Day-to-day troubleshooting
What is the link between datastore extension and root password?
Last week, I was asked for help: a junior VMware administrator, fresh VCP certified, was facing issues with extending a datastore.
After a storage team extended the SAN LUN under the datastore, you can see the new size in Extent Device window. But when my colleague tried to extend the datastore the free space was not visible.
First of all, google finds quickly KB1011754:
Symptoms
- You cannot extend a datastore when vSphere Client is connected to vCenter Server.
- vCenter Server does not show available unused space.
- The SAN LUN is extended and the new storage appears under Storage Adapters when viewed in vCenter Server. When you extend the datastore through the datastore properties, the free space does not appear.
- You can extend a datastore when vSphere Client is connected directly to the ESX host.
Resolution
vCenter Server calls a specific function to get all the available extents for that datastore. After getting extents, vCenter Server displays the extents as available if they meet these criteria filters:
- LUNS are not used as datastores on that host or on any other host (with exceptions to force mounted volumes).
- LUNS are not used as Raw Device Maps(RDMs) on that host or any other host.
vCenter Server applies these filters to stop any possible data corruption on already used LUNs by other host.
VMware recommends you to not to turn off the filters. Before making any changes to the LUN filters, contact the VMware Support team. For more information, see Turn off Storage Filters.
When you connect directly to the ESX host, these filters do not exist and hence you can see the extents.
To work around the issue:
- Connect to the ESX host using vSphere Client and log in as root.
- Extend the datastore from the GUI.
- Rescan all hosts from vCenter Server.
Playing with Storage Filters was not safe. Furthermore, the admin was not able to connect directly to host because he didn’t know the root password. And nobody available and working for that client did.
HowTo reset ESXi root password
Google again. According to KB1317898,
Reinstalling the ESXi host is the only supported way to reset a password on ESXi. Any other method may lead to a host failure or an unsupported configuration due to the complex nature of the ESXi architecture. ESXi does not have a service console and as such traditional Linux methods of resetting a password, such as single-user mode.
was out of discussion.
What to do?
Here it comes the dialog which led to the solution:
– Do you use AD authentication services?
– Yes, it is best practices.
– Is it configured the group “ESX Admins” in AD?
– … No. It would be a security issue.
– Are you able to create users and groups in AD?
– No… I am only member of “VMware-vCenter-Server-Admins” group in Active Directory, granting me rights to work in vCenter.
– Perfect… What I will show you now, don’t tell anyone 😉
Epilog and a question
The problem is solved; the admin was able to connect to ESXi host using his AD credentials and extend the datastore. He could even reset the root password.
My concern is this may be exploited in harmful ways. What is your opinion?
Cool one. In my opinion having access to vCenter and have permissions to change host advanced settings implies already ESXi host root permissions.