Check vMotion Compatibility

  By ,
Share this:

Recently I had situation where a few misconfigured VMs were not able to be migrated to any of the other hosts in the cluster, which came as a surprise during our maintenance window. This made me want to be a step ahead and minimize unplanned troubleshooting for future. If you appreciate your own time the next two examples for checking vMotion compatibility can be helpful for you.
Basically both of them are doing the same thing, but with one main difference – the PowerCLI script offers an option to actual migrate test VM across the hosts in a given cluster.

  • PowerCLI script
    Prerequisite: VMware PowerCLI installed

    As I mentioned earlier, the PowerCLI scrip offers the opportunity to perform actual vMotion of a specified VM across the hosts in a cluster to ensure proper network connectivity over the vMotion network. My personal opinion is that if you know well your environment and you worry only about the VM configurations you can skip the network test.
    Once you save it as .ps1 file, you can either open it from PowerCLI or just right click on it and select “Open in PowerShell”. You’ll be asked to specify vCenter Server to connect to, cluster name, if you wish to run the network test and if so, a name of VM to me migrated. Then the script will go through each of your ESXi hosts in the specified cluster, pull all VMs from it and test vMotion compatibility against the next ESXi host. When it reaches the last, it will test against the first one.

    Add-PSSnapin VMware.VimAutomation.Core -ErrorAction SilentlyContinue

$vCenter = Read-Host -prompt "Enter vCenter Server instance"

Connect-VIServer -server $vCenter

$CSVObjects = @()
$Results = @()
$err = @()
$destpath = [Environment]::GetFolderPath("Desktop")
$ClusterName = Read-Host -prompt "Enter cluster name"
$hosts = Get-VMHost -Location $ClusterName | Where {$_.ConnectionState -eq "Connected"} | Sort Name

while($confirmNetTest -ne "y") {
	$confirmNetTest = Read-Host -prompt "Do you want to perform vMotion network test? This will migrate specified VM across the hosts ensuring the network connectivity (y/n)"
	If ($confirmNetTest -eq "n") {break}
	}

If ($confirmNetTest -eq "y") {
		$TestVMName = Read-Host -prompt "Enter test VM name"
		while ($TestVM -eq $null) {
		$TestVM = Get-VM -Name $TestVMName -ErrorAction SilentlyContinue
			If ($TestVM -eq $null) {$TestVMName = Read-Host "No such VM. Please provide VM name"}
		}
}

$IsTestVMAffected = $false

$viewSI = Get-View 'ServiceInstance'
$viewVmProvChecker = Get-View $viewSI.Content.VmProvisioningChecker

for($i=0; $i -le $hosts.GetUpperBound(0); $i++){

	$vms = Get-VM -Location $hosts[$i]

	$o = $i+1
	If ($i -eq $hosts.GetUpperBound(0)) { $o = 0}

	Write-Host "Testing VM vMotion compatibility between" $hosts[$i] "and" $hosts[$o]

	foreach($vm in $vms){

		$Results = $viewVmProvChecker.QueryVMotionCompatibilityEx($vm.Id, $hosts[$o].Id)
			foreach ($Record in $Results) {
				If ($Record.Error -ne $null) {
				   foreach ($Error in $Record.Error) {
					$CSVObject = new-object PSObject
					$CSVObject | add-member -membertype NoteProperty -name "VM" -value $Record.VM
					If ($Record.VM -eq $TestVM.Id) {$IsTestVMAffected = $true} 
					$CSVObject | add-member -membertype NoteProperty -name "VMHost" -value $Record.Host
					$CSVObject | add-member -membertype NoteProperty -name "Error" -value $Error.LocalizedMessage
					$CSVObjects += $CSVObject
				   }					
				}
			}

	}

}

If ($confirmNetTest -eq "y") {
	If (!$IsTestVMAffected) {
	Write-Host "Testing vMotion network connectivity by migrating" $TestVM "across the hosts in" $ClusterName
	$InitialVMHost = $TestVM.VMHost
		for ($i=0; $i -le $hosts.GetUpperBound(0); $i++) {
				If ($i -eq $hosts.GetUpperBound(0)) {
				$DestHost = $hosts[0]
				} else { $DestHost = $hosts[$i+1] }
			
			Move-VM -VM $TestVM -Destination $DestHost -ErrorVariable +err 2>> $destpath\vMotionNetTest.txt | out-null
		}	
		If ($InitialVMHost -ne $hosts[0]) { Move-VM -VM $TestVM -Destination $InitialVMHost }
		$datetime = Get-Date
		If ($err.count -eq 0) {Write-Output "$datetime Network test completed successfully" >> $destpath\vMotionNetTest.txt}
	} else { Write-Output "$datetime The specified test VM cannot be migrated. Please check vMotionTest.csv file for details" >> $destpath\vMotionNetTest.txt }
}

Disconnect-VIServer -Confirm:$False

If ($CSVObjects.count -eq 0) {
	$CSVObjectNoErr = new-object PSObject
	$CSVObjectNoErr | add-member -membertype NoteProperty -name "Result" -value "No errors found"
	$CSVObjectNoErr | Export-Csv -Path $destpath\vMotionTest.csv -NoTypeInformation
	}
else { $CSVObjects | Export-Csv -Path $destpath\vMotionTest.csv -NoTypeInformation }

    After starting the script you can choose to actually migrate a specified VM across all hosts in the cluster. At the end the VM will be migrated back to the host where it was running in the first place. This can help you determine if there are any issues for example on the vMotion network level, which can lead to unsuccessful migration even though all other requirements are met. The output of this check is redirected to a .txt file on your Desktop.

    The result after checking the vMotion compatibility is .csv file exported to your Desktop which contains the following columns:
    VM – VM ID of the virtual machine which returned error
    Host – Host ID of the destination host
    Error – Error message in human readable format. In other words it is showing what exactly prevents the vMotion

  • vCenter Orchestrator workflow
    Prerequisite: vCenter Orchestrator configured and registered with vCenter Server

    It is a simple workflow with just one “Scriptable task”. Let’s go through the steps really quick:
  1. Open vCenter Orchestrator and log in
    vCO-login
  2. Make sure you are on “Design” layout, right click on a folder and select “New workflow”
    VCO-newWorkflow
  3. Provide a name for the new workflow. On the next screen select “Inputs” tab and add parameter of type VC:ClusterComputeRecource
    vCO-inputs
  4. Go to “Schema” tab. Drag and drop “Scriptable task” between the Start and End points. Click “Edit” above the newly added task
    vCO-schema
  5. Open “IN” tab and add the “ClusterName” parameter
    vCO-inputs
  6. Open “Visual Binding” and confirm the Input parameter of our workflow is linked to the Scriptable task
    vCO-schema-visual-binding
  7. Now let’s move on to the “Scripting” tab
    vCO-schema-scripting
    Copy/paste the script below:

    var ProvisioningChecker = ClusterName.sdkConnection.vmProvisioningChecker;
var VMHosts = System.getModule("com.vmware.library.vc.cluster").getAllHostSystemsOfCluster(ClusterName);
var VMs = System.getModule("com.vmware.library.vc.cluster").getAllVMsOfCluster(ClusterName);
var today = new Date();
var now = today.toGMTString();
var fw = new FileWriter("C:/Orchestrator/vmotion-results.txt");
var task = ProvisioningChecker.queryVMotionCompatibilityEx_Task(VMs, VMHosts);
var Results = System.getModule("com.vmware.library.vc.basic").vim3WaitTaskEnd(task, false, 1);
var Errors = 0;
fw.open();
fw.write(now + "\r\n");
for each( var Record in Results )
{
   if( Record.error != null )
   {
      for each( var Error in Record.error )
      {
		 fw.write("Unable to migrate " + Record.vm.name + " to " + Record.host.name + " - " + Error.localizedMessage + "\r\n");
		 Errors = Errors+1;
      }
   }

}

if (Errors == 0) {
	fw.write("No errors found \r\n");
}
fw.close();

    I want to focus your attention on the row where the path to the output file is defined. By default vCO has the following permissions:
    - rwx C:/
    + rwx C:/orchestrator
    # relative to user.dir which is %orchestrator_install_dir%\app-server\bin
    + rx ../../app-server/logs/
    + rx ../../configuration/logs/
    + rx ../bin/
    - rwx ../../app-server/conf/security/
    + rx ../../app-server/conf/
    + rx ../../apps/
    + r ../../version.txt

    Those accesses are specified in the js-io-rights.conf file. If you want to store your output file in different directory you should add it in this file and restart the vCO service.

    If you installed vCO as part of the vCenter Server Installation the js-io-rights.conf file is located at install_directory\VMware\Infrastructure\Orchestrator\app-server\conf\
    If you installed the standalone version of Orchestrator the path is install_directory\VMware\Orchestrator\app-server\conf\
    If you use vCO Appliance the js-io-rights.conf file is located in /etc/vco/app-server/ and the default permissions are as follows:
    - rwx /
    + rwx /var/run/vco
    - rwx /etc/vco/app-server/security/
    + rx /etc/vco
    + rx /var/log/vco/

  8. Look like we are ready to run the workflow. Click “Run”, select cluster and click “Submit”
    vCO-run
  9. In my case the workflow completed successfully and here is the output:
    Thu, 28 Jan 2016 23:09:45 GMT
    Unable to migrate Windows7 to esxi01.vlab.local – Virtual disk ‘Hard disk 3’ is not accessible on the host: Unable to access file [esxi02-local] Windows7/Windows7_1.vmdk
    Unable to migrate Windows7 to esxi01.vlab.local – Virtual disk ‘Hard disk 4’ is not accessible on the host: Unable to access file [esxi02-local] Windows7/Windows7.vmdk

    Looks like I’ll have troubles entering Maintenance Mode on one of my hosts.

    This looks simple enough, right? Well, let’s make it even easier to run. Go ahead and “Save and Close” you workflow. Then open web browser and log in to vSphere Web Client. Open vCenter Orchestrator, click “Manage” tab and then select “Context Actions”. Hit the “green plus” icon to add your workflow. Make sure you select “Cluster” as Available Types.
    WebClient-addWorkflow
    Now you can just right click on your cluster, select “All vCenter Orchestrator plugin Actions” and run your workflow.WebClient-Run-Workflow

I hope this can help you avoid “Unable to enter Maintenance Mode” surprises when planning your next patching.

Credits:
To the VMware community and especially this post
To this blog post which came in handy when I was “stuck” in the middle of the workflow

The following two tabs change content below.
My Twitter profileMy Facebook profileMy LinkedIn profile

Ivaylo Ivanov

Ivaylo has 5 years of professional IT experience. Most of it in server administration area, network and virtualization technologies. From 2014 he specializes in VMware products family. He holds VCIX6-DCV and VCP7-CMA certifications. vExpert 2016/2017
My Twitter profileMy Facebook profileMy LinkedIn profile

Latest posts by Ivaylo Ivanov (see all)

About Ivaylo Ivanov

Ivaylo has 5 years of professional IT experience. Most of it in server administration area, network and virtualization technologies. From 2014 he specializes in VMware products family. He holds VCIX6-DCV and VCP7-CMA certifications. vExpert 2016/2017
Tagged  , , , .
Bookmark the permalink.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.