Killing a virtual machine in ESXi 4.1

26 02 2011

* I’m pretty sure at least one step I take in this post is unsupported by VMware, so please use at your own risk.  If you don’t feel entirely comfortable with any part of this, and have a support contract – use it.

I was re-IPing a virtual machine (VM) today when it locked up and showed no signs of life (zero CPU usage, disk I/O, etc.). I went in through the host that it runs on using the vSphere Client and tried to hard reset it… which stuck at 95%. I restarted the management agents on the host, which successfully killed the reset command.

Now the VM showed as invalid, so I tried unregistering the VM, then selecting “Add to Inventory” by right-clicking on the <vmname>.vmx file in the VM’s folder on the datastore.  While this made the host recognize the virtual machine correctly again, it didn’t help with booting the VM up – I was now getting an error while booting that told me the virtual machine swap file (<vmname>.swp) was locked.

I’d seen similar errors before, and have a good history with the tried-and-true hard kill method for resolving these:

ps aux | grep <vmname>  –> which gave me a series of process IDs (pid) for the VM’s world
kill -9 <pid>

which resulted in:

cannot kill pid 123123: No such process

Uh-oh.  At this point I began contemplating contacting all of the customers whose VMs (about 40 of them) are on this host for an emergency reboot of their systems so I can reboot the physical host.  Then I found some threads that talked about the vm-support command and how it can hard crash the VM world for you. Huh.  Cool.  Here’s how I used that to make my day much better:

vm-support -x   (this lists out all of the VM worlds running on your host)

When I found the numeric world ID (wid) for my VM, I then ran this command and allowed it to kill the VM:

vm-support -X  <wid>

This process took a while (10-15 minutes), but it worked!  Here’s a good KB article on the use of vm-support and other means of killing virtual machines that just won’t stop:

http://kb.vmware.com/kb/1014165


Actions

Information

3 responses

30 06 2011
Sam

I ran into the same scenario, but for me vm-support -x failed with:

/sbin/vm-support: cd: line 2448: can’t cd to /var/tmp
Could not set working directory to ‘/var/tmp’.

/tmp and /var/tmp seem to be in recursive symlink hell on my box, but it turns out I can force the temp directory with the -w switch:

vm-support -x -w /tmp

3 07 2011
Curtis

Thanks for adding that note Sam, I’m sure you won’t be the only one to run into the same issue.

11 07 2011
Phil

Awesome – thanks Sam – the -w switch saved my life as I’d lost control of the VCentre.
Mental Note – do not mount a DVD iso from the local hard drive of the VC to the VC from a VI Client running on the VC

Leave a comment