Painting yourself into a corner... VMware Style

Forgive me, I have not posted. Now I appear to have some time tonight. VMware used to be ESX.  Then it became ESXi then View (excuse me.. Horizon View) on top of ESXi.  You get the picture.

I was planning on patching 4 ESXi hosts used for VMware View..  I moved the VMs off, placed the host into maintenance mode and waited for the VMs to move.  Once in maintenance mode go to Update Manager and then go to the patches and apply. The host reboots and NORMALLY comes back. You exit maintenance mode and move on to the next host.  No muss, no fuss.

NOT THIS TIME. The host did not reconnect with vCenter. Was it down? I could ping it and connect to the host with a direct client connection. Now what?!?

Off to the Google for some suggestions. Apparently in VMware 5.5 you cannot get an ESXi host with Update 3 to connect to a vCenter with LESS than Update 3 on it also.

Now lets go to the remote console (iDRAC in this case) to remedy the situation. NO. I did not configure it.  This is not good...

Off to the customer site at 9PM.

Once on site at the customer, I did save my bacon with something else I came across in my searching. The "SHIFT+R". The SHIFT+R at the boot screen allows you to revert to the previous build. So I:

1. Booted
2. Shift+R
3. Said "Y" to confirm
4. Rebooted and all was good.

What did we learn:

1. RESEARCH the patches
2. HAVE THE iDRAC SET UP
3. KNOW YOUR ESCAPE ROUTE

P2V and the GPT Disk.. Part 2

In the previous post I had installed the Double Take - Move product and synchronized the servers. Then on 12/27 I was able to execute the fail-over to the VM. I logged into the software and it gave the option of using one of the "point in time" replications or using "live data". Since I wanted to make sure that the copy was the most current possible I chose the "live data" option. Double Take then shut down the source server, finalized the target server and then rebooted the target and then was done. It was almost anti-climactic. I then installed VMware Tools (which the mouse driver gave me an issue.. not sure why) and then turned it over to the customer. They logged into the server and checked. The databases were online and a user checked the application. All was operating as advertised. Thank you DoubleTake for making me look good!

P2V and the GPT Disk

For the better part of 2010 I have been working on performing P2V migrations. Taking a Physical server and turning it into a Virtual Machine for the unwashed masses. For the most part the process of turning a physical server into a VM is fairly painless.

However, I did most recently run across an issue with a server with a dreaded GPT (GUID Partition Table) boot disk and NONE of the P2V utilities that I would normally use would work. This GPT boot disk is a fairly recent phenomenon since it requires a EFI BIOS in order to understand what to do with it. To make matters worse the RAID group was 2095 or so GB in size (there was only 350GB used however).

As I stated previously, I did try everything I could think of in my arsenal. VMware Converter, Vizioncore vConverter, Novell PlateSpin.. NOTHING was working. I even thought of using Windows System Backup.

What was our superhero to do...

To the rescue came Vision Solutions - DoubleTake Move . It is a bit different in that it requires that there be a SOURCE and a TARGET rather than a copy of the SOURCE BECOMING the TARGET. So I loaded Windows 2008 onto the target and installed the software. I then installed the software onto the source server and created a mirror set... they connected and began to synchronize. After they are in sync you select "Failover" and the source server shuts down and the target is renamed to the source name. I have not done this part yet.. stay tuned for part 2 of our exciting adventure (This is where Max my dog would yawn).

Merry Christmas to all..

Long Time Coming

Ok.. So it has been a while since I have entered a post. It does not mean that I have not been doing anything. Quite the contrary. Now that I have had a chance to breathe I will begin entering some posts that will hopefully be of interest. They will probably be more on the lines of VMware ESX 4 and ESXi 4 since that is what I have been working with as of late. Without any further waste of time, here is the first post:

Setting up your own iSCSI...

The average person is probably thinking "What is iSCSI and why would I care" but the VMware folks in our midst are saying "Tell me more...". I needed to set up a VMware ESX environment at home to test ESXi 4 and some other VMware products. Working on a budget, I sourced a used Dell PE2850 and downloaded the free ESXi from VMware. Now back to the budget. I managed to get the server with 2 - 18GB disks but I did not have much in the way of money left for additional space. So out with the iSCSI solution. Since ESXi will connect to iSCSI over a normal Ethernet network I could add an larger SATA drive to a normal PC and then present it via the iSCSI software to the ESXi host and BANG.. more ESXi storage.

Can I speak with root?

Once again... this is not a very exciting post. Unless you are having to re-install a Linux system because you do not have the root password, then it could be like sex. Maybe just a piece of candy but you get the idea.

First get the installation media for said system. In this example it is a Red Hat 4.6 or 5.1 system. Then perform the following:
1. Boot from the media. When it comes up with the installation prompt type "linux rescue"
2. The system will continue to boot and ask about the network interface - it is not needed
3. When it asks about mounting the installation allow it to mount with read-write

Now for the fun part.. ok as fun as it gets

4. The hard drive installation will be under /mnt/sysimage
5. Change to /mnt/sysimage/etc and then edit shadow with vi
6. Find the root entry in the shadow file it will look like:

root:Ep6mckrOLChF.:10063:0:99999:7:::

7. Change the line in the file to look like the following:

root::10063:0:99999:7:::

8. Save and exit vi, and then reboot the system
9. Voila, no root password.
10. Log in as root and set the password with the passwd command ASAP!

What to expect....

Ok when it comes to programming ANYTHING I am pretty slow. I needed to change the firewall ruleset on quite a few ESX servers and it seemed like a script was in order to make this change for me. However, since I could not get the Opsware client to run a script remotely (that was part of the problem) i needed to figure out an alternative method.

Enter EXPECT.. I had heard it mentioned before but not really understood what it did. Basically you can open a connection to another system and if you can tell the script what to "expect" then it will know what to send it in return.. pretty cool huh.. This is the part where my wife's eyes glaze over.

Basically what I was doing is this...

spawn ssh some_user@somemachine.com
expect "someuse@somemachine's password:"
send password\r
expect "someuser@somesmachine"

And so on.. you get the idea. BUT.. is there not always a but... the issue I was having was that the prompt was on a VMware ESX server and looked like [usename@hostname username]$. I entered the text verbatim so that the expect showed the brackets etc. It puked. But by removing the brackets and only going to the end of the first word it works. Example..
expect "someuser@hostname" works but expect "[usename@hostname username]$" does not. Go figure. So I am going to stop while I am ahead today and revel in my glory....

Just rember... You Stay Classy San Diego.

Backup a minute..

Ok.. another yawner of a post but hey that is me. I am working on process of backing up the VM's in an environment and making it as quick to recover as possible. In my search I ran across a product called esXpress v3.1. It is pretty cool (ok as cool as VM software can be). It does a couple things:

1. It will put a VM into "Snapshot Mode" and then perform a backup of the VMDK.
2. It can put the VMDK backup to SMB/FTP or SSH.
3. After it is done it will remove the snapshot.

THIS IS DONE WHILE THE VM IS RUNNING! HOW COOL IS THAT!

Oh and I did not mention the best part. The software is FREE! Now there are additonal features such as being able to make backups local and remote at the same time, having additional helper machines, encryption and other things.

Check it out at www.esxpress.com