A lot of my day-to-day work involves building and tearing down various configurations containing ESXi-hosts. For this, I use my laptop; a Dell Precision M4600. This means that the ESXi-hosts are virtual machines. I use vESXi templates and linked clones inside VMware Workstation to speed things up and save on disk space, instead of rebuilding each host anew.

I use the concept of templates and linked clones for my Windows virtual machines as well. With Windows, we get the advantage of sysprep to generalize the Guest OS, but with ESXi, such a tool doesn’t exist. We need some workarounds to make sure the Guest OS installation is generalized so each linked clone has its own unique identity.

A friend and co-worker (thanks Sjors!) pointed out that the ‘Reset System Configuration’ option in the DCUI might offer the required sysprep-like functionality. This option executes ‘/sbin/firmwareConfig.sh’, which basically removes the state (state.tgz and boot.cfg files in /bootbank) of an ESXi-host and reboots the machine. By executing this script before we shutdown the virtual machine and marking it as a template, we follow the same basic workflow as compared to sysprep (where you run sysprep and choose generalized OOBE).

Build the Virtual Machine

Make sure the virtual machine is created in the correct fashion, for either VMware Workstation 8 or vSphere 5 as the host platform. The document ‘Running Nested VMs‘ on VMTN explains it pretty well. Eric Gray does an even better job explaining how to nest ESXi on a physical ESXi 5 host.

For Workstation, the process is pretty easy. Create a VM, Guest OS type ‘VMware ESXi 5’. Make sure you enable the virtualized hardware-assisted virtualization in the VM’s settings (‘Virtualize Intel VT-x/EPT or AMD-V/RVI’).

Install ESXi

Installing ESXi isn’t a hard thing to do. Just randomly hit [enter] and [f11], and you’ll have ESXi installed. I used vSphere Update Manager to apply the latest firmware to the host (to save time on deployment later on).

Generalize ESXi

I actually took two separate actions to generalize ESXi:

  • From the DCUI, I deselected vmnic0 as a network adapter to be used for the management network and removed the DNS suffix. I then proceeded to restart the management network.
  • I then executed ‘Reset System Configuration’. Power the virtual machine off as soon as it reboots, though!

The script behind ‘Reset System Configuration’ does re-generate SSL thumbprints (certificates) and SSH fingerprints. Along with new MAC adresses (generated when you deploy linked from the template) and a different name and network identity (which you configure the newly deployed host with), each linked clone you deploy based on the template is unique.

Caveat: VMFS UUID

I used a small SCSI-disk of 5 GB, since I rarely use local storage for virtual machines. But when I used a 40 GB disk (I think the New Virtual Machine Wizard defaults to this size), the ESXi installer automatically creates a local VMFS-datastore called datastore1. The script behind ‘Reset System Configuration’ does not change or erase the VMFS-datastore. It doesn’t touch the VMFS-datastore at all. When deploying multiple vESXi VMs based on a template that contains a VMFS-datastore, you probably will encounter all kind of funky stuff as the VMFS UUID is not re-generated. Please make sure your template vESXi does not contain a local datastore!

Mark as template

The last step in creating the template (on VMware Workstation) is to mark the virtual machine as a template (‘Enable Template Mode (to be used for cloning)’) and create a snapshot (I named it ‘Reset System Configuration’).