Let’s see the steps to create an ESXi image that we can convert to a template for later use to spin up all our nested ESXi VM’s.
There had been numerous challenges we faced for nested esx VM install so I’ve made this writing for you to make your job easy.
Here is our base config for our master ESXi VM
- 2 vCPU (Expose hardware-assisted virtualization to the guest OS – checked on)
- 30 GB vRAM
- 40GB vDisk ( thin disk )
- 2x Network Adapters (VMXNET3)
- SSH Enabled
- NTP Configured to router
- Working DNS ( Please check this series for Linux DNS )
- Install ESXI on it
- Deploy 4 x hosts from the same template
- Add disks for the new VSAN hosts
These vSAN disks are not required for template VM but after all nested ESXi VM’s are created, we will need to create these disks on every ESXi nested host – The choice of separate SCSI adapters is good
- vSAN Disk1 – 100 GB Capacity Disk
- vSAN Disk2 – 150 GB Capacity Disk
- vSAN Disk3 – 150 GB Capacity Disk
Creating an ESXi template via Cloudbuilder vm
This is by far the easiest way to deploy ESXI for VCF. Cloudbuilder has the softwares for VCF will be available under /mnt/iso
- Download and install the Cloudbuilder VM, this is just a standard ova deployment.
- Establish an ssh session to the cloud builder and navigate to the path shown
- Copy the esxi ISO image from the /mnt/iso path to your desktop
- Mount the ISO to a data store.
- Follow the above posts to create an esxi VM – You don’t need to specify any flags above if your CPU is compatible
Unsupported CPU issue
If you are on an older generation CPU as we had, as you try to install ESX 7.0 you will get this error
The CPU in this host is not supported by ESXi 7.0.0.
Please refer to the VMware Compatibility Guide for supportability.
In order to solve this CPU incompatibility issue all you need to do is, as soon as the esx OS boots hit shift+O to edit boot option and add the below as shown. Setting this below will convert errors into warnings and you should go to boot the OS as it won’t block you anymore from installing unsupported CPU.
The below steps are fairly basic ESX install but I thought to cover it fully just in case someone finds it useful.
Create persistent bindings for unsupported cpu in ESXi config
This step can’t be omitted if your CPU was not compatible in the first place. This ensures the changes we did are persistent across reboots.
Login to our template VM and execute vi /bootbank/boot.cfg
Add the line as shown and save and exit the file
Also, there is one more change we need to do, /altbootbank/boot.cfg.
Add the line as shown and save and exit the file
Reboot the template VM
Changing the MAC ID
We’ve got few more tweaks to do before we can make our template VM ready for cloning. We will get a duplicated MAC Address of the VMkernel interface(s) because the Nested ESXi configuration is the same so execute these commands as shown below
esxcli system settings advanced set -o /Net/FollowHardwareMac -i 1
Changing the System UUID
The next issue is that we will have a duplicated ESXi System UUID which can be found by running the command esxcli system uuid get. The easiest way to fix this issue is open the esx.conf and remove the line containing UUID.
sed -i ‘s#/system/uuid.*##’ /etc/vmware/esx.conf
Next, we need to save, and these changes in the file esx.conf are persistent, we need to run the backup.sh in the ESXi: /sbin/auto-backup.sh
Changing the System Certificate
The next issue is that we will need to change the certificates on the template VM.
Set the hostname and fdqn as shown from CLI
[root@sfo01-m01-esx02:~] esxcli system hostname set –host=sfo01-m01-esx02
[root@sfo01-m01-esx02:~] esxcli system hostname set –fqdn=sfo01-m01-esx02.ash.local
To regenerate new certificate on esx, just run the command as shown
/sbin/generate-certificates
Restart hostd and vpxa services by executing the following command:
/etc/init.d/hostd restart && /etc/init.d/vpxa restart At this stage, our template is now fully ready for cloning operations.