I had a discussion with my colleague about mixed version of ESXi hosts. Our customer needed to migrate their vSphere environment from version 5.1 (caused of End of Support Life) to the latest version supported on their physical infrastructure. As they did not build a cluster based on identical physical servers and two of servers are not supported by vSphere 6.0, it was necessary to build a “mixed cluster" - based on 5.5.x and 6.0.x ESXi hosts. There are some important points that we should discuss when we have to keep such cluster for a while: 😉
- Virtual Hardware
- VMware Tools
- vSphere features such as HA/DRS
Virtual Hardware
Each version of ESXi host provides new version of virtual hardware. Newer ESXi hosts can run virtual machines with an older virtual hardware version but VMs with newer virtual hardware versions cannot be powered on on older versions of ESXi. In case mentioned above, customer have VMs with Hardware (HW) Version 9 (vSphere 5.1) and VMs will be migrated to vSphere 5.5 (HW 10) and vSphere 6.0 (HW 11). The highest version of virtual hardware compatible for both version of ESXi is HW 10. However, an upgrade of HW is not required if new features (e.g. MEM, CPU limits) are necessary. When we have to upgrade HW, we should do it on the older ESXi hosts as we can not choose the hardware version during the HW upgrade. The upgrade is a one click process (available when a VM is powered off).
What's about new VMs? We have to ensure that a virtual hardware version is compatible with all the hosts in the cluster. There are ways to address it:
- Create VMs using a custom option and always choose the proper (=compatible for all ESXi hosts) version of HW.
- Select a default virtual machine compatibility at the cluster level (recommended).
For the latest ESXi/ESX hosts and compatible virtual machine hardware versions list please follow VMware KB 2007240.
VMware Tools
VMware Tools is an important component installed on Virtual Machines. When we have mixed version of ESXi in vSphere cluster, VM created on "older" ESXi host will be running older version of VMware Tools than VM created on later ESXi version. However, as shown on below figure, the latest VMware Tools are supported on older ESXi hosts. 
There are many ways to make sure that the latest version of VMware Tools are installed on each VM in mixed vSphere cluster:
- Create VM on later ESXi host, install VMware Tools.
- Create VM on older ESXi host, then VMotion to newer ESXi host and install VMware Tools.
- Download VMware Tools directly from VMware page and install it on VM (does not matter on which ESXi host. We don't use a "embedded" VMware tools).
vSphere features
Running mixed ESXi versions in an HA/DRS cluster is supported, however we should avoid it. E.g. vSphere 6.x added many new features in HA that are not available in version 5.5. vMotion is supported but we have to enable Enhanced vMotion Compatibility (EVC) that simplifies vMotion compatibility issues across CPU generations.
My thoughts
Mixed clusters are supported by VMware and at least temporarily all of us have had it during an upgrade process if hardware refresh is not used. When we have to build a mixed cluster, probably this cluster also will be unbalanced (CPU, MEM) and calculating Admission Control in earlier vSphere than 6.5 is not automatic. When is possible, we should avoid such configuration and sometimes it may be better to split clusters for: "old" and "new". It "costs" generally one host more (for HA), but we have less points to be taken into consideration 🙂
Useful links
Enhanced vMotion Compatibility (EVC) processor support (1003212)