ESXi 7.x and beyond , SD cards & system storage layout

I am using VMWare ESXi since version 4 was shipped in 2009 in both, home lab and enterprise scale installations. The product evolved over the years, and it worked mostly reliable; except a couple of minor glitches in the matrix. Best of all. It is FREE.

If you are running a home lab, resources are limited. there is a 50% chance that you are running the ESXi from a USB flash drive or SD card.

Although it is and was never recommended by VMWare it works. I saw a few USB sticks dying over the years. When that happened for the very first time, I panicked. Today, I know better.

Simply reinstall ESXi on a new device, boot the server, apply your license and register your VMs. Takes 5 minutes, and you are back on track.

ESXi 7.x is the current release, and VSphere.next ( version 8 ) will come out soon. There are already a lot of articles around where VMware announced not to support USB / SD devices as a boot device. But they revised guidance in VMWare KB 85685

VMware will continue supporting USB/SD card as a boot device through the 8.0 product release, including the update releases. Both installs and upgrades will be supported on USB/SD cards.

I encourage you to read VMware KB 85685 in its entirety. The information is critical to ANY virtualization home lab enthusiast, and/or any VMware administrator who is planning out how and where they’ll be installing or upgrading to VMware ESXi 8.0 hypervisor.

Here is an image that shows the changes in the system storage layout in ESXi 6.x / 7.x. There will be no changes in ESXi 8.0

There is one important change, and that is the consolidation of some partitions in the ESXi 6.x layout into the new ESX-OS Data partition in ESXi 7.0. While the partitions in ESXi 6.x were static, the ESX-OS Data partition can vary in size depending on the size of the boot device.

If you want to use an USB / SD card, it should have a minimum size of 32 GB. I recommend using a high or max endurance microSD card like the SanDisk MAX ENDURANCE microSD-Karte 32GB.

I have used it since ESXi 7, but I decided to reconfigure my home lab and use a persistent 500GB SSD drive as boot device.

And here is where ESX-OS Data dynamic partition size comes into play. The partition can have a maximum size of 138GB.
On a 32 GB drive, ESX-OS Data will be 25 GB, and 55 GB on drives with less or equal 128 GB.

In a home lab there is no need for a 138 GB ESX-OS Data partition. As you can see from this image, the partition size will be pre-allocated, but it does not contain much data.

ESXi 7.0 Update 1c  release adds the boot option systemMediaSize to customize the space used by system-storage during installation and better match the purpose and size of the server. (VMWare KB 81166)

I first tried to edit boot.cfg as described in the KB article, but for some unknown reason this did not work. I recommend to enter the parameter interactive during the installation.

Start the host with the install image and when the ESXi installer window appears, press Shift+O within 5 seconds to edit the boot options.

In the lower left corner you will see something like

runweasel cdromboot

Replace cdromboot with

systemMediaSize=small

for a 55 GB ESX-OS Data partition.

After the installation has finished, you can SSH into the ESXi.

Type ls -ltrh /vmfs/devices/disks/ to get information about all your disks and partitions.

401.8G Aug 15 04:37 t10.NVMe__Samsung_SSD_970_EVO_500GB:8  // datastore
 55.9G Aug 15 04:37 t10.NVMe__Samsung_SSD_970_EVO_500GB:7  // ESX-OS Data
  4.0G Aug 15 04:37 t10.NVMe__Samsung_SSD_970_EVO_500GB:6  // boot-bank 1
  4.0G Aug 15 04:37 t10.NVMe__Samsung_SSD_970_EVO_500GB:5  // boot-bank 0
100.0M Aug 15 04:37 t10.NVMe__Samsung_SSD_970_EVO_500GB:1  // system boot
465.8G Aug 15 04:37 t10.NVMe__Samsung_SSD_970_EVO_500GB    //  

If your persistent SSD drive already contains a VMFS datastore, you must unregister the existing VMs and move them off of the datastore. the ESXi installer needs to repartition the device and this will also delete an existing VMFS datastore.

Although SD cards will still be supported in newer versions of the ESXi with options to move the ESX-OS Data partition off of the SD card to a VMFS datastore, you should consider to put the boot partition(s) on a persistent SSD drive.

The week couldn’t start worse when to come into work on a Monday and find out that one or more of them had self-destructed.