The Ghetto vSphere (ESXi) Home Lab

Project (Build a Ghetto vSphere (ESXi) Home Lab):

Summary:
This is detailed summary of my home lab. I wanted a low power, low noise, and low cost home lab. This will keep the wife from beating you. I know this setup is a ramshackle bag of ghetto parts. This is not the latest and greatest. This home lab will get the job done. You can find several blogs out there where they built a Xeon monster machine for $4,000+. This is not that. This is for the low budget user. You will have NIC driver issues, display driver issues, and bios issues. I will show you how to fix them with custom ESXi ISO and/or GRUB USB boot/install. You can build one ESXi host for about $180 (5GHZ CPU, 16GB RAM, 160GB Local HDD) to $270 (5GHZ CPU, 16GB RAM, 250GB Local SSD). For my total four ESXi host cluster (22 GHZ CPU, 80GB RAM, 500GB RAID1 SSD, 1TB RAID1 SSHD, Dual NIC(s)), the estimated cost is $1,935. You can reduce this by several hundred dollars. The RAID1 Enclosures with redundant hard drives add about $420. The 2nd NIC cards add about $140. Without redundant drives and NIC(s), the estimated cost is $1375. That’s not bad for a four host ESXi cluster with 22 GHZ CPU, 80GB RAM, 500 GB SSD, 1 TB SSHD, and Single NIC(s).

I use an EMC vVNX (virtual VNXe) for my shared storage (I may switch to the UnityVSA VM, it is the new vVNX with vVols, HTML5, extras.). I am running two vVNX virtual machines to provide ISCSI shared VMware datastores. The source vVNX is running all VM(s) via ISCSI on the 500GB RAID1 SSD. The source vVNX is replicating (RPO 1 hour) to the remote vVNX. The remote vVNX is using the 1TB RAID1 SSHD for its shared data store. I also added a speedy NAS drive for home use on the vVNX. Some people might say: “why use a SAN/NAS at all?”. My answer: “I wanted to know how to do it”. I may turn my home lab into vSAN. Also vVNX and UnityVSA are free community editions up to 4TB. 4TB is a lot for a home lab. You could buy a NAS device, but the price for a good one can be prohibitive. Cheap NAS devices usually have crappy software. The EMC vVNX and UnityVSA have more features than you know what to do with.

Hardware:
– CPU/Host: (Cost: $150 – $300 each)
— Quantity: 1, Lenovo W510 (esata port, 4 RAM slots, Intel NIC, Upgrade BIOS first for overheat issue): https://www.amazon.com/gp/product/B01AUK4KRS/ref=oh_aui_search_detailpage?ie=UTF8&psc=1
— Quantity: 1, Lenovo L420 (esata port, 2 RAM slots, Realtek NIC driver Issue): https://www.amazon.com/gp/product/B00K3TNKF0/ref=oh_aui_search_detailpage?ie=UTF8&psc=1
— Quantity: 2, HP EliteBook 2560p (esata port, 2 RAM slots, Intel NIC, Display driver Issue): https://www.amazon.com/gp/product/B00518M928/ref=oh_aui_search_detailpage?ie=UTF8&psc=1
— Recommendation: ESXi install is easier with all Intel CPU(s) and Intel NIC(s). 1st/2nd Gen Intel i5/i7 processors in USED laptops from Amazon (low power, low noise, low cost).

– RAM: (Cost: $30/each)
— Quantity: 10, Corsair 8GB (1x8GB) DDR3 1333 MHz (PC3 10666): https://www.amazon.com/gp/product/B005T63BEM/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1
— Recommendation: This gives you 16GB in the 2 slot host, 32GB in the 4 slot host.

-RAID: (Cost: $85/each)
— Quantity: 2, MiniPro 2.5-Inch RAID 1 Enclosure (esata, USB3, FireWire): https://www.amazon.com/gp/product/B00P2UAQ3E/ref=oh_aui_search_detailpage?ie=UTF8&psc=1
— Recommendation: Not Required. I highly recommend at least one RAID 1 Enclosure

– HDD/SDD/SSHD: (Cost: $80 – $150 each)
— Quantity: 2, Samsung 500GB SSD 2.5-Inch: https://www.amazon.com/dp/B00OBRE5UE/ref=twister_B00PRDMHLU?_encoding=UTF8&psc=1
— Quantity: 2, Seagate 1TB SSHD 2.5-Inch: https://www.amazon.com/gp/product/B00B99JUBQ/ref=oh_aui_search_detailpage?ie=UTF8&psc=1
— Quantity: 4, SanDisk Cruzer Fit CZ33 16GB USB 2.0 Low-Profile Flash Drive ($7): https://www.amazon.com/gp/product/B005FYNSZA/ref=oh_aui_search_detailpage?ie=UTF8&psc=1
— Recommendation: Use 2.5 inch drives for low power and low noise. (Optional) 16GB USB Flash for ESXi OS install)

-Switch: (Cost: $25 each)
— Quantity: 2, TP-LINK 8-Port Gigabit Ethernet Desktop Switch (TL-SG108): https://www.amazon.com/gp/product/B00A128S24/ref=oh_aui_search_detailpage?ie=UTF8&psc=1
— Recommendation: There may be less expensive switches, this is what I found.

-Cables: ($1 – $10 each)
— Quantity: 10, cat5e and cat6 for 1GB/s speeds (cat5 is only rated for 100MB/s): https://www.amazon.com/gp/product/B017C18TM0/ref=oh_aui_search_detailpage?ie=UTF8&psc=1
— Quantity: 2, eSATA Cable for 2.5″ SATA Laptop Hard Drive: https://www.amazon.com/gp/product/B002MKKTWA/ref=oh_aui_search_detailpage?ie=UTF8&psc=1
— Recommendation: Get cat6 or higher. The esata cable is optional, but is very useful to add extra drives.

-NIC: ($35 each)
— StarTech.com 1 Port ExpressCard Gigabit Laptop Ethernet NIC Network Adapter Card, EC1000S (Realtech NIC driver Issue, driver required to be preloaded with custom ESXI ISO install): https://www.amazon.com/gp/product/B003N3G1BI/ref=oh_aui_search_detailpage?ie=UTF8&psc=1
— Recommendation: 2nd NIC is optional. This will come in handy when you start playing with HA.

-UPS: ($20 – $45 each)
— Quantity: 2, APC BE350G Back-UPS 350VA 6-outlet Uninterruptible Power Supply (UPS): https://www.amazon.com/gp/product/B001985SWW/ref=oh_aui_search_detailpage?ie=UTF8&psc=1
— Recommendation: Optional. Your hosts (laptops) have ups batteries attached to them. This is for the switches.

 

Software:
-ESXi:
— Download the ISO (no extra drivers needed): https://my.vmware.com/en/web/vmware/evalcenter?p=free-esxi6
— Create custom ISO (for Realtek NIC Driver).
1. Install PowerShell (may require WinPRO)
1.a. Install PowerCLI: https://www.vmware.com/support/developer/PowerCLI/
2. Download ESXi-Customizer-PS-v2.4.ps1 (or latest version): http://www.v-front.de/p/esxi-customizer-ps.html#download
3. Using PowerShell/PowerCLI, run command(s):
cd DIRECTORY_OF_ps1_SCRIPT
.\ESXi-Customizer-PS-v2.4.ps1 -v60 -vft -load net-r8168,net-r8169,net-sky2
4. HELP, Step by Step Here: http://www.v-front.de/2014/12/how-to-make-your-unsupported-nic-work.html
5. Burn custom ISO to dvd disk, install from dvd disk
— Create bootable USB (Display Driver workaround)
1. Download YUMI: http://www.pendrivelinux.com/yumi-multiboot-usb-creator/
2. Add custom ISO to USB from YUMI GUI
3. Boot from USB on laptop (host)
4. Follow GRUB (USB Boot) prompts to boot from ESXi custom ISO
— (Untested) Install vib (Realtek Driver) from command shell: http://www.vladan.fr/realtek-8169-nics-not-detected-under-esxi-5-5/

-Vcenter
— Download VMware vCenter Server 6.0 Update 2 Appliance: https://my.vmware.com/group/vmware/details?downloadGroup=VC60U2&productId=491&download=true&fileId=686fb269e51713229148265d2bbdd00a&secureParam=cc5439447bcf292ae4d10ec80f7b077e&uuId=6102ab15-fc90-4fcc-b5d7-2c3adc7882f4&downloadType=
— Note: This is where you need VMware credentials to download the file. Since I enjoy my employment, I am not allowed to help skirt license requirements. Google is also great for cat photos!
— Follow tutorial on install: http://www.opvizor.com/blog/installing-vsphere-6-vcsa-vcenter-server-appliance/

-Datastores
— Beginners: I would recommend playing around with local datastores for a week or two, then set up your vVNX or UnityVSA.

-SAN/NAS VM:
— List of all EMC trial and Community Edition Software/VM(s): http://www.emc.com/products-solutions/trial-software-download/index.htm
— vVNX: has been removed. UnityVSA is the new vVNX (Virtual VNXe 3200)
— UnityVSA v4.0 Main Page: http://www.emc.com/products-solutions/trial-software-download/unity-vsa.htm
— UnityVSA v4.0 Download: http://www.emc.com/auth/unityvsasoftwaredownload.htm
— UnityVSA Installation Video >> https://community.emc.com/docs/DOC-53037
— UnityVSA Installation Guide >> http://www.emc.com/collateral/TechnicalDocument/docu69316.pdf
— UnityVSA FAQs >> http://www.emc.com/collateral/TechnicalDocument/docu69362.pdf
— Unity All Documentation >> http://www.emc.com/en-us/documentation/unity-family.htm
— Recommendation: I have played with vVNX and UnityVSA v1.0. I will assume the latest UnityVSA v4.0 is better. Deploy the free OVA/OVF file to your host. The only gotcha is the 12GB RAM memory reservation. You can change the reservation after deployment. You can use YouTube or EMC documentation for implementation/setup.

Leave a Reply