Lab Overview

My homelab originally was exactly that; a test lab at home running a few game servers for friends and storage for me. That's changed a bit lately thanks to an opportunity to rent a small office space with a friend. This let me stretch the lab's legs a bit into a proper 42U rack. Thanks to some hardware from the "merger" my old gear has shifted roles a bit.

Software

From a software perspective the setup is much the same as it was:

  • Firewall: OPNsense
  • Hypervisor: XCP-NG
  • Management: Xen Orchestra
  • Storage: TrueNAS Core
  • Backups: Xen Orchestra
  • Switching: Brocade ICX 6610 (core) and ICX 6450 (edge)

Hardware

As mentioned hardware has changed a bit...

Running OPNsense is now a more modest, but still over kill, R320 with 24GB of RAM and a dual port Chelsio NIC for a redundant uplink to the Brocade ICX 6610 stack.

Switching is handled by a stacked pair of Brocade ICX 6610s with POE. These are stacked with a single 40G cable to allow use of the 2 remaining 40G ports to connect to the primary SAN.

Primary storage is provided primarily by a Dell R720xd with dual E5-2667v2 CPUs, 96GB of RAM, a dual port 40G Mellanox card for NFS traffic, and a Dell daughter card providing 2x 1Gbe and 2x 10Gb uplinks. This runs TrueNAS Core with 12x ZFS mirrors consisting of 600GB 10K SAS disks for fast storage.
Secondary and backup storage is handled by an R420 (my old firewall) also running TrueNAS core with a  RAIDZ-1 of 6x 600GB 10K SAS disks as well as a RAIDZ-1 of 6x 4TB SAS drives hosted in a Dell MD1200 connected via an external SAS HBA. This links back to the Brocades with another Dell daughter card providing 2x 1Gbe and 2x 10Gb uplinks.

Hypervisor duty is picked up by the pair of dual-node Supermicro chassis. Each node contains an E5-2650v2 and 64GB of RAM. All 4 nodes run XCP-NG and are controlled by a Xen Orchestra VM running on that pool. These are all uplinked back to the ICX 6610s by Intel X520-DA2 dual port 10G NIC for VM and storage traffic while the onboard dual 1Gbe NICs are used exclusively for management.

Outside of the main lab I have a rented dedicated server from OVH with a Xeon D-1540 and 32GB of RAM for VMs I need solid up time and network connectivity for. After the move I had to had something at home to provide essential services, so I acquired a Supermicro desktop chassis with an E5-2620v4 and 48Gb of RAM. Both of these servers run XCP-NG and are centrally managed by Xen Orchestra.

Networking

Prior to the move I added the rented dedicated server from OVH which complicated my networking requirements. Before everything was contained in a single rack in the basement of my house and everything was easily accessed from my LAN.

Now I have several wireguard tunnels connecting all of my sites. As is sits I have:

  • Home to OVH
  • Home to Office
  • Office to OVH

These 3 tunnels provide robust connectivity to all of my hypervisors. This is essential for offsite backups at the office as well as the OVH rented server.

Diagram depicting 3 site to site wireguard VPN tunnels connecting my various sites together for routing