Home network infrastructure

2017-present

Introduction

I don’t like it when computers are small and quiet. I think they’re a lot cooler when they’re unnecessarily large, intimidating, loud, and power-hungry. I think industrial things are a lot cooler than consumer things. Hence, my home network setup looks like this:

My server rack

I know this looks like a lot, but through careful scavenging from decomissioned HPC clusters and dying businesses my total investment into this setup (including server, switches, rack, and UPS) is about $450.

The server is a Supermicro CSE-747-20 case with:

  • Dual Intel Xeon E5-2650 V4
  • 64 GB (8x8 GB) of ECC DDR4
  • 8 drive bays
  • Enough space for 4 GPUs
  • Dual 2000 W power supplies (actually 1000 W on 120 V)

The hardware is definitely aging, but for the things I’m doing with it it’s more than enough, and the drive bays and GPU space is awesome to have.

What I run

Everything I run on this server is virtualized through Proxmox VE. I love Proxmox.

Some things I run or have run on it are:

  • My Discord bot and its MongoDB database
  • Instances of useful webapps like OctoPrint and Excalidraw, containerized through Docker and routed through a Traefik reverse proxy
  • A NAS running TrueNAS Core with 2 6 TB drives using ZFS
  • An ad-blocking DNS server using PiHole
  • An GPU-accelerated VM I use to experiment with LLMs with Ollama
  • A bunch of various game servers
  • Windows XP
  • A long-running livestreamed experiment to see the natural conclusion of a Sims 3 game without any human intervention

A pair of extremely overkill Cysco Catalyst switches connect all of the computers in our apartment.

I also use Tailscale to make all my self-hosted services accessible when I am out of the house.

Future work

Networking

I have two really powerful Cisco Catalyst switches that I’m currently using as dumb switches because my router is not VLAN-aware. Eventually, I want to set up VLANs for my servers, desktops, and IoT devices to make managing things eaiser and more secure.

Also, I’m interested in switching to a self-hosted Headscale instance to reduce my reliance on external services.

Power

You might notice that there’s an enormous UPS on the bottom of my rack. Unfortunately it is currently broken because the thermistor to monitor the battery temperature is reporting insane temperatures and I do not know how to get a replacement. The UPS is an APC SMT2200RM2U and here is what I know about the thermistor:

  • It appears to have a negative temperature coefficient as the resistance goes down as the temperature goes up
  • It reads 1-2 kΩ at room temperature (not that this means much since it is broken)
  • If I attach a 10 kΩ to the temperature sensor pin on the UPS it says it’s 0° C
  • It looks like this:

Thermistor

If anyone can help me identify a suitable replacement for this thermistor PLEASE contact me and there might be a small bounty in it for you.

Storage

I need a better backup system. I want to make a low-power, physically small backup server with 2 HDDs that I can put in a willing participant’s house to host encrypted backups for me. I know I could probably just get an off-the-shelf NAS, but that’s a lot less fun than trying to make something myself.