Building a Self-Healing Home Lab with Raspberry Pi

The Problem

I wanted remote access to my home network without paying for expensive VPN services or exposing ports to the internet. Most tutorials show you how to set up a basic Pi, but they skip the hard parts: reliability, security, and what happens when things break at 3 AM.

The Stack

After months of iteration, here's what I landed on:

• NetBird for zero-trust networking (no port forwarding needed)
• UFW + fail2ban for defense in depth
• Key-only SSH (passwords are for the weak)
• Pi-hole for network-wide ad blocking
• Grafana for monitoring and alerting

The Self-Healing Part

The magic is in the automation. I wrote systemd services that:

1. Monitor critical services every 30 seconds
2. Auto-restart failed services with exponential backoff
3. Send alerts to my phone via ntfy.sh
4. Run daily health checks and log rotation

# Example health check script
#!/bin/bash
services=("pihole-FTL" "netbird" "grafana-server")
for svc in "${services[@]}"; do
  if ! systemctl is-active --quiet $svc; then
    systemctl restart $svc
    curl -d "$svc restarted on $(hostname)" ntfy.sh/my-alerts
  fi
done

Results

• 97.7% uptime over 6 months
• MTTR dropped from 18 min to 4 min
• Total cost: ~$50 (Pi + SD card + case)

Lessons Learned

1. SD cards fail. Use a good one and set up read-only root or log2ram
2. Monitor everything. You can't fix what you can't see
3. Automate recovery. Manual intervention doesn't scale

The full setup guide and scripts are on my GitHub.