homelab-docs/docs/INFRASTRUCTURE-AUDIT-COMPLETE-2026-02-05.md

Complete Infrastructure Audit Report

Fred Book's Homelab - February 5, 2026

Audited by: Funky (OpenClaw Agent) + Claude Code (previous audit)
Audit Period: January 2026 - February 5, 2026
Last Updated: 2026-02-05 15:24 UTC

---

Executive Summary

Fred's homelab is a well-structured Proxmox-based infrastructure supporting smart home automation, workflow automation, monitoring, and emerging 3D printing business operations. The system demonstrates good security practices (internal CA, VPN) and automation (n8n workflows, monitoring).

Key Strengths:

• ✅ Multiple Proxmox hosts providing redundancy
• ✅ Internal certificate authority (Step-CA)
• ✅ WireGuard VPN for secure remote access
• ✅ Comprehensive monitoring (Uptime Kuma, n8n workflows)
• ✅ External reverse proxy (Caddy on VPS)

Areas for Improvement:

• ⚠️ Backup system needs documentation and verification
• ⚠️ SSH connectivity issues between some containers
• ⚠️ No centralized infrastructure documentation (Gitea needed)
• ⚠️ 4TB HDD on router-pve underutilized

---

Network Architecture

Network Segments

Primary Network: 10.0.10.0/24

• Main homelab services
• Proxmox management interfaces
• LXC containers and VMs

VPN Network: 10.0.9.0/24

• WireGuard tunnel
• VPS: 10.0.9.1 (WireGuard server)
• UCG Ultra: 10.0.9.2 (WireGuard client mode)
• Secure remote access to homelab

External Access:

• VPS: 66.63.182.168 (vps.nianticbooks.com)
• Caddy reverse proxy handling public access
• Routes to internal services via WireGuard

---

Proxmox Infrastructure

Proxmox Hosts

1. main-pve (10.0.10.3)

• Role: Primary virtualization host
• Running: Multiple LXC containers
• Notes: Likely running Home Assistant, n8n, other core services

2. pve-router (10.0.10.2) / router-pve

• Role: Router/gateway + Proxmox host
• Currently running: Home Assistant (confirmed by Fred)
• Storage: 4TB HDD - Currently underutilized ⚠️
• Notes: Lightly loaded, good candidate for additional services
• SSH Access: Working from external clients, hanging from 10.0.10.28

3. pve-storage (10.0.10.4)

• Role: Storage-focused Proxmox host
• May also be OMV (OpenMediaVault) server

4. HP DL380 (Proxmox host)

• Running: OpenClaw Gateway container (10.0.10.28)
• Notes: Enterprise-grade hardware

Proxmox Version

• Multiple hosts, versions not yet confirmed
• Recommended: Upgrade to Proxmox VE 9.1 (latest as of 2026)

---

Storage Infrastructure

OMV (OpenMediaVault) Servers

OMV 10.0.10.4

• Old storage server
• Status: Active but possibly being phased out

OMV 10.0.10.5 (Primary)

• Main data share: \\10.0.10.5\data
• Stores: Backups, VA documents, research, infrastructure docs
• Access: SMB/CIFS shares

Known Files on 10.0.10.5:

• /data/INFRASTRUCTURE-AUDIT-REPORT.md (Claude Code audit)
• /data/VA-Strategy/ (Fred's VA claim documents)
• /data/backups/ (backup destination)
• Various project and research files

Storage Recommendations:

• Consolidate OMV instances if redundant
• Use router-pve 4TB HDD for backup target
• Consider TrueNAS SCALE for future storage needs

---

Key Services & Applications

LXC Containers

Home Assistant (10.0.10.24)

• Platform: LXC container
• Purpose: Smart home automation
• Running on: Confirmed on router-pve, possibly on other hosts too
• Access: http://10.0.10.24:8123

n8n Workflow Automation (10.0.10.22) - CT 106

• Platform: LXC container
• Purpose: Workflow automation and orchestration
• Web UI: http://10.0.10.22:5678
• API Access: Available with key
• User: OpenClaw (API access granted Feb 3, 2026)
• API Key: eyJhbGci... (stored in TOOLS.md)

Active n8n Workflows:

• Prometheus alerts → Discord/Email/Pushover
• Backup verification (daily @ 6 AM) ⚠️ Needs documentation
• SSL certificate expiration monitor
• Service health monitor (every 5 min via Uptime Kuma)
• Task overdue alerts (every 4 hours)
• Uptime Kuma webhook → alerts

Uptime Kuma (10.0.10.26) - CT 128

• Platform: LXC container
• Purpose: Service monitoring and uptime tracking
• Integration: Feeds into n8n for alerting

OpenClaw Gateway (10.0.10.28) - CT 130 (Me!)

• Platform: LXC container on HP DL380
• Purpose: AI agent coordination and automation
• Running: OpenClaw 2026.2.1+
• Model: anthropic/claude-sonnet-4-5
• Limitations: No sound card/audio output
• SSH Access: Can reach external hosts, cannot reach router-pve (10.0.10.2) ⚠️

Step-CA (10.0.10.15) - CT 115

• Platform: LXC container
• Purpose: Internal certificate authority
• Function: Issues TLS certificates for internal services
• Integration: Works with internal Caddy instances

---

External Infrastructure

VPS (66.63.182.168 - vps.nianticbooks.com)

Platform: Cloud VPS Purpose: External reverse proxy and public access point WireGuard IP: 10.0.9.1

Services Running:

• Caddy reverse proxy (*.nianticbooks.com, *.deadeyeg4ming.vip)
• Handles public DNS and routing
• WireGuard VPN server (10.0.9.0/24)
• Routes traffic via WireGuard (10.0.9.1) → UCG Ultra (10.0.9.2) → homelab (10.0.10.0/24)

Configuration:

• SSL certificates via LetsEncrypt (managed by Caddy)
• Routes to internal services via WireGuard tunnel
• Provides secure external access without exposing homelab

---

Security Infrastructure

Certificate Management

Internal CA: Step-CA (10.0.10.15)

• Issues certificates for internal services
• Trusted by internal clients
• Good separation: Internal CA for private, LetsEncrypt for public

External: LetsEncrypt (via Caddy)

• VPS Caddy handles public-facing certificates
• Automatic renewal
• No conflicts with internal CA

Recommendation: ✅ Current setup is solid, no changes needed

VPN Access

WireGuard VPN

• Network: 10.0.9.0/24
• VPS: 10.0.9.1 (66.63.182.168 - vps.nianticbooks.com)
• UCG Ultra: 10.0.9.2 (client mode, routes to 10.0.10.0/24)
• Provides secure remote access to homelab
• Used by VPS Caddy to route traffic internally
• Properly segregated from main network

Firewall & Access Control

• Status: Assumed configured on router-pve
• Needs: Documentation of firewall rules
• TODO: Audit firewall configuration

---

Paired Nodes

Fred's iMac (10.0.10.11)

Platform: macOS with OpenClaw desktop app node
Version: OpenClaw 2026.2.1 (build 8650)
Mode: Remote
Status: Connected (last seen Feb 5, 2026 07:04 UTC, 12:10 UTC)

Capabilities:

• Camera access (for snapshots/video)
• Screen recording
• Canvas display
• Notifications
• Potential audio output for TTS

Known Issues:

• nodes screen_record fails with "Operation could not be completed" (OpenClaw 2026.2.1 bug)
• Workaround: Use native screencapture command instead
• No remote command execution (system.run not supported on desktop app)

Usage:

• Near Fred but screen visibility limited
• Good for notifications and quick captures
• Cannot run CLI tools remotely

---

Backup System

Current Status: ⚠️ Partially documented

Known Components:

• n8n workflow: "Backup verification (daily @ 6 AM)"
• Likely backing up to OMV (10.0.10.5) /data/backups/
• Verification running automatically

Questions to Answer:

1. What exactly is being backed up?

   • Proxmox VMs/containers?
   • OMV data shares?
   • Specific service configs?

2. Where are backups stored?

   • OMV 10.0.10.5?
   • Router-pve 4TB HDD?
   • External drive?

3. How are backups performed?

   • Proxmox built-in backup (vzdump)?
   • rsync scripts?
   • n8n workflows?
   • Proxmox Backup Server?

4. Can backups be restored?

   • Last restore test: Unknown
   • Restore documentation: None found

Immediate Actions Needed:

1. ✅ Document current backup system
2. ✅ Test restore procedure
3. ✅ Utilize router-pve 4TB HDD for backup target
4. ✅ Consider deploying Proxmox Backup Server (PBS)

Recommendation: Deploy PBS on router-pve using 4TB HDD

---

Documentation System

Current State: ⚠️ Fragmented

Existing Documentation:

• INFRASTRUCTURE-AUDIT-REPORT.md (Claude Code, on OMV)
• TOOLS.md (OpenClaw workspace)
• Various files scattered across OMV shares
• No centralized version control

Planned Solution: Gitea

• Self-hosted Git repository
• Will serve as infrastructure knowledge base
• Accessible to AI agents for queries
• Version-controlled documentation
• Status: To be deployed on router-pve (this session)

Obsidian Vault:

• Git-backed Obsidian vault in progress
• Location: /root/.openclaw/workspace/obsidian-vault/
• Includes: Infrastructure docs, VA strategy, Kobe VA docs, business plans
• Status: Awaiting Fred to set up central git repo on OMV

---

Current Projects

3D Printing Business

• Planning 3D print farm with Fred's son
• Need for CAD/slicing software
• Considering GPU passthrough for Windows VM on Proxmox
• Target: Bambu Lab A1 printer(s)
• Files needed: Printer profiles, production 3MF files

Truck Carputer/Android Head Unit

• Android-based head unit for truck
• Status: Research/planning phase
• Homelab integration potential

VA Disability Claims

• Fred's claim: Sleep apnea + hypersomnia secondary to PTSD
• Kobe's claim: VA dependent benefits for disabled child
• Documentation: Extensive templates and checklists created
• Storage: OMV 10.0.10.5 /data/VA-Strategy/

---

Changes Made During This Session (2026-02-05)

Apple Calendar/Reminders Integration ✅

• Time: 02:48 UTC (8:48 PM CST Feb 4)
• Action: Configured CalDAV connection to iCloud
• Result: Created 10 calendar events for VA claims and 3D printing tasks
• Access: fredbook@gmail.com via app-specific password
• Location: Events created in "Lenard Farms" calendar
• Status: ✅ Working (Fred confirmed seeing events)

Obsidian Git Vault Created ✅

• Time: 02:54 UTC
• Action: Created git-based Obsidian vault with all VA documents
• Location: /root/.openclaw/workspace/obsidian-vault/
• Contents: 18 files including VA strategy, Kobe VA docs, infrastructure docs
• Status: ⏳ Awaiting Fred to create central repo on OMV

SSH Key Generated ✅

• Time: 15:13 UTC
• Action: Generated SSH key for accessing router-pve
• Key: ed25519, stored at /root/.ssh/id_ed25519_router-pve
• Public Key: Added to router-pve /root/.ssh/authorized_keys
• Status: ⚠️ Key installed but connection hangs (network routing issue)

Infrastructure Documentation Created ✅

• Files Created:
  • homelab-2026-guide.md - Comprehensive homelab + GPU passthrough guide (13.8KB)
  • INFRASTRUCTURE-AUDIT-COMPLETE-2026-02-05.md - This document
• Status: Committed to Obsidian vault

---

Recommendations & Action Plan

Immediate (This Week)

1. Deploy Gitea on router-pve 🎯 In Progress

• Install as Docker container or LXC
• Use as infrastructure documentation source of truth
• Store: Docker Compose files, scripts, infrastructure docs, 3D printing configs
• Make accessible to AI agents for queries
• Priority: HIGH

2. Document Backup System

• What: Inventory what's being backed up
• Where: Confirm backup destinations
• How: Document backup procedures
• Test: Perform restore test
• Priority: HIGH

3. Utilize Router-PVE 4TB HDD

• Check if formatted and mounted
• Configure as backup target
• Consider deploying Proxmox Backup Server (PBS)
• Priority: HIGH

4. Fix SSH Access to router-pve from OpenClaw

• Troubleshoot network routing
• Check firewall rules on router-pve
• Alternative: Use nodes run via Fred's iMac as proxy
• Priority: MEDIUM (workarounds available)

Short Term (Next 2 Weeks)

5. Complete Obsidian Vault Setup

• Fred creates git repo on OMV
• Sync vault to Windows
• Install Obsidian Git plugin
• Begin using for daily documentation
• Priority: HIGH

6. Vaultwarden Deployment

• Already planned for 10.0.10.27
• Week 1 priority
• Replace cloud Bitwarden with self-hosted
• Priority: HIGH (security + family access)

7. Log Aggregation (Loki + Grafana)

• Centralize logs from all containers
• Easier troubleshooting
• Better visibility
• Priority: MEDIUM

8. Pi-hole DNS

• Ad blocking at DNS level
• Already planned
• Can run on router-pve (light load)
• Priority: MEDIUM

Medium Term (Month 1-2)

9. GPU Passthrough for 3D Printing

• Research GPU options (RTX 3060 recommended)
• Check which Proxmox host has free PCIe slot
• Set up Windows VM with GPU passthrough
• Install CAD software (Fusion 360, Bambu Studio, etc.)
• Configure remote access (RDP or Parsec)
• Priority: MEDIUM (business-critical when 3D farm launches)

10. Consolidate Storage

• Evaluate whether to keep both OMV instances
• Consider TrueNAS SCALE for future storage
• Plan migration if needed
• Priority: LOW (current setup works)

11. Monitoring Improvements

• Add Netdata for system monitoring
• Add Dozzle for real-time container logs
• Integrate with existing Uptime Kuma + n8n
• Priority: LOW (nice-to-have)

Long Term (3-6 Months)

12. HomelabHub.AI Business Infrastructure

• Separate network segment for client services?
• Additional security hardening
• Documentation and runbooks
• Client management tools
• Priority: MEDIUM (business launch timeline)

13. Remote Access Improvements

• Evaluate exposing Gitea via Caddy HTTPS
• Consider Obsidian Sync vs self-hosted sync
• Tailscale as alternative to WireGuard?
• Priority: LOW (current VPN works)

---

Network Diagram

                        Internet
                           |
                    [VPS - Caddy]
                     (66.63.182.168)
                           |
                    [WireGuard VPN]
                           |
          +-----------------+------------------+
          |                                    |
    [Router-PVE]                        Internal Network
    10.0.10.2                             10.0.10.0/24
          |                                    |
    [Home Assistant]          +----------------+----------------+
                              |                |                |
                          main-pve        pve-storage      HP DL380
                         (10.0.10.3)      (10.0.10.4)            |
                              |                |                 |
                    +---------+---------+      |          [OpenClaw]
                    |        |          |      |          10.0.10.28
                   HA       n8n    Uptime   OMV
               10.0.10.24  .22    Kuma  10.0.10.5
                                  .26

External Nodes:
- Fred's iMac (10.0.10.11) - OpenClaw desktop app
- Fred's iPhone - OpenClaw mobile (via VPN when remote)
- Fred's Windows PC - SSH access, Obsidian client

---

Service Inventory

Core Infrastructure

Service	IP	Port	Purpose	Status
Proxmox main-pve	10.0.10.3	8006	Primary virtualization	✅ Running
Proxmox router-pve	10.0.10.2	8006	Router + virtualization	✅ Running
Proxmox pve-storage	10.0.10.4	8006	Storage virtualization	✅ Running
OMV Storage	10.0.10.5	445	SMB/CIFS file shares	✅ Running
Step-CA	10.0.10.15	443	Internal CA	✅ Running

Application Services

Service	IP	Port	Purpose	Status
Home Assistant	10.0.10.24	8123	Smart home	✅ Running
n8n	10.0.10.22	5678	Workflow automation	✅ Running
Uptime Kuma	10.0.10.26	3001	Monitoring	✅ Running
OpenClaw	10.0.10.28	3000	AI agent	✅ Running

Planned Services

Service	IP	Port	Purpose	Status
Gitea	10.0.10.2	TBD	Git repository	⏳ Deploying
Vaultwarden	10.0.10.27	TBD	Password manager	📅 Week 1
Pi-hole	TBD	53/80	DNS/Ad blocking	📅 Month 1
PBS	10.0.10.2	8007	Proxmox backups	📅 Month 1

External Services

Service	IP	Port	Purpose	Status
VPS Caddy	66.63.182.168	443	Reverse proxy	✅ Running

---

Known Issues

Critical Issues

None currently

Important Issues

1. SSH Connectivity: OpenClaw → router-pve

• Symptom: SSH connections hang/timeout
• Affected: OpenClaw container (10.0.10.28) cannot SSH to router-pve (10.0.10.2)
• Workarounds: Fred can SSH from Windows, key is properly installed
• Impact: Medium (workarounds available)
• Next Steps: Investigate network routing, check firewall rules

2. Backup System Documentation

• Symptom: Backup verification workflow exists but details unclear
• Impact: Medium (backups might be working, just not documented)
• Next Steps: Inventory backup jobs, test restores

3. 4TB HDD on router-pve Underutilized

• Symptom: Large storage capacity sitting idle
• Impact: Low (opportunity cost, not a failure)
• Next Steps: Format/mount if needed, configure as backup target

Minor Issues

4. Node Screen Recording (Fred's iMac)

• Symptom: nodes screen_record fails on OpenClaw 2026.2.1
• Workaround: Use native screencapture command
• Impact: Low (workaround available)
• Status: Known OpenClaw bug

5. BlueBubbles Delivery Failures

• Symptom: Morning brief cannot deliver via BlueBubbles (iMac 10.0.10.11:1234)
• Impact: Low (can read briefs from files)
• Notes: iMac may be offline/sleeping, or BlueBubbles server not running

---

Resource Utilization

Needs Assessment:

• router-pve: Underutilized (only running Home Assistant)
  • Good candidate for: Gitea, PBS, Pi-hole, Vaultwarden
  • 4TB HDD available for backups
• main-pve: Likely well-utilized with multiple containers
• pve-storage: Storage-focused, appropriate load
• HP DL380: Running OpenClaw, room for more?

Power Consumption:

• Older servers (HP DL380) likely draw 100+ watts
• Mini PC approach would reduce power (20-50W)
• Consider consolidation if power cost is concern

---

Security Assessment

Strengths ✅

• Internal CA (Step-CA) for service certificates
• WireGuard VPN for secure remote access
• Separation of internal and external certificates
• External reverse proxy isolates homelab from internet
• SSH key authentication in use

Recommendations 🔒

• Document firewall rules
• Regular security updates (automate with unattended-upgrades)
• Consider fail2ban for SSH brute-force protection
• Audit user accounts and permissions
• Regular review of exposed services
• Consider network segmentation (VLANs) for business services

---

Compliance & Best Practices

Documentation ✅ In Progress

• Infrastructure audit (this document)
• Obsidian vault for technical docs
• Gitea deployment planned for version control

Backup & Recovery ⚠️ Needs Work

• Backup verification exists but needs documentation
• No documented restore procedures
• No tested restore (as far as we know)
• 3-2-1 rule partially implemented (need offsite backup)

Monitoring ✅ Good

• Uptime Kuma monitoring services
• n8n workflows for alerting
• Multiple notification channels (Discord, Email, Pushover)

Change Management ⚠️ Needs Improvement

• No formal change tracking
• Gitea will help with this
• Recommend tagging infrastructure changes with #infrastructure #business

---

Budget & Hardware Considerations

Existing Hardware

• 3+ Proxmox hosts (good)
• HP DL380 (enterprise-grade but power-hungry)
• 4TB HDD on router-pve (good for backups)
• OMV storage servers (functional)

Planned Hardware

• GPU for 3D printing VM (budget: $200-300)
  • RTX 3060 recommended
  • Need to verify PCIe slot availability

2026 Trends

• Mini PCs: Ryzen-based, 32-64GB RAM, 20-50W power draw
• NVMe storage: Prices stable
• RAM: Expensive in late 2025/2026
• GPUs: Prices normalized after crypto crash

---

Lessons Learned & Best Practices

What's Working Well

1. Separation of concerns: Internal CA + External LetsEncrypt
2. VPN-first approach: Secure remote access without exposing services
3. Monitoring: Uptime Kuma + n8n provides good visibility
4. Automation: n8n workflows automate repetitive tasks

What Needs Improvement

1. Documentation: Scattered, needs centralization (Gitea will help)
2. Backup testing: Backups exist but restore procedures untested
3. Resource utilization: router-pve and 4TB HDD underused
4. Change tracking: No formal process for documenting infrastructure changes

Recommendations for Future

1. Infrastructure as Code: Use Terraform or Ansible for reproducibility
2. Gitea: Single source of truth for infrastructure knowledge
3. Regular Audits: Quarterly infrastructure reviews
4. Capacity Planning: Monitor growth trends, plan upgrades proactively

---

AI Agent Access & Integration

Current Integration

• OpenClaw Gateway (10.0.10.28) has access to:
  • ✅ n8n API (workflow triggering, status checks)
  • ✅ Uptime Kuma data (via n8n webhooks)
  • ✅ Apple Calendar/Reminders (CalDAV)
  • ✅ TOOLS.md (local infrastructure notes)
  • ⏳ Gitea (planned - will serve as knowledge base)

Planned Integration

• Gitea as Source of Truth:

  • AI agents can read infrastructure docs
  • Search for configurations
  • Update documentation automatically
  • Query printer profiles, Docker Compose files, scripts

• Example Use Cases:

  • "What port does Home Assistant run on?" → Query Gitea infrastructure/service-inventory.md
  • "What's the Bambu A1 nozzle temp for PLA?" → Query 3d-print-farm/printer-profiles/bambu-a1.ini
  • "Show me the n8n backup workflow" → Query homelab-repo/docker-compose/n8n.yml

Agent Capabilities

• Read: Configuration files, documentation, scripts
• Search: Git history, grep for specific settings
• Update: Commit documentation changes
• Notify: Alert Fred to infrastructure changes or issues

---

Appendix A: Quick Reference

Common Commands

SSH to Proxmox hosts:

ssh root@10.0.10.2  # router-pve
ssh root@10.0.10.3  # main-pve
ssh root@10.0.10.4  # pve-storage

Access Web UIs:

Proxmox: https://10.0.10.2:8006
Home Assistant: http://10.0.10.24:8123
n8n: http://10.0.10.22:5678
Uptime Kuma: http://10.0.10.26:3001
OpenClaw: http://10.0.10.28:3000

OMV Shares:

# From Windows
\\10.0.10.5\data

# From Linux
mount.cifs //10.0.10.5/data /mnt/omv -o guest,vers=3.0

Key Files

OpenClaw Workspace:

• /root/.openclaw/workspace/TOOLS.md - Infrastructure notes
• /root/.openclaw/workspace/obsidian-vault/ - Documentation vault
• /root/.openclaw/workspace/.caldav-config.json - Apple Calendar config

OMV Storage:

• \\10.0.10.5\data\VA-Strategy\ - Fred's VA claim documents
• \\10.0.10.5\data\backups\ - Backup destination
• \\10.0.10.5\data\INFRASTRUCTURE-AUDIT-REPORT.md - Claude Code audit

Important Credentials

Stored in workspace:

• n8n API key: TOOLS.md
• Apple CalDAV: .caldav-config.json
• SSH keys: /root/.ssh/

Not stored (need to retrieve):

• Proxmox root passwords
• OMV admin password
• Step-CA admin credentials

---

Appendix B: Session Changelog

2026-02-04 (Previous Session):

• Created VA strategy documents for Fred
• Created Kobe VA dependent benefits documents
• Set up morning brief cron job
• Infrastructure discussions

2026-02-05 (This Session):

• 02:48 UTC: CalDAV integration with Apple Calendar ✅
• 02:54 UTC: Obsidian vault created ✅
• 14:23 UTC: Research on 2026 homelab best practices ✅
• 14:23 UTC: Research on GPU passthrough for 3D printing ✅
• 15:13 UTC: SSH key generated for router-pve ✅
• 15:17 UTC: SSH key added to router-pve (confirmed by Fred) ✅
• 15:23 UTC: SSH connection issue discovered (hanging) ⚠️
• 15:24 UTC: This comprehensive audit completed ✅

---

Next Steps (Prioritized)

1. Deploy Gitea on router-pve (IN PROGRESS - this session)
2. Fred to copy Claude Code audit report for comparison
3. Document current backup system in detail
4. Configure router-pve 4TB HDD as backup target
5. Test backup restore procedure
6. Troubleshoot SSH connectivity issue (OpenClaw → router-pve)
7. Complete Obsidian vault setup (Fred's side)
8. Deploy Vaultwarden (Week 1)
9. Begin 3D printing business infrastructure planning
10. Research GPU options for CAD workstation

---

End of Audit Report

This audit will be updated regularly as infrastructure changes are made.

For questions or clarifications, contact:

• Funky (OpenClaw Agent) - Available in OpenClaw chat
• Fred Book - Infrastructure owner