FreeAI unifies local LLM serving, autonomous SDLC agents, workflow automation, security tooling, and a full desktop environment — one self-hosted stack.
Every subsystem is independently deployable yet fully integrated into a single operational platform.
Classifies prompts, routes to best backend, automatic fallback chains, LRU caching, and degenerate-output detection. Runs on port 8010.
22+ agents across 7 SDLC phases: plan, code, verify, fix, review, document, package. Runs real compilers inside sandboxes.
Visual pipeline builder with parallel/sequential steps, validation, audit logs, and template library. Port 8040.
llama.cpp (:9001), vLLM (:9002), FreeToken MoE (:9100). GPU warmup, eco/balanced/perf tuning modes.
19-page dashboard with GPU telemetry, alert system, model shelf, workflow designer, SDLC run viewer. Port 8030.
Full XFCE desktop accessible via VNC/noVNC on port 6080. VS Code, terminal, file manager pre-installed.
Each service runs on its own port with LAN-only bindings by default.
| Port | Service | Description |
|---|---|---|
:8010 | Router | Prompt classification, model routing, fallback chains, response cache |
:8020 | Agents API | SDLC agent orchestration, session memory, sandbox control |
:8030 | Dashboard | 19-page web UI — GPU telemetry, alerts, workflow designer |
:8040 | Workflow Engine | Visual pipeline builder, validation, audit logs |
:8050 | Autonomous SDLC | Full project generation from natural language specs |
:8090 | Hermes CLI | CLI agent orchestrator, multi-provider SDKs |
:9001 | llama.cpp | GGUF inference with CUDA acceleration |
:9002 | vLLM | High-throughput PagedAttention serving (optional) |
:9100 | FreeToken | Edge MoE 290B+ on consumer GPUs |
:6080 | noVNC | Web-based desktop access (XFCE) |
Choose the deployment model that matches your infrastructure.
Ubuntu + NVIDIA drivers. Best GPU performance. Auto-driver detection, CUDA toolkit integration.
curl -fsSL install-stack.sh | bash
Split or all-in-one profiles. Runs on any host with NVIDIA Container Toolkit.
docker compose --profile allinone up -d
Auto-scaling, service mesh, horizontal pod autoscaling for production fleets.
kubectl apply -f k8s/manifests/
RunPod, Vast.ai, Lambda Labs, Hetzner. One-click deployment scripts included.
bash deploy/runpod.sh
Bootable USB with Ubuntu/Kodachi/Kali/NixOS variants. Install or try without touching disk.
dd if=FreeAI.iso of=/dev/sdX bs=4M
No GPU? Set MOCK_LLM=1 and run the full stack on CPU for development and testing.
MOCK_LLM=1 python3 router/router.py
Security is not an afterthought — it's baked into every layer.
All model servers bind to 127.0.0.1. UFW firewall exposes only SSH and dashboard ports.
Every agent runs in an isolated workspace with file path restrictions, size limits, and command blocking.
Human-in-the-loop gates for 6 danger categories — exploit, persistence, credential dump, and more.
Sliding-window failure detection on router backends. Automatic fallback when confidence drops below threshold.
All secrets encrypted with age key. Decryption on-demand at runtime — never stored in plaintext.
Self-hosting ensures ISO 27001, DOD IL4, FedRAMP, NIST 800-53, CMMC Level 2 & 3 compliance posture.