Local AI Installation - running on your hardware

Local AI means inference runs on your GPU, not on a server farm halfway across the country. Your prompts, your documents, your generated outputs - none of it crosses the internet unless you explicitly let it. Models live on your SSD, weights load into VRAM, and the only network traffic involved is the initial download.

You can run completely offline once installed. We configure firewall rules and platform settings so models do not phone home or send telemetry. If you choose to bridge to the internet (for example, a cloud-fallback model or a web search tool), that integration is opt-in and visible - no hidden traffic.

Open-weight models like Llama 3.x, Mistral, Qwen, Gemma, and Phi run as quantized files (GGUF, GPTQ, AWQ). Quantization compresses the model so a 7B-parameter model can fit comfortably in 8GB VRAM (Q4_K_M) or a 70B can run on a 24GB+ card with offloading. We help you pick the right size/quant tradeoff for your hardware.

Context window is how much the model can "see" at once - modern open models support 8k–128k tokens. We tune context length to your VRAM budget so long documents and chat history work without crashing. Expect 20–80 tokens/sec on a mid-range GPU; faster on Apple Silicon with Metal or on enthusiast NVIDIA cards.

• Recommended runners: Ollama, LM Studio, llama.cpp, OpenWebUI
• Use cases: chat, summarization, code assist, structured data extraction, agent workflows
• Drop-in OpenAI-compatible API for tools that expect ChatGPT (n8n, Cursor, Continue, etc.)

Stable Diffusion (SD 1.5, SDXL, SD3), Flux, and other diffusion models generate images from text. ComfyUI gives you node-graph control over every step (sampling, ControlNet, upscaling). SwarmUI puts a simpler, form-based UI on the same ComfyUI engine — most of the power with a far gentler learning curve. FOOCUS / Forge / A1111 are simpler form-based UIs for quick iteration. We install the runner you prefer and pre-load checkpoints from Civitai or Hugging Face.

LoRA files are small fine-tunes (50–200MB) that nudge a base model toward a style, character, or subject - you can stack them at generation time. Typical SDXL render: 5–15 seconds for 1024×1024 on an RTX 4070+; Flux is slower but higher quality. We tune VAE, sampler, and step counts so you get crisp output without waiting forever.

• ControlNet for pose, depth, edge, and reference-image control
• Upscalers: ESRGAN, Ultrasharp, SwinIR for crisp 4k output
• Inpainting / outpainting workflows for editing existing images

Open video models like Wan 2.x, Hunyuan Video, AnimateDiff, and CogVideoX let you generate short clips locally. Quality is now on par with mid-tier cloud generators for 3–6 second clips at 720p–1080p. AnimateDiff layers motion onto a Stable Diffusion image; Wan and Hunyuan are full text-to-video models with better temporal coherence.

Be realistic about generation time: a 5-second 720p clip on a 16GB card is typically 3–10 minutes. We configure ComfyUI workflows, install the right schedulers, and dial settings so you can iterate without locking up your machine for an hour per render.

Local audio tooling spans music generation (MusicGen, Stable Audio Open, AudioCraft), voice cloning (RVC, Tortoise, XTTS), and speech-to-text (Whisper). You can produce full instrumental tracks from a text prompt, clone a voice from a 30-second sample, or transcribe meetings with timestamps - all on your hardware.

For creators, the combination is powerful: write copy locally with an LLM, generate cover art with Stable Diffusion, narrate it with a cloned voice via XTTS, and score it with MusicGen - no subscriptions, no platform terms-of-service to worry about.

Different platforms suit different users. We recommend and install based on your goals, but you are never locked in - models are interchangeable across most runners.

• Ollama - simplest text-LLM runner; one-line model downloads, OpenAI-compatible API, perfect for backends and devs
• LM Studio - friendly desktop app for chat with local models, drag-and-drop GGUF, great for non-technical users
• OpenWebUI - self-hosted ChatGPT-style web interface that pairs with Ollama for multi-user, multi-model setups
• ComfyUI - node-graph UI; the standard for image/video and any complex multi-model workflow
• SwarmUI / A1111 / Forge / FOOCUS - form-based image-generation UIs; faster onboarding than Comfy (SwarmUI runs on the Comfy engine, so you can graduate to node graphs later)
• OpenClaw - privacy-focused desktop runner with built-in tooling for files, calendars, and integrations

Docker isolates the AI stack from the rest of your system. Your CUDA driver, Python version, and dependencies stay clean - upgrading or removing the stack is one command. Model weights persist in a mounted volume so reinstalls do not re-download tens of gigabytes.

Direct installation gives slightly better raw performance (no container overhead), tighter system integration, and easier GPU passthrough for some setups. We recommend Docker for most users; direct install for power users running multiple isolated environments or pushing every last token/sec out of high-end hardware.

Out of the box, models know general knowledge - they do not know your business. There are three ways to teach them:

• Prompting - system prompts and instruction templates; instant, free, works for most use cases
• RAG (retrieval-augmented generation) - index your documents (Obsidian vault, PDFs, code) and inject relevant chunks at query time; no model retraining required
• Fine-tuning (LoRA / QLoRA via Axolotl, LMTuner, DeepSpeed) - actually update model weights on your data; needed for style mimicry, domain jargon, or behavior changes prompting cannot reach

GPU VRAM is the most important spec. CPU and RAM matter for offloading large models and running multiple workloads simultaneously, but VRAM dictates which models you can run at full speed.

• Minimum (light text models, small image models): NVIDIA GPU with 8GB VRAM, 16GB system RAM, 50GB free SSD
• Recommended (most users - comfortable with SDXL, 13B LLMs, video at lower res): NVIDIA 16GB+ VRAM, 32GB RAM, 100GB SSD
• Enthusiast (70B LLMs, fast video, multi-model workflows): NVIDIA 24GB+ VRAM (4090 / 3090 / 4090 Ti), 64GB RAM, 500GB+ NVMe
• AMD ROCm and Intel Arc work for some tools but compatibility is uneven - NVIDIA is the safe default
• Apple Silicon (M1/M2/M3/M4 with unified memory) is excellent for LLMs via llama.cpp Metal backend