{"data":{"items":[{"id":"864346ae-bb89-4940-94a4-3510d1e84a3a","excerpt":"I Replaced $100+/month in GEMINI API Costs with a €2000 eBay Mac Studio — Here is my Local, Self-Hosted AI Agent System Running Qwen 3.5 35B at 60 Tokens/Sec (The Full Stack Breakdown) — # TL;DR: self-hosted \"Trinity\" system — three AI agents (Lucy, Neo, Eli) coordinating through a single Telegram chat, powered by a Qw","url":"https://www.reddit.com/r/MacStudio/comments/1ri8ldg/i_replaced_100month_in_gemini_api_costs_with_a/","role":"pain","weight":1.4796968,"occurredAt":"2026-03-01T20:46:56.000Z","sourceKey":"reddit","sourceName":"Reddit","credibility":0.62,"venue":"MacStudio","intent":"feature_request","painScore":0.4908783,"sentiment":-0.4375,"confidence":0.9925,"matchedPatterns":["free_tier","missing_feature","workaround","manual_process"],"statement":"MLX doesn't support the --tool-call-parser flag that vLLM/SGLang offer.","title":"I Replaced $100+/month in GEMINI API Costs with a €2000 eBay Mac Studio — Here is my Local, Self-Hosted AI Agent System Running Qwen 3.5 35B at 60 Tokens/Sec (The Full Stack Breakdown)","body":"# TL;DR: self-hosted \"Trinity\" system — three AI agents (Lucy, Neo, Eli) coordinating through a single Telegram chat, powered by a Qwen 3.5 35B-A3B-4bit model running locally on a Mac Studio M1 Ultra I got for under €2K off eBay. No more paid LLM API costs. Zero cloud dependencies. Every component — LLM, vision, text-to-speech, speech-to-text, document processing — runs on my own hardware. Here's exactly how I built it.\n\n# 📍 Where I Was: The January Stack\n\nI posted here a few months ago about building Lucy — my autonomous virtual agent. Back then, the stack was:\n\n* **Brain:** Google Gemini 3 Flash (paid API)\n* **Orchestration:** n8n (self-hosted, Docker)\n* **Eyes:** Skyvern (browser automation)\n* **Hands:** Agent Zero (code execution)\n* **Hardware:** Old MacBook Pro 16GB running Ubuntu Server\n\nIt worked. Lucy had 25+ connected tools, managed emails, calendars, files, sent voice notes, generated images, tracked expenses — the whole deal. But there was a problem: **I was bleeding $90-125/month in API costs**, and every request was leaving my network, hitting Google's servers, and coming back. For a system I wanted to deploy to privacy-conscious clients? That's a dealbreaker.\n\nI knew the endgame: **run everything locally**. I just needed the hardware.\n\n# 🖥️ The Mac Studio Score \n\n# I'd been stalking eBay for weeks. Then I saw it:\n\n**Apple Mac Studio M1 Ultra — 64GB Unified RAM, 2TB SSD, 20-Core CPU, 48-Core GPU.**\n\nThe seller was in the US. Listed price was originally around $1,850, I put it in my watchlist. The seller shot me an offer, if was in a rush to sell. **Final price: $1,700 USD+**. I'm based in Spain. Enter[ MyUS.com](https://myus.com/) — a US forwarding service. They receive your package in Florida, then ship it internationally. Shipping + Spanish import duty came to **€445**.\n\n**Total cost: \\~€1,995 all-in.**\n\nFor context, the exact same model sells for **€3,050+ on the European black market website** right now. I essentially got it for 33% off.\n\nWhy the M1 Ultra specifically?\n\n* **64GB unified memory** = GPU and CPU share the same RAM pool. No PCIe bottleneck.\n* **48-core GPU** = Apple's Metal framework accelerates ML inference natively\n* **MLX framework** = Apple's open-source ML library, optimized specifically for Apple Silicon\n* **The math:** Qwen 3.5 35B-A3B in 4-bit quantization needs \\~19GB VRAM. With 64GB unified, I have headroom for the model + vision + TTS + STT + document server all running simultaneously.\n\n# 🧠 The Migration: Killing Every Paid API on n8n\n\nThis was the real project. Over a period of intense building sessions, I systematically replaced every cloud dependency with a local alternative. Here's what changed:\n\n# The LLM: Qwen 3.5 35B-A3B-4bit via MLX\n\nThis is the crown jewel. Qwen 3.5 35B-A3B is a Mixture-of-Experts model — 35 billion total parameters, but only \\~3 billion active per token. The result? **Insane speed on Apple Silicon.**\n\n**My benchmarks on the M1 Ultra:**\n\n* **\\~60 tokens/second** generation speed\n* **\\~500 tokens** test messages completing in seconds\n* **19GB VRAM** footprint (4-bit quantization via mlx-community)\n* Served via mlx\\_lm.server on port 8081, OpenAI-compatible API\n\nI run it using a custom Python launcher (start\\_qwen.py) managed by PM2:\n\nimport mlx.nn as nn\n\n\\# Monkey-patch for vision\\_tower weight compatibility\n\noriginal\\_load = nn.Module.load\\_weights\n\ndef patched\\_load(self, weights, strict=True):\n\n   return original\\_load(self, weights, strict=False)\n\nnn.Module.load\\_weights = patched\\_load\n\nfrom mlx\\_lm.server import main\n\nimport sys\n\nsys.argv = \\['server', '--model', 'mlx-community/Qwen3.5-35B-A3B-4bit',\n\n'--port', '8081', '--host', '0.0.0.0'\\]\n\nmain()\n\n**The war story behind that monkey-patch:** When Qwen 3.5 first dropped, the MLX conversion had a vision\\_tower weight mismatch that would crash on load with strict=True. The model wouldn't start. Took hours of debugging crash logs to figure out the fix was a one-liner: load with strict=False. That patch has been running stable ever since.\n\n**The download drama:** HuggingFace's new xet storage system was throttling downloads so hard the model kept failing mid-transfer. I ended up manually curling all 4 model shards (\\~19GB total) one by one from the HF API. Took patience, but it worked.\n\nFor n8n integration, Lucy connects to Qwen via an OpenAI-compatible Chat Model node pointed at http://mylocalhost\\*\\*\\*/v1. From Qwen's perspective, it's just serving an OpenAI API. From n8n's perspective, it's just talking to \"OpenAI.\" Clean abstraction, I'm still stocked that worked!\n\n# Vision: Qwen2.5-VL-7B (Port 8082)\n\nLucy can analyze images — food photos for calorie tracking, receipts for expense logging, document screenshots, you name it. Previously this hit Google's Vision API. Now it's a local Qwen2.5-VL model served via mlx-vlm.\n\n# Text-to-Speech: Qwen3-TTS (Port 8083)\n\nLucy sends daily briefings as voice notes on Telegram. The TTS uses Qwen3-TTS-12Hz-1.7B-Base-bf16, running locally. We prompt it with a consistent female voice and prefix the text with a voice description to keep the output stable, it's remarkably good for a fully local, open-source TTS, I have stopped using 11lab since then for my content creation as well.\n\n# Speech-to-Text: Whisper Large V3 Turbo (Port 8084)\n\nWhen I send voice messages to Lucy on Telegram, Whisper transcribes them locally. Using mlx-whisper with the large-v3-turbo model. Fast, accurate, no API calls.\n\n# Document Processing: Custom Flask Server (Port 8085)\n\nPDF text extraction, document analysis — all handled by a lightweight local server.\n\n**The result:** Five services running simultaneously on the Mac Studio via PM2, all accessible over the local network:\n\n┌────────────────┬──────────┬──────────┐\n\n│ Service        │ Port     │ VRAM     │\n\n├────────────────┼──────────┼──────────┤\n\n│ Qwen 3.5 35B  │ 8081     │ 18.9 GB  │\n\n│ Qwen2.5-VL    │ 8082     │ \\~4 GB    │\n\n│ Qwen3-TTS     │ 8083     │ \\~2 GB    │\n\n│ Whisper STT   │ 8084     │ \\~1.5 GB  │\n\n│ Doc Server    │ 8085     │ minimal  │\n\n└────────────────┴──────────┴──────────┘\n\nAll managed by PM2. All auto-restart on crash. All surviving reboots.\n\n# 🏗️ The Two-Machine Architecture\n\nThis is where it gets interesting. I don't run everything on one box. I have two machines connected via **Starlink**:\n\n# Machine 1: MacBook Pro (Ubuntu Server) — \"The Nerve Center\"\n\nRuns:\n\n* **n8n** (Docker) — The orchestration brain. 58 workflows, 20 active.\n* **Agent Zero / Neo** (Docker, port 8010) — Code execution agent (as of now gemini 3 flash)\n* **OpenClaw / Eli** (metal process, port 18789) — Browser automation agent (mini max 2.5)\n* **Cloudflare Tunnel** — Exposes everything securely to the internet behind email password loggin.\n\n# Machine 2: Mac Studio M1 Ultra — \"The GPU Powerhouse\"\n\nRuns all the ML models for n8n:\n\n* Qwen 3.5 35B (LLM)\n* Qwen2.5-VL (Vision)\n* Qwen3-TTS (Voice)\n* Whisper (Transcription)\n* Open WebUI (port 8080)\n\n# The Network\n\nBoth machines sit on the same local network via Starlink router. The MacBook Pro (n8n) calls the Mac Studio's models over LAN. Latency is negligible — we're talking local network calls.\n\n**Cloudflare Tunnels** make the system accessible from anywhere without opening a single port:\n\nagent.\\*\\*\\*.com    → n8n (MacBook Pro)\n\narchitect.\\*\\*\\*.com → Agent Zero (MacBook Pro) \n\nchat.\\*\\*\\*.com     → Open WebUI (Mac Studio)\n\noracle.\\*\\*\\*.com   → OpenClaw Dashboard (MacBook Pro)\n\nZero-trust architecture. TLS end-to-end. No open ports on my home network. The tunnel runs via a token-based config managed in Cloudflare's dashboard — no local config files to maintain.\n\n# 🤖 Meet The Trinity: Lucy, Neo, and Eli\n\n# 👩🏼‍💼 LUCY — The Executive Architect (The Brain)\n\n**Powered by:** Qwen 3.5 35B-A3B (local) via n8n\n\nLucy is the face of the operation. She's an AI Agent node in n8n with a massive system prompt (\\~4000 tokens) that defines her personality, rules, and tool protocols. She communicates via:\n\n* **Telegram** (text, voice, images, documents)\n* **Email** (Gmail read/write for her account + boss accounts)\n* **SMS** (Twilio)\n* **Phone** (Vapi integration — she can literally call restaurants and book tables)\n* **Voice Notes** (Qwen3-TTS, sends audio briefings)\n\nHer daily routine:\n\n* 7 AM: Generates daily briefing (weather, calendar, top 10 news) + voice note\n* Runs \"heartbeat\" scans every 20 minutes (unanswered emails, upcoming calendar events)\n* Every 6 hours: World news digest, priority emails, events of the day\n\nHer toolkit (26+ tools connected via n8n): Google Calendar, Tasks, Drive, Docs, Sheets, Contacts, Translate | Gmail read/write | Notion | Stripe | Web Search | Wikipedia | Image Generation | Video Generation | Vision AI | PDF Analysis | Expense Tracker | Calorie Tracker | Invoice Generator | Reminders | Calculator | Weather | And the two agents below ↓\n\n**The Tool Calling Challenge (Real Talk):**\n\nGetting Qwen 3.5 to reliably call tools through n8n was one of the hardest parts. The model is trained on qwen3\\_coder XML format for tool calls, but n8n's LangChain integration expects Hermes JSON format. MLX doesn't support the --tool-call-parser flag that vLLM/SGLang offer.\n\nThe fixes that made it work:\n\n* **Temperature: 0.5** (more deterministic tool selection)\n* **Frequency penalty: 0** (Qwen hates non-zero values here — it causes repetition loops)\n* **Max tokens: 4096** (reducing this prevented GPU memory crashes on concurrent requests)\n* **Aggressive system prompt engineering:** Explicit tool matching rules — \"If message contains 'Eli' + task → call ELI tool IMMEDIATELY. No exceptions.\"\n* **Tool list in the message prompt itself**, not just the system prompt — Qwen needs the reinforcement, this part is key!\n\nPrompt (User Message):\n\n=\\[ROUTING\\_DATA: platform={{$json.platform}} | chat\\_id={{$json.chat\\_id}} | message\\_id={{$json.message\\_id}} | photo\\_file\\_id={{$json.photo\\_file\\_id}} | doc\\_file\\_id={{$json.document\\_file\\_id}} | album={{$json.media\\_group\\_id || 'none'}}\\]\n\n\\[TOOL DIRECTIVE: If this task requires ANY action, you MUST call the matching tool. Do NOT simulate. EXECUTE it. Tools include: weather, email, gmail, send email, calendar, event, tweet, X post, LinkedIn, invoice, reminder, timer, set reminder, Stripe balance, tasks, google tasks, search, web search, sheets, spreadsheet, contacts, voice, voice note, image, image generation, image resize, video, video generation, translate, wikipedia, Notion, Google Drive, Google Docs, PDF, journal, diary, daily report, calculator, math, expense, calorie, SMS, transcription, Neo, Eli, OpenClaw, browser automation, memory, LTM, past chats.\\]\n\n{{ $json.input }}\n\n\\+System Message:\n\n...\n\n\\### 5. TOOL PROTOCOLS\n\n\\[TOOL DIRECTIVE: If this task requires ANY action, you MUST call the matching tool. Do NOT simulate. EXECUTE it.\\]\n\nSPREADSHEETS: Find File ID via Drive Doc Search → call Google Sheet tool. READ: {\"action\":\"read\",\"file\\_id\":\"...\",\"tab\\_hint\":\"...\"} WRITE: {\"action\":\"append\",\"file\\_id\":\"...\",\"data\":{...}}\n\nCONTACTS: Call Google Contacts → read list yourself to find person.\n\nFILES: Direct upload = content already provided, do NOT search Drive. Drive search = use keyword then File Reader with ID.\n\nDRIVE LINKS: System auto-passes file. Summarize contents, extract key numbers/actions. If inaccessible → tell user to adjust permissions.\n\nDAILY REPORT: ALWAYS call \"Daily report\" workflow tool. Never generate yourself.\n\nVOICE NOTE (triggers: \"send as voice note\", \"reply in audio\", \"read this to me\"):\n\nDraft response → clean all Markdown/emoji → call Voice Note tool → reply only \"Sending audio note now...\"\n\nREMINDER (triggers: \"remind me in X to Y\"):\n\nCalculate delay\\_minutes → call Set Reminder with reminder\\_text, delay\\_minutes, chat\\_id → confirm.\n\nJOURNAL (triggers: \"journal\", \"log this\", \"add to diary\"):\n\nProofread (fix grammar, keep tone) → format: \\[YYYY-MM-DD HH:mm\\] \\[Text\\] → append to Doc ID: 1RR45YRvIjbLnkRLZ9aSW0xrLcaDs0SZHjyb5EQskkOc → reply \"Journal updated.\"\n\nINVOICE: Extract Client Name, E","offTopic":true},{"id":"66cf9349-5bae-4423-a476-f73973ac65c0","excerpt":"I Replaced $100+/month in GEMINI API Costs with a €2000 eBay Mac Studio — Here is my Local, Self-Hosted AI Agent System Running Qwen 3.5 35B at 60 Tokens/Sec (The Full Stack Breakdown) — # TL;DR: self-hosted \"Trinity\" system — three AI agents (Lucy, Neo, Eli) coordinating through a single Telegram chat, powered by a Qw","url":"https://www.reddit.com/r/n8n/comments/1ri8922/i_replaced_100month_in_gemini_api_costs_with_a/","role":"pain","weight":1.4796968,"occurredAt":"2026-03-01T20:33:42.000Z","sourceKey":"reddit","sourceName":"Reddit","credibility":0.62,"venue":"n8n","intent":"feature_request","painScore":0.4908783,"sentiment":-0.4375,"confidence":0.9925,"matchedPatterns":["free_tier","missing_feature","workaround","manual_process"],"statement":"MLX doesn't support the --tool-call-parser flag that vLLM/SGLang offer.","title":"I Replaced $100+/month in GEMINI API Costs with a €2000 eBay Mac Studio — Here is my Local, Self-Hosted AI Agent System Running Qwen 3.5 35B at 60 Tokens/Sec (The Full Stack Breakdown)","body":"# TL;DR: self-hosted \"Trinity\" system — three AI agents (Lucy, Neo, Eli) coordinating through a single Telegram chat, powered by a Qwen 3.5 35B-A3B-4bit model running locally on a Mac Studio M1 Ultra I got for under €2K off eBay. No more paid LLM API costs. Zero cloud dependencies. Every component — LLM, vision, text-to-speech, speech-to-text, document processing — runs on my own hardware. Here's exactly how I built it.\n\n# 📍 Where I Was: The January Stack\n\nI posted here a few months ago about building Lucy — my autonomous virtual agent. Back then, the stack was:\n\n* **Brain:** Google Gemini 3 Flash (paid API)\n* **Orchestration:** n8n (self-hosted, Docker)\n* **Eyes:** Skyvern (browser automation)\n* **Hands:** Agent Zero (code execution)\n* **Hardware:** Old MacBook Pro 16GB running Ubuntu Server\n\nIt worked. Lucy had 25+ connected tools, managed emails, calendars, files, sent voice notes, generated images, tracked expenses — the whole deal. But there was a problem: **I was bleeding $90-125/month in API costs**, and every request was leaving my network, hitting Google's servers, and coming back. For a system I wanted to deploy to privacy-conscious clients? That's a dealbreaker.\n\nI knew the endgame: **run everything locally**. I just needed the hardware.\n\n# 🖥️ The Mac Studio Score (How to Buy Smart)\n\nI'd been stalking eBay for weeks. Then I saw it:\n\n**Apple Mac Studio M1 Ultra — 64GB Unified RAM, 2TB SSD, 20-Core CPU, 48-Core GPU.**\n\nThe seller was in the US. Listed price was originally around $1,850, I put it in my watchlist. The seller shot me an offer, if was in a rush to sell. **Final price: $1,700 USD+**. I'm based in Spain. Enter[ MyUS.com](https://myus.com/) — a US forwarding service. They receive your package in Florida, then ship it internationally. Shipping + Spanish import duty came to **€445**.\n\n**Total cost: \\~€1,995 all-in.**\n\nFor context, the exact same model sells for **€3,050+ on the European black market website** right now. I essentially got it for 33% off.\n\nWhy the M1 Ultra specifically?\n\n* **64GB unified memory** = GPU and CPU share the same RAM pool. No PCIe bottleneck.\n* **48-core GPU** = Apple's Metal framework accelerates ML inference natively\n* **MLX framework** = Apple's open-source ML library, optimized specifically for Apple Silicon\n* **The math:** Qwen 3.5 35B-A3B in 4-bit quantization needs \\~19GB VRAM. With 64GB unified, I have headroom for the model + vision + TTS + STT + document server all running simultaneously.\n\n# 🧠 The Migration: Killing Every Paid API on n8n\n\nThis was the real project. Over a period of intense building sessions, I systematically replaced every cloud dependency with a local alternative. Here's what changed:\n\n# The LLM: Qwen 3.5 35B-A3B-4bit via MLX\n\nThis is the crown jewel. Qwen 3.5 35B-A3B is a Mixture-of-Experts model — 35 billion total parameters, but only \\~3 billion active per token. The result? **Insane speed on Apple Silicon.**\n\n**My benchmarks on the M1 Ultra:**\n\n* **\\~60 tokens/second** generation speed\n* **\\~500 tokens** test messages completing in seconds\n* **19GB VRAM** footprint (4-bit quantization via mlx-community)\n* Served via mlx\\_lm.server on port 8081, OpenAI-compatible API\n\nI run it using a custom Python launcher (start\\_qwen.py) managed by PM2:\n\nimport mlx.nn as nn\n\n\\# Monkey-patch for vision\\_tower weight compatibility\n\noriginal\\_load = nn.Module.load\\_weights\n\ndef patched\\_load(self, weights, strict=True):\n\n   return original\\_load(self, weights, strict=False)\n\nnn.Module.load\\_weights = patched\\_load\n\nfrom mlx\\_lm.server import main\n\nimport sys\n\nsys.argv = \\['server', '--model', 'mlx-community/Qwen3.5-35B-A3B-4bit',\n\n'--port', '8081', '--host', '0.0.0.0'\\]\n\nmain()\n\n**The war story behind that monkey-patch:** When Qwen 3.5 first dropped, the MLX conversion had a vision\\_tower weight mismatch that would crash on load with strict=True. The model wouldn't start. Took hours of debugging crash logs to figure out the fix was a one-liner: load with strict=False. That patch has been running stable ever since.\n\n**The download drama:** HuggingFace's new xet storage system was throttling downloads so hard the model kept failing mid-transfer. I ended up manually curling all 4 model shards (\\~19GB total) one by one from the HF API. Took patience, but it worked.\n\nFor n8n integration, Lucy connects to Qwen via an OpenAI-compatible Chat Model node pointed at http://mylocalhost\\*\\*\\*/v1. From Qwen's perspective, it's just serving an OpenAI API. From n8n's perspective, it's just talking to \"OpenAI.\" Clean abstraction, I'm still stocked that worked!\n\n# Vision: Qwen2.5-VL-7B (Port 8082)\n\nLucy can analyze images — food photos for calorie tracking, receipts for expense logging, document screenshots, you name it. Previously this hit Google's Vision API. Now it's a local Qwen2.5-VL model served via mlx-vlm.\n\n# Text-to-Speech: Qwen3-TTS (Port 8083)\n\nLucy sends daily briefings as voice notes on Telegram. The TTS uses Qwen3-TTS-12Hz-1.7B-Base-bf16, running locally. We prompt it with a consistent female voice and prefix the text with a voice description to keep the output stable, it's remarkably good for a fully local, open-source TTS, I have stopped using 11lab since then for my content creation as well.\n\n# Speech-to-Text: Whisper Large V3 Turbo (Port 8084)\n\nWhen I send voice messages to Lucy on Telegram, Whisper transcribes them locally. Using mlx-whisper with the large-v3-turbo model. Fast, accurate, no API calls.\n\n# Document Processing: Custom Flask Server (Port 8085)\n\nPDF text extraction, document analysis — all handled by a lightweight local server.\n\n**The result:** Five services running simultaneously on the Mac Studio via PM2, all accessible over the local network:\n\n┌────────────────┬──────────┬──────────┐\n\n│ Service        │ Port     │ VRAM     │\n\n├────────────────┼──────────┼──────────┤\n\n│ Qwen 3.5 35B  │ 8081     │ 18.9 GB  │\n\n│ Qwen2.5-VL    │ 8082     │ \\~4 GB    │\n\n│ Qwen3-TTS     │ 8083     │ \\~2 GB    │\n\n│ Whisper STT   │ 8084     │ \\~1.5 GB  │\n\n│ Doc Server    │ 8085     │ minimal  │\n\n└────────────────┴──────────┴──────────┘\n\nAll managed by PM2. All auto-restart on crash. All surviving reboots.\n\n# 🏗️ The Two-Machine Architecture\n\nThis is where it gets interesting. I don't run everything on one box. I have two machines connected via **Starlink**:\n\n# Machine 1: MacBook Pro (Ubuntu Server) — \"The Nerve Center\"\n\nRuns:\n\n* **n8n** (Docker) — The orchestration brain. 58 workflows, 20 active.\n* **Agent Zero / Neo** (Docker, port 8010) — Code execution agent (as of now gemini 3 flash)\n* **OpenClaw / Eli** (metal process, port 18789) — Browser automation agent (mini max 2.5)\n* **Cloudflare Tunnel** — Exposes everything securely to the internet behind email password loggin.\n\n# Machine 2: Mac Studio M1 Ultra — \"The GPU Powerhouse\"\n\nRuns all the ML models for n8n:\n\n* Qwen 3.5 35B (LLM)\n* Qwen2.5-VL (Vision)\n* Qwen3-TTS (Voice)\n* Whisper (Transcription)\n* Open WebUI (port 8080)\n\n# The Network\n\nBoth machines sit on the same local network via Starlink router. The MacBook Pro (n8n) calls the Mac Studio's models over LAN. Latency is negligible — we're talking local network calls.\n\n**Cloudflare Tunnels** make the system accessible from anywhere without opening a single port:\n\nagent.\\*\\*\\*.com    → n8n (MacBook Pro)\n\narchitect.\\*\\*\\*.com → Agent Zero (MacBook Pro) \n\nchat.\\*\\*\\*.com     → Open WebUI (Mac Studio)\n\noracle.\\*\\*\\*.com   → OpenClaw Dashboard (MacBook Pro)\n\nZero-trust architecture. TLS end-to-end. No open ports on my home network. The tunnel runs via a token-based config managed in Cloudflare's dashboard — no local config files to maintain.\n\n# 🤖 Meet The Trinity: Lucy, Neo, and Eli\n\n# 👩🏼‍💼 LUCY — The Executive Architect (The Brain)\n\n**Powered by:** Qwen 3.5 35B-A3B (local) via n8n\n\nLucy is the face of the operation. She's an AI Agent node in n8n with a massive system prompt (\\~4000 tokens) that defines her personality, rules, and tool protocols. She communicates via:\n\n* **Telegram** (text, voice, images, documents)\n* **Email** (Gmail read/write for her account + boss accounts)\n* **SMS** (Twilio)\n* **Phone** (Vapi integration — she can literally call restaurants and book tables)\n* **Voice Notes** (Qwen3-TTS, sends audio briefings)\n\nHer daily routine:\n\n* 7 AM: Generates daily briefing (weather, calendar, top 10 news) + voice note\n* Runs \"heartbeat\" scans every 20 minutes (unanswered emails, upcoming calendar events)\n* Every 6 hours: World news digest, priority emails, events of the day\n\nHer toolkit (26+ tools connected via n8n): Google Calendar, Tasks, Drive, Docs, Sheets, Contacts, Translate | Gmail read/write | Notion | Stripe | Web Search | Wikipedia | Image Generation | Video Generation | Vision AI | PDF Analysis | Expense Tracker | Calorie Tracker | Invoice Generator | Reminders | Calculator | Weather | And the two agents below ↓\n\n**The Tool Calling Challenge (Real Talk):**\n\nGetting Qwen 3.5 to reliably call tools through n8n was one of the hardest parts. The model is trained on qwen3\\_coder XML format for tool calls, but n8n's LangChain integration expects Hermes JSON format. MLX doesn't support the --tool-call-parser flag that vLLM/SGLang offer.\n\nThe fixes that made it work:\n\n* **Temperature: 0.5** (more deterministic tool selection)\n* **Frequency penalty: 0** (Qwen hates non-zero values here — it causes repetition loops)\n* **Max tokens: 4096** (reducing this prevented GPU memory crashes on concurrent requests)\n* **Aggressive system prompt engineering:** Explicit tool matching rules — \"If message contains 'Eli' + task → call ELI tool IMMEDIATELY. No exceptions.\"\n* **Tool list in the message prompt itself**, not just the system prompt — Qwen needs the reinforcement, this part is key!\n\nPrompt (User Message):\n\n=\\[ROUTING\\_DATA: platform={{$json.platform}} | chat\\_id={{$json.chat\\_id}} | message\\_id={{$json.message\\_id}} | photo\\_file\\_id={{$json.photo\\_file\\_id}} | doc\\_file\\_id={{$json.document\\_file\\_id}} | album={{$json.media\\_group\\_id || 'none'}}\\]\n\n\\[TOOL DIRECTIVE: If this task requires ANY action, you MUST call the matching tool. Do NOT simulate. EXECUTE it. Tools include: weather, email, gmail, send email, calendar, event, tweet, X post, LinkedIn, invoice, reminder, timer, set reminder, Stripe balance, tasks, google tasks, search, web search, sheets, spreadsheet, contacts, voice, voice note, image, image generation, image resize, video, video generation, translate, wikipedia, Notion, Google Drive, Google Docs, PDF, journal, diary, daily report, calculator, math, expense, calorie, SMS, transcription, Neo, Eli, OpenClaw, browser automation, memory, LTM, past chats.\\]\n\n{{ $json.input }}\n\n\\+System Message:\n\n...\n\n\\### 5. TOOL PROTOCOLS\n\n\\[TOOL DIRECTIVE: If this task requires ANY action, you MUST call the matching tool. Do NOT simulate. EXECUTE it.\\]\n\nSPREADSHEETS: Find File ID via Drive Doc Search → call Google Sheet tool. READ: {\"action\":\"read\",\"file\\_id\":\"...\",\"tab\\_hint\":\"...\"} WRITE: {\"action\":\"append\",\"file\\_id\":\"...\",\"data\":{...}}\n\nCONTACTS: Call Google Contacts → read list yourself to find person.\n\nFILES: Direct upload = content already provided, do NOT search Drive. Drive search = use keyword then File Reader with ID.\n\nDRIVE LINKS: System auto-passes file. Summarize contents, extract key numbers/actions. If inaccessible → tell user to adjust permissions.\n\nDAILY REPORT: ALWAYS call \"Daily report\" workflow tool. Never generate yourself.\n\nVOICE NOTE (triggers: \"send as voice note\", \"reply in audio\", \"read this to me\"):\n\nDraft response → clean all Markdown/emoji → call Voice Note tool → reply only \"Sending audio note now...\"\n\nREMINDER (triggers: \"remind me in X to Y\"):\n\nCalculate delay\\_minutes → call Set Reminder with reminder\\_text, delay\\_minutes, chat\\_id → confirm.\n\nJOURNAL (triggers: \"journal\", \"log this\", \"add to diary\"):\n\nProofread (fix grammar, keep tone) → format: \\[YYYY-MM-DD HH:mm\\] \\[Text\\] → append to Doc ID: 1RR45YRvIjbLnkRLZ9aSW0xrLcaDs0SZHjyb5EQskkOc → reply \"Journal updated.\"\n\nINVOICE: Extrac","offTopic":true},{"id":"57e80225-3171-4a96-aae3-e60fc9286116","excerpt":"I Replaced $100+/month in GEMINI API Costs with a €2000 eBay Mac Studio — Here is my Local, Self-Hosted AI Agent System Running Qwen 3.5 35B at 60 Tokens/Sec (The Full Stack Breakdown) — I spent 10 weeks and many late nights building this to run 100% locally on a Mac Studio M1 Ultra, successfully replacing a $100/mo AP","url":"https://www.reddit.com/r/LocalLLM/comments/1ri8451/i_replaced_100month_in_gemini_api_costs_with_a/","role":"pain","weight":1.4524461,"occurredAt":"2026-03-01T20:28:35.000Z","sourceKey":"reddit","sourceName":"Reddit","credibility":0.62,"venue":"LocalLLM","intent":"feature_request","painScore":0.49250782,"sentiment":-0.4375,"confidence":0.9731581,"matchedPatterns":["free_tier","missing_feature","workaround","manual_process"],"statement":"MLX doesn't support the --tool-call-parser flag that vLLM/SGLang offer.","title":"I Replaced $100+/month in GEMINI API Costs with a €2000 eBay Mac Studio — Here is my Local, Self-Hosted AI Agent System Running Qwen 3.5 35B at 60 Tokens/Sec (The Full Stack Breakdown)","body":"I spent 10 weeks and many late nights building this to run 100% locally on a Mac Studio M1 Ultra, successfully replacing a $100/mo API bill. I used Claude to help write and structure this post so I could actually share the architecture without typing a novel for three days.\n\nCLAUDE OPUS 4.6 THINKING\n\n**TL;DR:** self-hosted \"Trinity\" system — three AI agents, the brain is the Qwen, coordinating through a single Telegram chat, powered by a Qwen 3.5 35B-A3B-4bit model running locally on a Mac Studio M1 Ultra I got for under €2K off eBay. No more paid LLM API costs. Zero cloud dependencies. Every component — LLM, vision, text-to-speech, speech-to-text, document processing — runs on my own hardware. Here's exactly how I built it.\n\n# 📍 Where I Was: The January Stack\n\nI posted here a few months ago about building Lucy — my autonomous virtual agent. Back then, the stack was:\n\n* **Brain:** Google Gemini 3 Flash (paid API)\n* **Orchestration:** n8n (self-hosted, Docker)\n* **Eyes:** Skyvern (browser automation)\n* **Hands:** Agent Zero (code execution)\n* **Hardware:** Old MacBook Pro 16GB running Ubuntu Server\n\nIt worked. Lucy had 25+ connected tools, managed emails, calendars, files, sent voice notes, generated images, tracked expenses — the whole deal. But there was a problem: **I was bleeding $90-125/month in API costs**, and every request was leaving my network, hitting Google's servers, and coming back. For a system I wanted to deploy to privacy-conscious clients? That's a dealbreaker.\n\nI knew the endgame: **run everything locally**. I just needed the hardware.\n\n# 🖥️ The Mac Studio Score (How to Buy Smart)\n\nI'd been stalking eBay for weeks. Then I saw it:\n\n**Apple Mac Studio M1 Ultra — 64GB Unified RAM, 2TB SSD, 20-Core CPU, 48-Core GPU.**\n\nThe seller was in the US. Listed price was originally around $1,850, I put it in my watchlist. The seller shot me an offer, if was in a rush to sell. **Final price: $1,700 USD+**. I'm based in Spain. Enter[ MyUS.com](https://myus.com/) — a US forwarding service. They receive your package in Florida, then ship it internationally. Shipping + Spanish import duty came to **€445**.\n\n**Total cost: \\~€1,995 all-in.**\n\nFor context, the exact same model sells for **€3,050+ on the European black market website** right now. I essentially got it for 33% off.\n\nWhy the M1 Ultra specifically?\n\n* **64GB unified memory** = GPU and CPU share the same RAM pool. No PCIe bottleneck.\n* **48-core GPU** = Apple's Metal framework accelerates ML inference natively\n* **MLX framework** = Apple's open-source ML library, optimized specifically for Apple Silicon\n* **The math:** Qwen 3.5 35B-A3B in 4-bit quantization needs \\~19GB VRAM. With 64GB unified, I have headroom for the model + vision + TTS + STT + document server all running simultaneously.\n\n# 🧠 The Migration: Killing Every Paid API on n8n\n\nThis was the real project. Over a period of intense building sessions, I systematically replaced every cloud dependency with a local alternative. Here's what changed:\n\n# The LLM: Qwen 3.5 35B-A3B-4bit via MLX\n\nThis is the crown jewel. Qwen 3.5 35B-A3B is a Mixture-of-Experts model — 35 billion total parameters, but only \\~3 billion active per token. The result? **Insane speed on Apple Silicon.**\n\n**My benchmarks on the M1 Ultra:**\n\n* **\\~60 tokens/second** generation speed\n* **\\~500 tokens** test messages completing in seconds\n* **19GB VRAM** footprint (4-bit quantization via mlx-community)\n* Served via mlx\\_lm.server on port 8081, OpenAI-compatible API\n\nI run it using a custom Python launcher (start\\_qwen.py) managed by PM2:\n\nimport mlx.nn as nn\n\n\\# Monkey-patch for vision\\_tower weight compatibility\n\noriginal\\_load = nn.Module.load\\_weights\n\ndef patched\\_load(self, weights, strict=True):\n\n   return original\\_load(self, weights, strict=False)\n\nnn.Module.load\\_weights = patched\\_load\n\nfrom mlx\\_lm.server import main\n\nimport sys\n\nsys.argv = \\['server', '--model', 'mlx-community/Qwen3.5-35B-A3B-4bit',\n\n'--port', '8081', '--host', '0.0.0.0'\\]\n\nmain()\n\n**The war story behind that monkey-patch:** When Qwen 3.5 first dropped, the MLX conversion had a vision\\_tower weight mismatch that would crash on load with strict=True. The model wouldn't start. Took hours of debugging crash logs to figure out the fix was a one-liner: load with strict=False. That patch has been running stable ever since.\n\n**The download drama:** HuggingFace's new xet storage system was throttling downloads so hard the model kept failing mid-transfer. I ended up manually curling all 4 model shards (\\~19GB total) one by one from the HF API. Took patience, but it worked.\n\nFor n8n integration, Lucy connects to Qwen via an OpenAI-compatible Chat Model node pointed at http://mylocalhost\\*\\*\\*/v1. From Qwen's perspective, it's just serving an OpenAI API. From n8n's perspective, it's just talking to \"OpenAI.\" Clean abstraction, I'm still stocked that worked!\n\n# Vision: Qwen2.5-VL-7B (Port 8082)\n\nLucy can analyze images — food photos for calorie tracking, receipts for expense logging, document screenshots, you name it. Previously this hit Google's Vision API. Now it's a local Qwen2.5-VL model served via mlx-vlm.\n\n# Text-to-Speech: Qwen3-TTS (Port 8083)\n\nLucy sends daily briefings as voice notes on Telegram. The TTS uses Qwen3-TTS-12Hz-1.7B-Base-bf16, running locally. We prompt it with a consistent female voice and prefix the text with a voice description to keep the output stable, it's remarkably good for a fully local, open-source TTS, I have stopped using 11lab since then for my content creation as well.\n\n# Speech-to-Text: Whisper Large V3 Turbo (Port 8084)\n\nWhen I send voice messages to Lucy on Telegram, Whisper transcribes them locally. Using mlx-whisper with the large-v3-turbo model. Fast, accurate, no API calls.\n\n# Document Processing: Custom Flask Server (Port 8085)\n\nPDF text extraction, document analysis — all handled by a lightweight local server.\n\n**The result:** Five services running simultaneously on the Mac Studio via PM2, all accessible over the local network:\n\n┌────────────────┬──────────┬──────────┐\n\n│ Service        │ Port     │ VRAM     │\n\n├────────────────┼──────────┼──────────┤\n\n│ Qwen 3.5 35B  │ 8081     │ 18.9 GB  │\n\n│ Qwen2.5-VL    │ 8082     │ \\~4 GB    │\n\n│ Qwen3-TTS     │ 8083     │ \\~2 GB    │\n\n│ Whisper STT   │ 8084     │ \\~1.5 GB  │\n\n│ Doc Server    │ 8085     │ minimal  │\n\n└────────────────┴──────────┴──────────┘\n\nAll managed by PM2. All auto-restart on crash. All surviving reboots.\n\n# 🏗️ The Two-Machine Architecture\n\nThis is where it gets interesting. I don't run everything on one box. I have two machines connected via **Starlink**:\n\n# Machine 1: MacBook Pro (Ubuntu Server) — \"The Nerve Center\"\n\nRuns:\n\n* **n8n** (Docker) — The orchestration brain. 58 workflows, 20 active.\n* **Agent Zero / Neo** (Docker, port 8010) — Code execution agent (as of now gemini 3 flash)\n* **OpenClaw / Eli** (metal process, port 18789) — Browser automation agent (mini max 2.5)\n* **Cloudflare Tunnel** — Exposes everything securely to the internet behind email password loggin.\n\n# Machine 2: Mac Studio M1 Ultra — \"The GPU Powerhouse\"\n\nRuns all the ML models for n8n:\n\n* Qwen 3.5 35B (LLM)\n* Qwen2.5-VL (Vision)\n* Qwen3-TTS (Voice)\n* Whisper (Transcription)\n* Open WebUI (port 8080)\n\n# The Network\n\nBoth machines sit on the same local network via Starlink router. The MacBook Pro (n8n) calls the Mac Studio's models over LAN. Latency is negligible — we're talking local network calls.\n\n**Cloudflare Tunnels** make the system accessible from anywhere without opening a single port:\n\nagent.\\*\\*\\*.com    → n8n (MacBook Pro)\n\narchitect.\\*\\*\\*.com → Agent Zero (MacBook Pro) \n\nchat.\\*\\*\\*.com     → Open WebUI (Mac Studio)\n\noracle.\\*\\*\\*.com   → OpenClaw Dashboard (MacBook Pro)\n\nZero-trust architecture. TLS end-to-end. No open ports on my home network. The tunnel runs via a token-based config managed in Cloudflare's dashboard — no local config files to maintain.\n\n# 🤖 Meet The Trinity: Lucy, Neo, and Eli\n\n# 👩🏼‍💼 LUCY — The Executive Architect (The Brain)\n\n**Powered by:** Qwen 3.5 35B-A3B (local) via n8n\n\nLucy is the face of the operation. She's an AI Agent node in n8n with a massive system prompt (\\~4000 tokens) that defines her personality, rules, and tool protocols. She communicates via:\n\n* **Telegram** (text, voice, images, documents)\n* **Email** (Gmail read/write for her account + boss accounts)\n* **SMS** (Twilio)\n* **Phone** (Vapi integration — she can literally call restaurants and book tables)\n* **Voice Notes** (Qwen3-TTS, sends audio briefings)\n\nHer daily routine:\n\n* 7 AM: Generates daily briefing (weather, calendar, top 10 news) + voice note\n* Runs \"heartbeat\" scans every 20 minutes (unanswered emails, upcoming calendar events)\n* Every 6 hours: World news digest, priority emails, events of the day\n\nHer toolkit (26+ tools connected via n8n): Google Calendar, Tasks, Drive, Docs, Sheets, Contacts, Translate | Gmail read/write | Notion | Stripe | Web Search | Wikipedia | Image Generation | Video Generation | Vision AI | PDF Analysis | Expense Tracker | Calorie Tracker | Invoice Generator | Reminders | Calculator | Weather | And the two agents below ↓\n\n**The Tool Calling Challenge (Real Talk):**\n\nGetting Qwen 3.5 to reliably call tools through n8n was one of the hardest parts. The model is trained on qwen3\\_coder XML format for tool calls, but n8n's LangChain integration expects Hermes JSON format. MLX doesn't support the --tool-call-parser flag that vLLM/SGLang offer.\n\nThe fixes that made it work:\n\n* **Temperature: 0.5** (more deterministic tool selection)\n* **Frequency penalty: 0** (Qwen hates non-zero values here — it causes repetition loops)\n* **Max tokens: 4096** (reducing this prevented GPU memory crashes on concurrent requests)\n* **Aggressive system prompt engineering:** Explicit tool matching rules — \"If message contains 'Eli' + task → call ELI tool IMMEDIATELY. No exceptions.\"\n* **Tool list in the message prompt itself**, not just the system prompt — Qwen needs the reinforcement, this part is key!\n\nPrompt (User Message):\n\n=\\[ROUTING\\_DATA: platform={{$json.platform}} | chat\\_id={{$json.chat\\_id}} | message\\_id={{$json.message\\_id}} | photo\\_file\\_id={{$json.photo\\_file\\_id}} | doc\\_file\\_id={{$json.document\\_file\\_id}} | album={{$json.media\\_group\\_id || 'none'}}\\]\n\n\\[TOOL DIRECTIVE: If this task requires ANY action, you MUST call the matching tool. Do NOT simulate. EXECUTE it. Tools include: weather, email, gmail, send email, calendar, event, tweet, X post, LinkedIn, invoice, reminder, timer, set reminder, Stripe balance, tasks, google tasks, search, web search, sheets, spreadsheet, contacts, voice, voice note, image, image generation, image resize, video, video generation, translate, wikipedia, Notion, Google Drive, Google Docs, PDF, journal, diary, daily report, calculator, math, expense, calorie, SMS, transcription, Neo, Eli, OpenClaw, browser automation, memory, LTM, past chats.\\]\n\n{{ $json.input }}\n\n\\+System Message:\n\n...\n\n\\### 5. TOOL PROTOCOLS\n\n\\[TOOL DIRECTIVE: If this task requires ANY action, you MUST call the matching tool. Do NOT simulate. EXECUTE it.\\]\n\nSPREADSHEETS: Find File ID via Drive Doc Search → call Google Sheet tool. READ: {\"action\":\"read\",\"file\\_id\":\"...\",\"tab\\_hint\":\"...\"} WRITE: {\"action\":\"append\",\"file\\_id\":\"...\",\"data\":{...}}\n\nCONTACTS: Call Google Contacts → read list yourself to find person.\n\nFILES: Direct upload = content already provided, do NOT search Drive. Drive search = use keyword then File Reader with ID.\n\nDRIVE LINKS: System auto-passes file. Summarize contents, extract key numbers/actions. If inaccessible → tell user to adjust permissions.\n\nDAILY REPORT: ALWAYS call \"Daily report\" workflow tool. Never generate yourself.\n\nVOICE NOTE (triggers: \"send as voice note\", \"reply in audio\", \"read this to me\"):\n\nDraft response → clean all Markdown/emoji → call Voice Note tool → reply only \"Sending audio note now...\"\n\nREMINDER (triggers: \"remind me in X to Y\"):\n\nCalculate delay\\_minutes → call Set","offTopic":true},{"id":"75e1eca9-3092-4374-8c1b-ca3aaf34a3c1","excerpt":"Running a Mac as home server and couldn't be happier. Power efficient, fast, small. Roast me! — I finally decided to get a home server a while ago. I've built my own PCs since I was a kid, my first was an AMD K6-2 at 400 MHz with an NVIDIA Riva TNT2 Pro. So I did what I always do: hand-picked the components for a box t","url":"https://www.reddit.com/r/selfhosted/comments/1u9fgo1/running_a_mac_as_home_server_and_couldnt_be/","role":"pricing","weight":1.1235666,"occurredAt":"2026-06-18T19:12:31.000Z","sourceKey":"reddit","sourceName":"Reddit","credibility":0.62,"venue":"selfhosted","intent":"pricing_complaint","painScore":0.48,"sentiment":0.019607844,"confidence":0.75916666,"matchedPatterns":["too_expensive"],"statement":"# Not overpriced anymore The \"Macs are overpriced\" argument has gotten weak.","title":"Running a Mac as home server and couldn't be happier. Power efficient, fast, small. Roast me!","body":"I finally decided to get a home server a while ago. I've built my own PCs since I was a kid, my first was an AMD K6-2 at 400 MHz with an NVIDIA Riva TNT2 Pro. So I did what I always do: hand-picked the components for a box to host and back up our photos and videos. In November that build was €800. By December the same parts were €1,200. (The same setup is now €1800) for 16GB of RAM and no GPU. I hesitated. And the part that really bugged me: I'm a software engineer, and I wanted to be able to run local LLMs. And that build couldn't do it.\n\nI used to laugh at Apple fanboys back when they soldered Intel chips.  \nFast forward to March '26: I bought a used Mac Studio M1 Max (64GB, 4TB) for €1,700 and made it the home server instead.\n\nThen I plugged in a wattmeter and left it running for 16 days. Literally could not believe the numbers first and had to double  check.  It showed 8 watts during \"idle\" (No inference running)!\n\nReal use: 25 Docker containers always on (Immich, Paperless-ngx, Matrix, Synapse, Caddy, AdGuard, Forgejo, Open WebUI, Whisper (speech to text).  \nI used it as workstation too, to run benchmarks during that period.\n\nThe average result after 16 days:\n\n# 11.6 watts average. 50 watts peak, during LLM inference.\n\nThat's about 101 kWh a year, roughly €39 where I live (Germany, some of the most expensive electricity in Europe). For context: our ancient Bose 5.1 surround system pulls 30 watts sitting on standby. A surround system doing nothing draws more than the Mac averages while running my whole stack.\n\nThanks to the unified memory architecture I run a 35B model (Qwen3.6, MLX 4bit) on the same box that averages 12W. The x86 way to do local LLMs is a discrete RTX card in a x86 system, which idles around 40W? (no idea) headless and pulls \\~300W under load. Different league.\n\n# Some notes:\n\n**Docker.** Don't use Docker Desktop on Mac. It's kinda broken: unstable, suddenly eats CPU for nothing. But that's a Docker Desktop problem, not a Mac problem I figured. I switched to OrbStack and it was night and day, stable and light, I forget it's running. I just ran into a networking bug after an update. It was fixed quite fast.\n\n**Storage.** No room for spinning drives inside. I hung a Terramaster 2-bay enclosure off it, 2x6TB WD Red for backups (Time Machien and rsync), plus an encrypted remote copy.\n\n**No ECC RAM.** At home I don't really care. My x86 build wouldn't have had ECC either.\n\n**Remote Access.** SSH works, remote Screen Sharing works (I use it all the time), and I can unlock the disk over SSH after a reboot. With 'Remote Access' enabled, you can SSH into the Mac pre-login. Use an Admin password to unlock the machine and finish booting. Afterward, you can connect via regular SSH or Screen Sharing. No real IPMI though. Console access when the OS is fully down, which hasn't happened yet. When it does, the literal box usually is in the next room.\n\n**Soldered RAM.** You buy what you need up front, no adding later. It is what it is. Buy second hand with as much ram as you can get for your budget.\n\n**macOS as a server.** It's not a server OS, and Apple's update policy is the one thing I actually worry about a bit. The runway is long though: Apple patches the latest three macOS versions, Macs get new OS releases for around 7 years, and no Apple Silicon Mac has been dropped yet, so a 2022 Studio has updates into the early 2030s. The real  occasional annoyance is that updates sometimes force reboots and with FileVault on the box you need to SSH and type in your password once to unlock.  I also set `sudo pmset -a autorestart 1` so it powers back on after an outage. Know those two and headless gets a lot less scary.\n\n# Not overpriced anymore\n\nThe \"Macs are overpriced\" argument has gotten weak. With RAM and SSD prices through the roof right now, a used M1 Max with 64GB and 4TB for €1,700 isn't the expensive option next to an equivalent x86 box anymore. The recent $399 are insane cpu power/efficiency for money for a home server. Mine is overpowered. But I use it for work too. So it's fine.\n\n# tl;dr:\n\nlow power, silent, great for local AI, and plenty of spare compute left for CPU-heavy services. Okayish remote access. Best machine I've bought in a long time. Honestly the best toy since Lego Technic, the whole package. And I think it makes a great home server package.\n\nAnyone else running one as a home server? Curious what bit you that I haven't hit yet. And did anyone else pick one for the power efficiency, or am I alone here?  \nWhat's your average power consumption? Anyone measured?\n\n\\------\n\nHere is the writeup with the numbers measured with a Wattmeter at the wall (and the Terramaster 2-drive bay). You will also find what I do with the server and local LLMs:  \n[https://famstack.dev/guides/mac-mini-mac-studio-home-server-power-consumption/](https://famstack.dev/guides/mac-mini-mac-studio-home-server-power-consumption/)\n\nWhat am I running on that Mac?  \nPhotos, memories, documents, chat, local AI: local and private by default, gets smarter over time. Open sourced, so it is usable for you too. A star and a follow would mean a lot <3  \n[https://github.com/famstack-dev/famstack](https://github.com/famstack-dev/famstack)","offTopic":true},{"id":"573c7e18-1186-40ff-b8a4-4172ac8fb706","excerpt":"I started a homelab because I didn't want to pay an extra $25/mo for a stock screener app I loved… And it now runs my house, powers a self-hosted LLM with the internet unplugged... and gives ~1,800 WoW bots their personalities! — **EDIT:** A bunch of you asked about the WoW bots, so I cleaned up the personality layer a","url":"https://www.reddit.com/r/homelab/comments/1uuwyvv/i_started_a_homelab_because_i_didnt_want_to_pay/","role":"pricing","weight":1.1151514,"occurredAt":"2026-07-13T00:56:41.000Z","sourceKey":"reddit","sourceName":"Reddit","credibility":0.62,"venue":"homelab","intent":"pricing_complaint","painScore":0.27324617,"sentiment":0.69863015,"confidence":0.87583333,"matchedPatterns":["paying_monthly","free_tier"],"statement":"**TL;DR:** I didn't want to pay $30/mo for a stock-screener app I loved, so I built my own — and three years later it's a silent \\~40-service homelab (which replaced a screaming Dell R710), runs my house, keeps working with the internet un…","title":"I started a homelab because I didn't want to pay an extra $25/mo for a stock screener app I loved… And it now runs my house, powers a self-hosted LLM with the internet unplugged... and gives ~1,800 WoW bots their personalities!","body":"**EDIT:** A bunch of you asked about the WoW bots, so I cleaned up the personality layer and open-sourced it → [**github.com/Merrymak3r/wow-llm-personas**](http://github.com/Merrymak3r/wow-llm-personas) — the tiny local-LLM shim that gives the bots their voices (per-bot personas, short memory, bot-to-bot banter). MIT, stdlib-only. The server itself is CMaNGOS + playerbots; this is just the glue that points it at Ollama.\n\n**TL;DR:** I didn't want to pay $30/mo for a stock-screener app I loved, so I built my own — and three years later it's a silent \\~40-service homelab (which replaced a screaming Dell R710), runs my house, keeps working with the internet unplugged, and uses a single RTX 3060 to give \\~1,800 vanilla-WoW bots AI personalities. Recurring cloud bill: basically $0. The whole lab draws an average of \\~161 W.\n\nA couple of years ago there was a stock-portfolio analysis app I genuinely loved — one day, they decided to lock a bunch of features I relied on behind a new subscription tier that took it from $4.99/month to $29.99/month. Rather than pay an extra $25/mo for it, I decided, in true self-proclaimed engineer fashion, that I could probably just build my own.\n\nThat decision escalated 3 years later into a homelab that:\n\n* Runs my house\n* Keeps working with the internet unplugged\n* Hosts \\~40 self-hosted services\n* Powers a fully local voice assistant\n* Gives \\~1,800 WoW bots AI personalities\n\nNone of it was planned. I just kept asking \"what if I self-hosted that too?\" ... and now I can't explain my setup in less than 5 minutes if I tried.\n\n**The evolution**\n\nMy first \"real\" server was a Dell R710. Powerful, cheap, and loud enough to qualify for a noise complaint. Whenever guests stayed over, I had to physically shut it down, because nobody could sleep in the same room as it — and my wife was entering the \"it's either the server or me\" phase of negotiations.\n\nThen I watched a YouTube video literally titled \"The EVERYTHING $300 Fanless Home Server,\" got completely hyped, and bought a Qotom Fanless PC:\n\n* 8-core Atom CPU\n* 64 GB ECC RAM\n* NVMe + SATA storage\n* More Intel NICs than I have ever actually used (I bought it half for the networking I was sure I'd need — reader, I have used exactly none of it)\n\nI was convinced it would replace my entire rack… it did not. What it did was replace the R710 — and that turned out to be the whole win. My \"server\" went from small jet engine to \"can't hear it from a foot away,\" power use dropped to \\~25 watts, and for the first time the thing felt like an appliance instead of an experiment. Honestly, even if it had drawn the same power, killing the noise alone would've been worth it.\n\nThe original goal was simple: build and run my stock portfolio analysis app and stop paying for someone else's. Then it spiraled — backups, then Prometheus, then Grafana, then Loki, then exporters, then OPNsense, then offsite backups, then Home Assistant, then local AI, then an offline library — until one day I looked up and realized I'd built an entire ecosystem. \\~40 services across two machines and the cloud, averaging about 161 watts.\n\n**The part I'm most proud of: one $250 GPU, four jobs**\n\nI only really have 1 capable video card, an RTX 3060 with 12 GB of VRAM. Instead of just gaming or editing videos with it, I kept finding it new jobs.\n\n**Job 1 — Stock analysis.** The original project. Retrieval over SEC filings plus a \"compute it, don't guess it\" step where the model writes the formula and a sandbox runs the actual math. No AI-invented P/E ratios.\n\n**Job 2 — Offline knowledge.** The same GPU answers a reference library I built from offline Wikipedia + Stack Exchange dumps. Unplug the internet and it keeps working — my little grid-down insurance policy.\n\n**Job 3 — My house.** It's the brain of a fully-local voice assistant I call MaUi: speech-to-text → local LLM → text-to-speech. No cloud, no subscription, nothing leaving the LAN.\n\n**Job 4 — \\~1,800 WoW bots (the newest AND dumbest thing I've built).** I decided to self-host a vanilla WoW server stuffed with \\~1,800 AI playerbots to make the world feel alive, and I recently wired the local LLM in so those bots have personalities — party banter, in-character reactions, the works. It's gloriously unfinished and occasionally ridiculous but watching an AI guildmate roast my gnome frost mage for making too ambitious a trash pull in the style of Gimli from LotR is exactly the kind of unnecessary engineering a homelab is supposed to enable. Right?\n\nSame weights, same 12 GB card. It just wears a different hat depending on who's asking.\n\n**How I actually pick the AI (a.k.a. the part where I benchmark everything)**\n\nHere's the thing that ties the whole lab together: I don't guess, I measure — and that goes for the models too.\n\nInstead of running whatever's trending, I built a frozen, reproducible bake-off: a fixed battery of prompts I put every candidate through, score head-to-head, and use to screen the field (I've run \\~80 models through it) down to a short list I trust. On my hardware. Same instinct as the Grafana dashboards — if I can't measure it, I don't believe it. A few things fell out of it:\n\n**The cheap option that keeps on winning.** The surprise wasn't that a bigger model is better — it's how little I needed. A modest, quantized \\~12B model punches so far above its weight that I have little reason to think about upgrading my GPU to run a 70B or reach for a Frontier AI API service that often. Then a pass of lossless tuning (quantization-aware weights, picking the right inference engine, KV-cache tricks) squeezed even more free speed out of it — same accuracy, meaningfully faster, $0 spent.\n\n**The lineup that won.** The serious interactive jobs — the stock takes and the voice assistant — run on a quantized Gemma4 12B (QAT): fast, well-calibrated, and it fits the card with headroom to spare. Heavier jobs that run overnight get a Gemma 26B. Embeddings are IBM Granite (768-dim) — swapping to it freed \\~2 GB of VRAM over my old embedder and improved retrieval accuracy at the same time, the rare win-win you don't plan for — paired with a tiny MiniLM cross-encoder reranker that runs on the mini-server's CPU so it never steals the GPU. The offline coding library runs Qwen 2.5 Coder 7B. And the WoW bots got their *own* bake-off and their own model — an uncensored fine-tune called **Tiger-Gemma 9B** (with an even lighter one aptly named *Fiendish* as backup), because the polite, well-behaved assistant models flat-out refuse to stay in character. I wanted bots that would get salty and roast me; you don't get that from a model trained to be helpful and harmless. One 12 GB card, a whole roster.\n\n**The speed demons.** The tuning rabbit hole turned up some genuinely fast setups — and the single biggest *free* win was the inference **engine**, not the model. Moving the right models from Ollama to llama.cpp roughly *doubled* throughput on the same card: gpt-oss:20b jumped to **\\~100 tokens/sec** (+102%) and deepseek-coder-v2:16b hit **\\~140 t/s** (+69%), zero quality lost. I don't run those as the daily driver — Gemma's the reliable all-rounder — but it's wild how much speed is just sitting in the engine you pick.\n\n**The benchmark saved me from a mirage.** At one point a hyped speed-up looked like a nearly-4× win in a quick test. I ran it through the full battery instead of the one lucky span, and it collapsed to a modest single-digit-to-actually-negative gains, depending on the task — nowhere near the headline. That's the entire reason the bake-off exists: one impressive run is a rumor; a battery is a result. I almost shipped the mirage. Glad I didn't.\n\n**Sometimes the best result is \"no.\"** I spent real time evaluating a time-series model to forecast prices. The unexpected win? It lost to a dumb random-walk baseline on basically every axis — so I didn't ship it. A lab where you can cheaply prove an idea is bad before it goes live is underrated.\n\n**The time I blue-screened the whole box**\n\nBenchmarking isn't free, and I have the crash logs to prove it. During one bake-off I was rapidly loading and unloading 10–20 GB models back-to-back to score them, and the entire machine hard-crashed — DPC\\_WATCHDOG\\_VIOLATION, full blue screen. Turns out machine-gunning that much VRAM churn at the NVIDIA driver tripped a bug deep in nvlddmkm.sys and took the whole system down with it. The fix was a nuke-from-orbit driver wipe (DDU), the Studio driver instead of the gaming one, and rewriting the benchmark's load pattern so it stopped hammering the card so violently. Bonus gotcha I found along the way: that same driver slowly leaks non-paged pool under sustained churn — \\~12 GB quietly gone after a week of runs, and only a reboot clears it. Homelabbing is 10% building and 90% discovering the specific way your hardware likes to betray you.\n\n**The dumbest fix that worked**\n\nNot every lesson is a crash. For the longest time my 3060 ran hotter than it should have under inference, and I couldn't work out why the chassis fans sat there doing nothing while it baked. Turns out HP's stock fan curve keys the case fans to *CPU* temperature, not GPU load — so during a GPU-pegged inference run (CPU barely awake), the fans figured \"cool CPU, nothing to do here\" and idled while the card cooked. Re-keying the chassis fans to follow GPU temperature instead dropped the 3060 from 84 °C to 75 °C. Nine degrees, zero dollars, one very confused afternoon.\n\n**Current setup**\n\n**Qotom mini-server** — the silent workhorse.\n\n* Atom C3758 · 64 GB ECC · 2× NVMe + 2 TB SATA · fanless · \\~25 W\n* Runs the entire \\~40-container Docker stack — it's all in the attached map.\n\n**HP Omen 40L** — the muscle (and my daily driver).\n\n* i5-12400F · 64 GB · RTX 3060 12 GB\n* Local AI, the WoW server, and my actual desktop.\n\n**Network**\n\n* Protectli FW4B running OPNsense (edge router) · TP-Link managed switch · Eero 6 in bridge mode · three VLANs · CrowdSec · AdGuard Home · Cloudflare Tunnel · Tailscale.\n* Nothing is port-forwarded — the only inbound path is an outbound Cloudflare Tunnel; everything else is Tailscale or LAN.\n\n**Cost**\n\nThe part I'm weirdly proud of is how much of this came from bargains:\n\n* Used firewall: $50\n* Both UPS units: free (just needed batteries)\n* Eero: came from my ISP\n* \"Rack\": literally a $20 Walmart shoe rack\n\nThe homelab infrastructure came in around $860. Including the AI/WoW machine (which is also my daily-driver desktop, so it kind of got drafted), it's roughly $1,900 all-in.\n\nAnd the cloud bill? This is the part I love: the whole thing runs on free tiers — Vercel, GitHub, Cloudflare, Tailscale, Neon, Clerk, PostHog, Resend, Healthchecks, and Backblaze B2 for immutable off-site backups. My total lifetime cloud spend is $5 of API credit I dropped in six months ago and still haven't used up, plus one domain registration. That's it. That's the bill.\n\nAnd the electricity to run all of it? The whole lab averages \\~161 watts — **$18.49/month** by my own Grafana (screenshot attached). And here's the part I didn't plan: even the *honest* number — the power **plus** the A/C that must haul its heat back out of the room — is $24.01/month, and **both are still less than the $29.99 subscription that started this whole thing.** I refused a $25 price hike and built a small datacenter that runs on less than the app it replaced. I'll let you decide whether that's a victory or a future mental health diagnosis.\n\n(Fair-play caveat, because I'm a \"measure it\" guy: my wattage is UPS-derived, not a metered wall plug — directionally right and measured the same way every time, but true-watt smart plugs are on the list.)\n\nThe best return on investment wasn't really even performance. It was removing a screaming Dell server from a room humans occasionally need to sleep in.\n\n**Why I don't (currently) run Proxmox**\n\nMy workloads are almost entirely containers. Everything on the mini-server is Docker Compose, so a hypervisor would mostly add a layer without much","offTopic":true},{"id":"797ae435-56b6-49ac-ad0e-e65d6cbb931c","excerpt":"Local host 3 Mac Studios stacked = private AI fleet for the whole office — A few days ago, I shared our 8x 4090Ds rig setup. It’s a beast, but let’s be real, not every office has the electrical infrastructure, the specialized cooling, or the massive budget to build and maintain a local supercomputer.  \n\nSo we also look","url":"https://www.reddit.com/r/LocalAIServers/comments/1ue97re/local_host_3_mac_studios_stacked_private_ai_fleet/","role":"demand","weight":1.0527111,"occurredAt":"2026-06-24T10:00:09.000Z","sourceKey":"reddit","sourceName":"Reddit","credibility":0.62,"venue":"LocalAIServers","intent":"tool_discovery","painScore":0.38666666,"sentiment":-0.6666667,"confidence":0.75916666,"matchedPatterns":["looking_for"],"statement":"Looking for a \"my auntie can join the fleet\" solution if anyone's solved this","title":"Local host 3 Mac Studios stacked = private AI fleet for the whole office","body":"A few days ago, I shared our 8x 4090Ds rig setup. It’s a beast, but let’s be real, not every office has the electrical infrastructure, the specialized cooling, or the massive budget to build and maintain a local supercomputer.  \n\nSo we also looked the other way: horizontal. 3 used Mac Studios on my desk + every junk laptop we could find in the office. Fully local AI fleet, no cloud, no data leaves the building. Here's the build:\n\n3 Studio M2 Ultra, 192GB / 2TB each. 100+ \"free-range\" office laptops. Qwen on each Studio, a LAN router + ComfyUI for img gen.\n\nDon't see many cross-platform Mac+Win fleet builds so here goes\n\nHope this shares some real value.\n\n100+ \"free-range\" office laptops. The kind that lag when a 3rd Chrome tab opens. No dGPU. Battery lasts 40 mins if you're lucky\n\nQwen 3.6-35B-A3B on each Studio via Ollama. A LAN router + ComfyUI + img gen for the rest\n\nWhy this exists:\n\nOur team needed auto-generated social media content, product posts, images, scheduling, research docs... without:  \n\\- uploading business docs to some random cloud  \n\\- leaking internal convos to whoever's API we're using  \n\\- paying per-seat SaaS tax for smth we can run ourselves\n\nWe ain't a dev shop. Half the team is happy with Windows Update weekly, the other half on Mac. \"Terminal\" is a scary word. They just click \"generate\" and get output.\n\nThe old way was tragic, one machine, one LLM, one API key. Growth team runs a SQL query -> model freezes -> everyone else's agents hang. System looks alive, nothing comes out. Then 3 more people retry the same query. Death by single-queue\n\nThe queue:\n\nConventional answer is scale vertical bigger GPU, more VRAM, one mega-machine. Doesn't fix the single queue. One heavy query still freezes everything. And when that machine dies, everyone stares at \"connection refused.\" Going horizontal instead 3 machines, each independent, each with its own queue means nobody blocks nobody. A router sends each request to the least-loaded engine. Linear throughput scaling, fault tolerance, and you expand by adding 1 more machine, not forklift-upgrading the whole rack.\n\n[Grid](https://www.autonomous.ai/grid)  (the router we use) saves us because it doesn't have one queue it has per machine. Each engine queues internally. Heavy query -> machine A. Caption gen -> machine B. Nobody blocks nobody. 3 machines = 3 queues, each clearing at \\~80 tok/s on Qwen's MoE. A heavy analysis might take 30s on one machine while the other two serve 20 lightweight requests at the same time.\n\nBottleneck went from \"one queue everyone fights over\" to \"how to stack 3 Mac Studios without them falling over.\" Way better problem\n\nThe math:\n\nM2 Ultra = 800 GB/s bandwidth. 192GB unified memory. Qwen 3.6-35B-A3B (MoE, 3B active) at 32-64K context per session. Per Studio handles \\~17 concurrent sessions. 3 Studios = \\~50 concurrent. At 25% concurrency, that's \\~200 employees. 500-token response at peak: \\~12s. Wait time under half a second. Headroom for days.\n\n24GB VRAM hits OOM at \\~2 concurrent 64K sessions. Not a dig, just physics of the hardware.\n\nThe cost:\n\n3 Mac Studios: \\~$17k total  \nPower draw all 3 under load: \\~300-385W  \nNo data leaves the building\n\nVs cloud equivalent comparable throughput but your data stays in-building\n\nScale:\n\nVertical scaling means buying a bigger machine. But you hit a ceiling, no bigger GPU exists, no more VRAM slots. Every upgrade means migrating everything, reconfiguring, downtime. Horizontal instead? Add another Mac to the stack. The router picks it up. 3 Studios today, 4 tomorrow, 6 next quarter. The ceiling isn't the hardware it's how much desk space you got left.\n\nThings you should keep in mind before stacked Mac Studios like this:\n\n1. ⁠LAN or nothing. No LAN = no agents. If the internet bill goes unpaid, or your wifi goes down ur entire fleet disappears. Just a room full of people staring at \"connection refused\"\n2. ⁠Employee takes laptop home at 6PM? They now own an expensive paperweight. Agents live in office LAN. \"It's a privacy feature not a bug.\" Remote Desktop may help if they really need it. Or tell them to touch grass idk\n3. ⁠Not zero-config yet. Each laptop needs an agent gateway configured once (\\~10 mins). My non-devs can't do that. Looking for a \"my auntie can join the fleet\" solution if anyone's solved this","offTopic":true}],"breakdown":[{"sourceKey":"reddit","sourceName":"Reddit","count":6}],"total":6}}