Testing a Local LLM
A developer running a 20-billion-parameter open-source language model on a personal desktop found it fast, accurate, and capable enough to replace a paid Claude Pro subscription for daily news research. This matters because it shows that capable AI is no longer locked behind clou...
Writing for his personal blog at lzon.ca, the author (who goes by the site handle lzon) published a detailed account on April 21, 2026, of his experiment running a local large language model on his home desktop. He cancelled his Claude Pro subscription before the test, citing both cost concerns and broader misgivings about Anthropic and the AI service industry, and set out to answer a simple question: could a small, locally-run model actually replace a cloud AI for real daily use?
Why This Matters
The answer, based on this hands-on test, is a clear yes for a specific and important use case. The author sustained 25 tokens per second on consumer hardware that retails for well under $2,000 combined, which puts genuinely useful local AI within reach of any serious hobbyist or independent developer. Cloud AI subscription costs, which run $20 per month for Claude Pro and $20 per month for ChatGPT Plus, add up fast, and the case for paying them weakens every time a test like this one succeeds. The moment local models can reliably handle citation-backed research and web search integration, the subscription model for casual AI users starts looking like a bad deal.
Daily briefing from 50+ sources. Free, 5-minute read.
The Full Story
The author built his test around a practical frustration most internet users will recognize: search has gotten worse. He describes the modern web as a "trash pile" that requires real effort to navigate, and says even his above-average search skills have not been enough to consistently find accurate, trustworthy information quickly. He had tried the AI-powered search summaries baked into Google and other engines but found them inconsistent and untrustworthy because they do not reliably cite their sources.
His solution was to build a local AI setup that would do what the free tools would not: cite every claim. He installed LM Studio with the ROCm llama.cpp runtime, which enables the software to run on AMD graphics cards rather than the NVIDIA hardware that dominates most AI guides. He then loaded the model openai/gpt-oss-20b, a 20-billion-parameter model released in summer 2025, and paired it with the Exa Search MCP Plugin to give it live web search capability.
The hardware running all of this is a Ryzen 9 5900XT processor with 16 cores, a Radeon RX 6800 graphics card with 16GB of video RAM, and 64GB of DDR4 system RAM. That is a capable but not exotic desktop build. The RX 6800 is a mid-to-high-end GPU from 2020, not a current-generation AI-optimized card, which makes the performance results more meaningful.
Performance held consistently above 25 tokens per second throughout the test, a threshold the author describes as subjectively "fast enough" with no frustrating waits. More importantly, the model integrated web search naturally, knowing when to search and what to search for without being explicitly prompted on each query. The author says the research experience was comparable to using Claude directly.
The system prompt he designed for the experiment, which he named "Ron Burgundy" after the fictional anchorman character, was engineered to strip out the AI's tendency toward hedging, opinion, and conversational filler. The prompt instructs the model to behave as a neutral news delivery assistant, write in the style of an objective news article, and never offer opinions or advice under any circumstances. The result was a local AI that surfaced cited, factual information in a clean, readable format without the performative personality that cloud models often inject into responses.
Key Details
- The test was published April 21, 2026, on the author's personal blog at lzon.ca.
- The model used was openai/gpt-oss-20b, released in summer 2025, with 20 billion parameters.
- Hardware includes a Ryzen 9 5900XT (16 cores), Radeon RX 6800 (16GB VRAM), and 64GB DDR4 RAM.
- Output speed never dropped below 25 tokens per second during testing.
- The software stack was LM Studio with ROCm llama.cpp runtime and the Exa Search MCP Plugin.
- The author cancelled a Claude Pro subscription (priced at $20 per month) before conducting the experiment.
What's Next
As open-source models continue improving through 2026, the gap between local model performance and cloud API quality will narrow further, making tests like this one look conservative rather than impressive in hindsight. The Exa Search MCP Plugin integration is worth watching specifically because combining local inference with real-time search citation is the piece that makes these setups practically useful rather than just technically interesting. Developers building research or news monitoring AI tools should treat this as a working proof of concept they can replicate today.
How This Compares
This experiment fits into a broader wave of developers abandoning or supplementing cloud AI subscriptions with local alternatives, a trend that accelerated significantly after Meta released Llama 3 in April 2024 and again when DeepSeek-R1 became widely available for local deployment in early 2025. What makes this particular test interesting is the AMD hardware angle. Most guides for local LLM setup assume NVIDIA GPUs and CUDA, which leaves a significant portion of the enthusiast PC market underserved. Running ROCm-based inference on an RX 6800 and hitting consistent 25 tokens per second is a practical data point that AMD users have needed.
Compare this to the Ollama-based setups that have become common on platforms like GitHub and Reddit, where users typically test 7-billion-parameter models on 8GB GPUs. Those setups are more accessible but produce noticeably lower quality output. The author's choice of a 20-billion-parameter model at 16GB VRAM hits a meaningful quality threshold that smaller models miss, particularly for tasks requiring coherent multi-step reasoning like web research synthesis.
Northwestern University research published in 2025 found that LLMs applied to journalism workflows achieved F1 scores of 0.94 for lead extraction and accuracy up to 92 percent in coarse newsworthiness assessment, but struggled with nuanced beat-specific editorial judgment. The "Ron Burgundy" system prompt approach the author used sidesteps that limitation cleverly by making the model a neutral information deliverer rather than an editorial decision-maker. It is a practical workaround that aligns with what the research actually shows these models can and cannot do reliably.
FAQ
Q: Can I run a useful AI model on a regular gaming PC? A: Yes, if your gaming PC has a GPU with at least 16GB of video RAM. The author ran a 20-billion-parameter model on a Radeon RX 6800 with 16GB VRAM and got response speeds above 25 tokens per second, which is fast enough for comfortable daily use in research and writing tasks.
Q: What is LM Studio and how does it work? A: LM Studio is a desktop application that lets you download and run open-source AI models locally without any cloud connection. It handles the technical setup, including runtime selection and model management, so you do not need to configure machine learning infrastructure manually. It supports both NVIDIA and AMD graphics cards.
Q: Is a local AI better than Claude or ChatGPT for privacy? A: For privacy, local AI wins outright because your queries and data never leave your machine. For raw capability on complex tasks, cloud models from Anthropic and OpenAI still hold an edge in most benchmarks. The gap is closing, though, and for structured tasks with clear prompts, a well-configured local model can match cloud quality closely enough to matter.
The author's experiment is a useful, honest snapshot of where local AI capability stands in mid-2026, and the answer is further along than most casual observers assume. Anyone paying a monthly AI subscription primarily for research and information synthesis should read this as a direct challenge to justify that cost. Subscribe to the AI Agents Daily weekly newsletter for daily updates on AI agents, tools, and automation.
Get stories like this daily
Free briefing. Curated from 50+ sources. 5-minute read every morning.