Nyquest – Open-source LLM token compression proxy in Rust (15–75% savings)
Nyquest is a new open-source token compression proxy built in Rust that sits between your app and any LLM API, cutting token usage by 15 to 75 percent depending on your workload. For teams paying real money on API calls, that kind of reduction can translate directly into lower mo...
Mike Simpson, the developer behind the Nyquest-ai GitHub organization, published the first public release of Nyquest on April 16, 2026, under a dual MIT and Apache-2.0 license. The project, version 3.1.1 at launch, is a full-stack Rust implementation designed to intercept LLM API requests, compress the token payload, and forward a leaner prompt to the model. Simpson tagged the release "Semantic Compression Proxy for LLMs," and the technical specs baked into that single commit message tell a surprisingly complete story.
Why This Matters
Token costs are the hidden tax on every AI product in production, and most teams are still ignoring it. OpenAI charges roughly $0.03 per 1,000 input tokens on GPT-4 standard pricing, which means a company pushing 10 million tokens monthly is spending $300 just on input alone. Nyquest's conservative 15 percent compression floor cuts that to $255, and at the 75 percent ceiling you are looking at $75 for the same workload. At enterprise scale, that gap funds engineering salaries.
Daily briefing from 50+ sources. Free, 5-minute read.
The Full Story
Nyquest works as a proxy, meaning it sits in front of your LLM API endpoint and intercepts every request before it reaches the model. Your application code does not need to change. You point your API calls at Nyquest instead of directly at OpenAI or whatever model you are using, and the proxy handles the compression transparently before forwarding the slimmer request.
The architecture is built on Axum 0.8, a popular async Rust web framework, and the compression engine runs through a 6-stage optimization pipeline. That pipeline starts with rule-based compression powered by more than 350 compiled regex rules organized across 18 categories. Compiled regex at that scale is fast, and the benchmark numbers back that up: Simpson reports a rule-stage latency of under 2 milliseconds. That is fast enough that the proxy adds negligible overhead to your request cycle.
After the rule-based stage, Nyquest hands the prompt to a local semantic compression layer powered by Qwen 2.5 1.5B running through Ollama. Using a 1.5 billion parameter model for the semantic stage is a smart tradeoff. It is small enough to run locally on commodity hardware, fast enough not to become a bottleneck, and capable enough to understand what parts of a prompt carry meaning and what parts are redundant. The semantic stage is where the higher end of that 75 percent savings figure comes from, because raw regex can only catch structural waste while a language model can identify conceptual repetition.
The system also includes a feature called OpenClaw agentic mode and prefix cache reordering. Prefix cache reordering is particularly clever. Modern LLMs can cache the KV (key-value) states of repeated prompt prefixes, which reduces both latency and cost on repeated calls. Nyquest reorders prompt segments to maximize prefix cache hits at the model level, squeezing out additional savings on top of the compression itself.
Throughput benchmarks show 1,408 requests per second under concurrent load. For most teams this is more than sufficient headroom, and it means the proxy itself will not become the bottleneck in high-volume deployments. The entire stack, including frontend tooling implied by the "fullstack" repository name, is written in Rust, which gives it memory safety guarantees without a garbage collector slowing things down at runtime.
Key Details
- Nyquest version 3.1.1 was published on April 16, 2026, by developer Mike Simpson under the Nyquest-ai GitHub organization.
- The project carries dual licensing under MIT and Apache-2.0, making it usable in both commercial and open-source projects without restriction.
- The 6-stage pipeline includes more than 350 compiled regex rules spanning 18 distinct compression categories.
- The semantic compression stage runs on Qwen 2.5 1.5B via Ollama, a locally hosted 1.5 billion parameter model.
- Rule-stage latency benchmarks at under 2 milliseconds per request.
- Concurrent throughput reaches 1,408 requests per second.
- Token savings range from 15 percent on structured, minimal prompts to 75 percent on verbose, redundant inputs.
- The repository had 1 star and 0 forks at the time of publication.
What's Next
The immediate test for Nyquest is whether the developer community picks it up and stress-tests it against real production workloads, because benchmark numbers on a single-commit repository tell only part of the story. Simpson will need to publish compression benchmarks across specific prompt types, such as RAG pipelines, agentic tool-call chains, and long-context summarization tasks, to help potential users predict savings for their specific use case. Watch the GitHub repository for additional commits and a growing issue tracker as early adopters surface edge cases.
How This Compares
Anthropic published research on prompt caching in 2024 that reduces the cost of repeated prompt prefixes at the API level, and Google's Vertex AI has begun baking similar caching features into its managed offerings. Both of those approaches are vendor-side optimizations, meaning you benefit only when using that specific provider and only under their pricing terms. Nyquest is provider-agnostic, which is a real structural advantage. You can point it at OpenAI, Anthropic, a local Ollama instance, or anything that speaks the standard API format.
The semantic caching tools that have gained traction in the developer community, such as GPTCache and similar AI tools, solve a different problem. They cache entire responses and return stored answers when a semantically similar question comes in later. Nyquest is not doing that. It is compressing the prompt itself before the model ever sees it, which means it works on novel requests where a cache would miss entirely. These two approaches are complementary rather than competing.
What makes Nyquest interesting compared to Python-based prompt optimization libraries like LLMLingua from Microsoft Research is the performance floor. LLMLingua achieves strong compression ratios but adds meaningful latency because it runs in Python with a larger model in the compression loop. Nyquest's sub-2-millisecond rule stage and lightweight 1.5 billion parameter semantic model are a deliberate architectural choice to make the proxy viable in latency-sensitive production environments, not just batch processing jobs. Whether that tradeoff costs compression quality compared to heavier approaches is the central question worth watching as the project matures.
FAQ
Q: What is a token compression proxy for LLMs? A: It is a piece of software that sits between your application and an LLM API. Before sending your prompt to the model, it removes redundant words, restructures sentences, and trims unnecessary content. The model receives a shorter prompt, which costs fewer tokens, and you pay less for the same result.
Q: Does Nyquest work with OpenAI and other commercial APIs? A: Yes. Because Nyquest operates as a proxy rather than a model replacement, it is designed to work with any LLM that accepts standard API requests. You redirect your existing API calls through Nyquest, and it handles compression before forwarding the request to OpenAI, Anthropic, or any other provider.
Q: How do I know how much money Nyquest will actually save me? A: The 15 to 75 percent range depends heavily on how verbose your prompts are. Highly repetitive prompts, long system instructions with reused content, and RAG pipelines that inject large documents tend to compress more aggressively. Minimal, tightly written prompts will land closer to the 15 percent floor. The best approach is to run your specific prompt types through the proxy and measure directly. Check the AI Agents Daily guides for practical walkthroughs on evaluating token optimization tools.
Nyquest is early, with a single commit and minimal community traction as of April 16, 2026, but the technical architecture is coherent and the performance numbers are credible enough to warrant serious attention from teams running meaningful LLM workloads. The open-source release under permissive licensing means there is no barrier to spinning up a test instance and measuring real savings against your own traffic. Subscribe to the AI Agents Daily weekly newsletter for daily updates on AI agents, tools, and automation.
Get stories like this daily
Free briefing. Curated from 50+ sources. 5-minute read every morning.
