Every time a new model comes out, the old one is obsolete of course
AI models are becoming obsolete faster than ever, with developers reporting noticeable performance drops within weeks of a new release. This pattern of rapid model turnover is creating real problems for businesses building production applications on top of these systems, and the ...
A thread on Reddit's LocalLLaMA community, submitted by user /u/FullChampionship7564, put a blunt label on something developers have been quietly complaining about for years: every new model release makes the previous one feel broken almost overnight. The post touched a nerve. It reflects a growing frustration documented across developer communities on Reddit and Hacker News, and covered directly by Chris Stokel-Walker writing for Fast Company in a piece titled "New AI models are losing their edge almost immediately."
Why This Matters
This is not just a complaint from power users who miss their favorite chatbot. Businesses are building production systems on top of models that can change without notice, and the cycle of obsolescence is accelerating. OpenAI cut pricing for its o3 model by 80 percent within a competitive window that saw multiple new models enter the market at roughly the same time, which tells you exactly how fast the floor is moving. A 2024 Nature study by Johnson and Obradovich found that the speed of AI development has directly accelerated the speed of model discontinuation, meaning researchers cannot even go back and verify old results. That is an accountability crisis, not just an inconvenience.
Daily briefing from 50+ sources. Free, 5-minute read.
The Full Story
The pattern looks like this: a new model launches, developers test it, performance feels sharp and capable, and then somewhere between a few weeks and a few months later, the same prompts start returning noticeably worse results. A Hacker News user identified as lispisok described it precisely about 10 months ago: "I swear every time a new model is released it's great at first but then performance gets worse over time." The comment resonated because it matched what hundreds of other developers were experiencing independently.
The causes are disputed, and that ambiguity is part of the problem. One leading theory is that companies implement safety fine-tuning after a model's public release to address harmful outputs flagged by users or internal red teams. The catch is that these adjustments do not live in a vacuum. When you constrain a model's behavior in one area, you often reduce its capability in adjacent areas, and users notice even if the company never announces a change. The model did not get safer and stay equally capable. It got safer and got worse.
A second explanation involves quantization, the process of reducing the numerical precision of a model's parameters to lower computational costs and improve inference speed. Quantization is a legitimate engineering tool, but it trades precision for efficiency, and that trade-off shows up in output quality. If a company quietly applies more aggressive quantization to a model after launch to reduce server costs, users see degraded performance without any announcement or changelog entry.
The model retirement problem runs even deeper than performance degradation. Research published on Vale.Rocks argues that proprietary AI models available only through hosted APIs are disappearing without any meaningful preservation mechanism. Unlike open-source models that can be downloaded, archived, and run indefinitely, hosted models simply vanish when companies retire them. OpenAI uses the word "deprecated" for models no longer available to new users. Anthropic draws a distinction between "deprecated" models that existing users can still access and "retired" models that no one can access at all. The end result is the same: the historical record of how these systems behaved gets erased permanently.
This creates a direct problem for AI safety research. If you published a paper in 2023 comparing the reasoning capabilities of two models, and one of those models no longer exists, your work cannot be replicated. The Johnson and Obradovich study in Nature from 2024 specifically called for new methods to deprecate AI systems in ways that preserve historical documentation and enable future research. So far, no major AI lab has responded with a concrete preservation framework.
For developers building commercial products, the uncertainty is operational. An application tuned around a model's launch-day capabilities may start producing worse results months later with no clear signal from the provider that anything changed. Debugging becomes nearly impossible when you cannot confirm whether the model itself was modified or whether something in your own stack drifted.
Key Details
- User /u/FullChampionship7564 posted the observation to Reddit's LocalLLaMA community, sparking community-wide discussion.
- Hacker News user lispisok documented the same performance degradation pattern approximately 10 months ago.
- OpenAI reduced pricing on its o3 model by 80 percent amid a wave of competitive model releases.
- A 2024 Nature study by Johnson and Obradovich formally identified the model deprecation cycle as a research preservation crisis.
- Chris Stokel-Walker covered the systemic nature of this issue for Fast Company in the article "New AI models are losing their edge almost immediately."
- Anthropic separates retired models, which have zero access, from deprecated models, which remain available only to existing users.
What's Next
Expect the deprecation cycle to compress further as competitive pressure from Anthropic, Google, and newer entrants like Zhipu AI forces faster release timelines across the board. Developers building on hosted APIs should treat any performance benchmark they run at model launch as a snapshot, not a guarantee, and build evaluation pipelines that test outputs on a regular schedule. The Johnson and Obradovich framework published in Nature in 2024 is the clearest roadmap the industry has for what responsible model retirement should look like, and it deserves direct adoption by the major labs.
How This Compares
Compare this to what happened with Google's Gemini rollout, where users reported significant quality variation between benchmark demonstrations and real-world API behavior within weeks of launch. The difference is that Google at least communicated some of the changes through versioned model identifiers. OpenAI has historically been less transparent, which is part of why the community speculation about silent quantization or post-launch fine-tuning persists. When users cannot get a straight answer, they fill the gap with theories.
Meta's release of multimodal tools and Anthropic's Project Glasswing both demonstrate that the release velocity is not slowing down. Each new product launch accelerates the obsolescence clock on whatever came before it. This is genuinely different from how software versioning has worked historically. When Microsoft shipped a new version of Office, the old version did not stop working and did not get silently modified. AI models deployed as hosted services operate under entirely different rules, and the industry has not built the transparency infrastructure to match.
The open-source side of this debate, which is exactly where LocalLLaMA lives, offers a partial answer. Models like Meta's Llama family can be downloaded, pinned to a specific version, and run locally with no risk of silent modification. That option is increasingly viable for AI tools at the application layer, and it is one reason the LocalLLaMA community has grown into a serious alternative to the hosted API ecosystem. The tradeoff is infrastructure cost and setup complexity, and for most companies, that is still a meaningful barrier.
FAQ
Q: Why do AI models seem to get worse after launch? A: Companies often update models after release to reduce harmful outputs or cut server costs through a process called quantization, which lowers the precision of the model's calculations. These changes are rarely announced publicly, so users experience declining performance without any explanation. The model is technically the same version but no longer behaves the same way it did on day one.
Q: What happens to old AI models when new ones come out? A: Most hosted models get deprecated or retired, meaning access is cut off either for new users or for everyone. Unlike open-source models that can be saved and run locally forever, hosted models disappear entirely once a company stops supporting them. Researchers studying AI development lose access to those systems permanently, which makes it impossible to replicate studies or verify historical benchmarks.
Q: How can developers protect their apps from AI model changes? A: The most practical approach is to build automated evaluation pipelines that run standardized test prompts against your chosen model on a regular schedule, so you catch performance changes quickly. You can also pin to specific model versions where providers offer them, or explore self-hosted open-source alternatives that cannot be silently modified. Check AI Agents Daily guides for practical walkthroughs on building model evaluation workflows.
The AI industry is cycling through models faster than it is building the infrastructure to manage that cycle responsibly, and developers are absorbing the cost of that gap in the form of unpredictable production systems. The conversation on LocalLLaMA is blunt, but the research from Johnson and Obradovich and the reporting from Stokel-Walker at Fast Company confirm it is also accurate. Subscribe to the AI Agents Daily weekly newsletter for daily updates on AI agents, tools, and automation.
Get stories like this daily
Free briefing. Curated from 50+ sources. 5-minute read every morning.