Easy Samples Are All You Need: Self-Evolving LLMs via Data-Efficient Reinforcement Learning
A team of six researchers has published a new method called EasyRL that trains large language models using just 10% of the labeled data that standard approaches require. The system beats current state-of-the-art baselines on math and science benchmarks by starting with simple, we...
Zhiyin Yu, Bo Zhang, Qibin Hou, Zhonghai Wu, Xiao Luo, and Lei Bai submitted their paper to arXiv on April 19, 2026, presenting EasyRL as a self-evolving training framework for large language models. The work has been accepted to the Findings of ACL 2026, which signals serious peer-reviewed validation for what looks like a genuinely clever rethinking of how reinforcement learning can work when you have very little labeled data to start with.
Why This Matters
The AI industry has been slowly choking on its own data requirements. Human annotation is expensive, slow, and does not scale, and the unsupervised workarounds that labs have tried keep falling apart through reward hacking or model collapse. EasyRL's ability to match or beat baselines using only 10% of labeled data is not a minor optimization, it is the kind of efficiency gain that changes what is economically viable for smaller labs and companies building on top of open models. If this holds up at scale, it significantly lowers the barrier to fine-tuning capable reasoning models for specialized domains.
Daily briefing from 50+ sources. Free, 5-minute read.
The Full Story
The core problem EasyRL is solving is one that anyone who has tried to fine-tune a language model for reasoning tasks will recognize immediately. Standard supervised approaches require large amounts of human-labeled examples, which costs money and time. The unsupervised alternatives, which use voting across model outputs or entropy-based reward signals, sound appealing in theory but regularly produce models that game their reward functions or collapse into repetitive, low-quality behavior. Neither path leads to a reliable, scalable solution.
What the researchers did was look at this problem through the lens of cognitive learning theory, specifically the human pattern of learning easy concepts first and gradually working up to harder material. They built EasyRL around this idea, structuring training as a three-stage process that mirrors how a competent student approaches a difficult curriculum.
The first stage is a warm-up phase. The model is trained using supervised reinforcement learning on a small set of easy, labeled examples. This gives the system a solid foundation, what the researchers call a "warm-up model," without requiring the massive annotation budget that traditional approaches demand. The few-shot labeled dataset is the 10% figure the paper highlights, and it is specifically drawn from simpler, high-confidence examples rather than a random slice of the full dataset.
The second stage is where the real engineering insight shows up. The team developed a divide-and-conquer pseudo-labeling strategy for handling the much larger pool of unlabeled, difficult data. They split this data into categories based on the model's uncertainty about each example. For low-uncertainty cases, they use consistency-based selection, meaning the model labels these with high confidence and those labels are trusted. For medium-uncertainty cases, they apply a reflection-based resolution process, where the model essentially reasons through its own uncertainty to produce a usable label. The high-uncertainty cases are set aside entirely, at least in early training rounds.
The third stage is difficulty-progressive self-training. The model iterates through the pseudo-labeled data in order of increasing difficulty, continuously updating via reinforcement learning. This progressive structure is what prevents the reward hacking and collapse issues that have plagued earlier unsupervised approaches. Because the model builds on reliable signals before confronting ambiguous ones, it does not get the chance to exploit weak reward mechanisms early in training.
The team tested EasyRL on mathematical and scientific benchmarks, the standard proving grounds for reasoning capability in language models, and found that it consistently outperformed state-of-the-art baselines despite being trained on a fraction of the labeled data.
Key Details
- Authors: Zhiyin Yu, Bo Zhang, Qibin Hou, Zhonghai Wu, Xiao Luo, and Lei Bai, submitted April 19, 2026.
- Paper accepted to Findings of ACL 2026, a top-tier venue in natural language processing research.
- EasyRL uses only 10% of easy labeled data compared to standard supervised approaches.
- The framework has three distinct training stages: warm-up supervised RL, divide-and-conquer pseudo-labeling, and difficulty-progressive self-training.
- Pseudo-labeling splits unlabeled data into at least 2 uncertainty tiers, with separate handling strategies for low-uncertainty and medium-uncertainty cases.
- Benchmarks cover mathematical and scientific reasoning tasks, the two domains most commonly used to evaluate LLM reasoning quality.
What's Next
ACL 2026 acceptance means the research community will get a formal presentation of this work, which typically accelerates reproduction attempts and follow-up studies. The most important next test is whether EasyRL's efficiency gains hold when applied to larger base models beyond the sizes tested in the paper. Researchers at other institutions will almost certainly run the framework against newer open-weight models throughout the remainder of 2026, and those results will tell the community whether the 10% data efficiency claim generalizes or is specific to the experimental setup.
How This Compares
EasyRL enters a space where several other teams are working on nearly identical problems from different angles. Researchers at Tsinghua University and Shanghai AI Lab have been developing test-time reinforcement learning, or TTRL, which distributes the learning process across deployment rather than concentrating it during training. TTRL is clever, but it depends on inference-time compute, which adds a different kind of cost. EasyRL's approach is cleaner because the self-evolution happens during post-training, not at runtime, which makes deployment simpler for production systems.
The R-Zero framework, presented at the MATH-AI workshop in October 2025 and accepted to ICLR 2026, pushes even further by having models generate their own training tasks from scratch. That is an ambitious goal and the ICLR acceptance confirms it works at some level, but fully autonomous task generation introduces its own instability risks. EasyRL's decision to anchor training in even a small set of easy labeled examples is a more conservative bet, and on benchmark performance that conservatism appears to pay off.
There is also work from Hangfan Zhang and colleagues, submitted in October 2025, on data-efficient self-evolving LLMs via intrinsic feedback. That paper focuses on internal model signals as a substitute for annotation, which is conceptually similar to EasyRL's pseudo-labeling but lacks the structured difficulty progression that is the heart of the new framework. The GitHub repository tracking reinforcement learning for large reasoning models has accumulated 2,400 stars, which suggests the broader community is actively following all of these threads. EasyRL's specific contribution, the cognitive learning curve as a structural principle, is distinct enough from its contemporaries that it will likely be cited as a reference point for how to think about data ordering and difficulty scheduling in self-training pipelines.
FAQ
Q: What is EasyRL and how does it train language models? A: EasyRL is a reinforcement learning framework that starts by training a language model on a small set of easy, labeled examples, then uses the partially trained model to generate labels for harder unlabeled data. It processes data in order of increasing difficulty, which prevents common training failures like reward hacking and model collapse. The full process requires only 10% of the labeled data that standard approaches demand.
Q: What is reward hacking in AI training? A: Reward hacking happens when a model learns to score well on its training metric without actually improving at the task the metric is supposed to measure. For example, a model trained to generate long answers might produce verbose nonsense because length correlates with its reward signal. EasyRL reduces this risk by anchoring early training on high-confidence, easy examples before the model encounters ambiguous cases where reward signals are weaker.
Q: Does EasyRL work on real-world tasks or just academic benchmarks? A: The paper tests EasyRL on mathematical and scientific reasoning benchmarks, which are the standard academic measures of LLM reasoning quality. These benchmarks are widely considered proxies for general reasoning ability, but the paper does not test the framework on production or domain-specific tasks. Follow-up work from other researchers using the ACL 2026 published version will likely clarify how well the method transfers to applied settings.
The convergence of EasyRL, TTRL, R-Zero, and similar frameworks in the same 18-month window is not a coincidence. The field has clearly identified the annotation bottleneck as the central obstacle to scalable LLM improvement, and multiple teams are now offering credible answers. EasyRL's acceptance to ACL 2026 puts it in the conversation as one of the more practically grounded solutions. Subscribe to the AI Agents Daily weekly newsletter for daily updates on AI agents, tools, and automation.
Get stories like this daily
Free briefing. Curated from 50+ sources. 5-minute read every morning.
