Reinforcement learning
QwenGyre: elastic RL framework for xLong-horizon LLM agent training
Alibaba Qwen introduces QwenGyre, an elastic RL framework that reallocates GPUs between rollout and training and merges branching agent executions into prefix-shared trajectory trees. The author reports up to 1.85× speedup over Colocate and about 44% lower end-to-end training time vs Async.
Shows how to keep GPUs busy and dedupe samples when RL rollouts for long-horizon agents span hours and many tool calls.
- 1 month ago @TheVixhal
An RL setup for calibrated decision probabilities
The post describes training a decision model that encodes input once, scores each option branch, and returns probabilities. Rewards combine the probability assigned to the observed outcome with a measure of confidence calibration, using proper scoring rules.
The setup may interest engineers building decision models that need calibrated probability outputs rather than generated text.
Training language models across multiple agent harnesses
A Hugging Face guide describes using a proxy to train models with RL across four agent harnesses while recording sampled token IDs and logprobs. The post reports improved task scores from multi-harness training and compares it with imitation learning.
Engineers can learn an approach for training models across different agent harnesses without modifying the harnesses.
- 1 month ago @cwolferesearch
Multi-Teacher On-Policy Distillation for Language Models
MOPD trains a student on its own completions, using teacher token-level log probabilities and a reverse-KL objective. One setup, consolidation, transfers capabilities from independently trained domain specialists into a student.
It describes a way to combine specialist models’ capabilities without training them together in multi-domain RL.
- 1 month ago @burkov
ProRL Agent decouples rollouts from LLM policy optimization
ProRL Agent is an open-source rollout infrastructure for multi-turn LLM agents. Its rollout-as-a-service design separates trajectory generation, environment orchestration, and evaluation from policy optimization.
Separating environment simulation from model optimization can ease framework migration and scaling of RL training systems.
A study of scalable data-aware positional encodings
The paper examines positional encodings for Transformers, noting RoPE’s bias toward nearby tokens and that alternatives have been evaluated under different settings. It aims to bring structure to this fragmented landscape.
Engineers working on language models can use this study to compare positional encoding approaches.
MiMo RL Environment Explorer
A Hugging Face Space for exploring MiMo-V2.6 RL environments and running rollouts.
Engineers can inspect RL environments and run rollouts in an interactive explorer.
- Reinforcement learningProject
FrontierSmith releases 200 open-ended RL problems
FrontierSmith releases 200 problems for studying RL on challenging, open-ended tasks. Its GitHub page describes a system that uses AI to synthesize open-ended coding problems at scale.
The problems offer engineers environments for studying RL on long-horizon, open-ended tasks.
On-Policy Self-Distillation for LLM Reasoning
The paper presents Self-Distilled Reasoner, an approach to on-policy self-distillation for large language models. The preview describes on-policy distillation as training on student-sampled trajectories with dense token-level teacher supervision.
It addresses distribution mismatch between training and inference in off-policy distillation, a concern for LLM reasoning systems.
LENS jointly trains reasoning and image segmentation with RL
LENS is a reinforcement-learning framework for text-prompted image segmentation that jointly optimizes reasoning and segmentation. The paper says supervised fine-tuning often omits explicit chain-of-thought at test time, limiting generalization to unseen prompts and domains.
Engineers working on vision systems can assess an approach that trains reasoning and segmentation together.
- Reinforcement learningArticle
Unsloth guide to reinforcement learning with GRPO
Unsloth documentation presents a beginner-to-advanced reinforcement learning guide, including how to train a DeepSeek-R1 reasoning model with GRPO.
Engineers can use it as a guide to applying GRPO to reasoning-model training with Unsloth.
- Reinforcement learningProject
Xiaomi MiMo releases RL environments and training code
A post points to Xiaomi MiMo’s GitHub repository for RL environments and training code, plus a MiMo-V2.6 RL dataset on Hugging Face.
The repository and dataset may help engineers explore or build RL workflows for language models.
- Reinforcement learningDataset
SmolDataEnvs: 5,000 verifiable RL tasks
SmolDataEnvs is an open-source collection of 5,000 verifiable reinforcement-learning environment tasks for hill-climbing small models in code and data science. The release includes environments, evaluations, and training.
Engineers can use the tasks and accompanying evaluations to train and assess small models.
- 1 month ago @teortaxesTex
DeepSeek Elastic Compute reports sandbox capacity at production scale
A post quotes a DeepSeek Elastic Compute paper: one production-scale DSec unit spans about 160 nodes and supports over 380,000 concurrent sandboxes, with more than 5,000 creations per second.
The reported figures offer a concrete reference point for large-scale sandbox infrastructure.
CodeMidas builds coding RL environments from source code
CodeMidas presents an agentic pipeline that turns implemented functionality in open-source codebases into executable RL environments. It targets diverse tasks with reliable verifiers.
Engineers building coding agents can use source code as a scalable source of RL tasks.
- 1 month ago @HuggingPapers
CodeMidas Generates Code-Based RL Training Tasks
Xiaomi’s CodeMidas uses agents to extract executable tasks from open-source codebases rather than relying on issues or commits. It generated 5,545 tasks across 23 languages.
The approach offers engineers a way to build RL training environments directly from source code.
- 1 month ago @nrehiew_
MiMo Pro improves its DeepSWE score after RL runs
The post says MiMo Pro's DeepSWE score rose from 58.41 to 72.57 after RL runs. It compares the result with a score of 74 attributed to Astra, Gemini 3.8 Flash, and Opus 5.
The reported scores offer a benchmark comparison for evaluating RL runs on DeepSWE.
Self-Rewarding Language Models use LLM-as-a-Judge feedback
The paper studies using a language model itself, prompted as an LLM-as-a-Judge, to provide feedback during training. It contrasts this approach with human-preference reward models that remain frozen during training.
Engineers can assess an approach to generating training feedback without relying solely on separate, frozen reward models.
- Reinforcement learningArticle
Never Give Up shifts RL compute toward harder LLM problems
The paper reports that RL improves LLM performance more on problems the model already solves easily than on harder problems. Its adaptive “Never Give Up” method samples until success to direct compute toward harder problems.
Engineers can assess an adaptive sampling strategy for allocating RL training compute across problems of different difficulty.
- Reinforcement learningProject
RULER uses LLM-ranked trajectories for GRPO rewards
The post describes RULER in OpenPipe ART: engineers specify evaluation criteria in plain English, an LLM ranks agent trajectories, and ART turns the comparisons into rewards for GRPO training. It reports using the setup with a Qwen3 1.4B agent playing 2048.
It offers an approach to evaluating multi-step agent behavior without hand-coding a set of scoring rules.
arxiv.org
unsloth.ai
