Reinforcement learning
EditScore uses reward models for RL in image editing
EditScore is a family of 7B–72B reward models for evaluating and guiding complex image edits, built on EditReward-Bench. The post says it enables RL training and outperforms GPT-5 on the benchmark.
Domain-specific reward models may help engineers train systems for complex image-editing tasks.
KL-Regularized RL and Mode Collapse
The paper studies reverse- and forward-KL regularization in reinforcement learning. It shows mathematically and empirically that the usual mode-seeking versus mass-covering intuition does not necessarily transfer to RL.
Engineers using KL regularization in language-model RL may need to reassess assumptions about diversity.
Meta-learning discovers reinforcement learning update rules
The paper describes a method that uses meta-learning across agents and environments to discover RL update rules. Its DiscoRL rule reports state-of-the-art benchmark performance, including on Atari and ProcGen.
Engineers can see how learned update rules may transfer across environments without being tuned for each one.
BAPO stabilizes off-policy reinforcement learning for LLMs
BAPO is a method for stabilizing off-policy reinforcement learning for large language models, using balanced policy optimization with adaptive clipping. The work addresses settings including partial rollout and experience reuse.
Engineers working on LLM reinforcement learning can review the method and its code for off-policy training settings.
- 1 year ago @omarsar0
Ring-1T combines trillion-parameter MoE with three-stage RL training
The post describes Ring-1T, a 1T-parameter MoE reasoning model with about 50B active parameters per token. Its training uses long-CoT SFT, verifiable-reward reasoning RL, and general RLHF, alongside IcePop, C3PO++, and ASystem.
The training phases and systems components offer concrete approaches to scaling RL for large reasoning models.
A Study of RL Design Choices for Agentic Reasoning
The paper systematically investigates reinforcement learning for agentic reasoning in LLMs across data, algorithms, and reasoning modes. The preview highlights a comparison of stitched synthetic trajectories with real end-to-end tool-use trajectories.
It examines design choices that can inform how engineers train LLM agents to reason and use tools.
- 1 year ago @HuggingPapers
LaSeR uses last-token self-rewarding for LLM reinforcement learning
Tencent introduces LaSeR, an algorithm that aligns last-token scores with true rewards for LLM reasoning and self-rewarding. The post says it uses one extra token inference.
The approach may be relevant to engineers exploring lower-cost reward optimization for language models.
- Reinforcement learningProject
SkyRL tx v0.0.2 supports training multi-LoRA models
SkyRL tx is an open backend for the Thinky Machines Tinker API. Version v0.0.2 supports training multi-LoRA models.
Engineers can explore an open backend for training multi-LoRA models through the Tinker API.
arxiv.org
