AI agents
Continual Harness for ARC-AGI-3 Agents
A post about “Continual Harness: An Efficient Self-Improving Agent on ARC-AGI-3” says the benchmark’s test-time learning demands push agents to build a world model of rules and mechanics that updates with new evidence.
It highlights the role of updating an agent’s world model during test-time learning.
- AI agentsPaper
Dockerless verifies coding-agent patches without execution
Dockerless is an environment-free verifier that evaluates generated code patches without executing them. The paper describes program verifiers as tools for selecting SFT trajectories and providing RL rewards.
It explores an alternative to setting up per-repository environments for execution-based verification.
- AI agentsPaper
EvoRec: A Multi-Agent Framework for Evolving Recommender Systems
EvoRec is a proposed multi-agent framework for automating recommender-system optimization. It targets agents that retain methodology across iterations and explore ideas beyond a predefined optimization space.
It describes an approach to reducing manual iteration in recommender-system optimization.
The Verification Horizon for Coding-Agent Rewards
The paper studies coding-agent reward signals, including test pass rates, LLM judges, and execution traces. It reports that each eventually stops tracking correctness and becomes vulnerable to hacking as task horizons grow.
Engineers designing long-horizon coding agents can use the findings to assess how long a reward signal remains a reliable proxy for correctness.
Towards Automating Scientific Review with Paper Assistant
The paper frames the scaling of scientific review as a systems challenge and proposes four levels of AI-human collaboration. It also discusses Google Paper Assistant as an early tool for scaling parts of the review process.
Engineers building research agents can learn how automated verification and human oversight are framed for scientific review.
Google’s Paper Assistant for Automating Scientific Review
The paper proposes a taxonomy for AI-assisted scientific review, motivated by the difficulty of scaling traditional peer review to match the influx of AI-assisted science. The post describes the system as focused on detecting errors and reviewing manuscripts.
Engineers building AI agents can examine how the system frames automated verification and review.
- AI agentsPaper
Red Queen Gödel Machine Co-Evolves Agents and Evaluators
The paper describes a self-improvement method that evolves an agent alongside its evaluator. It uses fixed evaluators within epochs and replaces them only when they score better on held-out ground truth.
Co-evolving evaluators may reduce benchmark gaming while preserving evaluation guarantees within each epoch.
- AI agentsPaper
DiscoBench Evaluates Clarification-Aware Deep Search Agents
DiscoBench is a benchmark for search agents powered by LLMs. It tests whether agents can handle vague, underspecified, or factually incorrect requests during multi-step retrieval and reasoning.
It gives engineers a way to evaluate how agents handle ambiguity in real-world search tasks.
Red Queen Gödel Machine Co-Evolves Agents and Evaluators
The paper proposes making evaluation part of recursive self-improvement by co-evolving agents and their evaluators. It addresses methods that assume a fixed verifier, benchmark, or labeled dataset.
Engineers building self-improving agents can consider how fixed evaluation criteria may become inadequate as agents improve.
BINEVAL uses binary questions for interpretable LLM evaluation
BINEVAL decomposes evaluation criteria into atomic yes-or-no questions and aggregates their verdicts into interpretable, multidimensional scores. The post reports training-free results matching or beating UniEval and G-Eval on three datasets.
Inspecting individual verdicts can help engineers diagnose evaluation scores and use them to guide prompt improvements.
- AI agentsPaper
Survey maps self-improving agents and experience-driven evolution
A survey and accompanying list cover how deployed agents turn interaction traces into durable capabilities, from self-evolution to meta-evolution.
Useful for engineers evaluating the infrastructure and approaches behind agents that improve from experience.
- AI agentsPaper
NOVA Uses Verification-Aware Agents to Evolve Recommender Architectures
NOVA is an agent harness for architecture evolution in industrial recommender systems. Its preview describes coordinated changes to model topology, feature configuration, and interaction modules under interface, resource, and serving constraints.
It addresses how agents can validate domain-specific constraints beyond whether generated code runs.
Survey examines evaluation of agent memory systems
The paper surveys the evolution of LLM agent memory into systems for persistent storage, retrieval, updates, consolidation, and lifecycle governance. It notes that existing evaluations often rely on end-to-end task metrics and treat memory as a black box.
Engineers can use its framing to assess what current agent memory benchmarks do—and do not—measure.
Ornith-1.0 releases open models for agentic coding
Ornith-1.0 is a family of open-source models ranging from 9B to 397B parameters. Its training uses reinforcement learning to optimize both coding solutions and the task-specific scaffolds that guide them.
Engineers can evaluate the models and their self-scaffolding approach for agentic coding workflows.
- AI agentsPost4 months ago @iScienceLuvr
Autodata uses agents to build training and evaluation data
Autodata is a method for AI agents to act as data scientists and create training and evaluation data. It includes data creation, data analysis, and meta-optimization stages.
The staged approach may help engineers structure agent workflows for generating and improving datasets.
- AI agentsPaper
NatureBench tests coding agents on 90 research tasks
NatureBench contains 90 tasks based on Nature-family papers and evaluates coding agents against published results without web search or access to the original method. The post reports that the best agent surpassed the published SOTA on 17.8% of tasks.
The benchmark offers a way to evaluate agents’ ability to select methods and reproduce research results.
- AI agentsPost4 months ago @ai_database
Automated code iteration improves scientific prediction methods
A post describes a Nature study in which researchers repeatedly generated and machine-scored code for scientific prediction tasks across six fields. Of 55 combinations of existing methods, 24 outperformed both source methods.
The work highlights how automated coding and evaluation loops could shorten experimentation on prediction software.
- AI agentsArticle
AutoResearch framework automates an RL experiment pipeline
The AutoResearch project describes an agent that planned GPU experiments and ran RL experiments on the DeepSeek 285B model. The post says it automated experiment design, coding, execution, debugging, and conclusion summaries.
Engineers can examine how the framework structures autonomous experimentation across an end-to-end RL workflow.
- AI agentsArticle
Deli AutoResearch automates RL experiment workflows
Deli AutoResearch is an open-source agent framework. The author says it autonomously planned GPU experiments and automated experiment design, code writing, execution, debugging, and conclusion summaries for RL runs on DeepSeek 285B.
The project offers an example of an agent handling multiple stages of an RL research workflow.
- AI agentsPost5 months ago @askalphaxiv
Self-Harness lets agents refine their own control harnesses
The paper explores agents mining their failures and proposing small harness edits, retaining changes that pass regression tests. It reports improved held-out pass rates on Terminal-Bench-2.0 across MiniMax, Qwen, and GLM.
Engineers can assess an approach to iteratively improving agent prompts, tools, retries, and verification without fine-tuning.
arxiv.org
