assessment_design

NeutronGym: Physics-Graded Neutron Instrument Design for LLM Agents

Authors: Lijie Ding, Changwoo Do
Venue / Source: arXiv (2026)

NeutronGym replaces unreliable LLM judges with a physically-grounded ray-tracing environment that forces AI agents to learn actual instrument design principles.

Key takeaways

Replacing LLM judges with physically-validated simulator rewards provides a robust, non-debatable metric for scientific instrument design.
A level-resolved reward ladder providing partial credit is necessary for effective reinforcement learning in complex engineering environments.
The environment includes a stringent 'admission gate' (no-model probes) to filter out degenerate tasks that could be solved by simple rules or memorization.
Imitation learning of successful agent episodes resulted in performance regression, whereas reinforcement learning successfully improved design capabilities.
Even with training, smaller 8B models could not close the performance gap compared to frontier models on the same held-out tasks.

Executive Summary (TL;DR)

In plain words In the paper's terms

This study creates a new virtual laboratory where AI agents can design scientific neutron instruments. Instead of asking an AI to judge the quality of a design, the researchers use a physics simulation that measures real performance, ensuring the scores are trustworthy. The authors found that training smaller AI models in this environment allowed them to significantly improve their design skills, though they still trail behind the most powerful models.

The paper introduces NeutronGym, an executable environment for neutron instrument design that replaces LLM-based judges with physical ray-tracing simulations and a level-resolved reward ladder. It demonstrates that reinforcement learning can train an 8B model to exceed the performance of a 32B model on gated tasks by learning physical design parameters from reward signals. Quantitative findings show that while reinforcement learning on this ladder achieves performance matching classical optimizers, frontier models still outperform these trained agents by a margin.

Authors' abstract (original)

Designing a scientific instrument tests whether language-model agents can do physics rather than recall it, provided the grading cannot be argued with. We introduce NeutronGym, to our knowledge the first executable environment for neutron instrument design: agents build instruments through validating tools, McStas ray-traces what they build, and a level-resolved ladder grades syntax, runtime, structure and science with no LLM judge. Procedural families supply unlimited instances of a fixed layout whose design parameters the agent must set, with held-out parameter regimes; a curated slice, McStasBench, adds 16 tasks from published instruments behind memorization probes and a sandbox. Seven models reproduce at most 7 of the 16, none retrieves a reference, and none meets an improvement target. The environment also trains. Reinforcement learning on its reward takes Qwen3-8B from 11% to 77% of held-out instances of a family whose targets come from a hidden design (69% at a second seed), past an untrained Qwen3-32B, and the recipe holds, at one seed each, on three further gated families. The analysis says what that gain is. Without the ladder's partial credit it collapses by 60 points. From reward alone the trained model reaches what a classical optimizer reaches, at the agent's simulation budget, only when handed the closed-form physics (77% against 81%, a gap that does not separate at this size), while frontier models still solve 98-99%. Getting a trustworthy result meant failing four task designs that no-model baselines could solve, and we release the probes that found them.

Quality & Relevance Evaluation Breakdown

Novelty

8.0/10

Methodological Rigor

8.0/10

Engineering Countermeasures

8.0/10

Consensus Challenge

5.0/10

Topic Relevance

8.0/10

Relevance for Educators & Research

Demonstrates the use of automated environments for verifying competence, relevant to the 'AI-resistant assessment' theme.
Provides a methodology for 'admission gates' to ensure assessment tasks aren't solvable via prompt-based shortcuts.
Highlights the validity risks of LLMs in assessment (contamination and recall) and proposes concrete engineering countermeasures.

Methodology and Dataset

The study evaluates seven models (Claude Sonnet 5, Claude Haiku 4.5, Gemini 3.6 Flash, Gemini 3.5 Flash-Lite, Llama 4 Maverick, Qwen3-32B, Qwen3-8B) using 16 benchmark tasks (McStasBench) and procedurally generated families (n=300 instances per family for testing, n=2,810 states for RL training).

Why Worth Reading

This paper offers a rigorous approach to assessment design where outcomes are verified by physical laws rather than LLM judges, providing a model for high-stakes, AI-resistant evaluation in scientific and engineering domains.

Key Claims & Verified Quotes

Dense partial credit (the reward ladder) is essential for training LLMs on complex engineering tasks.

Quote: “Replacing the level-resolved reward with pass/fail only, all else fixed, collapses the result to 16.7%” — 10.48550/arxiv.2610.03631, p. 9, para. 3 (exact)

Imitation learning (fine-tuning on successful model outputs) is ineffective for improving agent design capabilities. overclaim

Quote: “Cloning your own passing turns teaches a move that was right in the states you already handled and is a trap everywhere else” — 10.48550/arxiv.2610.03631, p. 7, para. 20 (exact)

Limitations & Study Design Flags

The benchmark slice is small (16 tasks) and lacks facility-validated instruments for hold-out tasks.
Training results for multi-task reinforcement learning were found to be unstable across different random seeds.
The procedural task families are limited to setting parameters for a fixed instrument layout, excluding topology choice.
The simulation environment is idealized (e.g., gravity off, no background noise).

Model inference (gaps & contradictions)

[method / contradiction]: The authors find that imitation learning (rejection sampling/behavioral cloning) degrades performance, contradicting common assumptions in some educational and alignment literature that successful student outcomes are optimal targets for imitation.

#AI-Resistant Assessment #Reinforcement Learning #Psychometric Modeling #Engineering Design #Validation