Papers tagged with: #rubrics

7.8
assessment_design

Rubrics Carry the Standard: Shapley Attribution for Checklist-Based LLM Judges

Saeid Asgari, Rui Ying, Sara Malvar, Rakshanda Agarwal, Bruce Sun, Claire Wang, Leonardo de Oliveira Nunes, Ranveer Chandra · arXiv (2026) · doi:10.48550/arxiv.2610.24971 · gemini-3.8-flash

Read full analysis →

8.0
psychometric_modeling

Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Why

Delip Rao, Chris Callison-Burch · arXiv (2026) · doi:10.48550/arxiv.2606.00093 · gemini-3.8-flash

Read full analysis →

7.4
psychometric_modeling

Inside the Scorer: Mechanistic Evidence on Rubric Control in LLM-Based Automated Essay Scoring from 25 Base Models

Chungche Huang, Po-Ya Angela Wang, Shu-Kai Hsieh · arXiv (2026) · doi:10.48550/arxiv.2610.25724 · gemini-3.7-flash

Read full analysis →

7.3
psychometric_modeling

From Scoring to Explanations: Evaluating SHAP and LLM Rationales for Rubric-based Teaching Quality Assessment

Ivo Bueno, Babette Bühler, Philipp Stark, Tim Fütterer, Ulrich Trautwein, Dorottya Demszky, Heather Hill, Enkelejda Kasneci · arXiv (2026) · doi:10.48550/arxiv.2606.05180 · gemini-3.6-flash

Read full analysis →

7.2
psychometric_modeling

CalibratedRubric: Task-Adaptive Rubric Banks for Open-Ended LLM Evaluation

Mengting Chen, Yanshu Sun, Wanting Liang, Beidi Luan, Rui Sun, Dezhi Chen, Jing Li, Zuo Bai · arXiv (2026) · doi:10.48550/arxiv.2607.29252 · gemini-3.1-flash-lite

Read full analysis →

7.2
assessment_design

Automated Refinement of Essay Scoring Rubrics for Language Models via Reflect-and-Revise

Keno Harada, Lui Yoshida, Takeshi Kojima, Yusuke Iwasawa, Yutaka Matsuo · Conference on Computational Natural Language Learning (2026) · doi:10.18653/v1/2026.conll-main.47 · gemini-3.6-flash

Read full analysis →