AI in Education
UK EN

Papers tagged with: #автоматизоване-оцінювання

8.7
psychometric_modeling

LLM Judges as Raters: A Pre-Registered Audit of Severity, Halo, Reliability, and Version Instability in LLM Essay Scoring on Public Corpora

Veerendra Kumar Sunkavalli · arXiv (2026) · doi:10.48550/arxiv.2608.29517 · gemini-3.1-flash-lite

Read full analysis →

8.0
assessment_design

A Judge Should Know What Changed:Construct Validity for LLM-as-a-Judge Evaluation

Jianlin Chen, Wenhui Chen, Ziyao Lin, Chi Man Vong · arXiv (2026) · doi:10.48550/arxiv.2608.24419 · gemini-3.1-flash-lite

Read full analysis →

7.7
assessment_design

GAUGE: When Not to Trust LLM-as-a-Judge in User-Simulated Evaluation of Task-Oriented Agents

Umesh Bodhwani, Thanh Tran, Kai Wei · arXiv (2026) · doi:10.48550/arxiv.2609.12191 · gemini-3.1-flash-lite

Read full analysis →

7.7
psychometric_modeling

Diagnosing the Reliability of LLM-as-a-Judge via Item Response Theory

Junhyuk Choi, Sohhyung Park, Chanhee Cho, Hyeonchu Park, Bugeun Kim · arXiv (2026) · doi:10.48550/arxiv.2602.00521 · gemini-3.7-flash

Read full analysis →

7.4
psychometric_modeling

Interpretable, Fairly Evaluated Automated L2 Speaking Assessment that Beats the Single-Human Ceiling and Why Pause Encoding Does Not Change LLM Fluency Scores

Eichi Uehara · arXiv (2026) · doi:10.48550/arxiv.2608.26137 · gemini-3.8-flash

Read full analysis →

7.8
assessment_design

Rubrics Carry the Standard: Shapley Attribution for Checklist-Based LLM Judges

Saeid Asgari, Rui Ying, Sara Malvar, Rakshanda Agarwal, Bruce Sun, Claire Wang, Leonardo de Oliveira Nunes, Ranveer Chandra · arXiv (2026) · doi:10.48550/arxiv.2610.24971 · gemini-3.8-flash

Read full analysis →

8.0
psychometric_modeling

Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Why

Delip Rao, Chris Callison-Burch · arXiv (2026) · doi:10.48550/arxiv.2606.00093 · gemini-3.8-flash

Read full analysis →

7.4
psychometric_modeling

Inside the Scorer: Mechanistic Evidence on Rubric Control in LLM-Based Automated Essay Scoring from 25 Base Models

Chungche Huang, Po-Ya Angela Wang, Shu-Kai Hsieh · arXiv (2026) · doi:10.48550/arxiv.2610.25724 · gemini-3.7-flash

Read full analysis →

7.3
psychometric_modeling

From Scoring to Explanations: Evaluating SHAP and LLM Rationales for Rubric-based Teaching Quality Assessment

Ivo Bueno, Babette Bühler, Philipp Stark, Tim Fütterer, Ulrich Trautwein, Dorottya Demszky, Heather Hill, Enkelejda Kasneci · arXiv (2026) · doi:10.48550/arxiv.2606.05180 · gemini-3.6-flash

Read full analysis →

7.3
assessment_design

Prompt Sensitivity in LLM-Based Essay Scoring is Model-Specific

Giulio Corsi, Alexandru Marcoci, Maryam Abo-Tabik, Georgiana Thorpe Apreutesei, Lyba Razzaq, Roni Tibon, Yael Benn, Deborah Talmi · CLOK (University of Central Lancashire) (2026) · doi:10.5281/zenodo.21039771 · gemini-3.7-flash

Read full analysis →

7.2
psychometric_modeling

CalibratedRubric: Task-Adaptive Rubric Banks for Open-Ended LLM Evaluation

Mengting Chen, Yanshu Sun, Wanting Liang, Beidi Luan, Rui Sun, Dezhi Chen, Jing Li, Zuo Bai · arXiv (2026) · doi:10.48550/arxiv.2607.29252 · gemini-3.1-flash-lite

Read full analysis →

7.2
assessment_design

Towards Scalable Measurement of Durable Skills

Amir Globerson, Amy Keeling, Anisha Choudhury, Anna Iurchenko, Aviad Segal, Avinatan Hassidim, Ayça Çakmakli, Ben Gomes, Benn Witt, Cathy Cheunga, Cristine Legare, Diana Akrong, Eliad Carmi, Elisabeth Bauer, Gal Elidan, Hadas Gelbart, Hairong Mu, Katherine Chou, Lev Borovoi, Nir Kerem, Niv Efron, Noa Kerrem Gilo, Preeti Singh, Rajvi Kapadia, Rena Levitt, Roni Rabin, Ronit Levavi Morad, Rotem Yulzary, Shashank Agarwal, Sophie Allweis, Tracey Lee-Joe, Tzvika Stein, Yael Bar Moshe, Yael Haramaty, Yaniv Carmel, Yishay Mor, Yoav Bar Sinai, Yoav Bergner, Yossi Matias, Yuri Lev · arXiv (2026) · doi:10.48550/arxiv.2609.15864 · gemini-3.1-flash-lite

Read full analysis →