assessment_design
A Judge Should Know What Changed:Construct Validity for LLM-as-a-Judge Evaluation
Jianlin Chen, Wenhui Chen, Ziyao Lin, Chi Man Vong · arXiv (2026) · doi:10.48550/arxiv.2608.24419 · gemini-3.1-flash-lite
Papers tagged with: #assessment-design
assessment_design
GAUGE: When Not to Trust LLM-as-a-Judge in User-Simulated Evaluation of Task-Oriented Agents
Umesh Bodhwani, Thanh Tran, Kai Wei · arXiv (2026) · doi:10.48550/arxiv.2609.12191 · gemini-3.1-flash-lite
quasi_experiment
Pressure, Context, and Machine Self-Control: A Criminological Test of Reward Hacking in Generative AI Models
Murat Ozer, Isaac Kofi Nti · arXiv (2026) · doi:10.48550/arxiv.2610.04793 · gemini-3.1-flash-lite
assessment_design
Fighting Fire with Fire: On the Feasibility of Protecting Exercises Against AI Cheating
Tobias Braun, Jonas Grebe, Louis Rethfeld, Marcus Rohrbach · arXiv (2026) · doi:10.48550/arxiv.2608.01112 · gemini-3.1-flash-lite
psychometric_modeling
Check The Scoreboard: An Analysis of Scoring Schemes on Multiple-Choice Evaluation
Nishant Balepur, Paiheng Xu, Wei Ai, Eunsol Choi, Rachel Rudinger, Jordan Boyd-Graber · arXiv (2026) · doi:10.48550/arxiv.2608.29887 · gemini-3.1-flash-lite
psychometric_modeling
Every Token Counts: Exact Likert-Scale Distributions for Measuring LLM Attitudes and Biases
Davood Wadi, Mohsen Ghodrat, Matthew Philp · arXiv (2026) · doi:10.48550/arxiv.2608.10503 · gemini-3.1-flash-lite
psychometric_modeling
What Makes Something Hard(er)? Explaining Question Difficulty in Natural Language
Peng Cui, Qiaoyuan Zheng, Rudolf Debelak, Mrinmaya Sachan · arXiv (2026) · doi:10.48550/arxiv.2610.01627 · gemini-3.1-flash-lite
assessment_design
Towards Scalable Measurement of Durable Skills
Amir Globerson, Amy Keeling, Anisha Choudhury, Anna Iurchenko, Aviad Segal, Avinatan Hassidim, Ayça Çakmakli, Ben Gomes, Benn Witt, Cathy Cheunga, Cristine Legare, Diana Akrong, Eliad Carmi, Elisabeth Bauer, Gal Elidan, Hadas Gelbart, Hairong Mu, Katherine Chou, Lev Borovoi, Nir Kerem, Niv Efron, Noa Kerrem Gilo, Preeti Singh, Rajvi Kapadia, Rena Levitt, Roni Rabin, Ronit Levavi Morad, Rotem Yulzary, Shashank Agarwal, Sophie Allweis, Tracey Lee-Joe, Tzvika Stein, Yael Bar Moshe, Yael Haramaty, Yaniv Carmel, Yishay Mor, Yoav Bar Sinai, Yoav Bergner, Yossi Matias, Yuri Lev · arXiv (2026) · doi:10.48550/arxiv.2609.15864 · gemini-3.1-flash-lite