Papers tagged with: #психометрія

8.1
psychometric_modeling

One Percent of the Bank: Adaptive Item Selection Reads Targeted Contamination Up While the Whole Bank Barely Moves

Guan-Yuan Chen, Ya-Fen Yeh · Zenodo (CERN European Organization for Nuclear Research) (2026) · doi:10.5281/zenodo.23156543 · gemini-3.1-flash-lite

Read full analysis →

8.7
psychometric_modeling

LLM Judges as Raters: A Pre-Registered Audit of Severity, Halo, Reliability, and Version Instability in LLM Essay Scoring on Public Corpora

Veerendra Kumar Sunkavalli · arXiv (2026) · doi:10.48550/arxiv.2608.29517 · gemini-3.1-flash-lite

Read full analysis →

8.0
assessment_design

A Judge Should Know What Changed:Construct Validity for LLM-as-a-Judge Evaluation

Jianlin Chen, Wenhui Chen, Ziyao Lin, Chi Man Vong · arXiv (2026) · doi:10.48550/arxiv.2608.24419 · gemini-3.1-flash-lite

Read full analysis →

7.7
assessment_design

GAUGE: When Not to Trust LLM-as-a-Judge in User-Simulated Evaluation of Task-Oriented Agents

Umesh Bodhwani, Thanh Tran, Kai Wei · arXiv (2026) · doi:10.48550/arxiv.2609.12191 · gemini-3.1-flash-lite

Read full analysis →

7.6
psychometric_modeling

Personality Without Persons? A Psychometric Critique of Big Five Testing in Large Language Models

Kim Zierahn, Cristina Cachero, Anna Korhonen, Nuria Oliver · arXiv (2026) · doi:10.48550/arxiv.2607.02325 · gemini-3.1-flash-lite

Read full analysis →

7.5
psychometric_modeling

Latent Impact and Differential Item Functioning Analysis for Asymmetric IRT Models

Gabriel Wallin, Qi Huang · arXiv (2026) · doi:10.48550/arxiv.2605.05684 · gemini-3.1-flash-lite

Read full analysis →

7.5
psychometric_modeling

A Dual-Dimensional LLM Framework for Automated Item Incidental Content Similarity Analysis in Large-Scale Assessments

Jing Huang, Jihong Zhang, Hua-Hua Chang · arXiv (2026) · doi:10.48550/arxiv.2608.24825 · gemini-3.1-flash-lite

Read full analysis →

7.5
psychometric_modeling

Check The Scoreboard: An Analysis of Scoring Schemes on Multiple-Choice Evaluation

Nishant Balepur, Paiheng Xu, Wei Ai, Eunsol Choi, Rachel Rudinger, Jordan Boyd-Graber · arXiv (2026) · doi:10.48550/arxiv.2608.29887 · gemini-3.1-flash-lite

Read full analysis →

7.5
psychometric_modeling

Every Token Counts: Exact Likert-Scale Distributions for Measuring LLM Attitudes and Biases

Davood Wadi, Mohsen Ghodrat, Matthew Philp · arXiv (2026) · doi:10.48550/arxiv.2608.10503 · gemini-3.1-flash-lite

Read full analysis →

8.0
psychometric_modeling

From 0-Order Selection to 2-Order Judgment: Combinatorial Hardening Exposes Compositional Failures in Frontier LLMs

Hanmeng Liu, Shichao Weng, Xiulai Liu, Zhicai Zhang, Anli Yan, Xiaozhang Liu · arXiv (2026) · doi:10.48550/arxiv.2605.07268 · gemini-3.6-flash

Read full analysis →

7.5
psychometric_modeling

Every Wrong Answer Counts: Option-Level Psychometrics for LLM Multiple-Choice Benchmarks

Xiao Fei, Yang Zhang, Sarah Almeida Carneiro, Michalis Vazirgiannis · arXiv (2026) · doi:10.48550/arxiv.2608.02966 · gemini-3.1-flash-lite

Read full analysis →

7.5
assessment_design

Do Coding Agents Deceive Us? Detecting and Preventing Cheating via Capped Evaluation with Randomized Tests

Thanawat Lodkaew, Johannes Ackermann, Soichiro Nishimori, Nontawat Charoenphakdee, Masashi Sugiyama, Takashi Ishida · arXiv (2026) · doi:10.48550/arxiv.2606.07379 · gemini-3.1-flash-lite

Read full analysis →

7.5
psychometric_modeling

An Interpretable and Scalable Framework for Evaluating Large Language Models

Xinhao Qu, Qiang Heng, Hao Zeng, Xiaoqian Liu · arXiv (2026) · doi:10.48550/arxiv.2605.07046 · gemini-3.7-flash

Read full analysis →

7.5
psychometric_modeling

Are Near-Tied LLM Rankings Robust to Family-DIF-Guided Benchmark Recomposition?

Qiaoyuan Zheng, Yiqu Yang · arXiv (2026) · doi:10.48550/arxiv.2609.00482 · gemini-3.1-flash-lite

Read full analysis →

7.8
psychometric_modeling

Detection of two-way outliers in multivariate data and application to cheating detection in educational tests

Yunxiao Chen, Yan Lu, Irini Moustaki · The Annals of Applied Statistics (2022) · doi:10.1214/21-aoas1564 · gemini-3.1-flash-lite

Read full analysis →

7.4
psychometric_modeling

What exam scores can and cannot prove about unauthorized AI assistance: Evidence from a highly public classroom episode

Chad M. Topaz, Utsav Bahl · arXiv (2026) · doi:10.48550/arxiv.2607.27978 · gemini-3.1-flash-lite

Read full analysis →

7.4
psychometric_modeling

Scalable Text-Embedding-informed Cognitive Diagnosis of Large Language Models

Jia Liu, Zhiyu Xu, Yuqi Gu · arXiv (2026) · doi:10.48550/arxiv.2603.14676 · gemini-3.7-flash

Read full analysis →

7.3
psychometric_modeling

Identifying Causes of Test Unfairness: Manipulability and Separability

Youmi Suk, Weicong Lyu · arXiv (2026) · doi:10.48550/arxiv.2601.13449 · gemini-3.1-flash-lite

Read full analysis →

8.0
psychometric_modeling

Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Why

Delip Rao, Chris Callison-Burch · arXiv (2026) · doi:10.48550/arxiv.2606.00093 · gemini-3.8-flash

Read full analysis →

7.3
psychometric_modeling

Which AI model is truly the best: how reliable are public benchmarks of frontier AI models?

Miloš Kankaraš · Zenodo (CERN European Organization for Nuclear Research) (2026) · doi:10.5281/zenodo.22998204 · gemini-3.7-flash

Read full analysis →

7.2
detector_evaluation

Why AI Detection Fails for Academic Integrity

Jonathan A. Karr, Grigorii Khvatskii, Ting Hua, Nitesh V. Chawla · arXiv (2026) · doi:10.48550/arxiv.2608.11256 · gemini-3.1-flash-lite

Read full analysis →

7.2
psychometric_modeling

Extending Item Response Theory for Efficient and Meaningful Multilingual Evaluation

Gili Lior, Tzviel Frostig, Gabriel Stanovsky, Matan Eyal · arXiv (2026) · doi:10.48550/arxiv.2606.15643 · gemini-3.7-flash

Read full analysis →

7.2
psychometric_modeling

CalibratedRubric: Task-Adaptive Rubric Banks for Open-Ended LLM Evaluation

Mengting Chen, Yanshu Sun, Wanting Liang, Beidi Luan, Rui Sun, Dezhi Chen, Jing Li, Zuo Bai · arXiv (2026) · doi:10.48550/arxiv.2607.29252 · gemini-3.1-flash-lite

Read full analysis →

7.2
assessment_design

Towards Scalable Measurement of Durable Skills

Amir Globerson, Amy Keeling, Anisha Choudhury, Anna Iurchenko, Aviad Segal, Avinatan Hassidim, Ayça Çakmakli, Ben Gomes, Benn Witt, Cathy Cheunga, Cristine Legare, Diana Akrong, Eliad Carmi, Elisabeth Bauer, Gal Elidan, Hadas Gelbart, Hairong Mu, Katherine Chou, Lev Borovoi, Nir Kerem, Niv Efron, Noa Kerrem Gilo, Preeti Singh, Rajvi Kapadia, Rena Levitt, Roni Rabin, Ronit Levavi Morad, Rotem Yulzary, Shashank Agarwal, Sophie Allweis, Tracey Lee-Joe, Tzvika Stein, Yael Bar Moshe, Yael Haramaty, Yaniv Carmel, Yishay Mor, Yoav Bar Sinai, Yoav Bergner, Yossi Matias, Yuri Lev · arXiv (2026) · doi:10.48550/arxiv.2609.15864 · gemini-3.1-flash-lite

Read full analysis →