Latent Impact and Differential Item Functioning Analysis for Asymmetric IRT Models
Authors: Gabriel Wallin, Qi Huang
Venue / Source: arXiv (2026)
Accounting for asymmetric item behavior in assessments allows for a cleaner distinction between genuine student bias and simple proficiency differences.
Key takeaways
- Asymmetric IRT models better capture empirical response processes where item characteristic curves are skewed.
- The proposed method uses l1-regularization to simultaneously estimate population impact and identify DIF without needing prior anchor knowledge.
- Accounting for ICC asymmetry reduces the risk of misattributing distributional differences to item-level bias.
- Latent class DIF results should be treated as exploratory unless supported by external validation like known demographics.
- The framework effectively handles cases where group membership (e.g., gender, language background) is unobserved.
Executive Summary (TL;DR)
Authors' abstract (original)
Differential item functioning (DIF) arises alongside latent population heterogeneity in many applications, and both must be accounted for when assessing measurement invariance. In many practical settings, however, the comparison groups are unobserved and anchor items are unknown. A further challenge is that item response theory models traditionally assume symmetric link functions, yet empirical response processes may exhibit substantial asymmetry. This paper proposes a general framework for jointly analysing impact and DIF under asymmetric item response models. Unobserved group differences are represented by latent classes within a mixture item response model, while item-specific shifts capture DIF effects. Assuming the number of DIF items is relatively small, an $\ell_1$-regularised estimator is used to simultaneously identify the latent classes and select DIF items without requiring observed group labels or pre-specified anchor items. A simulation study evaluates recovery of impact, item parameters, and DIF effects across a range of configurations. The method is illustrated using two empirical applications from educational testing. In one dataset, the selected model reveals both impact and item-level DIF, whereas in the other, the results indicate substantial impact but little evidence of item-level DIF.
Quality & Relevance Evaluation Breakdown
Relevance for Educators & Research
- Provides a sophisticated psychometric methodology for analyzing potential test bias in datasets where student metadata (e.g., AI vs non-AI use) might be latent.
- Offers a rigorous approach to parameter estimation using proximal EM algorithms.
- Relevant for discussions on test validity when assessment data is distorted by unsupervised or technological factors.
Methodology and Dataset
The study utilized a simulation design (N=500, 1000, 3000) and two empirical datasets: TIMSS mathematics items (N=605, Singapore/Morocco) and a university mathematics test (N=3000). The model uses a complementary log-log (CLL) link and Gumbel-distributed latent traits.
Why Worth Reading
It addresses a critical gap in psychometrics: how to detect test bias when comparison groups are latent and items exhibit the asymmetry common in complex mathematics assessments. It provides a robust, regularized alternative to standard symmetric IRT models.
Key Claims & Verified Quotes
No verified quotes available for this paper.
Limitations & Study Design Flags
- Monolingual: not mentioned in the text
- The framework does not currently provide confidence intervals or formal hypothesis tests for individual DIF effects.
- The latent class interpretation is irreducible without external validation and assumes the sparsity of DIF effects.
- Estimation accuracy for structural parameters (σ1) did not improve substantially with increased sample size in simulation.
- The model assumes the data-generating process correctly matches the CLL-Gumbel specification.
- [method / theoretical assumption]: The paper assumes DIF is sparse for identification, which might fail if DIF is pervasive or if non-zero delta parameters capture global model misfit rather than local bias.