A regularised latent‐class item response model for detecting measurement non‐invariance in ordinal response scales
Authors: Gabriel Wallin, Qi Huang
Venue / Source: British Journal of Mathematical and Statistical Psychology (2026)
Detect unfair test items and hidden student subgroups automatically without needing pre-existing group labels or anchor items.
Key takeaways
- The proposed latent-class model effectively detects differential item functioning without requiring pre-defined group identifiers or anchor items.
- Applying an l1-penalty to class-specific parameters helps resolve identifiability issues, particularly in identifying non-uniform response patterns.
- Simulation results suggest that test length significantly influences the classification accuracy of both respondents and problematic items.
- The reliance on a sparsity assumption for model selection remains a limitation, as does the current absence of formal post-selection hypothesis testing.
Executive Summary (TL;DR)
Authors' abstract (original)
Abstract Measurement non‐invariance arises when the psychometric properties of a scale differ across subgroups, undermining the validity of group comparisons. At the item level, this manifests as differential item functioning (DIF), where item responses differ across groups after controlling for the latent trait. This paper develops a framework for detecting DIF in ordinal scales without requiring known group labels or anchor items. We formulate a proportional‐odds latent‐class item response model in which individuals are assigned probabilistically to latent classes. DIF is captured through class‐specific intercept and slope shifts, allowing both uniform and non‐uniform DIF. Identification is achieved through an ‐penalised marginal likelihood under a sparsity assumption, with estimation implemented using a tailored EM algorithm. Because class‐specific slopes leave both the location and scale of each latent class unidentified, sparsity anchors the latent metric while selecting DIF effects. Simulation studies demonstrate accurate recovery of item parameters and both types of DIF. An empirical application to a personality test reveals latent subgroups with distinct response patterns and identifies items displaying potential class‐specific measurement non‐invariance. The framework provides a flexible approach for assessing measurement invariance in ordinal scales when comparison groups are unobserved or poorly defined.
Quality & Relevance Evaluation Breakdown
Relevance for Educators & Research
- Provides a robust quantitative method for identifying measurement non-invariance in large-scale datasets.
- Useful for modeling 'latent' cheating or AI-usage subgroups where specific student labels are unavailable.
- Demonstrates how sparsity constraints (l1-penalties) can solve identification problems in IRT.
Methodology and Dataset
The method uses a proportional-odds latent-class IRT model with an l1-penalized marginal likelihood and a proximal-gradient EM algorithm. The empirical study analyzes the Fisher Temperament Inventory (Analytical/Tough-minded subscale) with N = 4,967. Simulation studies test sample sizes of N = {1000, 5000}, item counts J = {15, 25, 50}, and varying latent class proportions.
Why Worth Reading
This paper offers a rigorous statistical solution to the 'hidden group' problem in assessment validity, enabling educators to detect if test items are functioning inconsistently across unlabeled student populations—an essential capability for maintaining test fairness in an era where AI-mediated learning might subtly alter student response behavior.
Key Claims & Verified Quotes
Limitations & Study Design Flags
- Standard errors and formal hypothesis tests are not provided for the DIF parameters due to the complexity of the post-selection inference.
- The model-selection procedure (using BIC) relies on the assumption that DIF is relatively rare across the item set (sparsity assumption).
- Results from the latent classes may not align with substantively labeled populations and require external validation.
- The computational load of the EM algorithm increases with the number of latent classes and quadrature nodes.
- Regularised DIF detection has been studied in multidimensional IRT models. «Regularisation-based DIF detection has also been studied in multidimensional IRT models using lasso and adaptive lasso penalties.», p. 11, para 43
- [method / methodological gap]: Previous latent DIF models required a fixed anchor set, whereas this framework treats anchor identification as a data-driven variable selection problem.
- [method / methodological gap]: Extends existing regularized models which typically focus on binary items to accommodate ordinal response scales and non-uniform DIF.