Explainable AI Isn't Enough! Rethinking Algorithmic Contestability
Authors: Timo Freiesleben, Kristof Meding, Gunnar König
Venue / Source: arXiv (2026)
Standard XAI explanations only prove errors exist somewhere nearby, failing to provide the evidence needed to actually overturn an unfair algorithmic decision.
Key takeaways
- Algorithmic contestability is distinct from recourse, as it aims to overturn decisions perceived as incorrect rather than changing input features to fit model requirements.
- Common XAI methods like LIME, Anchors, and counterfactuals only indicate that model errors exist in the vicinity of an observation, which is insufficient for individual-level decision reversal.
- Three forms of evidence—predictive multiplicity, incorrect feature values, and overruling evidence—provide a valid normative basis for challenging algorithmic outcomes.
- The European GDPR likely already grants individuals legal rights to access the specific types of evidence required for effective contestability.
- Real-world adversarial contexts between decision makers and individuals complicate the implementation of this contestability framework.
Executive Summary (TL;DR)
Authors' abstract (original)
Machine learning systems increasingly make life-changing decisions about individuals, such as loan approvals, hiring, and cheating detection, raising a pressing question: how can individuals respond to negative decisions made by these opaque systems? While explainable artificial intelligence (XAI) has largely focused on algorithmic recourse -- helping individuals change their features to obtain a desired outcome -- the parallel problem of algorithmic contestability -- helping individuals review and correct erroneous algorithmic decisions -- has received far less attention, despite its central ethical and legal importance. We trace this neglect to the absence of clear formal definitions and a systematic operationalization of contestability as an algorithmic problem. To address it, we propose an operational definition of contestability as a natural complement to recourse: contestability starts from the presumption that a decision may be incorrect and focuses on identifying evidence to challenge and potentially overturn it, whereas recourse assumes the decision is valid and instead provides pathways for changing it. We show that standard XAI explanations, such as counterfactuals, LIME, or Anchors, even when combined with human intuitions about decision continuity or monotonicity, reveal only errors in the neighborhood of the individual, but provide insufficient grounds for overturning the decision at hand. Going thus beyond traditional XAI, we identify three types of evidence warranting reversal according to the decision maker's own ethical standards: predictive multiplicity, incorrect feature values, and neglected overruling evidence. We argue that these render decisions normatively indefensible and thus successfully contestable. Finally, we analyze how existing EU legislation connects to our framework and argue that individuals already hold some legal rights to these forms of evidence.
Quality & Relevance Evaluation Breakdown
Relevance for Educators & Research
- Provides a formal, normative basis for student-facing AI appeals processes in academic assessment.
- Offers a rigorous way to think about 'epistemic contestability' in student cheating detection systems.
- Connects formal decision theory to legal frameworks relevant to AI in educational institutions.
Methodology and Dataset
Theoretical framework development. The paper uses the 'South German Credit' dataset for illustrative examples of XAI limitations. No human participants or AI model training benchmarks were conducted for evaluation purposes.
Why Worth Reading
It provides a foundational, rigorous framework for moving beyond the 'explanation' phase of AI transparency into the 'contestation' phase, which is critical for the future of fair and resilient academic assessment.
Key Claims & Verified Quotes
Limitations & Study Design Flags
-
No adversarial testing:
found
“Many real-world settings, however, are adversarial, and the interests of individuals and decision makers are not aligned” (p. 10)
- Unrepresentative sample: not mentioned in the text
- The paper lacks empirical validation on real-world datasets, focusing instead on formal definitions and philosophical grounding.
- The proposed framework assumes non-adversarial conditions between the decision maker and the individual, which may not hold in many high-stakes assessment settings.
- Practical implementation of predictive multiplicity and 'overruling evidence' faces significant technical and proprietary barriers for organizations.
- The analysis does not address the full scope of potential human intuitions or alternative XAI methods beyond the selected subset.
- supports Algorithmic contestability has remained comparatively underexplored in technical literature. «it is widely agreed that algorithmic contestability remains under-researched, especially from a technical perspective», p. 2, para 3
- supports Common XAI methods are insufficient for justifying the reversal of individual algorithmic decisions. «they do not provide sufficient grounds to overturn a decision», p. 2, para 2
- [method / theoretical vs empirical]: The paper identifies a strong theoretical need for contestability but acknowledges that the proposed evidence types (like predictive multiplicity) are often difficult to implement for proprietary models.