Applications of Risk Science to AI Fairness Evaluation: Principles, Challenges, and Best Practices

arXiv:2608.29478 · cs.CY, cs.AI · Submitted 2026-08-30 · Read on arXiv

cs.CY, cs.AI

Submitted: 2026-08-30

Updated: 2026-08-30

Code: https://github.com/kyrawilson/FairRisk

Terminology

• Epistemic Uncertainty — 16× in this paper · explained in 9 episodes
• Disparate Impact — 13× in this paper
• Accountability — 7× in this paper · explained in 2 episodes
• Accuracy — 5× in this paper · explained in 1 episode
• Demographic Disparity — 3× in this paper
• AUC — 2× in this paper · explained in 2 episodes
• Additivity — 2× in this paper · explained in 1 episode
• False-Negative Rate Ratio — 2× in this paper
• Gender Bias Score — 2× in this paper
• Generative AI — 2× in this paper · explained in 2 episodes
• Ground Truth Regression — 2× in this paper
• Interpretability — 2× in this paper · explained in 15 episodes
• Log Rank Regression — 2× in this paper
• Mean Absolute Error — 2× in this paper
• Mean Error Difference — 2× in this paper
• Mutual Information Difference — 2× in this paper
• Root Mean Square of TPRD — 2× in this paper
• AI Literacy — 1× in this paper · explained in 1 episode
• Adaptation — 1× in this paper · explained in 1 episode
• Artificial Intelligence (AI) — 1× in this paper · explained in 2 episodes
• Benchmark — 1× in this paper · explained in 2 episodes
• Benchmarking — 1× in this paper · explained in 3 episodes
• Explainability — 1× in this paper · explained in 3 episodes
• False Positive Rates — 1× in this paper
• Generalizability — 1× in this paper · explained in 6 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Personalization — 1× in this paper · explained in 2 episodes
• Predictive Modeling — 1× in this paper · explained in 2 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• True Positive Rates — 1× in this paper

Sources

• System Cards for AI-Based Decision-Making for Public Policy
• Don't Judge Me by My Face : An Indirect Adversarial Approach to Remove Sensitive Information From Multimodal Neural Representation in Asynchronous Job Video Interviews
• Degendering Resumes for Fair Algorithmic Resume Screening
• Closing the Gender Wage Gap: Adversarial Fairness in Job Recommendation
• Grading video interviews with fairness considerations
• Audit Cards: Contextualizing AI Evaluations

Related papers

• Reasoning Enhances Robustness to Prompt Injection in LLM-Based Consensus
• Generative AI Purpose-built for Social and Mental Health: A Real-World Pilot
• PersonaMem-v3: Toward Omni-Platform Personal Intelligence for Holistic User Understanding, Recommendation, and Agentic Tasks
• What is an intelligent system?
• AI University: An LLM-Powered Learning Assistant for Engineering---A Finite Element Method Case Study
• Generative AI Use in Entrepreneurship: An Integrative Review and an Empowerment-Entrapment Framework