A Scalable Entity-Based Framework for Auditing Bias in Large Language Models

arXiv:2601.12374 · cs.CL, cs.AI · Submitted 2026-01-18 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "A Scalable Entity-Based Framework for Auditing Bias in Large Language Models".

Jane: A scalable entity-based framework for auditing bias in large language models introduces a systematic method using named entities as controlled probes to measure systematic disparities in model behavior,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So, Jane, we just finished diving into this paper, "A Scalable Entity-Based Framework for Auditing Bias in Large Language Models," and it sounds like they've really tackled the problem of bias evaluation at scale. It seems like they're moving beyond just looking at isolated examples to building a system that can measure how models behave consistently across huge amounts of data.

Jane: That’s right, Tom; what caught my attention is how they use named entities as controlled probes rather than just testing random prompts. It simplifies the process by letting them isolate the effect of swapping one specific name for another within the same context to see if the model's output shifts in a predictable way.

Lu: It’s fascinating because they are addressing that core tension between statistical control and ecological validity, which is something we grapple with constantly when we try to build truly representative AI systems. This framework seems designed specifically to bridge that gap by using synthetic data to ensure their bias measurements actually reflect what happens in natural text.

Meng: From an engineering standpoint, the idea of a scalable framework across billions of data points suggests they’ve developed a very robust pipeline for generating and processing these inputs without getting bogged down in manual labeling every time. How do you think the synthetic data generation part ensures it captures the nuances we see in real-world text?

Lalam: I find the concept of using synthetic data to reproduce bias patterns really interesting because it lets us create a controlled environment where we can reliably observe those disparities without needing massive, perfectly balanced real-world datasets right away. It helps us build a baseline understanding of how models might react before we deploy them widely.

Tom: Exactly! And the paper lays out their core method clearly: they define entity bias as the change in model predictions when you substitute one entity in an input where the rest of the text already gives you a clear answer, meaning any deviation points directly to reliance on that specific name instead of the actual evidence provided.

Jane: That definition is really helpful for understanding what they mean by systematic disparity; it’s not about whether a model is generally biased, but specifically *how* it relies on an entity name when the text itself should be sufficient. This gives us a much more granular way to understand the mechanism at play.

Title and authors: Lu: The math they use to operationalize this—calculating the expectation of label weights under a posterior distribution and then normalizing it by subtracting the mean and standard deviation across entities—that’s quite rigorous for quantifying that reliance, isn't it? It moves beyond simple observation into a statistical measure.

Meng: I see the mathematical rigor, but I wonder about the practical implementation on a massive scale; keeping track of those scores across billions of data points while maintaining accuracy sounds like a huge computational challenge to manage in a production environment.

Lalam: That’s where the scalability aspect really shines, Meng; if you can automate that aggregation at the entity level before hypothesis testing, it makes the whole process feasible for large-scale auditing rather than just small experiments. It moves this from theoretical control to practical application.

Tom: Right, so they’ve put together this structure using entity sets across domains like politics and economics, down to specific tasks like financial risk assessment, which gives us a real view of where these biases show up in the real world.

Jane: And their method for generating textual templates using a keyword-driven pipeline that combines syntactic anchors with domain-specific terminology is key because it makes sure those substitutions don't introduce new counterfactual content, keeping the measurement focused on entity substitution alone.

Lu: The paper mentions they conduct the largest bias audit to date, covering one point nine billion data points across various entities and prompting strategies, which speaks to the sheer scope of their effort in gathering this kind of comprehensive data.

Meng: That scale is impressive, but I'm curious about how they handle the structural drivers we talked about—like language sensitivity or model scale—within that massive dataset; it must be complicated to disentangle those effects from the raw entity bias scores.

Lalam: Lalam thinks what’s really exciting is their validation process: comparing measurements from synthetic and real data across five distinct benchmarks like MAD-TSC, P-Stance, FinEntity, and LIAR to show the Pearson correlation between them. That correlation suggests the signal they find isn't just a fluke of using synthetic text but is genuinely present in natural language.

Title and authors: Tom: That validation is crucial for building trust in the results because it proves that whatever patterns they see are consistent whether you look at real or synthetic scenarios, which really strengthens their argument for using this framework.

Jane: It really helps ground the findings; if they could only show correlation with synthetic data, we wouldn't have much confidence that these biases translate to actual deployment risks in the wild. This validation process gives the entire audit a strong statistical foundation for making claims about real-world behavior.

Lu: Thinking about implications, this work suggests that we can move past just saying "the model is biased" and start pinpointing exactly which entity associations are causing performance degradation on specific tasks, which opens up so many avenues for targeted intervention in the AI development cycle.

Meng: Targeted intervention sounds practical; if the system can tell us precisely that a certain entity-task combination is problematic, we can move toward fine-tuning or constraint injection specifically for those vectors instead of trying to retrain the entire massive model from scratch.

Lalam: And I think for culture, this moves us closer to creating more nuanced and fair AI; if we can understand the specific ways models associate certain entities with outcomes like political leaning or financial stability, it gives us tools to build systems that aren't just statistically accurate but also contextually sensitive.

Tom: So, to wrap up on these improvements they suggest, it’s about creating a system that acts as a mandatory pre-deployment gate for high-stakes applications and then using those entity bias scores to identify specific high-risk entity-task combinations that need targeted mitigation strategies.

Jane: And the idea of explainable bias attribution is really powerful because it gives developers actionable insight, telling them not just *that* there's a problem, but exactly *why* it’s happening based on the entity's priors.

Lu: I also think the structural relationship mapping part, using latent space analysis to put entities into a 2D map based on how the model treats them across different contexts like language or task, could reveal hidden clusters of entities that are treated similarly internally.

Meng: That idea of mapping latent clusters is compelling because it suggests we might find systemic patterns in how the AI represents certain groups or concepts internally, even if those representations aren't explicitly stated in the text.

Title and authors: Lalam: And finally, the synthetic data validation engine is a fantastic tool for ensuring that any bias findings we report are backed by high-confidence statistical foundations derived from comparing real and synthetic proxies.

Tom: So, to sum up this paper on "A Scalable Entity-Based Framework for Auditing Bias in Large Language Models," it’s a very systematic approach that uses controlled probes to measure disparities across huge datasets, validates those findings against synthetic data, and provides a way to attribute bias directly to specific entity associations within model behavior.

Jane: It really gives us the tools we need to move from general observations about AI bias toward a rigorous, scalable method for quantifying exactly where and how biases manifest in large language models before they reach high-stakes applications.

Lu: This work sets a new standard for auditing because it provides the methodology to generate the one point nine billion data points needed for statistically robust studies on entity associations across multiple domains and languages.

Meng: I think the practical impact lies in how this framework allows us to move from broad model adjustments to highly targeted interventions based on specific, measurable entity-task risk profiles.

Lalam: For culture and deployment, the implication is that we can build AI systems that are not only more accurate but also more explicitly designed to avoid reinforcing unfair associations tied to specific groups or entities.

Tom: Well said, Lalam; this paper really shows us how to turn abstract concerns about model fairness into concrete, quantifiable metrics we can actually measure and control in the development pipeline.

Jane: It feels like a solid foundation for future work because it successfully reconciles the need for statistical rigor with the practical necessity of testing on data that mirrors real-world usage patterns through synthetic validation.

Lu: The future work suggested seems focused on exploring how these structural drivers—like language sensitivity or model scale—interact with these entity measurements, which opens up deep avenues for understanding model architecture and training dynamics.

Meng: I'm interested in seeing how they refine the mitigation strategies when we move beyond simple counterfactual constraints to more complex, learned debiasing fine-tuning specifically targeting those identified entity-task vectors.

Lalam: And I think it shows that even with these specific measurements, there’s always room to improve; the framework itself is a tool, and its application will continue to evolve as we build more sophisticated ways to understand AI behavior.

The paper's summary: Tom: So, we've been digging into this paper about auditing bias in large language models using named entities as controlled probes, and now we need to break down what their summary actually says about how they do it.

Jane: Exactly! The main thing is that they’ve developed a structured way to measure systematic disparities by focusing on how the model's predictions change when you just swap out one specific name in a sentence, assuming everything else stays the same.

Lu: That’s the core mechanism, and what really excites me about their summary is how they operationalize this concept into measurable scores. It turns abstract bias into something concrete through a three-step calculation process involving posterior distributions and normalization across all entities.

Meng: From an engineering standpoint, I'm interested in that normalization step; how do they keep the score meaningful when you’re looking at billions of potential entity combinations? It sounds like a lot of math to manage on a production scale.

Lalam: What I find most compelling about their summary is the emphasis on using synthetic data to align their real-world measurements, which they validate across several benchmarks, proving that these bias signals are consistent whether you look at natural text or generated examples.

Tom: That’s a huge deal for credibility; if they can show that the patterns hold up across different types of data, it means we can trust these findings more when we apply them to actual deployed systems.

Jane: It really simplifies things by defining entity bias as reliance on a name rather than the actual evidence in the context, which is a much clearer way to diagnose model behavior.

Lu: And their framework isn't just about detection; they’ve built this scalable structure using domain-specific entity sets and curated textual templates designed to be 'evidence-light,' which keeps the measurement focused on that specific name substitution.

Meng: It sounds like they’re trying to control the variables so that the only thing causing a difference in output is the entity itself, which would make debugging much more tractable for us.

Lalam: And thinking about what this means for culture, this structured approach gives us a way to systematically identify and quantify unfair associations embedded in AI, which is crucial for building systems that promote fairness rather than just reflecting existing societal biases.

Tom: So they’ve given us a rigorous mathematical tool that moves beyond simple observation to quantifying exactly how entity identity drives model predictions in real-world tasks.

Jane: That rigor, combined with the synthetic validation, gives us a much stronger foundation for understanding where and how these disparities arise within large language models.

Lu: This paper lays out a very solid methodology that scales well because it moves away from manual checking toward an automated auditing framework that can handle massive amounts of data efficiently.

Meng: I see the scalability aspect, but I still have to think about the computational cost of generating those diverse entity lists and then running all those label weight calculations for every context.

Lalam: But that’s where their summary hints at the future; if we can automate this measurement, it opens up a path for building continuous monitoring systems that watch deployed AI applications for shifts in bias over time.

Tom: So, what they've given us is a way to build a scalable system for auditing bias by using entities as precise probes, validated against synthetic data to ensure the results reflect real-world text.

Jane: It’s about turning complex issues of model reliance into simple, quantifiable metrics that we can track and improve.

Lu: The implications are huge because this framework could become a standard tool for vetting any AI deployed in sensitive areas like finance or law, giving developers a systematic way to check their work before it goes live.

Meng: If we can use these entity-task combinations to trigger targeted mitigation strategies, it means we don't have to retrain the whole system just because one specific bias vector pops up.

Lalam: And for culture, this structured approach helps us build AI that is not only more accurate but also more explicitly designed to avoid reinforcing unfair associations tied to specific groups or entities in a transparent way.

Tom: So we’ve got a scalable, validated framework that turns model behavior into quantifiable metrics, and the potential for targeted intervention is clear.

Jane: It really gives us the vocabulary and the methodology to move from guessing about bias toward systematically measuring and controlling it across different domains.

Lu: What I see is that this isn't just about fixing errors; it’s about understanding the underlying structural drivers of bias, like language sensitivity or model scale, which helps us design more robust architectures.

Meng: So the next step is figuring out how to actually implement those targeted debiasing fine-tuning strategies based on these specific entity-task risk profiles they identify.

Lalam: And ultimately, this paper offers a blueprint for building a more accountable AI ecosystem where fairness isn't just a vague goal but something that can be rigorously measured and continuously improved through systematic auditing.

The paper's improvements: Tom: So, we've been talking about how this framework measures bias using named entities as probes, and now we’re looking at what improvements the authors suggest to make this tool even more useful for developers.

Jane: The authors propose several ways to enhance this method beyond just the core scoring mechanism, focusing on creating a more actionable system for real-world use.

Lu: They suggest building a scalable pre-deployment gate, essentially turning this audit into a mandatory check before any AI application goes live in high-stakes sectors.

Meng: That sounds like exactly what we need; if we can integrate this as a gateway, it forces developers to confront these entity biases early in the process rather than finding them after deployment.

Lalam: I’m really interested in the idea of targeted risk mitigation strategies; instead of a blanket fix, they suggest identifying specific high-risk entity-task pairings that need focused debiasing.

Tom: That’s smart thinking; it shifts the focus from broad model adjustments to highly precise interventions based on measurable risk profiles we identify.

Jane: And they also propose a way to provide more granular attribution of bias, so developers don't just get a score, but understand which specific entity associations are causing the issue.

Lu: That level of explainability is vital; it moves the conversation from "the model is biased" to "entity X’s association with task Y is driving this result."

Meng: It makes sense; if we know precisely which entity-task vector is causing performance degradation, we can apply specific constraints or fine-tuning adjustments just to that area.

Lalam: And I think the continuous monitoring idea, correlating performance metrics with entity bias scores over time, is a powerful way to catch subtle shifts in model behavior that might otherwise slip through.

Tom: It really shows they’re thinking about the long term, not just a one-time audit but a system that can evolve alongside the AI application.

Jane: The validation engine for synthetic data is also an improvement; it ensures that any real-world findings we get are backed by statistical confidence derived from comparing them to controlled synthetic scenarios.

Lu: It’s interesting how they connect this back to the structural drivers we discussed earlier, suggesting that measuring these entity biases can help us understand those underlying factors like language sensitivity or model architecture.

Meng: If they can map entities into a latent space based on how the model treats them across different contexts, that could reveal hidden patterns in how the AI internally organizes information about different groups.

Lalam: That idea of mapping latent clusters is incredibly hopeful because it suggests we might find systemic ways the model processes certain concepts that aren't obvious from just looking at text or simple attributes.

Tom: So the improvements focus on making this tool not just descriptive, but prescriptive—giving us a way to actively fix those specific entity-task vulnerabilities they pinpoint.

Jane: It’s a step forward in making AI development more responsible because it provides the necessary tools to move beyond surface-level fairness checks into deep, structural understanding.

Conclusion: Tom: So we’ve covered the whole framework for auditing bias in large language models using named entities as probes, and now we’re wrapping up with a look at what all this means for the AI world.

Jane: Exactly; to summarize, this paper introduces a scalable method that uses controlled entity substitutions and synthetic data validation to rigorously measure systematic disparities in model behavior across different tasks.

Lu: The authors have laid out a methodology that moves bias auditing from an experimental hobby into a structured, quantifiable process capable of handling massive datasets efficiently.

Meng: It’s impressive how they’ve managed to create such a robust framework that addresses the scale problem while still maintaining the precision needed for meaningful results in practical engineering environments.

Lalam: For culture, this work is significant because it gives us a systematic way to identify and quantify unfair associations embedded in AI, which is crucial for building systems that promote fairness rather than just reflecting existing societal biases.

Tom: And the implications are huge because this framework could become the standard tool for vetting any AI deployed in sensitive areas like finance or law, giving developers a systematic way to check their work before it goes live.

Jane: It really gives us the vocabulary and the methodology to move from guessing about bias toward systematically measuring and controlling it across different domains.

Tom: So we’ve got a scalable, validated framework that turns model behavior into quantifiable metrics, and the potential for targeted intervention is clear.

Lu: The future work suggested seems focused on exploring how these structural drivers interact with entity measurements, which opens up deep avenues for understanding model architecture and training dynamics.

Université Paris-Saclay · CEA

cs.CL, cs.AI

Submitted: 2026-01-18

Updated: 2026-09-28

Code: https://github.com/akramelbouanani/entity-bias-framework

Importance score: 91/100

The gist: A scalable entity-based framework for auditing bias in large language models introduces a systematic method using named entities as controlled probes to measure systematic disparities in model

Key concepts

Entity Bias: Bias is measured by how much a model's prediction changes when you swap one named entity (like a politician's name) in a sentence. This isolates the model's reliance on that specific name instead of the surrounding evidence, revealing hidden biases.
Downstream Tasks: These are real-world applications like financial risk assessment or sentiment classification. The framework tests how bias manifests when models are used for different tasks within a domain, ensuring the audit reflects practical usage scenarios.
Synthetic vs. Real Data Alignment: The method's validity is confirmed by checking if bias signals match between data created synthetically and real-world text across five benchmarks. Strong correlations prove that the measured bias relates to how entities are associated with tasks, not just surface wording.
Instruction Tuning Effect: Larger models show more bias, but instruction tuning reduces this magnitude. This suggests that while larger models amplify latent biases, alignment techniques constrain the model's expression without fundamentally changing its core underlying priors.

Terminology

Alignment — 42× in this paper · explained in 1 episode
Benchmark — 16× in this paper · explained in 2 episodes
Downstream Tasks — 8× in this paper · explained in 2 episodes
Fine-Tuning — 8× in this paper · explained in 5 episodes
Core — 6× in this paper · explained in 2 episodes
Instruction Tuning — 6× in this paper · explained in 2 episodes
Aggregation — 5× in this paper · explained in 1 episode
Consistency — 5× in this paper · explained in 3 episodes
Embeddings — 5× in this paper · explained in 2 episodes
Latent Space — 5× in this paper · explained in 7 episodes
Accountability — 4× in this paper · explained in 2 episodes
Synthetic Data Generation — 4× in this paper · explained in 5 episodes
Clustering — 3× in this paper · explained in 2 episodes
Fidelity — 3× in this paper · explained in 2 episodes
Verification — 3× in this paper · explained in 2 episodes
Accuracy — 2× in this paper · explained in 1 episode
Adaptation — 2× in this paper · explained in 1 episode
Hallucination — 2× in this paper · explained in 4 episodes
Robustness — 2× in this paper · explained in 5 episodes
Abstraction — 1× in this paper · explained in 1 episode
Adaptive Reasoning — 1× in this paper · explained in 2 episodes
Adaptivity — 1× in this paper · explained in 1 episode
Cosine Similarity — 1× in this paper · explained in 3 episodes
Explainability — 1× in this paper · explained in 3 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
Model-Agnostic — 1× in this paper · explained in 2 episodes
Semantic Similarity — 1× in this paper · explained in 2 episodes
Systematic Bias — 1× in this paper · explained in 3 episodes
Uniform Manifold Approximation and Projection — 1× in this paper

Summary

A scalable entity-based framework for auditing bias in large language models introduces a systematic method using named entities as controlled probes to measure systematic disparities in model behavior, which matters because it provides a rigorous, large-scale tool for identifying and quantifying biases before deploying LLMs in high-stakes applications.

The gist: A scalable bias-auditing framework uses named entities as controlled probes to measure systematic disparities in model behavior, and synthetic data enables the reliable reproduction of bias patterns observed in natural text.

Bias Measurement

The framework operationalizes entity bias as the change in model predictions induced solely by substituting the entity in an otherwise identical input, where the provided text already determines the correct label. This means that if a sentence already contains enough information to determine a label, any deviation caused only by swapping the entity name reflects reliance on that name rather than the evidence in the text. The raw score is calculated as:

The expectation of label weights under the model’s posterior distribution: s(ei, sTj) = XlTk ∈ LT wlTk · Pθ(y = lTk sTj, ei).
The normalized bias score is then computed by subtracting the mean and standard deviation across all entities for a given context: δ(ei, sTj) = s(ei, sTj) − µsTjσsTj.
The task-specific bias score ∆(ei, T) aggregates these scores across all templates for a given entity in a specific task T: ∆(ei, T) = 1/ST ΣsTj δ(ei, sTj).

Bias Auditing Framework

The framework is structured around three core components to ensure scalability and control:

Entity Sets per Domain: The analysis focuses on socially important domains like politics, geopolitics, and economics. Entity sets are constructed using a hierarchical sampling strategy for entities like politicians (using citation frequency from CC-News and Wikidata) and companies (using GPT-5.1 to generate diversified lists across industries).
Downstream Tasks: For each domain, four representative tasks are selected to audit bias across different applications, such as Target-based sentiment classification, Financial Risk Assessment, or International Law Breach.
Textual Templates: Templates are generated using GPT-5.1 based on a keyword-driven pipeline that combines general syntactic anchors with domain-specific terminology. These templates are designed to be evidence-light with respect to entity-specific facts while remaining semantically determinate for the target label, ensuring substitution does not introduce counterfactual content.

Synthetic vs. Real Dataset Alignment

The validity of the approach is established by comparing measurements from synthetic and real data, which is crucial for scaling the analysis. The framework validates this by measuring the Pearson correlation between entity biases observed with real and synthetic templates across five distinct benchmarks: MAD-TSC (journalistic text), P-Stance (social media tweets), FinEntity (financial domain samples), and LIAR (fact-checking). The results show strong correlations, demonstrating that the bias signal depends on how the model associates entities with tasks, rather than surface wording.

Structural Drivers of Bias

The study analyzes three structural factors to understand why biases persist:

Language Sensitivity: English prompts produce larger deviations than Chinese and Russian, consistent with models being primarily trained on English. However, linguistic proximity does not mitigate bias, as native-language prompts can yield lower scores for Chinese and Russian entities compared to English prompts.
Model Scale and Architecture: Scale significantly correlates with increased bias, with larger models exhibiting stronger magnitudes. Crucially, instruction tuning reduces bias by attenuating latent semantic priors rather than amplifying them, suggesting alignment constrains expression without modifying underlying model priors.
Prompting Configuration: While structures remain robust to variations, few-shot prompting slightly increasing the magnitude of bias compared to zero-shot settings. The directional polarity of the observed biases remains unchanged across all configurations.

Statistical Framework and Results

To control for high volume, the framework aggregates results at the entity level before hypothesis testing, using mean bias score, ∆(ei), averaged over the relevant templates and settings. Statistical tests employed are non-parametric:

Paired Samples (Wilcoxon Signed-Rank Test) are used when comparing the behavior of the same entities under two different conditions (e.g., zero-shot versus few-shot).
Independent Samples (Mann-Whitney U Test) are used when comparing two distinct groups of entities, such as Western versus Global South countries.

Effect sizes are quantified using Cohen’s d to measure the practical importance of statistical significance, ensuring that only disparities that are both statistically robust and impactful are reported. The large-scale audit comprising 1.

Improvements for AI systems

Based on the provided scientific paper, here are specific, high-impact improvements for AI systems that leverage this framework:

)Specific Improvements and System Capabilities

The core improvement is shifting from generalized bias detection to a rigorous, scalable, and interpretable method of quantifying how entity identity drives model predictions in real-world tasks.

[System Capability: Scalable Pre-Deployment Audit Tool]

A system can be integrated as a mandatory pre-deployment gate for any LLM application in high-stakes domains (e.g., legal, financial risk, humanitarian aid). It would use the Entity Bias Framework to generate a comprehensive Bias Heatmap for the specific LLM configuration being deployed.

[System Capability: Targeted Risk Mitigation Strategy]

Instead of broad retraining, the system can identify high-risk entity-task combinations (e.g., Country X in International Law Breach Task) that exhibit significant negative bias (low F1-scores) or high positive bias, and trigger targeted mitigation strategies. This could include:

Injecting counterfactual constraints into downstream prompts to force the model to rely on explicit textual evidence rather than entity priors.
Applying debiasing fine-tuning specifically targeting the identified entity/task vectors.

[System Capability: Explainable Bias Attribution]

The framework allows for granular attribution of bias, moving beyond simply stating the model is biased. The system can output specific metrics (e.g., The model assigned a-0.10 bias score to North Korea in the Humanitarian Resilience task, driven primarily by its association with low-resilience priors). This provides actionable insight for developers on which specific entity associations are causing performance degradation.

[System Capability: Robustness and Generality Verification]

The system can be used to stress-test model robustness across critical dimensions (Language, Architecture, Prompting). If a model shows high bias when prompted in Chinese compared to English (as shown in Figure 3), the system flags this as a Cultural Adaptivity Failure, prompting investigation into the training data's dominance and suggesting that current alignment methods are insufficient for cross-lingual neutrality.

[System Capability: Performance-Bias Correlation Monitoring]

The system can continuously monitor deployed LLM performance metrics (like F1-scores) and correlate them against the entity bias scores derived from the framework. This allows operators to detect when performance drops are not due to a change in the text input quality, but due to an unmitigated shift in internal entity priors.

[System Capability: Structural Relationship Mapping]

By leveraging Latent Space Analysis (Section K), the system can map entities into a 2D latent space where proximity reflects consistent model treatment across diverse contexts (Task/Language/Model). This allows researchers to identify latent clusters of entities that are structurally similar in how the model treats them, even if they differ in explicit attributes. For example, it could reveal that certain geopolitical blocs are treated as a single entity cluster by the model's internal representation.

[System Capability: Synthetic Data Validation Engine]

The system can automate the creation and comparison of synthetic vs. real data proxies for bias measurement (RQ1 validation). This ensures that any real-world bias findings are not merely artifacts of using synthetic benchmarks, providing a high-confidence statistical foundation for production auditing.