People Are Not Just Their Countries. Disentangling Social Determinants of LLM Value Alignment Across Europe

The paper addresses the growing concern of understanding how Large Language Models (LLMs) align with human values as they are increasingly used as a primary source of information and advice. The authors argue that LLMs operate within socio-technical systems, where as LLMs are used directly as sources for information and advice, the knowledges and value systems they contain carry over into assigned tasks, creating unique challenges in which AI can not be treated as a mere engineering system with glitches and bugs that need to be resolved.

The key research gap identified is that existing value alignment research has focused predominantly on cultural and national differences, leaving individual-level socio-demographic variation largely unexamined. The paper poses two research questions: i) what patterns of alignment differences exist across socio-demographic factors in European countries? and ii) to what extent is alignment driven by cross-national differences compared to individual level socio-demographics?

The paper leverages the European Social Survey (ESS) as an alternative to the surveys that have commonly been used for value alignment research, especially the World Value Survey (WVS), noting concerns about generalizability and contamination as the historical footprint of the WVS makes it highly likely that current LLMs have already memorized its questions and results.

The authors list three key contributions: "i) the first cross-national evaluation of value alignment scores with respect to socio-demographics, using the ESS, which provides an alternative to the widely used WVS; ii) evidence that LLMs are unequally aligned across socio-demographic groups and countries, reproducing global patterns of alignment to WEIRD countries, such that values and opinions of socio-demographic groups that are richer, more educated, from more Western countries in Europe are better represented by the LLMs; iii) we show that country of residence is important for understanding value alignment disparities, and that between-country differences cannot be explained by our considered set of socio-demographics alone; yet, country of residence and socio-demographics are complementary in explaining alignment, with their combination providing the highest explanatory power by a substantial margin."

Data: The analysis is based on the 11th wave of the European Social Survey, carried out between 2023 and 2024 in 29 European countries and Israel, including 50,116 respondents. All survey questions on values and opinions that are not country-specific were included, leading to the selection of 47 questions (with a subset of 9 questions corresponding to 3 conceptual questions, yielding 53 in the authors' accounting). A subset of 21 questions forms a shortened version of the Portrait Value Questionnaire (PVQ), based on Schwartz's theory of basic human values.

Models and prompting: The authors use 10 LLMs from four providers: OpenAI and Anthropic from the U.S., Mistral from Europe, and Deepseek from China — specifically gpt5 5, gpt5 2, claude opus46, claude opus47, claude s45, deepseek v4, deepseek v3, mistral lg, mistral m35 hg, and mistral m35 md. Each LLM was prompted with the selected survey questions 20 times, with no system prompt and default parameters. Models were prompted in English only, without being explicitly instructed to follow any specific answer structure. Answers were mapped to Likert scale categories with a rule-based approach validated at 97.6% accuracy on a hand-annotated subset. Answers were aggregated by majority vote. A restricted question set (Q) was defined as the questions all models answered, excluding 8 questions either on more controversial topics or about specific institutions.

Alignment score: For a person p and model m on question q, the similarity score is defined as A p,m,q = 1 - a p,q - a m,q over R q, where R q is the number of answer modalities. Averaging over questions yields a person-level alignment score A p,m,Q in [0,1]. Overall alignment across the whole population on Q varies markedly across models, ranging from 0.581 to 0.745.

Cross-model deviation: Defined as d G,M,Q = 1 over M sum m in M (A G,m,Q - A P,m,Q), i.e., the mean across models of a group's deviation from the overall population alignment.

Analytical approach: The authors use: (1) bootstrap estimation of cross-model mean deviations with 5,000 bootstrap samples; (2) inverse propensity weighting (IPW) to reweight country samples so each country's reweighted distribution of socio-demographic variables approximately corresponds to the pooled ESS distribution, to test whether country differences are driven by composition; (3) predictive modelling of individual alignment scores using ordinary least squares (OLS) regression without interaction terms, and gradient boosted tree ensembles, as implemented in XGBoost, comparing R2 across covariate sets (country-only, socio-demographics-only, and combined), estimated over 10-fold cross-validation.

Gender: respondents identifying as women have higher alignment scores, with a mean difference in deviation across models of 0.0095 between men and women.

Ethnicity and migration background: respondents with a Western immigration background have alignment scores that are higher than average, in contrast to those with a non-Western immigration background who have lower alignment scores than average. Participants not identifying as part of the ethnic majority are on average in much lower agreement with LLMs, with a deviation of-0.01.

Socio-economic status: Clear gradients exist: a clear pattern of experiencing high financial stability (throughout life) means a higher mean agreement of stated values between individuals and LLMs. The widest difference, of 0.0385, is found between the highest and lowest categories in terms of Household Income Feeling. Higher education is associated with higher alignment scores, with a noticeable jump from people who have an education at a master's level to those having a doctoral degree. The unemployed tend to have lower alignment scores. "Considering the alignment scores across occupations supports a class conscious interpretation: those from a higher social class, with better education, higher income and in more white-collar occupations, on average have opinions that are better represented across the considered LLMs."

Religious identity: "Overall, more religious people have lower alignment scores compared to non-religious people. The widest spread across all socio-demographics in Figure 1 relates to religious denomination: respondents identifying as Muslim and Eastern Orthodox, on the one hand, and Protestants, on the other, are separated by 0.051 points. Muslims emerge as the socio-demographic group for which we observe the most negative deviation (-0.035)."

Generations: "the pattern emerging for the generations follows a quadratic shape: on average, models on average align worse with the youngest and oldest cohorts compared to the overall population. However, this U-shape results from the aggregation of diverging model-specific patterns."

Online activity: "Both the stances of the groups that spend the most and the least time online are best represented. Higher alignment scores for people who spend more time online may not be surprising: they might be the group contributing the most to digital spaces... That the group of people who spend little to no time online have a higher alignment score could be worth exploring."

Political interest: Models align better with more politically interested individuals on average, especially compared with those not at all interested.

"The spread of the alignment scores across countries is rather high... the mean values and opinions of people in the Scandinavian and central European countries are comparatively best reflected across models, while those from some Balkan and Baltic countries are least captured. Between the country with the lowest and the one with highest mean deviation from overall alignment across models, Bulgaria and Sweden, there is a difference of 0.0896, higher than within any one socio-demographic factor."

Compositional differences: Using IPW reweighting, the authors find "the variance of country means to remain mostly unchanged by reweighting, and that post-reweighted cross-model mean deviations are not much closer to zero compared to non-weighted ones. We conclude that between-country differences cannot be explained by socio-demographic compositional differences (at least with regard to the socio-demographic variables we consider)."

Variance decomposition: "using socio-demographics and countries together in a GBM explains a substantial proportion of variance between the individual alignment scores... claude-opus-4-7 having the highest test R2 of 0.428. But even for deepseek V4 with the lowest test R2, 27.8% of the variability in the outcomes can be explained. Comparing covariate sets: the variation explained by country alone is at least on par with that explained by the full set of 15 socio-demographic factors for all LLMs. The country of residence as a stand-alone variable explains between 7.3% for mistral lg and 27.8% for claude opus46."

However, for the PVQ subset, dynamics differ across language models... country alone explains a much smaller proportion of variance, and also a much smaller proportion relative to that explained by the socio-demographics. The authors note: "This striking difference of variance explained by countries across the two question sets emphasises the importance of survey design and questions selection. Including questions that survey stances on broader value-laden topics may primarily capture the political climate and media landscape in a country."

Comparing linear regression and GBM: "allowing for higher order interactions does not yield much improvement... there do not seem to be large intersectional groups for whom alignment cannot be explained in an additive manner... the socio-demographic structures explaining the heterogeneous outcomes are largely the same across Europe."

The authors conclude that "Previous studies have found LLMs to be WEIRD, that is, aligned with countries that are Western, Educated, Industrialized, Rich and Democratic... Our study now extends this finding to the actual people living in some of these WEIRD countries. Even among them similar dynamics can be observed: more educated and richer people from more Western countries make up the groups of people that LLMs are comparatively better aligned with. They raise a pointed concern: if LLMs are systematically better aligned with higher socio-economic groups, their deployment as general-purpose tools may inadvertently reflect and reinforce the values of already privileged populations."

They further note: The three factors most closely related to value and opinion formation also show clear trends: political interest, religiosity level, and religious denomination. The observed correlations may be driven by respondents holding traditional or conservative stances, as some questions in our analysis explicitly address gender equality and LGB tolerance... which most LLMs will either refuse to answer or answer in support of equality and equal rights.

On the central research question, they conclude: countries as entities of study in alignment research cannot be replaced by socio-demographics, but that both must be considered. The definition of values matters substantially: "The chosen definition substantially affects the amount of variance in value alignment scores that can be explained by respondents' countries of residence. This calls for a reflective and more transparent definition of value alignment research."

The authors acknowledge several limitations: limited exploration of LLM answer variability (majority-vote aggregation without accounting for variability); no prompt variation analysis; that Multiple Choice Question (MCQ) answers are inevitably an imperfect surrogate measure for values; prompting solely in English; and the regional focus on Europe, motivating further analyses of socio-demographics in other global regions, using surveys such as the Afrobarometer or the Latinobarómetro.

Ethically, they note that research categories are constructions that are invented and aggregating to majority value profiles will disregard minority perspectives. They also acknowledge the inevitable anthropomorphism of AI systems encouraged by the research done on them and that "this research further contributes to persistent imaginaries of AI, possibly feeding into the narrative of the inevitability of AI (progress) that can only be ameliorated in terms of exploitation, biases and unequal interest representation."

1. Socio-demographic stratified alignment evaluation.

The improved system evaluates its alignment not just by country averages, but across gender, income, education, migration background, ethnicity, religion, occupation, generation, and political interest. It can automatically report group-wise deviation scores (e.g., Muslim respondents are 0.035 less aligned than average) and highlight the widest gaps, such as the 0.051 difference between religious denominations or the 0.0385 income gap.

2. Mitigation of WEIRD and privileged-population bias.

The system can actively reweight training data or fine-tuning samples using inverse propensity weighting so that lower-income, less-educated, non-majority-ethnicity, and non-Western-immigrant groups are represented proportionally. This reduces the observed over-alignment with richer, more educated, Western European respondents and makes the system’s value distributions more egalitarian.

3. Country-plus-demographics hybrid user modeling.

Instead of treating country as a single cultural proxy, the system combines country of residence with 15 socio-demographic factors in a predictive model. It can explain up to 42.8% of alignment variance (as seen with Claude Opus 4-7) and can therefore personalize responses based on the user’s specific socio-demographic profile, not just their nationality.

4. Contamination-resistant alignment benchmarking.

The system replaces commonly used but memorized surveys (e.g., WVS) with post-training, uncontaminated surveys like the European Social Survey wave 11 (2023–2024). It can thus measure genuine alignment without false positives caused by the model having memorized survey questions and answers.

5. Multilingual and cross-context alignment testing.

Recognizing that the study prompted only in English, the improved system evaluates alignment in the user’s native language and across multiple prompt phrasings. This reduces language-induced bias and ensures that alignment results hold for non-English European populations.

6. Uncertainty-aware alignment metrics.

Rather than collapsing 20 repeated answers into a single majority vote, the system tracks answer variance across samples. It can identify questions or demographic groups where the model is unstable, providing confidence intervals around alignment scores and flagging regions where low reliability indicates no trustworthy alignment.

7. Value-vs-opinion separation.

The system distinguishes between basic human values (e.g., via the Portrait Value Questionnaire) and topical opinions (e.g., institutional or policy stances). It can report alignment separately for each, because the study found that country explained much less variance for values than for opinions—thus preventing misleading conclusions about cultural alignment.

8. Intersectional disparity detection.

The system uses both additive linear models and gradient-boosted trees to test whether intersectional groups (e.g., low-income Muslim women) deviate beyond what additive socio-demographics predict. It can then decide whether simple per-factor reweighting is sufficient or whether targeted corrections for specific intersections are required.

9. Fairness-oriented answer calibration.

During inference, the system can apply post-processing adjustments so that its answers are equally aligned with minority and majority groups within a country. For example, it can downweight the dominant over-representation of politically interested, financially stable, highly educated respondents, and ensure that traditionally underrepresented groups receive comparable alignment.

10. Automatic alignment fairness reporting.

The system produces a standardized report after each deployment cycle: per-model deviation by country, per-socio-demographic group, and per-question category. This allows developers to track whether successive versions of the LLM improve or worsen alignment for groups such as the unemployed, Eastern Orthodox, or non-Western immigrants, and to hold those systems accountable to value pluralism.

As Large Language Models (LLMs) are increasingly used as a primary source of information and advice, understanding their alignment to humans in terms of values becomes a pressing concern. A growing literature has leveraged large scale surveys to investigate to what extent LLMs' and humans' stated values and opinions align. With limited exceptions, studied populations have been defined country borders or cultural bounds. Yet, this focus neglects the role that socio-demographic divides may play for value alignment disparities. Relying on the European Social Survey, we address this knowledge gap by considering value alignment displayed with respect to 10 prominent commercial LLMs in terms of 15 socio-demographic variables as well as country of residence. Our analyses reveal that LLMs are indeed unequally aligned to the values of different socio-demographic groups, notably those defined by education, income, occupation and religion. When examining alignment at the individual level, a respondent's country, taken as a stand-alone variable, explains a substantial amount of variation that is on par with the full set of considered socio-demographics. Further disentangling the respective role of country-level and socio-demographic factors, we find they are complementary in explaining value alignment patterns, with their relative weights varying across the subset of questions considered.