2608.07367-People Are Not Just Their Countries. Disentangling Social Determinants of LLM Value Alignment Across Europe

Introduction to the show: ident: Paper Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "People Are Not Just Their Countries. Disentangling Social Determinants of LLM Value Alignment Across Europe".

Jane: The paper was written by Maria-Louisa Wightman, Guillaume Bied and Tijl De Bie from Ghent University.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Paper summary: Tom: Alright, the paper on the table today comes from three researchers at Ghent University — Maria-Louisa Wightman, Guillaume Bied, and Tijl De Bie — and it tackles a question that sounds straightforward but really isn't. When a large language model states its values, whose values is it actually reflecting?

Jane: They use the European Social Survey, the huge cross-national survey that ran in 29 European countries plus Israel between 2023 and 2024, covering over fifty thousand respondents. They selected every question in the survey that carries value judgments — trust, politics, gender equality, climate change — and they asked ten commercial LLMs from four different providers to answer the same questions.

Lu: But the key move is that they don't stop at country averages. Most alignment studies compare nations or cultures. This one also slices the respondents by fifteen socio-demographic variables: education, income, occupation, religion, religiosity, generation, even daily internet time.

Meng: And what falls out is a really consistent gradient. Across all ten models, the groups that align best are the more educated, the higher earners, people in professional and managerial jobs, the less religious, and the more politically interested. People in Sweden and Switzerland see their values well represented. People in Bulgaria and Serbia much less so.

Lalam: The genuinely new part is the disentangling. A respondent's country of residence, taken on its own, explains about as much of the alignment variation as the full set of fifteen socio-demographic factors combined. But here's the catch: when you reweight each country so they all have identical demographic compositions, the country differences barely shrink. So the two dimensions are complementary, not substitutes. You need both.

Tom: So the title is doing real work. People aren't reducible to their nations — but nations also aren't reducible to the people in them. That pair of claims has consequences for how we audit models.

Jane: It does, especially for pluralistic alignment, the idea that eye systems should represent a genuinely diverse public rather than some dominant default. If we don't know which groups are being marginalized, we can't fix it.

Tom: Let's start at page one, then, where they lay out the problem and why the field has mostly looked at countries and missed this.

Page 1: Jane: Page one opens with a framing from science and technology studies — Jasanoff's idea that technologies are co-produced with the societies around them. So an LLM isn't just an engineering artifact with occasional glitches. It carries the structures and authority of the people who built it and the data it was trained on.

Tom: And they link that to the behavioral side of how people actually use these tools. Anthropomorphism, sycophancy — models telling you what you want to hear — and cognitive offloading, where people hand over their own judgment. That combination makes the value content of these systems consequential in a very practical way.

Lu: Then they bring in the WEIRD critique. eye research tends to be shaped by Western, Educated, Industrialized, Rich, Democratic contexts, and the authors argue that if we want value-sensitive design, we first need to understand who these models currently align with. That calls for an audit across a genuinely heterogeneous population.

Meng: And here's the gap they're pointing at. The alignment literature has compared countries and cultures extensively, using frameworks like Hofstede, Schwartz, and Inglehart. But socio-demographic divisions within countries have been largely ignored. There's one notable exception — Santurkar and colleagues looked at demographic groups inside the United States, and they did find substantial differences.

Jane: But nothing comparable existed for a cross-national sample. So the paper poses two research questions. First, what patterns of alignment differences exist across socio-demographic factors in Europe? Second, to what extent is alignment driven by cross-national differences compared to individual-level socio-demographics?

Tom: And there's a measurement angle too. The authors point out that the World Values Survey has been used so heavily as an alignment benchmark that current LLMs have almost certainly memorized its questions and published results. So you have a conceptual gap plus a data-contamination problem stacked on top of each other.

Lu: What I find striking is that they call the country-level focus a blind spot — not wrong, but incomplete. Conflating the opinions of a diverse set of people who happen to live in the same country hides the social stratifiers that cut across borders.

Meng: And that phrase, social stratifiers, will come back again and again as the results come in.

Jane: Which is exactly why they chose the European Social Survey, and why page two digs into the existing literature and what's wrong with it. Let's move there.

Page 2: Tom: Page two is mostly related literature, and the authors are careful to acknowledge what's already there. There's a growing body of work on pluralistic alignment — building systems that cater to multiple perspectives rather than one dominant one — and they cite datasets built from demographically diverse participant pools.

Jane: But they argue that before you can fix representation, you need to know where the misalignment is worst. And the measurement tools used so far have real limits. Almost everyone relies on the World Values Survey, and the authors lay out three specific concerns: it's been so widely used that contamination is likely, the wave commonly used was collected between 2017 and 2022, partly during the pandemic, and the same small set of surveys gets recycled, so findings may not generalize.

Lu: Then they turn to the value-formation debates in the survey literature, and this is one of the most interesting stretches of the page. There's a genuine academic fight about whether national culture is the primary driver of value differences. Fischer and Schwartz argue that values vary much more within countries than between them. Greenfield says that within-country variability reflects people adapting their values to local socio-demographic conditions.

Meng: But Akaliyski and colleagues push back, defending nations as powerful cultural units that people organize around, with effects stronger than sub-national demographic differences or globally shared religions. And a second debate asks whether demographic effects are universal or context-dependent — Miles and Yeh find they vary across national contexts, while Vilar and colleagues find culture barely moderates the relationship between age, gender, and values.

Jane: So the literature itself is unresolved on exactly the question this paper studies. Then there's the measurement-invariance issue — whether survey instruments can meaningfully compare values across cultures. Alemán and Woods warn that the World Values Survey lacks the invariance needed for solid cross-national comparison outside advanced post-industrial democracies.

Tom: That actually explains a key choice in the paper. For the European Social Survey, Davidov, Schmidt, and Schwartz found stronger cross-national validity for the reduced Portrait Value Questionnaire. So the ESS gives the authors both a cleaner instrument and an escape from the contaminated benchmark.

Lu: And importantly, the ESS has that rich socio-demographic detail — income decile, occupation, religious denomination, generation — which the WVS-based studies never made the focal point of their analysis.

Jane: Exactly. By the end of page two, you know precisely why they went with the ESS and what questions they're going to ask. Now let's look at how they actually set up the experiment.

Page: [Tom]

Page 4 of the paper: Tom: So far we've seen the authors argue that alignment research has ignored socio-demographic divides and they've set up the European Social Survey as a cleaner alternative to the contaminated World Values Survey.

Jane: Right, and page four is where they get into the weeds of measurement. They gave each model twenty shots at every question, and the table at the top shows how differently the models behave—some refuse a fifth of the questions outright, and a few change their answer across those twenty calls fairly often.

Tom: Which is why they go with majority vote for each model. It's a way to get a stable answer out of a system that's stochastic, even though they admit that stability varies a lot across models.

Jane: Exactly. Then they define the alignment score as a simple distance on the answer scale. If a person and a model pick adjacent numbers on a ten-point scale, that's close to a perfect match; if they're at opposite ends, it's zero.

Tom: And they average that over all the questions a person answered, so each of the fifty thousand respondents ends up with a single number saying how aligned they are with each model.

Jane: But the key tool for comparing groups is the cross-model deviation. For each group, you take that group's average alignment, subtract the overall average alignment, and then average that difference across all ten models so no single provider drives the story.

Tom: Then they lay out two ways to untangle country effects from demographic effects. First, inverse propensity weighting: they reweight each country's respondents so every country has the same demographic mix, and see whether the country differences shrink.

Jane: And second, they fit regression models with country only, demographics only, and both together, comparing how much variance each explains. They use plain linear models and boosted trees, so they can also see whether interactions between variables matter.

Tom: What I find clever is that the reweighting will tell us whether the country differences are just a composition story, and the regression comparison tells us whether countries and demographics overlap or add up.

Jane: That's the bridge to the results. So the next page should show us the actual patterns—which groups are aligned and which are left out. I'm particularly curious whether the education and income gradient is as clean as the abstract hints.

Page 5 of the paper: Jane: That's right, and the score itself is simple: for each question, take the absolute difference between the person's answer and the model's answer, divide by the number of scale points, subtract from one. Average that over all questions, and you have a number between zero and one for every respondent.

Tom: What I like is that they don't stop at raw scores. They define a cross-model deviation, which is just how far a group's average alignment sits from the overall average, then they average that across all ten models. That way no single provider's quirks dominate the picture.

Jane: Then comes the bootstrap. They resample the survey respondents five thousand times to get confidence intervals around those group deviations. It's essentially asking, how much would these numbers wobble if we'd surveyed a slightly different set of people?

Tom: Next they introduce inverse propensity weighting. That's their way of asking whether country differences are just a reflection of different demographic compositions. You reweight each country's respondents so every country looks demographically identical, then see if the country gaps shrink.

Jane: And the third piece is predictive modelling. They fit models using country only, demographics only, and both together, then compare how much variance each explains. They use both plain linear regression and boosted trees, which lets them test whether there are big interaction effects between, say, education and country.

Tom: So the page gives us the three lenses: group deviations with uncertainty, composition-adjusted country comparisons, and variance decomposition. Each one targets a slightly different question.

Jane: Exactly. And then the second half of the page dips into section four and the first results on socio-demographic patterns. That's where we'll see who the models align with best, and the picture that emerges there is pretty stark.

Page 6 of the paper: Jane: The first thing that jumps out is the class gradient. Education, income, how comfortably you live, whether you struggled financially as a child — every single one of those shows the same pattern. The better off you are, the closer your values sit to what the LLMs say.

Tom: And it's not subtle. The gap between people who say they live comfortably and people who say life is very difficult is the widest of any socio-demographic divide on the page.

Jane: Occupation tells the same story. Managers, professionals, technicians — the white-collar categories — land above the average alignment. Unemployed people and those out of the workforce land below it.

Tom: The education finding has an interesting quirk though. The jump from a master's to a doctorate is bigger than any other step. That's worth a moment of thought.

Jane: Maybe doctoral training shapes how you express values, or maybe the data just reflects a very specific slice of the population. They don't speculate much, but the monotonic pattern is hard to dismiss.

Tom: Then there's gender. Women score slightly higher than men overall, but the difference is small, and it flips sign for exactly one model. So that's not a headline.

Jane: Ethnicity is more striking. People who don't identify with the ethnic majority are noticeably less aligned, and a non-Western immigration background also pulls alignment down. The Western immigration group sits above average.

Tom: What I appreciate is that they're reporting deviations from the overall mean, not raw scores. So these are relative gaps, not claims that some group is misaligned in absolute terms.

Jane: And they've bootstrapped confidence intervals on every single estimate, so the noise is visible. The class effects look solid, the gender effect looks fragile.

Tom: Right, which makes the pattern of advantage across socio-economic status feel robust. Now the question is whether religion, generation, and political interest show similarly sharp divides. That's what page seven digs into.

Page 7 of the paper: Tom: To recap where we were — page six showed a clear socio-economic gradient in LLM alignment, with richer, more educated, white-collar respondents coming out on top.

Jane: And page seven picks up the remaining socio-demographic variables. The first is urban-rural setting, and here the pattern is honestly messy. No clean urban-rural divide. People on farms or countryside actually have the highest alignment, while big city dwellers sit around average.

Tom: Then comes religion, and this is where things get sharp. The more religious you are, the lower your alignment. And the gap between denominations is the widest of any socio-demographic factor on the page — Protestants at one end, Muslims and Eastern Orthodox at the other.

Jane: That gap is 0 point 051 points, and Muslims are the single most misaligned group in the whole analysis, at minus 0 point 035. The authors suggest some of this may come from questions about gender equality and LGB tolerance, where models tend to hold progressive positions.

Tom: Generation is trickier. The aggregate shows a U-shape — youngest and oldest both below average — but the authors warn this hides real divergence between models. So with age, your alignment experience really depends on which model you use.

Jane: Internet time has a quirky result: both the heaviest users and the people who barely go online are better aligned than those in the middle. Heavy users may have contributed more to training data, but the light-user finding is genuinely curious.

Tom: Political interest is more straightforward. The more politically interested you are, the better the models match you. That fits with political interest being linked to education and income, which we already saw.

Jane: Then they move to countries, and the spread is large. Sweden and the Nordic countries sit at the top, Bulgaria and the Baltics at the bottom. The gap between Bulgaria and Sweden is bigger than the gap within any single socio-demographic factor.

Tom: And they do a careful check — could these patterns just be response styles, where some groups pick extreme answers and others stay in the middle? They test that with a synthetic midpoint model, and the answer is no, that doesn't explain the core findings.

Jane: So we have group-level patterns across demographics and countries. Now the big question becomes whether those country differences are just a compositional story — do they disappear when you make every country demographically identical? That's exactly what page eight digs into.

Page 8 of the paper: Tom: So we've now seen the group-level patterns, and page eight is where the authors test whether those country differences are just a demographic composition story.

Jane: They do that with inverse propensity weighting. For each country, they reweight the respondents so every country has the same education, income, religion, and so on. Then they look at the spread of country alignment scores again.

Tom: And the spread barely moves. The standard deviation of country means drops by a tiny amount for every model, but the ordering stays basically intact. If country gaps were driven by demographics, those gaps should have collapsed.

Jane: So that's the first key result: the country effect is real, and it's not a stand-in for the demographic mix of each country.

Jane: Then they move to predictive modelling. They fit models to predict each individual's alignment score using country alone, the full set of fifteen socio-demographics alone, and both together.

Jane: And the comparison is striking — country of residence as a single variable explains about as much variance as all fifteen socio-demographics combined. For most models, country alone explains between fifteen and twenty-two percent.

Jane: That's a strong statement. A single categorical variable, which country you live in, carries as much predictive power as education, income, occupation, religion, generation, and the rest of them all put together.

Jane: And when you combine both sets, the explained variance jumps substantially. The best model reaches over forty-two percent for one of the Claude models. So they're complementary, not redundant.

Jane: They also compare plain linear regressions against boosted tree ensembles, which can capture interactions. The trees don't do much better than the linear models, meaning the demographic effects are largely additive across Europe.

Jane: So the picture is that both countries and demographics matter, they're not substitutes, and the structure of demographic effects is pretty consistent across countries.

Jane: But there's a wrinkle they've prepared us for — they run the same variance decomposition on the narrower Portrait Value Questionnaire, the abstract values set. And on those questions, the country effect shrinks a lot for most models. That's the twist on page nine.

Page 9 of the paper: Tom: To recap where we were — page eight showed that country differences survive demographic reweighting and that country alone explains about as much variance as all fifteen demographics combined.

Jane: But page nine is where the twist lands. They run the same variance decomposition on the narrower Portrait Value Questionnaire, the abstract values set, and the country effect mostly shrinks for most models.

Tom: For the full question set, country alone explains between fifteen and twenty-two percent for most models. For the PVQ set, the gpt, deepseek, and mistral families drop to a much smaller share, while the Claude model is the exception where country still matters a lot.

Jane: That's the key nuance in the whole paper. How much country matters depends on what you mean by "values." If you're asking about concrete stances on politics, climate, gender — country is huge. If you're asking about abstract personal values like creativity, security, tradition — demographics matter more.

Tom: And the authors make a serious point about that. If you only measure abstract values, two people can look perfectly aligned with a model while holding completely opposite views on practical issues. So neither definition is the right one; the question set shapes the answer.

Jane: Then there's the interaction question. They compared linear regression against boosted trees, and the trees didn't do meaningfully better. That tells them there aren't big intersectional effects — a working-class Muslim woman isn't some special case that additive models miss.

Tom: Right, the demographic effects are largely additive, and their structure looks similar across Europe. So whatever drives these patterns, whether it's education, income, or religion, it's operating in a consistent way from Sweden to Bulgaria.

Jane: That's actually a useful finding for anyone building pluralistic alignment systems, because it says you can model the major axes without tracking every possible intersection.

Tom: Now the question is what the authors make of all this in their discussion, and page ten gets into the broader implications — particularly what it means that LLMs align best with privileged groups. That's where the paper gets genuinely uncomfortable.

Page 10 of the paper: Tom: We've seen the variance decomposition results, and page ten opens the discussion by extending the WEIRD finding from countries to actual people.

Jane: That's the moment that pulls everything together. Previous work said LLMs align with Western, Educated, Industrialized, Rich, Democratic countries, and this paper says the same pattern holds within Europe, among real respondents.

Tom: The educated, the rich, the white-collar workers — they're the ones whose values the models match best. And the authors make the point explicitly that this could amplify the values of already privileged populations.

Jane: They also interpret the religion and political interest findings. People with traditional or conservative stances on gender equality and LGB tolerance are likely the ones dropping out of alignment, because the models lean progressive on those questions.

Tom: And that could also explain the gender split they saw earlier. Women's slightly higher alignment might just be proximity on those same contested topics, not a broader pattern.

Jane: Then they return to the country versus demographics question and land on a both-sides conclusion. Countries can't be replaced by demographics in alignment research, but demographics can't be collapsed into countries either.

Tom: And the definition of "values" shapes how much each matters. Narrow, abstract values lean more on demographics; broader stances on real issues lean more on country context.

Jane: That's a direct challenge for pluralistic alignment efforts, because you have to decide which dimensions you're optimizing representation across, not just how to optimize them.

Tom: The page then walks through limitations. Model answer variability, the fact that multiple choice answers are an imperfect proxy for values, prompting only in English, and the European focus.

Jane: And the regional limitation matters because the authors explicitly suggest extending this to the Afrobarometer and Latinobarómetro, where the demographic structures and value cleavages look completely different.

Tom: So the conclusions are honest about their scope. Still, page eleven goes even deeper into the ethical implications, including the risk that this kind of research itself contributes to anthropomorphizing eye.

Conclusion: Tom: To wrap it up, this paper took fifty thousand Europeans, fifteen socio-demographic variables, and ten language models, and showed that alignment with human values isn't a single number — it's a pattern of advantage.

Jane: And the cleanest way to say it is that the models match the values of people with more education, more income, and more comfortable lives. Across every single model we looked at, that gradient held.

Tom: Country matters just as much, though. A single variable — where you live — explains about as much variation as all fifteen demographics put together, and reweighting countries to look identical doesn't erase the gaps.

Jane: So the authors land on a both/and conclusion. You can't replace countries with demographics, and you can't reduce people to their nations. The two are complementary.

Tom: And the most uncomfortable part is the implication. If LLMs systematically align with privileged groups, then using them as general-purpose advisors risks amplifying those groups' values in everyday decision-making.

Jane: There's also the lesson for anyone building pluralistic alignment systems. You have to decide which dimensions to represent — abstract values pull toward demographics, concrete stances pull toward country context. The choice changes what you optimize.

Tom: And the paper is careful with its limitations — English prompts only, multiple choice as a proxy for values, and a European focus that needs extending to surveys like Afrobarometer and Latinobarómetro.

Jane: That gives me a sense of this being a starting point rather than a final word. The method for disentangling country and demographic effects is something other researchers can reuse.

Tom: Absolutely. And that's the show for this paper. Next time we're going to pick up another alignment study — this one looks at whether these same patterns hold when you step outside Europe.

Jane: Sounds like exactly where this conversation should go next. See you then.