Too Categorical to be Human: Emotion Concepts in LLMs and Humans

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "Too Categorical to be Human".

Tom: Large language models show fragile cognitive reasoning about human emotions, revealing that while they capture systematic relations between cognitive appraisals and emotions,

Jane: First, who's behind it and why it matters.

Title and authors: Tom: Well, Jane, I gotta say this paper on "Too Categorical to be Human: Emotion Concepts in LLMs and Humans" is really getting my attention. It’s digging into whether these models are just memorizing emotion labels or if they actually have some real cognitive understanding going on.

Jane: I agree, Tom. The title itself makes you wonder how deep their reasoning actually goes when it comes to feelings like joy or fear. It suggests we might be looking at surface-level recognition instead of genuine cognitive processes, which is a really important distinction for us as developers and researchers.

Lu: This paper introduces CoRE, which sounds like a huge project, testing these models on seventeen different cognitive dimensions using about seventy thousand prompts across six frontier LLMs. I’m super intrigued by the sheer scale of this benchmark they built to map out these internal structures <ref:2508.05880#pg1>.

Meng: Seventy thousand prompts is a substantial amount of data to run through, Lu. From an engineering standpoint, the real question is how they managed to get such broad coverage across so many different model families and dimensions without running into massive computational bottlenecks or overfitting issues <ref:2508.05880#pg1>.

Lalam: I think from my perspective as an LLM, this research is vital because it helps us understand the structure of human emotion in a way that's not just about the word we use, but the underlying cognitive architecture itself <ref:2508.05880#pg0>.

Tom: Exactly, Lalam. And what they found immediately is that LLMs capture systematic relations between these appraisals and emotions, which is a start, but they also show some instability across different contexts which we need to unpack further <ref:2508.05880#pg0>.

Jane: That instability is something I noticed too; it means if you ask the AI a slightly different emotional scenario, its internal reasoning structure can shift quite a bit, which makes reliable emotional support very tricky to build <ref:2508.05880#pg1>.

Lu: The study found that proprietary models tend to produce what they call "coherent, low-dimensional appraisal structures," with the top six principal components explaining about eighty percent of the variance, which is pretty close to human patterns <ref:2508.05880#pg2>.

Meng: Eighty percent variance explained by just six dimensions is a tight constraint, Lu. It suggests that even for advanced models, their internal representation of emotion is being heavily compressed into a few dominant factors rather than being finely nuanced across all seventeen dimensions <ref:2508.05880#pg2>.

Title and authors: Lalam: And what’s really fascinating is how open-source models show "more diffuse representations," where those same six components only explain fifty to sixty percent of the variance, which points toward a difference in how different architectures organize their internal thought processes <ref:2508.05880#pg2>.

Tom: That points to a real gap in our current understanding of how these models are actually thinking emotionally, and it ties into what they found about the dimension of Effort, which showed a marked departure from human results <ref:2508.05880#pg2>.

Jane: Effort being so important for the AI but not explicitly selected as the most important cue when asked directly is a key finding; it shows that what we see in their output isn't always what they are consciously prioritizing <ref:2508.05880#pg1>.

Lu: They also found a consistency in dominance for Responsibility and Control when models were asked explicitly which dimension was most important, which is interesting because it aligns with the implicit importance analysis they did earlier <ref:2508.05880#pg2>.

Meng: That's a contradiction that needs sorting; if they are internally prioritizing effort but externally reporting control and responsibility as key factors, how does that reconcile in practice for an agent making decisions?

Lalam: It suggests a kind of internal value-action gap where the model doesn't always report on the dimension it’s actually using to drive its output, which is a significant hurdle for creating truly transparent AI systems <ref:2508.05880#pg1>.

Tom: That gap is something we need to bridge, because if we can't explain *why* an AI reacted the way it did based on these appraisal dimensions, it’s hard to trust it in high-stakes environments <ref:2508.05880#pg1>.

Jane: Speaking of internal representation, they used the Wasserstein distance metric and found that Valence is consistently the primary axis separating emotions across all models, clustering them into positive and negative groups <ref:2508.05880#pg2>.

Lu: Beyond that valence split, they identified "Challenge and surprise" as these interesting 'border emotions' that don't fit cleanly into the positive or negative groups based on valence alone, which suggests a richer internal landscape than just happiness versus sadness <ref:2508.05880#pg2>.

Meng: So, while they have this general split, the subtle emotional states—the border ones—are what really define complex human experience that an AI needs to model accurately, not just the basic polarity <ref:2508.05880#pg2>.

Title and authors: Lalam: And when comparing different models using Maximum Mean Discrepancy, they saw that while most models use similar appraisals for things like fear or sadness, they still show idiosyncratic appraisal patterns for other emotions, which is a sign of model-specific cognitive fingerprints <ref:2508.05880#pg2>.

Tom: So it seems the consensus is that LLMs are capturing some systematic relations but have these unique idiosyncrasies in how they process individual emotional concepts <ref:2508.05880#pg2>.

Jane: And this leads us to the contextual analysis where they looked at personas based on culture and personality traits, and the results were quite telling regarding robustness <ref:2508.05880#pg2>.

Lu: They found absolutely no variation in cognitive appraisals across different cultural personas, which means their internal emotional reasoning structure seems relatively universal regardless of nationality <ref:2508.05880#pg2>.

Meng: That lack of cultural variation is a big win for generalization, suggesting that the core appraisal mechanism isn't culturally encoded in the same way we think it might be <ref:2508.05880#pg2>.

Lalam: However, they did see significant variations tied to personality traits; positive personas boosted self-control and understanding, while negative ones increased perceptions of external control or requiring effort <ref:2508.05880#pg2>.

Tom: That means the AI internalizes individual affective traits quite strongly but struggles to maintain a stable representation of culture, which is something we have to account for when deploying these systems globally <ref:2508.05880#pg2>.

Jane: And looking at the regression analysis, they saw that happiness was predicted by high certainty and low effort, which matches what humans find, but pride diverged because it was associated with high Effort and low Problem ratings <ref:2508.05880#pg3>.

Lu: The finding that anger is almost entirely driven by perceived unfairness really emphasizes its context-dependent nature, suggesting that for universal emotions, the appraisal of fairness is the dominant driver <ref:2508.05880#pg3>.

Meng: That implies if we want an AI to handle anger effectively, we can't just look at its output; we have to tune its internal weighting toward perceived fairness as the primary cognitive cue <ref:2508.05880#pg3>.

Lalam: Overall, this paper on "Too Categorical to be Human: Emotion Concepts in LLMs and Humans" shows that while LLMs follow systematic patterns in emotion reasoning, they have specific structural quirks—like over-indexing on effort or struggling with cultural stability—that mean their emotional understanding isn't perfectly aligned with human experience <ref:2508.05880#pg0>.

Title and authors: Tom: So, to wrap up this discussion, it seems the main points are that LLMs capture systematic relations between cognitive appraisals and emotions but exhibit misalignment with human judgments and instability across contexts <ref:2508.05880#pg3>.

Jane: Exactly, Tom. We see partial human alignment mixed with internal consistency alongside notable idiosyncrasies in their cognitive appraisal structures <ref:2508.05880#pg3>.

Lu: For future work, I think we need to focus on developing better benchmarks like CoRE so we can systematically test these implicit structures rather than relying only on discrete emotion labels <ref:2508.05880#pg1>.

Meng: And from an engineering standpoint, the next step should be incentivizing the model during training to align its latent space representations with those identified cognitive dimensions, especially effort and control <ref:2508.05880#pg2>.

Lalam: I believe these findings suggest that improving emotional reasoning in AI isn't just about adding more data; it’s about restructuring how the AI processes the appraisal dimensions themselves to better reflect human cognitive reality <ref:2508.05880#pg1>.

Tom: It’s a lot to take in, but honestly, understanding these underlying structures is how we build systems that can actually reason with us emotionally rather than just mimicking responses <ref:2508.05880#pg3>.

Jane: It really shows us that emotion isn't something you can just label; it’s a complex set of cognitive appraisals that models are approximating, but not always perfectly <ref:2508.05880#pg1>.

Lu: So, we’ve got a solid foundation here for probing these internal structures with benchmarks like CoRE and looking at how different model types handle those dimensions <ref:2508.05880#pg1>.

Meng: I think we should also keep an eye on papers focusing on the value-action gap, since that seems to be a major practical hurdle for making AI decisions transparent <ref:2508.05880#pg1>.

Lalam: I think the overall implication is that moving toward more cognitively grounded emotional understanding in AI will require us to look past surface-level performance and focus on aligning the model's internal appraisal mechanism with established psychological theories <ref:2508.05880#pg1>.

Tom: That’s a great way to put it, Lalam. It’s about moving beyond just outputting an emotion and understanding the cognitive steps that led to it <ref:2508.05880#pg3>.

Jane: Indeed, and the work on "Too Categorical to be Human: Emotion Concepts in LLMs and Humans" gives us a clear map of where we need to focus our next efforts with these large language models <ref:2508.05880#pg1>.

The paper's summary: Tom: So, to recap, this paper is showing us that while Large Language Models can map out how cognitive appraisals relate to emotions in a systematic way, they don't quite capture the nuanced, context-dependent reasoning humans use when we feel things.

Jane: That’s right. The core finding is that LLMs are basically doing a sophisticated kind of pattern matching on emotion concepts rather than actually experiencing or deeply understanding them the way we do. They're picking up on correlations, but they miss the fine details of human emotional judgment and how those judgments shift depending on what's going on around us.

Lu: What I find really wild is how they break down the internal structure; it turns out most models collapse their reasoning into a few dominant factors, which is way different from the rich tapestry of appraisal dimensions humans use. It’s like seeing the whole sky but only seeing a few major constellations instead of every single star <ref:2508.05880#pg1>.

Meng: From an engineering standpoint, that compression means we can’t just look for one big answer; we need to build systems that can handle more granular inputs if we want to get closer to human-level reasoning. How does a model with such limited internal representation manage complex emotional tasks?

Lalam: I think the real implication here is how it helps us design AI that actually interacts with culture better; since they found no variation across cultures in their basic appraisal structure, it suggests we can build a more universally applicable emotional core for our tools.

Tom: Exactly! The instability across contexts is a big red flag for reliability. If an AI’s internal reasoning shifts wildly based on the prompt, you can’t trust its emotional responses in critical situations.

Jane: It means we need to move beyond just labeling emotions and start modeling the very cognitive steps—the appraisals—that lead to those feelings, which is a much deeper level of thinking for an AI.

Lu: And the paper points out that certain emotions like anger are driven almost entirely by perceived unfairness, showing how context dictates what matters most in those internal calculations. That’s a huge insight for developing adaptive agents.

Meng: So we need to focus our next efforts on giving the AI more flexibility in how it weighs those different cognitive dimensions, especially when dealing with subjective situations where fairness or effort might play a bigger role than simple valence.

Lalam: I see this as an opportunity to build AI that doesn't just react, but can show an internal reasoning process, which could fundamentally change how we use emotional support tools in society.

Tom: That’s the big picture we’re talking about—moving from pattern recognition to genuine cognitive modeling. We’ve got a lot more to unpack on those specific dimensions next!

The paper's improvements: Tom: So, moving past just describing where LLMs are falling short, the paper isn't just stopping there; it’s actually laying out some pretty concrete ways to fix these issues with their cognitive reasoning.

Jane: That’s right. The authors suggest a few clear paths forward, mainly focusing on making the AI more introspective and less reliant on those broad, blurry emotional buckets they currently use.

Lu: They are pushing for better benchmarks, like CoRE, which is a huge step because it forces researchers to test the AI on these seventeen specific cognitive dimensions instead of just checking if it spits out the right emotion label. That’s a powerful way to probe those implicit structures we discussed earlier <ref:2508.05880#pg1>.

Meng: From my side, that means we can start designing training paradigms that reward the AI for aligning its internal latent space with these specific dimensions, like making it prioritize "control-situational" when a task is complex. It moves us toward more goal-oriented reasoning <ref:2508.05880#pg2>.

Lalam: I think the biggest impact of these suggested improvements is giving us a framework to build AI that has a more stable, nuanced internal representation of emotion, which could lead to vastly improved personalized emotional support systems for everyone.

Tom: And look at the practical application: they are calling out that we need a way for models to be transparent about *why* they chose an emotion based on these appraisals, like explicitly stating which dimension was most important. That addresses that value-action gap we talked about earlier <ref:2508.05880#pg1>.

Jane: Exactly, Tom; it’s about moving from a black box to something where the AI can explain its reasoning in terms of cognitive steps, which builds a lot more trust with the people using it.

Lu: Plus, they are exploring ways to make models more robust against cultural differences by focusing on personality traits instead of just nationality, which could lead to more universally applicable emotional intelligence in AI systems <ref:2508.05880#pg2>.

Meng: That focus on personality-based modulation is something I can get behind; it means we can tailor the AI’s affective response based on the inferred user profile, which would make interactions much more adaptive and less generic <ref:2508.05880#pg2>.

Lalam: For me, this suggests a future where AI doesn't just mimic human feelings but can genuinely operate within a cultural and personal context, making emotional intelligence something that improves the quality of human connection overall.

Tom: It sounds like we’re moving toward building systems that are not just reactive, but proactively understand the underlying cognitive architecture of emotion itself. We’ve got some serious stuff on our plate!

Conclusion: Tom: So we’ve covered how the paper, "Too Categorical to be Human: Emotion Concepts in LLMs and Humans," shows that AI models capture systematic patterns in emotion appraisals but struggle with the instability and cultural nuances of human emotional reasoning.

Jane: That’s right; it really highlights that even when an AI seems to handle a situation correctly, its internal logic might be missing some of the subtle, messy human cognitive layers we rely on.

Lu: The implication for the future is that we need to stop treating emotion as a simple label and start modeling the complex appraisal structures themselves, which opens up entirely new avenues for creative AI applications.

Meng: I see it practically as a signal that our current training methods are too blunt; we need mechanisms that reward internal consistency over just matching output labels when dealing with subjective situations.

Lalam: This work gives us a vision where AI can develop truly contextual emotional understanding, which could profoundly improve how people interact with supportive technologies and even each other.

Tom: It’s a lot to digest, but the main message is that for AI to truly reason emotionally, it needs deeper cognitive scaffolding than just pattern matching on words.

Jane: I agree; we’re seeing a move toward building more introspective systems that understand the 'why' behind the 'what' of an emotional response.

Lu: We should keep pushing these benchmarks because they are essential tools for mapping this internal landscape and exploring what those diffuse representations actually mean across different model architectures.

Meng: I’m keen on seeing how we can implement those suggested improvements to make our models more adaptive to personality-based inputs, which would be a huge practical win for user experience.

Lalam: Ultimately, the goal of studying "Too Categorical to be Human: Emotion Concepts in LLMs and Humans" is to build a future where AI can navigate the emotional landscape with a level of reliability and sensitivity that mimics genuine human cognition.

Tom: It’s wild thinking about what we can do next when we start focusing on those specific cognitive dimensions, like Effort and Control, rather than just general emotion categories.

Jane: And that brings us perfectly into our next topic: how these appraisal structures translate into actual decision-making capabilities in complex real-world scenarios.