Pragmatics beyond humans: meaning, communication, and LLMs

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Pragmatics beyond humans: meaning, communication, and LLMs".

Jane: The paper was written by Vít Gvoždiak from Association for Computational Linguistics.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Title: Tom: So, let’s dive into the core of this paper and what it means right from the start. It isn't just about whether ChatGPT *can* answer a question; it's about *how* it understands the unspoken layers of language.

Jane: The authors argue that traditional views—like those based on Grice’s cooperative principle—aren’t quite enough to capture this new reality where humans and AI interact.

Tom: They are questioning the entire established hierarchy of meaning, which is usually broken down into syntax, semantics, and pragmatics.

Lu: If we look at the old model, it assumes that pragmatics is just a final little layer added on top of semantics and syntax, but this paper suggests that view is fundamentally flawed.

Meng: It seems like they're saying that since the models are so complex and probabilistic, relying on old human-centric tests might be completely misleading us.

Jane: That's true, Meng; we are moving away from the idea of a simple "truth evaluation" to something much more complex.

Tom: This shift is about realizing that LLMs aren't just processing words; they are engaging with language in a way that feels like a dynamic interface for action.

Lu: It’s an incredible pivot, moving the conversation from *what* is said to *how* it functions as a tool.

Meng: But how does this function translate into actual measurable outcomes? That's what I need to know to see if we can actually build reliable systems around these linguistic nuances.

Lalam: We are shifting our definition of communication from being a purely human-to-human exchange to something that impacts culture globally.

Summary: Tom: The paper offers a lot of ground, summarizing the challenges facing modern pragmatic research in the age of AI. One major problem is called "substitutionalism."

Jane: It’s when we treat one specific model’s performance—like GPT-two or GPT-four—as if it represents all the LLMs in existence.

Tom: The authors point out that this substitution often makes us ignore the real human participants in the conversation, which is a huge blind spot.

Lu: It’s like trying to judge an entire species based on one single specimen, Tom; you just miss so much about the overall population.

Meng: If we're basing our research on a specific model without acknowledging this bias, how can we trust that our findings apply to other models?

Jane: We also need to look at the way researchers choose their subjects, whether they pick proprietary systems or open-source ones, and that’s often done without a systematic approach.

Tom: The paper argues that by focusing too much on the machine as an object of study, we end up with a very human-centered view of pragmatics.

Lu: It’s a bit circular, isn't it? We are using tools designed by humans to study concepts that are inherently human experiences.

Meng: So, if we aren't focusing on the model itself but on the whole interaction, what does that mean for the architecture of an actual deployment?

Lalam: It means we have to design systems where the machine and both users are seen as active participants in a new form of culture.

Improvements: Tom: To move past these old problems, the authors suggest several improvements, chief among them is this "Human-Machine Communication" or HMC framework.

Jane: The HMC framework is much better because it doesn's just look at the linguistic output; it looks at three different layers: functional, relational, and metaphysical.

Tom: The functional aspect deals with how AI functions as a conversational partner in different types of communication genres.

Lu: And the relational aspect is really interesting because we have to consider how LLMs influence *our* own roles and identities during that interaction.

Meng: That sounds like a massive sociological study mixed with an engineering problem, which is great, but what does "metaphysical" look like in terms of code?

Jane: It’s about the fundamental shift in our understanding that communication isn't just a human activity anymore; it’s something that involves the machine too.

Tom: The paper also introduces this concept called "context frustration."

Lu: Context frustration is where the massive amount of data we feed these models makes them seem like they have infinite context, but in reality, it's just causing a massive collapse of shared understanding.

Meng: It’s an engineering nightmare if the input is huge but the actual coherent grasp remains small.

Jane: The authors suggest that we are being pushed to co-create our own pragmatic conditions when interacting with these models.

Tom: Which is why they propose probabilistic pragmatics, which lets us model communication as a continuous, incremental process rather than just a binary success or failure.

Conclusion: Tom: So, let’s bring everything together and summarize what we’ve learned from "Does ChatGPT Resemble Humans in Processing Implicatures?"

Jane: This paper argues that pragmatics has to evolve beyond the traditional idea of a final layer of meaning.

Tom: We need to use frameworks like HMC to move past the outdated semiotic trichotomy and stop treating LLMs as just proxies for human intelligence.

Lu: It’s about recognizing that AI is fundamentally changing how we define what "intelligence" looks like in communication.

Meng: The key takeaway for me is that we can’t just use old linguistic tests; we have to build systems that are robust against context frustration and recognize the probabilistic nature of language.

Jane: We also need to remember that human roles aren't static, and they are actively being shaped by these interactions in a way the traditional theories couldn't see.

Tom: This paper forces us to rethink how we evaluate AI, moving away from simply asking if it "resembles humans" toward understanding *how* it functions.

Lu: It’s a massive shift that will fundamentally change our cultural conversation about technology and humanity.

Tom: Thanks so much for listening as we unpack this groundbreaking paper on the future of communication. We'll be right back after the break to discuss another fascinating piece of research!