Assessing and Refining ChatGPT's Performance in Identifying Targeting and Inappropriate Language: A Comparative Study

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Assessing and Refining ChatGPT's Performance in Identifying Targeting and Inappropriate Language".

Jane: The gist: ChatGPT performs well in detecting inappropriate content, showing notable improvements in accuracy through iterative refinements, particularly in Version 6,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: Let’s start by looking at the title and who wrote this stuff. The full title of this work is "Assessing and Refining ChatGPT's Performance in Identifying Targeting and Inappropriate Language: A Comparative Study." It’s pretty straightforward, though I like the "Assessing" part because it tells you they aren't just claiming something works perfectly.

Jane: And the authors are Barbarestani, Maks, and Vossen. They’re from a place that did some solid work on content moderation annotation methods earlier, which gives them a good foundation for this kind of comparison. It suggests this isn't just an internal test; they’re trying to put ChatGPT against established benchmarks.

Lu: The implication here is that we need structured ways to evaluate these large language models in specific, nuanced tasks like targeting detection, not just general text generation. The authors are setting up a framework to systematically probe where the current technology succeeds and where it still falls short compared to human judgment.

Meng: So, what’s the immediate takeaway for us? It seems like the focus isn't just on building bigger models, but on making sure these existing models are reliable enough for real moderation work before we deploy them widely.

Lalam: I think it means we need to be very careful when we talk about AI catching something sensitive, because this paper points out that even advanced models have variability in how they spot specific targeting patterns.

The paper's summary: Tom: Now for the summary of what the authors actually found in this study. They compared ChatGPT’s outputs against crowd-sourced annotations and expert evaluations on a gold set of conversation threads. The core finding is that ChatGPT does perform well when it comes to detecting general inappropriate content, showing improvements as they refined it, especially with Version six <ref:2505.21710#pg1>.

Jane: But the summary also flags a specific issue: its performance on targeting language detection was variable, and it showed higher false positive rates when compared directly to expert judgments. That’s a key distinction they are making about the model's current limitations in that specific area.

Lu: They are looking at three parts of conversations—the beginning, the middle, and the final parts—to see how targeting behavior shifts across a thread. The study found that crowd workers noticed targeting initiation early on, with forty-six percent of subthreads starting with it, but ChatGPT detected a higher prevalence at the very beginning, about ninety-seven percent.

Meng: That difference between what the crowds saw and what the AI saw at the start is significant. If we rely too heavily on initial detection in a stream of comments, that variability could cause serious issues in how we triage incoming content.

Lalam: They also found that when it came to attributing targeting, attributed targeting became specific to an individual in twenty-three percent of cases by human annotators, but ChatGPT caught it twenty-eight percent of the time. That gap between human and AI attribution is what they are trying to quantify here.

The paper's improvements: Tom: The paper details how they improved the model through iterative prompting, moving from Version one up to Version six. They found that these prompt enhancements actually made a difference, leading to Version six achieving the highest Cohen’s Kappa score of zero point six six in identifying targeting language against those expert annotations on the gold set.

Jane: The key improvement they highlighted was making the model more consistent and less varied through prompt adjustments. Specifically, they changed the GPT model’s temperature setting from zero point three to zero for Version six which made the responses much more deterministic.

Lu: That adjustment to temperature is a technical detail that really matters because it moves the AI away from being overly creative and towards being more consistent with what’s actually present in the data, which is exactly what they need for reliable moderation.

Meng: Consistency at scale is everything for me. If a model can be made more deterministic, it means we can predict its output quality better when it hits high traffic volumes, instead of dealing with wildly fluctuating results from a less tuned model.

Lalam: So the improvement isn't just about better detection; it’s about aligning the AI’s behavior with human judgment by making its decision-making process more predictable and focused on what actually matters in those targeted comments.

Conclusion: Tom: So to wrap up this assessment of "Assessing and Refining ChatGPT's Performance in Identifying Targeting and Inappropriate Language: A Comparative Study," the paper shows that while ChatGPT is a strong tool for spotting inappropriate language generally, we still need work on making its targeting detection more consistent.

Jane: The main implication is that continuous refinement through prompt engineering, like moving to Version six with a temperature of zero, can genuinely boost performance metrics when aligned with expert knowledge <ref:2505.21710#pg1>. It suggests that tweaking the input instructions matters as much as just increasing the model size.

Lu: I think it underscores that for complex tasks like identifying nuanced targeting across different parts of a conversation, we need these kinds of targeted refinement strategies to bridge the gap between raw capability and reliable application.

Meng: From an engineering standpoint, it tells us that tuning the inference parameters, like temperature, is a high-leverage way to improve reliability without necessarily needing a completely new model architecture every time.

Lalam: And for me, this paper confirms that we can use these comparative studies to pinpoint exactly where the AI is struggling—in this case, specifically with the variability in targeting detection—so we know exactly what needs to be addressed next in our refinement pipeline.