How Deep Do Large Language Models Internalize Scientific Literature and Citation Practices?

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "How Deep Do Large Language Models Internalize Scientific Literature and Citation Practices?".

Jane: Here's the gist of the paper. The gist The spread of scientific knowledge depends on how researchers discover and cite previous work, and large language models systematically reinforce established citation trends by consistently favoring highly cited papers when generating references How it works The study analyzed 10,000 focal papers sampled from SciSciNet Q1 journals between 1999 and 2021 to prompt GPT-4o to generate references based on the paper's title, authors, year, venue, and abstract The researchers compared the characteristics of human citation patterns with those generated by LLMs solely relying on their parametric knowledge to assess how LLMs align with human practices The experimental setup involved generating a total of 274,951 references for the focal papers based on the ground truth number of references made in the focal paper The generated references were verified against the SciSciNet database using fuzzy matching to classify them as “existing” or “non-existent” based on strict title similarity (0.90 threshold) and author similarity (0.50 threshold) Systematic Preferences in Generated References The analysis revealed several systematic biases in the LLM-generated references that reflect established human citation practices These systematic preferences include: 1.

Tom: Preference for highly cited papers: Existing generated references exhibit a "systematic preference for more recent references with shorter titles and fewer authors" and show a "strong preference toward highly cited papers" where approximately 90% of existing generated references appear in the top 10% most cited papers within their respective field and year 2.

Jane: Temporal bias: Generated references tend to be "relatively more recent than the ground truth" with a recency bias driven by non-existent generated references 3.

Tom: Author and title characteristics: Generated references "tend to favor documents with fewer authors with a peak around 2-3 (1-3 for existing generated references) authors" and show a "clear preference for shorter titles" which aligns with documented human citation behavior 4.

Title and authors: Tom: So let’s look at the title itself, "How Deep Do Large Language Models Internalize Scientific Literature and Citation Practices?". It immediately tells you this isn't just a surface-level test of whether the AI can list some papers.

Jane: Right, it suggests they are digging into the actual depth of internalization. They aren't just asking if the AI knows a paper exists; they want to know if it understands the *practice* of citing literature.

Lu: The authors are from places like Vrije Universiteit Brussel and Harvard University, which gives them a solid foundation in both logic and applied science. Their setup involves sampling ten thousand focal papers from Q1 journals between one thousand nine hundred ninety-nine and two thousand twenty-one <ref:2504.02767#pg1>.

Meng: That's a pretty big sample size for focused research. I wonder how they managed to curate that list to ensure it represented the kinds of papers actually being cited in those high-level journals during that time frame.

Lalam: They are comparing the generated reference lists against actual human citation patterns, which is the crucial part here, because we need a baseline for what's considered normal behavior.

Tom: The implication here is pretty direct: if these models are just making noise, or if they’re systematically biased toward certain things—like very old papers or only the biggest ones—then that affects how new scientific ideas get connected.

Jane: It means we need to understand if this AI tool helps accelerate knowledge building or just reinforces existing, potentially skewed, patterns in science.

Lu: The title frames it as a question about the *depth* of internalization, suggesting they are interested in whether the model captures the complex relationships between papers.

Meng: So if this paper shows a strong preference for certain types of citations, that tells us how much we should trust an AI to help us map out literature.

The paper's summary: Tom: Now let’s talk about what the paper actually found in terms of the results. They found something pretty specific about how LLMs handle these references.

Jane: They showed that LLMs don't just spit out random papers; they systematically reinforce what we call the Matthew effect, which means highly cited papers get favored over others when generating suggestions.

Lu: Specifically, the paper notes that existing generated references show a "strong preference toward highly cited papers" where about ninety percent of them appear in the top ten percent most cited papers within their respective field and year.

Meng: Ninety percent is a huge number for bias. That means the model is heavily leaning on established, famous work rather than exploring the broader landscape of less-cited but potentially important literature.

Lalam: They also found temporal biases, meaning the generated references tend to be "relatively more recent than the ground truth," which they attribute to non-existent references skewing that generation process.

Tom: And it's not just about recency; there’s a bias for shorter titles and fewer authors in those suggested citations. They also see a strong preference for high-impact journals, like Nature and Science.

Jane: So, the summary paints a picture where the AI defaults to what is already well-established in the literature, rather than what might be emerging or less famous but still relevant.

Lu: This points to how citation dynamics are being subtly steered by these models when they operate based only on their parametric knowledge.

Meng: If we deploy this for literature review, we have to be careful; it might just give us a curated list of the most famous papers instead of a comprehensive map of what's happening.

The paper's improvements: Tom: The authors didn't just stop at finding these biases; they actually proposed some ways to fix them. They suggested several improvements to mitigate this reinforcement effect.

Jane: They suggest implementing a post-generation filtering mechanism that specifically targets those patterns we saw earlier—like the preference for more recent papers with shorter titles and fewer authors.

Lu: That sounds like a practical way to counteract the Matthew effect by checking the output against these known undesirable characteristics before presenting it. It’s an attempt to inject some human-like constraints back into the AI’s generation process.

Meng: I like that idea of a filtering step because we can build specific rules around those citation patterns, rather than just hoping the model gets it right on the first try.

Lalam: They also suggest enhancing network modeling to better capture how these generated references internalize human citation structures at second-order graph structures, making them look more like real human networks.

Tom: And they propose a content alignment verification module, using embedding analysis, to check if the generated references actually have high similarity scores with what we expect from human curation in scientific fields.

Jane: That’s smart because it moves beyond just looking at the metadata like authors and titles; it checks the actual semantic relationship between the paper being analyzed and its suggested sources.

Lu: They also suggest a self-citation suppression feature, which is interesting because they found that LLM-generated references tend to reduce author self-citations, so they’re trying to make sure that tendency stays suppressed.

Meng: That’s useful for visibility issues; if the model naturally avoids self-citation, that might be a good thing for assessing impact based on external citations.

Conclusion: Tom: So to wrap up the main findings of "How Deep Do Large Language Models Internalize Scientific Literature and Citation Practices?", the paper confirms that LLMs systematically reinforce established citation trends by favoring highly cited papers when generating references.

Jane: The implication is that without intervention, these models might be subtly directing scientific attention toward already dominant literature, which could limit the discovery of novel connections.

Lu: The proposed improvements focus on filtering those known biases and using network analysis to ensure the generated structure mirrors human practices more closely.

Meng: From a deployment standpoint, it suggests we need robust post-processing checks if we want to use these LLMs reliably for literature mapping, because the raw output has these inherent structural preferences.

Lalam: Ultimately, this paper shows that understanding citation practices is key to understanding how much an AI truly internalizes scientific knowledge and how that affects the research ecosystem.

Tom: That’s our take on "How Deep Do Large Language Models Internalize Scientific Literature and Citation Practices?". It’s a reminder that even in seemingly simple tasks, the underlying structure of data—like citations—carries significant weight.

Jane: And we’ll be looking at how these kinds of systematic biases show up in other areas next time, so keep an eye on that.