2608.07254-SCALE: Scientific Concept Aggregation via LLMs and Embeddings for Fine-Grained Taxonomy Extension

Introduction to the show: ident: Paper Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "SCALE: Scientific Concept Aggregation via LLMs and Embeddings for Fine-Grained Taxonomy Extension".

Jane: The paper was written by Daniele Raimondi, Feichi Lu, Oliver Grun, Mariia Eremina and Andrea Perlato from MDPI.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Paper summary: Tom: Alright, let's give our guests the floor. Lu and Meng, you've both spent serious time with large-scale classification systems — what was the first thing that made you stop and think when you read this paper?

Lu: For me it was the decision to build a separate semantic graph for each OpenAlex Field instead of one giant graph over all keywords. They take roughly three million author keywords from MDPI, filter them down to concept-level terms, then assign each keyword to the fields it belongs to and let clusters form within each field. That way a term like "plasticity" can be a physics concept and a neuroscience concept without being forced to pick one.

Jane: So it's a deliberate move against the old one-size-fits-all taxonomy. Did they need special machinery to keep the clusters at the right size?

Meng: They did. They curated 68 must-link pairs and 68 should-not-link pairs, then grid-searched the number of nearest neighbors, the similarity threshold, and the Leiden resolution parameter. The winning setup gave 69 point 5 percent must-link precision and 87 point 5 percent should-not-link precision, which is pretty respectable for genuinely hard boundary cases.

Tom: Those clusters then become named Concepts. But how do they decide where each Concept sits in the OpenAlex hierarchy?

Jane: They embed the Concept's name and description, pull the most similar OpenAlex Topics, and let GPT-4o-mini pick from that shortlist. If a Concept is close to several Topics, they keep all links, so something like Industry 5 point 0 can live under multiple disciplinary paths instead of being squeezed into one.

Lu: That multi-attachment is actually a big philosophical choice. Most taxonomies force a single parent, but research doesn't work that way. Giving Concepts several paths makes them far more useful for interdisciplinary science.

Meng: And the coverage results are striking: 113,892 Concepts in total, attached to 98 point 8 percent of the 4,516 OpenAlex Topics. Physical Sciences gets the largest share at 57 point 6 percent, but every domain is represented.

Tom: Now, they didn't just stop at building the taxonomy. Jane, you were looking at the real-world test — what happened when they used it to tag papers?

Jane: They ran it on the full MDPI corpus, nearly two million papers. 94 percent of Concepts appear in at least one assignment, so the taxonomy isn't just decorative, and the median paper gets five Concepts. Then 22 domain experts judged whether the tagged Concepts were relevant to 110 papers across eleven fields.

Meng: Precision at rank five was 88 point 9 percent for the GPT model and 88 point 7 percent for Qwen3. The field breakdown is fascinating — history sits above 97 percent precision while physics and materials science sit around 75 to 80, which tells you where the vocabulary is hardest to pin down.

Tom: Wait, history is the easiest? I would not have guessed that.

Lu: It makes sense when you think about it. Historical terminology changes slowly and experts agree on its meaning. Physics and materials science

Page 1 of the paper: Tom: So this first page is really the setup. It explains why we need a new layer of scientific concepts in the first place. The authors argue that big classification systems like OpenAlex handle broad disciplines well, but they miss the highly specialized stuff that's actually happening in research today.

Jane: And that's where author keywords come in — and they're a mess.

Tom: Exactly. The paper describes them as "fragmented, redundant, and strongly affected by terminological variation." So the same idea might show up as ten different phrases, and the same phrase might mean something totally different in another field. That's the gap they want to fill: something between a broad research topic and an individual paper.

Jane: And they're building it with embeddings, LLMs, and graph clustering all combined.

Tom: Right. The abstract spells out the recipe: scientific text embeddings, large language models, and graph-based community detection. And they've already applied it to OpenAlex, producing roughly 114,000 concepts underneath the existing topics.

Jane: So each concept is a coherent group of related terms, but stable enough to sit in a hierarchy.

Tom: That's what matters here. They're promising compatibility, not a replacement. Each concept gets a path up through topics, subfields, fields, and domains — so you can go from a very specific research theme all the way to a whole domain.

Page 2 of the paper: Tom: So on this page the authors go through all the rival approaches, and honestly, they're showing why each one falls short for their goal of adding a stable layer beneath OpenAlex Topics.

Jane: They start with topic modeling, right? LDA and BERTopic find themes in text, but those themes are tied to whatever corpus you feed them. Run them on a different set of papers and you get different clusters.

Tom: Exactly, and those clusters don't come with any connection to an existing hierarchy. So even if you get a nice thematic grouping, you can't just slot it into OpenAlex. It's a dead end for their use case.

Jane: Then they move to taxonomy construction systems like TaxoGen and TaxoCom. Those actually build hierarchies, but again, they build them from scratch for a specific corpus.

Tom: And the latest one, TaxoAdapt, uses LLMs to adapt to evolving research, but the authors point out it still creates its own structure. SCALE deliberately does the opposite — it keeps the existing four OpenAlex levels untouched and only adds a new fifth level underneath.

Jane: That's a pretty sharp distinction. They're not reinventing the tree; they're just growing one extra branch level on a tree that's already there.

Tom: Then they bring up the Computer Science Ontology and Klink-2. Those are great examples of automated concept building, but they're limited to specific domains. CSO is computer science, Klink-2 is about linking topics over time, not covering all of science in one clean hierarchy.

Jane: And that sets up the key section on OpenAlex itself. This is the part I found really telling — OpenAlex actually had a concept layer before, inherited from Microsoft Academic Graph, but they deprecated it.

Tom: Right, those MAG concepts were notoriously unstable. Maintaining them consistently just didn't pan out, so OpenAlex replaced them with Topics and a supplementary set of keywords.

Jane: And those keywords aren't a structural level at all. They're just ten fixed labels per topic, generated to help with paper retrieval. No hierarchical path, no stability as conceptual units.

Tom: So that's the gap the authors are really circling: existing systems either give you fine detail without structure, or structure without enough detail. OpenAlex keywords give you neither — they're just static labels.

Jane: So page 3 basically justifies why they need a new framework. Every alternative gets ruled out for a specific reason, and that makes the case for SCALE much stronger.

Page 3 of the paper: Tom: So the really clever bit on this page is that they don't just throw every author keyword into the clustering pot. They first decide which keywords are actually at the right level of detail to become a Concept.

Jane: Right, and they use an LLM to sort everything into three buckets: high-level terms like "Physics" or "Artificial Intelligence," low-level things like specific genes or chemical compounds, and then the middle ground — Concept-level terms that are reusable methods or research themes.

Lu: That middle bucket is what actually goes into building the taxonomy.

Tom: Exactly. The point is they're not trying to preserve every keyword anyone ever used. They're filtering for terms that can serve as stable classification units, which is a much smarter goal than just inventorying everything.

Jane: Then comes the part I found fascinating: they build a separate semantic graph for each of the 26 OpenAlex Fields instead of one giant global graph. The reason is that a keyword like "model" or "optimization" can mean completely different things in engineering versus biology.

Meng: So they're respecting disciplinary context before any clustering happens.

Tom: Yes, and they even let a keyword belong to multiple fields if it's genuinely used in several disciplines. So "neural network" can be processed independently for computer science and for neuroscience, rather than being forced down one path.

Jane: To make those graphs meaningful, they first generate a short field-specific definition for each keyword using an LLM. Then they encode the keyword together with that definition using SPECTER2, so the embedding carries not just the word but also its disciplinary meaning.

Lu: And then they connect keywords with edges only if they're mutual nearest neighbors and pass a similarity threshold. That's a nice way to cut out accidental or weak connections.

Tom: Right, so the graph only contains genuinely close, bidirectional semantic relationships. After that, they run the Leiden algorithm on each field's graph to find communities of related keywords — those communities become the raw material for the Concepts themselves.

Jane: What's striking is how much careful design went into just this one page. It's not a single black-box step; it's a pipeline where each decision — the granularity filter, the field split, the reciprocal neighbor rule — is deliberately chosen to keep the final taxonomy coherent.

Meng: It does read like they learned from past taxonomy failures, especially the instability of Microsoft Academic Graph's old concepts.

Tom: That's a good point. By anchoring everything in fields and requiring mutual similarity, they're building in stability from the ground up. The Concepts that emerge later are only as good as these graphs they construct here.

Page 4 of the paper: Tom: So we've hit the results section, and the first thing they do is show how they calibrated the clustering algorithm. Because earlier they described building these field-specific semantic graphs and running Leiden, but they never said how they chose the settings.

Jane: They just tuned it until it looked right?

Tom: Not exactly. On this page they explain they ran a grid search over three parameters: the number of nearest neighbors, the minimum similarity threshold, and the Leiden resolution parameter. Each of those controls how many clusters emerge and how tightly related the words inside them have to be.

Jane: But how do you know which combination is actually good?

Tom: They built a small test set by hand, made of pairs of terms. Some are must-link pairs, meaning those terms should appear in the same cluster, and some are should-not-link pairs, meaning they should be forced apart. These weren't easy pairs either—they deliberately picked difficult boundary cases they kept running into during development.

Jane: So they're testing on the hard cases, not the obvious ones.

Tom: Right. Then for every parameter configuration, they measured how many must-link pairs stayed together and how many should-not-link pairs got separated. That gives two precision scores, and they also tracked the total number of communities, since each community will become a concept.

Jane: Because if you get too many clusters, concepts get fragmented, and too few makes them too broad.

Tom: Exactly. Figure 3 plots all those configurations on a scatter plot, with must-link precision on one axis and should-not-link on the other, and the bubble size shows how many concepts each setting would produce. The highlighted point is the one they chose as the best balance.

Jane: That's a much more principled way to set the granularity than just eyeballing the output. They're letting the evaluation decide where the sweet spot is.

Page 5 of the paper: Tom: So this page is where they actually hand you the keys to the whole system. There's a web app called Taxonomy Explorer that lets you click through the five levels, from Domains all the way down to Concepts, and the page shows a radial visualization on the left with the taxonomic path on the right.

Jane: Wait, so you can actually navigate the hierarchy yourself, not just read about it?

Tom: Exactly. And you can search, filter, and export from it. But there's a catch — it's a research prototype, and you have to email the corresponding author for access credentials.

Jane: Then they also built something called Atlantis, right? That one sounds more like a map.

Tom: Yeah, Atlantis is the opposite of the Explorer in a way. Every Concept is a point on a low-dimensional map, and you can look at it in 2D or three dee, color it by Domain or Field, and click a point to see the Concept's label, explanation, and its whole path in the hierarchy.

Jane: So the Explorer shows the official tree, and Atlantis shows how Concepts sit near each other in meaning-space. Do they warn you about interpreting distances on that map?

Tom: They do. The page explicitly says distances in the projected space should not be read as a formal measure of taxonomic relatedness. So it's for browsing and inspiration, not for drawing scientific conclusions.

Jane: And the third piece is the open dataset on Hugging Face. What's in that package?

Tom: It's a versioned snapshot with tables for every level — Domains, Fields, Subfields, Topics, and the new Concepts — plus a flat hierarchy view and stable identifiers. The top four levels match OpenAlex, and the fifth level is the SCALE output.

Jane: And they're clear that the explanations are eye-generated for browsing, not expert definitions. That's an honest limitation to put right in the release notes.

Tom: It is. And the whole thing is under a CC0 license, so anyone can reuse it freely. But the page also notes it's taxonomy metadata only — no full texts, no abstracts, no author lists, and no human-evaluation microdata.

Jane: So you get the structure itself, not the evidence behind it. That seems like a deliberate line they're drawing.

Tom: Exactly. The dataset is meant to be a clean, reusable artifact, while the evaluation details stay behind the scenes. They even point out that Atlantis needs no registration, while the Explorer is gated — that's an interesting split in how open each tool is.

Jane: I'd say that's fine for a prototype. The open dataset is the real deliverable here.

Page 6 of the paper: Tom: We've spent a lot of time on the mechanics of building this fifth level, but page eleven is where they step back and tell you why it matters.

Jane: Their opening line is pretty direct. They say the main contribution is introducing Concepts as a new unit for organizing scholarly knowledge. A named unit with a place in the hierarchy, rather than just a loose cluster of keywords.

Tom: That's the key contrast with what OpenAlex already had. The old Concepts were deprecated, and the current Keywords are just ten fixed labels per topic. SCALE builds this fifth level from the bottom up, straight out of author keywords.

Jane: Exactly. The paper makes that explicit — OpenAlex Keywords are derived from Topics and assigned to individual works, while SCALE Concepts are a separate, finer-grained structural level. They're not retrieval tags, they're a layer you can actually navigate.

Tom: And they put real numbers behind it. They ran this in production on MDPI's whole corpus of nearly two million papers.

Jane: That part struck me. 94 percent of the Concepts were used in tagging, and each paper gets a median of five Concepts. So the taxonomy isn't sitting on a shelf; it's actively doing work.

Tom: Still, they keep some humility. They admit the current thing is a taxonomy, not a fully formal ontology.

Jane: Right, and that's where page eleven gets interesting. They say future work could decompose Concepts into finer ontological entities, using citation patterns, co-occurrence signals, and publication metadata to infer explicit relations among them.

Meng: That sounds like a bridge toward knowledge graphs. And they actually name MarmotGraph from EBRAINS as a possible home for this.

Jane: Yes, that's a nice touch. A taxonomy built from MDPI keywords could eventually feed a much broader semantic network of scientific knowledge.

Tom: So the takeaway isn't just that they built a bigger hierarchy. It's that they made it reusable, tested it at scale, and left a clear path toward something richer.

Conclusion: Tom: So wrapping this one up, the big takeaway for me is that they actually built a working fifth layer under OpenAlex, not just a proof of concept.

Jane: Yeah, and they did it at real scale—over a hundred thousand concepts, and nearly every single OpenAlex topic got at least one concept attached to it.

Tom: What struck me was how they turned messy author keywords into something stable, by clustering them per field and then giving each cluster a proper name and description.

Jane: And the fact that they've already deployed it in production for tagging nearly two million papers makes it feel much more concrete than a lot of taxonomy research.

Tom: The expert evaluation giving around eighty-nine percent precision at five tags is solid evidence that the concepts actually match what papers are about.

Jane: Though I did appreciate that they were honest about the weaker spots, like the must-link clusters not always grouping related terms together.

Tom: Right, and the human judgment part was tricky too—the agreement between experts wasn't great, which suggests the concepts aren't always unambiguous.

Jane: Even so, the whole thing points toward a future where you can trace very specialized research threads across disciplines, not just broad topics.

Tom: And they left the door open for turning this into a proper ontology later, with real relations between concepts instead of just a hierarchy.

Jane: That would be a natural next step, especially if they tie in citation patterns and co-occurrence data to infer how concepts relate to each other.

Tom: For now, though, it's a genuinely useful resource, and it's openly released, so other people can build on it.

Jane: Definitely a paper worth keeping an eye on, and I'm curious to see what the next one has in store for us.

Tom: Same here. Let's move on and see what's up next.