An Agentic Hybrid Top-Down and Bottom-Up Approach to Knowledge Graph Generation

arXiv:2608.07023 · cs.CL, cs.AI · Submitted 2026-08-07 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Paper Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "An Agentic Hybrid Top-Down and Bottom-Up Approach to Knowledge Graph Generation".

Jane: The paper was written by Emma Jouffroy, Warren Jouanneau and Marc Palyart from Malt.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Paper summary: Tom: Alright, we finally get to dig into this one.

Jane: And it hits close to home for anyone who's ever cleaned up a messy spreadsheet.

Tom: The crew at Malt — the European freelancer marketplace — built a knowledge graph generator.

Jane: Their raw material is brutal. Freelancers typing "gestion de projet web", "Web PM", "Project Management".

Lu: Across five languages, with typos, jargon, and fresh skills appearing every week.

Meng: The paper's core thesis is a hybrid — marry top-down structure to bottom-up discovery.

Tom: Top-down means anchoring every recognized skill to a stable Wikidata entity.

Jane: Bottom-up means letting an LLM spot long-tail skills that Wikidata never registered.

Lu: Then wiring those new skills into the graph with proper metadata.

Meng: They wrap that in an "agentic reflection" loop — the model critiques and corrects its own output.

Tom: The headline results are strong. 36,037 raw strings went into the pipeline.

Jane: 77 percent of them resolved to a knowledge graph node. The rest got flagged as noise or non-skills.

Lu: Those mapped strings collapsed into 13,298 canonical skill nodes.

Tom: That's a 52.1 percent compression rate.

Jane: So the platform went from 36,000 messy strings to about 13,000 clean concepts.

Meng: And each node carries labels in all five languages. 66,490 labels total.

Lu: That matters for matching — a search in Dutch finds the same node as a search in French.

Tom: They also benchmarked against a gold standard annotated by five domain experts.

Jane: Found coverage reached 84.9 percent — when the model attempts a match, it's usually right.

Meng: The overall wrong guess rate sits at 19.1 percent.

Lu: Sounds high, until you see the input. Real profiles are beautifully chaotic.

Lalam: Step back — the broader promise is a living graph for a fast-moving labor market.

Tom: And that matters because matching humans to work is genuinely high-stakes.

Jane: Plus you can audit every node back to its source. That's rare in this space.

Meng: I want to see how they keep the LLM from hallucinating skills out of thin air.

Tom: Then page one is our stop. It frames that exact problem.

Page 1: Jane: We've got the thesis. Page one now frames where this fits in the world.

Tom: The paper opens with the two classic ways to build these structures.

Lu: Rigid top-down methods lean on fixed ontologies like ESCO.

Meng: ESCO gives you consistency and high precision, sure.

Jane: But it struggles with emerging and hyper-local concepts.

Tom: Then there's chaotic bottom-up — purely generative clustering.

Lu: That's flexible, it can catch novel trends.

Meng: But it has no guardrails, so equivalent concepts fragment into redundant clusters.

Tom: They use a project management story to make it concrete.

Jane: "Project Management" and "Web Project Management" should sit together in a hierarchy.

Meng: A rigid ontology misses the web specialization entirely.

Lu: A wild bottom-up system might split them apart with no relationship at all.

Tom: And worse — it can hallucinate links. Their example shows "Excel" somehow attached.

Jane: Excel got linked to project management? That's the kind of noise HR systems can't tolerate.

Lu: Exactly. The hybrid they propose anchors the baseline to Wikidata entity Q179012.

Meng: So "Project Management" gets a stable, verifiable anchor.

Tom: Then agentic reflection synthesizes "Web Project Management" as a distinct child node.

Jane: With relational metadata tying it back to the parent.

Lu: And that absorbs all the shortcuts — "Web PM" — into one clean structure.

Tom: The key move is dividing labor.

Jane: Recognized concepts get grounded in the knowledge graph. Unrecognized ones get generative reflection.

Meng: That split is what makes the whole thing auditable.

Lalam: The philosophical point is you don't trust the model with everything; you trust it only at the edges.

Tom: It's a nice division of responsibility.

Jane: And it quietly addresses the hallucination worry, at least in principle.

Meng: I want to know how previous attempts failed before they landed here.

Tom: That's literally page two. Related work has the autopsy.

Page 2: Meng: So we're on to the autopsy. Page two walks through the related work.

Tom: Taxonomy creation started with expert-curated ontologies and static knowledge bases.

Jane: Expensive and slow. Then automated ML methods arrived.

Lu: Early skill extraction models, even efficient encoders, produced flat lists.

Meng: Flat lists mean semantic ambiguity. "Java" the island, "Java" the language, "Java" the coffee.

Tom: Top-down methods tried to fix that by anchoring to knowledge graphs.

Jane: They used domain seeds, tree expansion, LLM-driven ranking.

Lu: But static methods can't keep up with a moving job market.

Meng: Bottom-up clustering scales well, yet its labels are often uninterpretable.

Jane: And the clusters lack relational metadata — you get bags, not graphs.

Tom: Then LLM-based work started structuring categories through abstractive prompting.

Lu: Localized induction, specialized schemas, end-to-end generation.

Jane: End-to-end batch processing still fragments hierarchies.

Meng: Recent multi-agent frameworks added reflection and dynamic alignment.

Tom: Self-correction, prompt optimization, tree search — the reasoning toolkit got bigger.

Jane: So with all that machinery, what's still missing?

Lu: The paper names three persistent failures.

Meng: Hallucinated skills, formatting instability, and bias.

Tom: And that's where the hybrid architecture makes its case.

Jane: The LLM drives the pipeline, but its reasoning is anchored to a deterministic knowledge graph for anything recognized.

Lu: Unconstrained generation only happens for unmapped skills and their relational metadata.

Meng: So the model's imagination is fenced in.

Lalam: That's the real contribution — freedom at the frontier, discipline at the core.

Tom: I like that framing. Discipline at the core.

Jane: Now I need to see that discipline in practice. Page three shows the machinery.

Meng: Five stages, one iterative loop. Let's look.

Page 3: Tom: So the machinery. The paper runs everything as an iterative loop, not a straight line.

Jane: Five stages, and the whole thing cycles until the graph stabilizes.

Lu: They chose Wikidata as the anchor because its multilingual coverage is massive and open.

Meng: And they drive it with Gemini 1.5 Flash — a lightweight model chosen for cost and availability.

Tom: Stage one is reconciliation. That's where raw text meets Wikidata.

Jane: The engine pulls the top ten Wikidata candidates for the input string.

Lu: But a bare string like "gestion de projet web" carries almost no context.

Meng: So they enrich it with two empirical features from the freelancer's profile.

Tom: The top fifteen co-occurring peer skills.

Jane: Plus the top five professional categories from their history.

Lu: That context lets the LLM disambiguate properly.

Meng: The output is a structured JSON — chosen QIDs, confidence scores, step-by-step reasoning.

Tom: And two critical flags: is skill and is compound.

Jane: So the model has to decide whether something is even a skill at all.

Lu: And whether it's actually several skills mashed together.

Meng: The multilingual trick here is smart. Candidates are evaluated in all five languages at once.

Tom: That forces "Réseaux sociaux" and "Social Media" onto the same global QID.

Jane: No language silos. No splitting by translation.

Lu: Stage two is canonicalization — grouping validated inputs by their QID combinations.

Meng: The model generates human-readable preferred labels for all five languages.

Tom: There's a strict fallback hierarchy.

Jane: First priority goes to the most-used label on the platform.

Lu: Then the official Wikidata title.

Meng: And only if neither works does the model synthesize something new.

Tom: Every label carries a provenance tag — malt, wikidata, or generative.

Jane: That tag is how you keep the whole graph explainable.

Lalam: And that's the quiet revolution. The model proposes, but the source of truth stays visible.

Tom: Wait until you see what happens to the skills that don't fit. That's page four.

Jane: The curation stage, and the orphan queue.

Page 4: Jane: Page four takes us into the heart of the system — the curation stage.

Tom: This is where the agentic reflection actually bites.

Lu: The curation agent checks whether every raw skill is truly equivalent to its group.

Meng: If it's not equivalent, the model explains why.

Tom: Seven granular rejection criteria — ambiguous, specialization, semantic mismatch, not a skill, methodology, context, sub-task.

Jane: The interesting part is that rejections aren't just thrown away.

Lu: The model generates a suggested pref label for each rejected concept.

Meng: So a rejected specialization keeps its identity, just under a better name.

Tom: There's a safety valve too. Any node with over 50 percent rejection gets flagged for human review.

Jane: That prevents one bad cluster from cascading through the whole graph.

Lu: Stage four handles consolidation — deduplication across batches.

Meng: They use asymmetric bootstrapping to avoid comparing everything to everything.

Tom: The first batch builds the baseline graph. Every later batch compares only against that.

Jane: Lightweight heuristics flag potential merges first — shared skills or low edit distance between labels.

Lu: Then the LLM makes the final call: merge or keep separate.

Meng: And those decisions get cached, so future epochs don't re-litigate them.

Tom: Stage five is the cleverest bit. Orphan recovery.

Jane: Remember "gestion de projet web" getting rejected as a specialization?

Lu: It becomes an orphan with a synthetic identifier derived from its suggested label.

Meng: Identical concepts get identical suggested labels, so they naturally group together.

Tom: Next epoch, those orphans are re-injected and form a stable sub-branch.

Jane: Linked back to the parent Wikidata entity. The graph heals itself.

Lu: The loop repeats until the orphan queue is empty — full semantic convergence.

Meng: They call it a self-healing loop. That's not hype; it's literally the architecture.

Lalam: This is where the hybrid earns its name. Wikidata gives the skeleton, orphans build the new muscle.

Tom: And the whole thing runs without a human babysitting each merge.

Jane: The question is whether it actually works at scale.

Lu: That's exactly what page five answers. Numbers time.

Page 5: Lu: Numbers time. Page five opens with the coverage results.

Tom: 36,037 raw strings went in. 27,743 got resolved — a 77 percent global coverage rate.

Jane: The remaining 8,294 got flagged as non-skills or semantic noise.

Meng: The mapped strings grouped into 15,010 semantic groupings first.

Tom: Then streamlined down to 13,298 canonical skill nodes.

Jane: That compression rate — 52.1 percent — is the whole point. Downstream redundancy vanishes.

Lu: And the average sits at 2.08 variations per canonical node.

Meng: Then comes the cross-lingual claim, and it's striking.

Tom: 100 percent of the 13,298 nodes are fully supported across all five locales.

Jane: That's exactly 66,490 standardized labels. Perfect symmetry.

Lu: The Pareto distribution backs it up too.

Tom: The top 1,000 canonical skills cover 82.74 percent of platform usage.

Jane: The top 5,000 capture 97.25 percent.

Meng: So the long tail is basically pure specialty — and mostly noise-free after cleanup.

Tom: The paper draws a sharp line between lexical long tail and semantic long tail.

Jane: Typos and redundant variants get compressed hard.

Lu: But rare, emerging capabilities get preserved through the orphan loop.

Meng: That's why the final graph stays rich despite the aggressive filtering.

Tom: Then comes the gold standard evaluation. Five domain experts, no overlap.

Jane: Global alignment coverage is 79.7 percent. Found coverage is 84.9 percent.

Lu: Wrong guess rate lands at 19.1 percent on the full input set.

Tom: And performance varies by domain — that's the honest part.

Jane: Video games hit 91.8 percent found coverage.

Lu: Communication lags at 81 percent — softer, more subjective terminology.

Meng: The rejection table tells its own story. SCORE REJECTED leads at 31.9 percent.

Tom: NODE CURATION follows at 28.1 percent. NOT A SKILL blocks 13.7 percent.

Jane: So the pipeline is rejecting aggressively and explaining every refusal.

Lalam: Those percentages are the real audit trail. Every decision leaves a paper trail.

Tom: Exactly. And page six pushes that explainability even further.

Jane: Provenance data and the bias discussion. That's next.

Page 6: Tom: Page six opens with provenance, and the numbers are reassuring.

Jane: 80.65 percent of the knowledge graph is anchored in factual data.

Lu: Split between 67.28 percent empirical platform usage and 22.08 percent Wikidata titles.

Meng: Only 19.35 percent of labels are purely generative.

Tom: So the model's imagination is a sliver, not the foundation.

Jane: They also report outlier rates of 0.01 to 0.06 per sub-graph after human review.

Lu: That's tiny. The curation agent is doing its job.

Meng: Then the discussion turns to real-world deployment.

Tom: A streamlined taxonomy from this graph is already running in Malt's candidate matching.

Jane: The graph gives the matching engine an interpretable intermediate layer.

Lu: That's huge for auditability. You can trace why a match happened.

Meng: They're also honest about Wikidata's weaknesses.

Tom: It lags on niche HR jargon, and its generalist nature creates structural inconsistencies.

Jane: But the orphan recovery loop acts as a safety net for exactly those gaps.

Lu: Then comes the bias section, and it's thoughtful.

Meng: LLMs are English-centric. Forcing everything into English structures erases local nuance.

Tom: They want to preserve cross-lingual variation instead of over-normalizing.

Jane: And gender-marked terms — French and German occupational titles carry grammatical gender.

Lu: Their goal is mapping gendered variants to shared concepts while keeping the distinct forms.

Meng: That's a genuinely hard problem, and they're naming it early.

Tom: The future work roadmap has three tracks.

Jane: Extended evaluations against ESCO, TnT-LLM, and CLIMB.

Lu: Cost and scalability — token consumption, latency, optimization.

Meng: And failure analysis — tracking manual interventions and systematic mapping errors.

Lalam: The responsible deployment checklist. Rare to see it laid out so explicitly.

Tom: And rare to see a paper admit what's still unbenchmarked.

Jane: The consolidation phase is new and hasn't been formally evaluated yet.

Lu: That honesty makes the whole thing more credible.

Meng: Alright. I think we're ready to close the book on this one.

Conclusion: Tom: So let's wrap this up. The paper gave us a hybrid pipeline that builds skills knowledge graphs.

Jane: Five stages — reconciliation, canonicalization, curation, consolidation, iteration.

Lu: Wikidata grounds the recognized concepts. Agentic reflection catches the rest.

Meng: 36,037 messy strings became 13,298 clean, multilingual skill nodes.

Tom: 77 percent coverage, 84.9 percent found coverage, and a 52.1 percent compression rate.

Jane: Every node traceable back to a source — platform usage, Wikidata, or a labeled synthesis.

Lalam: The bigger takeaway is the architecture philosophy. Ground first, generate only at the edges.

Tom: That's what makes it both scalable and defensible.

Jane: And the self-healing orphan loop means the graph keeps up with the market.

Lu: New skills don't break the structure; they grow it.

Meng: The bias discussion shows they're thinking about who gets represented and how.

Tom: For HR platforms, this is a blueprint.

Jane: For anyone building knowledge graphs from noisy text, it's a template.

Lu: And they published all the prompts in the appendix. You can reproduce the pipeline.

Meng: With a different model, different data — the architecture stands on its own.

Tom: The honest caveat remains. The consolidation phase still needs formal benchmarking.

Jane: But as ongoing work, this is a strong foundation.

Lalam: It moves the field from static taxonomies to living graphs.

Tom: And that's a genuinely useful step forward.

Jane: Alright, we're saying goodbye to this one.

Tom: Thanks to the Malt team — Emma, Warren, and Marc — for sharing the work.

Jane: Goodbye, paper. We'll see what's next on the arXiv feed.

Meng: I'm curious what follows this. The feed never sleeps.

Tom: Then let's find out together.

Emma Jouffroy, Warren Jouanneau, Marc Palyart

Malt

cs.CL, cs.AI

Submitted: 2026-08-07

Updated: 2026-08-10

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 59/100

The gist: The paper, authored by Emma Jouffroy, Warren Jouanneau, and Marc Palyart (Malt, Paris, France), addresses the challenge of organizing "thousands of unstandardized, multilingual expertise

Key concepts

Knowledge Graph: A structured network of entities and their relationships, used here to organize freelancer skills into a clean hierarchy. Each skill is a node with labels in multiple languages, linked to a stable Wikidata identifier, enabling consistent matching across languages and reducing redundancy.
Top-Down vs Bottom-Up: Top-down uses fixed ontologies like ESCO for consistency but misses emerging skills. Bottom-up clustering is flexible but creates fragmented, unlabeled groups. The paper's hybrid combines both: top-down anchors to Wikidata for known skills, bottom-up lets an LLM discover new skills, then integrates them with metadata.
Agentic Reflection: An iterative loop where the LLM critiques and corrects its own output. In this system, a curation agent checks if raw skills match their groups, rejects mismatches with specific reasons, and generates better labels. This self-correction reduces errors and creates an audit trail for every decision.
Orphan Recovery: A mechanism for handling skills that don't fit existing categories. Rejected specializations become 'orphans' with synthetic IDs, then are re-injected in later cycles to form new sub-branches linked to parent Wikidata entities. This allows the graph to evolve and heal itself over time.

Terminology

Knowledge Graph — 45× in this paper · explained in 7 episodes
Large Language Model — 11× in this paper
Clustering — 10× in this paper · explained in 2 episodes
Core — 10× in this paper · explained in 2 episodes
Grounding — 9× in this paper · explained in 3 episodes
Hallucination — 7× in this paper · explained in 4 episodes
Provenance — 7× in this paper · explained in 4 episodes
Agentic Reflection — 6× in this paper · explained in 1 episode
Consistency — 6× in this paper · explained in 3 episodes
Alignment — 5× in this paper · explained in 1 episode
Trace — 5× in this paper · explained in 2 episodes
Knowledge graphs — 4× in this paper · explained in 2 episodes
Downstream Tasks — 3× in this paper · explained in 2 episodes
Human Resources — 3× in this paper
Ontology — 3× in this paper · explained in 2 episodes
Abstraction — 2× in this paper · explained in 1 episode
Benchmark — 2× in this paper · explained in 2 episodes
EMMA — 2× in this paper · explained in 2 episodes
Global Wrong Guess Rate — 2× in this paper
Human-in-the-Loop — 2× in this paper · explained in 3 episodes
Interpretability — 2× in this paper · explained in 15 episodes
Large Language Model (LLM) — 2× in this paper · explained in 3 episodes
Large Language Models (LLMs) — 2× in this paper · explained in 6 episodes
Pairwise Comparisons — 2× in this paper · explained in 2 episodes
Redundancy — 2× in this paper · explained in 2 episodes
Structural Constraints — 2× in this paper · explained in 2 episodes
Benchmarking — 1× in this paper · explained in 4 episodes
Fidelity — 1× in this paper · explained in 2 episodes
Hybrid Approach — 1× in this paper · explained in 4 episodes
Iterative Refinement — 1× in this paper · explained in 3 episodes
Multi-Agent Framework — 1× in this paper · explained in 3 episodes
Multi-agent Collaboration — 1× in this paper · explained in 2 episodes
Paper Review — 1× in this paper · explained in 3 episodes
Reinforcement Learning — 1× in this paper · explained in 4 episodes
Robustness — 1× in this paper · explained in 5 episodes

Summary

The paper, authored by Emma Jouffroy, Warren Jouanneau, and Marc Palyart (Malt, Paris, France), addresses the challenge of organizing thousands of unstandardized, multilingual expertise declarations for Human Resources platforms, which directly impacts downstream tasks like talent matching. The authors identify that maintaining comprehensive, structured skills knowledge graphs is a persistent challenge for talent marketplaces such as Malt, the European freelancer marketplace, due to multilingual fragmentation, niche jargon, compound skills, and rapid temporal drift as occupations and work practices evolve.

The paper argues that while Large Language Models (LLMs) overcome the scalability issues of hand-crafted taxonomies, they suffer from a susceptibility to hallucinated skills, formatting instability, and a lack of grounding in verifiable metadata. Traditional generation methods fall into two paradigms, each with limitations:

Rigid top-down methods rely on predefined ontologies such as ESCO, which provide structural consistency and high precision but often struggle to capture emerging or highly localized concepts.
Purely generative bottom-up approaches are flexible for identifying novel trends but frequently lack structural constraints, causing semantically equivalent concepts to fragment into redundant clusters and noisy relationships to emerge.

The authors illustrate this via a project management narrative: a top-down approach misses emerging specializations like Web Project Management, while an unconstrained bottom-up approach causes semantic fragmentation and link hallucinations (e.g., wrongly connecting Excel).

To bridge this gap, the authors introduce a fully automated, agentic, hybrid pipeline that grounds an LLM in Wikidata, to ensure that the generated entities and their associated metadata remain verifiable and auditable, while leveraging an iterative multi-agent loop to capture, validate, and structure the emerging 'long-tail' skills that formal ontologies have not yet registered. The system uses Gemini 1.5 Flash, selected for its cost-efficiency and immediate availability within our infrastructure, constrained with strict schemas to prevent hallucinations while preserving linguistic variants and contextual structure.

The paper makes three main contributions:

Hybrid Skill Extraction: We combine structured knowledge graph integration with dynamic, bottom-up parsing to capture both established global standards and emerging long-tail expertise, enriching them with contextual metadata.
Multilingual Standardization: We enforce structural schema constraints on the underlying model to support consistent multilingual representations across five target languages and reduce hallucinated outputs.
Iterative Self-Refinement: "We introduce a reflection-based multi-agent loop in which agents iteratively analyze and correct intermediate outputs. This enables progressive consolidation of fragmented concepts and improves overall taxonomic coherence."

The system operates as an iterative loop rather than a strictly linear execution pipeline, processing batches through five distinct stages.

This phase maps noisy, unstandardized, and multilingual skill mentions to stable, unambiguous Wikidata entity identifiers (QIDs), which serve as the foundational anchor nodes. The system extracts the top ten Wikidata entries for each raw input alongside labels, descriptions, and regional variants across the five target languages. Because raw mentions are often ambiguous in isolation, the retrieval is structurally enriched with domain-specific semantic neighborhoods — specifically, from the Malt platform's user profile graph, the engine appends the top fifteen co-occurring peer skills and the top five overarching professional categories. The LLM then executes a disambiguation step and outputs a JSON payload containing the selected QIDs, confidence scores, step-by-step reasoning, and critical boolean flags (is skill and is compound). This dual validation approach, combining Wikidata's semantic recall with the LLM's context-aware precision, reduces potential hallucinated mappings.

This phase groups validated inputs by their unique QID combinations to generate human-readable, multilingual canonical names. The LLM synthesizes localized preferred labels across five target languages, using a strict fallback hierarchy: the model must prioritize empirical platform usage, fall back to official ontological titles, and synthesize novel labels only when necessary. Each label is explicitly tagged with its resulting provenance (malt, wikidata, or generative) to maintain algorithmic explicability.

An LLM-powered Curation agent validates the strict equivalence of every raw skill against its broader entity grouping and selected preferred label. If a skill is non-equivalent, the model outputs one of seven granular rejection criteria (AMBIGUOUS, SPECIALIZATION, SEMANTIC MISMATCH, NOT A SKILL, METHODOLOGY, CONTEXT, SUB TASK). Crucially, a suggested pref label is generated to define what the rejected concept should actually be named. The agent then re-evaluates and refines the core node's canonical label using strictly the accepted subset, bifurcating data into tightly curated baseline entities and a structured 'Orphan' queue. As a safeguard, any node with a rejection rate exceeding 50% is automatically flagged for human-in-the-loop review.

This phase merges overlapping sub-graphs to maintain global structural consistency across incremental runs. To avoid exhaustive pairwise comparisons, the system uses an asymmetric bootstrapping strategy: the first batch establishes the foundational graph, and subsequent nodes are compared against this growing baseline. Lightweight heuristics flag potential merges based on member intersection (shared raw skills) or lexical similarity (low edit distance between preferred labels), after which the LLM outputs a structural decision (MERGE or KEEP SEPARATE) and designates a surviving ID. Decisions are logged in a cache to prevent redundant re-evaluations in future Epochs.

This iterative loop primarily serves to fix granularity gaps inherited from the baseline anchor graph. For example, both Project Management and Web Project Management might map to the same broad Wikidata entity; the curation phase flags the latter as an outlier, but instead of discarding it, the pipeline captures it as an orphan. Each orphan is assigned a synthetic identifier derived directly from its LLM-generated suggested pref label, so that identical concepts receive the exact same suggested label from the model, naturally grouping matching specializations into a unified sub-graph in the subsequent Epoch. The loop repeats until the orphan queue is empty, achieving full semantic convergence.

The pipeline was evaluated on a proprietary dataset of unstructured, multilingual expertise declarations from the Malt freelancing platform, featuring a severe long-tail distribution of highly niche, emerging, or misspelled jargon.

The pipeline processed 36,037 raw expertise strings.
The reconciliation engine resolved 27,743 inputs, achieving a Global Coverage rate of 77%.
The remaining 8,294 unmapped entries were flagged as non-skills or semantic noise.
The 27,743 mapped variations were grouped into 15,010 semantic groupings, streamlined into 13,298 canonical skill nodes.
This represents a compression rate of 52.1% and an Average Skills per Node (ASpN) of 2.08 variations per canonical node.
The knowledge graph maintains 100% cross-lingual symmetry: all 13,298 canonical concept nodes are supported across all five locales (fr, en, de, nl, es), generating exactly 66,490 standardized preferred labels.
Empirical data shows a strong Pareto distribution: the top 1,000 canonical skills account for 82.74% of platform usage volume, and the top 5,000 capture 97.25%.
The system distinguishes between the lexical long-tail (which it aggressively compresses) and the semantic long-tail (which it systematically preserves through agentic reflection).

Evaluated against a hand-annotated gold standard independently curated by five domain experts, the pre-consolidation pipeline achieved:

Global baseline Alignment Coverage of 79.7% (proportion of inputs correctly mapped to the gold standard).
Found Coverage of 84.9% (precision on the subset where the model attempted retrieval).
Global Wrong Guess Rate (WGR) of 19.1%, which the authors note remains highly competitive and is mitigated by downstream curation.

Performance varied by domain: Highly structured domains like Video Games (91.8% Found Coverage) outperformed softer, more subjective fields like Communication (81% Found Coverage).

Across all 66,490 preferred labels: 80.65% of the knowledge graph is strictly anchored in factual, real-world data (comprising 67.28% empirical platform usage and 22.08% Wikidata titles). The generative engine synthesized only the remaining 19.35%. Structural coherence was measured using the Outlier Rate, yielding marginal error rates (0.01 to 0.06 outliers per sub-graph). The Active Curation agent successfully block[ed] 13.7% of rejected inputs via the NOT A SKILL filter.

The rejection reasons as percentage of total rejected inputs: SCORE REJECTED (31.9%), NODE CURATION (28.1%), NOT A SKILL (13.7%), WIKIDATA NOT FOUND (11.4%), NOT IN NODE (10.7%), ERROR NOT KNOWN (4.2%).

The authors position the pipeline as an initial step toward modeling the complexity of modern labor markets. A streamlined skill taxonomy extracted directly from this underlying knowledge graph, is already integrated into the platform's candidate-matching algorithms, with the graph providing an intermediate abstraction layer that significantly improves the interpretability and auditability of our automated matching systems.

The authors acknowledge limitations of relying on Wikidata as the primary anchor, including a lag in capturing niche HR jargon and structural inconsistencies due to its generalist nature, but argue the bottom-up orphan recovery loop acts as a safety net, autonomously structuring the emerging long-tail skills that Wikidata natively misses. They also flag potential representational biases: Large language and embedding-based models often exhibit English-centric tendencies, which can lead to over-normalization of non-English occupational structures, and note that in inflected languages like French and German, occupational terms are often gender-marked, requiring the graph to support the seamless mapping of these gendered variants to shared underlying occupational concepts, while preserving their distinct linguistic forms.

Future evaluation focuses on three areas: Extended Evaluations (benchmarking the consolidated graph against standards like ESCO and AI approaches like TnT-LLM or CLIMB), Cost and Scalability (measuring token consumption, processing latency, and optimization strategies), and Failure Analysis (tracking manual intervention rates, categorizing systematic mapping errors, and evaluating performance degradation under high-volume scenarios).

The paper concludes that the hybrid, multi-agent architecture creates a living knowledge graph capable of keeping pace with the rapid technological changes and linguistic shifts of the modern freelance market, noting that while the overarching framework operates as a rich, metadata-driven knowledge graph, its hierarchical output can be easily downstreamed as a clean, structured skills taxonomy. The supplementary material provides full prompt templates for all five stages (executed with Gemini 1.5 Flash at temperature 0 with structured output) to ease reproducibility, as the architecture is intrinsically dataset-agnostic and anchored to public Wikidata entities.

Improvements for AI systems

Based on the paper, here are specific improvements you can make to AI systems, followed by what the improved system would be able to do.

Ground LLM outputs in a public knowledge graph before generation.

Require the model to first map raw inputs to stable Wikidata QIDs (instead of generating free-form entities).
Enrich the retrieval context with semantic neighborhoods (co-occurring peer skills, overarching categories) to disambiguate short, ambiguous inputs.
Use a dual-validation design: Wikidata supplies recall, the LLM supplies context-aware precision. This directly reduces hallucinated entity mappings.

Use a strict structured-output schema with reasoning and confidence signals.

Constrain every agent to output JSON containing selected qid, confidence, step by step reasoning, is skill, and is compound.
Run with temperature 0 and full schema enforcement to prevent formatting drift and hallucinated metadata.

Add a curation/reflexion agent that validates equivalence before accepting a skill into the graph.

Instead of accepting every LLM-extracted entity, validate each raw skill against the broader grouping.
Categorize rejections with granular reasons: AMBIGUOUS, SPECIALIZATION, SEMANTIC MISMATCH, NOT A SKILL, METHODOLOGY, CONTEXT, SUB TASK.
Generate a suggested pref label for rejected items so the rejected concept can be re-processed instead of lost.

Implement an orphan-recovery iterative loop instead of discarding non-equivalent concepts.

When a skill is too specific for its anchor node (e.g., Web Project Management vs. Project Management), route it to an orphan queue.
Assign orphan IDs derived from the LLM's suggested label, enabling identical concepts to group naturally in the next epoch.
Loop until the orphan queue is empty, achieving convergence without losing long-tail skills.

Use a canonicalization step with explicit provenance tags.

Generate multilingual preferred labels using a strict fallback hierarchy: platform usage → official Wikidata titles → generative synthesis.
Tag every label as malt, wikidata, or generative, enabling auditability and explaining exactly where each label came from.

Use asymmetric cross-batch consolidation rather than exhaustive pairwise comparison.

The first batch builds a baseline graph; subsequent batches are compared only against that growing baseline.
Use lightweight heuristics (member intersection, edit distance) to flag merge candidates, then let an LLM output a structural decision (MERGE / KEEP SEPARATE) with a surviving ID.
Cache decisions to skip redundant evaluations in future runs.

Add automatic human-in-the-loop triggers based on rejection rates.

Flag any node whose rejection rate exceeds 50% for manual review, preventing systemic errors from propagating silently.

Preserve multilingual and gendered variants rather than over-normalizing to English.

Enforce cross-lingual support across all target locales.
Map inflected/gendered occupational terms (French, German) to shared underlying concepts while keeping distinct surface forms.

Use provenance-aware metrics during evaluation.

Track alignment coverage, found coverage, and wrong-guess rate against a hand-annotated gold standard.
Measure outlier rates per sub-graph to quantify structural coherence, not just entity accuracy.
Ingest thousands of noisy, unstandardized, multilingual expertise declarations and transform them into a clean, verifiable knowledge graph whose nodes are anchored to Wikidata QIDs.
Achieve high coverage and compression simultaneously. For example, processing 36k raw strings, the system can resolve about 77% of inputs, group mapped variations into 15k semantic groupings, and compress them into 13k canonical nodes, a 52% compression rate with an average of 2.08 variations per node.
Maintain 100% cross-lingual symmetry. If the graph has 13,298 canonical nodes, it will generate exactly 66,490 standardized preferred labels across five locales (fr, en, de, nl, es), with no missing language variants.
Distinguish between lexical long-tail and semantic long-tail. It aggressively merges lexical variations and typo variants, while systematically preserving genuinely novel or niche skills through the orphan-recovery loop.
Reduce hallucinated mappings to a controlled level. In the paper's evaluation, the system achieved 79.7% alignment coverage and 84.9% found coverage with a wrong-guess rate of 19.1%, which is then mitigated by the curation stage.
Keep the knowledge graph auditable. 80.65% of all generated labels are anchored in factual data (empirical platform usage or Wikidata titles), with only 19.35% synthetically generated, and every label carries an explicit provenance tag.
Block non-skill noise automatically. A NOT A SKILL filter rejects 13.7% of invalid inputs, preventing irrelevant concepts from entering the taxonomy.
Recover emerging specializations that formal ontologies miss. At the same time, it aligns established concepts with global standards like Wikidata, making the graph useful for both bottom-up trend detection and top-down structural consistency.
Self-refine across iterations. The reflection loop progressively consolidates fragmented concepts, fixes granularity gaps inherited from the baseline anchor graph, and converges when the orphan queue is empty.
Support downstream applications directly. The resulting knowledge graph can be exported as a clean, hierarchical skills taxonomy for candidate-matching algorithms, while retaining rich metadata for interpretability and auditability.

Abstract

Organizing thousands of unstandardized, multilingual expertise declarations is a persistent challenge for Human Resources (HR) platforms, directly impacting downstream tasks like accurate talent matching. To address this, we propose a hybrid knowledge graph generation pipeline that grounds a Large Language Model (LLM) in the Wikidata multilingual Knowledge Graph (KG) while employing an agentic reflexion pattern to synthesize emerging concepts and their associated metadata. Unlike rigid top-down methods or fragmented bottom-up approaches, our system anchors recognized concepts to stable Knowledge Graph entities while dynamically creating new nodes and relational metadata for unrecognized skills. Executed across five stages, entity reconciliation, multilingual canonicalization, active curation, deduplication, and the iterative recovery of unmapped concepts, the system autonomously adapts to rapidly evolving, noisy skill mentions across five European languages. Ultimately, this pipeline provides a highly scalable, explicable, and self-healing framework for generating a comprehensive skills knowledge graph, from which a structured taxonomy is derived, using unstructured, noisy text.