Skill Constellations: Tracing the Supply Chain of Agent Skills on GitHub

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia, and with me are Elias and Priya, guest researcher.

Elias: Today's paper: "Skill Constellations: Tracing the Supply Chain of Agent Skills on GitHub".

Nadia: The gist: The authors contribute the first dated copy network of agent skills,

Elias: First, who's behind it and why it matters.

Title and authors: Nadia: We're diving deeper into Skill Constellations: Tracing the Supply Chain of Agent Skills on GitHub and who actually put this research together. It’s important to know who we’re listening to when we talk about these supply chain challenges.

Elias: The paper is written by Fahd Seddik from the University of British Columbia, and it covers a huge dataset: two million one hundred ninety-three thousand one hundred nineteen skill adoptions across GitHub. It sets out to reconstruct that missing graph of skill spread using the git history of every SKILL.md file.

Priya: So, the sheer scale of this data collection is what makes this study significant; they built a comprehensive map from scratch instead of looking at just a small snapshot in time.

Nadia: It’s not just about counting skills; it's about mapping the directed network—how one skill leads to another through copying—which provides a dated view of that entire flow.

Elias: They focused on the "software supply chain without a registry, versions or provenance" problem that we all face when AI agents run scripts copied between repositories.

Priya: So, the main contribution here is this first dated and directed network of how agent skills spread, moving beyond static views to show the dynamic movement.

Nadia: And that network lets us see if we can identify high-risk sources or effective audit targets before dangerous capabilities get widely distributed through these copies.

Elias: The implication for us is that we need a way to track this flow because right now, the lack of tracking means we don't know the origin of a copied skill or the reach of any security fix.

Priya: It moves the conversation from just looking at individual skills to understanding the entire system dynamics of how those skills are being reproduced and adopted.

The paper's summary: Nadia: So, to summarize what this paper, Skill Constellations: Tracing the Supply Chain of Agent Skills on GitHub, is actually doing, it’s taking all that raw git data and building a massive network graph of skill adoptions.

Elias: They are essentially reconstructing the missing link in the software supply chain where skills are shared by copying them between repositories without any formal tracking mechanisms like version control or provenance records.

Priya: What they found is that these skills organize themselves into communities, with fifty communities, and twelve of those labeled by topic <ref:2610.11169#pg1>. This shows us how these agent functions cluster together based on what they actually do.

Nadia: And the core mechanism driving this spread isn't just one person adopting a skill; it’s bulk adoption where copying is the dominant force at thirty-three point three percent, compared to only eight point one percent from individual adoption <ref:2610.11169#pg2>.

Elias: They also found specific patterns in transmission, showing that while most adoptions are single copies, there's a high level of overdispersion in how many times a skill gets transmitted after being adopted; the dispersion coefficient k is zero point zero eight six <ref:2610.11169#pg2>.

Priya: From a measurement standpoint, they also looked at popularity metrics like GitHub stars and found they don't correlate well with the actual out-degree of the skills or predict future sources accurately.

Nadia: They found that current copy out-degree actually captures four times more later transmissions than stars do, which is a big signal for where we should be focusing our attention instead of just looking at how many stars a project has <ref:2610.11169#pg3>.

The paper's improvements: Elias: Now, the authors don't just present this snapshot; they suggest several ways to improve auditing and risk assessment based on what they observed in the network. They are proposing specific strategies for security engineers to use with this data.

Nadia: One key improvement they suggest is focusing audits on a specific subset of repositories, saying that auditing the one hundred repositories that the source-choice model scores highest prevents fourteen point nine percent of later adoptions of high-risk skills compared to only zero point five percent for the top starred repositories.

Priya: That’s a practical suggestion because it allows security teams to short list potential sources to check before a skill actually spreads widely, which is much more effective than relying on just size or popularity metrics like stars.

Elias: They also push for platform vendors to distribute skills as versioned references instead of just copies, because the paper shows that copies don't reliably follow their original sources.

Nadia: That ties directly into the risk analysis because when you have versioned references, a fix at the source is much more likely to reach those copies consistently across time windows.

Priya: And they propose using a conditional logit model, which they call the Source-Choice Model for Risk Ranking, to fit how repositories choose their sources by looking at things like logarithms of skills held and past copies.

Elias: That model is superior because it ranks repositories based on source-level measures rather than just authorship, which describes the distribution of where things are coming from.

Nadia: We also looked at how modified copies gain functionality, and they found that this capability gain usually comes from adopting another version of the skill, not necessarily from the edits made by their original owner.

Conclusion: Priya: So wrapping up, the main implication of Skill Constellations is that we need to move away from static snapshots when assessing AI software supply chains and instead look at how these skills are dynamically spreading through copying.

Elias: The paper's conclusion boils down to this: platform vendors should distribute skills as versioned references because copies don't reliably follow their sources, and reviewers should rank repositories by current copy out-degree rather than relying on GitHub stars.

Nadia: That’s the practical shift for security engineers—using dynamic metrics like copy out-degree gives us a better signal about where to focus our review efforts before dangerous capabilities spread widely through these copied skills.

Elias: And they suggest building a flagging mechanism based on high-risk patterns, like skills that bundle executables or pre-approve tools, using the precision and recall achieved by Claude Code to flag skills with ninety-eight point three percent precision and eighty point nine percent recall <ref:2610.11169#pg2>.

Priya: I just want to emphasize that while this map is powerful, the authors admit a limitation: they didn't track every single change consistently across all time windows; only four percent of genealogies ever show consistent changes, which means our model has some noise in it <ref:2610.11169#pg3>.

Nadia: So we have a better picture of the supply chain dynamics, but we still need to be careful because the tracking of consistent changes isn't perfect.

Elias: That’s right; Skill Constellations gives us a map to navigate this supply chain, but it points us toward versioning and dynamic metrics as the best way forward for managing risk in these AI-driven development environments.