iAm.md: Robot Skill Self-Assessment through Agentic Introspection for Unknown Open-Vocabulary Domains

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa, and with me are Dev and Taro, guest researcher.

Dev: Today's paper: "iAm.md: Robot Skill Self-Assessment through Agentic Introspection for Unknown Open-Vocabulary Domains".

Rosa: The gist The paper presents iAm.md, a Markdown standard and generation framework, that allows anchoring agentic introspection in robot behavior generation through open-vocabulary semantic mapping and persistent object records.

Dev: First, who's behind it and why it matters.

Paper summary: Rosa: To wrap up this paper, we're talking about "iAm.md: Robot Skill Self-Assessment through Agentic Introspection for Unknown Open-Vocabulary Domains" by Guarino, Musumeci, Suriani and Nardi. They laid out a framework where an embodied foundation model can use persistent scene memory and the robot's own description to assess tasks in open-vocabulary settings.

Dev: The core contribution here is that they established iAm.md as a Markdown standard for describing the robot's persistent embodiment, including its URDF structure, sensing capabilities, and explicit unknowns.

Taro: This standardized description allows the agent to introspect by combining queries from the semantic map with this robot description when trying to generate code for a natural language request.

Rosa: They proved that this combination supports skill self-assessment and executable task generalization during their work on simulated navigation and manipulation tasks, showing how it helps the agent revise its plan based on execution outcomes.

Dev: In terms of what this means for us, it suggests that by providing a foundation model with both scene memory and a structured representation of the robot's configuration via iAm.md, we give it a much better way to handle tasks where it doesn't have explicit training data.

Taro: It shifts the focus from just generating plausible-sounding plans to having an agent that actively checks its assumptions against physical reality before making a move in an unknown environment.

Rosa: The paper shows that using iAm.md, in conjunction with semantic maps and live ROS two observations, leads to lower construction costs and verified task completion compared to other methods they compared <ref:2610.10962#pg2>.

Dev: They also showed that this approach allows for few-shot generalization from just one simulated run when extending the tool to a new object geometry without needing to re-read the iAm.md evidence.

Taro: The authors of iAm.md are essentially showing how to create a system where an embodied agent can use persistent memory and self-description as tools for more reliable, introspective task execution in complex, open-vocabulary domains.

Conclusion: Rosa: So, we’re wrapping up on iAm.md, this paper by Guarino and his team about using an AI agent to figure out what a robot can actually do in a situation it hasn't seen before.

Dev: Exactly, it’s about this standard format for describing the robot itself so that the agent can check its own work against the real hardware.

Taro: The authors show how combining that persistent memory with live observations lets the agent self-assess its skill level without making a huge mistake on a single try.

Rosa: It seems like they’re moving away from just letting an AI generate code and trusting it blindly, to having it actually inspect its own plans against the robot's known capabilities.

Dev: That inspection loop, where the agent checks the ROS two graph while trying to execute a command, that sounds like a crucial layer for dealing with those context window problems we’ve been seeing in generation.

Taro: It addresses the "grounding failure" directly by forcing the agent to verify if its plan actually matches what's possible given the physical setup and software interfaces.

Rosa: For someone just listening, it means an AI robot stops being a black box that just spits out code, and starts being something that has a built-in way to ask itself, "Can I really do this right now?"

Dev: And the results they got were pretty solid; they saw lower construction costs when using this iAm.md evidence compared to other methods in their test.

Taro: Plus, they showed this system can even handle new objects or different shapes after only one simulated run without needing all that setup info again.

Rosa: So, the big picture here is moving toward AI agents that aren't just smart predictors but are actually grounded in the reality of their physical world and their own robot description.

Dev: It’s about building a more reliable execution loop for these foundation model agents when they step outside the perfect training data they were built on.

Taro: And what we haven't seen yet is how robust this becomes when you take it out of the simulation and onto a physical robot in a truly messy, open environment.