KITE: Decoupling Kinematics and Interaction for Zero-Shot Cross-Embodiment Manipulation

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "KITE: Decoupling Kinematics and Interaction for Zero-Shot Cross-Embodiment Manipulation".

Dev: Here's the gist of the paper. The gist The KITE framework introduces Kinematic Interaction Transfer across Embodiments, which decouples manipulation into embodiment-agnostic task reasoning and embodiment-specific motor control via a learned latent representation of interaction intent.

Rosa: How it works KITE decouples manipulation into two complementary problems connected by a contact-based interface: embodiment-agnostic task reasoning and embodiment-specific motor control The former is implemented by a shared policy learned from source demonstration data, while the latter is implemented by an intent-conditioned action decoder learned from each embodiment’s kinematic model Task reasoning is performed by a shared policy that predicts latent intents from source demonstrations, while motor control is performed by an intent-conditioned action decoder learned from its kinematic model The core mechanism involves defining a contact-based interaction intent as a sparse set of contacts ct = (xt,i, nt,i) specifying the intended robot-object interaction without committing to any specific joint or end-effector This intent is then mapped to a fixed-dimensional latent intent zt = ψ(ct) ∈ Z via a permutation-invariant encoder A policy π predicts the next latent intent from observation, and an embodiment-specific action decoder g e produces the next motor command a e t = g e(zt, qe t), where zt−1 is the previously predicted latent and qe t is the current configuration of embodiment e Embodiment-Specific Learning of Motor Control from Kinematics The decoder g e is trained without any task demonstrations on body e, aiming to learn a state-based generalist that maps an interaction intent to a configuration target for one specific body This training data is synthesized using the kinematic model β e alone, generating intent-configuration pairs De g where the goal configuration qgoal is guided by a joint mask m induced by S The mask forces each latent in Z to commit to a specific set of joints, ensuring that each latent carries consistent meaning across bodies The training objective for the decoder is Lg = X e E(c, qinit, qgoal)∼De g h∥g e(ψ(c), qinit) − qgoal∥22i Embodiment-Agnostic Learning of Task Reasoning from Demonstrations The policy π is trained as a standard imitation-learning problem in Z, where the action dimension is set to dim(Z) and the native proprioception input is replaced by the previous latent zt−1 Training data for π comes from source demonstrations preprocessed into latent intents, where trajectory-level contact points are aggregated into a fixed set of trajectory-level contact points The frozen encoder ψ converts each ct into a latent target zt = ψ(ct), yielding training triples (ot, zt−1, zt) The policy π is then trained with the base policy’s native loss on the latent targets zt in place of its native action targets Zero-Shot Cross-Embodiment Transfer and Evaluation KITE enables adaptation to a new embodiment without needing to recollect demonstration data, as adaptation requires only training a new action decoder using its kinematic model The framework is validated on three manipulation tasks spanning transfer between parallel grippers, dexterous hands, and composite embodiments KITE consistently achieves zero-shot transfer to structurally different target embodiments, outperforming state-of-the-art baselines in transfer success and task embodiment scope The paper further demonstrates that the latent intent is an embodiment-agnostic demonstration signal, as shown by comparing policies trained from Wuji demonstrations versus Allegro demonstrations for bottle pumping The results confirm that the source embodiment is largely abstracted away, confirming that the latent intent is an embodiment-agnostic demonstration signal.

Dev: The framework's success on physical hardware confirms zero-shot cross-embodiment transfer on real hardware under real perception and physics, reaching 7/10 to 8/10 on all three tasks.

Rosa: The central claim is that the latent intent is an embodiment-agnostic demonstration signal, as shown by comparing policies trained from Wuji demonstrations versus Allegro demonstrations for bottle pumping.

Paper summary: Rosa: So we’re looking at KITE today. It tackles that big problem where a robot policy you train on one type of arm just doesn't work on a completely different shaped arm without redoing all the training.

Dev: That’s the core issue, right? The paper introduces Kinematic Interaction Transfer across Embodiments.

Rosa: Exactly. The thesis is that we can decouple manipulation into two parts: figuring out what the task *is*, regardless of how you hold it, and figuring out how to *move* the specific robot body to achieve that task.

Dev: So it separates task reasoning from motor control entirely?

Taro: That sounds like a clean split. But I wonder if that decoupling holds up when things get messy in the real world.

Rosa: Well, the paper claims this is done through a learned latent representation of interaction intent based on contact patterns.

Dev: So instead of a policy learning direct actions, it learns something abstract first, and then an action decoder translates that abstract thing into motor commands for that specific robot.

Rosa: That’s right. The task reasoning part is handled by a shared policy that predicts these latent intents from the source demonstrations you have.

Dev: And the motor control part, the action decoder, is trained on each embodiment's kinematic model alone to realize those intents.

Taro: The idea that the decoder learns how *that body* achieves an interaction intent from just its kinematic model is interesting because it means we don't need demonstrations for every single robot shape.

Rosa: That’s the zero-shot part of it. You train on a source robot, and then you can deploy that skill on a target robot with only its kinematic model, no new task data needed.

Dev: The paper says adaptation to a new embodiment requires only training a new action decoder using its kinematic model without recollecting demonstration data one <ref:2606.22113#pg1,adaptation to a new embodiment requires only training a new action decoder>. That’s the key claim there.

Rosa: It’s pretty significant because it means task knowledge could be collected once and reused across hardware.

Taro: I like that idea of the latent intent being an embodiment-agnostic signal, even when comparing different types of demonstrations, like Wuji versus Allegro demonstrations for bottle pumping Rosa That suggests the underlying interaction is what matters most.

Dev: But we gotta be careful about how that intent is defined. The paper defines it as a sparse set of contacts, which are specific points like x t i n t i, without committing to any specific joint or end-effector Rosa So the policy learns this contact pattern first, and then the decoder figures out how to move the joints for that pattern on a given body.

Taro: And they use a permutation-invariant encoder A policy that predicts the next latent intent from observation, and an embodiment-specific action decoder g e produces the next motor command a e t = g e(z t, qe t) Rosa That structure is what makes it work across different kinematics.

Paper summary: Rosa: So how does the motor decoder learn without task demonstrations on the target body?

Dev: It’s trained using synthetic data generated from each embodiment's kinematic model alone to create intent-configuration pairs.

Taro: And that configuration target qgoal is guided by a joint mask m induced by S, which forces each latent in Z to commit to a specific set of joints across bodies.

Dev: The training objective for the decoder is L g = X e E(c, qinit, qgoal)∼De g h∥g e(ψ(c), qinit) − qgoal∥22i Rosa That’s how they ensure the latent intent carries consistent meaning across different robot shapes.

Taro: I'm curious about the performance numbers. The paper evaluates KITE on four manipulation tasks spanning parallel grippers, dexterous hands, and composite embodiments Rosa And it says it consistently outperforms state-of-the-art baselines in transfer success and task embodiment scope Dev Which is pretty strong for zero-shot transfer.

Rosa: It’s also interesting that they show the latent intent is an embodiment-agnostic demonstration signal by comparing policies trained from Wuji demonstrations versus Allegro demonstrations for bottle pumping Dev That really confirms the source robot is largely abstracted away in the learning process.

Taro: It extends beyond robot-to-robot transfer, too. They show success on human-hand source keyboard pressing and bottle pumping Rosa So this framework is being tested across different types of interaction sources, not just robot to robot.

Dev: The paper flags a limitation in that the method doesn't solve every problem in real hardware. They note that the success on physical hardware reaches seven/ten to eight/ten on all three tasks Rosa It’s good, but it’s not perfect yet for everything <ref:2606.22113#pg1>.

Rosa: So what does this mean for us who are just watching? It suggests that we should think about cross-embodiment generalization not just as a policy scaling problem, but as a representation problem Dev A way to learn an intent signal that is truly independent of the physical body you happen to be using.

Taro: I think the implication is that if we can find this shared latent intent, we don't have to relearn everything when we switch hardware. It shifts where we focus our effort in robot learning.

Rosa: So, to wrap up on KITE: it’s about decoupling the task from the kinematics using a contact-based interface and a learned intent representation Dev We looked at how this works and what it means for future work.

Rosa: The authors call their approach Kinematic Interaction Transfer across Embodiments Dev It shows how shared latent intents can bridge different robot morphologies with zero additional task demonstrations.

Taro: It’s a solid framework for moving beyond hardware-specific learning bottlenecks in manipulation tasks, showing that the intent signal is robust across different physical systems.

Dev: The work gives us a clear path forward for reusing learned skills more broadly across diverse robotic platforms.

Conclusion: Rosa: So we've been looking at KITE today, which is about decoupling kinematics and interaction for zero-shot cross-embodiment manipulation.

Dev: Yeah, that's the core idea here, separating what the task is from how the robot body actually moves to do it.

Taro: So it means you don't have to retrain a policy every time you switch to a different robot shape if the underlying interaction intent stays similar.

Rosa: Exactly. The paper, KITE, shows how they use a learned latent representation of interaction intent based on contact patterns.

Dev: It’s about training one shared policy for the task reasoning and then using an embodiment-specific decoder to handle the motor control for that specific robot body.

Taro: I'm interested in how this translates to real testing, because my concern is what happens when the world misbehaves outside a clean lab setup.

Rosa: The authors validate this on four tasks across parallel grippers, dexterous hands, and composite embodiments in both simulation and the real world.

Dev: And they report that KITE consistently achieves zero-shot transfer to structurally different target embodiments, outperforming previous baselines.

Taro: It’s important that they show it works on real hardware under real perception and physics because I'm always skeptical about lab-to-real gaps.

Rosa: They do, and they find it reaches a seven/ten to eight/ten score on all three tasks when tested in the physical world.

Dev: The main result is that this latent intent signal is an embodiment-agnostic demonstration signal, which means the source robot is largely abstracted away.

Taro: So if you can capture that intent as a fixed-dimensional vector, it doesn't matter if the robot arm has twenty degrees of freedom or forty.

Rosa: That's right. It suggests that cross-embodiment generalization should be treated more like a representation problem than just policy scaling.

Dev: It means we focus on learning that shared intent signal instead of trying to learn every possible joint movement for every single robot shape individually.

Taro: What this implies is that the effort moves from mapping physical actions to understanding the underlying contact interaction itself.

Rosa: It’s a big idea for how we approach generalization in robotics and AI systems.