Behavioral Persistence and Incomplete Functional Transfer of Co-evolved Communication in Evolutionary Robotics

arXiv:2609.38527 · cs.RO, cs.NE · Submitted 2026-09-29 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa, and with me are Dev and Taro, guest researcher.

Dev: Today's paper: "Behavioral Persistence and Incomplete Functional Transfer of Co-evolved Communication in Evolutionary Robotics".

Rosa: This work evaluates whether a co-evolved communication protocol can be directly transferred from a 2D simulation to a 3D physical environment without retraining the network weights,

Dev: First, who's behind it and why it matters.

Paper summary: Rosa: To summarize "Behavioral Persistence and Incomplete Functional Transfer of Co-evolved Communication in Evolutionary Robotics," the core thesis is that a communication protocol co-evolved between two agents in a 2D simulation cannot be directly transferred to a three dee physical environment without retraining the network weights. The paper claims that while some behavioral elements persist, the full functional transfer is only possible if you meticulously preserve the specific ecological and navigational conditions under which that protocol originally evolved.

Dev: Essentially, they are showing that when you try to move a controller from a simulation to a real physical world—like moving from Pygame to PyBullet—the translation layer requires specific corrections, such as calibrating terms based on measurable asymmetries in the trained residual weights. This is important because it shows that the learned behavior isn't just copied; it's deeply tied to the environment of its creation.

Taro: The main takeaway here is that this isn't just a simple reality gap problem where the simulation looks different from reality; it’s more about how the system interprets cues like hunger and proximity when those physical dynamics change, which has implications for how we design autonomous systems that learn on the fly.

Rosa: That's right; it matters because they demonstrated an asymmetric transfer outcome: one agent succeeded in reaching the food source in one of thirty tested seeds, while the other failed to reach it in any of those scenarios. This asymmetry shows that you can't assume a uniform success rate when moving these learned protocols between environments without careful consideration.

Dev: From a loop rate perspective, this highlights that even if we get the communication structure right, if the underlying physical dynamics introduce unforeseen friction or sliding—like what happened with the turn actions causing sliding because of constant forward momentum—the learned behavior breaks down immediately.

Taro: I wonder how often these types of failures happen in real-world deployments; are we looking at constant, small degradations, or are we seeing catastrophic failures when the physical constraints deviate significantly from the training setup?

Rosa: They are seeing scenarios where the deviation is significant enough to cause complete failure for one agent across all thirty seeds tested, which suggests that environmental shifts can have a very hard cutoff point for protocol success.

Conclusion: Dev: When we look at the conclusion of "Behavioral Persistence and Incomplete Functional Transfer of Co-evolved Communication in Evolutionary Robotics," the authors really underscore that you can’t just assume protocols are transferable across different physical contexts without addressing those underlying spatial coding issues they found. The asymmetry between Agent A and Agent B confirms that this isn't a general problem for all transfers, but depends entirely on the specific interaction between the learned protocol and its new physical constraints.

Rosa: Exactly; it brings us back to the idea that when we deploy these systems outside of a pristine lab setting, we can't just rely on the initial training data being sufficient because the underlying interpretation of sensory input gets tied into that specific physical context. It means we need to think about how much physical context is actually encoded in the communication signal versus how much is left for the agent to learn on its own when things get messy.

Taro: This has major implications for autonomy research, suggesting that simply improving the communication channel's information content isn't enough if the receiving agent can't correctly map that signal onto a new set of physical dynamics or perceive obstacles differently. It points toward needing world models to bridge that gap in sensorimotor correspondence during transfer.

Dev: So, for us as control engineers, this means we have to be hyper-aware of those residual connections and how agents prioritize cues like fear over hunger when the physical setup changes; otherwise, the system might operate on a fundamentally flawed assumption about its own needs in the new environment.

Rosa: It really highlights that the transferability of these co-evolved protocols should not be automatically assumed when the spatial and sensorimotor conditions of the environment change, which is a big caution for anyone thinking about deploying learned behaviors into physical systems. We need to be much more rigorous about validating those environmental dependencies before we push them into real-world scenarios.

Fernando Montes-Gonzalez

Instituto de Investigaciones en Inteligencia Artificial, Universidad Veracruzana

cs.RO, cs.NE

Submitted: 2026-09-29

Updated: 2026-09-29

Comments: 13 pages, 3 figures, 2 tables. Source code and data available at Zenodo

Code: https://github.com/ferdiex/essim

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 73/100

The gist: This work evaluates whether a co-evolved communication protocol can be directly transferred from a 2D simulation to a 3D physical environment without retraining the network weights, revealing that

Key concepts

Co-evolved Communication Protocol
This is a set of learned rules or signals developed by two agents interacting together in a simulation. They learn to coordinate their actions, like seeking food, based on shared signals they exchange with each other.
Direct Transfer
The attempt to move the trained brain (network weights) from the original 2D software into a new 3D physical robot without retraining it. The study tested if this direct transfer would result in successful behavior in the new environment.
Residual Connection
A specific part of the neural network architecture that allows information to flow directly from one layer to another, bypassing some standard processing steps. In this study, its importance varied between agents, suggesting different reliance on different types of input signals.
Implicit Spatial Coding
The idea that the protocol's success isn't just in the signals themselves but also in how those signals relate to the physical space where they were learned. The successful transfer suggests that information about the environment is encoded physically, not just digitally.

Terminology

Summary

This work evaluates whether a co-evolved communication protocol can be directly transferred from a 2D simulation to a 3D physical environment without retraining the network weights, revealing that while some behavioral elements persist, full functional transfer is contingent upon preserving the ecological and navigational conditions under which the protocol evolved.

The gist

One agent reached the food source in one of thirty tested seeds, while the other did not reach it in any.

How it works

The study utilized two e-puck-type robots controlled by a GRU network with a residual connection, evaluated in a food-seeking task involving social signaling. The protocol was developed through coevolution within a 2D environment using Pygame, where agents learned to coordinate behavior based on shared signals. The core contribution involves assessing the direct transfer of these trained weights into a 3D physics-based environment implemented in PyBullet, without retraining the network.

The controller architecture for each robot is a RES-GRU network with eleven input units and sixteen hidden units. The input vector comprises eight proximity readings, the relative bearing to the food source, a hunger term (calculated as the exponential decay of distance to food), and a binary flag indicating if the robot is stopped. The network output selects one of five discrete actions: forward, turn left, turn right, reverse, or signaling action. A direct residual link maps this input vector to the output logits.

Adaptations for 3D Transfer

The direct transfer required three specific corrections to ensure stable physical operation in the 3D environment:

  1. The mapping of turn actions to wheel speeds was corrected because the original 2D implementation produced pure rotation, whereas the new 3D setup caused sliding along walls due to maintaining a constant forward component on both wheels during turns. This required reducing the residual forward momentum present during turn maneuvers.

  2. The implementation of the filter regulating social signal injection into the front sensor input was revised and corrected.

  3. The hunger term in input x[9] was recalibrated for one agent based on an analysis of residual weights, which showed Agent B's residual connection exhibited a greater relative importance to proximity cues associated with obstacle avoidance (interpreted as fear) than the hunger term. To compensate without altering evolved weights, the divisor of the exponential decay determining x[9] was adjusted; specifically, for Agent B, this decay value was set to 1.25.

Experimental Protocol and Metrics

The evaluation involved running episodes under a fixed seed for each condition to control initial positions and orientations. An episode concluded when both agents reached the food source (Euclidean distance less than 0.08 meters) or upon reaching a maximum simulation step limit. Key metrics recorded included:

saw flip rate:

This measures how frequently the hunger term alternates between significant increases and decreases between consecutive steps, capturing short-term oscillatory patterns.

saw cycle rate:

This counts complete oscillation cycles, defined as two consecutive sign changes that return the trend to the initial state.

magnet max:

Defined as the maximum value of the hunger term x[9] during a run, this provides a summary measure of proximity achieved by an agent, with values close to 1 indicating proximity to food.

Analysis of Transfer Limitations

The results revealed an asymmetric transfer outcome: Agent A reached the food source in one seed, while Agent B failed on all thirty seeds. Analysis showed that recalibrating the hunger term via the residual pathway was insufficient to restore full functional transfer for Agent B. Furthermore, an experiment incorporating explicit directional information into the social channel—replacing x[8] with a relative bearing toward the sending agent when hunger was below 0.6—produced observable changes in magnet max in several seeds but failed to allow Agent B to reach the food source successfully. This suggests that the bottleneck is not solely explained by signal translation, but also by the ability to navigate under the new physical constraints.

Discussion and Implications

The paper proposes two primary explanations for the partial transfer:

  1. The asymmetry in evolved weights: Agent B's residual connection shows a greater relative importance to obstacle avoidance cues (fear) than hunger, a relationship approximately 1.78 times stronger in Agent B than in Agent A, which may be a limiting factor under new physical dynamics.

  2. Implicit spatial coding: The protocol may depend on the physical context where it was learned; the information might not reside solely in the signal but also in the physical context of evolution, analogous to sensorimotor correspondence stability during transfer.

The findings suggest that the transferability of a co-evolved protocol should not be automatically assumed when the spatial and sensorimotor conditions of the environment change. Future work is suggested to explore retraining only the receiving agent or employing world models as an intermediate layer for latent space refinement.

Improvements for AI systems

Based on the scientific paper, here are specific improvements for AI systems derived from its findings:

  1. Improve sim-to-real transfer robustness for co-evolved social protocols by incorporating a mechanism to preserve implicit spatial coding.

  2. Develop a controller adaptation layer that dynamically recalibrates internal state terms (like the hunger term) based on residual weight asymmetries detected during transfer, rather than relying solely on fixed scaling factors.

  3. Implement an explicit directional information injection module within the social communication channel, but design it to be context-aware—only injecting directional cues when the agent's current navigational uncertainty (e.g., low hunger term) exceeds a specific threshold, rather than universally.

  4. Utilize latent space world models as an intermediate layer between simulation and physical environments to allow agents to practice coordination within a simplified model of the target environment before interacting with the full physics engine, mitigating the reality gap.

  5. Design multi-agent reinforcement learning (MARL) architectures that explicitly decouple the transferability of low-level signaling mechanisms from the environmental constraints that shape high-level behavioral coordination.

These improved AI systems can:

  1. Successfully transfer co-evolved communication protocols between 2D and 3D simulation environments with higher fidelity, ensuring both agents reach a common goal (e.g., food source) under new physical dynamics.

  2. Maintain functional reciprocity in social behaviors even when the underlying physical constraints change significantly, by adapting internal weights to account for environmental asymmetries (like fear vs. hunger trade-offs).

  3. Navigate complex, high-dimensional physical spaces effectively by leveraging learned world models that bridge the gap between simulated and real sensorimotor correspondences.

  4. Perform robust multi-agent foraging or cooperative tasks in novel physical settings where the communication protocol was not explicitly trained for those specific dynamics, by dynamically modulating signal content based on perceived navigational difficulty.

Abstract

This work evaluates the direct transfer of a co-evolved communication protocol from a 2D simulation to a 3D physical environment, without retraining the network weights. Two e-puck-type robots, controlled by a GRU network with residual connection, were evaluated in a food-seeking task with social signaling. The sensory and motor translation layer required three corrections for stable physical operation, including the calibration of a hunger term based on a measurable asymmetry in the trained residual weights. Even with these corrections, the transfer was partial and asymmetric: one agent reached the food source in one of thirty tested seeds, while the other did not reach it in any. Task success was measured by both agents reaching the food area. An additional experiment incorporating explicit directional information in the social channel produced observable changes in the trajectory of the receiving agent and improvements in several specific cases. However, these improvements were not enough to allow the second agent to reach the food source, suggesting that the limitation may not be explained solely by signal translation, but also by the ability to navigate under the new physical constraints. The results suggest that successful transfer of emergent communication may depend not only on preserving the signaling process itself, but also on preserving the ecological and navigational conditions under which the protocol evolved.

Sources

Related papers