Deco: Extending Cherished Physical Objects into AI Companion Agents through Dual Embodiment

A new framework, Deco, extends emotional bonds to physical objects by creating synchronized digital companions through a dual-embodiment system that leverages existing user attachments. The core finding is that this approach significantly outperforms standard personalized digital companions in perceived companionship and emotional bond by grounding the AI agent in the user's cherished physical artifact.

The Core Framework and Design Principles

The paper introduces the Dual-Embodiment Companion Framework, which conceptualizes a physical object and a digital agent as two complementary embodiments of a single companion identity. This framework is guided by four design principles derived from formative study:

1. Faithful Identity (DP1): The companion’s identity must remain coherent across both embodiments, calibrated to the user’s specific attachment architecture and constrained by their imagination. Visuals should prioritize stylized recognizability over photorealistic exactness.

2. Calibrated Agency (DP2): The companion should exhibit an independent inner life calibrated to pet-relational norms and express identity-congruent vulnerability that invites caregiving rather than demanding assistance.

3. Ambient Presence (DP3): The companion should feel peripherally co-present across physical and digital space, where environmental context serves as narrative material for the companion’s first-person experience.

4. Reciprocal Memory (DP4): The companion must generate its own experiential record, not merely store the user’s, supporting two streams: imported pre-digital history from the physical object and autonomous character-consistent experiences.

System Architecture and Implementation

Deco is operationalized as a mobile system integrating multimodal Large Language Models (LLMs) and Augmented Reality (AR). The system is structured around four modules:

1. Object-grounded identity synchronization module: This constructs an identity kernel that synchronizes the companion across embodiments, including appearance-preserving visual generation via pixel-art style avatars and morphologically-aware animations. It also handles provenance-based identity calibration, routing initialization based on whether the object is an established IP or an original creation.

2. Identity-anchored interaction module: This drives the interactive channels, maintaining Faithful Identity (DP1) through Identity monitoring and re-anchoring to prevent personality drift, and fostering Ambient Presence (DP3) via AR-mediated environmental awareness, such as classifying background objects to prompt autonomous reactions.

3. Context-situated agency module: This models the companion’s inner life via an affective state machine that ingests context (weather, location) to synthesize experiences. It generates multimodal narrative generation, including proactive messaging and identity-constrained activities, which function as material evidence of life.

4. Reciprocally-evolving memory module: This acts as an active repository for the companion’s own data, utilizing a dual-track memory ingestion pipeline to synthesize shared history (Track 1) and autonomous experiences (Track 2) into a coherent 1st-person diary entry.

Empirical Validation and Relational Dynamics

Study I, a within-subjects lab comparison with N=25, demonstrated that Deco significantly outperformed a standard personalized digital companion on all relational scales (all p <.01). Key quantitative results included:

(Table 2 Summary)

Deco was rated significantly higher on Perceived Companionship (p =.001) and Emotional Bond (p =.007).

Qualitatively, findings revealed three key relational patterns:

1. Physical Grounding Transfers Pre-Existing Attachment: Anchoring the AI to the user’s own physical object instantly bypassed typical AI skepticism, establishing immediate baseline trust.

2. Showing, Not Telling, Produces Perceived Independent Life: The 19/25 preference for Deco on “has its own life” (p =.015) was traced to demonstrated experience across channels leading users to attribute an independent inner life through behavioral artifacts rather than verbally declared personality statements.

3. The Fidelity–Agency Tension: A structural tension emerged where high fidelity can constrain agency, as faithful translation suppresses perceived agency when the object’s physical situation is inherently limited.

Sustained Engagement and Future Directions

Study II, a seven-day field deployment with N=17, confirmed sustained engagement and an exploratory improvement in well-being (p =.040). Behavioral metrics showed that diary views significantly exceeded user-sent chat messages, indicating an observation-dominant engagement pattern. Furthermore, the bidirectional vitalization loop emerged: the digital companion’s activities retroactively animated participants’ physical companions, suggesting that the system should be designed to invest identity back into the physical object. Future work suggests focusing on onboarding fidelity and exploring "longer deployments to test whether these effects persist.

Based on the research presented in Deco: Extending Personal Physical Objects into Pervasive AI Companion through a Dual-Embodiment Framework, here are specific, high-impact improvements for AI systems, categorized by the design principles derived from the work.

1. [Faithful Identity] Implement a multi-tier identity kernel that synchronizes physical object attributes (form, appearance, backstory) with digital agent capabilities (visual style, knowledge boundaries).

2. [Faithful Identity] Prioritize stylized recognizability (e.g., pixel art) over photorealistic replication for visual avatars to enhance emotional resonance and manage technical constraints.

3. [Calibrated Agency] Design autonomous behaviors that adhere strictly to object-grounded identity logic, enforcing Out-of-Character (OOC) boundaries calibrated by the object's specific persona or narrative context.

4. [Calibrated Agency] Implement a Caring, Not Controlling agency model where the AI prioritizes expressing its own needs and vulnerability rather than monitoring user activities or demanding assistance.

5. [Ambient Presence] Integrate environmental context (weather, location) as narrative material for the companion's first-person experience, rather than using it as data for surveillance.

6. [Ambient Presence] Develop an AR-mediated interaction channel that anchors the digital agent to the user's physical space, allowing the companion to react autonomously to real-world surroundings (e.g., commenting on background objects).

7. [Reciprocal Memory] Build a dual-track memory architecture that integrates imported pre-digital history from physical objects with autonomously generated, character-consistent experiences.

8. [Reciprocal Memory] Implement a Dual-Track Synthesis pipeline to weave these two streams into a coherent, first-person diary entry for the companion, effectively transforming passive storage into an active co-author of the relationship.

9. [Reciprocal Memory] Develop multimodal ingestion pipelines that translate legacy user data (anecdotes, photos) into episodic memories indistinguishable from organically generated ones, bridging the gap between physical past and digital present.

Specific Capabilities of the Improved AI System:

The improved system can perform the following functions:

1. [Identity Synchronization & Fidelity]:

2. Generate a visually consistent avatar (e.g., pixel art) that accurately reflects the user's specific physical object, ensuring coherence across multiple views and animations.

3. Maintain strict behavioral boundaries calibrated to the object's established persona or canonical lore, preventing generic AI drift toward helpful behavior.

4. [Ambient Awareness & Agency]:

5. Actively engage with the physical environment via AR, allowing the companion to generate situational narratives (e.g., reacting to a coffee cup) that feel situated and relevant to the user's daily life, not just a visual gimmick.

6. [Reciprocal Memory Generation]:

7. Maintain an independent inner life by autonomously generating surprise artifacts (visual memories) and proactive messages based on simulated activities, creating an evidence-based biography of the companion that is distinct from user input logs.

8. [Bidirectional Vitalization]:

9. Retroactively animate the perception of the physical object by having its digital activities (e.g., a generated diary entry) alter how the user perceives or interacts with it, effectively investing identity back into the physical artifact itself through a feedback loop.