HelixWorld: A Real-time Interactive Audio-Visual World Model

arXiv:2609.38123 · cs.CV, cs.MM, cs.SD · Submitted 2026-09-29 · Read on arXiv

cs.CV, cs.MM, cs.SD

Submitted: 2026-09-29

Updated: 2026-09-29

Code: https://github.com/NoizAI/HelixWorld

Project page: https://helixworld.org

Terminology

• Distillation — 36× in this paper · explained in 2 episodes
• World Model — 32× in this paper · explained in 7 episodes
• Benchmark — 19× in this paper · explained in 2 episodes
• Transformer — 16× in this paper · explained in 2 episodes
• Consistency — 13× in this paper · explained in 3 episodes
• Alignment — 12× in this paper · explained in 1 episode
• Distribution Matching — 9× in this paper · explained in 2 episodes
• Fidelity — 8× in this paper · explained in 2 episodes
• Foundation Model — 7× in this paper · explained in 3 episodes
• Embeddings — 6× in this paper · explained in 2 episodes
• Fine-Tuning — 6× in this paper · explained in 5 episodes
• KV cache — 6× in this paper · explained in 3 episodes
• Ablation — 5× in this paper · explained in 1 episode
• Computer Vision and Pattern Recognition — 5× in this paper
• Diffusion Model — 5× in this paper · explained in 8 episodes
• Diffusion Models — 5× in this paper · explained in 6 episodes
• Adaptation — 4× in this paper · explained in 1 episode
• Foundation Models — 3× in this paper · explained in 7 episodes
• Pruning — 3× in this paper · explained in 4 episodes
• Aggregation — 2× in this paper · explained in 1 episode
• Cosine Similarity — 2× in this paper · explained in 3 episodes
• Exposure Bias — 2× in this paper · explained in 2 episodes
• Grounding — 2× in this paper · explained in 3 episodes
• Mode Collapse — 2× in this paper · explained in 2 episodes
• Steering — 2× in this paper · explained in 2 episodes
• Two-stage training — 2× in this paper · explained in 2 episodes
• Verification — 2× in this paper · explained in 2 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Actor — 1× in this paper · explained in 3 episodes
• Hallucination — 1× in this paper · explained in 3 episodes
• Semantic Alignment — 1× in this paper · explained in 3 episodes
• Temporal consistency — 1× in this paper · explained in 2 episodes
• Unified Framework — 1× in this paper · explained in 5 episodes
• alpha — 1× in this paper · explained in 1 episode

Sources

• Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control
• DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulation
• Infinite Worlds with Versatile Interactions
• World Models
• LTX-2: Efficient Joint Audio-Visual Foundation Model
• ViPE: Video Pose Engine for 3D Geometric Perception
• Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition
• Depth Anything 3: Recovering the Visual Space from Any Views
• PrismAudio: Decomposed Chain-of-Thoughts and Multi-dimensional Rewards for Video-to-Audio Generation
• ThinkSound: Chain-of-Thought Reasoning in Multimodal Large Language Models for Audio Generation and Editing
• Qwen3-Omni Technical Report
• Lyra 2.0: Explorable Generative 3D Worlds
• DINOv3
• WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling
• Movie Gen: A Cast of Media Foundation Models
• VGGT-$\Omega$
• Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory
• Diversity-Preserved Distribution Matching Distillation for Fast Visual Synthesis
• LongLive: Real-time Interactive Long Video Generation
• Alaya-EVOKE: From Linear-Scaling Supervision to Endless World

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models