Syn2RealTrack: Bridging the Gap Between Synthetic and Real-World Datasets for Online Multi-View Multi-Target Tracking

arXiv:2608.24130 · cs.CV, cs.AI · Submitted 2026-08-25 · Read on arXiv

cs.CV, cs.AI

Submitted: 2026-08-25

Updated: 2026-08-25

Comments: This paper has been accepted by the AI City Challenge Workshop of the European Conference on Computer Vision (ECCV 2026)

Code: https://github.com/SKKUAutoLab/aic26_mc3dp

License: http://creativecommons.org/licenses/by-sa/4.0/

The gist: Multi-camera 3D perception systems for warehouse scenes are trained largely on synthetic data and evaluated on physically captured environments.

Terminology

Abstract

Multi-camera 3D perception systems for warehouse scenes are trained largely on synthetic data and evaluated on physically captured environments. The resulting synthetic-to-real gap, which corrupts ground-plane localization and cross-camera identity association, is usually treated as one deficiency for a single domain-adaptation module to absorb; we argue instead that it enters the pipeline at three separable points: the camera calibration, the object shape prior, and the assumption that the object census is known, each admitting a different local remedy. Our online pipeline, Syn2RealTrack, follows this decomposition: lens distortion is recovered from images alone under a calibration that provides none, detections are fused across views by a visibility-weighted part-based descriptor that abstains on occluded parts rather than guessing, person height is measured in closed form from calibration instead of copied from a synthetic prior, and a closed-world cardinality prior is paired with a causal filter that removes the phantom boxes the prior manufactures. The system therefore adapts by reallocating trust between geometry and appearance without retraining a feature extractor. On the AI City Challenge 2026 Track 1 evaluation server it reaches a 3D Higher Order Tracking Accuracy (HOTA) of 52.0118%. The code will be released at https://github.com/SKKUAutoLab/aic26 mc3dp

Sources

Related papers