No Frames, Just Frequencies: Bandwidth-Efficient Deployment-Time Adaptation for Smart Glasses

Deep neural networks deployed on smart glasses must adapt to changes in the distribution of incoming data while adhering to strict privacy and device constraints. We present No Frames, Just Frequencies, a bandwidth-efficient source-assisted adaptation framework that characterizes distribution shifts using compact frequency descriptors computed on-device. Rather than transmitting camera frames, the glasses compute and send frequency descriptors to a host device that contains a replica of the model. The host device maps the received frequency descriptor to a distribution shift operator that synthesizes out-of-distribution data and adapts a model replica using a pre-defined adaptation protocol. We evaluate the framework on mixtures of ImageNet-C corruptions using convolution-based (ConvNeXtV2-Femto), self-attention-based (DeiT-Tiny), and hybrid (TinyViT-5M) vision models, on person classification (Visual Wake Words), and on object detection (MS-COCO). Our results reveal that adaptation driven solely by the transmitted descriptors recovers 93-95% of the supervised oracle adaptation gain on all three architectures, 82% on Visual Wake Words and 94 -- 98% on MS-COCO. Notably, adaptation quality degrades gracefully as the descriptor resolution decreases from 224 to 28 pixels, consistently across architectures, enabling bandwidth reductions from 213 times to 784 times across the frequency descriptors.