Neural Field Tokenizations with Hierarchy and Spatial Locality Priors
cs.LG, cs.CV
Submitted: 2026-06-06
Updated: 2026-09-05
Code: https://github.com/samuelepapa/spatial_functa
License: http://creativecommons.org/licenses/by/4.0/
The gist: Neural fields parameterize data as functions from coordinates to values, providing a unified framework for representation learning across modalities.
Terminology
Abstract
Neural fields parameterize data as functions from coordinates to values, providing a unified framework for representation learning across modalities. Existing approaches are dominated by per-sample meta-learning, which scales poorly due to memory-intensive inner-loop optimization. The natural alternative -- feed-forward encoding -- typically introduces modality-specific assumptions, sacrificing the generality that makes learning with neural fields attractive. We argue that locality and hierarchy are useful priors for learning field representations that can be injected without compromising modality-agnosticism. We propose LH-NeF, a framework to learn general-purpose tokenized representations of continuous signals. A locality-preserving hierarchical encoder maps raw coordinate-value field observations to structured tokens, from which the field is reconstructed during training. By replacing meta-learning's inner loop with a single forward pass, LH-NeF uses 42x less memory and supports 133x larger batches than the strongest modality-agnostic baseline. Across images, 3D shapes, and climate fields, our learned representations match or exceed performance of modality-agnostic, modality-specific, and specialized generative neural field baselines on both reconstruction and downstream tasks.
Sources
- Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
- HiP: Hierarchical Perceiver
- Dynamic Chunking for End-to-End Hierarchical Sequence Modeling
- Self-attention Does Not Need $O(n^2)$ Memory
- GPSToken: Gaussian Parameterized Spatially-adaptive Tokenization for Image Representation and Generation
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks