How Far Are We from Removing the Visual Encoder? Scaling Laws for Encoder-Free Multimodal Pretraining
cs.CV
Submitted: 2026-09-28
Updated: 2026-09-28
Terminology
Sources
- Qwen3-VL Technical Report
- Chinchilla Scaling: A replication attempt
- DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
- Chameleon: Mixed-Modal Early-Fusion Foundation Models
- SOLO: A Single Transformer for Scalable Vision-Language Modeling
- A Hitchhiker's Guide to Scaling Law Estimation
- From Pixels to Words -- Towards Native One-Vision Models at Scale
- SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture
- What Do Visual Tokens Really Encode? Uncovering Sparsity and Redundancy in Multimodal Large Language Models
- Let ViT Speak: Generative Language-Image Pre-training
- Gemma 3 Technical Report
- Gemma 4 Technical Report
- Scaling Laws for Autoregressive Generative Modeling
- Training Compute-Optimal Large Language Models
- Scaling Laws for Neural Language Models
- Kimi K2.5: Visual Agentic Intelligence
- Predictable Scale: Part I, Step Law -- Optimal Hyperparameter Scaling Law in Large Language Model Pretraining
- Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation
- Mono-InternVL-1.5: Towards Cheaper and Faster Monolithic Multimodal Large Language Models
- Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models