Lossy Compressive Text Autoencoders

arXiv:2610.10738 · cs.CL · Submitted 2026-10-07 · Read on arXiv

cs.CL

Submitted: 2026-10-07

Updated: 2026-10-07

Terminology

• Latent Representation — 20× in this paper · explained in 2 episodes
• LLM judge — 19× in this paper · explained in 2 episodes
• Quantization — 19× in this paper · explained in 12 episodes
• Transformer — 19× in this paper · explained in 2 episodes
• Latent Representations — 9× in this paper · explained in 3 episodes
• Accuracy — 7× in this paper · explained in 1 episode
• Core — 7× in this paper · explained in 2 episodes
• Adaptation — 6× in this paper · explained in 1 episode
• Redundancy — 6× in this paper · explained in 2 episodes
• Representation Learning — 6× in this paper · explained in 2 episodes
• Semantic Similarity — 5× in this paper · explained in 2 episodes
• Adam — 3× in this paper · explained in 2 episodes
• Distillation — 3× in this paper · explained in 2 episodes
• Benchmark — 2× in this paper · explained in 2 episodes
• Embeddings — 2× in this paper · explained in 2 episodes
• Fidelity — 2× in this paper · explained in 2 episodes
• Loss Function — 2× in this paper · explained in 2 episodes
• Agentic Coding — 1× in this paper · explained in 1 episode
• Attention Mechanism — 1× in this paper · explained in 2 episodes
• Clustering — 1× in this paper · explained in 2 episodes
• Commentary — 1× in this paper · explained in 2 episodes
• Downstream Tasks — 1× in this paper · explained in 2 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Hallucination — 1× in this paper · explained in 3 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Reinforcement Learning (RL) — 1× in this paper · explained in 25 episodes
• Thinking mode — 1× in this paper · explained in 2 episodes
• Transformer Models — 1× in this paper · explained in 2 episodes

Sources

• Language Modeling Is Compression
• Nacrith: Neural Lossless Compression via Ensemble Context Modeling and High-Precision CDF Coding
• Big Bird: Transformers for Longer Sequences
• Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
• SONAR: Sentence-Level Multimodal and Language-Agnostic Representations
• Attention Is All You Need
• BERTScore: Evaluating Text Generation with BERT
• The Multi-Range Theory of Translation Quality Measurement: MQM scoring models and Statistical Quality Control
• FineZip : Pushing the Limits of Large Language Models for Practical Lossless Text Compression
• Longformer: The Long-Document Transformer
• ETC: Encoding Long and Structured Inputs in Transformers
• Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
• Efficiently Modeling Long Sequences with Structured State Spaces
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• Gated Delta Networks: Improving Mamba2 with Delta Rule
• DeepSeek-OCR: Contexts Optical Compression
• Context Cascade Compression: Exploring the Upper Limits of Text Compression
• MTEB: Massive Text Embedding Benchmark
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• Omnilingual SONAR: Cross-Lingual and Cross-Modal Sentence Embeddings Bridging Massively Multilingual Text and Speech

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering