KaLM-Reranker-V1: Fast but Not Late Interaction for Compressed Document Reranking

arXiv:2606.22807 · cs.CL · Submitted 2026-06-22 · Read on arXiv

cs.CL

Submitted: 2026-06-22

Updated: 2026-09-22

Comments: Technical Report, 31 pages;

Code: https://github.com/LiuHC0428/LAW_GPT

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 18× in this paper · explained in 2 episodes
• Matryoshka Embedding Pooling — 13× in this paper
• Embeddings — 11× in this paper · explained in 2 episodes
• Transformer — 8× in this paper · explained in 2 episodes
• AUC — 7× in this paper · explained in 2 episodes
• Distillation — 7× in this paper · explained in 2 episodes
• Empirical Methods in Natural Language Processing — 6× in this paper
• Fine-Tuning — 6× in this paper · explained in 5 episodes
• Generalization — 6× in this paper · explained in 13 episodes
• Adaptation — 5× in this paper · explained in 1 episode
• Adam — 3× in this paper · explained in 2 episodes
• Contrastive Learning — 3× in this paper · explained in 5 episodes
• Foundation Model — 3× in this paper · explained in 3 episodes
• RAG — 3× in this paper · explained in 3 episodes
• Robustness — 3× in this paper · explained in 5 episodes
• Verification — 3× in this paper · explained in 2 episodes
• Foundation Models — 2× in this paper · explained in 7 episodes
• Instruction Tuning — 2× in this paper · explained in 2 episodes
• Knowledge Distillation — 2× in this paper · explained in 11 episodes
• Trace — 2× in this paper · explained in 2 episodes
• Aggregation — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Decoupling — 1× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Reasoning Traces — 1× in this paper · explained in 3 episodes
• Redundancy — 1× in this paper · explained in 2 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Representation Learning — 1× in this paper · explained in 2 episodes
• Retrieval-Augmented Generation (RAG) — 1× in this paper · explained in 16 episodes
• Supervised Fine-Tuning (SFT) — 1× in this paper · explained in 8 episodes
• alpha — 1× in this paper · explained in 1 episode

Sources

• jina-embeddings-v5-text: Task-Targeted Embedding Distillation
• MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
• mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset
• M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation
• SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
• Distilling the Knowledge in a Neural Network
• KaLM-Embedding: Superior Training Data Brings A Stronger Embedding Model
• Adam: A Method for Stochastic Optimization
• ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models Reranking
• Towards General Text Embeddings with Multi-stage Contrastive Learning
• WebGPT: Browser-assisted question-answering with human feedback
• Large Dual Encoders Are Generalizable Retrievers
• Passage Re-ranking with BERT
• RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models
• Shopping Queries Dataset: A Large-Scale ESCI Benchmark for Improving Product Search
• DRCD: a Chinese Machine Reading Comprehension Dataset
• jina-embeddings-v3: Multilingual Embeddings With Task LoRA
• BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
• jina-reranker-v3: Last but Not Late Interaction for Listwise Document Reranking
• Text Embeddings by Weakly-Supervised Contrastive Pre-training

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering