JustFit: Just-in-Time State Management for Local LLM Serving

arXiv:2609.17475 · cs.AI, cs.PF · Submitted 2026-09-15 · Read on arXiv

cs.AI, cs.PF

Submitted: 2026-09-15

Updated: 2026-09-20

Comments: 13 pages, 4 figures, 9 tables

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

Decode — 24× in this paper · explained in 2 episodes
State Management — 14× in this paper · explained in 3 episodes
Quantization — 10× in this paper · explained in 12 episodes
KV cache — 4× in this paper · explained in 3 episodes
Trace — 3× in this paper · explained in 2 episodes
Verifier — 3× in this paper · explained in 2 episodes
Ablation — 2× in this paper · explained in 1 episode
Agent Memory — 2× in this paper · explained in 1 episode
Allocator — 2× in this paper · explained in 1 episode
Benchmark — 2× in this paper · explained in 2 episodes
Provenance — 2× in this paper · explained in 3 episodes
Speculative Decoding — 2× in this paper · explained in 2 episodes
Transformer — 2× in this paper · explained in 2 episodes
AI Agents — 1× in this paper · explained in 2 episodes
Activation Sparsity — 1× in this paper · explained in 1 episode
Core — 1× in this paper · explained in 2 episodes
Edge Devices — 1× in this paper · explained in 2 episodes
Embeddings — 1× in this paper · explained in 2 episodes
Sparsity — 1× in this paper · explained in 4 episodes
Verification — 1× in this paper · explained in 2 episodes

Related papers