When Entropy Is Not Enough: Reclaiming Lost Semantics in LLM Output Length Prediction

Efficient LLM serving is often bottlenecked by the need to pad sequences to a fixed maximum length, and this wastes compute and degrades throughput. Output length prediction enables length-aware scheduling and reduces overhead. Existing methods, including entropy-guided token pooling, rely on token-level entropy but overlook semantic differences between tokens, causing them to underweight important tokens and overemphasize less informative tokens. We introduce ESTP(Entropy-and-Semantic Token Pooling), a lightweight framework that combines entropy representations with attention-based semantic representations. By deriving both representations directly during prefill, ESTP captures uncertainty and semantic importance while incurring negligible additional inference latency and memory overhead compared with methods using external auxiliary models. On the ForeLen benchmark, ESTP outperforms baseline methods, achieves better prediction accuracy and lower prediction errors across most scenarios. When integrated with a length-aware scheduler in end-to-end system tests, it further improves overall throughput and reduces the padding ratio.