MLIPS: Multi-Layer Indexer Pool Sharing
En palabras de los autores
Sparse attention limits the keys used by each query, but finding those keys can still require a full-prefix scan at every layer. We introduce MLIPS (Multi-Layer Indexer Pool Sharing), a training-free method that shares candidate search while retaining each consumer's own scoring and selection. A producer publishes a top- pool larger than the attention budget ; subsequent layers select their own top- from that pool. Measurements across three sparse-attention model families show substantially higher coverage from broad pools than overlap between final selections. We implement layerwise sharing, persistent decode reuse, and a coarse producer in vLLM for DeepSeek-V4-Flash. Across four long-context suites, three consumer layers per producer give -- speedups and -- lower GPU energy, with observed accuracy changes of to percentage points in single runs. More extensive sharing exposes task-dependent quality losses. Sampled prefill traces also reveal recurring selected KV positions across query groups and layers. We characterize this shared working-set structure and use it to motivate Top-, an exploratory bounded pool construction for prefill. Its deployment, task accuracy, and serving benefits remain open.
Apareció: miércoles, 21 de octubre. arXiv. Preprint, todavía sin revisión por pares.