Tensor Cache: Eviction-conditioned Associative Memory for Transformers
The article introduces Tensor Cache, a novel two-level cache system designed for Transformers. It combines sliding-window softmax attention with a fixed-size outer-product fast-weight memory to enhance memory efficiency. This approach aims to improve the memory-quality frontier in long-context language modeling tasks.
- ▪Tensor Cache addresses the limitations of autoregressive Transformer KV caches that grow linearly with context length.
- ▪The system utilizes a first-level cache (L1) and a second-level cache (L2) to manage evicted tokens more effectively.
- ▪The proposed method enhances associative recall and memory capacity diagnostics in machine learning applications.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Machine Learning arXiv:2605.22884 (cs) [Submitted on 21 May 2026] Title:Tensor Cache: Eviction-conditioned Associative Memory for Transformers Authors:Kabir Swain, Sijie Han, Daniel Karl I. Weidele, Mauro Martino, Antonio Torralba View a PDF of the paper titled Tensor Cache: Eviction-conditioned Associative Memory for Transformers, by Kabir Swain and 4 other authors View PDF HTML (experimental) Abstract:Autoregressive Transformer KV caches grow linearly with context length; sliding-window caching bounds memory but discards evicted tokens entirely, so relevant evidence outside the window becomes inaccessible.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
