Block-Based Double Decoders
The paper introduces a new transformer architecture called block-based double decoders. This model combines the efficiency of decoder-only training with the inference advantages of encoder-decoder models. The authors demonstrate that their approach significantly reduces memory and compute requirements during inference without compromising performance.
- ▪Block-based double decoders utilize doubly-causal block-based attention masks for training.
- ▪The model achieves substantial inference-time savings compared to traditional encoder-decoder models.
- ▪In experiments, block-based double decoders outperform encoder-decoders and closely match decoder-only models.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Machine Learning arXiv:2605.18807 (cs) [Submitted on 11 May 2026] Title:Block-Based Double Decoders Authors:Asher Labovich, Benjamin Bradley, Vanessa Alexander, Chaitanya Harsha View a PDF of the paper titled Block-Based Double Decoders, by Asher Labovich and 3 other authors View PDF HTML (experimental) Abstract:Encoder-decoder models offer substantial inference-time savings over decoder-only models, but their pretraining objectives suffer from sparse supervision and dynamic sequence lengths, keeping them out of practice at scale.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
