LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series
The paper explores the effectiveness of language-pretrained transformers in forecasting time series data. It demonstrates that pretraining creates a reusable manifold that facilitates cross-modal transfer, allowing for competitive forecasts without paired supervision. The findings suggest that finetuning aligns existing directions rather than starting from scratch, enhancing optimization and performance.
- ▪Language-pretrained transformers can effectively forecast time series data.
- ▪Cross-modal transfer occurs due to a reusable manifold created during pretraining.
- ▪Finetuning aligns existing directions, improving optimization and performance.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Machine Learning arXiv:2605.20449 (cs) [Submitted on 19 May 2026] Title:LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series Authors:Alexis Roger, Prateek Humane, Zhenghan Tai, Gwen Legate, Andrei Mircea, Vasilii Feofanov, Irina Rish View a PDF of the paper titled LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series, by Alexis Roger and 6 other authors View PDF HTML (experimental) Abstract:Can language-pretrained transformers become effective time-series forecasters, and why? In this paper, we show that cross-modal transfer arises because language pretraining preconditions time series training with a reusable manifold.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
