Sensitivity of Transformer-Based PM2.5 Forecasting to Input Sequence Length

Transformer-based time series models are already widely used in PM_2.5 prediction studies due to their ability to learn long-term dependencies. However, despite input sequence length being a key design factor governing prediction performance, systematic evaluations that independently control this factor and examine how its effects vary across forecast lead times and atmospheric conditions remain limited. In particular, quantitative evidence is lacking on whether extending the input sequence length consistently improves long-term forecasting skill, or whether, under certain conditions, excessive historical information can instead degrade forecast stability.

In this study, Seoul—characterized by frequent high-pollution episodes and pronounced seasonal variability—is selected as a case study region. Using hourly observational data from 2018 to 2024, we quantitatively analyze the effects of input sequence length (3, 7, and 15 days) on Transformer-based PM_2.5 prediction performance. Vanilla Transformer, Informer, and Autoformer models are evaluated under identical data partitioning, preprocessing, input variable configuration, training strategies, and output structures, allowing the effects of input sequence length to be isolated from other modeling factors. Prediction performance is assessed for short-term (24 h) and long-term (72 h) forecast horizons using MAE and RMSE, enabling joint analysis of error reduction, error accumulation, and forecast stability as input sequence length increases.

The results show that extending the input sequence length from 3 to 7 days leads to reduced short-term prediction errors and improved stability in long-term forecasts across all models. However, further extension to 15 days yields diminishing returns and, in some cases, increased errors for long-term forecasts. In particular, differences in MAE associated with input sequence length reach up to approximately 10–15% for 72 h predictions, indicating that longer input sequences can introduce not only useful long-term dependencies but also redundant or irrelevant historical patterns. Seasonal analyses further reveal that sensitivity to input sequence length is amplified during wintertime conditions with frequent high-pollution events, suggesting that the utilization of accumulated historical information plays a more critical role under stagnant atmospheric regimes.

This study demonstrates that longer input sequences are not universally optimal across all forecast horizons and highlights the need to tailor input sequence length according to forecast lead time and environmental context, even within the same Transformer architecture. By reframing input sequence length as a purpose-driven design parameter rather than a fixed hyperparameter, this work provides empirical guidance for the development and application of Transformer-based PM_2.5 forecasting models.

Acknowledgments
"This research was supported by Particulate Matter Management Specialized Graduate Program through the Korea Environmental Industry & Technology Institute(KEITI) funded by the Ministry of Environment(MOE)“