Enhancing intelligent transportation systems with a more efficient model for long-term traffic predictions based on an attention mechanism and a residual temporal convolutional network

Abstract

Accurate traffic state prediction is fundamental to Intelligent Transportation Systems, playing a critical role in optimising traffic management, improving mobility, and enhancing the efficiency of transportation networks. Traditional methods often rely on feature engineering, statistical time-series approaches, and non-parametric techniques to model the inherent complexities of traffic states, incorporating external factors such as weather conditions and accidents to refine predictions. However, the effectiveness of long-term traffic state prediction hinges on capturing spatial-temporal dependencies over extended periods. Current models face challenges in dealing with (i) high-dimensional traffic features, (ii) error accumulation for multi-step prediction, and (iii) robustness to external factors effectively. To address these challenges, this study proposes a novel model with a Dynamic Feature Embedding layer designed to transform complex data sequences into meaningful representations and a Deep Linear Projection network that refines these representations through non-linear transformations and gating mechanisms. These two features make the model more scalable when dealing with high-dimensional traffic features. The model also includes a Spatial-Temporal Positional Encoding layer to capture spatial-temporal relationships, masked multi-head attention-based encoder blocks, and a Residual Temporal Convolutional Network to process features and extract short-and long-term temporal patterns. Finally, a Time-Distributed Fully Connected Layer produces accurate traffic state predictions up to 24 timesteps into the future. The proposed architecture uses a direct strategy for multi-step modelling to help predict timesteps non-autoregressively and thus circumvents the error accumulation problem. The model was evaluated against state-of-the-art baselines using two benchmark datasets. Experimental results demonstrated the model's superiority, achieving up to 21.17% and 29.30% average improvements in Root Mean Squared Error and 3.56% and 32.80% improvements in Mean Absolute Error compared to the baselines, respectively. The Friedman Chi-Square statistical test further confirmed the significant performance difference between the proposed model and its counterparts. The adversarial perturbations and random sensor dropout tests demonstrated its good robustness. On top of that, it demonstrated good generalizability through extensive experiments. The model effectively mitigates error accumulation in multi-step predictions while maintaining computational efficiency, making it a promising solution for enhancing Intelligent Transportation Systems.