A citywide TD-learning based intelligent traffic signal control for autonomous vehicles: Performance evaluation using SUMO

Abstract

An autonomous vehicle can sense its environment and operate without human involvement. Its adequate management in an intelligent transportation system could significantly reduce traffic congestion and overall travel time in a network. Adaptive traffic signal controller (ATSC) based on multi-agent systems using state-action-reward-state-action (SARSA (lambda)) are well-known state-of-the-art models to manage autonomous vehicles within urban areas. However, this study found inefficient weights updating mechanisms of the conventional SARSA (lambda) models. Therefore, it proposes a Gaussian function to regulate the eligibility trace vector's decay mechanism effectively. On the other hand, an efficient understanding of the state of the traffic environment is crucial for an agent to take optimal actions. The conventional models feed the state values to the agents through the MinMax normalization technique, which sometimes shows less efficiency and robustness. So, this study suggests the MaxAbs scaled state values instead of MinMax to address the problem. Furthermore, the combination of the A-star routing algorithm and proposed model demonstrated a good increase in performance relatively to the conventional SARSA (lambda)-based routing algorithms. The proposed model and the baselines were implemented in a microscopic traffic simulation environment using the SUMO package over a complex real-world-like 21-intersections network to evaluate their performance. The results showed a reduction of the vehicle's average total waiting time and total stops by a mean value of 59.9% and 17.55% compared to the considered baselines. Also, the A-star combined with the proposed controller outperformed the conventional approaches by increasing the vehicle's average trip speed by 3.4%.