PhD defence of Xingshuai Huang – Sample-Efficient Algorithms for Intelligent Transportation Systems
Abstract
Intelligent Transportation Systems (ITSs) represent an integration of advanced technologies into transportation infrastructure to enhance efficiency, safety, and sustainability. Recent advancements in Artificial Intelligence (AI) have boosted the development of ITSs, enhancing urban mobility, reducing congestion, and improving sustainability. However, AI-enhanced ITSs also face significant challenges in achieving real-world scalability due to the high sample complexity of data-driven learning algorithms, e.g., machine learning (ML) and reinforcement learning (RL), which require vast amounts of data for training and adaptation. This thesis explores multiple strategies to improve sample efficiency in two critical real-world ITS domains: charging load forecasting of electric vehicle (EV) charging stations and traffic signal control (TSC).
To address data scarcity in EV charging stations, we propose MetaProbformer, a Transformer-based meta-learning approach for probabilistic load forecasting. It enables rapid adaptation to new stations with limited historical data through meta-training across diverse datasets. The method achieves robust performance across unseen scenarios, reducing reliance on extensive training samples on target charging stations while maintaining forecasting performance.
For TSC, we present two model-based and one offline-to-online RL methods: ModelLight, FM2Light, and DTLight. ModelLight is a model-based meta-RL framework for optimizing single signalized intersections. In this framework, world models capturing the dynamics of signalized intersections are acquired and employed to generate imaginary trajectories within an optimization-based meta-learning approach. It enables traffic signal controllers to quickly adapt to varying traffic patterns with minimal additional training, improving both sample efficiency and generalization ability. For multi-intersection TSC scenarios, we propose FM2Light, a model-based multi-agent RL framework that enhances sample efficiency and introduces fairness constraints to ensure equitable traffic flow distribution, addressing both congestion reduction and social equity in urban networks. FM2Light employs an ensemble of global world models and a fairness-refined reward structure to enhance both sample efficiency and intersection-level fairness in large-scale deployments. Additionally, we propose DTLight, a lightweight Transformer-based traffic signal controller that leverages offline historical data to pre-train control policies, followed by online fine-tuning with real-time feedback. By doing so, we aim to take advantage of the high sample efficiency of offline RL while incorporating valuable feedback from the online environment. This hybrid approach minimizes the need for costly online exploration while adapting dynamically to changing traffic conditions.
Collectively, these methods in our thesis aim to provide practical, sample-efficient solutions for ITS applications, reducing the need for large datasets and expensive real-world experimentation. The proposed approaches aim to improve forecasting accuracy and traffic management, contributing to more intelligent, responsive, and sustainable urban transportation systems.