PhD defence of Yitian Zhang – Advanced Sequential Machine Learning Models for Time-Series Signals

Abstract

Time-series analysis and sequential machine learning have emerged as fundamental pillars of modern data science, with applications spanning quantitative finance, weather prediction, electricity management, healthcare monitoring, and industrial process control. The increasing complexity and scale of temporal data necessitate sophisticated methodologies that can capture intricate patterns, long-range dependencies, and nonlinear dynamics. This thesis addresses fundamental challenges in sequential machine learning by proposing novel architectures and methodologies that advance the state-of-the-art in time-series modeling, forecasting, and representation learning. The main contributions of this thesis are organized into three primary categories.

First, we propose a novel Multi-resolution Time-Series Transformer (MTST) architecture for multivariate time series forecasting. This framework employs a multi-branch architecture that simultaneously models diverse temporal patterns at different resolutions by adjusting patch-level tokenization, enabling the capture of both short-term fluctuations and long-term seasonal trends. Unlike previous works that rely on subsampling, MTST constructs multi-resolution representations through different patch sizes, with each branch processing temporal patterns at distinct frequencies. The architecture employs relative positional encoding, which is naturally aligned with capturing periodic temporal patterns. Extensive experimental evaluation demonstrates that MTST achieves state-of-the-art performance across seven benchmark datasets and four prediction horizons, outperforming previous patch-based transformers with statistical significance in the majority of cases.

Second, we establish SKOLR, a novel approach that connects Koopman operator theory with linear Recurrent Neural Networks. By leveraging an extended state space of lagged observations, we demonstrate an equivalence between structured Koopman operators and linear RNN updates, enabling the development of forecasting architectures that combine theoretical rigor with computational efficiency. SKOLR implements a structured Koopman operator through a highly parallel linear RNN stack, where learnable spectral decomposition of the input signal allows different RNN chains to attend to different dynamical patterns from different representation subspaces. The resulting architecture achieves exceptional performance on various forecasting benchmarks and dynamical systems, demonstrating superior capabilities in handling both short-term and long-term forecasting tasks across diverse temporal patterns.

Third, we introduce GraphTNC, a framework for learning joint representations of graph-structured time series through contrastive learning. The framework addresses the challenge of unsupervised representation learning for multivariate time-series data, particularly in settings where the data exhibits graph-structured relationships that evolve over time. GraphTNC incorporates both temporal smoothness and graph-structured relationships into the contrastive learning objective, assuming piecewise smooth dynamics in both time-series and graph evolution. This enables joint learning of graph and temporal representations that can be effectively utilized for downstream tasks such as classification. Experimental results demonstrate that GraphTNC learns meaningful representations that improve performance on various graph-structured time-series tasks.

Collectively, these contributions advance both the theoretical understanding and practical capabilities of time-series modeling, with demonstrated improvements in forecasting accuracy, computational efficiency, and representation quality across diverse benchmark datasets and application domains.