Time series clustering and characterization

Abstract

Among the vast volumes of data generated daily in our modern digital world, time series data represents a major category with broad applicability. Time series data refers to a collection of data points indexed by time, and it has major practical uses in the fields of finance, business, environment, healthcare, and more. There are multiple types of time series analysis, the most common of which are clustering and forecasting. While forecasting predicts future trends based on historical data, clustering plays a pivotal role in pre-processing, grouping time series into homogenous clusters based on their temporal trends and underlying characteristics. This unsupervised learning task provides valuable insights for pattern discovery, anomaly detection, and data organization. Without accurate cluster labels in most real-world data, we need to rely on tuning the parameters that affect time series clustering, primarily the clustering algorithm and distance metric, as well as suitable clustering metrics to evaluate our results across methods for different contexts. In this paper, I carefully evaluate the shape-based, feature-based, and model-based clustering algorithms, focusing on the most widely used approaches. The algorithms will be tested on a well-known synthetic time series dataset and the Standard & Poor 500 (SP500) financial time series for real-world data. I have chosen to explore the clustering of financial time series into market regimes, recognizing it as one of the lesser-known yet compelling applications of clustering relevant to our daily lives. This analysis proves invaluable in identifying recurring economic regimes and aiding retail investors in making informed investment decisions. Even in large portfolio and wealth management funds such as StashAway, Syfe, and Endowus, regime analysis plays a crucial role in guiding key investment decisions and portfolio rebalancing as market conditions change. The applicability of regime analysis also extends to diverse industries such as environmental science, engineering, healthcare, politics, and the social sciences.