Skip to main content

PyTorch for Time Series Data

Time series data is a critical component in various applications, ranging from financial markets to weather forecasting and sensor data analysis. PyTorch provides powerful tools to process, analyze, and prepare time series data for machine learning models. This article covers the essential techniques for working with time series data in PyTorch, focusing on data preprocessing, feature extraction, and advanced manipulation techniques.

1. Understanding Time Series Data​

1.1 What is Time Series Data?​

Time series data is a sequence of data points collected or recorded at successive points in time. Examples include stock prices, temperature readings, or daily sales figures. The key characteristic of time series data is the temporal dependency, where each observation is dependent on previous ones.

1.2 Challenges in Time Series Data​

Working with time series data presents several challenges:

  • Temporal Dependencies: Each data point is not independent but is related to its predecessors.
  • Seasonality: Repeating patterns within the data over time.
  • Trend Analysis: Identifying the underlying trend over time, whether it’s increasing, decreasing, or remaining constant.
  • Handling Missing Data: Dealing with gaps in data which can impact the analysis and predictions.

These challenges make it necessary to preprocess and handle the data correctly before feeding it into a model.

2. Preprocessing Time Series Data in PyTorch​

2.1 Normalization and Standardization​

Normalization and standardization are key preprocessing steps that ensure the data is on a similar scale, which is critical for the training stability and performance of machine learning models.

Example: Normalizing Time Series Data​

import torch

# Generate synthetic time series data
torch.manual_seed(17)
time_series = torch.arange(100, dtype=torch.float32) + torch.randn(100) * 10

# Normalize the data
mean = torch.mean(time_series)
std = torch.std(time_series, correction=0)
normalized_series = (time_series - mean) / std

print("Normalized Time Series:\n", normalized_series)

Explanation: The example uses the population standard deviation (correction=0). For forecasting, estimate scaling parameters on the training interval only and reuse them for validation, test, and production observations to prevent future-data leakage.

2.2 Handling Missing Data​

Time series data often has missing values that must be addressed to ensure continuity.

Example: Imputing Missing Values with Mean​

# Simulate missing data
time_series_with_nan = time_series.clone()
time_series_with_nan[10:20] = float('nan')  # Introduce missing values

# Replace NaNs with the mean of the series
nan_mask = torch.isnan(time_series_with_nan)
time_series_with_nan[nan_mask] = mean

print("Time Series with Imputed Values:\n", time_series_with_nan)

Explanation: This synthetic example can reuse the mean of its known complete series. Real forecasting code must impute using information available at prediction time; a global mean can leak future observations and erase local temporal structure.

2.3 Windowing and Reshaping​

In time series modeling, it is often necessary to create windows or segments of the data to serve as input features for the model.

Example: Creating Sliding Windows​

from torch.utils.data import DataLoader, TensorDataset

# Define the window size
window_size = 5

# Create input sequences and targets
def create_sequences(data, window_size):
    sequences = []
    targets = []
    for i in range(len(data) - window_size):
        sequences.append(data[i:i+window_size])
        targets.append(data[i+window_size])
    return torch.stack(sequences), torch.stack(targets)

sequences, targets = create_sequences(time_series_with_nan, window_size)

# Create a DataLoader for batch processing
dataset = TensorDataset(sequences, targets)
dataloader = DataLoader(dataset, batch_size=10, shuffle=True)

for batch in dataloader:
    print("Sequences:\n", batch[0])
    print("Targets:\n", batch[1])
    break

Explanation: This example shows how to create sliding windows from the time series data and package them into a DataLoader for batch processing. This is a crucial step when preparing data for training time series models.

3. Advanced Data Manipulation Techniques​

3.1 Lag Features​

Lag features are created by shifting the time series data backward by a certain number of steps. These features are commonly used to predict future values based on past observations.

Example: Creating Lag Features​

# Align lag 1 and lag 2 without wrapping future values to the beginning
lagged_features = torch.stack(
    (time_series_with_nan[1:-1], time_series_with_nan[:-2]), dim=1
)
lagged_targets = time_series_with_nan[2:]

print("Lagged Features:\n", lagged_features)

Explanation: Row i contains values from one and two steps before lagged_targets[i]. torch.roll would wrap the last observations to the beginning, silently introducing future information unless the wrapped rows were removed correctly.

3.2 Rolling Statistics​

Rolling statistics such as moving averages or standard deviations are used to smooth out noise in time series data or to create additional features that capture trends.

Example: Computing a Moving Average​

window_size = 5

# Compute the moving average
moving_average = torch.nn.functional.conv1d(
    time_series_with_nan.reshape(1, 1, -1),
    torch.ones(1, 1, window_size) / window_size,
).flatten()

print("Moving Average:\n", moving_average)

Explanation: The moving average is computed to smooth out the time series data, helping to capture the underlying trend and reduce the impact of short-term fluctuations.

3.3 Time-Based Feature Engineering​

Extracting time-based features like hour of the day, day of the week, or month of the year can be crucial for improving model performance, especially in seasonal time series data.

Example: Extracting Time-Based Features​

import pandas as pd

# Simulate a time index
dates = pd.date_range(start="2023-01-01", periods=100, freq="D")

# Extract time-based features
day_of_week = torch.tensor([date.weekday() for date in dates])
month_of_year = torch.tensor([date.month for date in dates])

print("Day of Week:\n", day_of_week)
print("Month of Year:\n", month_of_year)

Explanation: Time-based features are extracted to capture the seasonal and cyclical patterns in the data. These features can significantly enhance the predictive power of models.

Conclusion​

PyTorch provides a powerful framework for handling and analyzing time series data. From preprocessing and feature extraction to advanced manipulation techniques, PyTorch's capabilities allow for robust and flexible handling of time series data. Understanding and applying these techniques is essential for building effective time series models that can forecast future trends, detect anomalies, and gain insights from temporal data.