Models for Time Series Forecasting

Here’s something we’ve all wished for at least once: to peek into the future and find answers to the problem you’re facing today.

This may sound like science fiction, but many companies currently possess this capability — and they’re using it to strengthen their IT infrastructures and industrial systems, sales and marketing efforts, long-term product development, and more.

One popular way to do this is time series forecasting. One of many statistical methods, time series forecasting can help you in many areas.

Let’s take a look at this important topic.

What is time series forecasting?

The term “time series forecasting” refers to the prediction of a future sequence of events, by using historical and timestamped data. Forecasting time series is not about predicting a single data point in general — instead, you’re predicting a series of data points that lead to a future state, given the present state of event sequences.

Time series forecasting takes the knowledge from the given series of historical time series events and predicts what will happen over the next sequence of time instances in the future. These predictions can be seen as trajectories of event instances at given time intervals between a present state and the required future state.

Time series forecasting example

For example, weather and stock price events are time series incidents that change over time in a sequence (of events). Time series forecasting allows you to analyze the given information to predict how the weather or the stock price will change every few minutes over the next day.

Take a look at the time series data below. The data is charted out and binned with 1 hour granularity. It shows a cyclical pattern as traffic begins ramping up at 8PM, hits its peak at midnight, and then dwindles down entirely by 9AM.

As you can see, the volume of traffic on each day is not identical. For example, Mondays usually have much less traffic than any other day except Tuesday, which appears to be flat.

The takeaway here is that — in this example — you must account for the hour of the day and the day of the week in order to make a proper forecast.

Time series forecasting use cases

Time series forecasting is an important problem for all industry verticals and domains.

• Businesses analyze market trends to predict customer spending patterns.
• Economists study financial markets to predict future volatilities.
• Cybersecurity experts study network logs and traffic trends to predict cyberattacks.

These predictions are typically temporal trajectories. That is: data events that take place over distinct sequential time instances.

Time series forecasting involves the study of these trajectories and relationships between data distributions to accurately forecast future events.

What are TSDBs: time series databases?

Time Series Databases (TSDBs) are specialized storage systems designed for handling time-stamped data, where each entry is associated with a specific point in time. The need for these specialized TSDBs is growing, as Austin Chia describes:

“Time series data is becoming more prevalent across many industries. Indeed, it is no longer limited to financial data. As the need to handle time-stamped data increases, the demand for specialized databases to handle this type of data has also grown.”

TSDBs can handle time-ordered data generated by IoT sensors, applications, and infrastructure, where new data is constantly being generated. Structured to ingest this influx of data continuously, TSDBs have capabilities such as data compaction, retention policies, real-time processing.

(Learn more about time series databases.)

Modeling themes for time series forecasting

While the applications of time series forecasting can be diverse, some mathematical models are universally adopted for time series forecasting across all domains. The choice for models comes down to:

• The problem complexity
• Resources
• Technical requirements such as interpretability and flexibility of the models

Here are some of the common modeling themes used across a variety of time series forecasting methods:

Temporal dependencies

Temporal, or time, dependencies. Models learn time-based relationships between the data points. For example, the autocorrelation of present data to the historical data over a moving window average.

 Such models assume time-related dependencies between the observations. Examples include Autoregressive models.

Parameter estimation

Parameter estimation models learn parameters of a function that best captures the trends in a time series data distribution.

Simple models such as Least Square Estimation or Maximum Likelihood Estimate may be used to best fit the observed data points. These models make an assumption: that future observations follow the same trends.

The goal is to learn parameters that best map the features and output values based on the given observations.

Residuals and error modeling

Residual models and error modeling capture some information on the residuals (the difference between predicted and true values) to fit a model.

Classical time series forecasting models such as ARIMA make parametric assumptions about these errors (as white noise) and model residuals as a normal distribution. Advanced deep learning models may model parameters that minimize the residuals without explicit assumptions on the data distribution.

These models optimize the residual loss to best capture relationships between all input and output combinations. This is useful for high dimensional time series analysis that contains information on several related (and unrelated) features.

Modeling linear relationships

Time series trajectories can take many forms and represent stochastic (or even random) processes.

Classical linear time series forecasting methods may rely on temporal (across time) and spatial dependencies (across variables at each time instant). Examples include:

• Regression models (linear, autoregressive, lasso and Generalized Linear Models)
• State-space models
• Support Vector Machines
• A single layer perceptron (non-deep neural network)

In these cases, the models assume fixed rules and mathematical relationships between the input and the output. Once these rules are learned, the models can be used to predict a future set of values.

Nonlinear modeling

For time series data points, where the observations between inputs are not represented by a linear equation, advanced models use nonlinear functions and polynomial terms to find a function that best captures a mapping between the input and output variables.

As a simple example, a large deep learning neural network model applies a nonlinear activation function that transforms weighted sums of an input into an output. By doing this for all data points over the training process, the model is able to map inputs to their corresponding outputs without having to explicitly model the rules underlying these relationships.

Once the model has generalized this relationship, it can be used to predict a future time series trajectory.

Probabilistic modeling

Another approach to forecast time series trajectories is to quantify uncertainties in predictions. This is achieved with a probabilistic framework to produce a distribution of outcomes (instead of a single data point). This distribution represents the model uncertainty: the model assigns high probability to the outputs that it considers as most likely prediction, and a low probability to the rest.

These models use historical data to find temporal and spatial dependencies. In the case of Bayesian models, this information can be used to produce the distribution of model parameters corresponding to the given data distribution.

Other models such as Hidden Markov Models (HMM) learn a sequence of updates from one observation state to another (given the discrete and structured state space of time series events). 

How to choose time series forecasting models

So which model should you use for your time series forecasting? There’s no universally applicable solution to this problem, unfortunately.  Take into consideration:

• The data characteristics, including the features and known relationships between them
• Noise and volatility
• The forecasting horizon (short- versus long-term forecast)
• Problem complexity
• Model interpretability