In this article I will explain the basics of time series forecasting and show how we can implement different forecasting models in Python.
Forecast is a word we usually associate with weather. As we listen to or watch the NEWS, there is always a separate segment called "Weather Report" where the NEWS commentator provides us with the weather forecast information. Why are forecasts so important? Well, simply because we can doinformed decisions.
Now, there are two main types of forecasting methods, namely qualitative forecasting and quantitative forecasting.
In qualitative forecasting, forecasting decisions depend on expert opinions. No data is available to examine the patterns to make forecasting decisions. Since human decisions are involved, there is a possibility of bias.
In quantitative forecasting, data with patterns are available and computers can capture these patterns well. Therefore, human decision-making is not involved, so there is no possibility of human bias.
If we assign a time or time component to the forecast, it willtime series predictionand the data is called astime series data.Statistically, time series forecasting is the process of analyzing the time series data using statistics and models to make predictions and make informed strategic decisions. It falls under quantitative forecasting.
Examples oftime series predictionare weather forecasts for the next week, predicting the closing price of a stock for each day, etc.
To make near-accurate forecasts, we need to collect the time series data over a period of time, analyze the data, and then create a model to help us make the forecast. But for this process, certain rules must be followed, which will help us to get almost accurate results.
Granularity Rule:This rule states that the more aggregated your forecasts are, the more accurate your predictions will be. This is because aggregated data has less variance and therefore less noise.
Frequency rule:We frequently need to update the data to capture new information that becomes available, which makes our forecasts more accurate.
Horizon rule:Avoid making predictions too far into the future. That is, we should make predictions over a short period of time and not too far into the future. This will provide more accurate forecasts.
Components of time series data
Let's understand the meaning of each component one by one.
- Level: Every time series has a baseline. To this baseline we add various components to form a complete time series. This baseline is known asLevel.
- Trend: Defines whether the time series increases or decreases over a period of time. That is, it has an uptrend (increasing) or a downtrend (decreasing). For eg. The above time series has an increasingTrend.
- seasonality :It defines a pattern that repeats itself over a period of time. This pattern that repeats itselfregularlymeans asseasonality. In the chart above we can clearly see the seasonality component present.
- cyclicality: Cyclicity is also a pattern in the time series data, but it repeats itselfaperiodic, which means it doesn't repeat at fixed intervals.
- Noise: After we extract level, trend, seasonality/cyclicality, what is leftNoise. Noise is a completely random fluctuation in the data.
We get the above components if we decompose the time series. There are mainly two types of time series decomposition vizadditive seasonal decompositionandmultiplicative seasonal decomposition.
This is easy to understand: If the individual components of the present time series are added together to obtain the original time series, this is referred to as additive seasonal decomposition. If the individual components have to be multiplied in order to obtain the time series data, this is called multiplicative seasonal decomposition. The main reason for using one decomposition type over the other is that the residue should not show a pattern. That means it should just be random fluctuation.
Time Series Forecasting Python implementation
Using an example, we will now see how different forecasting techniques are implemented in Python and how effective they are.
First, let's understand the importance of the evaluation metrics that we will use to evaluate these forecasting techniques.
RMSE: RootMeanStormentedError is the square root of the mean square error (MSE). MSE is nothing more than a representation of how predicted values differ from actual or true. We take the square root to avoid the negative sign, since errors can be positive or negative. It is represented by the following formula:
CARTE: MeanAabsolutePpercentEError is the measure of how accurate a forecasting system is. It measures this accuracy as a percentage and can be calculated as the average absolute percentage error for each period minus actual values divided by actual values. It is represented by the following formula:
From whereIndeedis the true value andYpredictedis the predicted value at that particular point in time.nis the number of observations.
Both RMSE and MAPE should be as low as possible.
So here is the problem: Global Mart is an online super giant that operates worldwide. It takes orders and ships worldwide, serving 7 different geographic market segments – (Africa, APAC (Asia Pacific), Canada, EU (European Union), EMEA (Middle East), LATAM (Latin America), USA (United States). ) ). It covers all major product categories – consumer, business and home office. We mustforecast sales for the consistently most profitable market segment.
note: The code and graphics used in the article are present in the Python file whose link is given at the bottom of the article.
1. Import the required libraries
2. Read and understand the data
3. Exploratory data analysis
4. Data Preparation
5. Time Series Decomposition
6. Create and evaluate time series forecast
- Import the required libraries
2.Read and understand the data
Our data has 51290 rows and 5 columns and there are no missing values.
3.Exploratory data analysis
We run the outlier analysis of various attributes and find that there are indeed outliers in the profit and revenue columns.
In the time series data there are observations at all timestamps and therefore we cannot delete the outliers as this leads to data loss and affects their continuity.
We ran univariate, bivariate and multivariate analysis and here are the graphs.
From the graphs above we can see thatCanada consumer is the most profitable market segmentandAPAC-Home Office is the leading market segment combination in terms of revenue.
According to the problem statement, we need to find 21 market segments by combining the respective 7 geographic markets for each of the 3 product segments. We create a columnmarket segmentby combining 2 columns, market and segment.
Train-Test-Split :We split the data so that the train set contains 42 months and the test set contains 6 months of data.
Persistently profitable market segment:The coefficient of variation is the ratio of the standard deviation to the mean. We need to find the market segment for which the value of the coefficient of variation brings the least profit. This is because a lower standard deviation means a lower earnings variability, which means earnings numbers for that region are more consistent over the given time period. We calculate the coefficient of variation for each of the 21 market segments for 42 months (train data) to decide which market segment is consistently profitable.
We think, thatAPAC consumersis the market segment with the lowest coefficient of variation. This means that the winning numbers for the APAC consumer market segment have been consistent over the period of the train set. Therefore, we choose this market segment to further calculate and predict sales values.
We filter the dataAPAC consumersMarket Segment and group the resulting data frame by order date to get the time series data that includes order date and sales. We call it data1.
Our time series data looks like this:
We perform the additive and multiplicative seasonal decomposition as follows:
The data clearly contain seasonal components. We build different time series forecasting models and compare themRMSE(Root Mean Squared Error) undMAP(Mean Absolute Percentage Error) values for all models. Lower values of RMSE and MAPE are desirable to conclude that a model is performing better. Accuracy is calculated as (100 — MAPE). The lower the MAPE value, the higher the accuracy.
We will now see different forecasting methods to forecast sales values.
Simple time series forecasting methods
3 methods that fall under it are the naive method, the simple average method and the simple moving average method.
The naive method simply transfers the last observation. The simple average uses the average of all observations for forecasting and the simple moving average method uses moving averages for forecasting.
The RMSE and MAPE values are as follows:
As we can see from the figures above, among the simple forecasting methods, the simple moving average method performs best.
Exponential smoothing techniques
Namely, they are the Simple Exponential Smoothing technique, the Holt method with trend, and the Holt-Winter method.
While the simple averaging method gives equal weight to past observations, simple exponential smoothing gives more weight to recent observations than past ones. It captures the level in the data but does not capture trends or seasonality. The Holt method, on the other hand, can capture level and trend, but not seasonality. Holt Winter's method can capture all levels, trends and seasonalities.
We conclude that Holt Winters' additive method in the smoothing techniques is able to predict sales closer to actual values. The RMSE and MAPE values for this method are lower than other model methods. This method is very good at capturing the trend and seasonality in the data.
Autoregressive methods use the regression technique to predict future observations using a linear combination of past observations. But for that the time series should follow 2 assumptions: stationarity and autocorrelation.
For a time series to be stationary, the mean, variance, and covariance should be constant. Autocorrelation helps us know how a variable is affected by its own lagged values.
There are 2 tests to confirm stationarity, as follows:
- Null hypothesis (H 0 ): The series is stationary: p−value>0.05
2. Alternative hypothesis (H a ): The series is not stationary: p−value ≤ 0.05
Extended Dickey-Fuller (ADF) test:
- Null hypothesis (H0 ): The series is not stationary : p−value>0.05
2. Alternative hypothesis (Ha): The series is stationary: p−value ≤ 0.05
We perform these tests on our time series data and conclude that the time series is not stationary. To make it stationary, we have to performdifferentiate(make mean constant) andTransformation(make variance constant).
We perform a train-test split and continue with the autoregressive techniques for prediction.
Automatic regression method (AR)
This method uses linear regression to predict the future observation using one or more past observations.
Moving Average (MA) Method
Here, future values are forecast using past forecast errors in a regression-like model.
Automatic Regression Moving Average (ARMA) Method
It is a combination of AR and MA models.
Automatic Regressive Integrated Moving Average (ARIMA)
It is the same as the ARMA model, only has an additional integrated differentiation component. Previously, we applied both the Box-Cox transform and differentiation to the data to make the time series data stationary. Here we simply apply Box-Cox before building the model and let the model differentiate, i.e. H. the trend component itself.
Seasonal Auto-Regressive Integrated Moving Average (SARIMA)
SARIMA is the same as ARIMA, it just has an additional seasonal component.
After we have implemented all forecasting models, we calculate the RMSE and MAPE for all methods.
We conclude thatHolt Winters' additive methodandSeasonal Autoregressive Integrated Moving Average (SARIMA) technique.are the best for predicting sales for the data. Both methods have lower RMSE and MAPE values and are able to capture the trend and seasonality components in the data well.
This completes our analysis. I hope the article was informative and easy to understand. Also, I hope you enjoyed analyzing the colorful graphics included in the analysis.
Feel free to comment and give your feedback.
You can connect with me on LinkedIn:https://www.linkedin.com/in/pathakpuja/
Please visit my GitHub profile for the Python codes. The code mentioned in the article as well as the graphics can be found here:https://github.com/pujappathak/Retail Giant Sales Forecast
Time series forecasting occurs when you make scientific predictions based on historical time stamped data. It involves building models through historical analysis and using them to make observations and drive future strategic decision-making.What are the four 4 main components of a time series? ›
- Trend component.
- Seasonal component.
- Cyclical component.
- Irregular component.
A time series analysis consists of two steps: (1) building a model that represents a time series (2) validating the model proposed (3) using the model to predict (forecast) future values and/or impute missing values.How do you evaluate a time series forecast? ›
Steps for validating the time-series model
Compare the predictions of your model against actual data. Use rolling windows to test how well the model performs on data that is one step or several steps ahead of the current time point. Compare the predictions of your model against those made by a human expert.
An observed time series can be decomposed into three components: the trend (long term direction), the seasonal (systematic, calendar related movements) and the irregular (unsystematic, short term fluctuations).What are the 3 key characteristics of time series data? ›
- Seasonal and nonseasonal cycles.
- Pulses and steps.
Cross-covariance and Cross-correlation Functions. Two key characteristics of the univariate time series model are the autocorrelation function and the covariance. The autocorrelation function measures the correlation of a univariate series with its own past values.What is time series analysis for beginners? ›
Time-series analysis is a method of analyzing a collection of data points over a period of time. Instead of recording data points intermittently or randomly, time series analysts record data points at consistent intervals over a set period of time.Why is time series analysis so hard? ›
The difficulty with time series is that it is not a binary task. If your test forecast is the same as your original data, there is a great great chance that your model is overfitting your data.How do you solve time series problems? ›
Today, time series problems are usually solved by conventional statistical (e.g., ARIMA) and machine learning methods, including artificial neural networks (ANN), support vector machines (SVMs), and some others.
Currently, the most popular metrics for evaluating time series forecasting models are MAE, RMSE and AIC. To briefly summarize, both MAE and RMSE measures the magnitude of errors in a set of predictions. The major difference between MAE and RMSE is the impact of the large errors.What are the best evaluation metrics for time series forecasting? ›
The mean absolute percentage error (MAPE) is one of the most popular used error metrics in time series forecasting. It is calculated by taking the average (mean) of the absolute difference between actuals and predicted values divided by the actuals.Is time series forecasting difficult? ›
This method is not scalable; it's challenging and very time-consuming. Often it is not accurate and reliable. Many factors can influence the data and need to be considered when generating forecasts. Machine learning methods are designed to address these factors and are efficient and scalable.What are the five 5 steps of forecasting? ›
The major steps that should be addressed in forecasting include: Establishing the business need. Acquiring data. Building the forecasting model. Evaluating the results.What is the simplest method of time series forecasting? ›
This is one of the simplest methods. It says that the forecast for any period equals the last observed value. If the time series data contain seasonality, it'll be better to take forecasts equal to the value from last season. This is often used for bench-marking purposes.
Examples of time series are heights of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.Which are the basic patterns in a time series? ›
There are three types of time series patterns: trend, seasonal, and cyclic.What are the four types of data patterns in time series? ›
There are typically four general types of patterns: horizontal, trend, seasonal, and cyclical.What is the objective of time series? ›
There are two main goals of time series analysis: identifying the nature of the phenomenon represented by the sequence of observations, and forecasting (predicting future values of the time series variable).What are common techniques of time series analysis? ›
Common types include: Autoregression (AR), Moving Average (MA), Autoregressive Moving Average (ARMA), Autoregressive Integrated Moving Average (ARIMA), and Seasonal Autoregressive Integrated Moving-Average (SARIMA).
Time series analysis also suffers from a number of weaknesses, including problems with generalization from a single study, difficulty in obtaining appropriate measures, and problems with accurately identifying the correct model to represent the data.What is the problem with time series? ›
In time series problems, we expect observations close to each other in time to be more similar than observations far away, after accounting for seasonality. For example, the weather today is usually more similar to the weather tomorrow than the weather a month from now.Is time series analysis a skill? ›
Knowing how to model time series is an essential skill within data science, as there are structures specific to this type of data that can be explored in all situations.Which algorithms is used for time series analysis? ›
- Autoregressive Integrated Moving Average (ARIMA)
- Exponential Smoothing.
- Seasonal Trend Decomposition.
Time series analysis involves different methods for analyzing data to extract useful statistics, and other characteristics related to the data. Whereas, time series forecasting involves the prediction of future values as per previously seen values using the time series model.How do you solve missing data in time series forecasting? ›
- omit the entire record that contains information.
- Impute the missing information.
To solve for time, divide the distance traveled by the rate. For example, if Cole drives his car 45 km per hour and travels a total of 225 km, then he traveled for 225/45 = 5 hours. Created by Sal Khan.What is the best way to measure forecast accuracy? ›
One simple approach that many forecasters use to measure forecast accuracy is a technique called “Percent Difference” or “Percentage Error”. This is simply the difference between the actual volume and the forecast volume expressed as a percentage.How is time series forecasting accuracy? ›
The error is measured by fitting points for the time periods with historical data and then comparing the fitted points to the historical data.What is the most accurate forecasting model? ›
A causal model is the most sophisticated kind of forecasting tool. It expresses mathematically the relevant causal relationships, and may include pipeline considerations (i.e., inventories) and market survey information. It may also directly incorporate the results of a time series analysis.
Step 1: Problem definition. Often this is the most difficult part of forecasting. Defining the problem carefully requires an understanding of the way the forecasts will be used, who requires the forecasts, and how the forecasting function fits within the organisation requiring the forecasts.Which is the #1 rule of forecasting? ›
RULE #1. Regardless of how sophisticated the forecasting method, the forecast will only be as accurate as the data you put into it. It doesn't matter how fancy your software or your formula is. If you feed it irrelevant, inaccurate, or outdated information, it won't give you good forecasts!What does complete forecast mean? ›
With the 'Forecast to Complete' column, you can: Estimate the 'Projected Over/Under' cost for each budget line item. This ensures that the total in the 'Estimated Cost at Completion' column is accurate and consistent with your previous weekly or monthly projection.What is the main objective of time series forecasting? ›
There are two main goals of time series analysis: identifying the nature of the phenomenon represented by the sequence of observations, and forecasting (predicting future values of the time series variable).What are the four types of forecasting? ›
While there are a wide range of frequently used quantitative budget forecasting tools, in this article we focus on four main methods: (1) straight-line, (2) moving average, (3) simple linear regression and (4) multiple linear regression.What is a time series forecasting model quizlet? ›
Time Series. a forecasting technique that uses a series of past data points to make a forecast. Components of a forecasting model. - Independent variable is time. - Dependent variable is what you're measuring and what you think is affected by time.How do you complete a forecast? ›
- List out the goods and services you sell.
- Estimate how much of each you expect to sell.
- Define the unit price or dollar value of each good or service sold.
- Multiply the number sold by the price.
- Determine how much it will cost to produce and sell each good or service.
- Set Clear Objectives. Be specific about the objectives of the forecast. ...
- Gather the Right Data. ...
- Analyze the Data. ...
- Budget and Plan Accordingly. ...
- Use Demand Forecasting Technology.
Forecast error numbers range from 0 - 100%. 100% forecast accuracy is perfect, obviously, but if your data is right, you won't be seeing it very often, and this is OK. Depending on the selected period and other operational factors, anything north of 70% can be perceived as acceptable.What is the purpose of time series data? ›
Time series data is used in time series analysis (historical or real-time) and time series forecasting to detect and predict patterns — essentially looking at change over time.
There are two types of forecasting methods: qualitative and quantitative. Each type has different uses so it's important to pick the one that that will help you meet your goals. And understanding all the techniques available will help you select the one that will yield the most useful data for your company.What are the 3 forecasting techniques? ›
There are three basic types—qualitative techniques, time series analysis and projection, and causal models.What is the difference between time series prediction and forecasting? ›
Time Series Forecasting. Time series analysis involves different methods for analyzing data to extract useful statistics, and other characteristics related to the data. Whereas, time series forecasting involves the prediction of future values as per previously seen values using the time series model.Which of the following is an example of time series forecasting problem? ›
Estimating number of hotel rooms booking in next 6 months. 2. Estimating the total sales in next 3 years of an insurance company.Is time series analysis forecasting? ›
Time series forecasting is a technique for predicting future events by analyzing past trends, based on the assumption that future trends will hold similar to historical trends. Forecasting involves using models fit on historical data to predict future values.