Correlation generally determines the relationship between two variables. Correlation is calculated between the variable and itself at previous time steps, such a correlation is called Autocorrelation.
Method 1 : Using lagplot()
The daily minimum temperatures dataset is used for this example. As the first step, the autocorrelation can be quickly checked using lagplot() function provided by pandas.
Syntax:
pd.plotting.lag_plot(data, lag=1)
where,
- data is the input dataframe
- lag specifies integer to get the lags
Data Used: daily-minimum-temperatures-in-blr
Python3
# import modulesimport pandas as pd# read the data from the csvdata = pd.read_csv("daily-minimum-temperatures-in-blr.csv", header=0, index_col=0, parse_dates=True, squeeze=True)# display top 15 datadata.head(15)# lagplotpd.plotting.lag_plot(data, lag=1) |
Output:
Method 2: Creating lagged variables at different time steps
As we are aware of the fact that, the values of the observation at the current and previous time steps are significant in predicting the future step, let’s create lagged variables at different timesteps say, t+1, t+2, t+3. This is done using pandas.concat() and shift() function. Shift function shifts the timestep by a specified value and the Concat function joins the lagged variables at different timesteps as shown below.
Use pandas.corr() function on the new dataframe to calculate the correlation matrix.
Syntax:
pandas.DataFrame.corr(method = 'pearson')
where, method – pearson which is for calculating the standard correlation coefficient
Example:
Python3
data = pd.read_csv("daily-minimum-temperatures-in-blr.csv", header=0, index_col=0, parse_dates=True, squeeze=True)# extracting only the temperature valuesvalues = pd.DataFrame(data.values)# using shift function to shift the values.dataframe = pd.concat([values.shift(3), values.shift(2), values.shift(1), values], axis=1)# naming the columnsdataframe.columns = ['t', 't+1', 't+2', 't+3']# using corr() function to compute the correlationresult = dataframe.corr()print(result) |
Output:
Method 3: Using plot_acf()
A plot of the autocorrelation of a time series by lag is called the AutoCorrelation Function (ACF). Such a plot is also called a correlogram. A correlogram plots the correlation of all possible timesteps. The lagged variables with the highest correlation can be considered for modeling. Below is an example of calculating and plotting the autocorrelation plot for the dataset chosen.
Statsmodel library provides a function called plot_acf() for this purpose.
Syntax:
statsmodels.graphics.tsaplots.plot_acf(x,lags,alpha)
where,
- x – An array of time-series values
- lags – An int or array of lag values, used on the horizontal axis. Uses np.arange(lags) when lags is an int.
- alpha – If a number is given, the confidence intervals for the given level are returned. For instance, if alpha=.05, 95 % confidence intervals are returned where the standard deviation is computed according to Bartlett’s formula. If None, no confidence intervals are plotted.
Example:
Python3
# import the required modulesimport pandas as pdfrom statsmodels.graphics.tsaplots import plot_acf# read the csv datadata = pd.read_csv("daily-minimum-temperatures-in-blr.csv", header=0, index_col=0, parse_dates=True, squeeze=True)# plot the auto correlationplot_acf(data) |
Output:
