Lecture 31 Introduction to Linear Modelling for Time Series
31.1 Time series
A time series comprises observations taken at a sequence of time points.
In most situations these time points will be (approximately) evenly spaced. For example:
- Monthly rainfall figures
- Quarterly unemployment rates
Linear models can be applied to time series data, but typically it will prove necessary to generalize the structure of the error terms from the simple independent errors that we have assumed so far.
31.2 Tourism in Victoria
Data are number of room nights occupied in hotels, motels and guesthouses in Victoria. Observations are monthly from January 1980 to December 1994. Data source: Australian Bureau of Statistics.
## tourism <- read_csv(file = "motel.csv")
tourism <- tourism |>
mutate(Time = round(time_length(Date - min(Date), unit = "month")), Month = factor(month(Date,
label = TRUE), ordered = FALSE), Year = year(Date))
head(tourism)# A tibble: 6 × 6
Date RoomNights AvePrice Time Month Year
<date> <dbl> <dbl> <dbl> <fct> <dbl>
1 1980-01-01 276986 27.7 0 Jan 1980
2 1980-02-01 260633 28.7 1 Feb 1980
3 1980-03-01 291551 28.6 2 Mar 1980
4 1980-04-01 275383 28.3 3 Apr 1980
5 1980-05-01 275302 28.7 4 May 1980
6 1980-06-01 231693 28.6 5 Jun 198031.3
Note that
Timeis constructed so that it increases by 1 every month. This is to give us a nice easy integer valued time variable for interpretation.Yearis constructed from the date.Monthis constructed from the date and converted to an unordered factor (for modelling).
31.4
Figure 31.1: There is clear evidence of upward trend and seasonal (monthly) variation.
31.5 Variation in a Time Series
Possible sources of variation in a time series are:
- Secular trend (or just trend): tendency of the series to increase or decrease over a long period of time.
- Seasonal variation: describes fluctuations that recur during specific parts of the year (e.g. quarterly or monthly).
- Residual variation (or innovations): the part of the variation which is not explained by long term trend or seasonal effects.
- An additional cyclical source of variation (corresponding to business cycles, for example) is sometimes identified.
31.6 Modelling A Time Series
Time series data can be modelled using linear models (although there are a number of alternative approaches).
- Long term trend can be modelled using polynomial regression.
- Seasonal effects can be represented by specifying the seasons (e.g. months, quarters) as a factor in the model.
- Additional covariates can sometimes be incorporated in such models (e.g. standard economic indicators may be included to help explain variation in sales data)
31.7 Back to the Tourism Data
Model fitting and anova()
Analysis of Variance Table
Response: RoomNights
Df Sum Sq Mean Sq F value Pr(>F)
Time 1 5.4099e+11 5.4099e+11 2494.398 < 2.2e-16 ***
Month 11 1.5589e+11 1.4172e+10 65.346 < 2.2e-16 ***
Residuals 167 3.6219e+10 2.1688e+08
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 131.8
Call:
lm(formula = RoomNights ~ Time + Month, data = tourism)
Residuals:
Min 1Q Median 3Q Max
-37721 -8900 -1826 7931 48624
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 275040.43 4197.80 65.520 < 2e-16 ***
Time 1058.81 21.17 50.010 < 2e-16 ***
MonthFeb -30740.95 5377.53 -5.717 4.90e-08 ***
MonthMar 24115.91 5377.65 4.484 1.35e-05 ***
MonthApr -1464.50 5377.86 -0.272 0.785712
MonthMay -18682.52 5378.15 -3.474 0.000654 ***
MonthJun -65076.46 5378.53 -12.099 < 2e-16 ***
MonthJul -43764.27 5378.99 -8.136 8.89e-14 ***
MonthAug -29006.15 5379.53 -5.392 2.35e-07 ***
MonthSep -11274.03 5380.15 -2.095 0.037636 *
MonthOct 27159.22 5380.86 5.047 1.16e-06 ***
MonthNov 17231.14 5381.65 3.202 0.001635 **
MonthDec -57892.74 5382.53 -10.756 < 2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 14730 on 167 degrees of freedom
Multiple R-squared: 0.9506, Adjusted R-squared: 0.947
F-statistic: 267.8 on 12 and 167 DF, p-value: < 2.2e-1631.9
augment(tourism.lm, tourism) |>
ggplot(mapping = aes(x = Date)) + geom_point(mapping = aes(y = RoomNights)) +
geom_line(mapping = aes(y = .fitted), col = "blue")
31.11 More Model Fitting in R
tourism.lm.2 <- lm(RoomNights ~ poly(Time, 2) + Month, data = tourism)
anova(tourism.lm, tourism.lm.2)Analysis of Variance Table
Model 1: RoomNights ~ Time + Month
Model 2: RoomNights ~ poly(Time, 2) + Month
Res.Df RSS Df Sum of Sq F Pr(>F)
1 167 3.6219e+10
2 166 3.5762e+10 1 456501120 2.119 0.1474No evidence that quadratic trend improves on linear (P=0.1474).
31.13
The time plot of residuals for the linear trend model suggests that there are extended periods when the residuals are almost all negative (1986-1988) and extended periods where the residuals are almost all positive (1989-1990).
Such behaviour should not be observed if the errors are independent.
However, for time series data it is common for some residual correlation between residuals to remain even when the trend and seasonal variation has been removed.
This type of correlation in the sequence of residuals is usually called autocorrelation.
31.14 Stationary Processes and Autocorrelation
Consider a random process in time: \(Z_t\) where \(t=1,2,\ldots\) and \(Z_t\) represents the value of the process at time \(t\).
This process is said to be (weakly) stationary if:
- \(\mathbf{E}[Z_t]\) and \(\mathsf{Var}(Z_t)\) do not change with time \(t\).
- The correlation \(\mathsf{Corr}(Z_t, Z_{t+k})\) depends only on the time lag \(k\).
It is common to model the residuals from a time series as a stationary random process with zero mean.
For a stationary process, the autocorrelation function (or ACF) is defined by \[\rho(k) = \mathsf{Corr}(Z_t, Z_{t+k})\]
The ACF (autocorrelation against time lag) can be plotted in R using the acf() command.
31.16
- The ACF plot indicates a correlation of about \(0.4\) at lag one. Hence consecutive residuals are positively dependent.
- The dashed horizontal lines on the plot are a 95% confidence interval under the assumption that the true autocorrelation is zero.
- Any correlation lying within this confidence interval may be just noise.
- Any correlation lying outside this confidence interval is probably indicative of true serial dependence in the data.
For the tourism data it seems that there is serial dependence in the data, since the correlations at lags 1, 2, 3 and 4 all extend beyond the confidence interval bounds.
31.17
The existence of several significant correlations is common when autocorrelation exists. If the correlation between the \(i\)th and \(j\)th variables is high, and so is the correlation between the \(j\)th and \(k\)th variables, we ought to expect the correlation between the \(i\)th and \(k\)th variables to also be high.
When we are thinking about time series, the correlation for data from lags 0 and 1 is the same as the data from lags 1 and 2 because they are both sets formed by pairs of successive observations.
If observations are correlated with their preceding observations, and those preceding observations are correlated with their preceding observations, then there is likely to be a correlation between observations and those that are back two time steps.
This logic continues to three, four, and greater time lags and is especially likely when the correlation at the first lag is very, very high.
31.10 Comments
Timeis the appropriate covariate to track trend (not usingYear, which would ignore secular trend during a year).It is important to remember to code
Monthas a factor so as to represent seasonal effects.There is strong evidence of trend (\(P < 2.2 \times 10^{-16}\) for
Time) and of seasonality (\(P < 2.2 \times 10^{-16}\) forMonth) in the data.As might be expected, room bookings tend to be low in the winter months. The pattern over summer is less clear. Perhaps December is a bad month because of the Christmas effect?
We have assumed a linear secular trend. Would quadratic be better?