Lecture 34: Introduction to Linear Modelling for Time Series

161.251 Regression Modelling

Time series

A time series comprises observations taken at a sequence of time points.

In most situations these time points will be (approximately) evenly spaced. For example:

• Monthly rainfall figures
• Quarterly unemployment rates

Linear models can be applied to time series data, but typically it will prove necessary to generalize the structure of the error terms from the simple independent errors that we have assumed so far.

Tourism in Victoria

Data are number of room nights occupied in hotels, motels and guesthouses in Victoria. Observations are monthly from January 1980 to December 1994. Data source: Australian Bureau of Statistics.

Download motel.csv

## tourism <- read_csv(file = "motel.csv")
tourism <- tourism |>
    mutate(Time = round(time_length(Date - min(Date), unit = "month")), Month = factor(month(Date,
        label = TRUE), ordered = FALSE), Year = year(Date))
head(tourism)

# A tibble: 6 × 6
  Date       RoomNights AvePrice  Time Month  Year
  <date>          <dbl>    <dbl> <dbl> <fct> <dbl>
1 1980-01-01     276986     27.7     0 Jan    1980
2 1980-02-01     260633     28.7     1 Feb    1980
3 1980-03-01     291551     28.6     2 Mar    1980
4 1980-04-01     275383     28.3     3 Apr    1980
5 1980-05-01     275302     28.7     4 May    1980
6 1980-06-01     231693     28.6     5 Jun    1980

Note that

• Time is constructed so that it increases by 1 every month. This is to give us a nice easy integer valued time variable for interpretation.

• Year is constructed from the date.

• Month is constructed from the date and converted to an unordered factor (for modelling).

ggplot(tourism) + geom_line(aes(x = Date, y = RoomNights))

There is clear evidence of upward trend and seasonal (monthly) variation.

Variation in a Time Series

Possible sources of variation in a time series are:

• Secular trend (or just trend): tendency of the series to increase or decrease over a long period of time.
• Seasonal variation: describes fluctuations that recur during specific parts of the year (e.g. quarterly or monthly).
• Residual variation (or innovations): the part of the variation which is not explained by long term trend or seasonal effects.
• An additional cyclical source of variation (corresponding to business cycles, for example) is sometimes identified.

Modelling A Time Series

Time series data can be modelled using linear models (although there are a number of alternative approaches).

• Long term trend can be modelled using polynomial regression.
• Seasonal effects can be represented by specifying the seasons (e.g. months, quarters) as a factor in the model.
• Additional covariates can sometimes be incorporated in such models (e.g. standard economic indicators may be included to help explain variation in sales data)

Back to the Tourism Data

Model fitting and anova()

tourism.lm <- lm(RoomNights ~ Time + Month, data = tourism)
anova(tourism.lm)

Analysis of Variance Table

Response: RoomNights
           Df     Sum Sq    Mean Sq  F value    Pr(>F)    
Time        1 5.4099e+11 5.4099e+11 2494.398 < 2.2e-16 ***
Month      11 1.5589e+11 1.4172e+10   65.346 < 2.2e-16 ***
Residuals 167 3.6219e+10 2.1688e+08                       
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

summary(tourism.lm)

Call:
lm(formula = RoomNights ~ Time + Month, data = tourism)

Residuals:
   Min     1Q Median     3Q    Max 
-37721  -8900  -1826   7931  48624 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept) 275040.43    4197.80  65.520  < 2e-16 ***
Time          1058.81      21.17  50.010  < 2e-16 ***
MonthFeb    -30740.95    5377.53  -5.717 4.90e-08 ***
MonthMar     24115.91    5377.65   4.484 1.35e-05 ***
MonthApr     -1464.50    5377.86  -0.272 0.785712    
MonthMay    -18682.52    5378.15  -3.474 0.000654 ***
MonthJun    -65076.46    5378.53 -12.099  < 2e-16 ***
MonthJul    -43764.27    5378.99  -8.136 8.89e-14 ***
MonthAug    -29006.15    5379.53  -5.392 2.35e-07 ***
MonthSep    -11274.03    5380.15  -2.095 0.037636 *  
MonthOct     27159.22    5380.86   5.047 1.16e-06 ***
MonthNov     17231.14    5381.65   3.202 0.001635 ** 
MonthDec    -57892.74    5382.53 -10.756  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 14730 on 167 degrees of freedom
Multiple R-squared:  0.9506,    Adjusted R-squared:  0.947 
F-statistic: 267.8 on 12 and 167 DF,  p-value: < 2.2e-16

augment(tourism.lm, tourism) |>
    ggplot(mapping = aes(x = Date)) + geom_point(mapping = aes(y = RoomNights)) +
    geom_line(mapping = aes(y = .fitted), col = "blue")

Comments

Time is the appropriate covariate to track trend (not using Year, which would ignore secular trend during a year).

It is important to remember to code Month as a factor so as to represent seasonal effects.

There is strong evidence of trend (\(P < 2.2 \times 10^{-16}\) for Time) and of seasonality (\(P < 2.2 \times 10^{-16}\) for Month) in the data.

As might be expected, room bookings tend to be low in the winter months. The pattern over summer is less clear. Perhaps December is a bad month because of the Christmas effect?

We have assumed a linear secular trend. Would quadratic be better?

More Model Fitting in R

tourism.lm.2 <- lm(RoomNights ~ poly(Time, 2) + Month, data = tourism)
anova(tourism.lm, tourism.lm.2)

Analysis of Variance Table

Model 1: RoomNights ~ Time + Month
Model 2: RoomNights ~ poly(Time, 2) + Month
  Res.Df        RSS Df Sum of Sq     F Pr(>F)
1    167 3.6219e+10                          
2    166 3.5762e+10  1 456501120 2.119 0.1474

No evidence that quadratic trend improves on linear (P=0.1474).

Residuals Versus Time

augment(tourism.lm, tourism) |>
    ggplot(mapping = aes(x = Date, y = .resid)) + geom_line()

The time plot of residuals for the linear trend model suggests that there are extended periods when the residuals are almost all negative (1986-1988) and extended periods where the residuals are almost all positive (1989-1990).

Such behaviour should not be observed if the errors are independent.

However, for time series data it is common for some residual correlation between residuals to remain even when the trend and seasonal variation has been removed.

This type of correlation in the sequence of residuals is usually called autocorrelation.

Stationary Processes and Autocorrelation

Consider a random process in time: \(Z_t\) where \(t=1,2,\ldots\) and \(Z_t\) represents the value of the process at time \(t\).

This process is said to be (weakly) stationary if:

• \(\mathbf{E}[Z_t]\) and \(\mathsf{Var}(Z_t)\) do not change with time \(t\).
• The correlation \(\mathsf{Corr}(Z_t, Z_{t+k})\) depends only on the time lag \(k\).

It is common to model the residuals from a time series as a stationary random process with zero mean.

For a stationary process, the autocorrelation function (or ACF) is defined by \[\rho(k) = \mathsf{Corr}(Z_t, Z_{t+k})\]

The ACF (autocorrelation against time lag) can be plotted in R using the acf() command.

Tourism Data: ACF Plot for Residuals

acf(resid(tourism.lm))

• The ACF plot indicates a correlation of about \(0.4\) at lag one. Hence consecutive residuals are positively dependent.
• The dashed horizontal lines on the plot are a 95% confidence interval under the assumption that the true autocorrelation is zero.
• Any correlation lying within this confidence interval may be just noise.
• Any correlation lying outside this confidence interval is probably indicative of true serial dependence in the data.

For the tourism data it seems that there is serial dependence in the data, since the correlations at lags 1, 2, 3 and 4 all extend beyond the confidence interval bounds.

The existence of several significant correlations is common when autocorrelation exists. If the correlation between the \(i\)^th and \(j\)^th variables is high, and so is the correlation between the \(j\)^th and \(k\)^th variables, we ought to expect the correlation between the \(i\)^th and \(k\)^th variables to also be high.

When we are thinking about time series, the correlation for data from lags 0 and 1 is the same as the data from lags 1 and 2 because they are both sets formed by pairs of successive observations.

If observations are correlated with their preceding observations, and those preceding observations are correlated with their preceding observations, then there is likely to be a correlation between observations and those that are back two time steps.

This logic continues to three, four, and greater time lags and is especially likely when the correlation at the first lag is very, very high.