Lecture 24: Factors with Interactions

161.251 Regression Modelling

View the latest recording of this lecture

The two factor models that we have considered so far have assumed that the overall treatment effect is purely the sum of the main effects of the factors.

To put it another way, we have assumed that the effect of receiving level 3 of factor A is the same irrespective of the level of factor B.

Sometimes this will not be the case because the factors ‘interact’.

We will explore this issue in this lecture.

An Intuitive Introduction to Interactions

Suppose we are looking at two drugs that may reduce hypertension (high blood pressure).

Factor A has two levels: drug 1 absent, and drug 1 present.

Factor B has two levels: drug 2 absent, and drug 2 present.

Suppose that drug 1 on its own reduces blood pressure by 3 units.

Suppose that drug 2 on its own reduces blood pressure by 5 units.

It is not necessarily reasonable to assume that combined effect of drugs will be to reduce blood pressure by 8 units. Drugs may interact to give combined effect smaller than this.

The ‘main effects’ model for two factors is

\[Y_{ijk} = \mu + \alpha_i + \beta_j + \varepsilon_{ijk}\]

For the two drug example we would have \(\alpha_2 = -3\) and \(\beta_2 = -5\).

According to the main effects model, the treatment effect with both drugs (relative to the baseline level) is \(\alpha_2 + \beta_2 = -8\).

To represent an interactive effect between the drugs we need to introduce an extra term in the model:

\[Y_{ijk} = \mu + \alpha_i + \beta_j + (\alpha\beta)_{ij} + \varepsilon_{ijk}\]

Here \((\alpha\beta)_{ij}\) is the so-called interaction term in the model.

The notation \((\alpha\beta)_{ij}\) represents new parameters (not a ’multiplication’ of the \(\alpha\) and \(\beta\) parameters).

It is necessary to impose constraints on the interaction parameters to avoid overparameterization.

The treatment constraint sets all \((\alpha\beta)_{ij}=0\) when either i or j is one.

Hence, in the two drug example, only \((\alpha\beta)_{22}\) is non zero.

For the model with interaction, the treatment effect in the presence of both drugs is \(\alpha_2 + \beta_2 + (\alpha\beta)_{22} = -8 + (\alpha\beta)_{22}\), so that \((\alpha\beta)_{22}\) here represents the change from the main effects model that is caused by the interaction of the drugs.

Two-Way Model with Interaction

For a general model with two factors, the model with interaction is

\[Y_{ijk} = \mu + \alpha_i + \beta_j + (\alpha\beta)_{ij} + \varepsilon_{ijk}\]

As we have seen, both the main effects parameters (\(\alpha_i\), \(\beta_j\)) and the interaction parameters \((\alpha\beta)_{ij}\) must be constrained. We assume use of the treatment constrained (all parameters indexed by level 1 of either factor are zero).

If factor A has K levels and factor B has L levels then there are (K-1)(L-1) unconstrained interaction parameters.

Modelling with an interaction term allows any pattern of means between the different factor levels (no particular structure).

Inference for the Two-Way Model with Interaction

Parameter estimation is done by the method of least squares. Fitted values and residuals continue to be defined in the usual way.

The first test that is commonly done with the interaction model is to assess whether the interaction term in the model is statistically significant.

This test will involve multiple parameters (in general), and so will be done using the F test.

If the interaction term is important, then no more testing is necessary.

In particular, if a model contains an interaction term then it must also contain the corresponding main effects (even if these appear to be non-significant).

Testing for an Interaction Between Factors

Testing for the importance of the interaction is equivalent to comparing models

\(M0:~~~Y_{ijk} = \mu + \alpha_i + \beta_j + \varepsilon_{ijk}\) and \(M1:~~~Y_{ijk} = \mu + \alpha_i + \beta_j + (\alpha\beta)_{ij} + \varepsilon_{ijk}\)

In terms of hypothesis tests for parameters, we wish to test

H₀: \((\alpha\beta)_{ij} = 0~\mbox{for all}~i, j\)

H₁: \((\alpha\beta)_{ij}~\mbox{not zero for all}~i, j\)

The F test statistic is

\[F = \frac{[RSS_{M0} - RSS_{M1}]/[(K-1)(L-1)]}{RSS_{M1}/(n-KL)}\]

with the corresponding P-value computed in the usual manner.

ANOVA Table for the Interaction Model

The information for conducting an F test for an interaction is usually displayed in an (extended) ANOVA table.

	Df	Sum Sq	Mean Sq	F value	P value
Factor A	\(K-1\)	\(SSA\)	\(MSA\)	\(f_a\)	\(P_A\)
Factor B (adj.A)	\(L-1\)	\(SSB|A\)	\(MSB|A\)	\(f_{B|A}\)	\(P_{B|A}\)
Interaction	\((K-1)(L-1)\)	\(SSI\)	\(MSI\)	\(f_I\)	\(P_I\)
Residual	\(n-KL\)	\(RSS\)	\(RMS\)
Total	\(n-1\)	\(TSS\)

The sum of squares for the interaction (\(SSI\) in the table) can be computed by subtraction (difference in RSS for models M0 and M1).

The F test statistic for the interaction is \(f_I = MSI/RMS\)

Rat Weight Gain and Diet

This Example concerns the data introduced in an earlier Task. The response is rat weight gain. There are two explanatory factors:

1. Protein: either beef or cereal;
2. Amount: either low or high.

The data contains ten replicates at each treatment.

Those rats

In this example we will fit a model with interaction.

Design Matrix for the Rat Diet Data

Write down the model for the rat diet data in matrix format.

Take care to specify clearly the design matrix.

Assume the treatment constraint on parameters (with reference levels chosen as Beef for Protein and High for Amount, following R’s default alphabetical allocation).

Analysis of Rat Diet Data

Download ratdiet.csv

## RatDiet <- read.csv(file = "ratdiet.csv", header = TRUE)
RatDiet.lm <- lm(Gain ~ Amount * Protein, data = RatDiet)

By default (alphabetical ordering), levels Beef for Protein and High for Amount are set as reference (level 1).

On the RHS of the R formula, the syntax A*B for factors A and B indicates the main effects of the factors themselves and their interaction, which is written A:B.

Hence, for example,
Gain ~ Amount*Protein
and
Gain ~ Amount + Protein + Amount:Protein
are the same formula.

Anova table:

anova(RatDiet.lm)

Analysis of Variance Table

Response: Gain
               Df Sum Sq Mean Sq F value  Pr(>F)  
Amount          1 1299.6 1299.60  5.8123 0.02114 *
Protein         1  220.9  220.90  0.9879 0.32688  
Amount:Protein  1  883.6  883.60  3.9518 0.05447 .
Residuals      36 8049.4  223.59                  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The P-value for testing for an interaction between Protein and Amount is P=0.055, from an F-statistic of 3.95 on 1, 36 degrees of freedom.

There is some weak evidence of the existence of an interaction, although it does not quite reach formal statistical significance at the 5% level.

Summary Table:

summary(RatDiet.lm)

Call:
lm(formula = Gain ~ Amount * Protein, data = RatDiet)

Residuals:
   Min     1Q Median     3Q    Max 
-29.90  -9.90   2.05  10.85  25.10 

Coefficients:
                        Estimate Std. Error t value Pr(>|t|)    
(Intercept)              100.000      4.729  21.148  < 2e-16 ***
AmountLow                -20.800      6.687  -3.110  0.00364 ** 
ProteinCereal            -14.100      6.687  -2.109  0.04201 *  
AmountLow:ProteinCereal   18.800      9.457   1.988  0.05447 .  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 14.95 on 36 degrees of freedom
Multiple R-squared:   0.23, Adjusted R-squared:  0.1658 
F-statistic: 3.584 on 3 and 36 DF,  p-value: 0.02297

Fitted Values

In the previous example, write down the fitted values for each of the four possible treatments.

Obtain these using R code.

cbind(RatDiet, RatDiet.lm$fitted.values)

   Gain Protein Amount RatDiet.lm$fitted.values
1    90    Beef    Low                     79.2
2    76    Beef    Low                     79.2
3    90    Beef    Low                     79.2
4    64    Beef    Low                     79.2
5    86    Beef    Low                     79.2
6    51    Beef    Low                     79.2
7    72    Beef    Low                     79.2
8    90    Beef    Low                     79.2
9    95    Beef    Low                     79.2
10   78    Beef    Low                     79.2
11   73    Beef   High                    100.0
12  102    Beef   High                    100.0
13  118    Beef   High                    100.0
14  104    Beef   High                    100.0
15   81    Beef   High                    100.0
16  107    Beef   High                    100.0
17  100    Beef   High                    100.0
18   87    Beef   High                    100.0
19  117    Beef   High                    100.0
20  111    Beef   High                    100.0
21  107  Cereal    Low                     83.9
22   95  Cereal    Low                     83.9
23   97  Cereal    Low                     83.9
24   80  Cereal    Low                     83.9
25   98  Cereal    Low                     83.9
26   74  Cereal    Low                     83.9
27   74  Cereal    Low                     83.9
28   67  Cereal    Low                     83.9
29   89  Cereal    Low                     83.9
30   58  Cereal    Low                     83.9
31   98  Cereal   High                     85.9
32   74  Cereal   High                     85.9
33   56  Cereal   High                     85.9
34  111  Cereal   High                     85.9
35   95  Cereal   High                     85.9
36   88  Cereal   High                     85.9
37   82  Cereal   High                     85.9
38   77  Cereal   High                     85.9
39   86  Cereal   High                     85.9
40   92  Cereal   High                     85.9