Lecture 24 Factors with Interactions
The two factor models that we have considered so far have assumed that the overall treatment effect is purely the sum of the main effects of the factors.
To put it another way, we have assumed that the effect of receiving level 3 of factor A is the same irrespective of the level of factor B.
Sometimes this will not be the case because the factors ‘interact’.
We will explore this issue in this lecture.
24.1 An Intuitive Introduction to Interactions
Suppose we are looking at two drugs that may reduce hypertension (high blood pressure).
Factor A has two levels: drug 1 absent, and drug 1 present.
Factor B has two levels: drug 2 absent, and drug 2 present.
Suppose that drug 1 on its own reduces blood pressure by 3 units.
Suppose that drug 2 on its own reduces blood pressure by 5 units.
It is not necessarily reasonable to assume that combined effect of drugs will be to reduce blood pressure by 8 units. Drugs may interact to give combined effect smaller than this.
The ‘main effects’ model for two factors is
\[Y_{ijk} = \mu + \alpha_i + \beta_j + \varepsilon_{ijk}\]
For the two drug example we would have \(\alpha_2 = -3\) and \(\beta_2 = -5\).
According to the main effects model, the treatment effect with both drugs (relative to the baseline level) is \(\alpha_2 + \beta_2 = -8\).
To represent an interactive effect between the drugs we need to introduce an extra term in the model:
\[Y_{ijk} = \mu + \alpha_i + \beta_j + (\alpha\beta)_{ij} + \varepsilon_{ijk}\]
Here \((\alpha\beta)_{ij}\) is the so-called interaction term in the model.
The notation \((\alpha\beta)_{ij}\) represents new parameters (not a ’multiplication’ of the \(\alpha\) and \(\beta\) parameters).
It is necessary to impose constraints on the interaction parameters to avoid overparameterization.
The treatment constraint sets all \((\alpha\beta)_{ij}=0\) when either i or j is one.
Hence, in the two drug example, only \((\alpha\beta)_{22}\) is non zero.
For the model with interaction, the treatment effect in the presence of both drugs is \(\alpha_2 + \beta_2 + (\alpha\beta)_{22} = -8 + (\alpha\beta)_{22}\), so that \((\alpha\beta)_{22}\) here represents the change from the main effects model that is caused by the interaction of the drugs.
24.2 Two-Way Model with Interaction
For a general model with two factors, the model with interaction is
\[Y_{ijk} = \mu + \alpha_i + \beta_j + (\alpha\beta)_{ij} + \varepsilon_{ijk}\]
As we have seen, both the main effects parameters (\(\alpha_i\), \(\beta_j\)) and the interaction parameters \((\alpha\beta)_{ij}\) must be constrained. We assume use of the treatment constrained (all parameters indexed by level 1 of either factor are zero).
If factor A has K levels and factor B has L levels then there are (K-1)(L-1) unconstrained interaction parameters.
Modelling with an interaction term allows any pattern of means between the different factor levels (no particular structure).
24.3 Inference for the Two-Way Model with Interaction
Parameter estimation is done by the method of least squares. Fitted values and residuals continue to be defined in the usual way.
The first test that is commonly done with the interaction model is to assess whether the interaction term in the model is statistically significant.
This test will involve multiple parameters (in general), and so will be done using the F test.
If the interaction term is important, then no more testing is necessary.
In particular, if a model contains an interaction term then it must also contain the corresponding main effects (even if these appear to be non-significant).
24.4 Testing for an Interaction Between Factors
Testing for the importance of the interaction is equivalent to comparing models
\(M0:~~~Y_{ijk} = \mu + \alpha_i + \beta_j + \varepsilon_{ijk}\) and \(M1:~~~Y_{ijk} = \mu + \alpha_i + \beta_j + (\alpha\beta)_{ij} + \varepsilon_{ijk}\)
In terms of hypothesis tests for parameters, we wish to test
H0: \((\alpha\beta)_{ij} = 0~\mbox{for all}~i, j\)
H1: \((\alpha\beta)_{ij}~\mbox{not zero for all}~i, j\)
The F test statistic is
\[F = \frac{[RSS_{M0} - RSS_{M1}]/[(K-1)(L-1)]}{RSS_{M1}/(n-KL)}\]
with the corresponding P-value computed in the usual manner.
24.5 ANOVA Table for the Interaction Model
The information for conducting an F test for an interaction is usually displayed in an (extended) ANOVA table.
| Df | Sum Sq | Mean Sq | F value | P value | |
|---|---|---|---|---|---|
| Factor A | \(K-1\) | \(SSA\) | \(MSA\) | \(f_a\) | \(P_A\) |
| Factor B (adj.A) | \(L-1\) | \(SSB|A\) | \(MSB|A\) | \(f_{B|A}\) | \(P_{B|A}\) |
| Interaction | \((K-1)(L-1)\) | \(SSI\) | \(MSI\) | \(f_I\) | \(P_I\) |
| Residual | \(n-KL\) | \(RSS\) | \(RMS\) | ||
| Total | \(n-1\) | \(TSS\) |
The sum of squares for the interaction (\(SSI\) in the table) can be computed by subtraction (difference in RSS for models M0 and M1).
The F test statistic for the interaction is \(f_I = MSI/RMS\)
24.6 Rat Weight Gain and Diet
This Example concerns the data introduced in an earlier Task. The response is rat weight gain. There are two explanatory factors:
- Protein: either beef or cereal;
- Amount: either low or high.
The data contains ten replicates at each treatment.
In this example we will fit a model with interaction.
24.6.1 Design Matrix for the Rat Diet Data
Write down the model for the rat diet data in matrix format.
Take care to specify clearly the design matrix.
Assume the treatment constraint on parameters (with reference levels
chosen as Beef for Protein and High for Amount, following R’s default alphabetical
allocation).
24.6.2 Analysis of Rat Diet Data
## RatDiet <- read.csv(file = "ratdiet.csv", header = TRUE)
RatDiet.lm <- lm(Gain ~ Amount * Protein, data = RatDiet)By default (alphabetical ordering), levels Beef for Protein and High for Amount are set as
reference (level 1).
On the RHS of the R formula, the syntax A*B for factors A and
B indicates the main effects of the factors themselves and their
interaction, which is written A:B.
Hence, for example,
Gain ~ Amount*Protein
and
Gain ~ Amount + Protein + Amount:Protein
are the same formula.
Anova table:
Analysis of Variance Table
Response: Gain
Df Sum Sq Mean Sq F value Pr(>F)
Amount 1 1299.6 1299.60 5.8123 0.02114 *
Protein 1 220.9 220.90 0.9879 0.32688
Amount:Protein 1 883.6 883.60 3.9518 0.05447 .
Residuals 36 8049.4 223.59
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1The P-value for testing for an interaction between Protein and Amount is
P=0.055, from an F-statistic of 3.95 on 1, 36 degrees of freedom.
There is some weak evidence of the existence of an interaction, although it does not quite reach formal statistical significance at the 5% level.
Summary Table:
Call:
lm(formula = Gain ~ Amount * Protein, data = RatDiet)
Residuals:
Min 1Q Median 3Q Max
-29.90 -9.90 2.05 10.85 25.10
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 100.000 4.729 21.148 < 2e-16 ***
AmountLow -20.800 6.687 -3.110 0.00364 **
ProteinCereal -14.100 6.687 -2.109 0.04201 *
AmountLow:ProteinCereal 18.800 9.457 1.988 0.05447 .
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 14.95 on 36 degrees of freedom
Multiple R-squared: 0.23, Adjusted R-squared: 0.1658
F-statistic: 3.584 on 3 and 36 DF, p-value: 0.0229724.6.3 Fitted Values
In the previous example, write down the fitted values for each of the four possible treatments.
Obtain these using R code.
Gain Protein Amount RatDiet.lm$fitted.values
1 90 Beef Low 79.2
2 76 Beef Low 79.2
3 90 Beef Low 79.2
4 64 Beef Low 79.2
5 86 Beef Low 79.2
6 51 Beef Low 79.2
7 72 Beef Low 79.2
8 90 Beef Low 79.2
9 95 Beef Low 79.2
10 78 Beef Low 79.2
11 73 Beef High 100.0
12 102 Beef High 100.0
13 118 Beef High 100.0
14 104 Beef High 100.0
15 81 Beef High 100.0
16 107 Beef High 100.0
17 100 Beef High 100.0
18 87 Beef High 100.0
19 117 Beef High 100.0
20 111 Beef High 100.0
21 107 Cereal Low 83.9
22 95 Cereal Low 83.9
23 97 Cereal Low 83.9
24 80 Cereal Low 83.9
25 98 Cereal Low 83.9
26 74 Cereal Low 83.9
27 74 Cereal Low 83.9
28 67 Cereal Low 83.9
29 89 Cereal Low 83.9
30 58 Cereal Low 83.9
31 98 Cereal High 85.9
32 74 Cereal High 85.9
33 56 Cereal High 85.9
34 111 Cereal High 85.9
35 95 Cereal High 85.9
36 88 Cereal High 85.9
37 82 Cereal High 85.9
38 77 Cereal High 85.9
39 86 Cereal High 85.9
40 92 Cereal High 85.9