View the latest recording of this lecture

What is an Orthogonal Factorial Design?

What is a balanced and complete factorial design?

Consequences of an Orthogonal Factorial Design

In a two-way orthogonal design:

The idea of orthogonality can be extended to three or more factor models.

A Designed Experiment for Dairy Cattle

An experiment was performed to investigate butterfat content of milk (the response variable, measured as a percentage). The factors are:

Some cows
Some cows

Analysis of Dairy Cattle Data

Download cows.csv

## Cows <- read.csv(file = "cows.csv", header = TRUE)
head(Cows)
  X Butterfat    Breed    Age
1 1      3.74 Ayrshire Mature
2 2      4.01 Ayrshire  2year
3 3      3.77 Ayrshire Mature
4 4      3.78 Ayrshire  2year
5 5      4.10 Ayrshire Mature
6 6      4.06 Ayrshire  2year
Cows.lm.1 <- lm(Butterfat ~ Breed + Age, data = Cows)
anova(Cows.lm.1)
Analysis of Variance Table

Response: Butterfat
          Df Sum Sq Mean Sq F value Pr(>F)    
Breed      4 34.321  8.5803 50.1150 <2e-16 ***
Age        1  0.274  0.2735  1.5976 0.2094    
Residuals 94 16.094  0.1712                   
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Cows.lm.2 <- lm(Butterfat ~ Age + Breed, data = Cows)
anova(Cows.lm.2)
Analysis of Variance Table

Response: Butterfat
          Df Sum Sq Mean Sq F value Pr(>F)    
Age        1  0.274  0.2735  1.5976 0.2094    
Breed      4 34.321  8.5803 50.1150 <2e-16 ***
Residuals 94 16.094  0.1712                   
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The figures in the ANOVA tables for models Cows.lm.1 and Cows.lm.2 are identical, despite the difference in order in which the factors are considered. This occurs because of the orthogonal design.

There is overwhelming evidence of a breed effect (P-value smaller than \(2 \times 10^{-16}\)) on mean butterfat content.

There is no evidence of an age effect.

Summary Table

summary(Cows.lm.1)$coefficients
                      Estimate Std. Error   t value     Pr(>|t|)
(Intercept)             4.0077 0.10135464 39.541356 2.234024e-60
BreedCanadian           0.3785 0.13084828  2.892663 4.746491e-03
BreedGuernsey           0.8900 0.13084828  6.801771 9.480645e-10
BreedHolstein-Fresian  -0.3905 0.13084828 -2.984372 3.621196e-03
BreedJersey             1.2325 0.13084828  9.419306 3.155879e-15
AgeMature               0.1046 0.08275572  1.263961 2.093694e-01
summary(Cows.lm.2)$coefficients
                      Estimate Std. Error   t value     Pr(>|t|)
(Intercept)             4.0077 0.10135464 39.541356 2.234024e-60
AgeMature               0.1046 0.08275572  1.263961 2.093694e-01
BreedCanadian           0.3785 0.13084828  2.892663 4.746491e-03
BreedGuernsey           0.8900 0.13084828  6.801771 9.480645e-10
BreedHolstein-Fresian  -0.3905 0.13084828 -2.984372 3.621196e-03
BreedJersey             1.2325 0.13084828  9.419306 3.155879e-15

Note that by default, level one of Age (2 years old) and level one of Breed, Ayrshire, are set as the reference levels for the treatment constraint.

The Jerseys seem to provide the highest butterfat concentration.

We should look at model diagnostics.

Rats – Not Another Task!

This task concerns a complete and balanced experiment into rat weight gain.

Two factors:

  1. Protein: either beef or cereal;
  2. Amount: either low or high.

We have ten replicates at each treatment.

The rats again
The rats again

The following ANOVA table (with certain elements obscure by #) was obtained using R.

Df Sum Sq Mean Sq F value Pr(>F)
Amount 1 1299.6 # # 0.026
Protein 1 220.9 # # 0.345
Residuals 37 8933.0 #
  1. Calculate the obscured values.

  2. What can you conclude (if anything) about the effect of Protein ignoring the effects of Amount?

Mathematical formulation of factorial models

As we move towards more complex factorial models, with more than two factors and interactions, we need to start using the mathematical formulation of these models as they are more concise.

One-way ANOVA model

Remember that the one-way model \[Y_i = \mu + \alpha_2 z_{i2} + \ldots + \alpha_K z_{iK} + \varepsilon_i~~~~~(i=1,2,\ldots,n)\]

can be written as \[\boldsymbol{Y_{ij} = \mu + \alpha_i + \varepsilon_{ij}}\] where

  • \(Y_{ij}\) is the response of the \(j\)th unit at the \(i\)th level of the factor \((i=1,\ldots,K;~j=1,\ldots,n_i)\);

  • \(K\) denotes the number of levels, and \(n_i\) the number of observations at level \(i\) of the factor.

  • Assume treatment constraint \(\alpha_1=0\).

  • \(\varepsilon_{11}, \varepsilon_{12}, \ldots, \varepsilon_{Kn_K}\) are random errors satisfying assumptions A1 to A4.

Two-way (main effects) ANOVA model

A two-way model with factors A and B in multiple regression form \[Y_i = \mu + \alpha_2 z_{Ai2} + \ldots + \alpha_K z_{AiK} + \beta_2 z_{Bi2} + \ldots + \beta_L z_{BiL} + \varepsilon_i~~~~~(i=1,2,\ldots,n)\] becomes: \[\boldsymbol{Y_{ijk} = \mu + \alpha_i + \beta_j + \varepsilon_{ijk}}\]

where \(Y_{ijk}\) is the response for the \(k\)th unit at level \(i\) of factor A and level \(j\) of factor B (\(i=1,\ldots,K;~j=1,\ldots,L;~k=1,\ldots,n_{ij}\)).

  • \(\alpha_1, \ldots, \alpha_K\) and \(\beta_1, \ldots, \beta_L\) are parameters describing the ‘main effects’ of A and B respectively.

  • Assume treatment constraints, \(\alpha_1 = 0\) and \(\beta_1 = 0\).

  • \(\varepsilon_{111}, \varepsilon_{112}, \ldots, \varepsilon_{KLn_{KL}}\), are error terms satisfying assumptions A1–A4.

LS0tDQp0aXRsZTogIkxlY3R1cmUgMjM6IE9ydGhvZ29uYWwgRmFjdG9yaWFsIE1vZGVscyINCnN1YnRpdGxlOiAxNjEuMjUxIFJlZ3Jlc3Npb24gTW9kZWxsaW5nDQphdXRob3I6ICJQcmVzZW50ZWQgYnkgSm9uYXRoYW4gR29kZnJleSA8YS5qLmdvZGZyZXlAbWFzc2V5LmFjLm56PiIgIA0KZGF0ZTogIldlZWsgOCBvZiBTZW1lc3RlciAyLCBgciBsdWJyaWRhdGU6OnllYXIobHVicmlkYXRlOjpub3coKSlgIg0Kb3V0cHV0Og0KICBodG1sX2RvY3VtZW50Og0KICAgIGNvZGVfZG93bmxvYWQ6IHRydWUNCiAgICB0aGVtZTogeWV0aQ0KICAgIGhpZ2hsaWdodDogdGFuZ28NCiAgaHRtbF9ub3RlYm9vazoNCiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlDQogICAgdGhlbWU6IHlldGkNCiAgICBoaWdobGlnaHQ6IHRhbmdvDQogIGJlYW1lcl9wcmVzZW50YXRpb246IGRlZmF1bHQNCiAgd29yZF9kb2N1bWVudDogZGVmYXVsdA0KICBpb3NsaWRlc19wcmVzZW50YXRpb246IGRlZmF1bHQNCiAgc2xpZHlfcHJlc2VudGF0aW9uOiANCiAgICB0aGVtZTogeWV0aQ0KICAgIGhpZ2hsaWdodDogdGFuZ28NCiAgcGRmX2RvY3VtZW50OiBkZWZhdWx0DQotLS0NCg0KDQoNCg0KDQpbVmlldyB0aGUgbGF0ZXN0IHJlY29yZGluZyBvZiB0aGlzIGxlY3R1cmVdKGh0dHBzOi8vUi1SZXNvdXJjZXMubWFzc2V5LmFjLm56L3ZpZGVvcy8yNTFMMjMubXA0KQ0KPCEtLS0gRGF0YSBpcyBvbg0KaHR0cHM6Ly9yLXJlc291cmNlcy5tYXNzZXkuYWMubnovZGF0YS8xNjEyNTEvDQotLS0+DQoNCmBgYHtyIHNldHVwLCBwdXJsPUZBTFNFLCBpbmNsdWRlPUZBTFNFfQ0KbGlicmFyeShrbml0cikNCm9wdHNfY2h1bmskc2V0KGRldj1jKCJwbmciLCAicGRmIikpDQpvcHRzX2NodW5rJHNldChmaWcuaGVpZ2h0PTYsIGZpZy53aWR0aD03LCBmaWcucGF0aD0iRmlndXJlcy8iLCBmaWcuYWx0PSJ1bmxhYmVsbGVkIikNCm9wdHNfY2h1bmskc2V0KGNvbW1lbnQ9IiIsIGZpZy5hbGlnbj0iY2VudGVyIiwgdGlkeT1UUlVFKQ0Kb3B0aW9ucyhrbml0ci5rYWJsZS5OQSA9ICcnKQ0KbGlicmFyeSh0aWR5dmVyc2UpDQpsaWJyYXJ5KGJyb29tKQ0KYGBgDQoNCg0KPCEtLS0gRG8gbm90IGVkaXQgYW55dGhpbmcgYWJvdmUgdGhpcyBsaW5lLiAtLS0+DQoNCjwhLS0tIEdMTXNEYXRhIHBhY2thZ2UgaGFzIGJ1dHRlcmZhdCAtLS0+DQoNCg0KLSBXZSBrbm93IHRoYXQgdGhlIGltcG9ydGFuY2Ugb2Ygb25lIHByZWRpY3RvciBjYW4gZGVwZW5kIG9uIHdoZXRoZXIgb3INCiAgICBub3Qgd2UgaGF2ZSBhZGp1c3RlZCBmb3IgYW5vdGhlci4gVGhpcyBpcyBhbHNvIHRydWUgaWYgdGhlIHByZWRpY3RvcnMgYXJlIGZhY3RvcnMuDQoNCi0gVGhpcyBtZWFucyB0aGF0IHdlIG5lZWQgdG8gbG9vayBhdCBtdWx0aXBsZSBBTk9WQSB0YWJsZXMgaW4gb3JkZXIgdG8NCiAgICBwZXJmb3JtIGFsbCB0aGUgcG9zc2libGUgdGVzdHMgKHdpdGggcGFydGljdWxhciBwYXR0ZXJucyBvZg0KICAgIGFkanVzdG1lbnQpLg0KDQotIEhvd2V2ZXIsIGZvciBvcnRob2dvbmFsIGZhY3RvcmlhbCBkZXNpZ25zIGl0IHR1cm5zIG91dA0KICAgIHRoYXQgdGhlIGlzc3VlIG9mIGFkanVzdG1lbnQgaXMgb2Ygbm8gY29uY2Vybi4NCg0KIyMgV2hhdCBpcyBhbiBPcnRob2dvbmFsIEZhY3RvcmlhbCBEZXNpZ24/DQoNCi0gVGhlIHBhdHRlcm4gb2YgZmFjdG9yIGxldmVscyAoaS5lLiB0aGUgbnVtYmVyIG9mIG9ic2VydmF0aW9ucyBpbiB0aGUgZGlmZmVyZW50IGxldmVsIGdyb3VwcyBvZiB0aGUgZGlmZmVyZW50IGZhY3RvcnMpLCB3aGVuIGNob3NlbiBieSBhbiBleHBlcmltZW50ZXIsIGlzIGNhbGxlZCB0aGUgZmFjdG9yaWFsIGRlc2lnbi4gDQoNCi0gSW4gZXNzZW5jZSwgdGhlIHBhdHRlcm4gb2YgZmFjdG9yIGxldmVscyAodGhlIGZhY3RvcmlhbA0KICAgIGRlc2lnbiBpZiB0aGlzIHdhcyBjaG9zZW4gYnkgYW4gZXhwZXJpbWVudGVyKSBpcw0KICAgICoqb3J0aG9nb25hbCoqIGlmIHRoZSBzdW0gb2Ygc3F1YXJlcyBhdHRyaWJ1dGFibGUgdG8gb25lDQogICAgZmFjdG9yIGlzIHRoZSBzYW1lIHdoZXRoZXIgb3Igbm90IHRoZSBvdGhlciBmYWN0b3IgaGFzIGJlZW4gaW5jbHVkZWQNCiAgICBpbiB0aGUgbW9kZWwuDQoNCi0gSW4gdGVybXMgb2YgQU5PVkEgdGFibGVzLCB0aGlzIG1lYW5zIHRoYXQgKlNTQSogd2lsbCBiZSB0aGUgc2FtZQ0KICAgIHdoZXRoZXIgKkEqIGFwcGVhcnMgb24gdGhlIGZpcnN0IG9yIHNlY29uZCBsaW5lIG9mIHRoZSBBTk9WQSAgICAgdGFibGUuDQoNCi0gQSBiYWxhbmNlZCBhbmQgY29tcGxldGUgZmFjdG9yaWFsIGRlc2lnbiBpcyBvcnRob2dvbmFsLiBXZSB3aWxsIHNlZSB3aGF0IHRoZXNlIHR3byB0ZXJtcyBtZWFuLg0KDQojIyBXaGF0IGlzIGEgYmFsYW5jZWQgYW5kIGNvbXBsZXRlIGZhY3RvcmlhbCBkZXNpZ24/DQoNCi0gQSBmYWN0b3JpYWwgZGVzaWduIGlzICoqY29tcGxldGUqKiBpZiBvYnNlcnZhdGlvbnMgYXJlIG1hZGUNCiAgICBhdCBldmVyeSBwb3NzaWJsZSBjb21iaW5hdGlvbiBvZiBmYWN0b3IgbGV2ZWxzLCBvcg0KICAgIHRyZWF0bWVudC4NCg0KLSBGb3IgZXhhbXBsZSwgaWYgZmFjdG9yICpBKiBoYXMgMyBsZXZlbHMsIGFuZCBmYWN0b3IgKkIqIGhhcyA0DQogICAgbGV2ZWxzLCB0aGVuIGEgY29tcGxldGUgZGVzaWduIHJlcXVpcmVzIHRoYXQgd2Ugb2JzZXJ2ZSByZXNwb25zZXMgYXQNCiAgICBlYWNoIG9mIHRoZSBwb3NzaWJsZSAkM1x0aW1lczQgPSAxMiQgdHJlYXRtZW50cy4NCg0KLSBBIGZhY3RvcmlhbCBkZXNpZ24gaXMgKipiYWxhbmNlZCoqIGlmIHRoZSBzYW1lIG51bWJlciBvZg0KICAgIGV4cGVyaW1lbnRhbCB1bml0cyBhcmUgb2JzZXJ2ZWQgYXQgZWFjaCB0cmVhdG1lbnQuIEluIG90aGVyIHdvcmRzLCAqbn5pan4gPSByKiBpcyB0aGUgKGNvbnN0YW50KSAqKm51bWJlciBvZiByZXBsaWNhdGlvbnMqKiwNCiAgICBpLmUuIHRoZSBudW1iZXIgb2Ygb2JzZXJ2YXRpb25zIGluIGVhY2ggdHJlYXRtZW50Lg0KDQotIEJhbGFuY2UgYW5kIGNvbXBsZXRlbmVzcyBuZWVkIHRvIGJlIGFjaGlldmVkIGJ5IGRlc2lnbg0KICAgICh0aGV5IHdpbGwgbm90IHVzdWFsbHkgYmUgdGhlIGNhc2UgaW4gYW4gb2JzZXJ2YXRpb25hbCBzdHVkeSBvciBzdXJ2ZXkpLg0KDQojIyBDb25zZXF1ZW5jZXMgb2YgYW4gT3J0aG9nb25hbCBGYWN0b3JpYWwgRGVzaWduDQoNCkluIGEgdHdvLXdheSBvcnRob2dvbmFsIGRlc2lnbjoNCg0KLSBUaGUgKlAqLXZhbHVlcyBmb3IgZWFjaCBmYWN0b3IgaW4gdGhlIEFOT1ZBIHRhYmxlIHJlbWFpbiBwcmVjaXNlbHkgdGhlDQogICAgc2FtZSBpcnJlc3BlY3RpdmUgb2YgdGhlIG9yZGVyIGluIHdoaWNoIHRoZSBmYWN0b3JzIGFyZSBsaXN0ZWQuDQoNCi0gSW4gY29uc2lkZXJpbmcgdGhlIGltcG9ydGFuY2Ugb2YgZmFjdG9yICpCKiBpdCBkb2VzIG5vdCBtYXR0ZXINCiAgICB3aGV0aGVyIG9yIG5vdCB3ZSBoYXZlIGFkanVzdGVkIGZvciAqQSogKGFuZCB2aWNlIHZlcnNhKS4NCg0KVGhlIGlkZWEgb2Ygb3J0aG9nb25hbGl0eSBjYW4gYmUgZXh0ZW5kZWQgdG8gdGhyZWUgb3IgbW9yZSBmYWN0b3INCm1vZGVscy4NCg0KIyMgQSBEZXNpZ25lZCBFeHBlcmltZW50IGZvciBEYWlyeSBDYXR0bGUNCg0KDQoNCkFuIGV4cGVyaW1lbnQgd2FzIHBlcmZvcm1lZCB0byBpbnZlc3RpZ2F0ZSBidXR0ZXJmYXQgY29udGVudCBvZiBtaWxrICh0aGUgcmVzcG9uc2UgdmFyaWFibGUsIG1lYXN1cmVkIGFzIGEgcGVyY2VudGFnZSkuDQpUaGUgZmFjdG9ycyBhcmU6DQoNCi0gQ293IGJyZWVkLCB3aXRoIGZpdmUgbGV2ZWxzOiBBeXJzaGlyZSwgQ2FuYWRpYW4sIEd1ZXJuc2V5LCBIb2xzdGVpbi1GcmVzaWFuLCBKZXJzZXkuDQotIENvdyBhZ2UsIHdpdGggdHdvIGxldmVsczogbWF0dXJlLCBhbmQgMiB5ZWFycyBvbGQuDQoNCiFbU29tZSBjb3dzXSguLi9ncmFwaGljcy9jb3dzLmpwZykNCg0KDQoNCg0KLSAxMCByZXBsaWNhdGVzIChjb3dzKSBvYnNlcnZlZCBhdCBlYWNoIHRyZWF0bWVudCAoaS5lLiBjb21iaW5hdGlvbiBvZg0KICAgIGJyZWVkIGFuZCBtYXR1cml0eSkuDQoNCi0gVGhlIGRlc2lnbiBpcyBjb21wbGV0ZSAqKmFuZCoqIGJhbGFuY2VkLA0KICAgIHNvIHRoZXJlZm9yZSBvcnRob2dvbmFsLg0KDQojIyMgQW5hbHlzaXMgb2YgRGFpcnkgQ2F0dGxlIERhdGENCg0KYHIgeGZ1bjo6ZW1iZWRfZmlsZSgiLi4vLi4vZGF0YS9jb3dzLmNzdiIpYA0KDQpgYGB7ciBDb3dzLmxtLjEsIGVjaG89LTEsIGV2YWw9LTJ9DQpDb3dzIDwtIHJlYWQuY3N2KGZpbGU9Ii4uLy4uL2RhdGEvY293cy5jc3YiLCBoZWFkZXI9VFJVRSkNCkNvd3MgPC0gcmVhZC5jc3YoZmlsZT0iY293cy5jc3YiLCBoZWFkZXI9VFJVRSkNCmhlYWQoQ293cykNCkNvd3MubG0uMSA8LSBsbShCdXR0ZXJmYXQgfiBCcmVlZCArIEFnZSwgZGF0YT1Db3dzKQ0KYW5vdmEoQ293cy5sbS4xKQ0KYGBgDQoNCmBgYHtyIENvd3MubG0uMn0NCkNvd3MubG0uMiA8LSBsbShCdXR0ZXJmYXQgfiBBZ2UgKyBCcmVlZCwgZGF0YT1Db3dzKQ0KYW5vdmEoQ293cy5sbS4yKQ0KYGBgDQoNClRoZSBmaWd1cmVzIGluIHRoZSBBTk9WQSB0YWJsZXMgZm9yIG1vZGVscyBgQ293cy5sbS4xYCBhbmQgYENvd3MubG0uMmAgYXJlIGlkZW50aWNhbCwNCiAgICBkZXNwaXRlIHRoZSBkaWZmZXJlbmNlIGluIG9yZGVyIGluIHdoaWNoIHRoZSBmYWN0b3JzIGFyZSBjb25zaWRlcmVkLg0KICAgIFRoaXMgb2NjdXJzIGJlY2F1c2Ugb2YgdGhlIG9ydGhvZ29uYWwgZGVzaWduLg0KDQpUaGVyZSBpcyBvdmVyd2hlbG1pbmcgZXZpZGVuY2Ugb2YgYSBicmVlZCBlZmZlY3QgKCpQKi12YWx1ZSBzbWFsbGVyIHRoYW4NCiAgICAkMiBcdGltZXMgMTBeey0xNn0kKSBvbiBtZWFuIGJ1dHRlcmZhdCBjb250ZW50Lg0KICAgIA0KVGhlcmUgaXMgbm8gZXZpZGVuY2Ugb2YgYW4gYWdlIGVmZmVjdC4NCg0KIyMjIFN1bW1hcnkgVGFibGUNCg0KYGBge3Igc3VtbWFyaWVzQ293c30NCnN1bW1hcnkoQ293cy5sbS4xKSRjb2VmZmljaWVudHMNCnN1bW1hcnkoQ293cy5sbS4yKSRjb2VmZmljaWVudHMNCmBgYA0KDQpOb3RlIHRoYXQgYnkgZGVmYXVsdCwgbGV2ZWwgb25lIG9mIGBBZ2VgICgyIHllYXJzIG9sZCkgYW5kIGxldmVsIG9uZSBvZiBgQnJlZWRgLCBBeXJzaGlyZSwNCiAgICBhcmUgc2V0IGFzIHRoZSByZWZlcmVuY2UgbGV2ZWxzIGZvciB0aGUgdHJlYXRtZW50IGNvbnN0cmFpbnQuDQoNClRoZSBKZXJzZXlzIHNlZW0gdG8gcHJvdmlkZSB0aGUgaGlnaGVzdCBidXR0ZXJmYXQgY29uY2VudHJhdGlvbi4NCg0KV2Ugc2hvdWxkIGxvb2sgYXQgbW9kZWwgZGlhZ25vc3RpY3MuDQoNCg0KIyMgUmF0cyDigJMgTm90IEFub3RoZXIgVGFzayENCg0KVGhpcyB0YXNrIGNvbmNlcm5zIGEgY29tcGxldGUgYW5kIGJhbGFuY2VkIGV4cGVyaW1lbnQgaW50byByYXQgd2VpZ2h0IGdhaW4uDQoNClR3byBmYWN0b3JzOg0KDQoxLiBQcm90ZWluOiBlaXRoZXIgYmVlZiBvciBjZXJlYWw7DQoyLiBBbW91bnQ6IGVpdGhlciBsb3cgb3IgaGlnaC4NCg0KV2UgaGF2ZSB0ZW4gcmVwbGljYXRlcyBhdCBlYWNoIHRyZWF0bWVudC4NCg0KIVtUaGUgcmF0cyBhZ2Fpbl0oLi4vZ3JhcGhpY3MvcmF0cy5qcGcpDQoNCg0KDQoNClRoZSBmb2xsb3dpbmcgQU5PVkEgdGFibGUgKHdpdGggY2VydGFpbiBlbGVtZW50cyBvYnNjdXJlIGJ5IGAjYCkgd2FzDQpvYnRhaW5lZCB1c2luZyBSLg0KDQpgYGB7ciBlY2hvPUZBTFNFfQ0KUmF0RGlldCA8LSByZWFkLmNzdihmaWxlPSIuLi8uLi9kYXRhL3JhdGRpZXQuY3N2IiwgaGVhZGVyPVRSVUUpDQpNeVRhYmxlID0gYXMuZGF0YS5mcmFtZShhbm92YShsbShHYWluIH5BbW91bnQgKyBQcm90ZWluLCBkYXRhPVJhdERpZXQpKSkNCk15VGFibGVbLCJQcig+RikiXSA9IHJvdW5kKE15VGFibGVbLCJQcig+RikiXSwgNCkNCk15VGFibGVbLDM6NF0gPSIjIg0KTXlUYWJsZVszLDQ6NV0gPSIiDQprbml0cjo6a2FibGUoTXlUYWJsZSkNCmBgYA0KDQoxLiAgQ2FsY3VsYXRlIHRoZSBvYnNjdXJlZCB2YWx1ZXMuDQoNCjIuICBXaGF0IGNhbiB5b3UgY29uY2x1ZGUgKGlmIGFueXRoaW5nKSBhYm91dCB0aGUgZWZmZWN0IG9mIGBQcm90ZWluYCBpZ25vcmluZyB0aGUNCiAgICBlZmZlY3RzIG9mIGBBbW91bnRgPw0KDQojIyBNYXRoZW1hdGljYWwgZm9ybXVsYXRpb24gb2YgZmFjdG9yaWFsIG1vZGVscw0KDQpBcyB3ZSBtb3ZlIHRvd2FyZHMgbW9yZSBjb21wbGV4IGZhY3RvcmlhbCBtb2RlbHMsIHdpdGggbW9yZSB0aGFuIHR3bw0KZmFjdG9ycyBhbmQgaW50ZXJhY3Rpb25zLCB3ZSBuZWVkIHRvIHN0YXJ0IHVzaW5nIHRoZSBtYXRoZW1hdGljYWwNCmZvcm11bGF0aW9uIG9mIHRoZXNlIG1vZGVscyBhcyB0aGV5IGFyZSBtb3JlIGNvbmNpc2UuDQoNCiMjIyBPbmUtd2F5IEFOT1ZBIG1vZGVsDQoNClJlbWVtYmVyIHRoYXQgdGhlIG9uZS13YXkgbW9kZWwNCiQkWV9pID0gXG11ICsgXGFscGhhXzIgel97aTJ9ICsgXGxkb3RzICsgXGFscGhhX0sgel97aUt9ICsgXHZhcmVwc2lsb25faX5+fn5+KGk9MSwyLFxsZG90cyxuKSQkDQoNCmNhbiBiZSB3cml0dGVuIGFzICQkXGJvbGRzeW1ib2x7WV97aWp9ID0gXG11ICsgXGFscGhhX2kgKyBcdmFyZXBzaWxvbl97aWp9fSQkDQp3aGVyZQ0KDQotICRZX3tpan0kIGlzIHRoZSByZXNwb25zZSBvZiB0aGUgJGokdGggdW5pdCBhdCB0aGUgJGkkdGggbGV2ZWwNCiAgICBvZiB0aGUgZmFjdG9yICQoaT0xLFxsZG90cyxLO35qPTEsXGxkb3RzLG5faSkkOw0KDQotICRLJCBkZW5vdGVzIHRoZSBudW1iZXIgb2YgbGV2ZWxzLCBhbmQgJG5faSQgdGhlIG51bWJlciBvZg0KICAgIG9ic2VydmF0aW9ucyBhdCBsZXZlbCAkaSQgb2YgdGhlIGZhY3Rvci4NCg0KLSBBc3N1bWUgdHJlYXRtZW50IGNvbnN0cmFpbnQgJFxhbHBoYV8xPTAkLg0KDQotICRcdmFyZXBzaWxvbl97MTF9LCBcdmFyZXBzaWxvbl97MTJ9LCBcbGRvdHMsIFx2YXJlcHNpbG9uX3tLbl9LfSQgYXJlIHJhbmRvbSBlcnJvcnMNCiAgICBzYXRpc2Z5aW5nIGFzc3VtcHRpb25zICoqKkExKioqIHRvICoqKkE0KioqLg0KDQojIyMgVHdvLXdheSAobWFpbiBlZmZlY3RzKSBBTk9WQSBtb2RlbA0KDQpBIHR3by13YXkgbW9kZWwgd2l0aCBmYWN0b3JzICpBKiBhbmQgKkIqIGluIG11bHRpcGxlIHJlZ3Jlc3Npb24gZm9ybQ0KJCRZX2kgPSBcbXUgKyBcYWxwaGFfMiB6X3tBaTJ9ICsgXGxkb3RzICsgXGFscGhhX0sgel97QWlLfSANCisgXGJldGFfMiB6X3tCaTJ9ICsgXGxkb3RzICsgXGJldGFfTCB6X3tCaUx9ICsgXHZhcmVwc2lsb25faX5+fn5+KGk9MSwyLFxsZG90cyxuKSQkDQpiZWNvbWVzOg0KJCRcYm9sZHN5bWJvbHtZX3tpamt9ID0gXG11ICsgXGFscGhhX2kgKyBcYmV0YV9qICsgXHZhcmVwc2lsb25fe2lqa319JCQNCg0Kd2hlcmUgJFlfe2lqa30kIGlzIHRoZSByZXNwb25zZSBmb3IgdGhlICRrJHRoIHVuaXQgYXQgbGV2ZWwgJGkkIG9mDQogICAgZmFjdG9yICpBKiBhbmQgbGV2ZWwgJGokIG9mIGZhY3RvciAqQiogKCRpPTEsXGxkb3RzLEs7fmo9MSxcbGRvdHMsTDt+az0xLFxsZG90cyxuX3tpan0kKS4NCg0KLSAkXGFscGhhXzEsIFxsZG90cywgXGFscGhhX0skIGFuZCAkXGJldGFfMSwgXGxkb3RzLCBcYmV0YV9MJCBhcmUNCiAgICBwYXJhbWV0ZXJzIGRlc2NyaWJpbmcgdGhlIOKAmG1haW4gZWZmZWN0c+KAmSBvZiAqQSogYW5kICpCKg0KICAgIHJlc3BlY3RpdmVseS4NCg0KLSBBc3N1bWUgdHJlYXRtZW50IGNvbnN0cmFpbnRzLCAkXGFscGhhXzEgPSAwJCBhbmQgJFxiZXRhXzEgPSAwJC4NCg0KLSAkXHZhcmVwc2lsb25fezExMX0sIFx2YXJlcHNpbG9uX3sxMTJ9LCBcbGRvdHMsIFx2YXJlcHNpbG9uX3tLTG5fe0tMfX0kLCBhcmUgZXJyb3INCiAgICB0ZXJtcyBzYXRpc2Z5aW5nIGFzc3VtcHRpb25zICoqKkEx4oCTQTQqKiouDQo=