In the previous lecture we looked at:
- the assumptions underlying the (simple) linear regression
model;
- tools for detecting failure of the assumptions.
In this lecture we shall consider what to do if the model assumptions
fail.
What to Do When You Find Outliers
A Reminder
- The existence of outliers impacts on all model assumptions.
- If there are outliers, check whether data have been mis-recorded (go
back to data source if possible).
- If an outlier is not due to transcription errors, then it may need
removing before refitting the model.
- In general one should refit model after removing a single outlier,
because removal of one outlier can alter the fitted models and hence
make other points appear less (or more) inconsistent with the
model.
What to do When Assumption A1 Fails
Failure of A1 indicates that the relationship between the mean
response and the predictor is not linear (as specified by the simple
linear regression model).
Failure of this assumption is very serious — all conclusions from the
model will be questionable.
One possible remedy is to transform the data. e.g. if responses curve
upwards then a log transformation of responses may help.
However, important to recognize that any transformation of the
response distribution will also effect the error variance. Hence fixing
one problem (lack of linearity) by transformation may create another
problem (heteroscedasticity).
An alternative approach is to use polynomial regression (covered
later in the course).
What to do When Assumption A2 Fails
Failure of A2 occurs most frequently because the data exhibit serial
correlation in time (or space).
Failure of this assumption will leave parameter estimates
unbiased, but standard errors will be
incorrect.
Hence failure of A2 will render test results and confidence intervals
unreliable.
We will look at regression models for time series later in the
course.
What to do When Assumption A3 Fails
Failure of this assumption will leave parameter estimates unbiased,
but standard errors will be incorrect.
Hence failure of A3 will render test results and confidence intervals
unreliable.
Perhaps most common form of heteroscedasticity is when error variance
increases with the mean response.
A common strategy is to transform the response variable. (We will see
this in an example later in this lecture.)
If the relationship looks linear, then a logarithmic transformation
of both response and predictor variables can sometimes help in this
case.
An alternative strategy is to use weighted linear
regression (covered later in the course).
What to do When Assumption A4 Fails
Failure of the assumption of normality for the distribution of the
errors is not usually a serious problem.
Removal of (obvious) outliers will typically improve the normality of
the standardized residuals.
Example: Fatalities on Mt Everest
Climbing Mt Everest is a dangerous activity. This dataset gives the
number of fatalities among climbers, per year, from 1960-2019. Records
date right back to 1922, but our focus is on the most recent and
relevant years. The mountain was closed to climbers in 2020, while in
2021 there were fewer climbers allowed on the mountain than in previous
years. Once we know that the number of climbers is as free to change as
in the years before 2020, we must either restrict our analysis (Plan A),
or find some way to incorporate that extra knowledge (Plan B). Plan B is
too advanced for the purposes of today’s lecture so let’s stick with
Plan A.
You could quite easily find plenty of articles online about this
scenario. e.g. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0236919
Download Everest.csv
if you want to replicate the examples below.
Everest = read.csv("../../data/Everest.csv", header = TRUE)
head(Everest)
Year Fatalities
1 1960 2
2 1961 0
3 1962 1
4 1963 1
5 1964 0
6 1965 0
Year Fatalities
55 2014 17
56 2015 19
57 2016 7
58 2017 5
59 2018 5
60 2019 12
Taking a look at this data graphically:
`geom_smooth()` using formula = 'y ~ x'
Clearly on average the number of Fatalities has been increasing.
Fitting a simple model to this data and checking the assumptions
gives…
Everest.lm1 = lm(Fatalities ~ Year, data = Everest)
plot(Everest.lm1)
We see that the spread of residuals has been increasing. This has
carried through to affect the normality in the residuals.
What about curvature? This is more subtle because it looks as if the
straight-line model is fine, but your eye does not take into account the
fact that there is a lower bound on the Y-values: there cannot be
negative numbers of Fatalities.
Now the fitted line does not go negative, so there is no problem,
right?
Wrong. A statistical model is not just about the fitted line, but
about modelling the whole data, including understanding the errors or
variation around the line. The constant variance assumption implies that
we should be able to see the same spread of errors above and below the
regression line all the way along the x-axis.
This is illustrated by the following ‘prediction interval’ graph
which indicates we should expect to be able to get data all through the
shaded area which is roughly \(\hat{y} \pm 2 S\) where S is the residual
standard error (we will see in the next topic how exactly this graph is
created).
`geom_smooth()` using formula = 'y ~ x'
To get a shaded area which stays above zero, we are going to need to
use a transformation, which will also reduce the distance between the
bounds at the left, perhaps something like this -
LS0tDQp0aXRsZTogIkxlY3R1cmUgNzogV2hhdCB0byBkbyBXaGVuIEFzc3VtcHRpb25zIEZhaWwiDQpzdWJ0aXRsZTogMTYxLjI1MSBSZWdyZXNzaW9uIE1vZGVsbGluZw0KYXV0aG9yOiAiUHJlc2VudGVkIGJ5IE1hdHRoZXcgUGF3bGV5IDxNLlBhd2xleUBtYXNzZXkuYWMubno+IiAgDQpkYXRlOiAiV2VlayAzIG9mIFNlbWVzdGVyIDIsIGByIGx1YnJpZGF0ZTo6eWVhcihsdWJyaWRhdGU6Om5vdygpKWAiDQpvdXRwdXQ6DQogIGh0bWxfZG9jdW1lbnQ6DQogICAgY29kZV9kb3dubG9hZDogdHJ1ZQ0KICAgIHRoZW1lOiB5ZXRpDQogICAgaGlnaGxpZ2h0OiB0YW5nbw0KICBodG1sX25vdGVib29rOg0KICAgIGNvZGVfZG93bmxvYWQ6IHRydWUNCiAgICB0aGVtZTogeWV0aQ0KICAgIGhpZ2hsaWdodDogdGFuZ28NCiAgaW9zbGlkZXNfcHJlc2VudGF0aW9uOg0KICAgIHdpZGVzY3JlZW46IHRydWUNCiAgICBzbWFsbGVyOiB0cnVlDQogIHdvcmRfZG9jdW1lbnQ6IGRlZmF1bHQNCiAgc2xpZHlfcHJlc2VudGF0aW9uOiANCiAgICB0aGVtZTogeWV0aQ0KICAgIGhpZ2hsaWdodDogdGFuZ28NCiAgcGRmX2RvY3VtZW50OiBkZWZhdWx0DQotLS0NCg0KDQoNCg0KPCEtLS0gRGF0YSBpcyBvbg0KaHR0cHM6Ly9yLXJlc291cmNlcy5tYXNzZXkuYWMubnovZGF0YS8xNjEyNTEvDQotLS0+DQoNCmBgYHtyIHNldHVwLCBwdXJsPUZBTFNFLCBpbmNsdWRlPUZBTFNFfQ0KbGlicmFyeShrbml0cikNCm9wdHNfY2h1bmskc2V0KGRldj1jKCJwbmciLCAicGRmIikpDQpvcHRzX2NodW5rJHNldChmaWcuaGVpZ2h0PTYsIGZpZy53aWR0aD03LCBmaWcucGF0aD0iRmlndXJlcy8iLCBmaWcuYWx0PSJ1bmxhYmVsbGVkIikNCm9wdHNfY2h1bmskc2V0KGNvbW1lbnQ9IiIsIGZpZy5hbGlnbj0iY2VudGVyIiwgdGlkeT1UUlVFKQ0Kb3B0aW9ucyhrbml0ci5rYWJsZS5OQSA9ICcnKQ0KbGlicmFyeSh0aWR5dmVyc2UpDQpsaWJyYXJ5KGJyb29tKQ0KYGBgDQoNCg0KPCEtLS0gRG8gbm90IGVkaXQgYW55dGhpbmcgYWJvdmUgdGhpcyBsaW5lLiAtLS0+DQoNCkluIHRoZSBwcmV2aW91cyBsZWN0dXJlIHdlIGxvb2tlZCBhdDoNCiAgICANCi0gdGhlIGFzc3VtcHRpb25zIHVuZGVybHlpbmcgdGhlIChzaW1wbGUpIGxpbmVhciByZWdyZXNzaW9uDQogICAgICAgIG1vZGVsOw0KLSB0b29scyBmb3IgZGV0ZWN0aW5nIGZhaWx1cmUgb2YgdGhlIGFzc3VtcHRpb25zLg0KDQpJbiB0aGlzIGxlY3R1cmUgd2Ugc2hhbGwgY29uc2lkZXIgd2hhdCB0byBkbyBpZiB0aGUgbW9kZWwNCiAgICBhc3N1bXB0aW9ucyBmYWlsLg0KDQojIyBXaGF0IHRvIERvIFdoZW4gWW91IEZpbmQgT3V0bGllcnMNCg0KQSBSZW1pbmRlcg0KDQotIFRoZSBleGlzdGVuY2Ugb2Ygb3V0bGllcnMgaW1wYWN0cyBvbiBhbGwgbW9kZWwgYXNzdW1wdGlvbnMuDQotIElmIHRoZXJlIGFyZSBvdXRsaWVycywgY2hlY2sgd2hldGhlciBkYXRhIGhhdmUgYmVlbiBtaXMtcmVjb3JkZWQgKGdvDQogICAgYmFjayB0byBkYXRhIHNvdXJjZSBpZiBwb3NzaWJsZSkuDQotIElmIGFuIG91dGxpZXIgaXMgbm90IGR1ZSB0byB0cmFuc2NyaXB0aW9uIGVycm9ycywgdGhlbiBpdCBtYXkgbmVlZA0KICAgIHJlbW92aW5nIGJlZm9yZSByZWZpdHRpbmcgdGhlIG1vZGVsLg0KLSBJbiBnZW5lcmFsIG9uZSBzaG91bGQgcmVmaXQgbW9kZWwgYWZ0ZXIgcmVtb3ZpbmcgYSBzaW5nbGUgb3V0bGllciwNCiAgICBiZWNhdXNlIHJlbW92YWwgb2Ygb25lIG91dGxpZXIgY2FuIGFsdGVyIHRoZSBmaXR0ZWQgbW9kZWxzIGFuZCBoZW5jZQ0KICAgIG1ha2Ugb3RoZXIgcG9pbnRzIGFwcGVhciBsZXNzIChvciBtb3JlKSBpbmNvbnNpc3RlbnQgd2l0aCB0aGUgbW9kZWwuDQoNCiMjIFdoYXQgdG8gZG8gV2hlbiBBc3N1bXB0aW9uIEExIEZhaWxzDQoNCkZhaWx1cmUgb2YgQTEgaW5kaWNhdGVzIHRoYXQgdGhlIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIHRoZSBtZWFuDQogICAgcmVzcG9uc2UgYW5kIHRoZSBwcmVkaWN0b3IgaXMgbm90IGxpbmVhciAoYXMgc3BlY2lmaWVkIGJ5IHRoZSBzaW1wbGUNCiAgICBsaW5lYXIgcmVncmVzc2lvbiBtb2RlbCkuDQoNCkZhaWx1cmUgb2YgdGhpcyBhc3N1bXB0aW9uIGlzIHZlcnkgc2VyaW91cyAtLS0gYWxsIGNvbmNsdXNpb25zIGZyb20gIHRoZSAgbW9kZWwgd2lsbCBiZSBxdWVzdGlvbmFibGUuDQoNCk9uZSBwb3NzaWJsZSByZW1lZHkgaXMgdG8gdHJhbnNmb3JtIHRoZSBkYXRhLiBlLmcuIGlmIHJlc3BvbnNlcyBjdXJ2ZSB1cHdhcmRzIHRoZW4gYSBsb2cgdHJhbnNmb3JtYXRpb24gb2YNCiAgICByZXNwb25zZXMgbWF5IGhlbHAuDQoNCkhvd2V2ZXIsIGltcG9ydGFudCB0byByZWNvZ25pemUgdGhhdCBhbnkgdHJhbnNmb3JtYXRpb24gb2YgdGhlDQogICAgcmVzcG9uc2UgZGlzdHJpYnV0aW9uIHdpbGwgYWxzbyBlZmZlY3QgdGhlIGVycm9yIHZhcmlhbmNlLiBIZW5jZQ0KICAgIGZpeGluZyBvbmUgcHJvYmxlbSAobGFjayBvZiBsaW5lYXJpdHkpIGJ5IHRyYW5zZm9ybWF0aW9uIG1heSBjcmVhdGUNCiAgICBhbm90aGVyIHByb2JsZW0gKGhldGVyb3NjZWRhc3RpY2l0eSkuDQoNCkFuIGFsdGVybmF0aXZlIGFwcHJvYWNoIGlzIHRvIHVzZSBwb2x5bm9taWFsIHJlZ3Jlc3Npb24NCiAgICAoY292ZXJlZCBsYXRlciBpbiB0aGUgY291cnNlKS4NCg0KIyMgV2hhdCB0byBkbyBXaGVuIEFzc3VtcHRpb24gQTIgRmFpbHMNCg0KRmFpbHVyZSBvZiBBMiBvY2N1cnMgbW9zdCBmcmVxdWVudGx5IGJlY2F1c2UgdGhlIGRhdGEgZXhoaWJpdA0KICAgIHNlcmlhbCBjb3JyZWxhdGlvbiBpbiB0aW1lIChvciBzcGFjZSkuDQoNCkZhaWx1cmUgb2YgdGhpcyBhc3N1bXB0aW9uIHdpbGwgbGVhdmUgcGFyYW1ldGVyIGVzdGltYXRlcyAqKip1bmJpYXNlZCoqKiwNCiAgICBidXQgc3RhbmRhcmQgZXJyb3JzIHdpbGwgYmUgaW5jb3JyZWN0Lg0KDQpIZW5jZSBmYWlsdXJlIG9mIEEyIHdpbGwgcmVuZGVyIHRlc3QgcmVzdWx0cyBhbmQgY29uZmlkZW5jZQ0KICAgIGludGVydmFscyB1bnJlbGlhYmxlLg0KDQpXZSB3aWxsIGxvb2sgYXQgcmVncmVzc2lvbiBtb2RlbHMgZm9yIHRpbWUgc2VyaWVzIGxhdGVyIGluIHRoZSBjb3Vyc2UuDQoNCiMjIFdoYXQgdG8gZG8gV2hlbiBBc3N1bXB0aW9uIEEzIEZhaWxzDQoNCkZhaWx1cmUgb2YgdGhpcyBhc3N1bXB0aW9uIHdpbGwgbGVhdmUgcGFyYW1ldGVyIGVzdGltYXRlcyB1bmJpYXNlZCwNCiAgICBidXQgc3RhbmRhcmQgZXJyb3JzIHdpbGwgYmUgaW5jb3JyZWN0Lg0KDQpIZW5jZSBmYWlsdXJlIG9mIEEzIHdpbGwgcmVuZGVyIHRlc3QgcmVzdWx0cyBhbmQgY29uZmlkZW5jZQ0KICAgIGludGVydmFscyB1bnJlbGlhYmxlLg0KDQpQZXJoYXBzIG1vc3QgY29tbW9uIGZvcm0gb2YgaGV0ZXJvc2NlZGFzdGljaXR5IGlzIHdoZW4gZXJyb3IgIHZhcmlhbmNlIGluY3JlYXNlcyB3aXRoIHRoZSBtZWFuIHJlc3BvbnNlLg0KDQpBIGNvbW1vbiBzdHJhdGVneSBpcyB0byB0cmFuc2Zvcm0gdGhlIHJlc3BvbnNlIHZhcmlhYmxlLiAoV2Ugd2lsbCBzZWUgdGhpcyBpbiBhbiBleGFtcGxlIGxhdGVyIGluIHRoaXMgbGVjdHVyZS4pDQoNCklmIHRoZSByZWxhdGlvbnNoaXAgbG9va3MgbGluZWFyLCB0aGVuIGEgbG9nYXJpdGhtaWMgdHJhbnNmb3JtYXRpb24gb2YgYm90aCByZXNwb25zZSBhbmQgcHJlZGljdG9yIHZhcmlhYmxlcyAgICBjYW4gc29tZXRpbWVzIGhlbHAgaW4gdGhpcyBjYXNlLg0KDQpBbiBhbHRlcm5hdGl2ZSBzdHJhdGVneSBpcyB0byB1c2UgKip3ZWlnaHRlZCBsaW5lYXIgcmVncmVzc2lvbioqIChjb3ZlcmVkIGxhdGVyIGluIHRoZSBjb3Vyc2UpLg0KDQojIyBXaGF0IHRvIGRvIFdoZW4gQXNzdW1wdGlvbiBBNCBGYWlscw0KDQpGYWlsdXJlIG9mIHRoZSBhc3N1bXB0aW9uIG9mIG5vcm1hbGl0eSBmb3IgdGhlIGRpc3RyaWJ1dGlvbiBvZiB0aGUNCiAgICBlcnJvcnMgaXMgbm90IHVzdWFsbHkgYSBzZXJpb3VzIHByb2JsZW0uDQoNClJlbW92YWwgb2YgKG9idmlvdXMpIG91dGxpZXJzIHdpbGwgdHlwaWNhbGx5IGltcHJvdmUgdGhlIG5vcm1hbGl0eSBvZiB0aGUgc3RhbmRhcmRpemVkIHJlc2lkdWFscy4NCg0KIyMgVXNpbmcgVHJhbnNmb3JtYXRpb25zIHRvIGNvcnJlY3QgZm9yIGhldGVyb3NjZWRhc3RpY2l0eQ0KDQpOLkIuIHRoaXMgaXMgYSBzdHJhdGVneSBub3QgYSByZWNpcGUuIFRoZXJlIGlzIGEgY2hhbmNlIHRoYXQgYSB0cmFuc2Zvcm1hdGlvbiBjYW4gcmVzb2x2ZSB0aGUgcHJvYmxlbSwgYnV0IHlvdSB3aWxsIG5vdCBmaW5kIG91dCBpZiBpdCB3YXMgc3VjY2Vzc2Z1bCB1bnRpbCBpdCBoYXMgYmVlbiB0cmllZCBhbmQgZXZhbHVhdGVkLg0KDQoNCiMjIEV4YW1wbGU6IEZhdGFsaXRpZXMgb24gTXQgRXZlcmVzdA0KDQoNCg0KQ2xpbWJpbmcgTXQgRXZlcmVzdCBpcyBhIGRhbmdlcm91cyBhY3Rpdml0eS4gIFRoaXMgZGF0YXNldA0KZ2l2ZXMgdGhlIG51bWJlciBvZiBmYXRhbGl0aWVzIGFtb25nIGNsaW1iZXJzLCBwZXIgeWVhciwgZnJvbSAxOTYwLTIwMTkuIFJlY29yZHMgZGF0ZSByaWdodCBiYWNrIHRvIDE5MjIsIGJ1dCBvdXIgZm9jdXMgaXMgb24gdGhlIG1vc3QgcmVjZW50IGFuZCByZWxldmFudCB5ZWFycy4gVGhlIG1vdW50YWluIHdhcyBjbG9zZWQgdG8gY2xpbWJlcnMgaW4gMjAyMCwgd2hpbGUgaW4gMjAyMSB0aGVyZSB3ZXJlIGZld2VyIGNsaW1iZXJzIGFsbG93ZWQgb24gdGhlIG1vdW50YWluIHRoYW4gaW4gcHJldmlvdXMgeWVhcnMuIE9uY2Ugd2Uga25vdyB0aGF0IHRoZSBudW1iZXIgb2YgY2xpbWJlcnMgaXMgYXMgZnJlZSB0byBjaGFuZ2UgYXMgaW4gdGhlIHllYXJzIGJlZm9yZSAyMDIwLCB3ZSBtdXN0IGVpdGhlciByZXN0cmljdCBvdXIgYW5hbHlzaXMgKFBsYW4gQSksIG9yIGZpbmQgc29tZSB3YXkgdG8gaW5jb3Jwb3JhdGUgdGhhdCBleHRyYSBrbm93bGVkZ2UgKFBsYW4gQikuIFBsYW4gQiBpcyB0b28gYWR2YW5jZWQgZm9yIHRoZSBwdXJwb3NlcyBvZiB0b2RheSdzIGxlY3R1cmUgc28gbGV0J3Mgc3RpY2sgd2l0aCBQbGFuIEEuDQoNCllvdSBjb3VsZCBxdWl0ZSBlYXNpbHkgZmluZCBwbGVudHkgb2YgYXJ0aWNsZXMgb25saW5lIGFib3V0IHRoaXMgc2NlbmFyaW8uIGUuZy4gaHR0cHM6Ly9qb3VybmFscy5wbG9zLm9yZy9wbG9zb25lL2FydGljbGU/aWQ9MTAuMTM3MS9qb3VybmFsLnBvbmUuMDIzNjkxOQ0KDQoNCmByIHhmdW46OmVtYmVkX2ZpbGUoIi4uLy4uL2RhdGEvRXZlcmVzdC5jc3YiKWAgaWYgeW91IHdhbnQgdG8gcmVwbGljYXRlIHRoZSBleGFtcGxlcyBiZWxvdy4NCg0KYGBge3IgZ2V0RXZlcmVzdH0NCkV2ZXJlc3QgPSByZWFkLmNzdigiLi4vLi4vZGF0YS9FdmVyZXN0LmNzdiIsIGhlYWRlcj1UUlVFKQ0KaGVhZChFdmVyZXN0KQ0KdGFpbChFdmVyZXN0KQ0KYGBgDQoNClRha2luZyBhIGxvb2sgYXQgdGhpcyBkYXRhIGdyYXBoaWNhbGx5Og0KDQpgYGB7ciBmaXJzdEV2ZXJlc3RQbG90LCBlY2hvPUZBTFNFfQ0KbGlicmFyeShnZ3Bsb3QyKQ0KZ2dwbG90KEV2ZXJlc3QsIG1hcHBpbmcgPSBhZXMoeD1ZZWFyLCB5PUZhdGFsaXRpZXMpKSArIGdlb21fcG9pbnQoKSArDQpnZW9tX3Ntb290aChtZXRob2QgPSAibG0iLCBzZT1GQUxTRSAgKSArDQp5bGFiKCJOdW1iZXIgb2YgQ2xpbWJpbmcgRmF0YWxpdGllcyIpICsgeGxhYigiWWVhciIpIA0KYGBgDQoNCg0KQ2xlYXJseSBvbiBhdmVyYWdlIHRoZSBudW1iZXIgb2YgRmF0YWxpdGllcyBoYXMgYmVlbiBpbmNyZWFzaW5nLiBGaXR0aW5nIGEgc2ltcGxlIG1vZGVsIHRvIHRoaXMgZGF0YSBhbmQgY2hlY2tpbmcgdGhlIGFzc3VtcHRpb25zIGdpdmVzLi4uDQoNCmBgYHtyIEV2ZXJlc3QubG0xfQ0KRXZlcmVzdC5sbTEgPSBsbShGYXRhbGl0aWVzfiBZZWFyLCBkYXRhPUV2ZXJlc3QpDQpwbG90KEV2ZXJlc3QubG0xKQ0KYGBgDQoNCldlIHNlZSB0aGF0IHRoZSBzcHJlYWQgb2YgcmVzaWR1YWxzIGhhcyBiZWVuIGluY3JlYXNpbmcuIFRoaXMgaGFzIGNhcnJpZWQgdGhyb3VnaCB0byBhZmZlY3QgdGhlIG5vcm1hbGl0eSBpbiB0aGUgcmVzaWR1YWxzLiANCg0KV2hhdCBhYm91dCBjdXJ2YXR1cmU/ICBUaGlzIGlzIG1vcmUgc3VidGxlIGJlY2F1c2UgaXQgbG9va3MgYXMgaWYgdGhlIHN0cmFpZ2h0LWxpbmUgbW9kZWwgaXMgZmluZSwgYnV0IHlvdXIgZXllIGRvZXMgbm90IHRha2UgaW50byBhY2NvdW50IHRoZSBmYWN0IHRoYXQgdGhlcmUgaXMgYSBsb3dlciBib3VuZCBvbiB0aGUgWS12YWx1ZXM6IHRoZXJlIGNhbm5vdCBiZSBuZWdhdGl2ZSBudW1iZXJzIG9mIEZhdGFsaXRpZXMuICAgDQoNCk5vdyB0aGUgZml0dGVkIGxpbmUgZG9lcyBub3QgZ28gbmVnYXRpdmUsIHNvIHRoZXJlIGlzIG5vIHByb2JsZW0sIHJpZ2h0PyAgDQoNCldyb25nLiBBIHN0YXRpc3RpY2FsIG1vZGVsIGlzIG5vdCBqdXN0IGFib3V0IHRoZSBmaXR0ZWQgbGluZSwgYnV0IGFib3V0IG1vZGVsbGluZyB0aGUgd2hvbGUgIGRhdGEsIGluY2x1ZGluZyB1bmRlcnN0YW5kaW5nIHRoZSBlcnJvcnMgb3IgdmFyaWF0aW9uIGFyb3VuZCB0aGUgbGluZS4gIFRoZSBjb25zdGFudCB2YXJpYW5jZSBhc3N1bXB0aW9uIGltcGxpZXMgdGhhdCB3ZSBzaG91bGQgYmUgYWJsZSB0byAgc2VlIHRoZSBzYW1lIHNwcmVhZCBvZiBlcnJvcnMgYWJvdmUgYW5kIGJlbG93IHRoZSByZWdyZXNzaW9uIGxpbmUgYWxsIHRoZSB3YXkgYWxvbmcgdGhlIHgtYXhpcy4gIA0KDQpUaGlzIGlzIGlsbHVzdHJhdGVkIGJ5IHRoZSAgZm9sbG93aW5nICdwcmVkaWN0aW9uIGludGVydmFsJyBncmFwaCB3aGljaCBpbmRpY2F0ZXMgd2Ugc2hvdWxkIGV4cGVjdCB0byBiZSBhYmxlIHRvIGdldCBkYXRhIGFsbCB0aHJvdWdoIHRoZSBzaGFkZWQgYXJlYSB3aGljaCBpcyAqcm91Z2hseSogJFxoYXR7eX0gXHBtIDIgUyQgd2hlcmUgUyBpcyB0aGUgcmVzaWR1YWwgc3RhbmRhcmQgZXJyb3IgKHdlIHdpbGwgc2VlIGluIHRoZSBuZXh0IHRvcGljIGhvdyBleGFjdGx5IHRoaXMgZ3JhcGggaXMgY3JlYXRlZCkuICANCg0KDQpgYGB7ciwgZWNobz1GQUxTRSwgd2FybmluZz1GQUxTRX0NCm5ld3ggPC0gc2VxKG1pbihFdmVyZXN0JFllYXIpLCBtYXgoRXZlcmVzdCRZZWFyKSwgYnk9MSkNCnByZWRfaW50ZXJ2YWwgPC0gcHJlZGljdChFdmVyZXN0LmxtMSwgbmV3ZGF0YT1kYXRhLmZyYW1lKFllYXI9bmV3eCksIGludGVydmFsPSJwcmVkaWN0aW9uIiwgbGV2ZWwgPSAwLjk1KQ0KcHJlZF9pbnRlcnZhbCA8LSBhcy5kYXRhLmZyYW1lKGNiaW5kKHByZWRfaW50ZXJ2YWwsRXZlcmVzdCkpDQpnZ3Bsb3QocHJlZF9pbnRlcnZhbCwgbWFwcGluZyA9IGFlcyh4PVllYXIsIHk9RmF0YWxpdGllcykpICsgZ2VvbV9wb2ludCgpICsgZ2VvbV9zbW9vdGgobWV0aG9kID0gImxtIiwgc2U9RkFMU0UpICsgZ2VvbV9yaWJib24oICBhZXMoeW1pbiA9IGx3ciwgeW1heCA9IHVwciksIGZpbGwgPSAibWVkaXVtIGJsdWUiLCBhbHBoYSA9IDAuMikgKyB5bGFiKCJOdW1iZXIgb2YgQ2xpbWJpbmcgRmF0YWxpdGllcyIpICsgeGxhYigiWWVhciIpDQpgYGANCg0KVG8gZ2V0IGEgc2hhZGVkIGFyZWEgd2hpY2ggc3RheXMgYWJvdmUgemVybywgd2UgYXJlIGdvaW5nIHRvIG5lZWQgdG8gdXNlIGEgdHJhbnNmb3JtYXRpb24sIHdoaWNoIHdpbGwgYWxzbyByZWR1Y2UgdGhlIGRpc3RhbmNlIGJldHdlZW4gdGhlIGJvdW5kcyBhdCB0aGUgbGVmdCwgcGVyaGFwcyBzb21ldGhpbmcgbGlrZSB0aGlzIC0gIA0KDQoNCmBgYHtyIEV2ZXJlc3QubG0yLCBlY2hvPUZBTFNFfQ0KRXZlcmVzdC5sbTI9IGxtKHNxcnQoRmF0YWxpdGllcysxKSB+ICBZZWFyICwgZGF0YT1FdmVyZXN0KQ0KcHJlZF9pbnRlcnZhbCA8LSAgICBwcmVkaWN0KEV2ZXJlc3QubG0yLCBuZXdkYXRhPWRhdGEuZnJhbWUoWWVhcj1uZXd4KSwgaW50ZXJ2YWw9InByZWRpY3Rpb24iLCBsZXZlbCA9IDAuOTUpIF4yDQpwcmVkX2ludGVydmFsIDwtIGFzLmRhdGEuZnJhbWUoY2JpbmQocHJlZF9pbnRlcnZhbCxFdmVyZXN0KSkNCmdncGxvdChwcmVkX2ludGVydmFsLCBtYXBwaW5nID0gYWVzKHg9WWVhciwgeT1GYXRhbGl0aWVzKSkgKyBnZW9tX3BvaW50KCkgICArIGdlb21fcmliYm9uKCAgYWVzKHltaW4gPSBsd3IsIHltYXggPSB1cHIpLCBmaWxsID0gIm1lZGl1bSBibHVlIiwgYWxwaGEgPSAwLjIpICsgeWxhYigiTnVtYmVyIG9mIENsaW1iaW5nIEZhdGFsaXRpZXMiKSArIHhsYWIoIlllYXIiKSsgZ2VvbV9saW5lKGFlcyh5PWZpdCkgICkNCmBgYA0KDQoNCiMjIFdoYXQgc29ydCBvZiB0cmFuc2Zvcm1hdGlvbiBzaG91bGQgd2UgY2hvb3NlPyANCg0KSWYgdGhlIHJlc3BvbnNlIHZhcmlhYmxlIFkgIGlzIGJvdW5kZWQgYmVsb3cgIChlLmcuIG5lZ2F0aXZlcyBhcmUgaW1wb3NzaWJsZSkgIGFuZCByaWdodC1za2V3ZWQgKHRlbmRpbmcgdG8gaGlnaCBwb3NpdGl2ZSBvdXRsaWVycykgdGhlbiBjb21tb24gdHJhbnNmb3JtYXRpb25zIGFyZSBzcXJ0KFkpIGFuZCBsb2coWSkuIFNvbWV0aW1lcyBzcXJ0KFkrMSkgb3IgbG9nKFkrMSkgaXMgdXNlZCB0byBpbXByb3ZlIHRoZSBmaXQsICAgZXNwZWNpYWxseSBpZiBzb21lIFk9MCAob25lIGNhbm5vdCB0YWtlIGxvZyBvZiAwKS4gIFRoZSBncmFwaCBhYm92ZSB3YXMgZml0dGVkIHVzaW5nIHRoZSBsaW5lYXIgbW9kZWwgDQokJCAgWV97bmV3fSAgPSAgXGJldGFfMCArIFxiZXRhXzEgXGhib3h7WWVhcn0gKyBcdmFyZXBzaWxvbiQkDQp3aGVyZSAkWV97bmV3fSA9IFxzcXJ0e1xoYm94e0ZhdGFsaXRpZXN9KzF9JC4gICAgVGhpcyBpcyBzdGlsbCByZWdhcmRlZCBhcyBhIGxpbmVhciBtb2RlbCBzaW5jZSB0aGUgJFxiZXRhJHMgYXJlIHNlcGFyYXRlZCBieSBwbHVzc2VzLiANCg0KVGhlIGZpdHRlZCBsaW5lIGhhcyB0aGVuIGJlZW4gYmFjay10cmFuc2Zvcm1lZCB0byB0aGUgb3JpZ2luYWwgc2NhbGUsIGhlbmNlIGl0IGhhcyBiZWNvbWUgYSAgY3VydmUsIHdoaWNoIGNhbiBiZSB3cml0dGVuIGFzICQkICBcaGF0e3l9ICA9ICAoXGhhdHtcYmV0YV8wfSArIFxoYXR7XGJldGFfMX0gXGhib3h7WWVhcn0pXjIgLTEgIC4kJA0KDQojIyMgSXMgdGhlIHRyYW5zZm9ybWF0aW9uIHN1aXRhYmxlPyANCg0KSW4gb25lIHNlbnNlIHRoZSBncmFwaCBhYm92ZSBhbnN3ZXJzIHRoZSBxdWVzdGlvbjogIHRoZSBzaGFkZWQgYXJlYSBzZWVtcyB0byBmaXQgdGhlIGRhdGEgcXVpdGUgd2VsbC4gQnV0IHRoZSBwcm9wZXIgYW5zd2VyIGlzIHRvIGxvb2sgYXQgdGhlIHJlc2lkdWFscyBpbiB0aGUgdHJhbnNmb3JtZWQgKCRZX3tuZXd9JCkgdW5pdHMuIA0KDQpgYGB7ciBzaG93RXZlcmVzdC5sbTJ9DQpFdmVyZXN0LmxtMj0gbG0oc3FydChGYXRhbGl0aWVzKzEpIH4gIFllYXIgLCBkYXRhPUV2ZXJlc3QpDQpwYXIobWZyb3c9YygyLDIpKQ0KcGxvdChFdmVyZXN0LmxtMikNCmBgYA0KDQpJdCBsb29rcyBsaWtlIHRoZSAgaGV0ZXJvc2NlZGFzdGljaXR5IChBMykgYW5kIG5vcm1hbGl0eSAoQTQpIGhhdmUgYmVlbiBpbXByb3ZlZCwgYW5kIHRoZXJlIGlzIG5vIHByb2JsZW0gd2l0aCBjdXJ2YXR1cmUgKEExKS4gVGhlIGdyYXBoIGluIG9yaWdpbmFsIHVuaXRzIGRvZXMgbm90IGdvIGludG8gaW1wb3NzaWJsZSB0ZXJyaXRvcnkuICBTbyB0aGUgbW9kZWwgYXBwZWFycyBzdWl0YWJsZS4gDQoNCiMjIyBXaGF0IGlzIHRoZSBqdXN0aWZpY2F0aW9uIGZvciB0aGlzIHRyYW5zZm9ybWF0aW9uPyANCg0KVGhlcmUgaXMgc29tZSBtYXRocyBiZWhpbmQgd2h5IHNxcnQgYW5kIGxvZyBhcmUgY29tbW9uIHRyYW5zZm9ybWF0aW9ucywgZXNwZWNpYWxseSBmb3IgZGF0YSBjb25zaXN0aW5nIG9mIGNvdW50ZWQgb2NjdXJyZW5jZXMsIGJ1dCB0aGUgcmVhc29uaW5nIHdpbGwgaGF2ZSB0byBiZSBsZWZ0IHRvIGFub3RoZXIgY291cnNlLiANCkZvciBvdXIgcHVycG9zZXMsIHdlIGp1c3QgbG9vayBmb3IgYSAgdHJhbnNmb3JtYXRpb24gIHRoYXQgZml4ZXMgdGhlIHJlc2lkdWFscyBvbiB0aGUgdHJhbnNmb3JtZWQgc2NhbGUuIFRoZW4gZm9yIHByZWRpY3Rpb24gd2UgYmFjay10cmFuc2Zvcm0gdGhlIHJlc3VsdC4gIFlvdSBjYW4gcmVhZCBtb3JlIGFib3V0IHRyYW5zZm9ybWF0aW9uIG1ldGhvZHMgaW4gYW4gQXBwZW5kaXggdG8gdGhpcyBib29rIG9mIG5vdGVzLiANCg0KVGhlIGNvdXJzZSAxNjEuMzMxIGNvbnNpZGVycyBzb21ldGhpbmcgY2FsbGVkIEdlbmVyYWxpemVkIExpbmVhciBNb2RlbHMsICB3aGljaCBhcmUgYSBjbGFzcyBvZiByZWdyZXNzaW9uIG1vZGVscyB1c2VkIHdoZW4gdGhlIHJlc3BvbnNlIHZhcmlhYmxlICRZJCBkb2VzIG5vdCBoYXZlIG5vcm1hbCBlcnJvcnMuIE1hbnkgb2YgdGhlIHRlY2huaXF1ZXMgeW91IHdpbGwgbGVhcm4gaW4gMTYxLjI1MSBnZW5lcmFsaXplIGVhc2lseSBpbnRvIHRoYXQgd2lkZXIgdmFyaWV0eSBvZiBjb250ZXh0cy4NCg0KDQo=