|
A note on Repeated Measures ANCOVA
Thomas, M. S. C., Annaz, D., Ansari, D., Serif, G., Jarrold, C., & Karmiloff-Smith, A. (2009). Using developmental trajectories to understand developmental disorders. Journal of Speech, Language, and Hearing Research, 52, 336-358. Click here for PDF version (837K).
To reference this worksheet, please cite the paper that it accompanies.
Conceptually, the addition of a covariate (which is a between-subjects measure) to a repeated measures analysis should not alter the main effects of within-subjects factors. The two are orthogonal: e.g., Bill scores 70% for task 1 and 80% for task 2, but for each data point his age is the same. Ted scores 60% and 75%, but for each data point his age is the same. The mean performance of the whole group on task 1 versus task 2 should not depend on each participant's age; that is, it should not be based on a computation that involves the covariate of age, since the means for task 1 and task 2 are collapsed across age.
Of course, the covariate may interact with the repeated measure: the difference between Bill’s performance on task 1 and task 2 and Ted’s performance on task 1 and task 2 may indeed partly depend on their ages (this would produce a significant Task*Age interaction). But the reliability of the overall difference in the mean performance on task 1 versus task 2 should not.
Unfortunately, it turns out that the repeated measures ANCOVA does change the main effect of the repeated measure compared to assessing this main effect via a simple repeated measures ANOVA. Delaney and Maxwell (1981, p.107) discuss the origin of this difference and calculate the difference in sums of squared error that arises from the addition of a covariate. Broadly, the difference arises because the repeated measures factor is calculated in terms of a difference score (i.e., task1 - task2 for each individual); the covariation of the covariate and the difference score is used to adjust the sums-of-squares. More specifically, the estimate of the main effect of the repeated measure will only remain the same after the addition of the covariate if either the regression coefficient for predicting the mean difference score from the covariate is zero, or the mean of the covariate is itself zero. The change in sum of squared error for the model after the addition of the covariate can be shown to be
where X is the covariate and D is the difference score (Delaney & Maxwell, 1981, equation 3).
Typically, the result of this adjustment is to make the main effect of the repeated measure weaker, as if the between-subject covariate is taken to explain some of the variability in the (t1-t2) difference score. This is particularly the case when the variability explained by the covariate (e.g., change in performance across age) is much larger than the variability explained by the repeated measure (the task effect). This is quite a common occurrence in our data. The result is an overly conservative test of the within-subjects effect.
Delaney and Maxwell (1981) discuss a method for eliminating the change in the main effect by mean centring the covariate prior to running the ANCOVA. For example, in a trajectory analysis, you first calculate the mean age of the group. Then each individual’s age would be replaced by (age – mean age). The new mean age of the overall group when this adjustment is carried out is then (by definition) equal to zero. This protects the covariate from altering the main effect of the repeated measure. (For discussion of the Delaney-Maxwell method, see Aligna, 1982, esp. p.127-128).
Our preferred solution is to treat the analysis of the repeated measures factors separately to the influence of the covariate. This is because, as with the between-groups comparison, we wish to rescale the age covariate only to ensure that we measure the task difference at the earliest age – consistent with the idea of measuring different at onset. If the covariate is rescaled so the earliest measured age is zero, the mean of the age covariate will not be zero. Using the 2-phase analysis, we first assess main effect of task (and any other repeated measures and interactions between repeated measures) in a separate ANOVA. Then the covariate is introduced in order to examine the main effect of age and the interaction of age with any repeated measures factors in an ANCOVA. Importantly, note that the main effect of the covariate and the interactions of the covariate with repeated-measures factors in the ANCOVA are the same whether the Delaney-Maxwell method of rescaling is used or not. It is the main effect of the repeated measures factor that is sensitive to the rescaling.
However, if one wishes to make a claim like the following: "the main effect of task was present from the earliest age measured", then this refers to a difference between the intercepts of two regression lines. This claim must be assessed by the main effect of task taken from the ANCOVA. The test will be more conservative than the main effect of task taken from the ANOVA.
An example of the use of repeated-measures and mixed-design ANCOVA can be found in this paper (see p. 466 onwards).
An alternative approach to carrying out repeated measures linear regression analyses via the use of dummy variables can be found in Pedhazur (1977; see the ‘criterion scaling method’). See Williams (1959) for an approximate method for testing the significance of a difference between two non-independent correlation coefficients, due to Hotelling [1940] based on a deriving a t statistic (e.g., for comparing the correlations of y with x1 and x2, based on a sample of n observations).
References:
Aligna, J. (1982). Remarks on the analysis of covariance in repeated measures designs. Multivariate Behavioral Research, 17, 117-130.
Delaney, H. D., & Maxwell, S. E. (1981). On using analysis of covariance in repeated measures designs. Multivariate Behavioral Research, 16, 105-123.
Pedhazur, E. J. (1977). Multiple regression in behavioral research: Explanation and prediction, 3rd Edition. London: Harcourt Brace.
Williams, E. J. (1959). 136. query: Significance of difference between two non-independent correlation coefficients. Biometrics, 15(1), 135-136.
Reader Query
Question:
I am hoping you can finally clarify a question I have had for a long time.
I employ A-B-A' designs for experimental studies of psychophysiological and emotional reactions to social stressors. The B portion of the design is the stressor, usually a speech, that is preceded and followed by resting baseline periods. Thus, the main effect of the within-subjects factor is starkly quadratic. My interest lies in finding factors to account for individual differences in those quadratic profiles. Thus, I use both randomized (i.e., experimental) and empirically derived (i.e., median splits) groups to compare profiles. However, I also have time-invariant covariates such as age, traits, and baseline levels of related measures (e.g., emotional state before start of the study and before experimental instructions given). It is this last issue about which I have a question.
When I include some time-invariant covariates in a repeated-measures profile analysis using GLM in SPSS, the quadratic (A-B-A') main effect of a primary DV (e.g., heart rate) disappears - that is I have a partial eta squared of .6 or more without the covariate that then drops to a non-significant .1 or less when the covariate is included in a second analysis. My interpretation has been that the covariate therefore "explains" individual differences in those profiles. However, from the few sources that tackle this issue directly (including yours), it seems that (a) any change in the repeated main effects due to the covariate are artifacts.
Answer: Yes, that's correct.
(b) centering the covariate will get rid of those artifacts
Answer: Yes.
(c) only the between subjects effects (main effects and interactions) can be interpreted (d) it is not appropriate to interpret the difference between models that do and do not include the covariate.
Answer: I think the key distinction here is between the main effect of the primary DV (e.g., of heart rate) and the interaction between this factor and the covariate (e.g., of age). If you add a covariate to the repeated measures design and find that the main repeated measures effect is no longer reliable, this is an artifact. It can be demonstrated by mean centering the covariate.
However, there is a direct test of whether the effect of the repeated measure (your individual profile of response) is changing across the range of the covariate. This is the interaction between the repeated measures factor and the covariate.
I use separate models to assess (a) the pure repeated measures effects (excluding the covariate) and (b) interactions between repeated measures and between-participant factors such as covariates. The separation of the models is mainly on conception grounds - the repeated measure is independent of the covariate, and so its reliability can be assessed in a separate model.
I suppose you could compare these two models for the proportion of variance they account for; but in developing the methods on the website, I have generally been driven by getting the stats to reflect the patterns I am observing in the data. The conceptual side drives the statistical methods.
If you do find an interaction between the quadratic repeated measure and the covariate, this would still need to be interpreted. My preference would probably be to reduce it to a 2-level factor to visualise the interaction, e.g. by plotting two trajectories, one linking the covariate to differences score of A-B and a second linking the covariate to the difference scores of B-A'.
© Michael Thomas 2009
Last edited by MT 06/10/09