MANCOVA and MANOVA Discussion > Queries regarding ANCOVA and MANCOVA
1) I'm afraid I can't tell you if it is "OK" to do that or not, because it is a conceptual questions/issue that you must determine based on what you'd expect from previous literature. If what you want to know is simply the difference between groups, holding pre-test scores constant then an ANCOVA (as you've described) is fine. HOWEVER, if previous literature would suggest that additional variables are likely influential on your DV, and may confound results, then it would be advisable to hold those variables constant also.
2) I'm not sure I understand this question: if you are running a MANCOVA then don't you have both the pretest and posttest as DVs (repeated measures)? If so, then it can't be included as a covariate. Alternatively, if you meant including the multiple subscales in the MANCOVA (not repeated measures), I might have some concerns about that, because of how the partial estimates would impact your results (each effect would control for the pre-test for all subscales).
3) I think using a bonferroni is a reasonable strategy for testing multiple subscales, yes. As for whether it is "OK" or not, there is no objective "right" or "wrong", these are decisions that are made for different conceptual reasons, not based on right/wrong answer.
4) The most common post-hoc used is Tukey's.
I hope this helps and let me know if my responses are unclear or if you have follow-up questions!
Jeremy Taylor
Respected Jeremy,
Thanks a lot for your reply.
Let me elaborate my previous queries 1 & 2 a little bit more.
Query 1 :
The most common observation, I have made so far while going through the previous literature, is that ANCOVA is being used most commonly in all types of of experimental studies (quasi-experimental or true experimental) by taking pre-test measures, IQ & previous year scores as covariates.
The same covariates are being used even if the groups are not significantly different from each other as far as their scores on these covariates are concerned.
Your suggestions in this regard please.
Query 2 :
Since achievement test consists of 3 subsclaes (namely, knowledge, comprehension, application), so I thought of doing MANCOVA on achievement (overall, including 3 subscales) by taking pre-achievement as covariate & ANCOVA on each subscale by taking pre-test measures on each subscale as covariate.
Same thing for other 2 dependent variables.
I observed that a no. of researchers have done this in their studies. Your suggestions please.
I also thought of doing MANCOVA with repeated measures but I still need to go through its assumptions.
Additional Query:
Can the same covariates be used for all dependent variables except their respective pre-test measures? I mean to say:
(a) Can I use the following as covariates for Achievement:
Pre-test measures on achievement, previous year achievement scores, IQ
(b) Can I use the following as covariates for Attitude:
Pre-test measures on Attitude, previous year achievement scores, IQ
So, for achievement & attitude, there are 2 common covariates: previous year achievement scores, IQ
Your suggestions please.
Thanks.
Waiting for your reply.
Uzaima
Hello again Uzaima!
In most of your questions you're asking whether you "can" do one thing or another In your analysis. My answer would be that you absolutely can, because most questions about how to build a model in statistics are not a matter of right or wrong, but rather a matter of what your intentions are in the model and what influence it will have to run it differently.
You mentioned that most of the literature in your field conduct their analysis in a certain way, so it may be a good idea to replicate that method. However, I generally try to avoid including extraneous variables in my model, just for the sake of consistency across models, because it introduces unnecessary noise into my model, which can have various consequences (such as losing power to detect effects… for just one example).
Again, I don't mean to be difficult or vague, but I can't really answer the question about whether you "can" include certain variables, because it isn't a matter of whether it is right or wrong. Instead, the question is what influence will it have on your analysis. Just remember, whatever you include as a covariate in your model, will be held constant in the estimation of any other effect in that model. With that mind, you should only include in your model Variables that you want held constant when testing the effects in that model.
Jeremy Taylor
Respected Jeremy,
Thanks a lot for your reply.
I think I will go for ANOVA with repeated measures. I need some help.
1. If control group and experimental groups are not significantly different from each other at pre-test stage as far as their mean pre-test scores, IQ scores & previous year achievement scores are concerned, can they be considered as equivalent or matched groups?
2. As there are 2 between-group factors (teaching method & gender) & one within-group factor (time: pre & post), I am confused about its procedure, interpretation & presentation of results in tabular form. I searched a lot but did not find any satisfactory information on these.
Please tell the procedure of ANOVA with repeated measures in SPSS, its interpretation & presentation of results (APA style).
3. How to do MANOVA with repeated measures in SPSS?
According to you, which one will be more appropriate in my case: ANOVA with repeated measures OR MANOVA with repeated measures?
Waiting for your reply...
Uzaima
Uzaima,
I think what you are looking for is repeated measures in the General Linear Model menu in SPSS. Also, even if there are no differences in those covariates ate baseline, I would still test them as covariates in the model, to see if they are predictive when partial estimates are calculated.
Jeremy Taylor
Is there any problem in my experimental design? Please see the following:
In a QUASI-EXPERIMENTAL PRE-POST TEST STUDY (factorial design 3x2, instructional method x gender), 6 intact classes were taken & 2 instructors were involved. Out of 6 classes, 2 were randomly assigned to instructional method X, other 2 to method Y & the remaining 2 to method Z.
Each instructor was randomly assigned to teach 3 classes by following a different method so as to control teacher differences.
Thus, there are
1. two independent variables:
(a) teaching methods (3 levels/groups) (b) gender
2. three dependent variables:
(a) achievement (3 sublevels-knowledge, comprehension, application)
(b) attitude (measured by likert scale having 5 subscales)
(c) attitude (measured by likert scale having 5 subscales)
At pretest stage, it was found that all the 6 classes (comprising 3 different groups) were not significantly different from each other as far as their scores on relevant variables (covariates which are pretest scores) is concerned.
I have done 2-way ANCOVA and found the main effects of teaching method and gender as well as the ineraction effect of teaching method and gender on each dependent variable.
My problem is: Since 2 instructors/teachers were involved, each taught 3 classes & made use of each teaching method, shall I include teacher also as one of the independent variable and find its main effect as well its interaction effect with other independent variables on each dependent variable?
Uzaima
In a pre-post test experimental study having 3x2 factorial design, there are:
1. two independent variables:
(a) teaching methods (3 levels/groups) (b) gender
2. three dependent variables:
(a) achievement (3 sublevels-knowledge, comprehension, application)
(b) attitude (measured by likert scale having 5 subscales)
(c) attitude (measured by likert scale having 5 subscales)
Query 1: If the 3 groups are found to be equivalent at pre-test stage (either quasi or true experimental method is used), is it ok to go for ANCOVA using pre-test measures as covariates or is there a need to include intelligence or other relevant variables as covariates?
Query 2: Is it ok if MANCOVA is conducted on each dependent variable with pre-test measures as covariates to detect any significant differences among the 3 groups?(MANCOVA because sublevels/subscales of each dependent variable are correlated with each other)
Query 3: Is it ok if ANCOVA is conducted on each sublevel of each dependent variable with pre-test measures as covariates, by adopting Bonferroni method for controlling the type I error rates?
Query 4: What post hoc tests (in SPSS) can be used in this case?
Waiting for your reply,
Thanks