STATS MAKE ME CRY

I am using (or trying to use) SPSS v. 19.0.1 to perform multiple imputation.

Because SPSS seems to provide only some pooled results (e.g., for t, unstandardized b), how would I go about obtaining other pooled parameter estimates (e.g., F statistic, B, R square change, confidence intervals, df's, partial eta squared, etc)?

Any help would be much appreciated!

Thank you, Jeremy, that helps. I am wondering if this is the macro you are referring to:

http://www.spss.com/devcentral/index.cfm?pg=downloadDet&dId=55

I am not sure how to go about installing it, and if this, at all, is what I need?

Another question I have is how to go about reporting results? Is there an APA style for this?

Hi Jeremy,

I came across this post and it relieves me to see what you wrote - that I can get pooled (or summary) statistics for all of by simply calculating the "mean" across the imputation results. I've thought about it before but I've been hesitant and spent few days now to find a way.

So, I am planning now to take your suggestion. I have few questions though:
1. My data is categorical so is it acceptable to use "mode"?
2. Do I include the original data with missing values in this process? Intuitively I do not want to but not sure if I'm right.

Thank you!

hi
just wondering if a 'solution' to this was found. I am trying to pool estimates from repeated measures ANOVA following MI and was hoping there was an easy solution as PASW/SPSS does not seem to provide this option.
thanks
mitch

I am having the same issue discussed by various authors in this thread, and after reading the responses, I am still a little unclear about how I would proceed.

I ran a 2(time1 and time2 measure) x 2(gender) x 2(couple type) repeated measures ANOVA using 5 different data sets (create from 5 multiple imputations). I now have 5 different SumSquare, F-values, p-values, and partial eta-squared values for time main effect, gender main effect, time*coupletype, and time*coupletype*gender. What do I aggregate across the five data sets to get one final reported value?

Any assistance would be greatly appreciated!

Thanks!

Hi there,

I've run the default 5 multiple imputations in spss (v 19) on my dataset and I have split the cases according to the imputation_ variable and I have run a series of univariate GLMs with bonferoni correction. My output matches all of the text books and forums apart from the crucial part of providing the "pooled" section. When I run the analysis it has the swirly icon next to it suggesting it is supported but I don't understand why the output doesn't contain the pooled summary.

Do you know why this might be?

Best,
Emma

Hi Jeremy,

My question is similar to Allen's. Could you provide more explanation on how to get pooled effect for the statistics listed by Allen (e.g., F value for the main effect; F value for the interaction effect)? I am using SPSS.

Thanks much for your time!

I wish that were true Jeremy, but at least up to v.20, SPSS won't pool GLMs. A shame as it'd be extremely useful.

Hi Jeremy!

Could you recommend a way of pooling repeated-measures MANOVA analyses across imputation results in R? Following your suggestion I tried to tackle the problem in R; learned how to do repated measures MANOVA (using 'car' package), how to use the 'mi' package, but still I have a problem with pooling, because 'mi' package handles pooling only for more standard analyses.

You also recommended above calculating the mean of across the imputation results, to obtain a pooled result. What did you mean by that - averaging F-values, for example?

thanks in advance!
Jakub
ps. I believe the macro you referred to was this one: http://www.socialsciences.leiden.edu/educationandchildstudies/childandfamilystudies/organisation/staffcfs/van-ginkel.html

Jeremy, thanks!
I will check this out.
Jakub

Hi Jeremy

Im trying to do a one way repeated measures anova in spss with 5 imputated datasets
In earlier posts you mentioned pooling your results by simply calculating the mean of your results.
I am interested in the pooled F-value and corresponding p-value, so could I just take the mean of my 5 F-values from the 5 imputated datasets?
How would I find the corresponding pooled p-value?
Because my data violates the assumption of sphericity I would use the Greenhouse-Geisser corrected results.
These results show different df's per imputated dataset. Should I also take the mean degrees of freedom to find the significance level in an F-table?

I'm really confused so I hope I made some sense to you and you can help me out!

According to the SPSS manual, descriptive statistics is one procedure that should support pooling. However, I get this error message: At least one of the vectors or matrices in the model being added is different in size from the corresponding vector or matrix in previously added models. This model cannot be added. I have not found anything helpful online (in general or through SPSS's site). Does anyone here have any idea what the problem is? Thank you!

Can you pool this way?...

For example, you have five imputations. So you could use the aggregate function to create a new dataset in which the five imputations for each case's variable are averaged and aggregated into a single cell. Then, for each missing value, the value that replaces it would be the average value of all the five imputations. From there, you could run your analyses as all the data are already pooled ... wouldn't this equate to the same output response? Or am I missing something?

So the bottom line, but correct me if I'm wrong, when it comes to SPSS and imputed data analysis is :

1. Pooled statistics are available (and automatically displayed) when asking SPSS for descriptives

2. SPSS is not able to provide you with a pooled F-value, p-value, etc. when conducting AN(C)OVA and other GLM analyses

3. Use a different program for the latter analyses

The question then remains: are there any calculations you can do to pool the statistics yourself?

Hi Jeremy,

I'm in the middle of completing my dissertaion for my maths degree using SPSS for collected data regarding side effects of radiotherapy. My dissertation involves writing about how reliable multiple imputation is for medical research purposes. SPSS use is a requirement as my dissertation will be used in further research to compare results from SPSS with results from other statistical software programs.

I'm currently in the process of trying to carry out ANOVA with my data however, I am unable to get a pooled result. I read one of your previous posts from 19th June and followed your instructions, however on the last step, it does not give me the option of pooling the results in the drop down menu. Can you recommend anything for me?

Hope to hear back.

Kate

Hi Kate and Yannick

I have just recently submitted my thesis involving spss and multiple imputation. The process was a big learning curve, especially as I used 20 MI and I searched both the literature and the web. I was initially pooling each parameter estimate manually which took quite some time. But I found that the best way to do it is to double click on the output table, click pivot then pivoting trays, and then click and drag most of the items at the bottom of the tray to the top left tray. As you may already know, this allows you to control which items to display. By allowing full control of what is viewed, you can filter the information so that, for instance, the table only displays the p value for all imputations. As they are all together and in a column, if you hold shift and click on the first and last p value displayed for the imputations., you can select them all. Right click then click 'copy'. Now go to any online mean calculator on google, and press control v to paste all the p values. This may seem lengthy, but it is a lot quicker than having to calculate and input each p value manually. And you just do this for the rest of the parameters. I hope that makes sense. I saved much time this way.

Hi Steve,
Thank you so much!!! You have rescued my dissertation!!

Hi, I want to use MI in SPSS for a RM-AVOVA as I have a lot of missing data. I have carried out the MI and now have a dataset that has 1000 subjects (rather than 100, as I did 10 MIs) If i split the file by imputation my analysis is not pooled, as discussed above, but my question is this - can I just use my new MI dataset as a complete dataset and analyse it without the split file command?
#thanks

Hi Sheila,

I had the same problem when I was carrying out repeated measures anova in SPSS. Unfortunately, there is no way of getting a pooled analysis from SPSS when carrying this out with a multiply imputed dataset (I tried so hard to find out that I even called IBM for help and they told me it's not in any of the current models but they are looking to introduce it in future versions of the programme).

Using your full dataset without splitting the file first wont work since it will see it as one massive dataset rather than individual ones. So it will treat it as though you have 1000 subjects.. it's not the same. MI creates all the individual datasets to account for the errors and the uncertainty about creating imputes. So you need to use them individually.

I had to copy and paste the RM ANOVA output analysis into an excel file and find the pooled data manually. I did this by taking the averages of all the outputs from SPSS. Take into account the max and min data so you can see how far your average value is from the actual data. Its a tedious job but not too difficult!

Good luck!

Hi,

i have the same problem. I would like to use the imputed data set for repeated measures anova. i tried out what you suggested above (as far as i understood ;-) ) and wondered if one could do the following:
build the average of every single spss cell above the 5 imputations. this would lead to a new "mean imputed" matrice with new "raw data" that could then be used for repeated measures anova, couldn't it?
Is it allowed to build those means? And if yes, is there any faster way to do it than manually?

Hope to hear back and thanks in advance!

Hi all,

Very helpfull discussion so far. I have a very specific question for which I have not been able to find a good answer.

If you have an imputed dataset and you want to obtain a 95CI for a logistic regression OR or beta using bootrapping, can you pool the upper and lower limits across the MI sets similar to what was described previously?

Hi all,

The answer to all the questions about multiple imputation and repeated-measures ANOVA can be found in:

Van Ginkel, J. R. & Kroonenberg (2014). Analysis of variance of multiply imputed data. Multivariate Behavioral Research, 49, 78-91.

Best regards,

Joost van Ginkel

Hi, I read that it might be ok (or is often done) to just pool a list of pvalues by taking the average of them?

I would need a reference for my thesis for this.. can anyone help me?

I have several imputed sets, ran chisquare tests on them and posthoc test to compare each group with eachother.. I will take the average of those now and present those as my results since I am running out of time.. but it would be great if I could point to someone saying that this is how it can be done..

Unfortunately, there is no reference for this because it is not justified to average p-values. :( Pooling significance tests is not a simple matter, unless it concerns t- or z-tests.

I am new to MI and have been using SPSS MI to replace missing data. I have run 5 datasets and was able to analyze the pooled results. I am looking to create a dataset that would have 1 final value for the missing data based on the 5 imputed datasets. Can I simply aggregate the 5 imputed datasets (average the results) to create the final dataset. If so is this an acceptable solution for replacing missing data or does it not account for the error associated with each imputed dataset.

Thanks,

Ken

Here is how I resolve the situation in SPSS for ANOVA, ANCOVA...
Regression is a special case of ANOVA so you could just run a regression model and SPSS will give you POOLED result for your p value and so on. The t-test squared will equal your F-test in anova.

The computation made is not an actual average of all result, it's almost that. see this reference for a simple explanation of this type of averaging (pooled)
http://www.tandfonline.com/doi/pdf/10.1080/1743727X.2014.979146

In the end, you cant just average results, i dont think it would be accepted in the scientific community.
Use the formula in the reference if you have to do it by hand or use regression to estimate your model as SPSS gives POOLED ouput with regression.

Sorry for my english, I am working on that :)

I have used SSPS MI to replace missing data before carrying out some model testing (CFA) using AMOS 22. However when I use the imputed data set , I am unable to use Modification Indices as the message is telling me that there is missing data. Can I use the 5th imputation only as it is the data-set which corrects for all the missing data?
Many thanks

Eamon

Hi all,

Steve, I have a question regarding what you suggested: taking the mean of all your imputed data to calculate the pooled p-value, F-ratio, etc.
Do you have a reference that supports this?
It would be of great help! Thank you,
Jess.

Hi everyone. I have done the MI with 5 imputations and run regressions using the pooled data - that's all great. However, I'm doing a moderation analysis with Process and that doesn't seem to allow me to split the file and run the analysis. As a result I have df(3, 1892)! Does anyone have any suggestions how to use the pooled data for Process? My data set is a bit complicated as I have 5 variables: 4 of which data were missing completely at random and 1 was MNAR. So I did the imputation for the 4 variables MCAR but not the MNAR one. Does this make sense? I should add that stats is so not my strong suit! Any help would be appreciated. Thanks.

I have run a MANCOVA with a multiply imputed data set in SPSS. How would I go about calculating the pooled partial eta squared and the pooled standard error ? Can I simply calculate the mean of the estimates (partial eta squared, standard error) across all 5 datasets or is there some other step I need to take?