Why am I getting different outputs between Matlab & SPSS?

I am running a 3-way mixed ANOVA (2*6*2).

The between variable is Consciousness with 2 levels (conscious/ subconscious)

The within variables are: Congruency with 6 levels (RH, RF, UH, UF, NH, NF) & Target with 2 levels (BodyPart, Object).

I have run this in Matlab and had assumed I had generated the correct output, Code:

%Within Design Table
WithinDesign=table(categorical({'Related Hand','Related Foot','Unrelated Hand','Unrelated Foot','Neutral Hand','Neutral Foot','Related Hand','Related Foot','Unrelated Hand','Unrelated Foot','Neutral Hand','Neutral Foot'}'),categorical({'BP','BP','BP','BP','BP','BP','O','O','O','O','O','O'}'),'VariableNames',{'Congruency','Target'});
%%
%3-Way Mixed Anova
OverallMixedANOVA=fitrm(OverallDataTable,'BP_RH_Con-O_NF_Sub~Consciousness',WithinDesign=WithinDesign,WithinModel='separatemeans');
%Sphericity Calculation & Error Correction
OverallSphericity=mauchly(OverallMixedANOVA)
OverallErrorCorrect=epsilon(OverallMixedANOVA)
%Anova Table output
[OverallMixedANOVATable,A,C,D]=ranova(OverallMixedANOVA,"WithinModel",'Congruency*Target')
%Corrections for significant sphericity
%CorrectedOverallMixedANOVATable=OverallMixedANOVATable;
%CorrectedOverallMixedANOVATable.DF=OverallMixedANOVATable.DF*OverallErrorCorrect.GreenhouseGeisser

Below is the Matlab Output:

The same data was used in SPSS to check my code. For all previous (2-way RM-) ANOVAs I have performed on Matlab the output has been the same, but in this case the output was different.

Here is the SPSS code pasted to the log:

GLM BP_RH O_RH BP_RF O_RF BP_UH O_UH BP_UF O_UF BP_NH O_NH BP_NF O_NF BY Consciousness

/WSFACTOR=Congruency 6 Polynomial Target 2 Polynomial

/MEASURE=Tradeoff

/METHOD=SSTYPE(3)

/PRINT=DESCRIPTIVE ETASQ OPOWER HOMOGENEITY

/CRITERIA=ALPHA(.05)

/WSDESIGN=Congruency Target Congruency*Target

/DESIGN=Consciousness.

Output from SPSS pasted below:

Different values are highlighted in blue.

I'd like to know why I am getting different values. I know that the p values are still signficant for both methods of analysis, but when looking for replicability this doesnt seem to be satisfactory.

My Questions are:

Have I input the factors correctly in the Matlab code?

Why are the values different between the two softwares?

Does it matter that Matlab and SPSS generate different statistics?

Do you have any reccomendations for what I can do to improve my code?

Thanks in advance

*Edit 1, Ive uploaded the data if anyone wants to have a look.

** PS, I ran the same analyiss using JASP and generated the same output as SPSS.

[dpb]

I am not familiar with the newer MATLAB statistics toolset, but that the p values match (I would put no significance on SPSS reporting <0.001 and MATLAB outputting the actual computed values) implies the same basic test is being done excepting the two have normalized the models internally somewhat differently.

That the DF differences are almost a factor of 2 different is most likely a clue as to what is being done...I would guess one will have to delve into the references that MATLAB typcally provides if there are not 'Algorithm' notes that provide more details.

I think for anybody here to actually do anything specifically they would need the data itself and probably also access to SPSS (which I don't have).

[Dan]

Thanks for your knowledge.

It was something like that I was worried about, I'd have to look deeper to understand what the actual formula Matlab uses in its calculation.

Does the difference in F statistic reflect anything of interest?

I can upload the dataset if thats of use, i'll try to add it as an excel to the above.

PS. I reran the analysis in JASP as well, (which I believe is free) and generated the same output as SPSS.

[dpb]

WithinDesign=table(categorical({'Related Hand','Related Foot','Unrelated Hand','Unrelated Foot','Neutral Hand','Neutral Foot','Related Hand','Related Foot','Unrelated Hand','Unrelated Foot','Neutral Hand','Neutral Foot'}'),categorical({'BP','BP','BP','BP','BP','BP','O','O','O','O','O','O'}'),'VariableNames',{'Congruency','Target'});
OverallMixedANOVA=fitrm(OverallDataTable,'BP_RH_Con-O_NF_Sub~Consciousness',WithinDesign=WithinDesign,WithinModel='separatemeans');

Unrecognized function or variable 'OverallDataTable'.

[Dan]

Sorry for the late reply, I didn't understand what you ment by this to begin with.

OverallDataTable=readtable('Data.xlsx')
WithinDesign=table(categorical({'Related Hand','Related Foot','Unrelated Hand','Unrelated Foot','Neutral Hand','Neutral Foot','Related Hand','Related Foot','Unrelated Hand','Unrelated Foot','Neutral Hand','Neutral Foot'}'),categorical({'BP','BP','BP','BP','BP','BP','O','O','O','O','O','O'}'),'VariableNames',{'Congruency','Target'})
OverallMixedANOVA=fitrm(OverallDataTable,'BP_RH_Con-O_NF_Sub~Consciousness',WithinDesign=WithinDesign)
%Here is the actual ANOVA command
[OverallMixedANOVATable,A,C,D]=ranova(OverallMixedANOVA,"WithinModel",'Congruency*Target')

This should fit the model, but it is still generating results different to those generated by JASP & SPSS. Any ideas?

Answer 1

[Dan]

Just marking this as solved

Conscious needed to be a categorical array

OverallDataTable.Consciousness=categorical(OverallDataTable.Consciousness)