Plot Cook’s distance for mixed effects model

Hello, I have a mixed effects regression model ‘lme’ and I want to plot the Cook’s distance. Apparently the code for this is ‘plotDiagnostics(lme, ‘cookd’)’ but when I enter that I get the error ‘Undefined function ‘plotDiagnostics’. Does anyone know what I am doing wrong here?

[Adam Danz]

That command goes all the way back to r2012a and requires the Statistics and Machine Learning toolbox.

To confirm that you do not have the function/method,

which plotDiagnostics -all

It looks like you do have the toolbox since you seemed to have generated the model (lme).

[Adam Fitchett]

Thank you Adam

I thought I did have the toolbox installed, but when I entered ‘which plotDiagnostics -all’ it said it wasn’t found

[Adam Fitchett]

Indeed, I do have the toolbox as it is listed when I enter “ver” But plotDiagnostics is still not found And I have version 2019a, so should be there

[Adam Danz]

Physically check that you have the file that contains this method. plotDiagnostics is a function stored in the classdef LinearModel.m.

In windows, on my system, that file can be found in

'C:\Program Files\MATLAB\R2019a\toolbox\stats\classreg\@LinearModel\LinearModel.m'

Do you have that file? If you open the file, can you find the plotDiagnostics function?

[Adam Fitchett]

My version is 2019a. I have tried closing and reopening MATLAB, but that didn’t fix it. I tried restoredefaultpath, but that didn’t fix it. I don’t think anything has been deleted or renamed in the toolbox, but maybe it has happened somehow without my knowledge

[Adam Fitchett]

Yes, I have the LinearModel.m file and when I search through it plotDiagnostics does seem to be there

[Adam Fitchett]

Yes it’s in that exact directory. I followed through the directory you described in order to find it, and it was precisely there

[Adam Danz]

Copy the directory that stores LinearModel.m and add that path using addpath('...'). Then run the which plotDiagnostics -all again.

If that fixes it, then somehow the matlab path got messed up.

[Adam Fitchett]

Well that’s curious. I did what you said above, and MATLAB returns “Warning: Name is nonexistent or not a directory”. But I definitely entered the directory of the file.

[Adam Danz]

Oh right, because method directories are not allowed to be added that way. Do this instead (make sure this path is the same on your system); then run the which command

addpath(genpath('C:\Program Files\MATLAB\R2019a\toolbox\stats'))

[Adam Fitchett]

Ah thank you, the which command seems to be finding it now

[Xuelong Fan]

Hi!

I have the same problem. And I tried addpath ... but it still says 'plotDiagnostics' not found.

I installed both 2019b and 2020b. But that shouldn't be a problem, should it?

I wonder if anyone has the same problem.

[Adam Danz]

@Xuelong Fan what is returned by the following commands?

license('test','Statistics_toolbox')
which LinearModel

[Katharina]

I do have the same problem and would be grateful for help!

>> license('test','Statistics_toolbox')

ans =

1

>> which LinearModel

/Applications/MATLAB_R2020b.app/toolbox/stats/classreg/@LinearModel/LinearModel.m % LinearModel constructor

>>

[Adam Danz]

Please share

1. the line of code you're using when calling plotDiagnostics.
2. the full error message you receive after calling the function.

[Xuelong Fan]

0.

I got something similar after running the codes:

ans =

1

C:\Program Files\MATLAB\R2020b\toolbox\stats\classreg\@LinearModel\LinearModel.m % LinearModel constructor

1&2.

And the full error message is:

lme is a linear mixed-effect model.

3.

After running addpath ...,

I got a warning like this:

maybe relevant?

[Adam Danz]

Could you share the line of code you used that led to this error? Ideally, you should share the code and the inputs so we can reproduce the error or trace the stack on our end.

[Katharina]

>> plotDiagnostics(lme,'cookd')

Unrecognized function or variable 'plotDiagnostics'.

[Katharina]

plotDiagnostics(lme,'cookd') using any output of fitlme gives that error

[Katharina]

for same lme, I got this error this time:

Undefined function 'plotDiagnostics' for input arguments of type 'LinearMixedModel'.

[Xuelong Fan]

Similar to Katharina.

It does not work on any lme.

What could be interesting to you is that:

1) when you type in the command line till the point "plotDiagnostics(", the hint of how to use this function can pop up.

But when you run it, it cannot be found:

>> Unrecognized function or variable 'plotDiagnostics'.

2) when you run addpath ... as you mentioned before, other functions such as plotResidual cannot be found neither (it can before).

I think it might have to do with conflict of function names??

[Adam Danz]

@Katharina thanks for the sample of your data. It was helpful to identify the problem.

I'll add an answer to this in a few minutes.

[Adam Danz]

This thread keeps getting attention now and then. I wonder where users get the idea that they can use plotDiagnostics with non-LinearModel objects. If there is a demo or some other souce that is leading people to this problem, please let us know here in the comments section.

Answer 1

[Adam Danz]

plotDiagnostics is a public method of the LinearModel class. The function is defined in LinearModel.m and recieves a LinearModel object as input.

plotDiagnostics is not defined in other model classes such as LinearMixedModel objects. Determine the class of your model using class(mdl) to confirm if it's a LinearModel object. If not, you cannot use plotDiagnostics with the model.

If the LinearModel class has not been constructed, help('plotDiagnostics') will not find the function unless you specify the class name,

clear all 
help plotDiagnostics % r2020b

plotDiagnostics not found.

Use the Help browser search field to search the documentation, or
type "help help" for help command options, such as help for methods.

help LinearModel.plotDiagnostics

 plotDiagnostics Plot diagnostics of fitted model
     plotDiagnostics(LM,PLOTTYPE) plots diagnostics from LinearModel LM in
     a plot of type PLOTTYPE. The default value of PLOTTYPE is 'leverage'.
     Valid values for PLOTTYPE are:
 
        'contour'      residual vs. leverage with overlayed Cook's contours
        'cookd'        Cook's distance
        'covratio'     delete-1 ratio of determinant of covariance
        'dfbetas'      scaled delete-1 coefficient estimates
        'dffits'       scaled delete-1 fitted values
        'leverage'     leverage (diagonal of Hat matrix)
        's2_i'         delete-1 variance estimate
 
     H = plotDiagnostics(...) returns handles to the lines in the plot.
 
     The PLOTTYPE argument can be followed by parameter/value pairs to
     specify additional properties of the primary line in the plot. For
     example, plotDiagnostics(LM,'cookd','Marker','s') uses a square
     marker.
 
     The data cursor tool in the figure window will display the X and Y
     values for any data point, along with the observation name or number.
     It also displays the coefficient name for 'dfbetas'.
 
     Example:
       % Plot the leverage in a fitted regression model
       load carsmall
       d = dataset(MPG,Weight);
       d.Year = ordinal(Model_Year);
       lm = fitlm(d,'MPG ~ Year + Weight + Weight^2')
       plotDiagnostics(lm,'leverage')
 
       % Look at the data for the high-leverage points, and note that
       % their Weight values are near the extremes
       high = find(lm.Diagnostics.Leverage>0.11)
       d(high,:)
 
     See also LinearModel, plotResiduals.

    Documentation for LinearModel/plotDiagnostics
       doc LinearModel.plotDiagnostics

The which function only works when the LinearModel class is constructed.

clear all
which plotDiagnostics

'plotDiagnostics' not found.

fitlm(1:5,1:5);
which plotDiagnostics

/MATLAB/toolbox/stats/classreg/@LinearModel/LinearModel.m  % LinearModel method

[Katharina]

Thank you so much for this clarification, Adam! Are you aware of a way one could calculate Cook's distance for LinearMixedModel in MATLAB?

[Adam Danz]

You'll have to compute it directly. There may be functions on the file exchange that could help out. Matlab's documentation shows how it's computed.

[Katharina]

Thanks, Adam

[Xuelong Fan]

Now I know!

Thank you!