Iteration limit reached - too few observations in a category
1 view (last 30 days)
Show older comments
Hey!
In order to identify important variables in a dataset, I am starting out with making a univariate logistic regression model for each of the variables. However, for some categorical variables, there are only a few observations in some categories (the total number of observations is 1155; there are no missing values). Running the code
modelspec1='Y ~ X';
B1 = fitglm(table,modelspec1,'Distribution','binomial');
returns the warning
Warning: Iteration limit reached.
In glmfit (line 324)
In GeneralizedLinearModel/fitter (line 575)
In classreg.regr.FitObject/doFit (line 94)
In GeneralizedLinearModel.fit (line 882)
In fitglm (line 142)
and the estimate of beta becomes around 100, with a huge standard error and a p-value close to 1. I have tried to increase the number of iterations using
opts = statset('glmfit');
opts.MaxIter = 10000; % default value for glmfit is 100.
but it did not help.
I cannot share my data, but I have 1155 observations, where for example I have a dummy variable where 0 occurs 1132 times, and 1 occurs only 23 times. Where the dummy is 1, the response variable is always 0, while for dummy 0, the responses are both 0 and 1.
I really need to get some kind of an estimate (just dropping a category is sadly not an option). How can I fix the problem? I read ( here ) that I could only use a part of the data where the dummy is zero, but I would like to use it all (as only using a part of it becomes a problem when I include more variables in the model), is it possible?
Thank you!
0 Comments
Answers (0)
See Also
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!