Problem with initial guess in Newton-Raphson iteration method
1 view (last 30 days)
Show older comments
I'm working on estimating the four parameters of Exponentiated Modified Weibull Extension Distribution introduced by Sarhan and Apaloo (2013) with the Maximum Likelihood Estimation (MLE). Because the first derivative of the log-likelihood function of the four parameters give implicit solutions then I tried to continue with the Newton-Raphson iteration method. In any optimization problem, we need some initial guess depend on how much the parameter of distribution. But this thing gives me a little bit problem. I'm not familiar in determining the good initial guess for this method. I have already tried some but nothing give the best solution. My other problem is that I got to exhibit the bathtub shaped hazard curve for the data set I used because my main purpose is to get the good estimated parameters for modelling the bathtub shaped hazard function of the data set. So actually how to choose the good initial guess for this optimization method? Is there any technique in choosing this initial guess?
0 Comments
Answers (2)
John D'Errico
on 21 Mar 2016
To add to what Alan has said...
I like to introduce the concept of the "basin of attraction". This is a region in your parameter space, that is specific to a given objective function, AND to the optimizer used. Effectively, for a given solution, the basin of attraction for that solution, using your chosen optimizer, is the set of points (X) that converge to the solution (Y). The various solutions (Y) need not be the globally optimal solution. They may even be points of divergence, where no local solution is found, but the optimizer decides to go there anyway.
A basin of attraction is not necessarily a convex set. It may even be dis-contiguous.
As well, the concept of basin of attraction applies to essentially all numerical optimization problems - root finding, min/max problems, etc.
So, the problem is, you need to start your choice of optimizer inside the basin of attraction for the solution you hope to find. Yeah, not always easy to do, since some basins may be terribly small.
Worse, MLE problems are often difficult. They often have domains where you get logs of negative numbers, causing the optimizer to fail rather unpleasantly. Or, you have domains where the objective is just so tiny as to cause underflows (why you want to work on the log-likelihood function).
So choosing a good start point tends to be a nasty thing. Of course, since you provide the data, it makes sense that you have an intelligent start point, based on your knowledge of the process. Hey! It is YOUR problem after all. Nobody knows it better than you.
Some optimizers are less sensitive to a poor choice of starting values. Partitioned Least Squares methods are good for some problems, because they allow you to reduce the dimensionality of the parameter space that must be searched. (They are not applicable to your problem of course.)
A choice of solver that tries to be robust to starting value problems is the family of stochastic solvers. This includes genetic algorithms, bee or ant colony optimizers, simulated annealing. A virtue of these tools is they try not to get stuck in a local minimum.
Another choice is the set of algorithms called global solvers. At the very least, start with a multi-start method. Thus choose a random sample of start points, and see where the solver ends up. This improves the odds that at least ONE of those start points lies inside the basin of attraction of the globally optimal solution.
0 Comments
Alan Weiss
on 21 Mar 2016
I don't know of anything that is generally applicable, and don't understand your case well enough to give you reasonable ideas. The MultiStart solver enables you to start your search from a variety of points automatically at random within bounds, or by any other method you dream up. In general, it is hard to find effective start points.
Good luck,
Alan Weiss
MATLAB mathematical toolbox documentation
0 Comments
See Also
Categories
Find more on Multiobjective Optimization in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!