Improve fit quality for a custom function with near-perfect starting values

3 views (last 30 days)
I'm trying to fit data with a relatively complicated custom function (combined logistic+linear function; combined effect of my data+noise). By trail-and-error I can manually find a good fit (indicated below), but I want MATLAB to do the final fine-tuning. Instead, the MATLAB fit is way worse than what I can do manually. As this takes minutes to do by hand and I will need to process hundreds of images, I really want to automate this.
This is the starting fit I provide manually:
This is the fit after MATLAB is done (using the manual fit values as startpoint):
I tried:
  • Setting lower and upper bounds
  • Parameter scaling (all coefficients between 1E-2 and 1E2)
  • Changing algorithm
  • Setting DiffMinChange and DiffMaxChange to small values
  • Changing TolX and TolFun
The main thing I don't understand is why MATLAB worsens the fit. Especially with low DiffMinChange and DiffMaxChange, I would expect MATLAB to give the starting fit or something better.
In case people want to try for themselves, below I provide the data and custom function. The data-to-fit is in 'data.txt', while the function I use to fit it is:
I=@(Ac,b,AE,bg,bgx,x) AE.*(1./(1+1E15./Ac.*exp(-b.*x)))+bg+bgx.*x;.
The manual startpoint is
start=[1.75 0.065 14 16 0.015];
To help with interpretation of the function:
  • It is a logistic function '1/(1+1E15./Ac*exp(-b*x))',
  • a scale-factor for the logistic function 'AE', (the 1E15 is to scale the parameter)
  • a background offset 'bg', and
  • a background slope 'bgx'.
  2 Comments
Star Strider
Star Strider on 5 Jun 2018
Edited: Star Strider on 5 Jun 2018
Which of: (Ac,b,AE,bg,bgx,x) are your independent variable, and which are the parameters?
Are there any constraints on the parameters?
Pieter Hamming
Pieter Hamming on 6 Jun 2018
Edited: Pieter Hamming on 6 Jun 2018
Ac,b,AE,bg,bgx are parameters, and only x is my independent.
Bounds:
  • 0 < Ac < 10
  • 0 < b < 0.1
  • 0 < AE < 40
  • 0 < bg < 30
  • 0 < bgx < 0.1

Sign in to comment.

Accepted Answer

Star Strider
Star Strider on 6 Jun 2018
I like genetic algorithms, because it is possible to set them to search the entire parameter space. They tend to ignore local minima, although with challenging problems, it is usually best to run them several times. The best fit I got using this code:
D = load('data.txt');
[~,idx] = max(D(:,2));
x = D(1:idx,1);
y = D(1:idx,2);
I = @(Ac,b,AE,bg,bgx,x) AE.*(1./(1+1E15./Ac.*exp(-b.*x)))+bg+bgx.*x;
Ifcn = @(p,x) I(p(1),p(2),p(3),p(4),p(5),x);
ftns = @(p) norm(y - Ifcn(p,x));
opts = optimoptions('ga', 'PopulationSize',5000, 'InitialPopulationMatrix', randi(1000, 5000, 5)*1E-2, 'PlotFcn','gaplotbestf', 'MaxGenerations',2E+3);
[B,fval] = ga(ftns, 5, [], [], [], [], [0 0 0 0 0], [10 0.1 40 30 0.1], [], [], opts)
figure(2)
plot(x, y, '.b')
hold on
plot(x, Ifcn(B,x), '-r')
grid
were these parameters and norm of the residuals:
B =
7.518752366084703 0.062124979009373 13.986849416774319 16.989374475316001 0.013494379513801
fval =
10.904527751274076
The genetic algorithm converged on these values after about 280 generations. (I am using R2018a.)
  2 Comments
Pieter Hamming
Pieter Hamming on 7 Jun 2018
Great answer, thank you! I had not dabbled in genetic algorithms thus far, but from now on I sure will.
Star Strider
Star Strider on 7 Jun 2018
As always, my pleasure! Thank you!
Genetic algorithms are straightforward to understand, and the MATLAB implementation is the most robust I have seen.

Sign in to comment.

More Answers (0)

Products


Release

R2017b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!