How can I use a trained neural network as an objective function in GA optimisation

I have trained neural networks from experimental data and I want to use the neural networks simulation as feed to the genetic algorithm, is there a way of doing that in Matlab? I guess I will still need a parametric fitness function for optimization using the GA. The only way I can think of for generating a parametric objective function is by regression, is there a way I can use the trained neural network to generate a parametric equation?

Answer 1

[Greg Heath]

Use the trained network to calculate the MSE on non-training data.

Normalized MSE NMSE = MSE/mean(var(target',1)) and "Fraction of target variance explained (Wikipedia/R-squared) R^2 = 1-NMSE are the most convenient optimization objectives.

[Hossein]

Hello Greg, I have the same question as Givemore's. That is I want to use a trained neural network as an input fitness function for the GA optimization toolbox. If I understand you right the R^2 then will be a good objective for optimizing the network ITSELF (weights , ...) not the output (in my case noise) with three inputs.

Specifically I need to minimize noise (output) then ga gives me my Three corresponding inputs.

Does R^2 minimization still works in this case??

The way I was trying to do it was after running my network code and constructing the "net" structure in workspace define a function file that accepts the input 'x' and runs:

ftn=sim(net,x');

But GA toolbox says "undefined function/variable 'net'"!!!!! I'll appreciate any hints or comments in advance.

[Greg Heath]

I still don't understand what you are trying to do. A trained neural network just represents a nonlinear function of the input. For example,

 y = f(x) = b2+LW*tanh(b1+IW*x);

where b2,LW, b1 and IW are fixed parameters obtained from training on pairs of examples (xi,yi;i=1:N).

Now, what are you trying to minimize?

Greg

[Hossein]

Thanks for your prompt response Greg. The thing that I am looking for is exactly the nonlinear function you mentioned. I know about the simple function you wrote however the relation is not that clear for me I first tried to code all the simulink blocks that the nftool generates to actually have the function code! But that never gave me the results that are even close to my targets. I get good output (close to target) though from :

ftn=sim(net,x');

However as I mentioned in my last comment the "net" seems to be undefined for the GA toolbox.

You asked what really I am trying to minimize? from your example I am trying to minimize y with respect to x (my input) so when y got minimum then GA give me the corresponding x (which has three elements: speed,height and width).

As you know the "nftool" also generates a simple code for the network. I even tried to embed that code into my fitness function file to get around the "undefined variable/function 'net'" error. But the problem with that is the network gets repeatedly trained for each iteration of the ga and that is not what I want since I need to train a network once and use the result trained network as nonlinear input-output function!!!

Hope I was able to clarify my problem a bit.

Thank you,

[Greg Heath]

Sorry for being so dense. The expression for y is the most common form of the network implemented by nftool for regression (and classification). Other forms will also have differentiable analytic representations. You start with multidimensional data examples (inputs,targets) = {xi,ti,i=1:N) which can be searched for minimum values(s) tj such that |tj| or |tj|^2 is a minimum where the notation |tj| indicates vector norm.

However, your premise is that the known multidimensional target data points are probably not those with minimum vector norms. So, you fit a smooth multidimensional function to those points to obtain y=f(x) and search for the minimum of the function characterized by |y| or |y|^2.

If I am correct, why not just use the analytic form instead of calling net(x)?

Greg

[Hossein]

You are fine Greg. What do you mean exactly with "analytic form"? you knew the only reason I am using a net is that I do not have the analytic equation relating inputs to output. right?

With two different files one for nftool to generate the net and save it and the other one the fitness function file load ing that "net" the error is now gone!!! and the result is not bad...

Thanks,

[Greg Heath]

The analytic form of the I-O relationship between the nornalized input and output for the regression neural net FITNET is

 yn = B2 + LW * tanh( B1 + IW * xn );

For details, search on subsets of these variables in the NEWSGROUP and ANSWERS.

[Anurag Trivedi]

Following code uses Neural Network as Objective Function for SA:

 for i=1:5; 
    ObjectiveFunction = @(x) sim(net{22,3}, x);
    p1 = [30.00;20.30;737;1000;0.20;];

    % // (starting point for the algo)
    % ERROR: // is not used in MATLAB for comments. Use %

    lb = [0 16.80 713 884 0.1]; 
    ub = [100 22.90 762 1100 0.7];
    [x,fval,exitFlag,output] = simulannealbnd(ObjectiveFunction,p1,lb,ub); 
    XX(1:5,i)=x; 
    FVALL(1,i)=fval;
end;

[GEORGIOS BEKAS]

what does the '{22,3}' expression mean?