Learning functional relationship through NN where only postprocessed target values are known

My Problem: I have a functional relationship between an Input (x_i) and an Output (f_n) which I want my neural network to learn. However, I can only train my neural network with O_m, which is a postprocessed f_n. The relationship between f_n and O_m is non-injective but known, i.e. from O_m I cannot get back to f_n but given f_n I can calculate O_m.

My question is how can I train my neural network using O_m while defining the (known) functional relationship between f_n and O_m. In the end what I am interested in is the neural network giving me the relationship between x_i and f_n.

The following is just a simple minimum working example showing my Problem:

• X_i is my Input parameter
• O_m are the known values I can train
• the relationship between f_n and O_m is known only in the sense f_n->O_m

The code works but the resulting NN correlates X_i with O_m but I would like to have a NN that predicts f_n based on x_i

%Input/ functional relation 
x_i=[0:1000]';
f_n=x_i.^2;

% Classification vector O_m as the Model  
O_m = zeros(size(x_i));
O_m(log(f_n)>12.8)=1;

x = x_i';
t = O_m';

trainFcn = 'trainscg';
hiddenLayerSize = 10;
net = patternnet(hiddenLayerSize, trainFcn);
net.input.processFcns = {'removeconstantrows','mapminmax'};
net.output.processFcns = {'removeconstantrows','mapminmax'};
net.divideFcn = 'dividerand';  
net.divideMode = 'sample';  
net.divideParam.trainRatio = 80/100;
net.divideParam.valRatio = 10/100;
net.divideParam.testRatio = 10/100;

net.performFcn = 'crossentropy';  

[net,tr] = train(net,x,t);

% Test the Network
y = net(x);
e = gsubtract(t,y);
performance = perform(net,t,y)
tind = vec2ind(t);
yind = vec2ind(y);
percentErrors = sum(tind ~= yind)/numel(tind);

Thanks in advance. David

[Maria Duarte Rosa]

Hi David,

One suggestion that might help is to create a custom performance function that takes into account this relationship between f_n and O_m. You can use x_n as your input variable and O_m as your target variable to the network but your performance function could try to update the network as to minimize the difference between your target variable (O_m) and a transformed version of the predictions, F(y), where F is this relationship between f_n and O_m. This way the network is encouraged to find y values close to f_n. Please see here for how to create custom functions.

I hope this helps.

[David Rieger]

Thanks for your answer. I managed to have 'mymse' as a template for my own performance function, but where exactly is it possible to define the difference between target variable and a transformed version of the predictions in the +mse folder?

Answer 1

[Greg Heath]

There is no solution to your problem.

The functional relationship between input and target is a step function.

Obviously, a step function has no unique inverse.

Hope this helps.

Thank you for formally accepting my answer

Greg