How to calculate the MSE for multi-output neural network?

I want to compare different neural network architectures using MSE. I know that for a single output network, it is straightforward. But I am not sure how to calculate MSE for multi-output network especially outputs have different units? Is it okay to calculate the MSE for each output set separately and get average value? Can someone help me with creating a code to calculate the MSE of the following network?

Thank you

x = Input;
t = Output;
trainFcn = 'trainlm'; 
net = fitnet([25,25],trainFcn);
net.trainParam.max_fail=10;

net.layers{1}.transferFcn = 'tansig';
net.layers{2}.transferFcn = 'logsig';

net.input.processFcns = {'removeconstantrows','mapminmax'};
net.output.processFcns = {'removeconstantrows','mapminmax'};

net.divideFcn = 'dividerand';  % Divide data randomly
net.divideMode = 'sample';  % Divide up every sample
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;

net.performFcn = 'mse';  % Mean Squared Error

   net.plotFcns = {'plotperform','plottrainstate','ploterrhist', ...
      'plotregression', 'plotfit'};

[net,tr] = train(net,x,t);

y = net(x);
e = gsubtract(t,y);
performance = perform(net,t,y);

trainTargets = t .* tr.trainMask{1};
valTargets = t .* tr.valMask{1};
testTargets = t .* tr.testMask{1};
trainPerformance = perform(net,trainTargets,y);
valPerformance = perform(net,valTargets,y);
testPerformance = perform(net,testTargets,y);

%siimulating the outputs 
alloutputs = sim(net,x);

% Getting the training targets
trainTargets = gmultiply(t,tr.trainMask);
SSE = sse(net,trainTargets,alloutputs); % Sum of Squared Errors for the training set
n = length(tr.trainInd); % Number of training cases
p = length(getwb(net)); % Number of parameters (weights and biases)
% Schwarz's Bayesian criterion (or BIC) (Schwarz, 1978)
    SBC = n * log(SSE/n) + p * log(n);
% Akaike's information criterion (Akaike, 1969)
    AIC = n * log(SSE/n) + 2 * p;
% Corrected AIC (Hurvich and Tsai, 1989)
    AICc = n * log(SSE/n) + (n + p) / (1 - (p + 2) / n);

Answer 1

[Greg Heath]

It is better to combine scalars into vectors and matrices when they have the same numerical scale. My ideal is to STANDARDIZE inputs and targets to zero mean and unit variance. As shown below, this leads to the dimensionless normalized mean-square-error figure of merit related to the regression R-squared (see Wikipedia) via

           NMSE = 1 - Rsq
with
        0 <= NMSE, Rsq <= 1

Calculate the mean and standard deviations of all input and target rows. Remove or modify values which are more than a specified number of standard deviations from the mean (scatter plots are especially useful here). Then recalculate means and standard deviations and standardize.

I always use NMSE (in [0,1]), the normalized mean-square error obtained using the constant reference output

outref = mean(target')'
MSEref = mse( target- outref )
       = mean( var(target',1) )

Then

NMSE = mse(target-output)/ MSEref
    = 1 - Rsquare      % See Wikipedia

Hope this helps.

Thank you for formally acceoting my answer

Greg