how can i evaluate my artificial neural network when the targets' range is very short and mse doesnt work?

I'm working with a.n.n in matlab and I want to fit a network to my experimental datas. Totally, most of MLPs I train can work easily, and no matter how many neurons I have used, most of networks have a mse less than 0.001 which is acceptable.

but the problem is regression, in the regression plots for test data, result is not accurate enough. and when I check the network manually with my datas, i dont get what I need.

what shoud I do? remembering that my targets have a rang of : 0.1000 to 0.1800 .

is there any other parameter for evaluating the network accuracy?

Answer 1

[Greg Heath]

Yes, the normalized MSE

 NMSE = mse(t-y)/mean(var(t',1))

and corresponding coefficient of determination or Rsquared

 R2 = 1 - NMSE

============================================================

NEWSGROUP and ANSWERS search info:

 greg NMSE
 greg R2

SEARCH ENGINE INFO:

Coefficient of determination - Wikipedia, the free encyclopedia

 en.wikipedia.org/wiki/Coefficient_of_determination

R2 - Wikipedia, the free encyclopedia

 en.wikipedia.org/wiki/R2

Thank you for formally accepting my answer

Greg

[Poooneh mmm]

 Dear prof. Heath thanks for your detailed answer.

after you introduced normalized mse, i searched about it on the intenet. and I found some equations explaning nrmse and nmse (which were totally diffrent. )

any way, I still dont know how to compare results with nmse. which nmse is better? how can I find optimume number of neurons and layers?

I guesse a normalizing between target datas is needed, but i dont know how . and also I think when matlab itself normalizes datas( by 'mapminmax') are there better ways of annalyzing datas?

[Greg Heath]

It sounds like you didn't concentrate on searching my posts in

 1. The NEWSGROUP 
 2. ANSWERS

With targets standardized to zero-mean/unit-variance, NMSE = MSE.

In short I usually recommend

 1. Standardizing inputs and outputs to zero-mean/unit-variance (zscore or mapstd)
2. MSEgoal <= 0.01 (Rsquare >= 0.99)
3. MinGrad <= MSEgoal/100
4. Minimize the number of hidden nodes using a double loop search: 
   a. outer loop over Hmin:dH:Hmax 
   b. inner loop over 1:Ntrials initial random number states (determines random trn/val/tst division and initial weights)

I have posted zillions of examples.

Greg