File Exchange

image thumbnail

MLP Neural Network trained by backpropagation

version 1.0.0.2 (2.15 KB) by Mo Chen
Multilayer Perceptron (MLP) Neural Network (NN) for regression problem trained by backpropagation (backprop)

36 Downloads

Updated 21 Nov 2018

View License

Very compact implementation of backpropagation for MLP regression. Mean to be read and learn.

This package is a part of the PRML toolbox (https://github.com/PRML/PRMLT).

Comments and Ratings (21)

Dan Walker

I am having the hardest time understanding your code, it seams easy enough but on feed forward why do you require Z{T+1} (outside the loop) without the activation function? When you backpropagate you need the derivative of tanh (1/cosh^2x) but I don't see it in your code. is that because this df is specific to your problem? is that also why you are squaring your values there? I think this code could benefit from more comments

% feedforward
for t = 1:T-1
Z{t+1} = tanh(W{t}'*Z{t}+b{t});
end
Z{T+1} = W{T}'*Z{T}+b{T};

% backward
R{T} = E; % delta
for t = T-1:-1:1
df = 1-Z{t+1}.^2; % h'(a)
R{t} = df.*(W{t+1}*R{t+1}); % delta
end

% gradient descent
for t=1:T
dW = Z{t}*R{t}'+lambda*W{t};
db = sum(R{t},2);
W{t} = W{t}-eta*dW;
b{t} = b{t}-eta*db;

yulong zhu

it train very well。but how to predict?

Adolfo

mr morgan

Hi all,
I used this code to train a sample of 8 inputs and one output and it worked. However, I am wondering about how to use it to predict using testing data.
anybody can help, please!

Nice, but not compatible with R2013b, it's not true compatible with any release

I ran the demo, but I got
>> mlp_demo
Error using +
Matrix dimensions must agree.

Error in mlpReg (line 33)
Z{t+1} = tanh(W{t}'*Z{t}+b{t});

Error in mlp_demo (line 8)
[model, L] = mlpReg(x,y,k);

Excellent and simple

Mo Chen

@dsmalenb, [4,5] means, two hidden layer, one with 4 nodes, and one with 5 nodes

dsmalenb

Dumb question - does "h = [4,5]" mean "4 neurons in 5 layers" or "4 layers with 5 neurons". I can't deduce this by reading the code. It is nice and compact but that point is not clear to me.

hi, every body, why not do the following? please explain it, thanks
E = W{l}*dG;

LEE ZISHENG

hana razak

nima safari

Ignas A.

Dan

Amazing - works really well and is super compact in terms of code. Great work!

Rain

Updates

1.0.0.2

change title

1.0.0.1

rewrite for regression

MATLAB Release Compatibility
Created with R2018b
Compatible with any release
Platform Compatibility
Windows macOS Linux
Acknowledgements

Inspired by: Pattern Recognition and Machine Learning Toolbox