How can I use a normal node and an activation function together in a hidden layer?

How can I use a normal node and an activation function node together in a hidden layer?

In this paper, the author defined the hidden layer as a vector of normal nodes and activation function node.

References are attached with pictures.

now, I'm using this command

hiddenlayer_number=1;

net = feedforwardnet(hiddenlayer_number)

net.layers{1}.transferFcn = 'tansig';

Answer 1

[Shreeya]

Hello,

According to my understanding, you want to create a neural network and define the activation function on the hidden layers.

If my understanding of your problem statement is correct, you can define the activation function on each hidden layer with the below command, as highlighted by you as well:

net.layers{i}.transferFcn =  ‘tansig’;

In case of multiple layers, the activation function can be defined for each layer using the above command.

Hope this helps!

Answer 2

[Udit06]

Hi,

I understand that you want to have more than one type of activation in a given hidden layer. You can customize a deep learning network by defining custom layers for the model. You can refer to the following MathWorks documentation to understand more about the same:

https://www.mathworks.com/help/deeplearning/ug/define-custom-deep-learning-layers.html

In the code template present in the above link, you can modify the "predict" and "forward" function to create a hidden layer with more than one activation function.

I hope this helps.