Custom use of Softmax activation function in LSTM MAtlab for Solar forecasting

How can i use softmax activation function in the below code?What changes should i make?

With this code i am getting RMSE 8.6.How can i reduce it further ?

Kindly advice.

%Creating LSTM regression network
numFeatures = 1;
numResponses = 1;
numHiddenUnits = 200;

layers = [sequenceInputLayer(numFeatures),lstmLayer(numHiddenUnits),fullyConnectedLayer(numResponses),regressionLayer];

% Specifying the training options
options = trainingOptions('adam','MaxEpochs',250,'GradientThreshold',1,'InitialLearnRate',0.005,'LearnRateSchedule','piecewise','LearnRateDropPeriod',125,'LearnRateDropFactor',0.2,'Verbose',0, 'Plots','training-progress');

%Train LSTM Network
net = trainNetwork(XTrain,YTrain,layers,options);

Answer 1

[Mahesh Taparia]

Hi

Softmax layer bounds the output between [0,1] and usually it is used while training a classification network. In your case, it seems a regression problem. To reduce the RMSE, you can change the network architecture/ increase the network depth by increasing the hidden layers/ follow the existing solution from the literature related to the problem statement. Also try with different learning rate, optimizer etc.

Hope it will help!