Why does my CNN model's validation accuracy vary so much everytime I run the model with the same data?
2 views (last 30 days)
Show older comments
Hello,
I am training a CNN model with a total of 80 images and 2 categories of classification. Every time, I run the model the validation accuracy significantly varies e.g. the first time it was 95.8% and the second time it was 61.5%. Can someone please help me in figuring out why there is such a huge difference in that. I would really appreciate any assistance. Thank you,
I am using the following code:
clear all
clc
rootfolder= fullfile ('CNN Data');
imds = imageDatastore(rootfolder,'IncludeSubfolders',true,'LabelSource','foldernames');
[imdsTrain, imdsValidation] = splitEachLabel(imds,0.70,'randomize');
numTrainImages = numel(imdsTrain.Labels);
idx = randperm(numTrainImages,16);
figure
for i = 1:16
subplot(4,4,i)
I = readimage(imdsTrain,idx(i));
imshow(I)
end
net = alexnet ();
rng ('default');
net.Layers(1);
net.Layers(end);
numel(net.Layers(end).ClassNames);
inputSize = net.Layers(1).InputSize;
augimdsTrain = augmentedImageDatastore(inputSize,imdsTrain, 'ColorPreprocessing','gray2rgb');
augimdsValidation = augmentedImageDatastore(inputSize,imdsValidation, 'ColorPreprocessing','gray2rgb');
w1 = net.Layers(2).Weights;
w1 = mat2gray(w1);
featurelayer = 'drop7';
trainingFeatures = activations(net,augimdsTrain,featurelayer,'MiniBatchSize', 32,'OutputAs', 'columns');
miniBatchSize = 32;
options = trainingOptions('sgdm', ...
'InitialLearnRate',0.001,...
'ValidationData',augimdsValidation, ...
'ValidationFrequency',3, ...
'MaxEpochs',5,...
'MiniBatchSize',32,...
'Plots','training-progress',...
'Shuffle','once');
numClasses = 2;
layersTransfer = net.Layers(1:end-3);
layers = [
layersTransfer
fullyConnectedLayer(numClasses,'WeightLearnRateFactor',20,'BiasLearnRateFactor',20)
softmaxLayer
classificationLayer];
trainedNet = trainNetwork(augimdsTrain,layers,options);
trainingLabels = imdsTrain.Labels;
classifier = fitcecoc(trainingFeatures, trainingLabels, 'Learner', 'Linear', 'Coding', 'onevsall','observationsIn','columns');
validationFeatures = activations(trainedNet, augimdsValidation, featurelayer,'MiniBatchSize', 32,'OutputAs', 'columns');
predictLabels = predict(classifier,validationFeatures,'ObservationsIn','columns');
validationLabels = imdsValidation.Labels;
confMatt = confusionmat(validationLabels, predictLabels);
confusionmean= mean(diag(confMatt));
plotconfusion(validationLabels, predictLabels)
figure, cm= confusionchart(validationLabels, predictLabels)
title ('Confusion Chart')
2 Comments
Mahesh Taparia
on 15 Dec 2020
Hi
Can you try with more epochs, as it might be the case that loss function is not getting saturated in your 2nd case due to less number of epoch. Every time weights are initialized, the accuracy will be different for each initialization but there will not be much difference.
Answers (0)
See Also
Categories
Find more on Sequence and Numeric Feature Data Workflows in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!