Clear Filters
Clear Filters

Error in using trainNetwork with training data and labels

23 views (last 30 days)
I am trying to classify image datasets using deep learning.
after getting feature vector of each single image I ve got a matrix 18000x24000 which indicates to No. of images x features.
I used:
trainNetwork (X, Y, Layers, Options)
Where X is the train data and Y is the Labels which is 18000x1. But there is an error says Invalid training data. X and Y must have the same number of observations.
I think I should change the train matrix to 4-D but I don't know how, and if it is correct?
  3 Comments
Ali Yar Khan
Ali Yar Khan on 6 Dec 2019
Edited: Ali Yar Khan on 6 Dec 2019
i have row of feature in data and one distinct value in label for each row of data ... my network is also giving me the same error ... here is my code.
%loading train and test data
trainingData = load('train.mat');
trainData = trainingData.data;
trainLabels = categorical(trainingData.labels);
testingData = load('test.mat');
testData = testingData.data;
testLabels = testingData.labels;
% Define Network Architecture
% Define the convolutional neural network architecture.
layers = [
imageInputLayer([1 144 1]) % 22X1X1 refers to number of features per sample
convolution2dLayer(3,16,'Padding','same')
reluLayer
fullyConnectedLayer(384) % 384 refers to number of neurons in next FC hidden layer
fullyConnectedLayer(384) % 384 refers to number of neurons in next FC hidden layer
fullyConnectedLayer(2) % 2 refers to number of neurons in next output layer (number of output classes)
softmaxLayer
classificationLayer];
options = trainingOptions('sgdm',...
'MaxEpochs',500, ...
'Verbose',false,...
'Plots','training-progress');
%train the network
net = trainNetwork(trainData,trainLabels,layers,options);
%test the network
predictedLabels = classify(net,trainD)'
accuracy = sum(predictedLabels == testLabels)/numel(testLabels)
It gives me this error
Invalid training data. X and Y must have the same number of observations.
Error in convnet1d (line 27)
net = trainNetwork(trainData,trainLabels.',layers,options);
Caused by:
Error using nnet.internal.cnn.util.NetworkDataValidator/assertXAndYHaveSameNumberOfObservations (line 142)
Invalid training data. X and Y must have the same number of observations.
Any help regarding this . I have this work space ... that may help you about the dimensions of dataScreenshot (13).png

Sign in to comment.

Answers (4)

Bhartendu
Bhartendu on 8 Apr 2018
Try the following:
If your data ( 18000 data points) is in form of images with dimensions say, 120*200 (equals to 24000), then reshape it as follows:
X_train = reshape(X, [120,200,1,size(X,1)]);
This should create 4-D Matrix X_train of size (120,200,1,18000), then train the network using:
net = trainNetwork(X_train,Y,Layers,Options)
  1 Comment
Radians
Radians on 5 Feb 2020
I have done exactly that but to no avail.
I Capture.PNG
where 'X_1' is my X and 'correspondingvalues' is my Y. basically, x is a 16*9500 items collection of 16*16 images. Y is their labels.
Please help.

Sign in to comment.


US
US on 26 Jul 2020
Edited: US on 26 Jul 2020
I am also getting the same error (my layers structure is a prediction CNN):
>> whos X Y
Name Size Bytes Class Attributes
X 64x64x52x1800 3067084800 double
Y 1800x1 14400 double
>> MyPrediction = trainNetwork(X, Y, layers, opts);
Error using trainNetwork (line 165)
Number of observations in X and Y disagree.
It gives me the same error when I transpose Y, or when I reshape X to [64*64,52,1,1800].
And I have:
MATLAB Version: 9.6.0.1072779 (R2019a)
I am starting to think that this MATLAB function trainNetwork or a sub-function that it calls is somehow not behaving properly or I am missing something.
Any help will be appreciated.

michael scheinfeild
michael scheinfeild on 9 Jun 2018
hi i have similar issue and cant solve it i also look at the example Sequence Classification Using Deep Learning
i try to classify between different signals based frequency future
%%generete the signal
fs=500;
f=20;
t=[0:1/fs:5-1/fs];
xsig=10*sin(2*pi*f*t);
sampleLen=length(xsig);
NFFT = 2.^nextpow2(sampleLen);
% signal fft
hwin=hamming(length(xsig))';
Y = fft(xsig.*hwin,NFFT);
magnitudeY = abs(Y);
xsampleFFT=20*log10(magnitudeY(1:NFFT/2));
figure,plot(xsampleFFT);title('signal fft')
% noise fft
xnoise= randn(size(xsig));
Yn = fft(xnoise.*hwin,NFFT);
magnitudeYn = abs(Yn);
xnoiseFFT=20*log10(magnitudeYn(1:NFFT/2));
figure,plot(xnoiseFFT);title('noise')
%%make the data
sampleLen = 100;
NFFT = 128;
hwin=hamming(sampleLen)';
nsamples=length(xsig)/sampleLen;
xsampleFFT={};%zeros(nsamples,NFFT/2);
hwin=hamming(sampleLen)';
kj=1;
for(k=1:sampleLen:length(xsig)-sampleLen+1)
cursig=xsig(k:k+sampleLen-1);
Y = fft(cursig.*hwin,NFFT);
magnitudeY = abs(Y); % Magnitude of the FFT
xTrain{kj}=20*log10(magnitudeY(1:NFFT/2));
%figure,plot(xTrain{kj})
yTrain(kj)=categorical(1);
kj=kj+1;
end
disp('data types')
[size(xTrain) size(yTrain)]
[size(xTrain{1})]
class(xTrain)
class(yTrain)
%append
for(k=1:sampleLen:length(xnoise)-sampleLen+1)
cursig=xnoise(k:k+sampleLen-1);
Y = fft(cursig.*hwin,NFFT);
magnitudeY = abs(Y); % Magnitude of the FFT
xTrain{kj}=20*log10(magnitudeY(1:NFFT/2));
%figure,plot(xTrain{kj})
yTrain(kj)=categorical(0);
kj=kj+1;
end
disp('data types')
[size(xTrain) size(yTrain)]
[size(xTrain{1})]
class(xTrain)
class(yTrain)
figure,plot(yTrain)
figure,subplot(2,1,1),plot(xTrain{10})
subplot(2,1,2);plot(xTrain{30})
function [net] = train_lstm(XTrainLoc,YTrainLoc)
%%lstm
inputSize = 1;
numHiddenUnits = 100;
numClasses = 2;
layers = [ ...
sequenceInputLayer(inputSize)
lstmLayer(numHiddenUnits,'OutputMode','sequence')
fullyConnectedLayer(numClasses)
softmaxLayer
classificationLayer];
maxEpochs = 1;
miniBatchSize = 100;
options = trainingOptions('adam', ...
'ExecutionEnvironment','cpu', ...
'GradientThreshold',1, ...
'MaxEpochs',maxEpochs, ...
'MiniBatchSize',miniBatchSize, ...
'SequenceLength','longest', ...
'Shuffle','never', ...
'Verbose',0, ...
'Plots','training-progress');
% options = trainingOptions('adam', ...
% 'ExecutionEnvironment','cpu', ...
% 'GradientThreshold',1, ...
% 'MaxEpochs',maxEpochs, ...
% 'MiniBatchSize',miniBatchSize, ...
% 'SequenceLength','longest', ...
% 'Shuffle','never', ...
% 'Verbose',0, ...
% 'Plots','training-progress',...
% 'ValidationData',{XValidation,YValidation},...
% 'ValidationPatience',Inf);
%%train
net = trainNetwork(XTrainLoc,YTrainLoc,layers,options);
====== i recive error
[net] = train_lstm(xTrain,yTrain)
*Error using trainNetwork (line 154)
Invalid training data. If all recurrent layers have output
mode 'sequence', then the responses must be a cell array of
categorical sequences, or a categorical sequence.*
_Error in train_lstm (line 42) net = trainNetwork(XTrainLoc,YTrainLoc,layers,options);
Caused by: Error using nnet.internal.cnn.util.NetworkDataValidator/assertOutputModeCorrespondsToDataForClassification (line 380) Invalid training data. If all recurrent layers have output mode 'sequence', then the responses must be a cell array of categorical sequences, or a categorical sequence._
so what is the issue i tried also change y to cell array of category , transpose the internal x, change network in. i think in this fft i have actually one sample each time with nfft feature. this is the same as the Japanese sample but they have 12 features

SARAH LONER
SARAH LONER on 29 Nov 2019
sir i also have the same issue cant able to train network for an image .
my work was image segmentation based on unet segment cant able to clear the code
it showing error at
net = trainNetwork(imds,layers,options);
where in imds i have taken image
kindly help to solve this error
  2 Comments
SARAH LONER
SARAH LONER on 30 Nov 2019
inp=b % my input image is dicom image
DatasetPath=fullfile('C:\Users\Desktop\to');
imds=imageDatastore(DatasetPath, 'IncludeSubfolders', true,...
'LabelSource','foldernames','fileextension',{'.dcm'});
labelDir = fullfile(DatasetPath,'testImages');
I = readimage(imds,1);
I = histeq(I);
imshow(I)
classes = [
"MALIGNANT","BENIGN"
];
labelIDs=[255 0]
inputlayer = imageInputLayer([512 512 1],'Name','inp')
numFilters = 64;
numLayers = 16;
layers = [ ...
imageInputLayer([512 512 1])
convolution2dLayer(5,20)
batchNormalizationLayer
reluLayer
maxPooling2dLayer(2,'Stride',2)
convolution2dLayer(5,20)
batchNormalizationLayer
reluLayer
maxPooling2dLayer(2,'Stride',2)
transposedConv2dLayer(4,numFilters,'Stride',2,'Cropping',1);
convolution2dLayer(5,20)
batchNormalizationLayer
reluLayer
transposedConv2dLayer(4,numFilters,'Stride',2,'Cropping',1);
convolution2dLayer(5,20)
batchNormalizationLayer
reluLayer
convolution2dLayer(5,20)
fullyConnectedLayer(4)
softmaxLayer
pixelClassificationLayer
]
pxds = pixelLabelDatastore(labelDir,classes,labelIDs);
options = trainingOptions('sgdm', ...
'InitialLearnRate',0.01, ...
'MaxEpochs',1, ...
'Shuffle','every-epoch', ...
'ValidationFrequency',30, ...
'Verbose',false);
ds = pixelImageDatastore(imds,pxds);
net=trainNetwork(ds,layers,options);
I = read(imds);
C = read(pxds)
C = semanticseg(I, net);
% Overlay pixel label data on the image and display.
B = labeloverlay(I, C);
figure(12)
imshow(B)
i got error at
ds = pixelImageDatastore(imds,pxds);
once i cleared that error means again i got error at
net=trainNetwork(ds,layers,options);

Sign in to comment.

Categories

Find more on Image Data Workflows in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!