349 views (last 30 days)

I am trying to classify image datasets using deep learning.

after getting feature vector of each single image I ve got a matrix 18000x24000 which indicates to No. of images x features.

I used:

trainNetwork (X, Y, Layers, Options)

Where X is the train data and Y is the Labels which is 18000x1. But there is an error says Invalid training data. X and Y must have the same number of observations.

I think I should change the train matrix to 4-D but I don't know how, and if it is correct?

Bhartendu
on 8 Apr 2018

Try the following:

If your data ( 18000 data points) is in form of images with dimensions say, 120*200 (equals to 24000), then reshape it as follows:

X_train = reshape(X, [120,200,1,size(X,1)]);

This should create 4-D Matrix X_train of size (120,200,1,18000), then train the network using:

net = trainNetwork(X_train,Y,Layers,Options)

Radians
on 5 Feb 2020

I have done exactly that but to no avail.

I

where 'X_1' is my X and 'correspondingvalues' is my Y. basically, x is a 16*9500 items collection of 16*16 images. Y is their labels.

Please help.

michael scheinfeild
on 9 Jun 2018

hi i have similar issue and cant solve it i also look at the example Sequence Classification Using Deep Learning

i try to classify between different signals based frequency future

%%generete the signal

fs=500;

f=20;

t=[0:1/fs:5-1/fs];

xsig=10*sin(2*pi*f*t);

sampleLen=length(xsig);

NFFT = 2.^nextpow2(sampleLen);

% signal fft

hwin=hamming(length(xsig))';

Y = fft(xsig.*hwin,NFFT);

magnitudeY = abs(Y);

xsampleFFT=20*log10(magnitudeY(1:NFFT/2));

figure,plot(xsampleFFT);title('signal fft')

% noise fft

xnoise= randn(size(xsig));

Yn = fft(xnoise.*hwin,NFFT);

magnitudeYn = abs(Yn);

xnoiseFFT=20*log10(magnitudeYn(1:NFFT/2));

figure,plot(xnoiseFFT);title('noise')

%%make the data

sampleLen = 100;

NFFT = 128;

hwin=hamming(sampleLen)';

nsamples=length(xsig)/sampleLen;

xsampleFFT={};%zeros(nsamples,NFFT/2);

hwin=hamming(sampleLen)';

kj=1;

for(k=1:sampleLen:length(xsig)-sampleLen+1)

cursig=xsig(k:k+sampleLen-1);

Y = fft(cursig.*hwin,NFFT);

magnitudeY = abs(Y); % Magnitude of the FFT

xTrain{kj}=20*log10(magnitudeY(1:NFFT/2));

%figure,plot(xTrain{kj})

yTrain(kj)=categorical(1);

kj=kj+1;

end

disp('data types')

[size(xTrain) size(yTrain)]

[size(xTrain{1})]

class(xTrain)

class(yTrain)

%append

for(k=1:sampleLen:length(xnoise)-sampleLen+1)

cursig=xnoise(k:k+sampleLen-1);

Y = fft(cursig.*hwin,NFFT);

magnitudeY = abs(Y); % Magnitude of the FFT

xTrain{kj}=20*log10(magnitudeY(1:NFFT/2));

%figure,plot(xTrain{kj})

yTrain(kj)=categorical(0);

kj=kj+1;

end

disp('data types')

[size(xTrain) size(yTrain)]

[size(xTrain{1})]

class(xTrain)

class(yTrain)

figure,plot(yTrain)

figure,subplot(2,1,1),plot(xTrain{10})

subplot(2,1,2);plot(xTrain{30})

function [net] = train_lstm(XTrainLoc,YTrainLoc)

%%lstm

inputSize = 1;

numHiddenUnits = 100;

numClasses = 2;

layers = [ ...

sequenceInputLayer(inputSize)

lstmLayer(numHiddenUnits,'OutputMode','sequence')

fullyConnectedLayer(numClasses)

softmaxLayer

classificationLayer];

maxEpochs = 1;

miniBatchSize = 100;

options = trainingOptions('adam', ...

'ExecutionEnvironment','cpu', ...

'GradientThreshold',1, ...

'MaxEpochs',maxEpochs, ...

'MiniBatchSize',miniBatchSize, ...

'SequenceLength','longest', ...

'Shuffle','never', ...

'Verbose',0, ...

'Plots','training-progress');

% options = trainingOptions('adam', ...

% 'ExecutionEnvironment','cpu', ...

% 'GradientThreshold',1, ...

% 'MaxEpochs',maxEpochs, ...

% 'MiniBatchSize',miniBatchSize, ...

% 'SequenceLength','longest', ...

% 'Shuffle','never', ...

% 'Verbose',0, ...

% 'Plots','training-progress',...

% 'ValidationData',{XValidation,YValidation},...

% 'ValidationPatience',Inf);

%%train

net = trainNetwork(XTrainLoc,YTrainLoc,layers,options);

====== i recive error

[net] = train_lstm(xTrain,yTrain)

*Error using trainNetwork (line 154)

Invalid training data. If all recurrent layers have output

mode 'sequence', then the responses must be a cell array of

categorical sequences, or a categorical sequence.*

_Error in train_lstm (line 42) net = trainNetwork(XTrainLoc,YTrainLoc,layers,options);

Caused by: Error using nnet.internal.cnn.util.NetworkDataValidator/assertOutputModeCorrespondsToDataForClassification (line 380) Invalid training data. If all recurrent layers have output mode 'sequence', then the responses must be a cell array of categorical sequences, or a categorical sequence._

so what is the issue i tried also change y to cell array of category , transpose the internal x, change network in. i think in this fft i have actually one sample each time with nfft feature. this is the same as the Japanese sample but they have 12 features

SARAH LONER
on 29 Nov 2019

sir i also have the same issue cant able to train network for an image .

my work was image segmentation based on unet segment cant able to clear the code

it showing error at

net = trainNetwork(imds,layers,options);

where in imds i have taken image

kindly help to solve this error

SARAH LONER
on 30 Nov 2019

inp=b % my input image is dicom image

DatasetPath=fullfile('C:\Users\Desktop\to');

imds=imageDatastore(DatasetPath, 'IncludeSubfolders', true,...

'LabelSource','foldernames','fileextension',{'.dcm'});

labelDir = fullfile(DatasetPath,'testImages');

I = readimage(imds,1);

I = histeq(I);

imshow(I)

classes = [

"MALIGNANT","BENIGN"

];

labelIDs=[255 0]

inputlayer = imageInputLayer([512 512 1],'Name','inp')

numFilters = 64;

numLayers = 16;

layers = [ ...

imageInputLayer([512 512 1])

convolution2dLayer(5,20)

batchNormalizationLayer

reluLayer

maxPooling2dLayer(2,'Stride',2)

convolution2dLayer(5,20)

batchNormalizationLayer

reluLayer

maxPooling2dLayer(2,'Stride',2)

transposedConv2dLayer(4,numFilters,'Stride',2,'Cropping',1);

convolution2dLayer(5,20)

batchNormalizationLayer

reluLayer

transposedConv2dLayer(4,numFilters,'Stride',2,'Cropping',1);

convolution2dLayer(5,20)

batchNormalizationLayer

reluLayer

convolution2dLayer(5,20)

fullyConnectedLayer(4)

softmaxLayer

pixelClassificationLayer

]

pxds = pixelLabelDatastore(labelDir,classes,labelIDs);

options = trainingOptions('sgdm', ...

'InitialLearnRate',0.01, ...

'MaxEpochs',1, ...

'Shuffle','every-epoch', ...

'ValidationFrequency',30, ...

'Verbose',false);

ds = pixelImageDatastore(imds,pxds);

net=trainNetwork(ds,layers,options);

I = read(imds);

C = read(pxds)

C = semanticseg(I, net);

% Overlay pixel label data on the image and display.

B = labeloverlay(I, C);

figure(12)

imshow(B)

i got error at

ds = pixelImageDatastore(imds,pxds);

once i cleared that error means again i got error at

net=trainNetwork(ds,layers,options);

US
on 26 Jul 2020 at 6:52

Edited: US
on 26 Jul 2020 at 6:54

I am also getting the same error (my layers structure is a prediction CNN):

>> whos X Y

Name Size Bytes Class Attributes

X 64x64x52x1800 3067084800 double

Y 1800x1 14400 double

>> MyPrediction = trainNetwork(X, Y, layers, opts);

Error using trainNetwork (line 165)

Number of observations in X and Y disagree.

It gives me the same error when I transpose Y, or when I reshape X to [64*64,52,1,1800].

And I have:

MATLAB Version: 9.6.0.1072779 (R2019a)

I am starting to think that this MATLAB function trainNetwork or a sub-function that it calls is somehow not behaving properly or I am missing something.

Any help will be appreciated.

Opportunities for recent engineering grads.

Apply Today
## 2 Comments

## Direct link to this comment

https://in.mathworks.com/matlabcentral/answers/368182-error-in-using-trainnetwork-with-training-data-and-labels#comment_507064

⋮## Direct link to this comment

https://in.mathworks.com/matlabcentral/answers/368182-error-in-using-trainnetwork-with-training-data-and-labels#comment_507064

## Direct link to this comment

https://in.mathworks.com/matlabcentral/answers/368182-error-in-using-trainnetwork-with-training-data-and-labels#comment_775125

⋮## Direct link to this comment

https://in.mathworks.com/matlabcentral/answers/368182-error-in-using-trainnetwork-with-training-data-and-labels#comment_775125

Sign in to comment.