Invalid training data. X and Y must have the same number of observations.

17 views (last 30 days)
I have long ECG signals segmented into 300 points segments/heartbeats. I want to use CNN for feature extraction with a bidirectional LSTM layer for classification. I have the following network:
inputSize=[1 300 1]; %the heartbeat size
Layers=[
sequenceInputLayer(inputSize,'Normalization', 'zscore', 'Name','input');
sequenceFoldingLayer('Name','fold')
convolution2dLayer([1 7], 16,'stride',[1 1], 'padding','same','Name','conv1')
batchNormalizationLayer('Name','bn1')
maxPooling2dLayer([1 2],'stride',[1 2],'Name','mpool1')
convolution2dLayer([1 7], 32,'stride',[1 1], 'padding','same','Name','conv2')
batchNormalizationLayer('Name','bn2')
reluLayer('Name','relu1')
maxPooling2dLayer([1 2],'stride',[1 2],'Name','mpool2')
convolution2dLayer([1 5], 64,'stride',[1 1], 'padding','same','Name','conv3')
batchNormalizationLayer('Name','bn3')
reluLayer('Name','relu2')
convolution2dLayer([1 5], 128,'stride',[1 1], 'padding','same','Name','conv4')
batchNormalizationLayer('Name','bn4')
reluLayer('Name','relu3')
convolution2dLayer([1 3], 256,'stride',[1 1], 'padding','same','Name','conv5')
batchNormalizationLayer('Name','bn5')
reluLayer('Name','relu4')
maxPooling2dLayer([1 2],'stride',[1 2],'Name','mpool3')
convolution2dLayer([1 3], 512,'stride',[1 1], 'padding','same','Name','conv6')
batchNormalizationLayer('Name','bn6')
reluLayer('Name','relu5')
maxPooling2dLayer([1 2],'stride',[1 2],'Name','mpool4')
sequenceUnfoldingLayer('Name','unfold')
flattenLayer('Name','flatten')
bilstmLayer(200,'Name','lstm')
reluLayer('Name','relu6')
fullyConnectedLayer(256,'Name','fc1')
reluLayer('Name','relu7')
fullyConnectedLayer(128,'Name','fc2')
reluLayer('Name','relu8')
fullyConnectedLayer(5,'Name','fc3')
softmaxLayer('Name','softmax')
classificationLayer('Name','classification')
];
I connect the layers using:
lgraph = layerGraph(Layers);
lgraph = connectLayers(lgraph,'fold/miniBatchSize','unfold/miniBatchSize');
When I train the network I have the following error:
Error using trainNetwork (line 170)
Invalid training data. X and Y must have the same number of observations.
Error in CNN_LSTM (line 152)
convnet = trainNetwork(Xtrain,Ytrain,lgraph,options);
Xtrain has the size 1 300 1 91147 (it contains 91147 segments of 300 datapoints each)
Y train has the size 91147 1
Can someone please tell me how could I solve this error? I checked the documentation Sequence Classification Using Deep Learning but it did not help me.. If I introduce the S= sequence length (91147) I have another error

Accepted Answer

Tomaso Cetto
Tomaso Cetto on 26 May 2021
Hi Ioana,
There are a few modifications to your data and network that are required to make your workflow possible.
The difficulty is associated with the fact that the convolutional part of your network needs to operate on data of the form S x S x C x B (in your case, this is equal to 1 x L x 1 x 91147, where L = 300 to start with, as you rightly pointed out), but the recurrent part of your network (that is, the bilstmLayer) needs data in the format C x B x T (in your case, 1 x 91147 x L, where L is the length of the segment once it's gone through all the convolution layers).
The issue here is that your dimension which I've marked as L is at first a spatial dimension, and then a time dimension. To make this switch possible, we are going to need a custom layer. More on that later - for now, let's start by thinking about how you want to specify your training data.
The sequenceInputLayer requires data in cell array format, as Srivardhan pointed out.
For your training data, you're going to need one cell array for each observation (so 91147 cell arrays), and in each of these cell arrays, you'll need a S x S x C x T array (see this part of the trainNetwork documentation for more details). In your case, these arrays will be of size 1 x 300 x 1 x 1 - the singleton T dimension is irrelevant here, because we've marked the segment length as a spatial dimension, but it'll become relevant later, when we use it in our custom layer. The following line of code shows how I've created random data in this format - you're going to want to get your 1-by-300-by-1-by-91147 data array in this format too:
inputSize = [1 300 1 1];
numTrainSamples = 91147;
xtrain = arrayfun(@(x)rand(inputSize),1:numTrainSamples,'UniformOutput',false)';
For your responses, because each of the 91147 sequences has a single label, you can keep the 91147 x 1 vector of categorical responses.
Now, on to the network. The main changes you need to make are the following:
1) Instead of the flatten layer, you will need to create a custom layer which transforms your S x S x C x B data to C x B x T data. If you are unfamiliar with custom layers, all you need to do is save the code below into its own file called 'imageToSequenceCustomLayer.m' and include this file in the directory with your main network training file:
classdef imageToSequenceCustomLayer < nnet.layer.Layer & nnet.layer.Formattable
methods
function this = imageToSequenceCustomLayer(nanvargsme)
arguments
nvargs.Name (1,:) {mustBeText} = "imageToSequence"
end
this.Name = nvargs.Name;
end
function Y = predict(~, X)
% Permute from S x S x C x B to C x B x T x S
Y = permute(stripdims(X), [3 4 2 1]);
% Re-label the array with the correct dimensions
Y = dlarray(Y, 'CBT');
end
end
end
You can then just replace your flattenLayer with
imageToSequenceCustomLayer()
The flattening is somewhat implicit in the permutation of the data that happens in the customLayer, so we don't need it anymore!
The final thing you'll need to do is to specify the 'OutputMode' NVP of your biltsmLayer as 'last', so that you don't classify every single time step of the sequence. Instead, you output the last time step and use that to classify the entire sequence.
Your network should look as shown below:
layers = [
sequenceInputLayer([1 300 1],'Normalization', 'zscore', 'Name','input');
sequenceFoldingLayer('Name','fold')
convolution2dLayer([1 7], 16,'stride',[1 1], 'padding','same','Name','conv1')
batchNormalizationLayer('Name','bn1')
maxPooling2dLayer([1 2],'stride',[1 2],'Name','mpool1')
convolution2dLayer([1 7], 32,'stride',[1 1], 'padding','same','Name','conv2')
batchNormalizationLayer('Name','bn2')
reluLayer('Name','relu1')
maxPooling2dLayer([1 2],'stride',[1 2],'Name','mpool2')
convolution2dLayer([1 5], 64,'stride',[1 1], 'padding','same','Name','conv3')
batchNormalizationLayer('Name','bn3')
reluLayer('Name','relu2')
convolution2dLayer([1 5], 128,'stride',[1 1], 'padding','same','Name','conv4')
batchNormalizationLayer('Name','bn4')
reluLayer('Name','relu3')
convolution2dLayer([1 3], 256,'stride',[1 1], 'padding','same','Name','conv5')
batchNormalizationLayer('Name','bn5')
reluLayer('Name','relu4')
maxPooling2dLayer([1 2],'stride',[1 2],'Name','mpool3')
convolution2dLayer([1 3], 512,'stride',[1 1], 'padding','same','Name','conv6')
batchNormalizationLayer('Name','bn6')
reluLayer('Name','relu5')
maxPooling2dLayer([1 2],'stride',[1 2],'Name','mpool4')
sequenceUnfoldingLayer('Name','unfold')
imageToSequenceCustomLayer()
bilstmLayer(200,'Name','lstm', 'OutputMode', 'last')
reluLayer('Name','relu6')
fullyConnectedLayer(256,'Name','fc1')
reluLayer('Name','relu7')
fullyConnectedLayer(128,'Name','fc2')
reluLayer('Name','relu8')
fullyConnectedLayer(5,'Name','fc3')
softmaxLayer('Name','softmax')
classificationLayer('Name','classification')
];
This should not throw any errors!
Hope this helps!
Tomaso
  12 Comments
Tomaso Cetto
Tomaso Cetto on 21 Jun 2021
Hi Ioana!
No problem at all, glad to see it's training for you!
As far as using the network for prediction, the 'Train Network Using Custom Training Loop' example in the documentation will hopefully be able to point you in the right direction, in particular the 'Test Model' section.
You'll see that what needs to be done is to define a seperate modelGradients function, called 'modelPredictions', which follows similar steps to the original modelGradients function. Importantly for your use case, you want to be sure to modify the for loop inside the function (which loops over batches of data), so that, when passing data through the network, you do what your modelGradients does - that is, call predict on the first network, permute the data, and then call predict on the second network, to return the right dlYPred .
Hope this helps!
Tomaso
Ioana Cretu
Ioana Cretu on 21 Jun 2021
Dear Tomaso,
Yes, it really helps. Thank you!
I also managed to install the new version of matlab on my personal pc (2021a), and I tried the first solution out of curiosity (the one suggested by you above) using the imageToSequenceCustomLayer and I got this error:
Error in CNN_LSTM (line 145)
convnet = trainNetwork(Xtrain,Ytrain,lgraph,options);
Caused by:
Layer 'imageToSequence': Input size mismatch. Error using 'predict' in Layer imageToSequenceCustomLayer. The function threw an
error and could not be executed.
I checked the dimensions and they seem fine. I attached the used files to this comment. Can you please tell me why it does not work?
Best regards,
Ioana

Sign in to comment.

More Answers (1)

ytzhak goussha
ytzhak goussha on 21 May 2021
hey,
You should check the dimentions of your features and targets of your training data.
These are the dimentions you need:
features/inputs:
{HxWxCxS}
targets:
{1xS}
H - height
W - width
C - channle
S - time point
I think that in your case, you need to transpose the targets into 1xS array instead of Sx1
  1 Comment
Ioana Cretu
Ioana Cretu on 21 May 2021
@ytzhak goussha thank you for your response. I am not sure about how I should reshape the signal. I tryed to transpose the targets but it did not solve my error.
Taking into account the comment made by by Joss Knigh at this question I reshaped the signals along the 4th dimension like that:
  • Xtraining: 1-by-300-by-1-by-91147; where 300 is the length of each signal and 91147 are the number of signals
  • Ytraining: 91147-by-1- which contains the labels for 5 classes of all the 91147 samples (these are categorical)
How do you think I should reorganise them?

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!