Neural Network Training - Input Data with Targets

Hello dear community,
I'm pretty new to all neural network training subjects and stuff so be gentle with me and all the glossary that is required, I'm really sorry :)
I'm trying to train a neural network model that I've imported from MATLAB. The model should perform depth estimation - the input layer of the net should recieve RGB images, sized 384X384 , and the output supposedly should give me back 2D images, of the same input image, but the estimated depth of each pixel in the image.
The problem is that I really don't know how to define the input in the following command line:
netTrained = trainnet(images,net,lossFcn,options)
I thought about defining 'images' as a combined dataset cell (1X2) , when I'm combining between the 2D RGB images (this is the 1x1 element within the cell), and the depth ground truth depth corresponding images datastores (this is the 1X2 elemnet within the cell). when the ground-truth should be my targets in that case because this is what used in order to calculate the loss function.
Is it the correct way to define the input ?

If it is the correct way, how should I define the validation data accordingly ?
is it fine to do the same for the validation data as the images combined dataset ?

Hope someone can help me please,
I'm really confused.
Thanks in advance,

Answer 1

[Aastha]

Hi Maxim,

As per my understanding, you want to train an image-to-depth network in MATLAB. To do this, you may refer to the steps outlined below:

1. Create imageDatastore objects for your input RGB images and corresponding depth images for training, testing, and validation datasets. Here's how you can do it in MATLAB:

imdsTrain = imageDatastore("PathToTrainData", ... 
    IncludeSubfolders=true, ... 
    LabelSource="foldernames"); %create a datastore for the training input images 

imdsTest = imageDatastore("PathToTestData", ... 
    IncludeSubfolders=true, ... 
    LabelSource="foldernames"); %create a datastore for the test input images 

imdsVal = imageDatastore("PathToValData", ... 
    IncludeSubfolders=true, ... 
    LabelSource="foldernames"); %create a datastore for the validation input images 

imdsTargetTrain = imageDatastore("PathToTrainTargetData", ... 
    IncludeSubfolders=true, ... 
    LabelSource="foldernames"); %create a datastore for the training target images 

imdsTargetTest = imageDatastore("PathToTestTargetData", ... 
    IncludeSubfolders=true, ... 
    LabelSource="foldernames"); %create a datastore for the test target images 

imdsTargetVal = imageDatastore("PathToValTargetData", ... 
    IncludeSubfolders=true, ... 
    LabelSource="foldernames"); %create a datastore for the val target images

Refer to the following documentation link for more information on formatting data and training the network:

https://www.mathworks.com/help/deeplearning/ug/datastores-for-deep-learning.html?#mw_204e9632-bce5-461a-a0b1-acd35e089b52

2. Combine the input and target datastores using the combine function to create paired datasets for training, testing, and validation.

% combine the inputs and targets 

imdsTrainInput = combine(imdsTrain, imdsTargetTrain); 

imdsTestInput = combine(imdsTest, imdsTargetTest); 

imdsValInput = combine(imdsVal, imdsTargetVal); 

For more information on combine function, you may refer to the link of MathWorks documentation mentioned below:

https://www.mathworks.com/help/symbolic/sym.combine.html

3. Use the trainingOptions function in MATLAB to specify the validation data that will be used by the network. This can be done as follows:

options = trainingOptions(...,ValidationData=imdsValInput); % ... represents other options 

Refer to the following documentation link of trainingOptions function in MATLAB for more information on it:

https://www.mathworks.com/help/deeplearning/ref/trainingoptions.html

I hope this is helpful!

[Maxim]

Hi Aastha :)

Thank you for your help ! I'll give it a try !