Why is my Trained SSD Object Detector not giving a bounding box?

I have been working on implementing an object detection network that takes an image of a car and returns a bounding box around the license plate. I have tried using Fast and Faster RCNN networks for this, but the training images are too large and my hardware is not adiquitte enough causing the network to run out of memory immediatly.

My training data is 427 images of size [720 1280 3] with each image having a bounding box to identify the license plate, I am trying to train an SSD Object Detection Network and though I can train the network, when I try to use the trained network on an image it does not return any bounding box.

My Code

my_data = load('Copy_of_mydata.mat');
%loads .mat file with bounding boxes and path to image for the bbox
trainingData = my_data.vehicleTrainingData;
my_dataDir = 'D:/Senior_Project_Matlab/dataset/train/images';
trainigData.imageFilename = fullfile(my_dataDir,trainingData.imageFilename);
%datastores for bbox and images
imds = datastore('./dataset/train/images/*.jpg');
blds = boxLabelDatastore(trainingData(:,2:end));
%combines datastores
ds = combine(imds,blds);
%input size for network and number of bbox labels as number of classes
inputImageSize = [360 640 3];
numClasses = 2;

network = 'resnet101';
%create layer graph for ssd network based on resnet101 or whatever network
%is assigned to network variable
lgraph = ssdLayers(inputImageSize, numClasses, network);

options = trainingOptions('sgdm','InitialLearnRate',18.0e-4,'LearnRateSchedule','piecewise', ...
    'LearnRateDropFactor',0.7,'LearnRateDropPeriod',1,'Verbose',true,'MiniBatchSize',9,'MaxEpochs',8, ...
    'Shuffle','every-epoch','VerboseFrequency',10);
%train the network
detector = trainSSDObjectDetector(ds, lgraph, options);

%trying the network
Files = dir('dataset/train/images/*.jpg');
for x = 1:round(length(Files)/10)
    %gets file name
    FileName = append('dataset/train/images/',Files(x).name);
    %reads image
    I = imread(FileName);
    %runs image through network
    [bboxes, scores, labels] = detect(detector, I)
end

Training:

*************************************************************************

Training an SSD Object Detector for the following object classes:

* licensePlate

* Other

Training on single GPU.

Initializing input data normalization.

|=======================================================================================================|

| Epoch | Iteration | Time Elapsed | Mini-batch | Mini-batch | Mini-batch | Base Learning |

| | | (hh:mm:ss) | Loss | Accuracy | RMSE | Rate |

|=======================================================================================================|

| 1 | 1 | 00:00:03 | NaN | 25.86% | NaN | 0.0018 |

| 1 | 10 | 00:00:26 | NaN | 100.00% | NaN | 0.0018 |

| 1 | 20 | 00:00:51 | NaN | 100.00% | NaN | 0.0018 |

| 1 | 30 | 00:01:16 | NaN | 100.00% | NaN | 0.0018 |

| 1 | 40 | 00:01:41 | NaN | 100.00% | NaN | 0.0018 |

| 2 | 50 | 00:02:07 | NaN | 100.00% | NaN | 0.0013 |

Output:

labels = 1x0 catagorical

bboxes = empty array

scores = empty array

I have tried training both with just the license plate bounding boxes(mydata.mat) and with license plate bounding boxes and other bounding boxes(Coby_of_mydata.mat) with both trained networks producing the same output for every image I try. I have also tried and image input size of both [720 1280 3] and [360 640 3] again with the same outputs and I have tried all of the above options with resnet101, resnet50 and vgg16.

Answer 1

[Divya Gaddipati]

Here are a couple of things I would recommend you to try:

1. Firstly I would recommend you to increase the training data or include data augmentations. 427 is too less data to train a deep neural network. You can refer to this example to check out the typical data augmentation methods used for object detection in images.

2. Make sure all of your ground truth bounding boxes in your training data are valid. You can refer to the validataInputData function used in the 'Load Data' section of this example.

3. If there are memory contstraints, you should reduce your MiniBatchSize to 2 or 3 instead of 9.

4. From the training progress information, I see that the loss is always NaN. This means that the network is not learning anything and hence it is unable to detect any bounding boxes on the test set. In this case, you can try to play around with the training options to find the best set for your data. Firstly, I would recommend to increase the number of training epochs and also lowering the initial learning rate (maybe 1e-6). If the NaN issue is still not resolved, you could try using a different solver.

5. Since, you have resized the images to a lower size, you could try other object detectors like YOLOv3 which is known to perform better than SSD. You can refer to this link which explains in detail the workflow and you can easily adapt it to your dataset.