Why do I run into "Out of memory" error using Faster R-CNN method?

7 views (last 30 days)
I am trying to use the "fasterRCNNobjectDetector", but it keeps getting "out of memory" errors:
Error using +
Out of memory on device. To view more detail about available memory on the GPU, use
'gpuDevice()'. If the problem persists, reset the GPU by calling 'gpuDevice(1)'.
This happens even when lowering the "miniBatchSize" to 4 (which is the minimum for this algorithm) and even when working on the CPU instead of the GPU:
options = trainingOptions('sgdm', ...
'MiniBatchSize', 4, ...
'InitialLearnRate', 1e-6, ...
'MaxEpochs', 10, ...
'ExecutionEnvironment','cpu',...
'CheckpointPath', tempdir);
It should be noted that this happens after restarting MATLAB. The simple "RCNNobjectDetector" (not the faster version) works, but how can I run the faster version?

Accepted Answer

MathWorks Support Team
MathWorks Support Team on 26 Jan 2024
Edited: MathWorks Support Team on 15 Feb 2024
 This issue is likely caused due to the following reasons:
1) Large input dimensions of the images (Big Images).
2) Limited memory availability on the GPU card.
3) Size and complexity of the network architecture (Big Network like VGG-16).
4) Combination of (1) and (3).
Further, the Faster R-CNN method processes the entire input image without resizing. This is in contrast to the "RCNNObjectDetector" method which performs inherent cropping and resizing of regions to be comparable to the input dimensions of the network.
Additionally, the Faster R-CNN methods trains a Region Proposal Network whose architecture is derived from the network passed into the function. More precisely, the layers after the last convolutional layer are removed and replaced with two smaller convolutional layers, thus increasing the complexity of the algorithm. These two conditions perhaps may explain why the "trainRCNNObjectDetector" method was able to successfully train on the same network.
A couple of workarounds to continue using the "trainFasterRCNNObjectDetector" method are mentioned below:
1) Setting the "SmallestImageDimension" parameter of the "trainFasterRCNNObjectDetector" to a value of 400. However, this parameter may need to be adjusted to account for GPU specifications. Setting this parameter will resize the images during training and help avoid "Out of memory" errors.
2) Using a smaller network in addition to setting the "SmallestImageDimension" parameter as shown in this example:
We suggest setting the "SmallestImageDimension" parameter to something <= 600. This will auto resize the images during training. Alternatively, the user may manually resize the images themselves along with the bounding box ground truth data before calling the training functions.
3) Updating the GPU card. To train a network like VGG-16 using the Faster R-CNN method, a GPU with more memory might be required.
4) For versions of R2021a and above, converting the large image into a "blockedImage" object, and then use this "blockedImage" object when using the "detect" function (instead of the original image):
Blocked image objects is an image made from discrete blocks. Functions that are meant to be applied to images in MATLAB are then performed on each of these blocks, rather than the entire image at once, so as to save memory. The "apply" function is used to execute a function on a large image. The documentation for "blockedImage" and "apply" are shown below:
"blockedImage":
"apply":
In the case of using the "detect" function with a "blockedImage" object, one uses the "apply" function to execute the "detect" function on each block of the object. Assuming that the image data is named "imageVariable" in the MATLAB workspace, the code would look as follows:
>> blockIMG = blockedImage(imageVariable, 'BlockSize', [1500 1500]); % This creates a blockedImage object with blocks no smaller than 1500 x 1500
>> bblocks = apply(blockIMG, @(x) detect(detector, x.Data, 'ExecutionEnvironment', 'cpu')) % This will call the detect function on the blockedImage object
You can look into altering the parameters passed into the "blockedImage" and "apply" functions to try and decrease this run time more.
The returned value, "bblocks", will be an n x 2 cell array, where n is the number of blocks in the "blockedImage" object. Each row of this cell array has two cells, the first will contain the detected bounding boxes for the given block, while the second contains the origin of the block in the larger image.
This output format is described in the "output" section of the "apply" documentation:
5) For versions R2019b to R2021a, use "bigImage" instead of "blockedImage". The documentation for "bigImage" and "apply" are shown below:
"bigImage":
"apply":

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!