Convolutional neural networks: What is the best practice training approach using graphics cards?

1 view (last 30 days)
Training a convolutional neural network (CNN) for image classification, I successfully used the trainNetwork function employing 4 CPU cores. However, the process takes quite a lot of time (hours) and must be accelerated, e.g by using a graphics card.
Currently, I pass a tbl to trainNetwork containing the image paths and labels. I suppose that images are read from the disk and then sequentially processed by the function. This might work for CPU based processing system to some extent. However, using a GPU, I assume that this approach will significantly slow down the training process due to a number of GPU accesses and related delays. Is it e.g. possible to transfer the training data batch-wise to the graphics card or is this automatically done using the parallel processing toolbox? How do I have to adapt my code in this case? It would be great to have a minimalistic code snippet.
Thank you! Best, Stephan
P.S.: I should mention that I cannot use an imageDatastore, since this datatype apparently does not work for regression CNNs which I use.

Accepted Answer

Joss Knight
Joss Knight on 27 Oct 2017
Edited: Joss Knight on 27 Oct 2017
You needn't worry too much about the efficiency of file i/o. Even with a file-path table, data is prefetched in the background during training. Your only concern is setting the mini-batch size appropriately for maximum efficiency.

More Answers (1)

Corey Silva
Corey Silva on 24 Oct 2017
You can use the "trainingOptions" function to tell the "trainNetwork" function to use the GPU.
For example, if we already have "trainDigitData" and "layers" defined, then the following example does this:
>> options = trainingOptions('sgdm','ExecutionEnvironment','gpu');
>> convnet = trainNetwork(trainDigitData,layers,options);
  1 Comment
quincy
quincy on 25 Oct 2017
Edited: quincy on 25 Oct 2017
Dear Corey, thanks for your answer! I know about this option and it should work using an image database as data source. As far as I understood, data handling is then optimized such that the number of graphics card accesses are minimal. Using a CNN in regression mode, the targets need to be the regression values. This, however, seems not to be supported by the image database. Here, the workaround is to use a table. The question is if tables also work optimal with graphics cards or if there is another best practice.
Best, Stephan

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!