gpu limit on 3070 with a simple CNN
1 view (last 30 days)
Kuno Bruswachtl on 5 Jan 2022
I have had this problem for the past two days and I have ran out of options how to solve this. I am training a basic CNN with the input and output mentioned in the code down below. However I use my data, wheter I split it up or use smaller samples, I always get the error " Out of memory on device. To view more detail about available memory on the GPU, use 'gpuDevice()'. If the problem persists, reset the GPU by calling 'gpuDevice(1)'." I also changed my minisizebatch to one, which doesn´t help neither. The last thing I tried is to run it on my CPU where it stops after the 3rd epoch out of three. Any Ideas?
imageInputsize = [6570 7000 1];
layers = [
convolution2dLayer(3,16, 'Padding', 'same' )
convolution2dLayer(3,10, 'Padding', 'same' )
resize3dLayer('OutputSize',[10 70000 5], 'Name', 'resize10x10x5')
options = trainingOptions('sgdm', ...
'InitialLearnRate', 0.01, ...
'MiniBatchSize', 1, ...
'Verbose', false, ...
net = trainNetwork(arr_B,arr_V,layers, options);
note: the input as well the outputs are double arrays.
thank you so much
Joss Knight on 5 Jan 2022
You have 2-D data so you can't use a resize3dLayer. Use a resize2dLayer. But actually it doesn't look like that's what you want.
If you have 70000 samples, they will be the batch dimension not one of the spatial dimensions, so you shouldn't include that dimension at all when specifying your layers. You'll probably need to permute both the data and the response to S-by-C-by-B format (ie. permute the last two dimensions). Perhaps what you want is a series of 1-D convolutional layers?
It seems like your response requires your network to output a result of length 10 with 5 channels? So you're going to need to make your network reduce the spatial dimension to 10 and ensure that the last convolution has 5 filters. Use analyzeNetwork(layers) to check what you're doing and make it work.