How can I recompile the GPU libraries ?

Hello,

I want to test my machine learning code on GPU environment but it doesn't work.

I constantly getting warnning below

Warning: The CUDA driver must recompile the GPU libraries
because your device is more recent than the libraries.
Recompiling can take several minutes. Learn more.
>In run_MNIST_pinv (line 9)

and continuously another errors erupt

Error using gpuArray/hTimesTranspose
Call to dgemm in CUBLAS failed with error status:
CUBLAS_STATUS_EXECUTION_FAILED.
...
    etc

so I want to fix warnning(require recompile the GPU libraries) first to solve these errors

how can I recompile the GPU library

ref. my HW

OS : ubuntu 18.10

CPU : AMD Ryzen Threadripper 2990WX

GPU : NVIDIA Geforce RTX2080 Ti

MATLAB ver : R2018b

I already checked GPU driver version.!

Thanks !

[Matt J]

Did you try rebooting?

[Chanho Park]

Yes, I already did, but the messages still rise

Answer 1

[Joss Knight]

This is a known issue with CUDA 9.1 and the new Turing generation of GPUs. You don't need to do anything except ignore the initial warning or error.

There are multiple issues. The warning about recompiling of libraries is spurious and not relevant. The error from cublas is one-time only. And there is an equivalent error in cuDNN. A permanent workaround is to put the following into your startup.m file:

warning off parallel:gpu:device:DeviceLibsNeedsRecompiling
try
    gpuArray.eye(2)^2;
catch ME
end
try
    nnet.internal.cnngpu.reluForward(1);
catch ME
end

[Chanho Park]

Thanks a lot!

but Error from cuBlas still rises. cublas error isn't one-time only

Is there any way to access 'gpuArray/hTimesTranspose'??

[Joss Knight]

That's annoying for you. In my tests on my Turing card the error is a one-off between each device reset. Perhaps you are resetting the device? You should give me your reproduction code so I can investigate.

[mao]

Hello, I added the startup.m file to the path as you said and restarted matlab. However, when I re-run the following code, I still get the previous error

[Joss Knight]

This is a bug in R2018b. You should upgrade MATLAB if you can.

[向前 刘]

I have the same problem using matlab2018b. Does this bug exist in other matlab version?

[Steven Lord]

I suspect this is Bug Report 1898305 which is listed as fixed in release R2019a. Joss can correct me if that's not the correct bug report.

Answer 2

[JIE WANG]

Hello, Did you solve this problem? I also meet this problem and want to obtain some experience from you.

[Joss Knight]

This issue is fixed in R2019a.

[fei wong]

Hello, Did you solve this problem? I also meet this problem, it annoyed me a lot.

[Ricardo Gonzales]

I have R2019a but it still shows me the problem. I have cuda v11.1

[Joss Knight]

You have a Turing card and are receiving the library recompiling warning?

Answer 3

[yun di]

我也遇到了相同的问题，是可以忽视吗？我的matlab版本是2018b

I also encountered the same problem. Can I ignore it? My MATLAB version is 2018b.

[Joss Knight]

This question is answered above.