GPU performance significantly slow when running multiple MATLAB

Hi:

I have a code using GPU calculation, when it used by 1 MATLAB, it runs OK.

but when I open two MATLAB.exe in my computer, and run this GPU-relate function at the same time, the GPU performance becomes significantly slow.

for example, when run 1 MATLAB, it cost about 20 seconds, but when 2 MATLAB runs at the same time, it cost about 2000 seconds.

one thing I'm sure is that the GPU memory is enough for more than two tasks. therefore, the memory should not be the problem.

my GPU is GTX 980ti, which has 6GB memory.

Just want know if there is any mistake with my operation, or is there anyway to improve this performance.

Thanks!

Yu

Answer 1

[Joss Knight]

I'd be very surprised if memory isn't the issue here, because contention between your two MATLABs will be forcing continual synchronization and raw allocation on both. I'd have to see what it is you're doing.

Ultimately you should explain why you are using two MATLABs if you only have one GPU. You may have a good reason of course but it's worth looking into.

[Yu Li]

Hi:

Thanks for your reply, I record my operation into a video, which answered you question and give a brief description of what I'm doing, please see link below:

https://www.youtube.com/watch?v=Q4srDHxcsJU&feature=youtu.be

Thanks!

Yu

[Joss Knight]

Wow, that's an interesting way to post your response!

Your peak memory usage isn't the amount of memory taken up by gpuArray variables at the end of your calculations. Each operation requires working memory. So for instance, to perform a matrix multiplication may require space for the input variables, space for the output variable, and space in which to perform the calculation which is as big as the input. The need for working memory varies from operation to operation. To monitor memory usage you can use the nvidia-smi utility and set it running in a command window or terminal:

nvidia-smi -i 0 --query-gpu=memory.used --format=csv,noheader -lms 50

One way to check this might be to reduce the size of your data. If the unusual behaviour (performance when running on two GPUs) persists for small data then perhaps there is a real problem. I would expect it to take more than twice as long, because there is significant overhead due to the need for the GPU driver to continually switch context. 100x slower is unusual for that though.

I'm concerned that you have to reboot your computer, however if you are using this GPU to drive your display that is not surprising since it isn't being given any chance to render your desktop. If not, then perhaps the real issue here is CPU memory, since it tends to be virtual memory access that makes a system appear to hang due to continually thrashing of the disk.

[Yu Li]

Hi:

Thanks for your reply.

the command: nvidia-smi... does not work in my side, perhaps I'm using GTX GPU but this command is target on TITAN, QUADRO, etc.

I slight revise my code to monitor the available GPU memory at end of each operation, which should be able to provide the detail information, as you can see, the max memory usage for this code is about 2.5GB, if I run 2 MATLABs the max GPU memory usage should be about 5GB<6GB.

could you please give me some more suggestions?

https://www.youtube.com/watch?v=Xjltr0UR_FY&feature=youtu.be

Thanks!

Yu

[Joss Knight]

On Windows, nvidia-smi is not on the path. You have to navigate to C:\Program Files\NVIDIA Corporation\NVSMI to find it.

It's no good just to look at the memory usage at the end of each operation. MATLAB allocates, uses, and deletes memory internally during the operations.

[Yu Li]

Hi:

Ok I got it. I have achieved the memory usage using the command, and found that the maximum usage is 4246 MiB. thus, if I run 2 MATLABs at the same time, the total memory usage may exceeds 6GB.

But, as you mentioned, even though in this situation, the total calculation time should only be about 2*normal time consuming, 200 times is obviously not reasonable.

do you have any suggestions regarding this?

Thanks!

Yu

[Joss Knight]

It wouldn't be normal if memory was not the issue. Given that it is the issue, I suspect that this is not unexpected behaviour. I recommend that you either reduce the amount of data that you are processing at once, or do not use two MATLAB processes at once.

[Yu Li]

Hi:

Thanks for your reply.

The size could not be reduced. Open two Matlab is just a way to save time.

Is there no any other way to improve the performance? Or is it possible that there are some problem with gpu or gpu code?

Thanks.

Yu

[Joss Knight]

Focus on doing as efficient processing as you can on one MATLAB by vectorizing your code. I would never advise someone on Windows to share a GPU, it just isn't a supported workflow for NVIDIA hardware.

[Yu Li]

Hi:

Thanks for your suggestion, I use GPU because it has about 2-3 times faster than vector calculation using MATLAB even when parallel situation. Now, in order to conduct multiple MATLAB calculation, I have changed the code back to normal MATLAB code. looks there is a trade-off relation.

However, since your answer does not solve the specific question for this topic, I could not accept your answer. but I really appreciate your suggestions and thanks for your time for discussion here.

Best!

Yu

[Walter Roberson]

"Just want know if there is any mistake with my operation"

Yes, it is a mistake to share the GPU between multiple processes in Windows.

A document I was examining recently on how NVIDIA GPU handle multiple processing indicated that task synchronization is expensive in relative terms, and that moving tasks on / off of the GPU is the most expensive operation for their GPU.

When you run two different processes each trying to access the GPU, you are forcing it to run the most expensive operation often.