Using both GPUs of Nvidia Tesla k80 in one MALTAB code simultaneously

I am aiming to run a MATLAB code on both Tesla k80 GPUs and take advantage of both GPUs at the same time. Can someone tell me if it is possible in practice to do that with MATLAB? I noticed that it is possible to transfer data to only one gpuDevice using gpuArray command. However, I beileve that there should be a trick to activate and force both GPUs on Tesla k80 to work together simultaneously.

I will apprecite any comments and your experience on this!

Thank you.

[scottneh]

Hello, I see thsi post is less than a year old so I hope you may see my question. I am setting up a machine with the following parameters :

Dual Xeon Silver 4216

128 GB Memory

3 Tesla K80 cards

Asus WS C621E SAGE main board

HDDs are all solid state NVMe

Asus GeForce RTX2070 video card

PSU: Corsair AX1600i (main) Corsair RX850 (for a little extra power)

Latest BIOS installed

I have read is other forum posts this is impossible but It looks like you made it work. I am still working on a 3D print for an air duct to cool them with a couple 120mm fans so it will be a few weeks before I am really trying to run them. Wanted to see if you have any pointers from your experience.

Answer 1

[Walter Roberson]

The trick is that the limitation is per worker, so you can use parfor or parfeval to run computation on multiple gpu.

Mathworks has also started supporting dual communicating GPUs in fairly limited circumstances, deep learning if I recall correctly.

[Jason Ross]

We have a multi-GPU example using parfor and parfeval here

Answer 2

[Amir Masoud Kordbacheh]

Thank you very much Walter and Jason for your useful advise and information.

I worked it out and could run my code on my two GPUs. At this time, I have got one NVIDIA GeForce GTX 1080 Ti and one NVIDIA TESLA K40. When running my code on my workers, I noticed that it runs a bit slower than the time I am only running it on one of those GPUs. I note that TESLA K40 generally performs faster as it has got more double precision.

My code includes a large 2D-array matrix (a wavefunction of the non-linear Schrodinger Equation) that propagates and is evoluted over time by a for-loop including a number of FFT in 2D. What I have used is something like this:

spmd
    for ii=1:(the end time)
        statement...
    end
end

I have done a bit of research and noticed that it might be due to the communication slowness between two GPUs that Windows 10 causes. Do you think installing Linux can solve the issue? What else do you recommend to speed up working with 2 GPUs. I am also wondering if having 2 GPUs on one PCB can help, like using a TESLA K80? Would it increase the communication speed between the two GPUs?

Thank you for your reply!

[Jason Ross]

The K40 is going to be about 5-6 times faster at doubles than the 1080. So the worker that uses the 1080 is going to be taking considerably longer for the same work. Depending on how the scheduling ends up, that could be adding to your overall time.

To MATLAB, the K80 is going to appear as two GPUs but they share the same PCI interface. You could get speedup simply because you will pretty much have the same as putting two K40 cards in your machine. Since the workers are independent, communication speeds don't matter all that much. If your machine can accomodate two K40s plus your 1080 you could get similar performance as one K80.

If you could upgrade to something like a Titan V, it would offer about 4.5X speedup versus your K40 -- the Titan V offers good doubles performance.

Depending on your workflow, you can always look into doing work locally and then scaling up to AWS to get access to V100 cards to run larger data sets -- a V100 is going to be about 5 times faster, and you can rent machines with multiple V100s.

[Amir Masoud Kordbacheh]

Such a perfect explanation! Thanks Jason!

My motherboard (ASUS WS X299 sage) has got 7 PCI-e 3.0 slots meaning that I can accomodate 4 high-end double-slot GPUs running at x16. Your advice is just at the very right time as it's been a while I am thinking of getting the TITAN V for its high doubles performance. However, I was a bit unsure how much it could speed up compared to my current K40. Your answer just resoleved all my doubts about the TITAN V.

Thanks also for suggesting rental machines with multiple TESLA V100. I have noticed that there are two different types of V100 in the market. One runs at 300 Watt and the other at 150 Watt. I couldn't find any explicit specifications for the 150 W one, but based on my research through some online marketting websites, I reached to the following conclusion regarding their diffrences:

NVIDIA TESLA V100 (150 Watt):

Peak Double Precision Performance: 7 TFlops

Peak Single Precision Performance: 14 TFlops

NVIDIA TESLA V100 (300 Watt):

Peak Double Precision Performance: 7.5 TFlops

Peak Single Precision Performance: 15 TFlops

May I ask your idea about these details? Since TESLA V100 - 300 Watt is pretty much out of my budget, I am just thinking of buying the 150 Watt one. Now I'm actually stuck between two options: TESLA V100 (150 Watt) or TITAN V? Which one do you recommend more?

I appreciate your thoughts on this.

[Jason Ross]

It looks like the difference between the 150 watt and 300 watt V100 is that the clock speed is turned down, and delivers 80% of the performance according to nVidia. SOURCE

The source of the performance numbers for the cards is found on Wikipedia:

• GeForce / workstation cards
• Tesla cards

As for V100 vs. Titan V, that comes down to what your chassis will accept and support (plus budget, of course!). With the V100 you do need to know that your chassis provides adequate cooling for the card, as it is entirely passive with no fan of its own. The Titan V has a built-in fan so it can provide its own cooling. But it looks like the performance is very similar.

[Amir Masoud Kordbacheh]

Thanks again Jason for providing the refrences and information. It was all really useful, much appreciated!