GPU multiplication slowdown after ~500 ops. (Image Attached)

An optimization task requires I perform many large matrix multiplication operations sequentially. However, when I do many multiplication operations in sequence, after roughly the 500th operation the GPU multiplication becomes much slower. The code I used to demonstrate this is as follows:

     A = gpuArray(rand(1000));
     x = gpuArray(rand(1000,1));
     fff=0;
     i=1;
     while(i<=1000)
         tic;
         A*x;
         fff = [fff,toc];
         i=i+1;
     end
     figure
     semilogy(fff)

When we examine the time it took for each individual operation to complete, we observe the following behaviour:

Why does the time cost suddenly spike to normal CPU speeds at about 500 operations in? Is there any way I can prevent this?

[N.B. I'm using a GeForce 940MX and MATLAB 2016b]

[Sharath Chandran]

Does this behavior consistently occur every-time you run the script?

I tried running your script on my system with NVIDIA Quadro K620 using R2016B and plotted about 150 graphs. There was not much deviation in time taken per iteration. There were couple of spikes indicating that time taken for that particular iteration was more than '1e-4', however I could not reproduce the behavior, that you've reported above, consistently.

I reckon that this is an exception and if you try running the script multiple times and obtain the plot for each of them, you'd observe the expected behavior.

Please let us know the results after you've carried this experiment on your system.

Answer 1

[Joss Knight]

This is just because you are not accounting for the asynchronous behaviour of the GPU in your timing. This question is asked regularly on this forum, refer for instance to this recent post.