parfor is slower than for

Dear Matlab community, I am trying to implement parfor function and have 5000 iterations. What kind of settings and preferences should I select? looking at different resources I did such commands:

   parpool(8)
    Result:
            Connected: true
           NumWorkers: 8
              Cluster: local
        AttachedFiles: {}
          IdleTimeout: 30 minutes (30 minutes remaining)
          SpmdEnabled: true

Then I run a function:

    clc;
    clear;
    modulation = 4;
    iterations = 5000;
    tic
    parfor i=1:iterations
        if(modulation == 4)
            result(1,i) = modulation*i;
        elseif(modulation == 16)
            result(1,i) = 2*modulation*i;
        end
    end
    time_1 = toc; 

    tic
    for i=1:1:iterations
        if(modulation == 4)
            result(1,i) = modulation*i;
        elseif(modulation == 16)
            result(1,i) = 2*modulation*i;
        end
    end
    time_2 = toc; 

    fprintf('time Parallel =  %.9f;\n', time_1);
    fprintf('time Serial   =  %.9f;', time_2);
    fprintf('\n');

Result time is as following: time Parallel = 0.150103298; time Serial = 0.000062877;

How to improve parallel run time?

Thank you

[Stephen23]

"parfor is slower than for"

"How to improve parallel run time?"

Why do you think that parfor should be faster than for for your problem? It is not the case that parfor is always faster. Have you read the MATLAB documentation on this?:

https://www.mathworks.com/help/distcomp/decide-when-to-use-parfor.html

[Talgat Manglayev]

Thank you Stephen, of course I looked documentation and other resources. I don't see a reason why parfor should be slower than for. That's why I am asking, and if someone with more experience would share knowledge.

[Adam]

You need to consider the actual problem you want to solve when it comes to parallelisation. Creating a simple example is fine for understanding syntax and some of the warnings and errors about variable classification you may get, but for simply looking at the speed it is erroneous.

In your example was the parallel pool already open before the parfor loop? If not then this takes time to open. Even if it is data has to be copied to the workers.

Your serial process took 0.000062877s in this case. Just how fast do you want this to be???!!!! Do you really expect that copying data to multiple workers would be faster than that? That serial time is so fast I can't even be bothered to work out how to say that time in words!

[Adam]

From the link Stephen posted:

A parfor-loop might not be useful if you have:

Code that has vectorized out the for-loops. Generally, if you want to make code run faster, first try to vectorize it. For details how to do this, see Vectorization (MATLAB). Vectorizing code allows you to benefit from the built-in parallelism provided by the multithreaded nature of many of the underlying MATLAB libraries. However, if you have vectorized code and you have access only to local workers, then parfor-loops may run slower than for-loops. Do not devectorize code to allow for parfor; in general, this solution does not work well.

Loop iterations that take a short time to execute. In this case, parallel overhead dominates your calculation.

Answer 1

[Stephen23]

"I don't see a reason why parfor should be slower than for."

I don't see a reason why parfor should be faster than for. The only time it is worth using parfor is when the time saved running the operations inside the loop is greater than that of the extra time for the parfor overhead. This is computing, and nothing comes for free (something that beginners seem to forget when they expect infinitely fast computers with infinite memory, infinite-precision numerics, and overhead operations that take no time at all).

Running a parfor is a much more complicated thing than running a simple for loop because the data needs to be partitioned and the results grouped again afterwards, so it is entirely possible that a parfor might be slower. This is clearly explained in the documentation: "There is overhead in calling parfor instead of for. If function evaluations are fast, this overhead could become appreciable. In particular, solving a problem in parallel can be slower than solving the problem serially."

source: https://www.mathworks.com/help/optim/ug/improving-performance-with-parallel-computing.html

With your problem the actual task you perform in the loop is so fast that the parfor overhead will take considerably more time than the operation itself.

[Talgat Manglayev]

Thank you Stephen, I got your idea: background and preparation work takes more time rather than work within the iteration, but now I increased iteration to 5 000 000 and still have:

time Parallel =  0.735779271;
time Serial   =  0.053804071;

[Adam]

And the answer is still the same. Parallel processing is not suitable for trying to increase the speed of something that takes fractions of a second. Why would you even bother trying to do this? All you did was increase the amount of data that has to be carted back and forth to/from the workers.

[Talgat Manglayev]

I saw in this video an author used simple operation for 5000 iterations only. (sorry, I have edited link) https://youtu.be/pd9oVP2yEtY

[Stephen23]

@Talgat Manglayev: did you notice that their parfor loop is slower than their for loop? The narrator even states this in the voiceover. The final values shown in their command window are:

t_normal_for = 0.0827
t_parfor = 0.5912

The fact that parfor is nearly ten times slower than for (for that trivial addition that they used as a demonstration) does not surprise me at all.