parfor schedule not very smart?

I noticed that the parfor scheduler isn't distributing the tasks smartly. For example, why does the following simple code take 250 sec to execute? My guess is that a single worker is hogging both loops 1 and 2 while the remaining 7 sit idle. Why should that be the case? Since it takes the 7 workers no time to finish executing loops 3 to 16, one would expect that one of them gets assigned loop 2, in which case it would probably take only 130 sec to finish the job.

Is anyone else seeing this? or am I deluded...

 % code
    matlabpool open 8
    tic
    answer = cell(16,1);
    parfor ii = 1:16,
        if (ii == 1 || ii == 2)
            pause(120);
        end
        answer{ii} = ii;
    end
    toc

> Elapsed time is 257.125697 seconds.

[William Smith]

The scheduler appears to pre-assign blocks of work to the individual workers, presumably because the communication overhead to send individual tasks to workers is deemed too heavy.

I also see many cases when I'd appreciate a better allocation of work to workers for heterogeneous workloads.

Answer 1

[Matt J]

Splitting work the "smart" way would require that parfor know a priori the outcome of expressions and commands within the body of the loop for all ii, in this case the logical expressions

 (ii == 1 || ii == 2)

Doing a pre-analysis like that would essentially mean running the entire loop serially prior to doing any parallelization, killing the intended benefit of parfor.

Answer 2

[Edric Ellis]

The PARFOR scheduler divides loop iterations up in a fixed way to try and achieve a good balance for 'normal' workloads. In particular, it tries to send out decent sized chunks of work to overcome data transmission overhead. Pathological cases like this will always exist. If your long-running cases vastly exceed the time taken by the 'quick' cases, you might just be better off submitting a series of batch jobs and letting the cluster schedule each iteration individually.

Answer 3

[Spencer Chen]

Thanks, Matt and Edric. Yes, indeed, Edric. I was hoping not have to move that way. Really, parfor can be a bit smarter and reschedule unfinished loops to idle workers.

[Matt J]

That can require reslicing and/or re-broadcasting data in some cases, which would add overhead. Seems to me that it would be hard for parfor to know when it is optimal to do so.