Why does parfor slow down with large shared arrays?

I've been trying to speed up a simulation loop using parfor, but noticing it actually runs slower than a regular for loop when I use a large matrix that gets read (not written) inside the loop.

Read-only variables were broadcast efficiently to workers. Is there a size threshold where broadcasting overhead outweighs the parallelism gains? Should I be using parallel.pool.Constant instead?

Running MATLAB R2024b with a local 6-worker pool. TIA!

[Torsten]

Should I be using parallel.pool.Constant instead?

Try it.

% Create a constant object outside the loop
constArray = parallel.pool.Constant(myLargeArray);
parfor i = 1:numIterations
    % Access the data using .Value
    chunk = constArray.Value; 
    ...
end

[William]

Yeah, that's exactly the fix here. It sends the data to workers once instead of re-broadcasting every iteration, which kills performance with large arrays. Thank you, Torsten!

Answer 1

[John D'Errico]

I can't test this myself, and I lack a great understanding of the parallel tools. But it looks like there are a few issues, mainly based on the need to pass that array around. Each worker in a parfor pool normally wants its own copy of any such array, as it does not know what you will be doing with that data. If you would modify that array, then you certainly need to have private copies.

First is the case of data broadcasting overhead: If a large array is read but not sliced by the loop counter, MATLAB decides it is a broadcast variable. The client must serialize and transfer the entire array to every worker in the pool. This causes communication issues, and therefore bottlenecks.

Next is memory thrashing and swapping: If you have 6 workers and an 8 GB shared array, MATLAB may try to allocate 6×8=48 GB of RAM. If this exceeds your system's physical memory, the OS resorts to hard drive swapping, slowing down performance.

You can probably figure out which of these events is happening, by looking at the memory requirement, and by looking at an activity monitor. But it is simple enough. If the total RAM required is too much for you exisitng RAM, then you will need to get into swapping, and that will always be a bad thing. If the array is just large, but the multiple copies will not come that close to your total RAM, then your system is just having communication problems. It is still going to be nasty but you need to avoid it. Now a constant array might help.

If I understand correctly, I think you want to shift to a thread based pool to avoid the communication/swapping issues. It depends on what you will be doing with that large array.

% Shut down any existing process pool
delete(gcp('nocreate'));
% Now open a thread-based pool, which can share the existing RAM
parpool('threads'); 

From what you are saying, I don't think you want a Constant array, since that still needs to pass the entire array at least once to each worker, but you can try the idea too.

The thread based pool can share the same memory. Again, I think what matters is how you will be using that array, and how large is that array relative to your overall memory. I would suggest trying the various options, while carefully watching what happens to your system using an activity monitor. At least, that is what I always did when I was more actively using these tools. A good approach in MATLAB is often to try it out on your system and your problem.