Is there a way to change number of worker in a created parpool?

Hi,

I have launch a very long simulation wih parfor loop inside. To keep working on my PC, I let one core free. Now that I'm leaving my office, I would like to add a worker on my parallel pool so that all the cores will be used during the night.

Does anyone know if there is a way to change the number of worker in a running parrallel pool?

Can I pause my simulation and enter a command line to change it?

Thanks!

GL.

Answer 1

[Edric Ellis]

Unfortunately, there is currently no way to increase the number of workers in a parallel pool. One approach that might possibly work for you is to do the following:

1. Launch the pool using the full number of workers
2. Divide your code into multiple parfor loops
3. Have each parfor loop specify a number of workers to use using the "M" parameter.

For example, you might do something a bit like this:

p = parpool('local');
nw = p.NumWorkers;
for outerLoop = 1:1000
    % Simple time-based schedule for how many workers to use
    currentHour = hour(datetime);
    if currentHour < 8 || currentHour >= 18
        % Out of work hours, use all workers
        M = nw;
    else
        % Work hours, don't use all workers
        M = nw - 1;
    end
    parfor (innerLoop = 1:1000, M)
        ... % do stuff
    end
end

[langrg]

Hi Edric,

Thanks for your answer. I understand your solution, but that's not what I'm looking for.

In fact, while I said "a very long simulation wih parfor loop inside", I mean that the code inside the parfor loop, itself, takes long time. My code is mainly based on one parfor loop (not a lot of code before the loop, and not a lot of code after the loop). I hoped that there were a solution to pause code, enter a command line (or do an action in Matlab interface) and resume code, so that the current parpool could take the modification into account immediatly (and if you decrease number of worker, take it into account as soon as any worker would have finish is work).

[Edric Ellis]

Unfortunately, I think my code example is the best that can be achieved for now.

Your other option is to use OS facilities to set the process priority of the workers to be extremely low, and simply use all the workers all the time.