Is there a way to change number of worker in a created parpool?

35 views (last 30 days)
Hi,
I have launch a very long simulation wih parfor loop inside. To keep working on my PC, I let one core free. Now that I'm leaving my office, I would like to add a worker on my parallel pool so that all the cores will be used during the night.
Does anyone know if there is a way to change the number of worker in a running parrallel pool?
Can I pause my simulation and enter a command line to change it?
Thanks!
GL.

Answers (1)

Edric Ellis
Edric Ellis on 19 Dec 2018
Unfortunately, there is currently no way to increase the number of workers in a parallel pool. One approach that might possibly work for you is to do the following:
  1. Launch the pool using the full number of workers
  2. Divide your code into multiple parfor loops
  3. Have each parfor loop specify a number of workers to use using the "M" parameter.
For example, you might do something a bit like this:
p = parpool('local');
nw = p.NumWorkers;
for outerLoop = 1:1000
% Simple time-based schedule for how many workers to use
currentHour = hour(datetime);
if currentHour < 8 || currentHour >= 18
% Out of work hours, use all workers
M = nw;
else
% Work hours, don't use all workers
M = nw - 1;
end
parfor (innerLoop = 1:1000, M)
... % do stuff
end
end
  2 Comments
langrg
langrg on 19 Dec 2018
Hi Edric,
Thanks for your answer. I understand your solution, but that's not what I'm looking for.
In fact, while I said "a very long simulation wih parfor loop inside", I mean that the code inside the parfor loop, itself, takes long time. My code is mainly based on one parfor loop (not a lot of code before the loop, and not a lot of code after the loop). I hoped that there were a solution to pause code, enter a command line (or do an action in Matlab interface) and resume code, so that the current parpool could take the modification into account immediatly (and if you decrease number of worker, take it into account as soon as any worker would have finish is work).
Edric Ellis
Edric Ellis on 19 Dec 2018
Unfortunately, I think my code example is the best that can be achieved for now.
Your other option is to use OS facilities to set the process priority of the workers to be extremely low, and simply use all the workers all the time.

Sign in to comment.

Categories

Find more on Parallel Computing Fundamentals in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!