MATLAB Answers

Multi-level parallelization

2 views (last 30 days)
Edward B
Edward B on 18 Jan 2021
Answered: Raymond Norris on 18 Jan 2021
I am running on a machine with 64 cores. My code can run in parallel efficiently only on about 12 workers. When I give it more than 12 workers, the code actually slows down.
However, I can sumultaneously run two instances of the same code (for parametric studues) by starting up two sessions of MATLAB and running one code on each session. Because the two codes do not interact and do not communicate with each other, they each seem to run independently at full speed without sacrificing parallel efficiency.
QUESTION: Can I do this from one session of MATLAB? Ideally I would want some kind of multi-level parallelization, which would look something like this proto-code:
pool1 = parpool("local1", 12);
pool2 = parpool("local2", 12);
result1 = runCodeInBackground(pool1, "myCode", parameterForParametricStudy(1));
result2 = runCodeInBackground(pool2, "myCode", parameterForParametricStudy(2));
I realize that the "batch" command is designed for this. However, batch involves a lot of overhead in starting up each batch job. I would like to set up two (or more) independent parallel pools at the beginning, and then use these existing pools interactively.

Accepted Answer

Raymond Norris
Raymond Norris on 18 Jan 2021
MATLAB does not support multiple parallel pools in the same session. With that said, as you've pointed out, batch will do what you want and has almost the same signature as runCodeInBackground.
result1 = runCodeInBackground(pool1, "myCode", parameterForParametricStudy(1));
versus
job1 = batch("myCode", nout, {parameterForParametricStudy(1)}, 'Pool',12);
Where nout is the number of output arguments of myCode you want returned.

More Answers (0)

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!