How do I get parfor to downgrade gracefully to for when no parallel seat is open?

My biggest client has many MatLab seats but only a few parallel toolbox seats. Often, a user is able to run MatLab but not able to run in parallel. Is there a way to configure MatLab to automatically downgrade a parfor to a for loop when that happens? Currently it crashes with "Unable to checkout a license for the Parallel Computing Toolbox".

Please don't bother suggesting that I change all the code everywhere to check for the pool then run either a for or a parfor. Yeah. I know how to do that.

I'm looking for "a more elegant weapon for a more civilized age".

This is R2018b.

Answer 1

[Edric Ellis]

parfor already has the innate ability to run without Parallel Computing Toolbox being present, and it should automatically handle the case where a PCT license is not available. Presuming the user settings are in the default state (i.e. a parallel pool is automatically created when encountering a parfor), then if the license checkout fails, parfor should already degrade gracefully. Here's what I tried:

>> parfor i = 1, end % this works
>> parpool('local', 2) % this fails because I don't have a license
Starting parallel pool (parpool) using the 'local' profile ...
License checkout failed.
License Manager Error -5
Cannot find a license for Distrib_Computing_Toolbox.

Troubleshoot this issue by visiting: 
https://www.mathworks.com/support/lme/R2018b/5

Diagnostic Information:
Feature: Distrib_Computing_Toolbox 
License path: /home/eellis/.matlab/R2018b_licenses/license_uk-eellis-deb9-64_40447121_R2018b.lic:/local-ssd2/MATLAB/R2018b/licenses/license.dat:/local-ssd2/MATLAB/R2018b/licenses/*.lic 
Licensing error: -5,0.
Error using parpool (line 113)
Unable to checkout a license for the Parallel Computing Toolbox. 

So, I believe that if your code has only calls to parfor and not parpool, then it should work correctly. Please give more details if this isn't what you're seeing.

[Dr D]

Good info & a nice clue. This code base is quite large (nearly 100,000 lines of code). Some of it is linear (1 cpu), some uses parfor, and other bits use parfeval and fetch... For the latter bits of code, I've already implemented a generic check function which checks gcp() and if it cannot get a valid pool returns handles to fake versions of parfeval, fetchNext, and fetchOutputs so that the calling code can continue oblivious to the fact that there is no actual parallel pool. It uses function handles which it believes are parfeval, etc... but runs on a single cpu instead of parallel.

If I understand what you're saying, then if any code like that runs at some point earlier in the process, the fact that it has tried to get a pool means that any later parfor is going to fail with an error (like I get now) instead of gracefully degrading like it would normally if I hadn't used gcp().

Surely there is some setting I can reset after a failed call to gcp() so that any future parfor's don't fail.

Note that it's even a little more complicated than this. Some of these codes can take anywhere from a few minutes to an entire day to run depending on the job to be performed. Same code; wildly varying runtime (this is normal & expected). So it's possible that part of the code gets a parallel seat, then some linear portion runs for so long that the parallel pool shuts down, then a later part of the code tries to use parfor and boom, the error happens.

So I have two different cases I need to account for: when I get a pool but it times out unbeknownst to me and when I don't get a pool. How do I reset MatLab so that those future parfor attempts degrade gracefully?