Is there an alternative to parfor?

Hello,

I run a simulation with several settings. Thus, it is possible to run more than one simulation at one time.

I get the settings from several loops, but before each simulation I check a MySQL database to find out if I run it before. Example:

for S1 = setting1
   for S2 = setting2
      parfor S3 = setting3
         % if S1, S2, S3 is not available in the DB, run it, otherwise continue
      end
   end
end

It is possible to open Matlab by own 4 times and run the script instead of the parfor with a normal for. In this case, the script is more stable and if there is an error, just one worker stops and not all. This option is way better for the simulation but in the beginning it is not nice to start all manually.

Is there an option to program it this way, or is there a alternative to parfor? Debuging is with parfor also not easy because I don't get the line of the error.

[Kirby Fears]

All parallel computing functionality:

http://www.mathworks.com/help/distcomp/functionlist.html

asynchronus execution:

http://www.mathworks.com/help/distcomp/parfeval.html http://www.mathworks.com/help/distcomp/fetchoutputs_fevalfuture.html

spmd:

http://www.mathworks.com/help/distcomp/spmd.html

Answer 1

[Mohammad Abouali]

Couple of notes here:

NOTE 1 instead of only parallelizing the inner for-loop you can parallelize all of them as follow:

nSetting1=numel(setting1);
nSetting2=numel(setting2);
nSetting3=numel(setting3);

parfor idx=1:(nSetting1*nSetting2*nSetting3)
  [idx3,idx2,idx1]=ind2sub([nSetting3,nSetting2,nSetting1],idx);
  S1=setting1(idx1);
  S2=setting2(idx2);
  S3=setting3(idx3);
  % if S1, S2, S3 is not available in the DB, run it, otherwise continue
end

The above code parallelize all the loops for you.

NOTE 2 The above code has the same issue as before, meaning if one worker fails all of them stop You can use try/catch block to overcome this as follow:

nSetting1=numel(setting1);
nSetting2=numel(setting2);
nSetting3=numel(setting3);

parfor idx=1:(nSetting1*nSetting2*nSetting3)
  [idx3,idx2,idx1]=ind2sub([nSetting3,nSetting2,nSetting1],idx);
  S1=setting1(idx1);
  S2=setting2(idx2);
  S3=setting3(idx3);

    try
      % if S1, S2, S3 is not available in the DB, run it, otherwise continue
    catch 
      warning('Some error happened. skipping %d,%d,%d',idx1,idx2,idx3);
    end
  end

The above code generates a warning whenever an error occurs; but it won't stop from working and it continues. Of course you can change the catch block appropriately to do whatever you want it to do.

[Walter Roberson]

I suspect you mean

S1=setting1(idx1);
S2=setting2(idx2);
S3=setting3(idx3);

[Mohammad Abouali]

Yes. Copy/Paste effect :D Thank you for pointing this out