parpool error in matlab docker image

Hi,

I am using the matlab docker image with Parallel Computing Toolbox. I got the following error when running

p = parpool(4);

The full message is attached below: (No other matlab process is running)

>> p = parpool(4);
Starting parallel pool (parpool) using the 'Processes' profile ...

Warning: Cannot cancel or destroy a job that was not created by this Local cluster. 
> In parallel.cluster.Local/cancelOrDestroyJob (line 34)
In parallel.cluster/Local/hCancelJob (line 299)
In parallel.internal.cluster.CJSJobMethods.cancelOneJob (line 51)
In parallel.job.CJSConcurrentJob>@(job,ce)CJSJobMethods.cancelOneJob(job.Parent,job,job.Support,ce) (line 57)
In parallel.job/CJSConcurrentJob/cancelJob (line 57)
In parallel.Job>iCancelJobs (line 1643)
In parallel.Job>@(jobs)iCancelJobs(jobs,exceptionBuilder) (line 1485)
In parallel.internal.cluster.hetfun (line 57)
In parallel/Job/cancel (line 1485)
In parallel/Cluster/hDeleteOneJob (line 1015)
In parallel.internal.pool.AbstractInteractiveClient>iDeleteJobs (line 491)
In parallel.internal.pool/AbstractInteractiveClient/pStopWorkersAndDisconnect (line 275)
In parallel.internal.pool/AbstractInteractiveClient/stopLabsAndDisconnect (line 188)
In parallel.internal.pool.AbstractClusterPool>iStopPoolResourcesAndNotifySessionAborted (line 886)
In parallel.internal.pool.AbstractClusterPool>@()iStopPoolResourcesAndNotifySessionAborted(client,sessionInfo) (line 624)
In parallel.internal.general/DisarmableOncleanup/delete (line 25)
In parallel.internal.pool.AbstractClusterPool.hBuildPool (line 622)
In parallel.internal.pool.doParpool (line 22)
In parpool (line 127) 
Warning: Cannot cancel or destroy a job that was not created by this Local cluster. 
> In parallel.cluster.Local/cancelOrDestroyJob (line 34)
In parallel.cluster/Local/hDestroyJob (line 303)
In parallel.internal.cluster.CJSJobMethods.destroyOneJob (line 71)
In parallel.job.CJSConcurrentJob>@(job)CJSJobMethods.destroyOneJob(job.Parent,job,job.Support,job.SupportID) (line 52)
In parallel.job/CJSConcurrentJob/destroyJob (line 52)
In parallel.Job>iDeleteJobs (line 1634)
In parallel.internal.cluster.hetfun (line 57)
In parallel/Job/delete (line 1429)
In parallel/Cluster/hDeleteOneJob (line 1031)
In parallel.internal.pool.AbstractInteractiveClient>iDeleteJobs (line 491)
In parallel.internal.pool/AbstractInteractiveClient/pStopWorkersAndDisconnect (line 275)
In parallel.internal.pool/AbstractInteractiveClient/stopLabsAndDisconnect (line 188)
In parallel.internal.pool.AbstractClusterPool>iStopPoolResourcesAndNotifySessionAborted (line 886)
In parallel.internal.pool.AbstractClusterPool>@()iStopPoolResourcesAndNotifySessionAborted(client,sessionInfo) (line 624)
In parallel.internal.general/DisarmableOncleanup/delete (line 25)
In parallel.internal.pool.AbstractClusterPool.hBuildPool (line 622)
In parallel.internal.pool.doParpool (line 22)
In parpool (line 127) 
Mismatch between number of environment names and values

I really appreciate any insights on the causes of this error message. Thanks in advance!

Answer 1

[Fushimi Tatsuki]

Hi guys,

If this is still relevant, but I had same issue and found the solution. I experienced this issue both in Amazon AWS, and Azure version. Hope this helps everybody!

parallel.internal.mwa.ensureLoggedIn()

https://docs.nuvolos.cloud/user-guides/application-specific-guides/matlab

Answer 2

[Harsha]

There are several issues that can stip parapool from starting. You can try below steps to resolve your issue.

Disable local mpiexec

If you are using R2010a or newer, you may experience issues with the new local mpiexec implementation. In that case, try the following command to disable this feature:

 distcomp.feature( 'LocalUseMpiexec', false )

This should allow the local scheduler to create and process parallel jobs and parpool.

Clear the local scheduler data folder

The error you are seeing might be the result of bad local scheduler data. In that case, the local scheduler data can be removed. To do so:

1. In MATLAB, run the command "prefdir" to find your preferences folder. Ex:

 >>prefdir
 ans =
 C:\Users\Administrator\AppData\Roaming\MathWorks\MATLAB\R2009b

This will output your preferences folder. The local scheduler data folder is one level up from the preferences in a folder called "local_scheduler_data" or "local_cluster_jobs". For example:

C:\Users\Administrator\AppData\Roaming\MathWorks\MATLAB\local_scheduler_data

2. Close MATLAB

3. Rename or delete the local_scheduler_data or local_cluster_jobs

4. Restart MATLAB

You can now try to run parpool to see if the data folder was the problem.