R2024b parpool crashing when being activated with 24 workers.

!!! Update: These crashes seem to be happening quite randomly, regardless of the number of workers that are used.

Dear all,

Whenever i try to start a parpool with >20 workers on the processes profile, an error occurs and the parallel pool automatically gets shut down. I have tried validating the profile with the cluster profile manager, and using any value above 20 workers seems to be producing this error for some reason, despite my CPU having 24 cores. I've never experienced this problem on Matlab 2024a, and I have always been able to start parallel pools with up to 24 workers.

Is there a known fix for this? It has only been happening since updating to Matlab R2024b. My CPU is an Intel Core i9-14900KF.

Thanks in advance, I attached the error below if it can be useful, and a few snapshots of the cluster profile manager validations.

Command window output:

Starting parallel pool (parpool) using the 'Processes' profile ...

Error using parpool (line 133)
Parallel pool failed to start with the following error. For more detailed information,
validate the profile 'Processes' in the Cluster Profile Manager.

Error in parallel.internal.ui.PoolHelper.startPool (line 12)
            parpool();
            ^^^^^^^^^
Caused by:
    Error using parallel.internal.pool.AbstractInteractiveClient>@()checker.checkState()
    (line 121)
    The parallel pool job errored with the following message: MATLAB worker shut down
    unexpectedly with status 1 during task execution.
 
Parallel pool using the 'Processes' profile is shutting down.
This parallel pool has been shut down.

Caused by:
    The client lost connection to worker 2 (Task 2; Host: localhost), potentially due to
    network issues or errors during the interactive communicating job.
    

With 16 workers (same output when using 20):

With 24 workers:

[Piyush Kumar]

@Matteo D'Ambrosio, Are the error messages same each time?

[Matteo D'Ambrosio]

Thanks for the reply!

Yes the error messages are the same, the only difference is the number (ID) of the worker that fails.

[Chao Wang]

I faced the same problem and I have no idea why it happened.

Answer 1

[Sergio E. Obando]

While not exactly the same error, this post covers some good troubleshooting steps: Validation Fails

If you prefer or if those steps do not resolve your issue, I would highly recommend contacting Technical Support.

[Matteo D'Ambrosio]

Thanks for the reply Sergio,

I tried to follow the troubleshooting steps and the problem persists, I will try contacting Technical Support. It could be a problem with the new Matlab update, since I've never experienced similar problems before running the same parallel pool profile.

[Henrik Teneberg]

Did you manage to get an answer? I am running into the same issue after upgrading to 2024b (from 2022b). However, my error status from the worker is 252, not 1 as in your screenshot above.

[Matteo D'Ambrosio]

Hi Henrik, After some back and forth with the mathworks support, we found that there is a bug in the current version of Matlab Service Host, which from what i understand is what checks for the licensing on your account. The problem seems to be known to mathworks and hopefully it will be fixed in the upcoming weeks by the matlab team. for now, the only way to fix the problem is to talk to mathworks support, and they will send you a temporary patch to use. This is the only thing that has worked for me, but as i said, it is only a temporary solution.

[Sergio E. Obando]

Henrik, my understanding is that a fix has been rolled out for MATLAB Service Host (latest version is MSH 2024.10). If it hasn't auto-updated, please create a help ticket, so tech support can assist you with that process:

http://www.mathworks.com/support/servicerequests/create.html

[Henrik Teneberg]

Hi Sergio,

I have installed 2024b Update 1. Does that include the update to Matlab Service Host? The issue is unfortunately still there.

I should also mention that I run Matlab as an .exe under MCR. I have used Update 1 both when compiling and as MCR version.

[Sergio E. Obando]

Hi Henrik, if this is through MATLAB runtime I am unsure if the root cause is the same. Please reach out to support as they may need to troubleshoot this with you.

[Raffael Kozerski]

Same here:

running a simulation with more than 60 workers crashed with R2024b on several machines.

The same simulation runs fine with R2024a using 700 cores/Matlab workers.

No idea why R2024b crashed; also running SPMD validation test.

in the Job log there is only a "Matlab crashed on worker XXX" message - no other useful information.

Raffael-

[Sergio E. Obando]

Raffael, please reach out to technical support. They can help you debug this issue and see if the root cause is similar to the one from the original post.