Why can't I open a parpool('Processes') of over 64 workers on my high core Windows machine?

I have a Windows machine with over 64 cores but the local or 'Processes' cluster profile says I can only have up to 64 workers, why won't it let me use more workers?

Answer 1

[MathWorks Support Team]

In order to use more than 64 workers on Windows, the number of workers in the local or 'Processes' cluster profile needs to be manually set. This can be achieved by editing the profile in the Create and Manager Clusters GUI or by using code similar to:

c = parcluster('Processes')
c.NumWorkers = N; % where N is the desired number of workers you want to run on this machine
saveAsProfile(c,'Processes');

The default number of workers in the 'Processes' cluster profile on Windows machines with multiple processor groups is capped at the number of processors in a single processor group due to a limitation in how the number of cores is detected.

Increasing the NumWorkers of the 'Processes' profile manually works around this limitation. 

Setting the number of parallel workers too high may impact your performance and machine's stability. Always ensure the machine has enough memory to run the number of MATLAB workers and code you wish to execute. We recommend that if you choose to exceed the number of physical cores on your system, you increase the NumWorkers setting gradually and measure your application's performance, memory usage and stability.