What happens if I create a parpool with more workers than the number of cores the OS has assigned me?

I requested a machine from our internal cluster requesting for a interactive session on a machine with 64 cores. After my job got processed, I received the requested session. I then opened a VNC connection to this session through which I could then start MATLAB. 

Problem:

When I execute the following command on my MATLAB Command Window, I see that MATLAB is restricted to only use 16 cores instead of the 64 I expected:

maxNumCompThreads()

ans = 16

Questions:

1. Is there a way to force MATLAB to use all the available cores on the machine?
2. What happens if I call parpool('Processes',64) inside this MATLAB? Does the restriction apply to those workers as well?

Answer 1

[MathWorks Support Team]

1. The given output indicates that the OS or the scheduling software is restricting MATLAB from using all available cores on the machine. MATLAB cannot control this behavior. However, you might be able to remove any restrictions by updating your environment (docker/ cluster/ scheduler) settings that are restricting the number of cores allocated to a process.
2. If the 'Processes' profile was updated to use NumWorkers = 64 on this machine and parpool was then called Parallel Computing Toolbox would start 64 workers but they would be restricted to just the resources the OS has assigned. This would likely lead to resource contention. *

*One exception to this situation is on high core count Windows machines with > 64 cores where an OS restriction may indicate the presence of "processor groups". In this will not limit the ability to start process parpools larger than 64 workers and the NumWorkers in the 'Processes' profile can be safely adjusted using these steps.