how to ask for a pool size when using parpool with threads

Dear all,

using R2020b, looks like the flag "threads" when using a parpool excludes providing a pool size... is there a reason for that? How can provide an explicite number of workers?

thanks,

Daniel

Answer 1

[Raymond Norris]

There isn't from within parpool. When you create a Threads pool, the size of the pool is identical to maxNumCompThreads, which by default is the number of physical cores on the machine.

One workaround is as follows. Assume you have 4 cores and you want a thread pool of 2.

old_thread_count = maxNumCompThreads(2);
p = parpool("threads");

Keep in mind, adjusting maxNumCompThreads could have an impact on other MATLAB functions (though perhaps not since you're now running a pool), so I'd suggest setting it back once you're done with the pool.

p.delete
maxNumCompThreads(old_thread_count);

Is there a particular reason you want to adjust the size?

[Arabarra]

Thanks! Well, I'll have to live with it. My fear is that the system does not use all available computation resources, and gets stuck at 12 (a bug I had some time ago in a machine with 26 cores)

[Raymond Norris]

A Threaded pool should abide by maxNumCompThreads. Here's an example where I've requested Slurm for a single core with 18 cpus-per-task. MATLAB sees 18 cores and will start a threaded pool of 18.

[Arabarra]

also, sometimes I need to adjust the size for different operations. I can use all my 28threads for a CPU-based job, but immediately after that I need to use a smaller pool to control my 8 GPUS.

[Rick Amos]

As an update to Raymond's answer, we added this capability to thread-based pools in R2022b. For R2022b or later, you can control thread-based pool size directly via parpool:

parpool("threads", 2);