In parfor-loop, can I call a multi-threaded mex and get some speed-up?

7 views (last 30 days)
I learned the concept of multi-threaded mex from undocumentedmatlab. (It seems this website is unaccessible now ...)
I am wondering if I can call a multi-threaded mex in parfor-loop.
My current code looks like
parfor k=1:1e6
result(k) = mex_wrapper(data(k));
end
mex_wrapper.c looks like
double calculate()
{
int N=50;
for (i=0;i<N;i++)
{
//...
}
}
void mexFunction()
{
calculate();
}
The iterations inside calculate() are independent, so I want to change the sub-routine calculate() to support multi-thread.
Although I am running parfor in process-based-environments, I am not sure if multi-threaded mex would confict with parfor.
So can I use multi-threaded mex in parfor? And would I get some speed-up by doing so?

Accepted Answer

Edric Ellis
Edric Ellis on 4 Jan 2021
You should be able to run a multi-threaded MEX file correctly inside a parfor loop. However, you will be oversubscribing your machine. For example, if your machine has 6 cores, your parfor loop will run 6 copies of your MEX function simultaneously. If each of those uses 6 threads each, you will have 36 threads active on your machine. This should work, but it will probably be less efficient than having a single-threaded MEX function. (A multithreaded MEX function inside parfor can be more useful when you have a cluster of machines - there, you might run single worker process per machine, and have each machine run the multithreaded MEX function).
  3 Comments
Xingwang Yong
Xingwang Yong on 4 Jan 2021
Thanks, Edric, I got your points. I tried to use maxNumCompThreads() in parfor-loop, it returns nothing. So this is infeasible.

Sign in to comment.

More Answers (0)

Categories

Find more on Parallel for-Loops (parfor) in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by