When I use parfor in the program, the program seems to fall into an endless loop in the remoteparfor.m file, resulting in the program's delay in output.What's going on?

When I use parfor in the program, the program seems to fall into an endless loop in the remoteparfor.m file, resulting in the program's delay in output.What's going on?

Answer 1

[Edric Ellis]

The file "remoteparfor.m" is part of the implementation of a parfor loop. The main job of this piece of code is to wait for results to arrive from the workers - it's not doing any work itself. So while it may appear to be sitting in an infinite loop, it really is just waiting for whatever the workers are doing, and will complete as soon as they have completed.

[Walter Roberson]

@Edric Ellis

Is there a keep-alive mechanism? Just in case the workers die?

[Edric Ellis]

Yes, the desktop MATLAB client monitors the state of workers while waiting for results. If workers die, then appropriate action is taken. The nature of the action depends on the state of the pool, and the work that's going on at the time - in most practical cases, the work is transparently rescheduled on another worker, but other times the entire pool has to be terminated. Pool termination on worker crash is required when spmd has been used to ensure consistency; otherwise, generally the pool stays running. Of course, if what's causing the workers to crash is deterministic, then you can end up simply crashing the workers one at a time until there aren't any more left.

[Walter Roberson]

Thanks! I was just curious. I had noticed that in one of my sessions, MATLAB went into a spin loop when my MuPad kernel died, but in a different session MATLAB thought for a moment and then returned a clean "fail"... so MuPAD must have some state monitoring, even if imperfect.

[jessie sun]

How to prevent long waiting？

[Walter Roberson]

@jessie sun

Write code that does less work.

See also https://www.mathworks.com/matlabcentral/fileexchange/71436-parfor-progress-monitor-progress-bar-v4