Why using rtwbuild in parallel is not recommended?

In the documentation for rtwbuild, there is a note: 'Do not use rtwbuild, rtwrebuild, or slbuild commands with parallel language features'. What is the reason for that? I wanted to speedup code generation for several models by using parallel toolbox, and it works. However the rtwbuild is slower on the workers than on the client for the same models, thus the speedup is lower than I expected. Could anyone explain/suggest how rtwbuild works on the client and on workers?

Answer 1

[Sharan Paramasivam Murugesan]

Model build generates several artifacts (e.g., several .mat files) for a model which are used in the code generation process. So if you are using one of these commands to build a single model multiple times via multiple threads, then these threads will run into resource contention for these artifacts. But from your explanation I assume you are building multiple models which are independent of each other. In this case, you can use rtwbuild with parallel computing features.

Now, there could be multiple factors affecting the speed of the build in workers. Are you running the workers locally or in a remote machine? - If they are running locally in the machine same as the client, then the increased overload on the machine's memory and CPU load could be slowing down the build. - If they are running remotely, then the remote machine's ability (memory and CPU load) to handle memory (or any resource) contention by parallel processes efficiently matters.

Having said all these, the one important information that you should consider is "How slower are these build?". So before diagnosing the effect of all these factors, please note that, in programming world, parallel computation is not always guaranteed to be faster than sequential computations. There would be scenarios where the parallel computations could be a bit slower than sequential.

In general, rtwbuild is designed for single-threaded application and hence there is no difference in the way it works on client and on workers.

Lastly, if you are using model references in your model, we do support parallel build of these references. For more information on this : https://www.mathworks.com/help/rtw/ug/reduce-build-time-for-referenced-models.html

[Bibigul Shektybayeva]

Thank you for your answer. Yes, I am building multiple models independent of each other. I am running the workers locally on the same machine as client.

After timing the code generation of each model sequentially and in parallel, I realized that the first call to the rtwbuild is slow, while the consequent calls are much faster. So, for example if I have 8 models, sequentially the first call to the rtwbuild is around 20 s, and the rest 7 calls take around 5-7 s -> the total time is around 60s. Now, in parallel on 4 cores, I have 4 parallel calls to rtwbuild that take around 20 s, and 4 consequent calls of 5-7 s. Additionally the time is needed for parpool command, which takes around 20s as well. The total time for the parallel execution is the around 50 s. Could you please suggest where can I find an explanation for such behavior of rtwbuild function?