Write to single file from multiple clients in parfor loop

Hello

Im running a parfor loop to run a set of jobs and in each loop I would like to update a my job monitoring file. My job monitoring file is a text file and in each loop I will update it using fopen, fprintf, flclose. I am aware that multiple writes to the same file from different clients may result in a corrupt file quickly. Now I was wondering if there is a method to block other clients while the file is opened.

In 2012 there was a thread concerning the same issue. At this time there were no such options, I hope there has been an update: https://ch.mathworks.com/matlabcentral/answers/33127-whats-the-best-command-to-write-to-file-inside-parfor-loop

If there is no solution to this problem, what might be the best way to write i job monitoring file?

Kind regards

Ben

Answer 1

[Walter Roberson]

If you open a file with 'a+' permission then each individual fprintf or fwrite would be done atomically at the end of file. Or at least that is the expected behavior on osx and Linux; I do not know if MS Windows atomically has the same behavior.

To use this to write a bunch of data you should assemble the data as a byte array or character vector with embedded newlines and only fwrite it when it is all assembled.

You could also consider sending the data by tcp to a server that is doing the logging.

Depending on the nature of your log file it might be the case that you can take advantage of your system event logging facilities and log file viewer.

[Ben]

Dear Walter, thank you for your quick response! Indeed I have been thinking about this method (a+). However this will not work because I am trying to read the status of each job in each loop, so i can make a statement about the overall progress.

Do you think it might be possible to simply edit the file with regexprep and close it again? Or do you think this will also corrupt my file?

Your other proposition with tcp logging sounds to complicated for me. I'm a novice...

Thank you!