While cycling home I thought of a possible solution: each worker just writes their own diary to a random filename report and passes the filename and the job name back to the dataQueue. The dataQueue can then use this to append the report content to the relevant overall log, deleting the temporary files as it goes.
Logging parellel pool worker diaries to single file
33 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Ben Wetherill
el 31 de En. de 2018
Editada: Ben Wetherill
el 2 de Feb. de 2018
I'm converting a procedural script into a parallel script. The existing script dumps the command window output using Diary. So each job's output was nicely lumped together. Now though, as I have tasks running in parallel my log file is very disjointed. Not ideal.
Is there a way I can collect each worker's diary and send it to a dataQueue for storage in a single file? In pseudo code it would be:
parfor ...
run task (including command window outputs)
send Diary content back to client dataPool for writing to log file
end
Is there some method within my parfor loop of saying 'send(q, thisWorker.Diary)'?
Respuesta aceptada
Edric Ellis
el 1 de Feb. de 2018
Instead of parfor, you could use parfeval to run stuff on the workers. This will require a bit of restructuring - you'll need to divide up your work into decent-sized chunks (this is something that parfor takes care of for you automatically). Then, you can use the Diary property of each parallel.Future instance.
For example
% request execution on a worker
f = parfeval(@() fprintf('worker diary text: %d\n', rand()), 0);
wait(f), f.Diary
1 comentario
Ben Wetherill
el 2 de Feb. de 2018
Editada: Ben Wetherill
el 2 de Feb. de 2018
Más respuestas (0)
Ver también
Categorías
Más información sobre Parallel for-Loops (parfor) en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!