system command in parfor loop
10 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
I am invoking a C++ executable from the Matlab command prompt.
Interactively, it goes like this:
>> [status,cmdout] = system('multi.exe','-echo')
Input unique file identifier. Ex: 2~04-13-013939 - 2~11-30-091444
And this works fine.
However, I need to invoke the C++ executable multiple times. So I created a parfor loop, like so:
parpool
tic
parfor i=1:nfiles
[status,cmdout] = system('multi.exe','-echo')
str3{i,1}
end
where str3 is a series of inputs:
str3 =
14×1 cell array
{'1~11-24-054646'}
{'1~11-24-095017'}
{'1~11-29-060908'}
{'1~11-29-101743'}
{'1~11-30-091225'}
{'2~11-24-055554'}
{'2~11-24-100903'}
{'2~11-29-060541'}
{'2~11-29-101814'}
{'2~11-30-091444'}
{'3~11-24-055933'}
{'3~11-29-101847'}
{'4~11-24-060505'}
{'4~11-29-101935'}
This ran for over 54 hours, but the task never completed, and I eventually hit Ctrl-C.
Any thoughts/ feedback?
2 comentarios
Mario Malic
el 17 de En. de 2022
Check this out. Parfor worked fine for me using Sysem.Diagnostics.Process.
In order to relay the messages from workers in parpool, there are functions which I don't remember the name exactly. Check out this example https://www.mathworks.com/help/parallel-computing/parallel.pool.dataqueue.html
jessupj
el 17 de En. de 2022
Editada: jessupj
el 17 de En. de 2022
just to play devil's advocate, i've approached a similar problem in the past by calling parallel instances of matlab from the shell. it was by no means the most I/O-efficient method and wasted a lot of time opening and closing matlab & pooling. the individual system jobs were geophysical models that took O(days) so the data importing and pool opening overhead was comparatively insignificant.
Respuestas (1)
Walter Roberson
el 17 de En. de 2022
parpool
tic
parfor i=1:nfiles
cmd = sprintf('echo "%s" | multi.exe', str3{i,1});
[status,cmdout] = system(cmd, '-echo')
end
12 comentarios
Walter Roberson
el 26 de En. de 2022
parpool
tic
parfor i=1:nfiles
tname{i} = tempname();
[fid, msg] = fopen(tname{i});
if fid < 0
error('iteration %d failed to create temporary file "%s" because "%s"', i, tname{i}, msg);
end
fprintf(fid, '%s\n\n', str3{i,1});
fclose(fid)
cmd = sprintf('multi.exe < "%s"', tname{i});
[status,cmdout] = system(cmd, '-echo')
end
Ver también
Categorías
Más información sobre Parallel Computing Fundamentals en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!