Memory issues using parfeval

7 visualizaciones (últimos 30 días)
Andre Zeug
Andre Zeug el 22 de Abr. de 2020
Comentada: Andre Zeug el 1 de Mayo de 2020
Hi everyone!
Since years I am using the nice function export_fig to save figures, which I create with my scripts and functions. To speed this up I tried to do this in background using 'parfeval'. Unfortunately, for some reason workers pile up memory consumption which can be monitored e.g. with the ProcessExplorer.
function export_figP(fname, varargin)
% same input as export_fig, but first input must be the filename, last
% input must be the figure handle (could be improved)
tParFeval = tic;
tt = []; fig_1 = [];
set(varargin{end},'Visible','off')
myEvalStr = 'tt = parfeval(@export_fig, 0, fname';
for n=1:nargin-2
myEvalStr = sprintf('%s, ''%s''',myEvalStr, varargin{n});
end
myEvalStr = sprintf('%s, fig_1);',myEvalStr);
n = nargin-1;
eval(sprintf('fig_1 = varargin{%d};',n))
eval(myEvalStr)
afterEach(tt, @(~)evalAfterAll(tt,fig_1,fname,tParFeval), 0);
function evalAfterAll(tt,hh,saveStr,tParFeval)
if ~isempty(tt.Error)
fprintf(2,'!!! Error saving Figure %s !!! \n',saveStr)
else
fprintf(2,'Figure ''%s'' saved within %.3fs (total time %.3fs)\n',saveStr,(datenum(tt.FinishDateTime)-datenum(tt.CreateDateTime))*24*60*60, toc(tParFeval))
close(hh)
tt = [];
clear all
end
end
end
The function can be tested with this lines (requires export_fig)
%% Test export_figP
if ~isfolder(fullfile(pwd, 'figs')); mkdir('figs');end
tic
for n = 1:1000
% create simple figure
fig1 = figure('Color','w','Visible','on');
imagesc(rand(1000, 1000))
colorbar
% save figure in Background
saveStr = sprintf('%04d_TestFig.png',n);
export_figP(sprintf('figs/%s',saveStr),'-q100','-nocrop','-r300',fig1);
fprintf('saving of %s started (%.3f)\n', saveStr, toc)
end
What do I do wrong?
How to clear the workers memory after the function call (except shuting down parallel pool)?
Probably my function 'export_figP' does something substantially wrong.
  1 comentario
Andre Zeug
Andre Zeug el 1 de Mayo de 2020
Is there an option to clear workers memory when no further job is processed?
p = gcp('nocreate')
p =
ProcessPool with properties:
Connected: true
NumWorkers: 16
Cluster: local
AttachedFiles: {}
AutoAddClientPath: true
IdleTimeout: Inf (no automatic shut down)
SpmdEnabled: true
q = p.FevalQueue
q =
FevalQueue with properties:
Number Queued: 0
Number Running: 0
But memory consumtion is still big after several calls of parfeval:

Iniciar sesión para comentar.

Respuestas (0)

Categorías

Más información sobre Asynchronous Parallel Programming en Help Center y File Exchange.

Productos


Versión

R2020a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by