How to reduce the usage of "broadcast variables" when using parfor
63 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Paula
el 10 de Ag. de 2022
Comentada: Paula
el 10 de Ag. de 2022
Hi, in a "parfor" loop, a same warning message is put on three variables which I named "el", "ds", and "vmin". The message is: "The entire array or structure 'el' (or 'ds', 'vmin') is a broadcast variable. This might result in unnecessary communication overhead."
This "parfor" loop requires four variables which were calculated before the loop: Variable "el" is 4-by-378 double, "NElem"=27, "ds" is 27-by-27 sparse double, and "vmin" is 1-by-378 double.
Part of this loop is as follows:
el_row1=el(1,:);
parfor e=1:NElem
k=el_row1-e==0;
ind=el(4,k);
ds_e=ds(:,ind);
vs_e=ds_e*vmin(k)';
...
end
Is there a way to avoid this "unnecessary communication overhead", so that this parfor loop can run faster please? I read on the various types of variables in parfor-loops in Matlab documentation, I'm guessing the reason these three variables get the warning message is the form of indexing: "k" or "ind" is not a loop variable, so "el", "ds" or "vmin" cannot be sliced. But I don't know how to turn the variables "el", "ds", and "vmin" into slice variables. Any help is appreciated.
0 comentarios
Respuesta aceptada
Walter Roberson
el 10 de Ag. de 2022
"Is there a way to avoid this "unnecessary communication overhead", so that this parfor loop can run faster please?"
No. You are accessing the values in random order.
k=el_row1-e==0;
ind=el(4,k);
You can build el4=el(4,:); and index that at el_row1==e so that el as a whole does not need to be sent.
You could findgroups el_row1 and splitapply or similar ahead of time to create cell ind{e} so that the index information could be sliced.
But ds and vmin are going to have to be sliced. Unless, that is, it turns out that the information needed for any given e turns out to be consecutive and can be split ahead of time.
Your arrays are not large at all. I would be concerned about whether they are doing enough work to make it worth using parfor.
6 comentarios
Bruno Luong
el 10 de Ag. de 2022
el_row1=[1 2 3 2 1 1];
[~,~,J] = unique(el_row1);
i = accumarray(J,(1:length(J))',[],@(x){x});
i{:}
Más respuestas (1)
Bruno Luong
el 10 de Ag. de 2022
Editada: Bruno Luong
el 10 de Ag. de 2022
You can use parallel constant to convert el, el_row1, ds, vmin to constant and reduce the data transfert, if they are not modified (the snipet of code you showed they are not).
0 comentarios
Ver también
Categorías
Más información sobre Parallel for-Loops (parfor) en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!