How to work with nested containers.Map()s inside a parfor loop

9 visualizaciones (últimos 30 días)
Guy Berreby
Guy Berreby el 18 de Abr. de 2022
Editada: Shlomo Geva el 27 de Mayo de 2023
Hi, I am currently having a problem related to using nested containers.Map()s inside parfor loops. The data structure I am working with the following:
I have an outer containers.Map(), whose keys are strings and whose values are other containers.Map()s. The inner containers.Map()s have keys which are strings (the same keys as the outer containers), and values which are arrays.
The purpose of this structure is to store arrays which will be graphed later on. The values inside these arrays are going to be calculated using a parfor loop, which is where I am running into a problem.
This is how I am trying to update the array values. Note, that at the end I want for inner_array_2 to have 2312312 in every entry of the array
outer_container = containers.Map();
outer_container("1") = containers.Map();
inner_container_1 = outer_container("1");
inner_container_1("2") = zeros([1 12]);
parfor x = 1:12
inner_container_2 = outer_container("1");
inner_array_1 = inner_container_2("2");
inner_array_1(x) = 2312312;
inner_container_2("2") = inner_array_1;
outer_container("1") = inner_container_2;
end
inner_container_3 = outer_container("1");
inner_array_2 = inner_container_3("2");
disp(inner_array_2)
However, when I do this, it only outputs an array of zeros. This is in contrast to when I use this exact same code, but with a for loop instead of a parfor loop like so:
outer_container = containers.Map();
outer_container("1") = containers.Map();
inner_container_1 = outer_container("1");
inner_container_1("2") = zeros([1 12]);
for x = 1:12
inner_container_2 = outer_container("1");
inner_array_1 = inner_container_2("2");
inner_array_1(x) = 2312312;
inner_container_2("2") = inner_array_1;
outer_container("1") = inner_container_2;
end
inner_container_3 = outer_container("1");
inner_array_2 = inner_container_3("2");
disp(inner_array_2)
This code outputs an array with 2312312 everywhere, as I wanted. Is there any way for me to modify the parfor loop code to behave the way it would in this for loop?
Note that this is just a toy example, in the actual code I am writing I am also looping over the keys of the inner containers.Map(), which is why they are nested.

Respuestas (1)

Edric Ellis
Edric Ellis el 21 de Abr. de 2022
containers.Map instances are MATLAB handle objects. These get copied to the workers, and so when you try to modify them on the worker, you're operating on a separate copy. There's more here in the doc. The short answer unfortunately is: don't try to modify containers.Map instances inside parfor. Send your result back in a cell array or something, and modify the containers.Map instance back on your desktop MATLAB.
  1 comentario
Shlomo Geva
Shlomo Geva el 27 de Mayo de 2023
Editada: Shlomo Geva el 27 de Mayo de 2023
I have another problem.
Here is the code of a script that calls a function inside the loop. The function is passed a Map object (created previously). I put in the comment the details of the Map. The function actually does nothing. I run this code with a for loop and it takes "Elapsed time is 0.000417 seconds."
When I run it with a parfor loop it goes into a black hole never to be seen again. I have to kill it. In the Activity Monitor (on a Mac) I can see all 8 processors are 100% busy.
I assume it may have to do with the distribution of a large map, but I tried it also on a server with 1.5TB of memory and it hangs up there too.
Any suggestions or explanantions will be greatly appreciated.
Perhaps there is a simple way to distribute the map as a shared variable (it is read-only inside the loop)
Here is the script:
% >> myMap
%
% myMap =
%
% Map with properties:
%
% Count: 10344256
% KeyType: char
% ValueType: any
%
% >>
tic
for i=1:8 % if this is cahanged to parfor it hangs
[x,y] = foobar(myMap,i);
end
toc;
function [x,y] = foobar(myMap,i)
x=[];
y=[];
end

Iniciar sesión para comentar.

Categorías

Más información sobre MATLAB Parallel Server en Help Center y File Exchange.

Productos


Versión

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by