I have a file 1x200 struct how can I distribute it to the worker (32 worker)
1 visualización (últimos 30 días)
Mostrar comentarios más antiguos
Student for ever
el 19 de En. de 2018
Comentada: Ahmed Abdalazeez
el 22 de En. de 2018
Dear all, I have a file 1x200 struct, I would like to send it to the cluster (parallel computation), how can I distribute it to the worker (32 worker). I can make it one by one, but it is really take very long time.
Thanks in advance.
Best regards, ahmed
2 comentarios
Walter Roberson
el 19 de En. de 2018
Is it a file (text or binary) or is it just a struct()? Do the workers need access to all of it or only to the portion they are acting on?
Respuesta aceptada
Walter Roberson
el 19 de En. de 2018
Just create the struct normally before the parfor() loop, and inside the parfor() loop, index the struct by the parfor variable. MATLAB will automatically distribute only the needed portion to each worker.
3 comentarios
Walter Roberson
el 22 de En. de 2018
data = struct('test', num2cell(rand(10,200),1) );
Now data is a 1 x 200 struct with one field named test, with each entry data(K).test being a 10 x 1 vector. The details of the above are not important: I just needed to create a 1 x 200 struct to show parfor.
Now:
parpool(32)
sums = zeros(1,length(data));
parfor K = 1 : length(data)
sums(K) = sum( data(K).test );
end
In this example, parfor does not need to copy all of data to every worker: it only copies the portion needed by the worker. Notice that no special code was needed for this: parfor handles it automatically.
This is an example of a "sliced variable", https://www.mathworks.com/help/distcomp/sliced-variable.html
If the use of the elements of the data structure was more complicated, then parfor could decide that the easiest thing to do is to copy all of data to every worker. In that situation, data would be referred to as a "broadcast variable", https://www.mathworks.com/help/distcomp/broadcast-variable.html
Más respuestas (0)
Ver también
Categorías
Más información sobre Parallel for-Loops (parfor) en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!