Slow access of large composite element
1 visualización (últimos 30 días)
Mostrar comentarios más antiguos
rp
el 10 de Abr. de 2017
Comentada: Walter Roberson
el 9 de Feb. de 2018
I am using spmd to run some parallel algorithms which periodically communicate. At the end, each process has a structure which I would like to pull back to the client for further analysis. Unfortunately, the size of the structure is large (but I don't think they are excessive), and trying to pull just one of the composite elements leads to a timeout of the cluster (30 min) and therefore fails. Is there any way to have the client as one of the spmd processes, or a more efficient way to transfer data from one or more of the workers back to the client in a more efficient way?
EDIT:
A clunky workaround is to have each process save their workspace to a directory. This seems to work pretty quickly, but it's obnoxious to have to import all the workspaces to do the diagnostics. I would appreciate more elegant solutions.
5 comentarios
Walter Roberson
el 9 de Feb. de 2018
Looking more closely, it is for "communicating jobs", of which spmd is one kind.
Respuesta aceptada
Más respuestas (0)
Ver también
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!