Matlab cluster - validation - parpool stuck

14 visualizaciones (últimos 30 días)
Maria
Maria el 20 de Mayo de 2021
Comentada: Maria el 1 de Jun. de 2021
Hi,
I am testing for the first time Matlab Parallel. I have a cluster in my local network, with 8 cluster nodes and a max of 32 workers that can run.
I created the cluster profile and ran "validate." If I validate only 1 of the cluster nodes (4 workers), everything is fine, and the validation is done in a couple of minutes or less. However, if I try with 2 cluster nodes, the validation stops at "Parallel pool test (parpool)" and keeps on running indefinitely; I waited up to 1 hour. If I try to "Stop" the validation, nothing happens, and I have to kill Matlab from the task manager. I also tried running the 2 clusters with only 1 worker each, but the same thing happens. I am not getting any error, and I don't get any report either, because, as I said, the parpool testing just keeps on running. If I try cluster node 2 alone, everything works fine again. The problem arises when I start both cluster nodes 1 and 2 (and, of course, also when I start all 8 cluster nodes).
I am new to Matlab parallel, and I currently have no clue where to start looking. Could someone give me some hint on what the problem could be?

Respuesta aceptada

Raymond Norris
Raymond Norris el 20 de Mayo de 2021
A couple of questions
  • What scheduler is your cluster running? MJS? HPC Server? PBS/Slurm/etc?
  • Sounds like all the stages are passing, except the last one (parpool) IFF the workers span >1 node. Otherwise, the last stage with workers running on a single node pass. Do I have that right?
Without knowing more, I betting that you don't have password-less SSH between the compute nodes and that mpiexec is hung. Get onto a compute node and ssh to another. Are you prompted for your password? If so, you've reproduced the issue.
  4 comentarios
Maria
Maria el 24 de Mayo de 2021
Thank you, I will do so!
Maria
Maria el 1 de Jun. de 2021
I have disabled all firewall, but the problem persists. I am now waiting for the Technical Support... In the meantime, do you have any other idea what could it be?

Iniciar sesión para comentar.

Más respuestas (0)

Categorías

Más información sobre MATLAB Parallel Server en Help Center y File Exchange.

Etiquetas

Productos


Versión

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by