This behavior is expected and is due to how MATLABPOOL OPEN command works. When MATLABPOOL OPEN is issued, the command starts a specified number of headless MATLAB workers. Each of these workers executes the startup.m file, and awaits user input. Because the headless workers do not have desktops, user is not aware that they are requesting input and thus it appears that the MATLABPOOL OPEN hangs the system.
As a workaround either remove the user input commands from the startup.m file, or choose a default setting that is executed when the script is executed on a headless node, using the JAVACHK('DESKTOP') command as shown in the example below:
disp('head less mode');