saving within SPMD or parfor

Why matlab does not allow to save something to a file when within spmd or parfor?

let's say I have:

parfor i=1:n or spmd(nWorker)
    VarA=some calculations
    save(filename(i), 'VarA');
end

Matlab gives me an error "SAVE can not be called in an SPMD block. (the same for parfor except it says within parfor)

Does anyone knows why is that?

The funny thing is that if I create a new function like

function savetofile(data,fullfilename)
    save(fullfilename,'data');
end

and then use this function, everything works fine. Well, of course, this is not recognized by matlab to flag an error and stop running the code. But I am trying to show that it means that the operation is perfectly natural and ok.

I have so many files, that some operation needs to be done, which each file could be processed completely independently. The output needs to be stored also in separate files. So, I want multiple worker, each opening one file at a time, and then storing their results once they are done, and then moving to next available file to process. But MATLAB somehow thinks it is not allowed to use save command within spmd and parfor and I need to trick it by kinda changing the function name. Why is that matlab thinks save shouldn't be used?

edit: writetable works just fine. It seems somehow it is save command that is not sitting well.

Answer 1

[Edric Ellis]

The problem with the save command is that it needs to inspect the contents of the workspace in which it is running. That's not allowed inside parfor and spmd. There's more on this constraint in the documentation. As you note, the constraint is about the body of the block itself, not a fundamental constraint on workers calling save or load, and that's because it's an analysis constraint.

[Mohammad Abouali]

Thank you.

Answer 2

[Thomas Koelen]

Transparency is violated by the SAVE command because in general MATLAB cannot determine which variables from the workspace will be saved to a file.

The solution is to move the call to SAVE to a separate function and to call that function from inside the PARFOR loop. Pass any variables that are to be saved as arguments to the function. That way MATLAB can determine which ones will be saved and transparency is not violated.

[Mohammad Abouali]

Dankjewel

[Thomas Koelen]

Geen probleem ;)