cplex object params not transferred to parfor loop
1 visualización (últimos 30 días)
Mostrar comentarios más antiguos
Katarzyna Furmanska
el 30 de Jul. de 2020
Comentada: Katarzyna Furmanska
el 31 de Jul. de 2020
I am trying to solve the batch of CPLEX problems in parallel using parfor loop. I'm not very fluent in Matlab data types, but CPLEX problems seem to be stored as handle class objects with two structs: Model and Param.
Now, say I have an array with the paths to my .lp problems and I am reading them into cell array:
subp_array = {}; % array to store problems
nSubp = 1; % for indexing problems
% read models into subp_array
for file = filepaths
prob = Cplex('uc');
prob.readModel(file{1});
prob.Param.threads.Cur = 1;
subp_array{nSubp} = prob;
nSubp = nSubp+1;
end
The param threads.Cur is particularly important, because it indicates how many threads can be utilized for a single CPLEX problem (and with anything other than 1, it does not bring any time benefit). Now when I run the loop to solve the problems and I print the param value:
parfor subp_index = 1:length(subp_array)
prob = subp_array{subp_index};
disp(prob.Param.threads.Cur);
% solve the problem
prob.solve();
end
They all happen to be 0 (default value). If I run the for loop instead, it keeps the value of 1.
What happens here? I'm actually trying to catch more differences in my original code as parfor returns completely different results than for - but this one could be an indication for me where the differences come from.
Thanks
0 comentarios
Respuesta aceptada
Edric Ellis
el 31 de Jul. de 2020
The most likely cause of the problem here is that the output of Cplex cannot be saved to a file and then loaded again. Transfer of data to the workers in a parallel pool is equivalent to calling save and then load (but the data isn't stored on disk at any point), and some data types in MATLAB do not correctly support save and load.
If you determine that the Cplex data types cannot be correctly saved and loaded, then the simplest workaround is to build the Cplex data directly on the worker, operate on it there, and return to the client only results (which are hopefully of a simple data type that can be correctly transferred). So, you might do something like this:
parfor idx = 1:numel(filepaths)
prob = Cplex('uc');
prob.readModel(filepaths{idx});
prob.Param.threads.Cur = 1;
prob.solve();
out{idx} = doSomething(prob); % Not sure what you need here.
end
This code assumes that the workers can see the same files as the client. This will be the case if you are using the 'local' cluster type (i.e. the default). If the workers can't see the same files, then you might need to use addAttachedFiles to make the files available on the workers.
3 comentarios
Edric Ellis
el 31 de Jul. de 2020
One thing that you need to bear in mind is that when you use a handle object inside a parfor loop, each worker gets a copy of that object. Updates that you make on the worker are not automatically reflected back at the client. So perhaps that's what's going on. I'd still be looking to try and build the handle objects directly on the workers inside the parfor loop, and sending back to the client only the results - presuming they can safely be copied. More here in the doc.
Más respuestas (0)
Ver también
Categorías
Más información sobre Parallel for-Loops (parfor) en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!