Problem using OPTIMOPTIONS with FMINCON for PARALLEL COMPUTING

Hi All, I am performing an optimization of parameters of a Simulink model using fmincon The options I am using are the followings

options = optimoptions('fmincon'); % Start with the default options
options = optimoptions(options,'Algorithm', 'interior-point');
options = optimoptions(options,'Display', 'iter-detailed');
options = optimoptions(options,'MaxIter', 1000);
options = optimoptions(options,'PlotFcns', { @optimplotfval });

So far so good, the optimization run and arrives to a minimum with no problems at all However, as the optimization takes few hours to perform I wish to speed up the process by using parallel computing (I have a 2 core processor computer)

Then, I add the following line to the options listed above

options = optimoptions('fmincon','UseParallel',true);

Before I run the optimization with the new line of code, I set the parallel computing on manually using the following command in Matlab Command Line

parpool

Matlab gives the following message

Starting parallel pool (parpool) using the 'local' profile ... connected to 2 workers.

ans = 

   Pool with properties: 

              Connected: true
             NumWorkers: 2
                Cluster: local
          AttachedFiles: {}
            IdleTimeout: 30 minutes (30 minutes remaining)
            SpmdEnabled: true

I then run the optimization but I get this error

Error using optimoptions (line 114)
Invalid value for OPTIONS parameter UseParallel:
 must be 'always' or 'never'.

Error in Damper_Optimization_v1 (line 134)
options = optimoptions('fmincon','UseParallel',true);

Please not the Damper_Optimization_v1 is the name of my script (where fmincon is used) and that line 134 is the line of

options = optimoptions('fmincon','UseParallel',true);

Does anybone know why I am getting this error?

I tried to use "always" instead of "true" but this did not work, I got the following error

Undefined function or variable 'always'.

Error in Damper_Optimization_v1 (line 151)
options = optimoptions('fmincon','UseParallel',always);

PS: I noted that when I add the line with the parallel computing option, the word "options" in the previous line is underlined in red and the warning message that I get for it is

The value assigned to variable 'options' might be unused

See below the rest of the code where I use fmincon

options = optimoptions('fmincon'); % Start with the default options options = optimoptions(options,'Algorithm', 'interior-point'); options = optimoptions(options,'Display', 'iter-detailed'); options = optimoptions(options,'MaxIter', 1000); options = optimoptions(options,'PlotFcns', { @optimplotfval }); %options = optimoptions('fmincon','UseParallel',true);

options = optimoptions('fmincon'); % Start with the default options options = optimoptions(options,'Algorithm', 'interior-point'); options = optimoptions(options,'Display', 'iter-detailed'); options = optimoptions(options,'MaxIter', 1000); options = optimoptions(options,'PlotFcns', { @optimplotfval }); %options = optimoptions('fmincon','UseParallel',true);

OPTIMIZATION

tic % Start stopwatch to measure elapsed time   

[Optimum_Damper,fun_value,exitflag,output_details] = fmincon(...
                @Objective,...                    % Cost Function
                Damper_Parameter_Initial_Guess,...% Initial Guess 
                A,B,...                           % Damper Parameters Constrains
                [],[],...                         % Damper Parameters Inequalities 
                Damper_Parameters_Lower_Bound,... % Damper Parameter Lower Bounds
                Damper_Parameters_Upper_Bound,... % Damper Parameter Lower Bounds
                [], ...                           % Non-linear conditions (NONE)
                options);                         % Options 

toc

Thanks in advance for your help

G

[Jill Reese]

When you say that "I tried to use "always" instead of "true" but this did not work.", what do you mean? Did you get a different runtime error? Or was it simply that your simulation did not speed up?

[Giuseppe Naselli]

Hi Jill,

I am sorry for not being very clear on that.

so I amended the question

Please see above

Thanks

Regards,

G

[SHAMSUL FAISAL]

Good day,

I am a beginner in Simulink Design Optimization toolbox. Currently I am estimating a system that has more than 20 parameters and it takes more than 2 hours to complete the estimation. Therefore, I decided to try utilising the Parallel Computing Toolbox.

I tried to practise the Parallel Computing Toolbox by modifying the existing example given by MATLAB, which is the 'Estimate Model Parameters and Initial States (Code)' example which can be referred in detail in the following link: The original code is as follow:

opt = sdo.OptimizeOptions;
opt.Method = 'lsqnonlin';

The reason why I found this thread is because initially I also made the same mistake like Giuseppe Naselli while using MATLAB R2013b, which is:

opt = sdo.OptimizeOptions;
opt.Method = 'lsqnonlin';
opt.UseParallel = true;

After reading this thread, I made the correction. The complete code is shown below:

opt = sdo.OptimizeOptions;
opt.Method = 'lsqnonlin';
opt.UseParallel              = 'always';
opt.OptimizedModel           = 'initialcondition';
opt.ParallelPathDependencies = sdo.getModelDependencies('initialcondition');
%parpool

Please take note that I don't use the parpool command because I commented it out. I don't use the parpool because so far I can't see what is the difference between using and not using it.

The source code can be executed without error. However, I noticed that the following message appear while running the 'pOpt = sdo.optimize(estFcn,p,opt)' command:

Failed to get RMI data for fl_lib/Electrical/Electrical Elements/Capacitor. Library "fl_lib" is not loaded.
Failed to get RMI data for elec_lib/Additional
Components/SPICE-Compatible
Components/Sources/DC Voltage Source. Library "elec_lib" is not loaded.
Failed to get RMI data for fl_lib/Electrical/Electrical Elements/Electrical Reference. Library "fl_lib" is not loaded.
Failed to get RMI data for nesl_utility/PS-Simulink
Converter. Library "nesl_utility" is not loaded.
Failed to get RMI data for fl_lib/Electrical/Electrical Elements/Resistor. Library "fl_lib" is not loaded.
Failed to get RMI data for nesl_utility/Solver
Configuration. Library "nesl_utility" is not loaded.
Failed to get RMI data for fl_lib/Electrical/Electrical Sensors/Voltage Sensor. Library "fl_lib" is not loaded.
Configuring parallel workers for optimization...

May I know what did I do wrong until this messages appear? Please feel free to inform me if you would like to see the full script and objective function.

Thank you.

[SHAMSUL FAISAL]

All right. I managed to get rid of those messages by deleting the 'opt.ParallelPathDependencies = sdo.getModelDependencies('initialcondition');' command. Therefore, my latest code is as follow:

opt = sdo.OptimizeOptions;
opt.Method = 'lsqnonlin';
opt.UseParallel              = 'always';
opt.OptimizedModel           = 'initialcondition';
%parpool

However, I still need some time to study what is the purpose of the 'parpool' command. I noticed that the simulation elapsed time is faster whenever I don't use the parpool command (90 seconds when not using vs 60 seconds when using).

Please advice if I miss any other setting that I should use for this Parrallel Computing Toolbox.

Thank you.

Answer 1

[Alan Weiss]

I think there might be two issues here.

1. The value true for UseParallel was added in R2014a. If you use an earlier version, set UseParallel to 'always'. Include the single quotes.
2. In your fmincon call, do you pass options? I mean, does your call look like this:

 [x,fval,exitflag] = fmincon(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,options)

Alan Weiss

MATLAB mathematical toolbox documentation

[Giuseppe Naselli]

Hi Alan,

thanks for your answer. To reply your questions

1. I have the R2013b version. So I changed my code

options = optimoptions('fmincon','UseParallel','always');

However I got the following error

Warning: The default trust-region-reflective algorithm does not solve problems with the
constraints you have specified. FMINCON will use the active-set algorithm instead. For
information on applicable algorithms, see Choosing the Algorithm in the documentation. 
> In fmincon at 501
  In Damper_Optimization_v1 at 159 
Warning: Your current settings will produce an error in a future release. 
> In fmincon at 511
  In Damper_Optimization_v1 at 159 
Starting parallel pool (parpool) using the 'local' profile ... connected to 2 workers.
Error using Objective (line 40)
Error due to multiple causes.

Error in finDiffEvalAndChkErr (line 26)

Error in parfinitedifferences>(parfor body) (line 129)

Error in parfinitedifferences (line 118)

Error in nlconst (line 338)
                [gf,gnc(:,nonlIneqs_idx),gnc(:,nonlEqs_idx),numEvals] = ...

Error in fmincon (line 846)
    [X,FVAL,LAMBDA,EXITFLAG,OUTPUT,GRAD,HESSIAN]=...

Error in Damper_Optimization_v1 (line 159)
[Optimum_Damper,fun_value,exitflag,output_details] = fmincon(...

Caused by:
    Error using Objective (line 40)
    Error due to multiple causes.
        Error using Objective (line 40)
        Invalid setting in 'Model_GR59_v23_Damper_Optimization/Body Displacement - Velocity -
        Angles /Body Corners Height/Gain1' for parameter 'Gain'
            Error using Objective (line 40)
            Error evaluating parameter 'Gain' in 'Model_GR59_v23_Damper_Optimization/Body
            Displacement - Velocity - Angles /Body Corners Height/Gain1'
                Error using Objective (line 40)
                Undefined function or variable 'a'.
        Error using Objective (line 40)
        Invalid setting in 'Model_GR59_v23_Damper_Optimization/CPL/Gain1' for parameter 'Gain'
            Error using Objective (line 40)
            Error evaluating parameter 'Gain' in 'Model_GR59_v23_Damper_Optimization/CPL/Gain1'
                Error using Objective (line 40)
                Undefined function or variable 'g'.
        Error using Objective (line 40)
        Invalid setting in 'Model_GR59_v23_Damper_Optimization/Force Calculator/Front Suspension
        Forces/Constant' for parameter 'Value'
            Error using Objective (line 40)
            Error evaluating parameter 'Value' in 'Model_GR59_v23_Damper_Optimization/Force
            Calculator/Front Suspension Forces/Constant'
                Error using Objective (line 40)
                Undefined function or variable 'FD_trans_speed_bump'.
        Error using Objective (line 40)
        Invalid setting in 'Model_GR59_v23_Damper_Optimization/Heave & Body Heights/Gain10' for
        parameter 'Gain'
            Error using Objective (line 40)
            Error evaluating parameter 'Gain' in 'Model_GR59_v23_Damper_Optimization/Heave &
            Body Heights/Gain10'
                Error using Objective (line 40)
                Undefined function or variable 'Mb'.
        Error using Objective (line 40)
        Invalid setting in 'Model_GR59_v23_Damper_Optimization/Pitch/Gain1' for parameter 'Gain'
            Error using Objective (line 40)
            Error evaluating parameter 'Gain' in
            'Model_GR59_v23_Damper_Optimization/Pitch/Gain1'
                Error using Objective (line 40)
                Undefined function or variable 'a'.
        Error using Objective (line 40)
        Invalid setting in 'Model_GR59_v23_Damper_Optimization/Roll/Gain1' for parameter 'Gain'
            Error using Objective (line 40)
            Error evaluating parameter 'Gain' in 'Model_GR59_v23_Damper_Optimization/Roll/Gain1'
                Error using Objective (line 40)
                Undefined function or variable 'tr'.
        Error using Objective (line 40)
        Invalid setting in 'Model_GR59_v23_Damper_Optimization/Vertical EOM Wheels/Gain1' for
        parameter 'Gain'
            Error using Objective (line 40)
            Error evaluating parameter 'Gain' in 'Model_GR59_v23_Damper_Optimization/Vertical
            EOM Wheels/Gain1'
                Error using Objective (line 40)
                Undefined function or variable 'mf'.

So, the simulation start and I think the problem I had before with the options of fmincon is now solved However, it seems that the parallel workers do not see the parameters that need to be given to the Simulink model to work (like a, Mb, tr,mf and so on)

I load the parameter for the Simulink model in Matlab Workspace declaring them as global variables (but my understanding is that this is not enough to make the parallel workers see those variables?)

I think that ideally I should load the parameters in the Simulink workspace so whoever will use the Simulink model will find the parameters in the model already, does anyone know a command to load variables into the Simulink workspace?

Alan, what you think about my comments above?

2. Yes I call fmincon with options parameters, see above my original question amended now shows all the code involved in fmincon

PS: I also attached the cost function to minimize "Objetive.m" and the script that load the CONSTANT parameters for the simulink model "Damper_Optimization_v1.m" and that perform the optimization with fmincon. Just to clarify the parameters I want to optimize are in the cost function, FD_p1, FD_p2 ....) Please note that all parameters have a nominal value of 10 as I do not want to share some information about my model (but this does not make any difference to my question)

Thanks

G

[Alan Weiss]

I'm just guessing here, because you did not show us your function call

 x = fmincon(...)

But it seems that you are not passing parameters properly. I suggest that you get all your parameters into your workspace, then make an anonymous function

 fun = @(x)yourobjectivefunction(x,a,b,c...)

then call fmincon with fun as the objective. fmincon will pass the parameters to all the workers.

I also suggest that you set the Algorithm option to 'interior-point', but that is entirely up to you.

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation

[Giuseppe Naselli]

Hi Alan,

I amended my question and now it show the fmincon call

Please have a look

I am trying your suggestion right now and will update the thread with the results

Many Thanks

G

[Giuseppe Naselli]

So I used your advice Alan and I got a different error now. Just to recap

This is the way I define the options for fmincon

% Define fmincon options

options = optimoptions('fmincon'); % Start with the default options
options = optimoptions(options,'Algorithm', 'interior-point');
options = optimoptions(options,'Display', 'iter-detailed');
options = optimoptions(options,'MaxIter', 1000);
options = optimoptions(options,'PlotFcns', { @optimplotfval });
options = optimoptions('fmincon','UseParallel','always');

Then, I then define

fun = @(Damper_Parameter)Objective_v2(Damper_Parameter)

and get

fun = 

      @(Damper_Parameter)Objective_v2(Damper_Parameter)

Then I run the fmincon

[Optimum_Damper,fun_value,exitflag,output_details] = fmincon(...
@fun,Damper_Parameter_Initial_Guess,A,B,[],[],Damper_Parameters_Lower_Bound,Damper_Parameters_Upper_Bound,[],options);

I get this error

Error using feval
Undefined function 'fun' for input arguments of type 'double'.

Error in fmincon (line 640)
      initVals.f = feval(funfcn{3},X,varargin{:});

Error in Damper_Optimization_v2 (line 184)
[Optimum_Damper,fun_value,exitflag,output_details] = fmincon(...

Caused by:
    Failure in initial user-supplied objective function evaluation. FMINCON cannot continue.

Any idea?

Thanks

G

[Alan Weiss]

You have a few errors.

1. When you write your last option line you undo all the previous option settings:

 options = optimoptions('fmincon','UseParallel','always');

2. You don't seem to be passing extra parameters in your call. In previous posts you had Objective_v2 depend on many parameters, not just Damper_Parameter. But maybe I misunderstand or misremember.

3. You passed fun incorrectly. By definition, fun is already a function handle, so you should write

 [outputs] = fmincon(fun,...) % not @fun

Alan Weiss

MATLAB mathematical toolbox documentation

[Giuseppe Naselli]

Hi Alan,

thanks for your answer, but I still have problems in making it works. Here my reply to your points

1. You are absolutely right, I see your point so I changed my code in

options = optimoptions('fmincon','UseParallel','always'); 
options = optimoptions(options,'Algorithm', 'interior-point');
options = optimoptions(options,'Display', 'iter-detailed');
options = optimoptions(options,'MaxIter', 1000);
options = optimoptions(options,'PlotFcns', { @optimplotfval });

However, once I run it I got these errors

Error using Objective_v4 (line 132)
Error due to multiple causes.

Error in finDiffEvalAndChkErr (line 26)

Error in parfinitedifferences>(parfor body) (line 129)

Error in parfinitedifferences (line 118)

Error in computeFinDiffGradAndJac (line 50)

Error in barrier (line 137)

Error in fmincon (line 905)
    [X,FVAL,EXITFLAG,OUTPUT,LAMBDA,GRAD,HESSIAN] = barrier(funfcn,X,A,B,Aeq,Beq,l,u,confcn,options.HessFcn, ...

Error in Damper_Optimization_v4 (line 184)
[Optimum_Damper,fun_value,exitflag,output_details] = fmincon(...

Caused by:
    Error using Objective_v4 (line 132)
    Error due to multiple causes.
        Error using Objective_v4 (line 132)
        Invalid setting in 'Model_GR59_v24_Damper_Optimization_v4/Body Displacement - Velocity - Angles /Body
        Corners Height/Gain1' for parameter 'Gain'
            Error using Objective_v4 (line 132)
            Error evaluating parameter 'Gain' in 'Model_GR59_v24_Damper_Optimization_v4/Body Displacement -
            Velocity - Angles /Body Corners Height/Gain1'
                Error using Objective_v4 (line 132)
                Undefined function or variable 'a'.
        Error using Objective_v4 (line 132)
        Invalid setting in 'Model_GR59_v24_Damper_Optimization_v4/CPL/Gain1' for parameter 'Gain'
            Error using Objective_v4 (line 132)
            Error evaluating parameter 'Gain' in 'Model_GR59_v24_Damper_Optimization_v4/CPL/Gain1'
                Error using Objective_v4 (line 132)
                Undefined function or variable 'g'.
        Error using Objective_v4 (line 132)
        Invalid setting in 'Model_GR59_v24_Damper_Optimization_v4/Force Calculator/Front Suspension
        Forces/Constant' for parameter 'Value'
            Error using Objective_v4 (line 132)
            Error evaluating parameter 'Value' in 'Model_GR59_v24_Damper_Optimization_v4/Force Calculator/Front
            Suspension Forces/Constant'
                Error using Objective_v4 (line 132)
                Undefined function or variable 'FD_trans_speed_bump'.
        Error using Objective_v4 (line 132)
        Invalid setting in 'Model_GR59_v24_Damper_Optimization_v4/Heave & Body Heights/Gain10' for parameter
        'Gain'
            Error using Objective_v4 (line 132)
            Error evaluating parameter 'Gain' in 'Model_GR59_v24_Damper_Optimization_v4/Heave & Body
            Heights/Gain10'
                Error using Objective_v4 (line 132)
                Undefined function or variable 'Mb'.
        Error using Objective_v4 (line 132)
        Invalid setting in 'Model_GR59_v24_Damper_Optimization_v4/Pitch/Gain1' for parameter 'Gain'
            Error using Objective_v4 (line 132)
            Error evaluating parameter 'Gain' in 'Model_GR59_v24_Damper_Optimization_v4/Pitch/Gain1'
                Error using Objective_v4 (line 132)
                Undefined function or variable 'a'.
        Error using Objective_v4 (line 132)
        Invalid setting in 'Model_GR59_v24_Damper_Optimization_v4/Roll/Gain1' for parameter 'Gain'
            Error using Objective_v4 (line 132)
            Error evaluating parameter 'Gain' in 'Model_GR59_v24_Damper_Optimization_v4/Roll/Gain1'
                Error using Objective_v4 (line 132)
                Undefined function or variable 'tr'.
        Error using Objective_v4 (line 132)
        Invalid setting in 'Model_GR59_v24_Damper_Optimization_v4/Vertical EOM Wheels/Gain1' for parameter 'Gain'
            Error using Objective_v4 (line 132)
            Error evaluating parameter 'Gain' in 'Model_GR59_v24_Damper_Optimization_v4/Vertical EOM
            Wheels/Gain1'
                Error using Objective_v4 (line 132)
                Undefined function or variable 'mf'.

Please note that 'a', 'tr', 'mf' and all the other parameters that are listed above in the error messages are all parameters that are needed for the simulink model (but stay constant during the optimization)

2. A part "Damper_Parameter", all the other parameters you saw on Optimization_v2 are parameters that are needed for the simulink model to work(but stay constant during the optimization). My aim is to use fmincon to get an Optimum "Damper_Parameter" only

3. If I take off the "@" character, fmincon does not work even with no parallel computing (please remember everything works with no parallel computing option). It gives this error

Error using Objective_v4 (line 114)
Not enough input arguments.

Error in Damper_Optimization_v4 (line 184)
[Optimum_Damper,fun_value,exitflag,output_details] = fmincon(...

Finally I wish to try something but I do not know how to do it.

I still think that the problem is due to the fact that the parallel workers do not see Matlab workspace. I think that if I will be able to assign values to the simulink model workspace from command line I can solve this problem. But I do not know how to do it. Any idea?

Thanks in advance

Regards,

G

[Alan Weiss]

I am sorry, but I do not know if we are communicating correctly or not. I think you somehow need to define your objective function as

 fun = @(Damper_Parameter)Objective_v2(Damper_Parameter,a,g,...)

where your objective function explicitly depends on the parameters a, g, etc. You say that these parameters do not change during the optimization. Then you can call

 [Optim_Damper,...] = fmincon(fun,...)

and you will not have a problem. In other words, you need to write your objective function so that it passes the variables, and does not pick up variables from your workspace. Then you will be able to compute in parallel as you desire.

I do not anything about Simulink, if that is what you are using to compute your objective function. So I am sorry not to be able to advise you on how to pass parameters to Simulink. But I am certain that you can.

One more thing. I would not be surprised if you have to change some options in order to successfully optimize a Simulink model. See Optimizing a Simulation or ODE.

Alan Weiss

MATLAB mathematical toolbox documentation

[Giuseppe Naselli]

Thanks a lot Alan, the last comment was the one which actually made me achieve my goal

Regards,

G

[Maher]

ms = MultiStart('UseParallel','true') and not 'UseParallel' in the options