A more systematic/efficient way to do optimization in Matlab

Dear ALL,

Part 1: Problem Statement

(1) I have an objective function, which is calculated from the simulation results of an external executable. That is to say, I don't have the exact formula for my objective function. Therefore, I don't know whether it is smooth or not.

(2) I totally have 8 parameters to optimize:

4 discrete parameters: (a) Opt_PerfNo (4 variables), (b) Opt_PerfDia (4 variables), (c) Opt_ProppSize (1 variable), (d) Opt_ProppDen (1 variable)

4 continous parameters/variables: (e) Opt_ClusterSpacing (1 variable), (f) Opt_MaxInjRate (1 variable), (g) Opt_MaxPropConc (1 variable), (h) Opt_RampUpSpeed (1 variable)

Note 0: For parameter (a) and (b), each parameter will have 4 variables. Therefore, totally, I have 14 (=4+4+1+1+1+1+1+1) variables.

Note 1: Please note that sometimes I might optimize all the variables, but sometimes I might ONLY optimize one variable, eg., 1 discrete variable.

Note 2: (f) (g) (h) (the last three variables) must be optimized together. That is to say, I have ONLY two options: (1) optimize (f) (g) (h) together; (2) (f) (g) (h) all these three are not optimized.

(3) Linear inequality constraint

I will have this constraint ONLY when I choose to optimize: (a) or (f,g,h) or both (a) and (f,g,h).

(4) Non-linear inequality constraint

I will have this constraint ONLY when I choose to optimize (f,g,h).

I want to run this optimization in a systematic way (more efficient way).

Part 2: Motivation

Previously, I always choose to run Genetic Algorithm (GA) to do the optimization. Then, I realized one problem.

For example, I am optimizing ONLY 1 discrete variable (parameter (d)) and I am giving GA two options.

Then, I thought GA only needs to run 2 cases since I only give it 2 options, and then compare the objective function values of these two cases and get the optimal solution.

However, GA actually runs a lot of cases, which is a waste of time.

Essentially, if I ONLY optimize parameter (d), then, according to the Problem Statement in Part 1, I don't have any constraints at all.

My point here is, sometimes, if I choose to optimize a lot of parameters, then my problem would be very complex, then at some point the only option I have is to use GA. However, in some cases, if I choose to optimize only several of them, then my problem would be degraded to be simple ones, then GA is not necessary at all.

Therefore, I want to do my optimization in a more efficient way.

Part 3: My Current Idea and Questions

My current strategy is as follows (Any comments and suggestions would be greatly appreciated!!!):

(1) If I only optimize (e), which is a continuous variable

Then I don't have any constraints.

Q1: Which optimization method I should use? Is GA a good option?

(2) If I only optimize (f,g,h), which are three continuous variables

Then I will have both the linear and non-linear constraints.

Q2: Which optimization method I should use? Is GA a good option?

(3) If I only optimize (b) and/or (c) and/or (d), which means I only have discrete variables

Then I I don't have any constraints.

Actually, in these cases, I can calculate the total number of valid combinations of variables based on the options of each parameter.

For example, if (b) has 8 options, (c) has 5 options, (d) has 4 options, then the total number of valid combinations = 8^4 * 5 * 4 = 81,920.

Usually, one run of my external executable will take around 50 seconds.

Then, the first idea in my mind is, wow, that's a lot of cases! Then, it is impossible to run all the forward models... Then let's use GA!!

Q3: Is GA a good option?

However (second thought...), if I only optimize (c) and/or (d), in this case, total number of valid combinations = 5 * 4 = 20.

Then, I would say, please run all the 20 cases of forward model and find the global optima. But how?

Q4: Am I correct?

Then, I would do like this (please remember that here I don't have any constraints):

Step 1: based on the options the user gives, I will first calculate the total number of valid combinations = A.

Step 2: Is A > 200 (Q5: how to choose this threshold)?

If "yes", please run GA. Q6: Is GA a good option? How about pattern search? Or Mixed-Integer Linear Programming?

If "no", please run all the forward models. Q7: But how? I mean for example I have 150 valid combinations, how could I run all these cases one by one?

Q8: What do you think of this strategy?

(4) If I choose to optimize (a)

Then I will have the linear constraint.

For example, if (a) has 7 options, then the total number of valid combinations = 7^4 = 2401.

Q9: Is GA a good option? How about pattern search? Or Mixed-Integer Linear Programming?

Any comments and suggestions would be greatly appreciated!

Thanks in advance!

Answer 1

[Matt J]

You've listed 9 questions, but as far as I can tell you really have two:

(1) Sometimes I can enumerate the solutions and sometimes I cannot. What should I use when there is a small number of possible solutions, which I can enumerate?

In cases where you can enmerate a small number of solutions, you should just evaluate the fitness function at all of those solutions in a loop. Then just use min() to obtain the minimum.

(2) When I cannot enumerate the solutions, should I use ga, patternsearch, or intlinprog (mixed integer linear programming solver)?

It is hard to know if ga or patternsearch would be better. You should test both. intlinprog would be best to use when your problem is a mixed integer linear programming, but as far as you have told us, it is not. You said you don't have an exact formula for your objective function, so it is not even clear that the objective is linear.