Fastest methods to solve nonlinear systems MANY times

I have an unconstrained nonlinear system of 1000 equations & 1000 variables. There are also parameters in the system:

F(x,params)=0

I want to numerically solve this system for different values of the parameters (parameter sweep in a sense). My current approach is:

set=[grid of trial parameter values]
big_x=size(length(set),1000)
for i=1:length(set)
    x=fsolve(@(x)Function(x,set(:,i)),x0,options)
    big_x(i,:)=x;
end

This approach is too slow. I provide the Jacobian of the system to improve speed but each iteration takes 0.3 seconds and I have 100 million iterations. I've tried other solvers (fminunc, lsqnonlin, manual Newton's method) with little difference. I also tried using previous solutions as new starting values with no performance boost.

Options I've tried:

• PARFOR: If I could get 1000x parallel then it could be feasible but I cannot seem to get more than 50 parallel processes going on the HPCC I have access to
• Compiling the code: I don't seem to get a major performance boost when I compile the code, but I will try more.
• Removing the loop with CELLFUN: I looked into avoiding any loops but people on this forum say that cellfun typically does NOT yield a speed increase and I have been unable to get cellfun to work when I manually code a Newton's method solver.

Question Is there any major approach paradigm I'm missing? If not, then I will abandon the current parameter sweep approach to my underlying estimation problem and seek alternatives.

Answer 1

[John D'Errico]

Big problems take big time.

1. Compiling will almost NEVER yield a speed increase. If it does, any speed boost will not be that much.

2. cellfun is an implicit loop. But it is still a loop!!!!! Simple loop overhead is not that significant in any case.

3. Parallel processing can give you a boost, limited by the number of processors you can use. If 50X is not sufficient, then you are trying to solve too large of a problem.

I have no idea why you need to solve this system 100 million times, but surely this is the real issue. Finding a viable alternative will surely give you far more of a speed boost than any of the alternatives.

Or, just sit down and read the complete works of Shakespeare a dozen times while you are waiting for this to finish. At least when it is done, you will know them by heart. Can't hurt.

[CJ]

So the 100 million comes from wanting to do a grid space of 10 parameters with 50 points each and then randomly choosing a subset of those. Perhaps I should phrase this question more as a way to do a parameter sweep/grid search that doesn't require a fine mesh for each parameter.