decrease the elapse time

this set of files organized to solve 'environment' function with ranges of different variables. I followed all possible instructions but the elapse time is growing with each iteration inside the script 'Ta_I_yearly_by_azimangle' so I have to take bigger steps. Maybe the whole thing could be organized some other way to make it faster, I don't know, so I need your help here, thanks.

[Asliddin Komilov]

the files

[Rik]

The answer is the same as with your previous question.

[Asliddin Komilov]

the last thing was the preallocation I needed help with, which is already implemented, I am just checking if there is anything else to do that I don't know of. thanks

[Jan]

Use the profiler to find the bottleneck of the code: which line needs te most computing time? Then post the corresponding code with some test data as input.

Start with omitting the clear all, because this wastes a lot of time: Removing all loaded frunctions from the memory requires to load them again from the slow disk. There is no benefit in such a brute clearing.

[Asliddin Komilov]

this is what I got running the 'Ta_I_yearly_by_azimangle' for i=5

Elapsed time is 101.771973 seconds.

Elapsed time is 37.928433 seconds.

Elapsed time is 428.026140 seconds.

Elapsed time is 297.756538 seconds.

Elapsed time is 138.780894 seconds.

code file 'environment' run time is almost the same always, around 28 sec., the rest is "Self time (built-ins, overhead, etc.)", that is so much different for each iteration (see the attachment). Actually I don't know what that means.

[Rik]

You should use the actual profiler. In most releases that is called 'run and time'. That will show you a breakdown by function and line by line.

Answer 1

[Asliddin Komilov]

appointing to 'simple' and 'int16' made huge difference

Answer 2

[Sulaymon Eshkabilov]

Hi,

You'd need to associate all of your calc's into one function file and no need to save all variable/calc values from every iteration. Save the only ones that you'd need as outputs from your main function file. ALso, put all of your external fuinction files as a nested functions within your main function file. All these steps should speed up your simulations.

Good luck.

[Asliddin Komilov]

thank you, it would be very nice if i also knew how to do it...

[Rik]

Why suggest such drastic changes before Asliddin has looked at the profiler? Have you tried the profiler yourself and concluded there isn't really a specific line of code that consumes too much time?

Making other functions internal functions also has downsides. It might not be required (or even desirable) to share variables between functions like that.

I suspect the largest gain in performance would be to find a way to express some or all of the loops as array operations.

[Asliddin Komilov]

as I wrote before, the whole code 'environment' is not of concern because it takes the same time each iteration around 28 seconds, but something else is happening outside of it which increase the elapse time in each iteration differently, as you can see above same iteration takes 38 seconds on time and next time its 428 seconds, and there is no distinct 'signature', the time is not increasing or decreasing with each next iteration, so if I run it once again, ellapse times for 5 differnt 'i' will be different. That is what I cannot understand why it is happening, and how to improve.

Answer 3

[Jan]

The output of the profile is useful - thanks.

The most time is spent here:

[days(:,:,i), time(:,:,:,i), I(:,:,:,i), Ta(:,:,:,i), ...
    angle(:,i), Ir(:,:,:,i), Id(:,:,:,i), Gincl(:,:,:,i), r(:,:,:,i)] = ...
    Ta_I_yearly_by_angle(azim_angle(i)); 

But this takes between 36 and 376 seconds. This is not an effect inside Ta_I_yearly_by_angle.

My guess is, that your memory is exhausted and sometime the slow disk caching is used. So please check the memory consumption. during the code runs. Pre-allocating is a good programming style already, but sometimes only more RAM helps.

[Asliddin Komilov]

thanks, I also had a thought it could be the computer, but I have 8g ram and 64 bit system, shall I uninstall some things? I have 2016a matlab, shall I look for a newer version? I need a solution because I need variable 'i' be not 5 but 36 and also variing 'r' variable for the best result.