Documentation on JIT compiler?

Can someone explain the workflow of the JIT compiler? I am interested in basic functionality. When is compiled code saved, what causes it to be deleted, and what causes it to be recompiled.

I read somewhere that changing directories forces a reset of previously compiled code, but I couldn't find any documentation explaining this.

My goal is to write maximally optimized code. Another Q - I've also noticed that a large software package opens faster the second time I open matlab, suggesting that JIT compiled code "lingers" even after closing and reopening matlab. Is this true?

[John D'Errico]

Don't. That is, don't make any assumptions about what goes on under the hood there. Maximally optimized code won't necessarily be so next year anyway. And maximally optimized code would be CPU & system dependent. It would be problem dependent, because problems of different sizes might involve differing tricks to be truly optimal.

Even something as simple as a matrix*matrix multiply has been shown to be non-obvious. For example, carefully time how long a matrix*matrix multiply takes, then plot the time required as a function of the order of the matrix. For nxn matrices, this should be an O(n^3) problem, scaling smoothly, because your CPU will just pump those adds and multiplies through as fast as it can. And we can count just how many such ops are needed. But various factors conspire here, from the blas to cache size, etc. So what happens is that curve of time versus order is not a smooth one, even after you reduce the noise by replication. The bumps will be system dependent, and consistent, but they will also potentially change with release, nor should you bother to try to find some subtle tweak to avoid them.

Spending a lot of time to chase subtle features in JIT is a waste of time. Instead, spend that time to profile your code if there are bottlenecks. Then spend time to either alleviate those specific identified bottlenecks, or just move on and accept tham.

[Andrew Landau]

Hm. Fair. I see the practicality in your advice. However, if you look at matlab documentation, the number of people who say "don't use 'clear'" is high. The reason is so that you don't force the engine to recompile code. Now, according to your advice, this shouldn't matter.

My question relates to these kinds of tips- are there things I am doing in my code for convenience of writing that lead to inefficient execution.

Importantly- I want to understand more about how Matlab works so I can make my own decisions about what to do.

[per isakson]

See Techniques to Improve Performance.

[Andrew Landau]

thanks for the link, but there's only a single tip there about JIT:

"Avoid clearing more code than necessary. Do not use clear all programmatically. For more information, see clear."

I already mentioned that I know this.

I want to see actual documentation that explains how JIT compiling works in matlab!

[Walter Roberson]

The blog entries are the only "actual documentation", aside from a couple of lines here and there in the Release Notes. The detailed workings of JIT is pretty much proprietary.

[Andrew Landau]

that's wild! thanks

[Stephen23]

"I want to see actual documentation that explains how JIT compiling works in matlab!"

You are very unlikely to get it, for the reasons that John D'Errico has already explained. The TMW blogs have mentioned this several times, e.g. "We recommend not writing specifically for the JIT since the JIT is constantly evolving. You should though keep in mind the general good MATLAB-coding practices, such as those listed above"

source: https://blogs.mathworks.com/loren/2008/06/25/speeding-up-matlab-applications

and made it clear that code should not be written around JIT enhancements: "Keep your code natural. Don't write unnatural code just to take advantage of this."

source: https://blogs.mathworks.com/loren/2007/03/22/in-place-operations-on-data/

See also:

https://www.mathworks.com/matlabcentral/answers/214625-matlab-jit-and-execution-time

"However, if you look at matlab documentation, the number of people who say "don't use 'clear'" is high. The reason is so that you don't force the engine to recompile code. Now, according to your advice, this shouldn't matter. "

Unlike the behavior of the JIT engine, the behavior of clear all is quite well documented, as are the multiple sources in the documentation that recommend avoiding clear all because it forces code to be recompiled. So that is a rather apples-with-oranges kind of comparison.

Answer 1

[Philip Borghesani]

For the most part I belive your question has been answered well in the comments above. One reason for limited availability of details, is that they keep changing, and no matter how much you think you are writing code for one specific version of MATLAB the reality is that large amounts of code get run over many years on many versions of MATLAB. Writing code that is easier for a human to understand frequently also makes it easier for the computer (or jit) to understand, if not now then it will in the future. If the jit/compiler can understand your code better then it can optimize it better. Most of the suggestions here: techniques for improving performance are made because following them makes the code work better with the jit.

Before R2015b MATLAB could only run some code with a limited jit. In R2015b and later a completely rewritten jit (we call it an execution engine) was introduced that had very different characteristics from the previous jit. Many micro-optimizations that worked well no longer applied and much problematic code was no longer problematic. Since then we have continued to add more optimization to all areas of MATLAB code.

I will answer one specific question: Any speedup you see between two executions of MATLAB is entirely due to operating system caching of files accessed from disk. Think of restarting MATLAB as the strongest possible form of clearing.

For the most part the causes of code recompilation are farly obvious and most recompiliation takes place on an only as needed bases when the code runs. If the code file or type of the data changes, or possible functions to be called (cd or path change) or the file is explictetly cleared then code must be recompiled. Function code compiles a bit better because MATLAB knows all variables in the workspace when it is compiled instead of having to check when running.

[Andrew Landau]

"Most of the suggestions here: techniques for improving performance are made because following them makes the code work better with the jit."

Now that you say that this makes more sense to me. Thanks!