cache_results

CACHE_RESULTS - caching function results to disk with transparent tracking of arguments

cache_resutls will evaluate a specified function with the provided arguments, then save the results to disk in a file named according to the first argument. On a subsequent call, it will check for such a file, and, if the arguments match, return the result read from file instead of re-evaluating the function. In a situation where the same function is being evaluated on the same data in multiple places (or mulitple times), and where the evaluation is at least moderately computationally expensive, cache_results can be used as a transparent replacement that will avoid redundant computations.

When the first argument is a string (for instance, a file name), this name is used (with small modifications) as the name of the cached results file (within a specific cache directory). When the first argument is some other Matlab type, it is converted into a hexadecimal hash (using Jan Simon's DataHash) which is then used as the cache file name.

Although (sometimes) only the first function argument is used to construct the cache file name, all the arguments to the function are recorded in the file, along with the corresponding output. When a cache file is found, cache_results then performs an exact comparison of all the arguments (as recorded in the cache file) with all the arguments provided for the current invocation, and only uses the cached result if the arguments match. If no match is found, the provided function is executed. This new result is then added to the cache file (along with the distinct set of arguments), so that in future both results will be returned from cache. Thus, while individual cache files distinguish only the first argument, the system can handle any number of different argument sets associated with this first argument. However, the search through different argument sets within a cache file is linear (at present), so the most efficient way to use the function.

While largely transparent, using a results cache of this kind has a few drawbacks. If the function does not give deterministic results given its input arguments, then caching the result will "freeze" the output, changing its behavior. If the first argument refers to a file on disk, but that input file itself is changed, cache_results will not know to re-evaluate the function on the new file. If the function itself is changed (so that even when evaluated with the same input arguments, its result will be different), cache_results will still return the cached value. There's some code within the function to check the modification date of both cache file and function which could possibly force a recalculation when the function is modified, but this behavior is currently disabled.

See http://www.ee.columbia.edu/~dpwe/resources/matlab/cache_results/ .