Differences in memory pre-allocation approaches.

Why should we prefer one pre-allocation approach over the other?

I have compiled three different approaches that apparently pre-allocates memory in MATLAB.

clear all
clc

N = 2^28;

tStart = tic;

% Approach #1: 
%X = zeros(N,1);

%Approach #2: 
% X = [];
% X(N) = 0;

%Approach #3: 
%X = [];
%X(N, 1) = 0;

%TEST CODE
%Code that uses the previous pre-allocation approaches. 
%for i = 1:N
%     
%     X(i) = i;
% 
% end

tStop = toc(tStart)

Observations (All approaches equally speed up the test code.):

Approach #1: When I use this pre-allocation approach, MATLAB effectively reserves the memory, and this memory usage shows up on the Task Manager. The task manager shows MATLAB is using 2.50 GB before and after executing the test code.

Approach #2: This approach has the same behavior as approach #1.

Approach #3: When I use this pre-allocation approach, before executing the test code, the Task Manager shows MATLAB is using only about 500MB of memory, but after executing the test code it uses 2.50 GB, as seen on approach #1 and #2.

Does it mean that Approach #3 is more memory efficient than the other two approaches? It seems as if Approach #3 is pre-allocating memory, but it is not using it until required. Thanks!

[the cyclist]

I would have expected all three of these approaches to have essentially same memory behavior.

I see differences in memory use, as you do, but here is some strangeness. For me, Approach #1 is the one that uses a smaller amount of memory after the preallocation (around 700 MB for me), and then grows to 2.5GB while the for loop is running. Approaches #2 and #3 use 2.5GB immediately after the preallocation.

I am running R2013a on a MacBook Pro (OS X 10.8.3).

[James Tursa]

I'm curious ... what behavior do you get if you use this mex routine for the pre-allocation?

http://www.mathworks.com/matlabcentral/fileexchange/31362-uninit-create-an-uninitialized-variable-like-zeros-but-faster

[Cedric]

I'll have to make a few tests about #3, but Yair discusses #1 and #2 in http://undocumentedmatlab.com/blog/preallocation-performance/ (and associated threads).

Answer 1

[per isakson]

This has been discussed more than once. I think Loren has discussed it in her blog.

Some comments

• The behavior of Matlab may change from one release to the next. Thus, always state which release.
• With R2012a #1 allocates memory and assigns the value 0 to all elements. #2 and #3 takes a "lazy" approach. Matlab "reserves" memory internally and only the last element is assigned a value. The memory is not used until values are assign by your for-loop.
• Try tic, ..., toc the three approaches and I think you will see that #2 and #3 are much faster the #1.

[Greg]

I know this is a really old thread, but I'm not comfortable with the implication of this answer. Methods #2 and #3 probably are faster at the pre-allocation, but the subsequent for loop will likely be much slower because MATLAB hasn't fully allocated that memory. This holds with my experience (sorry, I don't have any concrete examples to offer). I always suggest using zeros/ones/nan/inf/true/false functions to preallocate arrays if for no other reason than it's clear and obvious.

Answer 2

[Jan]

For real world applications the actual work is the same: Either Matlab can reserve the memory lazily at first or allocate zeroed memory directly. At least since the data are written to the array, the explicit allocation must have happend. Therefore measuring the run time should include the process of writing.