Faster way to concatenate arrays with unknown size ?

In my application, I have a huge for loop and at each iteration, I concatenate the 'result' array with the matrix obtained at the current loop. The function returns this 'result' array at the end of the for loop. Is there a way I can speed things up because concatenating seems very slow when the result array gets large enough. I won't know the final size of the array but I will know the size of the matrix at the current loop if that helps..

Answer 1

[Walter Roberson]

If you know the final data size then pre-allocate the output and store into the array as you go.

If you do not know the final data size then store the pieces in a cell array until the end (better yet if you can at least pre-allocate the cell array itself.) You can then cell2mat()

[RuiQi]

Hi thanks! Would it help if I know the maximum size it can take (not the exact final size) ? i.e. if I know the cell will always have lesser than a million rows would it help if I initialize the cell array with a million rows then copy the data into the cell at every iteration ? Would there be any issues ?

I would then remove all the empty cells at the end

[per isakson]

"know the maximum size it can take" &nbsp

One approach: Assume the result_array is &nbsp<kxm double>. Preallocate total_result as &nbsp<kxmxn double> where n is maximum number of loops that can occur.

total_result = zeros(k,m,n) 
loop over huge number
     total_result(:,:,jj) = result_array;  
end
delete unused space of total_result

See Preallocation performance

[Walter Roberson]

Yes, removing extra rows afterwards is efficient.