Global variables - is there something better?

I have a large structure; contains test and parameter descriptions and a large (2M+ Rows X 100+ Columns) matrix. I need to read from this structure using multiple functions (some function multiple times). The structure is large enough that I don't want multiple copies in memory. Most functions will read only, but some may create new columns that will need to be added to the current matrix. This structure always has the same name and structure (data matrix will vary) so I'd like to write functions to simply read from it but all I read is AVOID GLOBAL VARIABLES.

[per isakson]

• "[...] all I read is AVOID GLOBAL VARIABLES" As with all HeuristicRules, this is not a rule that applies 100% of the time.
• I think that a prerequisite to using GLOBAL is knowledge on why GLOBAL is problematic. If that doesn't apply to your situation use GLOBAL. If in doubt, don't use GLOBAL.
• "I don't want multiple copies [of a few megabytes] in memory" I think that's a weak argument. Furthermore, I think "multiple" is exaggerating. Replace it by "a few". See Memory Management for Functions and Variables and Internal Matlab memory optimizations
• I cannot remember when I last used GLOBAL, but I use Singletons.
• "something better" Pass the data needed to the functions. Use In-place Operations on Data.
• Make some experiment before you decide.

Answer 1

[Cedric]

Look at my answer there if you want to try using a handle class: https://www.mathworks.com/matlabcentral/answers/359410-is-it-possible-to-avoid-copy-on-write-behavior-in-functions-yet

An OOP approach would also be interesting in the sense that you could implement specialized methods for operating on your data structure and maybe even overload some operators to fine tune how they operate.

If you don't want to go for OOP, then nested functions could be a good alternative to globals:

 function buildAndProcess( locator, flag )
     largeArray = load( locator ) ;
     process1() ;
     if flag
        process2() ;
     end

     % === Nested Functions ===
     function process1()
          largeArray(:,end) = ... ;
     end
     function process2()
          largeArray(end,:) = ... ;
     end
 end

[Jun Yup Kim]

this doesn't do

Answer 2

[Walter Roberson]

There is no way to avoid having multiple copies of a data item in memory if you are adding more data to it: it is always necessary to find new memory large enough to hold the combined results, copy the data out of the existing item into the new data item, and then copy as well the new values in as well.

You can avoid this by pre-allocating the data item as large as it could possibly get. You can reduce the impact of this pre-allocation if what you make the data type cell, or structure array (pre-allocated to the maximum number of possible entries), or table, or containers.Map: all of those data types essentially internally contain a number of pointers to chunks of data and in some circumstances only the one chunk has to be copied to update it (or, at most, copies of the pointers need to be made rather than copies of all of the associated data.)

It sounds as if perhaps your added columns do not always have the same meaning: if that is the case, then a struct or a table or a containers.Map might make a lot of sense, giving you the ability to name the item and retrieve it by name.

You might also see reference to handle variables. Only one copy of a handle object exists -- except during the time that it is being modified, at which point it has the same rules about having to allocate / copy when the size changes. It might, however, make some sense in your program to create your data as an "object" with a number of different named properties -- which is internally implemented as a struct and so modifying one property does not require deep copies of the memory for another property. This would give you the same named-item advantages I mentioned for struct and table and containers.Map