Faster indexing from Matfiles with similar names

Hi,

Suppose I have 5 matfiles called m1,m2,m3 up to m5 and they all have the same variable called a in it.

Is it possible for me to index each a from m1 to m5 using a for loop that can loop through m1 to 5 and call the variable a on each iteration?

So far the results I have found only help when it comes to setting up the matfile but not calling variables from it.

Any help would be greatly appreciated.

Answer 1

[Stephen23]

"What I meant when I said I had 5 matfiles called m1 to m5 is that m1 - m5 were matfile-type variables obtained by calling the matfile function in MATLAB, i.e it allows us to index some of the variables in the entire dataset without loading it into memory. Is there any way to replicate what you did above in this case?"

If you had stored the matfile-type variables in one cell array then this would be trivial and easy using indexing.

But because you made the mistake of using lots of separate variables and numbering the variable names, the first thing we should do is to join them together into one cell array.

A = {m1,m2,m3,m4,m5};

for k = 1:numel(A)

m = A{k};

% do whatever with matfile m

end

Lets try it right now. First we create some fake data in MAT files:

X = 11; save test1 X
X = 22; save test2 X
X = 33; save test3 X
X = 44; save test4 X
X = 55; save test5 X

Then we can access the MAT files:

m1 = matfile('test1.mat');
m2 = matfile('test2.mat');
m3 = matfile('test3.mat');
m4 = matfile('test4.mat');
m5 = matfile('test5.mat');
A = {m1,m2,m3,m4,m5};
for k = 1:numel(A)
    m = A{k};
    m.X
end

ans = 11

ans = 22

ans = 33

ans = 44

ans = 55

But rather than creating lots of numbered variables (which should be avoided), you should just use a cell array in the first place.

[Chun Hei Chan]

Thanks Stephen, it worked perfectly.

Answer 2

[Ayush]

Hi @Chun Hei Chan

The 'matfile' function in MATLAB allows you to index and access specific variables in a matfile without loading the entire dataset into memory. This can be useful when dealing with large datasets where loading everything into memory may not be feasible. Here is an example for that:

% Create a matfile object for the matfile 
mat = matfile('data.mat'); 

% Access and index specific variables without loading the entire dataset 
variable1 = mat.variable1; 
variable2 = mat.variable2; 

% Perform operations on the indexed variables 
result = variable1 + variable2; 

Where,

1. ‘matfile’: function is used to create a matfile object for the 'data.mat' file.
2. ‘variable1’, ‘variable2’: specific variables within the matfile that can be accessed directly without loading the entire dataset into memory.

You may read further from here:

https://in.mathworks.com/help/matlab/ref/matlab.io.matfile.html

Thanks,

Ayush Jaiswal

Answer 3

[Sufiyan]

Hi Chun,

You can follow the procedure shown in the below code to call the variable a on each iteration.

% m1.mat to m5.mat
for i = 1:5
    filename = ['m', num2str(i), '.mat'];  % filename
    data = load(filename);
    
    a_value = data.a;  % 'a' variable from the loaded data
    
    fprintf('Value of a: %f\n', a_value);
end

Hope this helps!

[Chun Hei Chan]

Hi Sufiyan,

Thanks for the reply. Perhaps I wasn't clear about it in my original question. What I meant when I said I had 5 matfiles called m1 to m5 is that m1 - m5 were matfile-type variables obtained by calling the matfile function in MATLAB, i.e it allows us to index some of the variables in the entire dataset without loading it into memory. Is there any way to replicate what you did above in this case?

Many thanks.