How can I multiply two big matrices, avoiding out of memory?

21 visualizaciones (últimos 30 días)
for example, I = A*B, in which size(A) = [1024^2, 3], size(B) = [3, 1001^2]. So, size(I) = [1024^2, 1001^2] which could cause out of memory.
I have tried using tall arrays like the following code:
AA = tall(A);
II = AA*B;
I = gather(II);
but the command line still shows error: out of memory.
Sincerely thanks!
%% update %%
Thanks for your answer very much! Originally, I want to calculate this equation:
微信截图_20190826095243.png
where
so I write the following matrix equation:
m and n are variable, for example,m = 1024, n=1001^2 whatever. It is worth noting that the two matrices in exp(...) are not too big, but the result of their multiplication is too large to be storaged in memory. I've tried to split the two matrices into several small patches, but it need to input the number of the patches manually rather than automatically, moreover, the for loop could not be avoided in this case.
I have no good ideal currently.
Best regards!
  2 comentarios
Bruno Luong
Bruno Luong el 25 de Ag. de 2019
Editada: Bruno Luong el 25 de Ag. de 2019
Storage of your matrix requires 8 Terabytes. Do you have a HD that big?
Xin Liu
Xin Liu el 26 de Ag. de 2019
I don't have so big HD - -、, so I'm searching for a better method to avoid big matrix :)

Iniciar sesión para comentar.

Respuesta aceptada

Bruno Luong
Bruno Luong el 26 de Ag. de 2019
Editada: Bruno Luong el 26 de Ag. de 2019
You can process by chunk of smaller (100 here);
% Generate small test data
m = 100;
n = 100^2;
C = rand(m^2,3); % your [alpha,beta,gamma]
XYZ = rand(3,n); % your [x; y; z];
A = rand(1,m^2);
psize = 100; % chunk size
E = zeros(1,n);
count = 0;
while count < n
p = min(psize,n-count);
j = count+(1:p);
E(j) = A*exp(C*XYZ(:,j));
count = count + p;
end
  4 comentarios
Xin Liu
Xin Liu el 26 de Ag. de 2019
Well, I think I should accept your answer, :), haha~thank you very much.
Bruno Luong
Bruno Luong el 26 de Ag. de 2019
Well if you have considered this option, then it is your answer too.

Iniciar sesión para comentar.

Más respuestas (0)

Categorías

Más información sobre Numerical Integration and Differentiation en Help Center y File Exchange.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by