Out of memory for Blkdiag
2 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Vahid Hematian Dehkordi
el 28 de Mzo. de 2017
Comentada: Matt J
el 28 de Mzo. de 2017
I have two big matrices ( 5000X329176 and 5000X328658 ) and I want to combine them into one matrix which is in a form of these two in diagonal and all the other elements 0. basically I want blkdiag(A,B).
The problem is even though I made both these guys in int8 variable type and sparse and they take a small size but for blkdiag function, it needs to make a bigger matrix first ,zeros (5000+5000 X 329176+328658) and this guy is by default made in double which takes a huge amount of Ram.
I know e.g. zeros(100000,650000) needs 484 GB memory and sparse(zeros(100000,650000)) because of execution order needs the same amount ! however zeros(100000,650000, 'int8') needs 60GB memory which is available.
Unfortunately, blkdiag does not have this option. So, any idea about solving the problem?
0 comentarios
Respuesta aceptada
Matt J
el 28 de Mzo. de 2017
Editada: Matt J
el 28 de Mzo. de 2017
If the matrices are sparse, why store them as int8? Why not store them as sparse doubles? If you pre-convert A and B to type sparse, then blkdiag will assemble them without creating non-sparse type matrices, e.g.,
>> A=sprand( 5000,329176,.0001 );
>> B=sprand( 5000,328658,.0001 );
>> C=blkdiag(A,B);
>> Whos C
Name Size Kilobytes Class Attributes
C 10000x657834 10279 double sparse
and sparse(zeros(100000,650000)) because of execution order needs the same amount !
You would never create a sparse matrix this way. You always use one of the following syntaxes, which avoid allocating a full sized matrix,
S=sparse(m,n);
S=sparse(I,J,S);
1 comentario
Matt J
el 28 de Mzo. de 2017
Vahid Hematian Dehkordi commented:
Thank you so much. I thought sparse logical or sparse int8 should have the minimum size but it seems sparse double is the right way.
Más respuestas (0)
Ver también
Categorías
Más información sobre Sparse Matrices en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!