Problem with processing large data set
1 visualización (últimos 30 días)
Mostrar comentarios más antiguos
Andrei-Ieronim Constantinescu
el 9 de Dic. de 2020
Comentada: Andrei-Ieronim Constantinescu
el 17 de Dic. de 2020
I want to train a BiLSTM-CNN (Bidirectional Long Short-Term Memory Convolutional Neural Network) with 203457 timeseries. I am trying to store them in a table with one line and 203457 columns, each cell containing an array with 100,000 values, but my server freezes whenever I run my code. I don’t get any error messages so I am not sure what the problem is and how to fix it. Do you have any idea why this happens and how to solve this?
Thank you!
0 comentarios
Respuestas (1)
Eric Sofen
el 14 de Dic. de 2020
I'm not sure of the BiLSTM-CNN input requirements, but for starters you're probably better off storing those timeseries as rows in a table rather than variables (columns). You've got some 20 billion total values, so you may be hitting memory limits. You may want to look into using a tall variable for out-of-memory computation.
Ver también
Categorías
Más información sobre Sequence and Numeric Feature Data Workflows en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!