Why won't jsonencode encode my entire structure array?
15 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Andrew Mistele
el 4 de Ag. de 2020
Editada: Andrew Mistele
el 13 de Ag. de 2020
I have a 1x2 struct named "Vortices" that contains data outputs for two objects in a simulation, and is approximately 1.2 GB in size. Each struct in the struct array has 6 fields, each of which are 1x6000 struct arrays as well. I want to encode "Vortices" in JSON and write out to a "vortices.json" file, using the jsonencode() function, as below:
file_handle = fopen('vortices.json','w');
str_vortices = jsonencode(Vortices);
fprintf(file_handle,str_vortices);
"Vortices" structure:
However, I have tried this with structure outputs from several different simulations, and found that my file write-out never results in a JSON file larger than 2,097,152 KB. I have backtracked and found that this is because the jsonencode(Vortices) is not encoding the entire structure, as you can see the encoded string output from jsonencode() does not end with any JSON termination characters like "}" or "]":
I know that the structure I'm trying to encode is large, but I'm running MATLAB 64-bit (R2020a) and the sizes of these objects are well below what inputting "memory" indicates is my largest possible array size, 43.8 GB.
Since I haven't exceeded the maxium array size (or my allocated RAM), why would jsonencode() not be encoding the entire structure?
0 comentarios
Respuesta aceptada
Gaurav Garg
el 13 de Ag. de 2020
Hey Andrew,
You can divide your data into small chunks and then enode those chunks (one-by-one) into a JSON file (using jsonencode).
You can start by opening a file in write mode, loading a chunk of data, writing the data into JSON, and repeating the same steps but opening the file in append mode this time.
It will lead to some discrepancies because the resultant JSON file will show the variable names multiple times.
For example -
I assume a struct 'data' with variable 'x' in it. For 2 data chunks, I would get something like -
{"x": values} {"x": values}
But, you can easily write a script to handle this case. HINT: Start off with small example to remove the second "x", "}{" and ":" symbols and then generalize the script for the whole JSON file (with all the chunks of data).
Using this approach, you can encode even larger data without any errors.
1 comentario
Andrew Mistele
el 13 de Ag. de 2020
Editada: Andrew Mistele
el 13 de Ag. de 2020
Más respuestas (0)
Ver también
Categorías
Más información sobre JSON Format en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!