Extending compound HDF5 dataset

Dear Forum,

I am trying to extend a dataset in Matlab with the low-level API. The underlying datatype is compound.

% create file
FILE    ='myFile.h5';
file    = H5F.create(FILE, 'H5F_ACC_TRUNC', 'H5P_DEFAULT', 'H5P_DEFAULT');

% create compound datatype
memtype = H5T.create ('H5T_COMPOUND', 32);
H5T.insert (memtype, 'serial_no',   0 ,'H5T_NATIVE_DOUBLE');
H5T.insert (memtype, 'location',    8, 'H5T_NATIVE_DOUBLE');
H5T.insert (memtype, 'wurz',        16, 'H5T_NATIVE_DOUBLE');

% create compound datatype
filetype = H5T.create ('H5T_COMPOUND', 32);
H5T.insert (filetype, 'serial_no',   0 ,'H5T_NATIVE_DOUBLE');
H5T.insert (filetype, 'location',    8, 'H5T_NATIVE_DOUBLE');
H5T.insert (filetype, 'wurz',       16, 'H5T_NATIVE_DOUBLE');

% create unlimited filespace
H5S_UNLIMITED = H5ML.get_constant_value('H5S_UNLIMITED');
maxdims = [H5S_UNLIMITED];
memspace = H5S.create_simple(1, fliplr(0), fliplr(maxdims));

% create data set
dcpl = H5P.create('H5P_DATASET_CREATE');
H5P.set_chunk (dcpl, fliplr(1));

dset = H5D.create(file, 'DS',filetype ,memspace, dcpl);
bSpace = H5D.get_space(dset);
%dset = H5D.create (file, 'DS', filetype, memspace, 'H5P_DEFAULT');

% fill element-wise
for i=1:10

    % create test data
    wdata.location      = i;
    wdata.serial_no     = i*10;
    wdata.wurz          = -i*10;

    % extend data set
    H5D.extend(dset, i);
    space = H5D.get_space(dset);    
    H5S.select_all(space);

    % select hyperslab
    H5S.select_hyperslab(space,'H5S_SELECT_SET', i-1, 1, 1 ,1);

    % write data       
    H5D.write(dset,  memtype, 'H5S_ALL', space, 'H5P_DEFAULT',wdata);

    % close file space
    H5S.close (space);

   end

H5D.close(dset);
H5T.close (filetype);
H5T.close (memtype);
H5F.close(file);

Unfortunately my program does not do what I intend: Instead of filling element-wise, it will only copy the first compound data to the hdf5-file:

Output of dataset DS from HDF5 viewer: 

serial_no  location wurz
1.0 10.0 -10.0 
0.0 0.0 0.0 
0.0 0.0 0.0 
-1.288480843725888E-231 -3.786504882356714E-270 -3.786504882356956E-270 
4.9E-324 9.8117197E-315 9.81183448E-315 
4.5792095E-316 9.8117197E-315 9.81183448E-315 
9.792595565E-315 9.8117197E-315 9.81183448E-315 
8.4E-323 4.9E-324 1.353122016E-315 
9.79259493E-315 8.49433059E-315 2.121477727E-314 
8.4E-323 9.790301677E-315 5.295032923E-315

Can anybody help?

Thanks,

Daniel

Answer 1

[MrDan]

I found the mistake myself by studying this: https://de.mathworks.com/matlabcentral/newsreader/view_thread/284377

Finally, replacing this

    memspace = H5S.create_simple(1, fliplr(0), fliplr(maxdims));

through this

memspace = H5S.create_simple(1, fliplr(1), fliplr(maxdims))

does the trick.

[Markus Krug]

I have exactly the same problem but the above mentioned change did not solve the 'random' numbers for the extend dataset starting from row 4. Could you please post you solution.

Additonally I'm interested in extending compound datasets that are compressed. Did you do something similar? If yes any hint will be appreciated

[bowen li]

To fix the random number bug, replace

H5D.write(dset, memtype, 'H5S_ALL', space, 'H5P_DEFAULT',wdata);

throuth

memspace_id = H5S.create_simple(1,1,[]);
H5D.write(dset, memtype, memspace_id, space, 'H5P_DEFAULT',wdata);