How can I extract the numerical array from a hypercube object?

Hi! I've been using the hyperspectral library for the image processing toolbox for some years now. In the beggining, when a variable of class hypercube was created, it was storing the nomerical array (i.e. the spectral image) in a struct way that you could acces via dot indexing, for instance:

spectral_image = cube.DataCube;

However this seems not to be working anymore. Instead if I try this, I get a string instead of a numerical array. I've cheked and there is a funcion called gather which is supposed to be for this. I have also tried:

spectral_image = gather(cube);

But it is also not working. I get the following error, but my spectral image fits more than enough in my RAM, so I am sure this is not a problem.

Dot indexing is not supported for variables of this type.

Error in hypercube/gather (line 1398)

if isa(obj.BlockedDataCube.Adapter,'images.blocked.InMemory')

^^^^^^^^^^^^^^^^^^^^^^^^^^^

The problem is that I have spectral images stored in files in hypercube class that I saved some time ago with Matlab R2023 version or even earlier, but now I can not access directly to this numerical arrays that previously were perfectly fitting in my RAM.

Any clue about how could I fix this?

Thanks in advance.

Answer 1

[dpb]

The <Version History> section of the doc for R2025a discusses that the new version doesn't contain the actual datacube and that the prior versions will as you've discovered return the property as text. Why on earth that is is bizarre, indeed, it seems. But, it is what it is. Fortunately, they provide a helper function to convert to the expected numeric form. Why this wasn't included as a a builtin feature is anybody's guess.

From that doc page (sans my somewht cheeky side remarks) <vbg> --

"If you have saved hypercube objects created using the hypercube function in older releases prior to R2025a, the DataCube property contains the actual data cube. However, starting from R2025a, the DataCube property will be read as a string. You cannot use the new gather and apply object functions with these hypercube objects. To extract the actual data cube from hypercube objects created in prior releases, use this helper function."

function [datacube,wavelength,metadata] = classConverter(hcube)
   datacube = str2double(hcube.DataCube);
   metadata = hcube.Metadata;
   datatype = metadata.DataType;
   datacube = cast(datacube,datatype);
   wavelength = hcube.Wavelength;
end

Good luck...

[Miguel Ángel]

Thanks a lot. I haven't tried yet, but indeed this is exactly my problem so I'm sure this will help fixing it. This is the information I could not find yesterday.

It is somewhat mesy since now I have different hypercube objects, some of them created with the old way and some with the new way, and I have no way of knowing it. So somehow I have to try an error handling function myself trying to do it one way and if it does not work try the other.

Anyway thanks a lot for the help.

[dpb]

I don't have the TB so never used the object so not familiar with it.

This penchant Mathworks has developed for changing functionality at the expense of backwards compatibility at the immediate release of a new version is quite disconcerting and imposes a real cost on users. It probably won't do any good with the apparent internal change in mindset, but I'd submit a very strong complaint as a user of the toolset.

Unfortunately, there doesn't seem to be a way to attach any user data such as having a Tag property that one could use to identify an object created with either the old or the new version...about all one would be able to do would be to try an operation on the object and catch the result.

I'd suggest this is worthy of an official support request to Mathworks at <Product Support Page> asking for suggested manner in which to be able to interoperate with the two incarnations as well as suggesting the enhancement of having a way to distinguish the two as well as having the helper function be fully incorporated into the product.

Would it be feasible to convert all the old instantiations to the new version? That would imply an irreversible decision to stick with R2025+ and that you don't have to support any legacy systems.

There are enough changes and complaints about R2025 that I've not gone that way yet.

[Miguel Ángel]

Update:

I have already tried to do the proposed method in the Matlab hypercube documentation (i.e. the datacube = str2double(hcube.DataCube); operation). Result is that for a 512 x 512 x 121 spectral image takes aproximately half an hour in a relatively powerful and fast computer. This is completelly useless for me to "fix" the old-way saved spectral images.

I agree about the complaining and I'll try to find a faster solution but for the moment maybe my fastest solution would be to keep a Matlab release that is older than 2025 and convert there all the old-way saved images to normal 3D single format matrices and then I'll decide either if I store the images with the new hypercube format using imhypercube or if I directly forget about Matlab format and store them in any other starndard format like ".h5".

I guess there should be a technical reason for loading the old-way saved cubes in string format instead of numerical array format, but it completelly ruining the thing.

[dpb]

str2double is notoriously slow -- with current MATLAB, you can convert directly and is quite a lot faster.

Try replacing

function [datacube,wavelength,metadata] = classConverter(hcube)
   datacube = double(hcube.DataCube);
   metadata = hcube.Metadata;
   datatype = metadata.DataType;
   datacube = cast(datacube,datatype);
   wavelength = hcube.Wavelength;
end

[Miguel Ángel]

Yes. This way of conveting is indeed faster than using str2double. Yet not very fast, but at least I gain som time. I am not sure why Mathworks help recommended str2double which takes forever. I actually directly cast to single in one line and then build the new hypercube object using the new imhypercube function:

datacube = single(double(hcube.DataCube));

hcbue = imhypercube(datacube,hcube.Wavelength);

For some reason there's not a direct way of converting to single so instead you have to go through double. Yet the difference in computation time is not large.

[dpb]

Once having done the work to scan the text and do the conversion to internal storage form, the cast to single() will be quick, yes. MATLAB doesn't natively support any alternatives to skip the conversion of floating point representations to double. Even in C, using the '%f' conversion format to a &float memory addresss, it's likely the io runtime code actually does the conversion to double internally and then the cast is done on storage.

Answer 2

[Parth Parikh]

Hi Miguel,

I completely understand the frustration that comes with adapting to changes in well-established workflows. Moving forward, the best solution is to adopt the latest MATLAB functions for hyperspectral data, namely imhypercube and geohypercube. These provide seamless integration with current features, improved compatibility, and enhanced data management, positioning your work for future updates and optimizations from MATLAB. The helper function is meant to recover data from legacy hypercube objects created before R2025a, for robust, long-term workflows it’s highly recommended to transition to these newer functions. This way, you’ll be able to take advantage of MATLAB’s ongoing improvements with greater ease and reliability.

Note: For multispectral data, MATLAB provides the corresponding immuticube and geomulticube functions.

[Miguel Ángel]

Hi!

Thanks for the answer.

I agree that keepig up to date with the new Matlab releases is mostly positive. This is the reason I always do. However when a new release makes this kind of non-back-compatible changes, a big and well-stablished workflow might be seriously compromised. That is why thinking in this backcompatibility is always crucial.

Now I am searhing for the fastest way to convert dozens of old hcube objects into the new version, and this will take me too much time. I am still going with the most important ones at least, but the thing is that nothing is really telling me that this will not change again in the future and maybe the conversion work I am doing now will be again useless.

On the other hand, as you can read in the previous answer, the proposed solution through str2double is way slower than directly casting the DataCube to double. This is also not straightforward for a naive Matlab programmer.

[Parth Parikh]

Sure, we will update the documentation.

[dpb]

@Parth Parikh, your response brushes off the real cost imposed on users as inconsequential and immaterial.

Why on earth if this was going to be done was it not in prior release notes as coming and were not efficient tools provided to aid in the transition rather than dumping all the work onto the user unannounced? This is quite rude.

Why, at least, since Mathworks has access to all the internals could not the new methods be able to discern an object of the prior type and directly return the data stored therein rather than the string? Or at least distribute the prior version (perhaps in an aliased form) in parallel so there would be an (at least relatively) painless way to make the transition? As it is now, if one reverts to a prior release to read the data, there isn't a tool to create the new version directly and as @Miguel Ángel noted, the conversion even replacing str2double is nontrivial.

This was just a very bad move on Mathworks' part in the way the transition was rolled out.

[dpb]

@Miguel Ángel - "Now I am searhing for the fastest way to convert dozens of old hcube objects into the new version, and this will take me too much time. .."

Given the lack of coexisting tools in either the prior or present versions (shame on you, Mathworks!), I suspect the best approach will be to write a batch script to run in the prior release that does a search for the saved objects (I'm assuming they're in a .mat file format?) and extracts the data from them and writes to either another .mat or binary file that can then be read in the later release and create the new objects from the raw data. That could be set to be a background or overnight unattended bach job. Painful and shouldn't be necessary to have to go to such extremes, but better than a one-at-a-time approach.

Another idea all within the later release might try would be to write the string data to a file and then read it in and see if the builtin io format conversion routines of the runtime are any faster than the in memory methods. Again, wny they would do this is beyond comprehension from the user's viewpoint, anyway.

Similar idea would be to compare direct sscanf with double().

ADDENDUM

Not having the TB no way to go look at just what the prior code might be, I wonder if it might be possible to move the previous version over to a private directory on the new and give it an aliased name. It may be it's too intertwined with other supporting internals to be possible, but I'd probably give it a look...

[Miguel Ángel]

Thanks @dpb.

Yes, the first approach you proposed is exactly how I will be working, We have some computer in our lab with older Matlab release so I guess it will work. Kind of time consuming but still faster than using concersion/casting from string to numbers.

Thanks a los for your help!

[dpb]

Glad to try, anyway...good luck and let us know how it goes.

I still think this is well deserving a strong ping/complaint about unduly burdening the user with extra overhead cost.

ADDENDUM

Last thought...I think I'd request a working version of the old hcube object code that can be run with the new release, or conversely, a method that can use in the prior release to create the new objects as part of that complaint. It's only what they should have done to begin with.