readtable with fixed column width from a txt file

Hi there,

So I have a text file that is generated from another computational code and looks like this:

What I'd like to do is to import this file into Matlab with proper data structure and pretty much use the key value pair to perform quick query about the "Matrix Eqn" with given (Node, DOF) keys. I realized readtable() is potentially a good candidate to import the file. But when I use the command, Matlab (I'm using Matlab R2014b) returns a warning like this:

>> T = readtable('Stiff_HB.mapping','FileType','text');
Warning: Variable names were modified to make them valid MATLAB identifiers.

And the table T looks like this:

>> T.Properties

ans = 

               Description: ''
      VariableDescriptions: {'Original column heading: 'Matrix Eqn          Node    DOF''}
             VariableUnits: {}
            DimensionNames: {'Row'  'Variable'}
                  UserData: []
                  RowNames: {}
             VariableNames: {'MatrixEqnNodeDOF'}

A possible solution for this is to first import the text data file into excel and set the column width constant, Excel will then divide the data set into a 3-column like structure. By calling readtable again with the Excel file as an input argument, the table would recover all three properties that I want. The warning shown below is because the property name "MatrixEqn" originally has a whitespace. However, reading from Excel file is way slower than reading from text file.

>> T = readtable('Stiff_HB_mapping.xlsx');
Warning: Variable names were modified to make them valid MATLAB identifiers. 
>> T.Properties

ans = 

               Description: ''
      VariableDescriptions: {'Original column heading: 'Matrix Eqn''  ''  ''}
             VariableUnits: {}
            DimensionNames: {'Row'  'Variable'}
                  UserData: []
                  RowNames: {}
             VariableNames: {'MatrixEqn'  'Node'  'DOF'}

Also, there are possibly thousands of such text files for processing and it's really not a good idea to introducing an excel interface for data importing. Thus, I wonder if it is possible to simply get the job done in Matlab solely. Thanks in advance for any tips and suggestions.

[Stephen23]

Is it necessary to use readtable for this? It would be trivial to use textscan with its MultipleDelimsAsOne option.

[Fu-Rui Xiong]

Thanks Stephen, you are right, textscan could solve the problem!

[Walter Roberson]

It looks to me as if you have tab delimited fields.

Answer 1

[dpb]

The warning is simply readtable complaining about the space in the first column heading; it's not smart enough to just smush the two strings together or insert and underscore or somesuch. Just use the optional 'Name','Value' arguments to handle it...

T = readtable('Stiff_HB.mapping','FileType','text', ...
                                 'ReadVariableNames',0, ...
                                 'Delimiter',' ');     % ignore headerline w/ malformed variable name
T.Properties.VariableNames={'MatrixEqn','Node','DOF'}; % set usable variable names
T.DOF=categorical(T.DOF);                              % convert last column to categorical

As Stephen notes, you could read the file with textscan but readtable reads using a filename directly eliminating the extra step of obtaining and subsequently releasing a valid file handle and return desired table albeit here you need fixup the variable name. But, you'd have to do that if read it via any other form as well as the problem is inherent in the name as written with the embedded blank that isn't legal Matlab variable name.

I don't understand why readtable doesn't have a larger subset of the rest of the features of table in the list; I suppose TMW figures if you have just enough to get the data in you can cleanup later; or perhaps, just haven't yet gotten it fully implemented to date.

[Fu-Rui Xiong]

Hi there,

Thanks for the useful hint. The code runs with an error like

    Error using readtable (line 129)
    Each line of a text file must have the same number of delimiters.

However, when I made the following adjustment, everything works okay.

T = readtable('Stiff_HB.mapping','FileType','text', ...
                                 'ReadVariableNames',0, ...                                 
                                 'HeaderLines',1,...
                                 'Format','%d %d %s'); % ignore headerline w/ malformed variable name
T.Properties.VariableNames={'MatrixEqn','Node','DOF'}; % set usable variable names
T.DOF=categorical(T.DOF);                              % convert last column to categorical

>> T.Properties

ans = 

               Description: ''
      VariableDescriptions: {}
             VariableUnits: {}
            DimensionNames: {'Row'  'Variable'}
                  UserData: []
                  RowNames: {}
             VariableNames: {'MatrixEqn'  'Node'  'DOF'}