Stroring data from text file into cell array

Hi everyone,

Apologies for the stupid question, I am new to extracting data from txt files. Here is the problem, I have a lot of data stored in a txt file (in total rows) which is of the following form:

802|201308|9|204307|40140|000|1|I|107|999|154000|107|4.125|R|N|FRM|CA|SF|92400|F113Q|N|360|01|Norges, N.A.|Norges, N.A.||F108X|

My attempt so far has been:

fileID = fopen('thetextfile.txt');
formatSpec = '%s';
N = 27;
C_text = textscan(fileID,formatSpec,N,'Delimiter','|');
fclose(fileID);

I am happy how the data has been stored, the only problem is that just the first row of the text file has been saved in C_text. In other words, C_text is a 27×1 cell array. The goal would be to have a 27x>2000000 celly array, which I can then manipulte further. I am not sure whehter this is feasible given the size of the txt file. If there is a better and more efficient way to do it then please let me know. Any help is appreciated.

Many thanks!

Robert

Answer 1

[Cris LaPierre]

I would suggest using readtable. It makes your data much more accessible. If there is a header row, it will use those names as the variable names (each column in a table is a variable). Without them, it assigns names (see below). You can then access your data using dot notation (table.variable).

Here's an example using the sample data you provided. It should be able to scale to handle your complete data set.

data=readtable("thetextfile.txt","Delimiter","|")

data = 1x28 table
    Var1       Var2       Var3       Var4       Var5     Var6    Var7    Var8     Var9    Var10     Var11      Var12    Var13    Var14    Var15     Var16     Var17     Var18     Var19      Var20      Var21    Var22    Var23         Var24               Var25          Var26      Var27      Var28
    ____    __________    ____    __________    _____    ____    ____    _____    ____    _____    ________    _____    _____    _____    _____    _______    ______    ______    _____    _________    _____    _____    _____    ________________    ________________    _____    _________    _____

    802     2.0131e+05     9      2.0431e+05    40140     0       1      {'I'}    107      999     1.54e+05     107     4.125    {'R'}    {'N'}    {'FRM'}    {'CA'}    {'SF'}    92400    {'F113Q'}    {'N'}     360       1      {'Norges, N.A.'}    {'Norges, N.A.'}     NaN     {'F108X'}    {' '}

data.Var3

ans = 9

You can see more about accessing data in a table here.

[FR2020]

Thanks Cris, this worked perfectly! Believe it or not, I also used readtable() at some point but it took quite a long time due the size of the txt file and hence I didn't pursue with it. Thanks again for your help! Robert

[Cris LaPierre]

It does try to do a lot of autodetecting under the hood, so sometimes performance can be improved by setting more of the import settings (data type, format, etc.) using detectImportOptions.

[FR2020]

Thanks Cris, I will give that a try.