textscan not reading all rows.

Hello,

I am using textscan to read a data file. It works correctly except that it stops on the 10008th row out of roughly 100,000 rows. There are 113 columns.

Is this a limitation of textscan? I tried increasing BufSize, but that doesn't seem to have any effect.

Thank you!

Answer 1

[Star Strider]

Without having your text file, it is not possible to provide specific code to read it. However, it is possilbe that that there is a non-numeric value at the row that stops the function.

There are a few different ways to deal with this, (one of which is the fileread function), however it might be worthwhile to see if something like this will work:

fidi = fopen('YourFile.txt','rt');
k1 = 1;
while ~feof(fidi)
    C = textscan(fidi, '%f%f%f%f', 'HeaderLines',1, 'CollectOutput',true);
    M = cell2mat(C);
    if isempty(M)                                       % Empty Matrix Indicates End-Of-File
        break
    end
    D{k1,:} = M;
    fseek(fidi, 0, 0);
    k1 = k1 + 1
end
Out = cell2mat(D);

This reads the file until it encounters a non-numeric value in a numeric field. It then re-starts the loop and continues until it encounters the next non-numeric value, or until the input is empty (a check to avoid an infinite loop if there is no valid end-of-file indicator), or a valid end-of-file indicator, if one exists. The ‘Out’ variable will then be a continuous matrix of the file contents, if all the rows that it reads have the same number of numeric values, including NaN elements.

Note that the format string and other textscan arguments are generic in this example. Provide the appropriate arguments to textscan to match your file format.

[Christopher Poletto]

Thank you for this. I will check if EOF is reached.

Doesn't the line "fseek(fidi, 0, 0);" cause the textscan to always go back to the beginning of the file?

[Star Strider]

‘Doesn't the line "fseek(fidi, 0, 0);" cause the textscan to always go back to the beginning of the file?’

No. The fseek call causes textscan to go to the next row (in this context). The ‘fidi’ argument is the file ID, and the ‘0,0’ tells it to advance only to the next row. At least that is how it has always worked when I have used this with every other file. The frewind function (that I do not call here) would cause it to begin again at the beginning of the file.