How to handle 1 GB - 2GB .txt files efficiently in matlab, I used textscan. Is it efficient?

Please let me know with any alternative methods to proceed with.

thanks

Answer 1

[Walter Roberson]

What do you need to do with the text file? Is it a text file that consists just of numbers, or does it have additional information? Do you want to read in all of the numbers in it or just some columns?

How often will you need to process any given text file? Sometimes pre-processing it outside of MATLAB to make it easier for MATLAB to read can be time-efficient, and the relative efficiency goes up the more often you re-use the same file.

[Aditya]

text file consists of all kinds of data like char,numbers,special chars. and I deal with retrieval of data in the text file. Help me with this context.

[Walter Roberson]

Could you give an extract of the file and indicate what you would want retrieved from it?

Also please respond about how often you will need to process any given file.

Answer 2

[Aditya]

POINT X1 X2

POINT -124.63-78.433 .... ....

above is how file exists and I need to extract required POINTS.

[Walter Roberson]

Are there lines that start with anything other than POINT ? Is the first line of the file the only one that says

POINT X1 X2

or does that occur every second line? Or does it appear after a group of numeric POINT objects, with those values to be grouped together (e.g., a face of an object) ? Do the numbers in "X1 X2" vary and if so are they significant or can they be ignored?

The numeric values you show all run together, and have variable number of decimal places. Are all of the values negative so there will always be a "-" between the values? If not then do the positive values have "+" or a space before them?

Answer 3

[Aditya]

I just want to search for the respective point and retrieve the data. Only once the file is used and is stored. I did using textscan - cellarrays.