remove duplicate rows from text file
6 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
I'd be glad to have some advise on the fastest way to read a text file which contains duplicate rows, remove all the duplicates and write it back.
Matlab version R14 SP3
2 comentarios
Paolo
el 19 de Jul. de 2018
What does the text file look like? Could you attach it to your question?
Respuestas (1)
Guillaume
el 19 de Jul. de 2018
lines = strsplit(fileread(yourtextfile), '\n')); %read file and split into lines
lines = unique(lines, 'stable'); %remove duplicate lines
fid = fopen(newfilename, 'w'); %open file for writing
fwrite(fid, strjoin(lines, '\n'), 'char'); %merge lines and write
fclose(fid);
4 comentarios
Guillaume
el 19 de Jul. de 2018
Matlab version R14 SP3
Wow! You're only 25 versions behind the current version, soon to be 26.
While it's probably fairly easy to write a version of strsplit using regexp that would work with your version and strjoin is probably not too complicated either, you'd also have to write your own unique function since the 'stable' option didn't even exist back then. That's a more complicated task particularly as I'd have to pore over the online documentation to find out what was possible or not back then.
I'm sorry you're on your own there. At some point you've got to let go of legacy code.
Walter Roberson
el 20 de Jul. de 2018
lines = strsplit(fileread(yourtextfile), '\n')); %read file and split into lines
[ulines, ia] = unique(lines);
lines = ulines(sort(ia));
fid = fopen(newfilename, 'w'); %open file for writing
fprintf(fid, '%s\n', lines{:});
fclose(fid)
Together with
function S = fileread(filename)
fid = fopen(filename, 'rt');
if fid < 0
error('file does not exist');
end
S = fread(fid, [1 inf], '*char');
fclose(fid);
function C = strsplit(S, delim)
C = regexp(S, delim, 'split');
Ver también
Categorías
Más información sobre Data Import and Export en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!