Reading data from a multiple csv's in a folder and read until end of a specific column and not starting from top of the file

So I have maybe a dozen folders and inside each folder is another folder containing a varying number of data files. I've been doing some digging and found that dir is probably my best option because the number of files to be read in will be different and I aim to use a for loop. In each file there are 2 columns that I am interested in and both start on the same row and each file has a different number of rows so the end row is different for each loop. I found some code that helped, but I still can't quite get it. Below is the code I found and what I tried. I got internal size mismatch.

I want the 2nd and 4th zero based columns read from the 4th row to the end of the column and stored into pos and time and each loop will do the same for the next file in the folder which will be taken from the struct. The loop will be completed when the files in a folder have been read then I will restart the code and change the directory to the next one.

    cd(uigetdir());
    dd = dir('*.DAT');
    fileNames = {dd.name}; 
    data = cell(numel(fileNames),2);
    data(:,1) = regexprep(fileNames, '.csv','');

    for ii = 1:numel(fileNames)

      data{ii,2} = csvread(fileNames{ii},4,2,[4,2,end,2]);
      data{ii,3} = csvread(fileNames{ii},4,4,[4,4,end,4]);
      pos=data{ii,2};
      time=data{ii,3};  

    end

And here's the top of one of the csv files I'm working with, I also attached a csv.

    "TOA5","laser","CR9000X","1085","CR9000X.Std.06","laser","10248","Table1"
    "TIMESTAMP","Distance","maxDist","mmPerSec","ticker","startTime","maxTime"
    "TS","mm","","","","",""
    "","Smp","Smp","Smp","Smp","Smp","Smp"
    "2016-03-15 13:45:03.34",0.3204956,2,0,0.01,0,0
    "2016-03-15 13:45:03.35",0.1595154,2,0,0.02,0,0
    "2016-03-15 13:45:03.36",0.004302979,2,0,0.03,0,0
    "2016-03-15 13:45:03.37",0.2714233,2,0,0.04,0,0

Edit: added csv file and below text for clarification

I want to assign column's 2 and 4 separately from row 4 to the end to 2 different arrays. These 2 columns are to be read from roughly 100 files of the same format with differing lengths, i.e. file 1 has 6000 rows, file 40 has 10000 rows. How can I program matlab to read all of these files and select the same 2 columns starting at the same row and read to the bottom which is a different row in each file?

Answer 1

[Mohammad Abouali]

could you upload a sample CSV file!

note that pos and time are both overwritten in each loop. So what's their purpose?

Also you define data as:

data = cell(numel(fileNames),2);

but then reference it as:

data{ii,3}

By the way, Are you sure the provided code doesn't produce error on:

data{ii,2} = csvread(fileNames{ii},4,2,[4,2,end,2]);

I don't think you can provide "end" in that format.

one of the cases that results in "internal size mismatch" error is once you try to read more columns/rows that there are in the file. So, if your code has 10 rows, and you specify any number more than 9 (the numbering starts from 0) you would get that error.

[Paul McDonald]

I uploaded it, I figured the sample csv I provided in the original post was sufficient enough as it contains the important part, after that the length just changes from file to file.

I'm going to use pos and time to get velocity and plot it and save the plot and do the same in the next loop so it's a waste of memory to store the values after the plot is made and saved. I did not completely finish writing my code, I was working on just figuring out how to assign the data I want to the appropriate arrays.

I define data{ii,3} in line 10.

That's exactly where I'm getting my error so I figured I couldn't use "end" as a command, hence the error. I'm searching for a replacement for end with this question post. I want to assign column's 2 and 4 separately from row 4 to the end to 2 different arrays. These 2 columns are to be read from roughly 100 files of the same format with differing lengths, i.e. file 1 has 6000 rows, file 40 has 10000 rows. How can I program matlab to read all of these files and select the same 2 columns starting at the same row and read to the bottom which is a different row in each file?