How to calculate average value in each row of data (Not a matrix) ?

Hello ^^

I have a lot of different file like this

File 1 : j1p001.txt

90      1.456   0.998   0.114  NaN   1.002  1.987   ..
100     0.411   1.223   0.01   NaN   0.201   ..     ..   ..
-98     1.446   0.998   0.023  NaN   3.244   0.88   ..   ..
-89     1.098   3.113   1.01   NaN   0.234   0.01  NaN   0.234
..
..

File 2 : j1p002.txt

And other file have the same structure, but with different number of rows and columns.

I want to calculate the mean value in each row (but it start at column 2 to end of the column) and save it in new file.

i just code something but it still error

lc;

clear;

format short;

Y = dir('D\Reformat new file\done\j1p*.txt');

for A=1:length(Y);

H = Y(A).name; gg = H(:,1:7);

data=load(H);

ref=data(:,1);

index_data=find(~isnan(data));

data_fix=data(index_data,1);

n=length(data);

out=mean(data_fix,2);

result=[ref out];

file_name = [gg '_result.txt'];

dlmwrite(file_name,result,'delimiter','\t');

end

Thank you

[Rik]

You're using data before it is defined.

Also, why are you using mean as a variable name? The mean function will to what you want, so why not use it instead?

[Retno Purwaningsih]

Yess, thanks for the correction. I have fixed it but it still doesn't work. Can you point out where is the problem? Thanks

[Rik]

You are not using valid Matlab syntax ("+..+" doesn't exist), and you aren't using the mean function. It can replace the entire loop.

Have you done a basic Matlab tutorial?

[Retno Purwaningsih]

thanks for the comment ^^

I will fix that, but i still doesn't understand how to use function mean if it has a different number of column in each row.

[Rik]

Based on your last edit, I would say you have found out how to calculate the mean. You might be interested in the 'omitnan' flag.

Also, arrays in Matlab always have the same number of columns for each row, so I don't know what you mean with your last remark.

[Stjepan Mamusa]

Hi @Retno Purwaningsih , can you verify if the answer I gave is working for you

Answer 1

[Stjepan Mamusa]

Here's my proposal. First read all the files and separate good from bad. Then, do the work on healthy files.

Click the green arrow to test it out in place ( you can copy - paste the code in a new answer and run it there, but it's better to copy it and run it locally on your files ).

The file j1p404.txt has errors created by me, to test if error detection will work.

UPDATE: Edit with @Rik's suggestion to use omitnan flag, it will give better results, as my solution meant the NaN's are zeroes actually.

clear;
clc;
format short;

file_list = dir('j1p*.txt');

ok_files = dir('');
nok_files = dir('');

% Check file list for errors in files
for i = 1 : length( file_list )
    try
        tmp = load( file_list(i).name );
        ok_files(i) = file_list(i);
    catch
        % In case of error remove the file from list
        fprintf('Error occured with file %s\n',file_list(i).name );
        nok_files(i) = file_list(i);
    end
    clear tmp;
end

Error occured with file j1p404.txt

file = fopen('ok_files.txt','w');
fprintf(file,'%s\n', ok_files.name);
fclose(file);
file = fopen('nok_files.txt','w');
fprintf(file,'%s\n', nok_files.name);
fclose(file);

clear i file nok_files;

for i = 1 : length( ok_files )
    data = load( ok_files(i).name );
    averaged = zeros( size(data,1), 2);
    averaged(:,1) = data(:,1);
    for j = 1 : size(data,1)
       averaged(j, 2) = mean( data(j, 2:end), 'omitnan' );
    end
    file = fopen(strcat('averages_', ok_files(i).name ),'w');
    fprintf(file,'%12s %12s\n','My Variable','Average');
    fprintf(file,'%12.6f %12.6f\n', averaged' );
    fclose(file);
    disp( ok_files(i).name );
    disp(averaged);
end

j1p001.txt

   90.0000    1.1114
  100.0000    0.4613
  -98.0000    1.3182
  -89.0000    1.0930

j1p002.txt

   90.0000    1.1114
  100.0000    0.6602
  -98.0000    1.3023
  -89.0000    1.0930

j1p003.txt

   90.0000    1.3892
  100.0000    1.0178
  -98.0000    1.4896
  -89.0000    1.0792

This will create a list of healthy (ok) files and, a list of files where you had issue with measurement. It will calculate all of your row averages and save them in a separate file.

I have attached all the files I used.

I took the liberty to create some dummy data in the form of j1p*.txt files, all the rest except script.m are generated by the script.

If this solved your problem, please mark the question as answered.

[Retno Purwaningsih]

I want to ask again, what should i do if i want to calculate the nok_files? Because, most of my file is created in the nok_files.

Thank youu

[Stjepan Mamusa]

Hi @Retno Purwaningsih

nok_files is what I called files that are, well, not okay in some way. Those are mainly files that have some issues in formatting or missing values, so the best thing would be to fill in those missing values with NaN variables.

[Retno Purwaningsih]

Could you help me to fill those missing value?