Find Mean from Duplicate Entries

Hello,

I have a matrix with 2 columns (X,Y) and a large number (hundreds or thousands) of rows. It's a time series, so the first column has year values and the second column, Y, corresponds to an observation at that time. However, some of the dates repeat with different values in Y (due to the fact that I did a "floor" function on the dates, some of which originally had monthly resolution).

What I want to do is locate all dates that are found more than once and take the mean of the corresponding Y values. For example, if I have

    1900 8
    1901 6
    1902 4
    1902 3
    1903 5
    .
    .
    .

I want the output to be,

    1900 8
    1901 6
    1902 3.5
    1903 5
    . 
    . 
    .

Any help would be appreciated! Thanks.

Answer 1

[Walter Roberson]

[uyear, yearjunk, yearidx] = unique( Matrix(:,1) );
OutMatrix = [uyear(:), accumarray( yearidx, Matrix(:,2) ) ];

[Chris]

Walter, thanks but this actually gives me the sum of all the elements in Y corresponding to the repeated years...

[Walter Roberson]

OutMatrix = [uyear(:), accumarray( yearidx, Matrix(:,2), [], @mean ) ];

[Chris]

Thanks!

Answer 2

[T. Guillod]

Thanks for this solution. It helped me a lot.

The solution by Walter is OK but very slow (call to mean()). A better one:

Instead: accumarray( yearidx, Matrix(:,2), [], @mean )

Use: accumarray( yearidx, Matrix(:,2))./accumarray( yearidx, ones(size(Matrix,1),1))./

[Sean de Wolski]

You could skip the accumarray call in the denominator and instead use histc() on the yearidx with edges being uyear. histc() should be faster than accumarray()

[John Yang]

Find this old thread. Your solution is really fast, 10 times faster of using histc than calling @mean.

Just curious, why using histc so fast? In my eye, calling a basic function such as mean should use the same order of time as histc.