Trying to Average Data sets

Hi all - I have two data sets I am trying to average and am a new MATLAB user. The first is 1 column with 14 rows and I want to every two data points to end with 1 column with 13 rows. Example = [1 2 3 4 ...] and final = [1.5 2.5 3.5. ...]

The second is a large data set with N columns and 75000+ rows. I would like to average this set by every 50th point to reduce down to N columns and 1500 rows.

I would appreciate any help ... I have been able to go nowhere with this. Thank you, RLP

Answer 1

[Matt J]

Using your first example:

>> A=(1:14).';
>> result= interp1(A,(1.5:13.5).')

result =

            1.5
            2.5
            3.5
            4.5
            5.5
            6.5
            7.5
            8.5
            9.5
           10.5
           11.5
           12.5
           13.5

The second case would work much the same. See "help interp1".

[R.L.P.]

Thanks Matt, but will this work if the intervals are not evenly spaced. I used example of equally spaced, but data sets are not.

Answer 2

[Image Analyst]

You want to try something a little more advanced than the intuitive brute force looping method? Try blockproc:

clc;
clearvars;
array2D = rand(75000, 5);

tic;
meanFilterFunction = @(theBlockStructure) mean2(theBlockStructure.data(:));
blockSize = [50 1];
blockMeanArray = blockproc(array2D, blockSize, meanFilterFunction);
[rows columns] = size(blockMeanArray)
toc

[Matt J]

It doesn't seem to me that the OP was asking for a block mean, judging by the example s/he posted. Even so, I wouldn't recommend blockproc for block mean calculations. I would recommend this:

function M=downsamp2d(M,bindims)
%DOWNSAMP2D - simple tool for 2D downsampling
%
%  M=downsamp2d(M,bindims)
%
%in:
%
% M: a matrix
% bindims: a vector [p,q] specifying pxq downsampling
%
%out:
%
% M: the downsized matrix

p=bindims(1); 
q=bindims(2);
[m,n]=size(M); %M is the original matrix
newdims=[m/p,n/q];

M=reshape(M,p,newdims(1),q,newdims(2));

M=sum(sum(M,1),3)/(p*q);
M=reshape(M,newdims);

[R.L.P.]

Would the block mean average every 50th point in each column? That is main goal of the second question is to average every 50th point in each column and put into new matrix.

Matt - thanks for suggestions. I moved comment and am trying this code

[Matt J]

No, it would not.

If you do downsamp2d(A,[50,1]) it would average together the first 50 points mean(A(1:50)) and put that in the first spot. Then it would average together the next 50 points mean(A(51:100)) and that would become the 2nd element of the result, etc...

If this is not what you want, you should use interp1, interp2, etc...

[Image Analyst]

My code averages rows 1-50 and puts it in blockMeanArray(1), and then 51-100 go into blockMeanArray(2), and 101-150 go into the third element. It then does that for every column independently so you'll have the same number of columns, but 1/50 the number of rows. That is the block average. If that's not what R.L.P. wants then he/she should clarify. There is also a sliding mean where it moves by one row, not in "jumps" and you can use conv2() for that method.

If he/she really wants interpolation rather then averaging, then the wording should be corrected to accurately reflect that fact. Matt could be right based on the comment that the data is not evenly spaced. It can be difficult to interpret what people want if they don't use precise terminology. For example in one comment they say "average every 50th point in each column" Now literally that would mean output = mean(columnVector(1:50:end)) so that output(1) = the mean of the 1st, 51st, 101st, 151st, etc., and output(2) = the mean of the 2nd, 52nd, 102nd, 152nd etc., but I doubt that is what was meant.

[R.L.P.]

I want averaging for both cases. The first case may or may not be even intervals. I suspect interp1 only works if distance is even between intervals. For both cases, more than often, the data are not equal intervals so I need to average between every 2 rows in Case 1 and output that value to new column. In the second case, I need to average every 50 values and put that into new column. Matt's solution seems to only work when I have equal intervals that are positive. One of my data series goes between positive and negative values, so interp1 does not work. I am currently trying the blockMean, but having difficulty applying it to the first case. Hope this clarifies?? Being new to MATLAB, I am trying to use correct language

[R.L.P.]

I realized confusion - I said every 50th point in earlier comments, but I meant every 50 points just like every 2 points in the first case.

[Image Analyst]

No it doesn't. "Average every 50 values" does not mean the same as "average every 50th point". Let's say you have 153 points.

"Average every 50 values" means output(1) = (signal(1) + signal(2) + signal(3) + ... + signal(50))/50, and so on, averaging elements from 51-100, 101-150, etc.

"average every 50th point" means (signal(1) + signal(51) + signal(101 + signal(151))/4 for the first output element, and so on for every set of 4 numbers from 2 on, 3 on, etc.

"Interpolating every 50th point" would mean that you want to replace (or insert) elements 50, 100, 150, etc. with the average of elements 49 & 51, 99&101, 149&151, etc. If you had another set of data, like an x or time measurement so that the signal was not evenly sampled but sampled at weird times, then you can use interp1 to get estimated signal that IS uniformly sampled, as Matt suggested. Of course you could do this for every point, not just every 50th point.

So, which is it?

[R.L.P.]

Average every 50 values is what I am looking for ... sorry for the confusion. I wasn't clear enough in my question

[Image Analyst]

That's what my blockproc demo above does. The usual way most people would do it is to extract a column at a time, and then reshape that column into a 2D matrix with 50 columns and then average across the columns. Then repeat for all of the other 4 columns. 6 of one, half a dozen of the other.

Answer 3

[Andrei Bobrov]

[ii ii] = ndgrid(ones(50,1),(1:1500)');
[i1 i2] = ndgrid(ii(:),1:N);
out = accumarray([i1(:) i2(:)],yourmatrix(:),[],@mean);

other variant

ii = ndgrid((1:1500)',ones(50,1));
[i1 i2] = ndgrid(ii(:),1:N);
out = accumarray([i1(:) i2(:)],yourmatrix(:),[],@mean);

[R.L.P.]

What is 'N' in this example?

[Andrei Bobrov]

The initial array yourmatrix with size (75000 x N).