How can I measure similarity between two signals with different lengths (and with different time points)?

Hi all,

I want to measure similarity between two signals with different lengths.

Following is my code.

 A = [1.2,1.5;
     2.4,0.5;
     3.2,1.5; 
     4.1,1]
 B = [1,1;
      2,0;
      3,1;
      4,0.5;
      5,1.5]
 BB = [1,1;
      2,0;
      3,1;
      4,0.5;
      5,1.5] 
 C = [1.1,1.2;
     2.2,1.3;
     3.3,1.5; 
     4.2,1.7]
%%%%% B and BB are the same signal 
plot(A(:,1), A(:,2)) % first column is the x-values
hold on;
plot(B(:,1), B(:,2))
figure(2)
plot(C(:,1), C(:,2)) % first column is the x-values
hold on;
plot(BB(:,1), BB(:,2))
 
%%% Similarity between B and A
B=imresize(B, size(A))
r_BA = corr2(B,A)  % Correlation between B and A
maxB = max(B)
maxA = max(A)
nor_B = bsxfun(@rdivide,B,max(B))
nor_A = bsxfun(@rdivide,A,max(A))
nor_r_BA = corr2(nor_A, nor_B)   % Normalized Correlation

%%% Similarity between B and C 
BB=imresize(BB, size(C))
r_BBC = corr2(BB,C)   % Correlation between BB and C
maxBB = max(BB)
maxC = max(C)
nor_BB = bsxfun(@rdivide,BB,max(BB))
nor_C = bsxfun(@rdivide,C,max(C))
nor_r_BBC = corr2(nor_A, nor_BB)  % Normalized Correlation

%plot between B and A

Plot between B and C

It looks like B and A have higher similarity than B and C.

However, the correlation (corr2) value between B and C is greater than B and A.

Normalized correlation values are the same for both.

Please let me know how I can measure similarity of the 2 signals above?

I can accommodate R, SPSS or any other tool to solve this problem.

Answer 1

[Ameer Hamza]

If you plot the signals after applying imresize you will see that A and B are not very similar to each other. It appears from the discussion on different forums, there is no universal way to address the issue of correlation between vectors of unequal length. The most appropriate strategy seems to be getting rid of extra data from signals to make them of equal length.

https://stackoverflow.com/questions/9973432/correlation-between-vectors-of-different-lengths

https://stats.stackexchange.com/questions/3589/correlation-between-two-variables-of-unequal-size

[SOPAE YI]

Thank you so much for your suggestion, Sir :--)

It can be one of the solutions, however, if both vectors are time series and the measurements are not taken at the same time points (as shown in the plots above), you cannot simply remove the data points of the longer signal.

[Ameer Hamza]

Yes, therefore, I mentioned there is no universal way, the solution will differ from case to case. One of the answers in the posted links mentions about upsampling or downsampling the of the timeseries to bring them at the same level is one of a solution.

[SOPAE YI]

Thank you so much Mr. Hamza :--)

I think upsampling/ downsampling can solve my problem. Maybe, I can duplicate the last point of A to match with B like the following:

A = [1.2,1.5;

2.4,0.5;

3.2,1.5;

4.1,1;

4.1,1;

4.1,1;];

1. What do you think?

If I may ask one more question,

2. Is the Normalized Correlation formula below correct?

maxA = max(A);

maxB = max(B);

nor_A = bsxfun(@rdivide,A,max(A));

nor_B = bsxfun(@rdivide,B,max(B));

corr2(nor_B, nor_A)

[Ameer Hamza]

I think it is better to remove the extra element from A than artificially adding an extra element in B. Since information is not already present, so artificially creating like this might not help.

As for the Normalized Correlation, it usually referred to the correlation coefficient, and you can directly use the MATLAB's function corrcoef: https://www.mathworks.com/help/matlab/ref/corrcoef.html. However, it does not involve division by the maximum value. It normalizes the vectors using mean and variance values.

[SOPAE YI]

Thank you so much for your answer, Mr. Hamza :--)

As for removing the extra elements, if both vectors are time series and the measurements are not taken at the same time points (as shown in the plots above), you cannot simply remove the data points of the longer signal. So, I am hesitant to remove the extra elements.

And.... I cannot decide which portion of the signal is 'extra' (beginning, middle or final portion?).

[Ameer Hamza]

In that case, I think it will be better for you to use interp1 instead to imresize to match the dimensions of the signals. For example, check this

A = [1.2,1.5;
    2.4,0.5;
    3.2,1.5;
    4.1,1];
B = [1,1;
    2,0;
    3,1;
    4,0.5;
    5,1.5];

figure();
plot(A(:,1), A(:,2)) % first column is the x-values
hold on;
plot(B(:,1), B(:,2))

B_new = [A(:,1) interp1(B(:,1), B(:,2), A(:,1), 'linear', 'extrap')];
figure();
plot(A(:,1), A(:,2)) % first column is the x-values
hold on;
plot(B_new(:,1), B_new(:,2))

Here I changed the dimension of B to match with A. You can also try the reverse. Note, this is not equivalent to removing some elements. This method creates a linear interpolation model from the B matrix and then gives the value at x-values of matrix A using that model.

[SOPAE YI]

Thank you so much, Mr. Hamza :--)

‘Interp1’ works WONDERFUL!!

I am very much appreciative of your sharing valuable expertise, sir.

My next problem is clustering the 2-dimensional time series signals with the similarity measure (corr2) you helped me use.

I have posted a question regarding this issue:

https://www.mathworks.com/matlabcentral/answers/510541-how-can-i-use-multidimensional-matrices-as-input-for-clustering-in-matlab

[Ameer Hamza]

SOPAE, check my answer to that question.

[SOPAE YI]

Thanks again Mr. Hamza :--)

I have been experimenting with ‘interp1’ function and found out that Matrix B is losing it’s time information after ‘interp1’. However, the time of each point is important in my research. Therefore, to keep the time information of both A and B, maybe I might be able to use maximum value of cross-correlation as a similarity measure.

I have calculated like below:

>> max(max(normxcorr2(A, B)))

ans = 0.9966

>> max(max(normxcorr2(C, B)))

ans = 0.9413

I do not think this is reasonable result because the scores (0.9966 vs 0.9413) are not different enough to show the difference between the signals properly.

I do not have to keep the actual time values, I just want to keep the proportional time interval between points (points within 1st column). (e.g. Z = [1, 1.2; 6, 2.1; 8, 3.4 ] can be rewritten as ZZ = [1, 1.2; 4, 2.1; 5, 3.4 ] . As they still have the same ratio (3 to 1) of time intervals, they have the same time information.

How can I compute correlation while keeping the time information?

[Ameer Hamza]

If you are don't want to loss the values in matrix in A and C, than you can also apply interp1 in reverse. For example check this

A = [1.2,1.5;
    2.4,0.5;
    3.2,1.5;
    4.1,1];
B = [1,1;
    2,0;
    3,1;
    4,0.5;
    5,1.5];
C = [1.1,1.2;
     2.2,1.3;
     3.3,1.5; 
     4.2,1.7];
 
A_new = [B(:,1) interp1(A(:,1), A(:,2), B(:,1), 'nearest', 'extrap')];
C_new = [B(:,1) interp1(C(:,1), C(:,2), B(:,1), 'nearest', 'extrap')];

figure();
plot(A_new(:,1), A_new(:,2)) % first column is the x-values
hold on;
plot(B(:,1), B(:,2))
figure();
plot(C_new(:,1), C_new(:,2)) % first column is the x-values
hold on;
plot(B(:,1), B(:,2))

corr2(A_new(:,2), B(:,2))
corr2(C_new(:,2), B(:,2))

This will make both A and C have dimension of 5x2. The time value (first column) will be same for A, B, and C.

I don't know much about normxcorr2 so cannot comment whether it is suitable for this situation.

[SOPAE YI]

Thank you so much for help, sir.

You code still lost the time information (original 1st column data) of A and C since all 3 matrices are using the 1st column of B as their 1st column.

Is there a way to keep the ratio of time intervals of all 3 matrices and compute correlation between them?

Note:

I do not have to keep the actual time values, I just want to keep the proportional time interval between points (points within 1st column). (e.g. Z = [1, 1.2; 6, 2.1; 8, 3.4 ] can be re-written as ZZ = [1, 1.2; 4, 2.1; 5, 3.4 ] . As they still have the same ratio (3 to 1) of time intervals, they have the same time information.

[Ameer Hamza]

You don't need to use B(:,1) as first column of matrix A_new. You can give you own time values to interp1,

A = [1.2,1.5;
    2.4,0.5;
    3.2,1.5;
    4.1,1];
B = [1,1;
    2,0;
    3,1;
    4,0.5;
    5,1.5];
C = [1.1,1.2;
     2.2,1.3;
     3.3,1.5; 
     4.2,1.7];

A_x = [1.2 2 2.9 3.9 5]';
A_new = [A_x interp1(A(:,1), A(:,2), A_x, 'nearest', 'extrap')];

However, my example manually inputs time values. You can use your own rule to automatically generate A_x, as you mentioned about keeping fix time ratios in your comment.

[SOPAE YI]

Thanks a million, Mr. Hamza :--)

I think your suggestion is a brilliant idea.

I will try to figure out a rule to automatically generate A_x.

By the way, I am wondering how you got A_x = [1.2 2 2.9 3.9 5] out of A = [1.2 2.4 3.2 4.1]? (extracting 5 elements out of 4 time points).

Suppose,

E = [1.2,1.5; 2.4,0.5; 3.2,1.5]

B = [1,1; 2,0; 3,1; 4,0.5; 5,1.5];

To compute E_x,

The 1st point is ‘1.2’

and…

(2.4-1.2)/(4.1-1.2) x (5-1.2) = 1.57 so, 2nd point is (1.2+1.57)

(3.2-2.4)/(4.1-1.2) x (5-1.2) = 1.04 so, 3rd point is (1.2+1.57+1.04)

How can I compute the 4th point?

(of course the 5th point is 5, the last point.)

Since I have signals with various lengths (of time), each and every size of my matrices should be converted to be matched with the size of the largest matrix I have.

I am thinking of making a function turning matrices into the size of the largest matrix using ‘interp1’. (input: a matrix, output: a new matrix with the same size of the largest matrix) to be used to cluster a bunch of matrices.

[Ameer Hamza]

Glad to be of help. Best of luck with your work.

[SOPAE YI]

Mr. Hamza,

You've been such a great help!!

I am very much appreciative of your sharing your expertise, sir.

Stay healthy and happy!!!