TIME SERIES COMPARISON WITH XCORR

Good afternoon: I have two time series datasets. Thier fields are time (in seconds from 1900) and salinity. The data sets are from sensors (J and G) about 9km apart. The J set is about 8500 samples longer than the G set. I need to correlate the two time find maximum lag at the highest correlation coefficient. The idea is to then use this information to derive velocity of a water parcel. I am using xcorr, and have followed the example on the 'xcorr' doc page closely, as it is a similar application. However, I get a plot that ends up being far too perfect for it to be right, (attached). I believe I need to interpolate time into the calculation somehow (which I have not done), but I am unsure how best to do so. I have researched numerous methods from answered questions up here, including some time interpolation techniques, but I am spinning my wheels.

 % XCORR
 FsJ = 0.02350  %Samples per second at J over the period
 FsG = 0.01975  % Samples per second at G over the period
 avFs = (FsJ+FsG)/2

 [corr,lag] = xcorr(s_salinityj,s_salinityg);
 [~,I] = max(abs(corr));
 cor = corr/max(corr);

 [MJG,IJG] = max(corr);
 tJG = lag(IJG);

 lagDiff = lag(I)
 timeDiffJ = lagDiff/FsJ
 timeDiffG = lagDiff/FsG
 timeDiffav = lagDiff/avFs

 figure
 set(gcf, 'Position',[10 50 2075 1000])
 plot(lag,acor,[tJG tJG],[-0.1 1],'r:')
 text(tJG+100,0.9,['MAX LAG IS:  ' int2str(tJG),' samples'])
 ylabel('Correlation  Coefficient_{J-G}')
 xlabel('Samples, (x10^4)')if true

I could use a shove in the right direction. Thank you.

[Jyotish Robin]

Could you give a sample of the data (s_salinityj,s_salinityg)?

[NATHAN MURRY]

Good morning Jyotish: Thank you for the response. I have attached two files with samples of the data I am working with from 2 moorings, J and G. In both cases, the first field is time in seconds from 1900, and the last (4th) field is salinity. The two middle fields are depth/pressure, which am not using in the calculations yet.

--NMM

[Jyotish Robin]

My hunch is that the correlation results are right and the non-bumpy nature may be because of the nature of the 2 datasets.

[NATHAN MURRY]

Good morning Jyotish: Thank you again for getting back to me. What do you mean by the 'nature of the two data sets'? The numerical results make some degree of sense (sample lag and current velocities), but the plot looks way too perfect to be correct. I also tried applying my data to the 'delay between correlated audio signals' example on the XCORR page, as the applications seemed very similar, but the resulting plot looks nothing like the example; I would have expected at least some similarity. I also tried the 'signal alignment' example for XCORR, but it actually threw the signals further out of alignment. The data is direct from the sensor, so I am not sure how else to process it for a proper XCORR applications. Do you have any suggestion on a possible next step? Thank you.

[David Goodmanson]

Hi Nathan,

Could you comment on the data? It appears that the sensor works in a cycle where it takes data for a few minutes and is inactive for about the next 2 1/2 hours. That will certainly have a large effect on any xcorr calculation.

[NATHAN MURRY]

Hi David: Thank you for the response. You are correct. The sensors are attached to an undulating apparatus on 2 static moorings, 'J' and 'G'. Approximately every 2 hours, the apparatus dives to around 400m, taking numerous readings as it goes. Mooring J is about 9km northeast of mooring G, and their salinity profiles are very similar (see the attachment), though G lags behind J. The idea is that if I can calculate the lag between the profiles, I can then derive a velocity of the water parcel. My research tells me that a cross-correlation calculation is the best way to figure out the lag between 2 time series. While, I have done a fair amount of reading on correlation functions, I am not sure why this data format would affect a cross-corr function. I want to do my own spade work, but I would appreciate 'shove in the right direction' as far as that theory goes. If I need to gain a better understanding of cross-correlation overall, I will. I am just not entirely sure how to proceed from here. Thanks again.

[David Goodmanson]

Hi Nathan,

that's a nice plot although I don't understand why it shows the J data going all the way to the end of June with a gap in the middle, whereas the text files you provided have the J data ending earlier, with no gap.

The reason you are getting the big triangle is that each data set has an average value around 36, with small variations about that. So you are convoluting together two rectangles, each with a small amount of variation on the top. That gives you a large triangle with some small, hard-to-see variations. If sG and sJ are the two salinity vectors, then if you take away the large constant part with

plot(xcorr(sg-mean(sg), sj-mean(sj)))

you will see something more like what you expect, although it doesn’t look very useful for finding a delay.

[NATHAN MURRY]

Hi David: The reason for the date discrepancy is that I only posted a sample of the data set, not the entire set that covers the entire date range you see in the first plot. Thank you for explaining to me what is going on with the plot. I will run the test code you suggest to see what that looks like, and also do some zooming on the original plot to be able to see the variations you made mention of.

I think I have enough to move forward again. THANK YOU for responding to me, I greatly appreciate it.

--NMM