MATLAB Answers

Synchronizing several large timetables

21 views (last 30 days)
Lutetium
Lutetium on 18 Mar 2021
Commented: Seth Furman on 1 Apr 2021
Dear community,
while I'm waiting for my synchronization to finish, I'm wondering if there is any faster way perform below synchronization. I had to use tall() to avoid running out of memory.
In my example, I have around 19 timetable with timevector and corrsponding signal. I take my first timetable and put it into variable TT1. Then, I put the second table in TT2 and synchronize TT1 and TT2. The resulting table is then transferred into TT1 and I synchronize the next timetable.
While this certainly works, it will take on the current speed (see screenshot below) approx 15 hours or more. Is this a very inefficient way? Is there a way to speed this up significantly? Any obvious mistakes?
I appreciate your input (This is local parallel pool with 16 workers, Xeon E-2286 2.4 GHz, 32 GB RAM)
TT1=tall(tbl{1,1});
for i=2:sigcount
TT2=tall(tbl{1,i});
TTSync=synchronize(TT1,TT2,'union','nearest');
TT1=TTSync;
end
location = 'E:\Matlab\Analysis';
write(location,TT1);
clearvars;

Accepted Answer

Seth Furman
Seth Furman on 18 Mar 2021
Where are your timetables coming from? Are you reading them from one or more files? If so, it probably makes more sense to create each tall timetable from a datastore rather than an in-memory timetable. See the link below for an example.
Assuming tblTall is a cell array of tall timetables, we can pass all tall timetables to synchronize at the same time as follows
TTSync = synchronize(tblTall{:,:},'union','nearest');
Please let me know if you are able to try this and, if so, whether you see better performance.
  3 Comments
Seth Furman
Seth Furman on 1 Apr 2021
Please also clarify, does code like the following work on your machine with your timetables or do you get errors about running out of memory?
>> tblCell = {timetable(1,'SampleRate',1),timetable(2,'SampleRate',1)};
synchronize(tblCell{:})
ans =
1×2 timetable
Time Var1_1 Var1_2
_____ ______ ______
0 sec 1 2

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by