Serial: Real-time plotting and storaging streaming data from Arduino slowing down badly over time

Hello everyone,

I'm working on a system where I collect data from an Arduino and display it on a real-time plot in MATLAB, while also storing it in a table for later analysis. The data is sent from the Arduino line by line, with each line consisting of two coordinates separated by a comma. I am developing a standalone app for this job.

Here’s my approach:

1. I use `readline` in MATLAB to read each received line.

2. Line is splitting on comma, to extract the two coordinates.

3. Dot is plotted in real time with previously extracted coordinates. Line is displayed on list box also.

4. The coordinates are stored in Table for further processing.

function RecordData(app)
            
            global Arduino
            global Table
            
            Table = table(0,0,'VariableNames', {'X', 'Y'});
            
            while app.StartRecordingButton.Value == 1 %State button strats and stops recording
                
                flush(Arduino);
                line = char(readline(Arduino));
                
                numElement = numel(str2double(split(line, ', '))); %Counts number of elements in received line
                
                if numElement == 2 %Only process line with both of coordinates received 
                  
                    coords = str2double(split(line, ', ')); %Extracting two comma-separated values from received line
                    
                    xCoord = coords(1); %X position
                    yCoord = coords(2); %Y position
                    
                    %Plot
                    plot(app.Graph, xCoord, yCoord, "Marker","*", "Color",'k');
                    hold(app.Graph, "on")
                    
                    %Append new data to Table
                    newLine = table(xCoord, yCoord, 'VariableNames', {'X', 'Y',});
                    Table = [Table; newLine];
                    
                    %Also update list box
                    app.ListBox.Items(end + 1) = {line};
                    scroll(app.ListBox,'bottom');

                end
                        
                pause(0.01);
                flush(Arduino);
          
            end
            
            msgbox("Recording finished", "Info");
      
        end

The problem is that the program gradually slows down over time. After about 5 minutes, it becomes too slow to be practical. I have attached the profiler report. It shows a highest number of calls within the Stream function. The biggest self time has Stream>Stream.isDeviceDone.

I’m unsure which part of the code is causing this slowdown.

I have tried without real time plotting. The issue is not solved.

Could anyone advise on what might be causing the slowdown or suggest a more efficient approach? I am sure that my approach is not optimal.

Thank you in advance for any help!

Answer 1

[Walter Roberson]

                    Table = [Table; newLine];                    

Each time that statement is executed, MATLAB needs to examine the current size of Table, and the current size of newLine, and allocate a new table() that is the new size. Then it copies all of the existing data from Table into the new area, and then copies the newLine at the bottom of the new area. Then it assigns Table the new data addresses and marks the old Table data area as being unused. Garbage collection is potentially triggered at this point.

The first time through, zero X and zero Y are copied to the new area, and then 1 X and 1 Y are added. Total that has been copied so far: 0.

The second time through, 1 X and 1 Y are copied to the new area, and then 1 X and 1 Y are added. Total that has been copied so far: 1+1 = 2.

The third time through, 2 X and 2 Y are copied to the new area, and then 1 X and 1 Y are added. Total that has been copied so far: 2 + (2+2) = 6.

The fourth time through, 3 X and 3 Y are copied to the new area and then 1 X and 1 Y are added. Total that has been copied so far: 6 + (3+3) = 12.

The fifth time through, 4 X and 4 Y are copied to the new area and then 1 X and 1 Y are added. Total that has been copied so far: 12 + (4+4) = 20.

And so on. After N steps, the total number of items that have been copied so far is N*(N-1)

After 100 steps, 100*99 = 9900 items will have been copied so far.

Your memory handling is dragging down performance, a lot.

You would be better off instead using

 Table{end+1,:} = {xCoord, yCoord};

which would have less overhead. But it would still have serious memory copy overhead.

So... what you should do is increase the Table storage in batches. For example

 sz = 256;
 Table = table(sz, sz, 'VariableNames', {'X', 'Y'});
 used_entries = 0;
 
 %...
 used_entries = used_entries + 1;
 if mod(used_entries, sz) == 0
     Table(end:end+sz-1,:) = {zeros(sz,1), zeros(sz,1)};
 end
 Table(used_entries,:) = {xCoord, yCoord};

Possibly the line = {zeros(sz,1), zeros(sz,1)} might need some adjustment.

Answer 2

[Adam Danz]

Calling plot(__) with hold on in a loop is inefficient. Every time new points are added to the axes, new line objects are created. An axes with many line objects will render slower than an axes with one line object.

Instead of this...

while app.StartRecordingButton.Value == 1 %State button strats and stops recording
   ...
    if numElement == 2 %Only process line with both of coordinates received
        ...
        %Plot
        plot(app.Graph, xCoord, yCoord, "Marker","*", "Color",'k');
        hold(app.Graph, "on")
        ...
    end
   ...
end

... create a single line object and update its data properties.

h = plot(app.Graph, NaN, NaN,"Marker","*", "Color",'k')); %<---- CREATE OBJECT
hold(app.Graph, 'on') % <--------------------------------------- APPLY HOLD
while app.StartRecordingButton.Value == 1 %State button strats and stops recording
   ...
    if numElement == 2 %Only process line with both of coordinates received
        ...
        % Update plot
        set(h, 'XData', [h.XData, XCoord], 'YData', [h.YData, YCoord]) %<------ UPDATE PROPERTIES
        ...
    end
   ...
end