How to remove NaN values from an array

I have an array 'AllTEC' of measurements from different sensors at different latitudes 'Latrayend' (see attached test.mat). Each column in 'AllTEC' and 'Latrayend' is from a different sensor. I plot these as different color lines but a gap occurs where there are NaN values:

I want the lines to be continuous. For one sensor (a vector) I simply use:

%find and remove nan values
Latrayend(find(isnan(AllTEC)))=[]
AllTEC(find(isnan(AllTEC)))=[]

%plot figure 3
 cmap = jet(m);
figure
 for k=1:m
    line(Latrayend(:,k),AllTEC(:,k), 'Color', cmap(k, :)), hold on% USE FOR LARGE NUMBER OF
    ylabel('Total Electron Content (TECu = 10^{16} electrons/m^2)','fontsize',24, 'fontweight','bold')
    xlabel('Satellite Latitude (Degree)','fontsize',24, 'fontweight','bold'); 
    title('FIG 3- STEC as a function of satellite latitude') 
 end
 legend(h)
 hold off

But this does not work for an array. I tried doing this within the loop to create the figure, but get a subscript assignment mismatch error. I understand it is because the array would then not have the same order (when the loop goes through the first vector there are empty cells, but i don't care if it is an array of columns with different lengths. In this case I could just delete the entire row, but this may not be the case for futre datasets.

Thank you in advance for your help!

best regards,

Alex

Answer 1

[Star Strider]

Since you want the lines to be continuous, consider using the fillmissing (link) function to fill in the NaN values with numbers. (It was introduced in R2016b.)

[Alex]

I'm using Matlab 2010a at home right now, but could try that in the computer lab tomorrow. I'd still be interested in a solution that doesn't need a package that I may not have access to in the future.

Cheers,

Alex

[Star Strider]

One possible way is to use interp1 (link), for example:

x = linspace(0, 5, 50);
y = sin(2*pi*x);
y(20:25) = NaN;

figure
plot(x, y)

yi = interp1(x(~isnan(y)), y(~isnan(y)), x, 'linear');

figure
plot(x, yi)

There may be File Exchange contributions, such as ‘inpaint_nans’ that could work with R2010a. Otherwise, try fillmissing tomorrow.

[Alex]

Using interp1 I get the error 'The values of X should be distinct.' likely because there are NaN values in both Latrayend(x) and AllTEC(y). Still need to try fillmissing and inpaint_nans but assuming the same problem will arise since these methods interpolate between data points to fill the matrix when I'd like to simply remove the cells with Nans (and the corresponding cells from the other array) and create two irregularly shaped arrays and then plot each column as a different color line (i.e. a particular column from each array will have the same length, but not the same length as all other columns).

[Star Strider]

This worked when I ran it:

D = load('test.mat');
Latrayend = D.Latrayend
AllTEC = D.AllTEC

xi = linspace(min(Latrayend(:,1)), max(Latrayend(:,1)), numel(Latrayend(:,1))); % Interpolating Vector

m = size(AllTEC,2);
cmap = jet(m);
figure
hold all
for k=1:m
    AllTEC(:,k) = interp1(Latrayend(~isnan(AllTEC(:,k)),k), AllTEC(~isnan(AllTEC(:,k)),k), xi, 'linear');
    line(xi, AllTEC(:,k), 'Color', cmap(k, :))  % USE FOR LARGE NUMBER OF
end
hold off
ylabel('Total Electron Content (TECu = 10^{16} electrons/m^2)','fontsize',24, 'fontweight','bold')
xlabel('Satellite Latitude (Degree)','fontsize',24, 'fontweight','bold');
title('FIG 3- STEC as a function of satellite latitude')
legend(h)
hold off

Experiment to get the result you want.

EDIT — (24 Apr 2019 at 15:59)

Added plot image —

[Alex]

Great! Thank you!

[Star Strider]

As always, my pleasure!