Replace multiple intervals in array with NaN - No loops
1 visualización (últimos 30 días)
Mostrar comentarios más antiguos
Michael Madelaire
el 14 de Feb. de 2017
Comentada: Jan
el 15 de Feb. de 2017
Hi
I am working on a project with a lot of data. In the area of 10.000.000 rows. I therefore cannot use for loops. The data has to be cleaned, with respect to certain parameters. These parameters are found independently and are not on measured with the same timesteps. Below is a simplified example that explains the idea of the problem. :)
The main dataset, let's call it:
dataset = 1:30;
Two arrays are now given. They indicate intervals within which measurements has to replaced with NaN.
upperbound = [2,5,10,18,27];
lowerbound = [0,3,8,15,25];
With a for loop I would normally say:
for i = 1:length(upperbound);
dataset(lowerbound(i):upperbound(i)) = NaN;
end
But with so much data this is not possible.
Do any of you have a good idea to solve this ?
2 comentarios
Jan
el 14 de Feb. de 2017
The smallest lowerbound index is zero. Is this a typo, because indices must be >= 1?
Respuesta aceptada
Jan
el 14 de Feb. de 2017
Editada: Jan
el 15 de Feb. de 2017
With logical indexing:
data = 1:30;
upper = [2,5,10,18,27];
lower = [1,3,8,15,25]; % Not 0 as 1st element
% FAILING for upper==lower or overlapping intervals:
index = zeros(1, numel(data));
index(lower) = 1;
index(upper) = -1;
toNaN = (cumsum(index) == 1) | (index == -1); % Thanks Adam!
data(toNaN) = NaN;
v = mcolon(lower, upper);
data(v) = NaN;
If this is the bottleneck of your code, use a C-Mex function.
// File: BlockCopy.c
#include "mex.h"
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
double *low, *up, *out, NaN = mxGetNaN(), *p, *pend;
size_t n, lowi, upi, maxi;
maxi = mxGetNumberOfElements(prhs[0]);
low = mxGetPr(prhs[1]);
up = mxGetPr(prhs[2]);
n = mxGetNumberOfElements(prhs[1]);
if (n != mxGetNumberOfElements(prhs[1])) {
mexErrMsgTxt("*** BlockCopy[mex]: Index vectors must have the same length.");
}
plhs[0] = mxDuplicateArray(prhs[0]);
out = mxGetPr(plhs[0]) - 1; // For 1-based Matlab indexing
while (n--) {
lowi = (size_t) *low++; // Check validity of indices
upi = (size_t) *up++;
if (lowi < 1 || upi > maxi) {
mexErrMsgTxt("*** BlockCopy[mex]: Index out of bounds.");
}
p = out + lowi;
pend = out + upi;
while (p <= pend) {
*p++ = NaN;
}
}
return;
}
This works with overlapping intervals also.
Attention: The C-code will crash if any index is <= 0 or > then the length of the input array or if the inputs are no DOUBLE arrays. Either care for proper inputs or add checks of each index.
7 comentarios
Jan
el 15 de Feb. de 2017
@Michael: Feel free to use the C-mex. I've attached the commented C-code and a pre-compiled function for Win/64.
Más respuestas (0)
Ver también
Categorías
Más información sobre Loops and Conditional Statements en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!