Faster alternate to all() function
108 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Abinesh G
el 22 de Ag. de 2024 a las 10:41
Comentada: Abinesh G
hace alrededor de 21 horas
I am running a simulation of 100s of thousands loop and I am noticing that inside the each loop an all() function consumes almost 95% of the time. I want a faster alternate to this.
So here is my algorithm:
varname=rand([24665846,4])
for i=1:1000000
idx=idxkeep(i);
ind1=all(varname(:,1:4)==varname(idx,1:4),2);
ind=find(ind1);
end
0 comentarios
Respuesta aceptada
Stephen23
el 22 de Ag. de 2024 a las 14:58
N = 10000;
varname = rand(246658,4)
idxkeep = randi(size(varname,1),1,N);
tic
for i=1:N
idx = idxkeep(i);
idy = all(varname(:,1:4)==varname(idx,1:4),2);
idz = find(idy);
end
toc
tic
for i=1:N
idx = idxkeep(i);
idy = ...
varname(:,1)==varname(idx,1) & ...
varname(:,2)==varname(idx,2) & ...
varname(:,3)==varname(idx,3) & ...
varname(:,4)==varname(idx,4);
idz = find(idy);
end
toc
Más respuestas (2)
Steven Lord
el 22 de Ag. de 2024 a las 18:13
Are you sure the longest time is spent in all? Reducing the size of the varname variable and the number of iterations a bit (so it doesn't time out in MATLAB Answers) and looking at the times for the == operation and the all call:
varname=rand([246658,4]);
n = 10000;
timingData = zeros(n, 2);
% You never defined the variable idxkeep, so defining it here using random data
idxkeep = randi(size(varname,1),1,n);
for i=1:n
idx=idxkeep(i);
tic
x = varname(:,1:4)==varname(idx,1:4);
timingData(i, 1) = toc;
tic
ind1=all(x,2);
timingData(i, 2) = toc;
ind=find(ind1);
end
format longg
seconds(sum(timingData, 1))
So it looks like the all call does not take the majority of the time. Most of the time is spent creating the logical array (named x in the modified example above.)
Now my choice of idxkeep is somewhat arbitrary. I'm guessing you have an alternate purpose for idxkeep. If you tell us in words not code what the purpose of this whole block of code is, we may be able to offer a more performant solution (perhaps one that avoids creating the large logical array.)
3 comentarios
Steven Lord
el 22 de Ag. de 2024 a las 19:52
So you want the rows with unique entries for the first four columns? What happens if you call unique with the 'rows' and 'stable' options? Or use groupsummary or grouptransform to perform your filtering on the rows based on unique combinations of the elements in the first four columns?
arushi
el 22 de Ag. de 2024 a las 11:13
Hi Abhinesh,
Here’s a possible approach that uses vectorized operations and logical indexing to improve performance:
% Assuming idxkeep is a pre-defined vector of indices
% Precompute the subset of varname
varname_subset = varname(:, 1:4);
% Preallocate for results if needed
results = cell(1, 1000000);
for i = 1:1000000
idx = idxkeep(i);
target_row = varname_subset(idx, :);
% Compare using vectorized operations
% Use implicit expansion (broadcasting) for comparison
ind1 = sum(varname_subset == target_row, 2) == 4;
% Find the indices
ind = find(ind1);
% Store results if needed
results{i} = ind;
end
Hope this helps.
Ver también
Categorías
Más información sobre Troubleshooting in Polyspace Products for Ada en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!