unique across multiple cells or arrays

Let's say I got a bunch of arrays in a cell array that are not the same size:

a = {[1 2 2], [2 3 6 3], [4 3 5 6 7]}

I want to remove all dupplicates accross the cells. I want to remove things in a cell that are in the other cells as well as the dupplicates in each cells:

Example output:

b = {[1 2], [3 6], [4 5 7]}

I would like to have this vectorized. I can simply loop but this is very slow:

others = [];
for i = 1:length(a)
    b{i} = setdiff(unique(a{i}), others);
    others = [others, b{i}];
end

I'm thinking of putting it all into a single array to call unique:

b = unique([a{:}])
% now b = [1 2 3 4 5 6 7]

But I don't think there's anyway to put it back into its respective cells since the order is lost and I don't know where the limits are

[per isakson]

Read the documentation on unique and start with something like this

len = cellfun( @numel, a ); % remember the origins of the elements  
[ C, ixa, ixc ] = unique( [a{:}], 'stable' );

Next an indexing excercise will do it - I think.

[Alexander Winter]

I ended up with something like this:

endIdx = cumsum(cellfun(@numel, a));
startIdx = circshift(endIdx, 1);
startIdx(1) = 0;
startIdx = startIdx + 1;
[C, ixa, ~] = unique([a{:}], 'stable');
a = arrayfun(@(i) C(ixa >= startIdx(i) & ixa <= endIdx(i)), 1:length(vertices), 'UniformOutput', false);

Not sure if there's simpler but at least it works. Big thanks!

[per isakson]

Thanks for sharing. I cannot think of anything simpler.

[Matt J]

The process you describe does not have a well defined output. For the example you gave,

a = {[1 2 2], [2 3 6 3], [4 3 5 6 7]}

this would also satisfy the requirements:

b = {[1], [2 3], [4 5 6 7]}

Answer 1

[Stephen23]

>> a = {[1,2,2],[2,3,6,3],[4,3,5,6,7]};
>> [u,x] = unique([a{:}],'first'); % or 'last' to allocate to the last cell where value occurs.
>> [~,y] = sort(x);
>> n = cumsum([1,cellfun(@numel,a)]);
>> z = sum(bsxfun(@ge,x(y),n),2);
>> c = accumarray(z,u(y),[],@(m){m});
>> c{:}
ans =
     1
     2
ans =
     3
     6
ans =
     4
     5
     7

[the cyclist]

Nice!

I don't know if it matters, but this solution will give a different result from Alexander's original one for and input like

a = {[1 2 2], [2 3 6 3], [4 3 5 6 7], [1 2]};

in which that last element has no unique elements not contained in the prior ones. Alexander's solution will include an empty array in its place, and this solution will have a shortened list instead.

Interestingly, they give equivalent results for

a = {[1 2 2], [2 3 6 3], [4 3 5 6 7], [1 2], [8 9]};

where the cell with no unique elements is not the last cell.

[Stephen23]

@the cyclist: well spotted!

The behavior you describe is caused by the presence/lack of indexing into that cell: if accumarray is not told to put anything in that cell, then that cell simply won't be created. That occurs for any number of trailing cells which only contain duplicates of prior numbers, not just the last cell.

It can be resolved quite easily by telling accumarray the required output array size:

c = accumarray(z,u(y),size(a),@(m){m});
%                     ^^^^^^^ output size

[Alexander Winter]

In my case the empty array is not desirable (I'm discarding them afterwards) so your initial solution is better for me. Thanks!