Dendrogram threshold doesn't work well

hello everyone i'm using 'linkage' to cluster vectors, I have 4 groups of vectors and I want to find a threshold such that each cluster can't contain more thane one vector from each group. the number of vectors in each group is not always the same. most of the time it works fine using this code:

        for i=1:size(C,1)
            if sum(ismember(Labels{C(i,1)},Labels{C(i,2)}))
                Thresh = C(i+1,3);
                break;
            else
                Labels{end+1} = [Labels{C(i,1)} Labels{C(i,2)}];
            end
        end

'C' is the output from 'linkage' and 'Labels' is the label for each vector from which group it came from.

and draw the dendrogram like that:

    dendrogram(C,'ColorThreshold',Thresh,'Labels',Labels1)

in most cases I get a good reasult like that:

but in other cases I get something like this:

I noticed that it is always when there is only one cluster. I tried tho give a threshold of C(1,3), C(2,3) and it remains all blue, and if I set it to be C(3,3) then there is one to many vector in the cluster

please help me, thanks