That is not a problem of ismember function. The machine accuracy is in matlab eps
to better understand you have to try this
I IEEE Standard “double precision floating point” is eps ≈ 1.11 × 10−16. In matlab the eps is 2^-52 but as you see +2*eps or less 2*eps deliverd the same resault.
in your case A(25)-5 is equal 4*eps which is numeric precision of calculation for 0.2:0.2:20.
If you use the diffrent of values (A(25)-5)<=4*eps the answer will be logical true.
I hope that is the answer of your quastion.