Your equation is wrong ; it must read
eqn = A^x==B_A
But you should not check for nilpotency this way.
In your case, it's obvious by inspection that A is nilpotent since it is upper triangular.
For the general case, I'd check whether A has only 0 as eigenvalue :
help eig
Or - provided n is small - just calculate A^n for an (nxn)-matrix A and see whether it's the null matrix.
Best wishes
Torsten.