You do that conditionally on some particle cost being found that is less than the initialized cost of infinity. However, that never comes true: your calculated costs for your initial population are all nan +/- nan*1i (complex nan).
You are working in ranges about 1e23 to 1e25.
lambda-(lambdaB-a3*(N-Nth)) is about 750. Squaring that is on the order of 1E5. a2 is on the order of 1e19, so the product of the two is on the order of 1e24
That is the same order of magnitude, with a small (roughly pi) complex component
exp(gamma1*pitch/2) and exp(-gamma1*pitch/2) are the same but with different signs on the exponent. If gamma1 is about 1e24 and negative, exp(gamma1*pitch/2) would be roughly exp(-1e24) which would come out as 0, but exp(-gamm1*pitch/2) would be roughly exp(1e24) which would overflow to infinity. If gamma1 is about -1e24, then which of the two exp() is 0 and which is inf switches roles. Either way, one of the two is going to be infinite when you are working in that range. And that is going to poison all the rest of the calculations, leading to NaN results.
You have large values on the order of 1e24 hard-coded, you aren't going to get around those at all easy; no mere typing mistake. But to get finite values out of the expression, your pitch would have to down-scale the 1e24 into the range of no more than 700-ish.
Or, you would have to switch to the symbolic toolbox for that calculation. For the symbolic toolbox, the limit is roughly
which gets you an extra factor of 1e6 to play with.