Error in the training of physics-informed neural network (PINN) when changing the settings

I am trying to train a PINN and the codes are adapted from the example at https://uk.mathworks.com/help/deeplearning/ug/solve-partial-differential-equations-using-deep-learning.html. I found that for some values of the neural network training settings (number of layers and neurons, number of epochs, initial learning rate, decay rate), although the error is huge, but the codes can still run successfully. However, when I change the values of these settings, it can sometimes give me the error shown below:

I am very confused by this error because even I only make a small change in some settings (for example, change the decay rate from 0.1 to 0.999) can trigger this error. Does anybody know what is happening here?

[Ben]

Could you paste your modelLoss function here, or whichever function you're passing to create accfun from dlaccelerate?

I wouldn't expect changing the settings you mention to cause this error. However I can see there being one subtle issue - using accfun = dlaccelerate(@modelLoss) creates an accelerated function which works by tracing through your modelLoss function. This creates a cached and accelerated version of modelLoss on accfun. What can be surprising about this is that if you change modelLoss-s implementation, the cache might not get updated when you re-run the script sections. To avoid this issue you can call clearCache(accfun) on the line following accfun = dlaccelerate(@modelLoss) - this is a good safeguard in general and doesn't cost too much performance.