How do I make my neural network consistent?

Hi,

I have a DSGE model, which I want to solve using neural network for my dynamic equations. First, I solved a simpler version of my model and everything was great. But when I tried to change slightly the model imposing zero lower bound (ZLB) (for neural network this change one of my input data) and then solve the model using the solution of simpler model as an initial step it fails and I got this kind of things. What can I change to make my neural network more consistent?