Move With Kids

I read an article here over a year old that won’t let me comment on about a mom wanting to move her and the kids away. I’m in the same boat. My ex and I were never married. No custody agreement. He’s on the birth certificates. He has a history, 1st child he was in prison when I gave birth, I was dumb enough to let him convince me to send the certificate to let him sign in prison. We now have two and have been broken up. I let him have his way and take the kids every 3-4 days. One day he decided he wants me back and stalked me with a tracker and showed up to the bar I was out with friends with on his night with the kids, I asked where is the kids he said with an adult. He seemed drunk called me all kind of nasty names because he thinks I’m chasing other men & ignoring his calls and texts. Long story short he left our toddler in the car and police were called and he had drugs on him. He was out on parole for selling drugs. He gets out on monitor soon and all I can think about is moving away from the drama and his family that condones his behavior. They said it’s my fault he did what he did because I led him on. Just like it was my fault he went to jail when he hurt me physically and the case got dropped he says because it didn’t happen so everyone believes him but it’s because I never showed up to court. I’m tired of him controlling my life. I’m sick of his friends and family on his side when he keeps making our lives hell. I want to move but my family said if I do he has rights and can force us to come back. Should I go to court first a request a move away or just move and hope a judge pieces the puzzle together to see why I left. I’m scared when he gets out he will fight for custody to see the kids. He’s even called me in jail to ask if he can see the kids again even though he has a temporary order not to speak to us.