My Boyfriend(ex Now) Filed For Full Custody Behind My Back While We Were Dating

I just had his baby and we were still dating I was living at his families home taking care of it 24/7 while he would play video games or be with his friends until he finally got a job. I felt something was off I kept asking him but he said it was nothing and I had a feeling to try and find out if he was trying to get full custody. His face went white and he started calling his mom. He then called the cops lying that I hit the baby and he yanked the baby across the bed. His grandma came in and tried to take the baby from me and I freaked out and got in the way where she couldn't grab her. When the cops got there I said I'm taking the baby to my family home just because I have been FULLY taking care of her. He said "you're not taking the car seat so you can't take her" and the cops stuck up for me and said he was being selfish. That's when he said "fine you can take the shitty one" and then proceeds to hand a car seat that is dirty and has MOLD on it where the police officer was disgusted and trying to scrub it. I'm scared because his family is rich and my family isn't I'm not going to be able to get a good lawyer and if they were manipulating me for that long I'm scared they will get full custody of my daughter. Please give me advice