Does A Landlord Get To Keep A “security” Deposit If I Never Signed A Lease? (wisconsin)

Okay soooo, my boyfriend and I recently started our house search (to rent not buy). We found one that we liked a lot and the process was seemingly going great. We were approved and after approval they requested the “security” deposit. I have this in quotations because we did not sign a lease or anything when we sent the money (not smart, I’m aware) but because of this I searched and found that this is more of a holding fee rather than a security deposit. We had all intentions to move forward when we received the lease via email but ended up finding a rental over $300 cheaper that met every single want/need that we were looking for. We emailed the landlord of the first place that we have decided to go a different direction and figured this really wouldn’t be too much of an issue since nothing had been signed. However, they did pull the rental from the market when we sent the money so we were definitely anticipating them keeping a portion of the deposit. But they are stating it is “non-refundable” - at no time over the phone or via email did they state this was non refundable if we didn’t sign. So I guess my question is: is there any legal action or statements we can make to get our money back? Any and all info is much appreciated ????