Neighbors Came Into My House Without Invitation. What Do I Do?

I posted this in r/neighborsfromhell and it was suggested that I post it here for advice:

I'm finally selling my house with the neighbors from hell. The issue is that yesterday, while my real estate agent was in my house with my termite and moisture inspector, those neighbors just walked into the house uninvited and proceeded to walk around. No knocking at the door, no "hey how you doing" to the real estate agent, just walked in like they owned the place. They then started making disparaging comments about me and my husband and our family. They even went so far as to tell my realtor that they think he beats me! (This is completely false and unfounded!) She had to be really direct to get them to leave, and they kept trying to "meet the buyers". I'm so afraid that they will meet my buyers,, and that my buyers will walk because of them! What can I do? Should I text these neighbors about this incident to start a "paper trail" or should I just tuck my head and pretend nothing happened and just try to get through this sale? Is this worth pursuing legal council?