I think sometimes get a sick kick out of making you feel like you've wronged them.

I often find that if you don't entertain their silliness, they soon get fed up! (trust me, I'm always putting my foot in it in some way or other....)

Good Luck!