He may be her legally wedded hubby, but still you dont run a hotel. Tell him he is not welcome here, if she does not like it, she can go..You do not have to accept a situation which you are not comfortable with, there is nothing cruel about it...And keep all your stuff locked up, he may already have cut a key..If I were you, absolutely no way I would accept him being rude to me in my house, and not accepting my boundaries..

You do not have to talk to him at all: just tell her she cannot have her visitors in your house, on her off days , she is free to go anywhere she wants with anybody..If he turns up and shouts at you, tell him get out , or you will be calling the police.