Well Azi looks like u answered ur own question, u don't want to be a doormat nor do u deserve to be one. So forget about him girl and move on :)

I don't even see a reason for u to talk him at all, even about work or other stuff, it'll only confuse u and might lure u into thinking u have feelings for him. Just shut him out of ur life if he's not worth it.

I'm not always right, but I'm never wrong -Garfield