Dont know what kind of response you would expect from a public forum on a very personal issue. Since people seem to be giving advice, I will also chip in my two cents.

Its always your choice. You can choose to ignore, stay and suffer or leave him or live your life and enjoy while with him. It all boils down to choices you make, I dont think anyone else can give you solutions, more than words.