No, He cannot touch. In the first instance itself you have to convey the same to him. If are not comfortable you might have been taken into his notice. there is no point to discuss now.