Pople can be very cruel. I am guessing that they want you to marry someone else in order to have children. The best way is to tell them that you love your wife and that although you are trying for children, It is the Lord's will.