No, it's smart. Women travelers are advised to just shake their heads politely when the elevator stops for her and there is a man or men already on it. And if you are alone and a man gets on, just step out the door and wait for the next one. Don't even think twice, just make this your habit.
Mandi