I wear shorts all the time and never have any problems.
I wear shorts that are just at the knee, i wouldn't wear gym shorts tho.

4 years ago I used to get indian guys give me the once over, but not any more.