I would simply ignore it. You're in a compound, not walking around in the middle of town, and you're only going to the gym, not just walking around the compound in short shorts. It's totally normal and allowed for you to wear shorts in the gym.

If one person has a problem with that, ignore it. They can't exactly do anything about it anyway.