with you at all times showing your stamped visa in it. I would also speak to your employer for them to issue some form of security pass if possible. I know with QP a guest (mother of a town resident here) was asked to produce ID and didn't have any. They had to organise a security pass for her to carry around with her for the duration of her stay.
I think it's unusual not to have an ID card that is what would have thrown the security. If you don't have an ID card, usually it means you are here on a temporary visa ... this is the visa you should have a copy of to show them instead of ID card.
Spot ID/paper checks happen frequently though-out the middle east. You should always be prepared.
Since you weren't taken away, you must have had enough papers for them to have been satisfied.
In central Doha it doesn't happen too often, but once you start heading out, especially toward the oil/gas/military areas it's not unusual.