you made mistake to stop your car over there as that guy is not a cop or something like that. By the way, you can go to police but it's a wise decision not to go to cop at this moment. As there was ladies in that car they might make another story if you go to police now.
May be that guy got annoyed due to your presence besides his car as he was (may) busy in some private moment with ladies.