I think I would have reacted the same way if someone had put my family in danger like that so good on ya mate. He was probably bluffing, thinking that you would be afraid to go to the police or traffic so you did well to go there. I often have similar confrontations and being firm in your stand is what would really save you.