If you are scared of the brawl, you can device other contraptions to replace the buzzer.
Like place a bucket just enough to hold one basket ball at the center of the hall and give each team a basketball marked with the team name. The team to put the ball in the bucket gets to answer first.
There will be one ball boy for each team.