Yeppo. The least you can do is say 'no English',, or "Sorry, I don't understand you", or put the phone down and go wander to look for someone who can speak to you. Especially if it is an English speaking company and that is how you traditionally do business with the company.

I'd do the same in Japan. Sometimes I'd have to answer the phone at school, and we were strictly Japanese-speaking. I don't mind saying that this was often a terrifying experience, especially when the person went on... and on... and on...

But I could at least tell the person on the phone I would call someone else to the phone, and I would call another teacher to the phone and explain what the person needed, or very apologetically explain that I didn't quite know what they needed and could they please talk to the person. It was no problem.

Not speaking Japanese fluently was no excuse. If I had just hung up, and my kyoto sensei (head teacher) found out I had done so, I would have gotten in severe trouble. I can guarantee that.