Talk to the bank.  They should be ok with opening a savings account.  You wont get any cheques or credit cards though until you get a residence permit.