I received a reply from an email to Treasury. Refer link posted below:

http://www.treasury.gov.au/contentitem.asp?NavId=037&ContentID=1538

It seems that if you have done all the things you need to, to declare yourself as a non-resident there should be no problem.