Tax free status is granted based on your answers to CRA's request for non-residency for tax purposes form. It is not automatic no matter how long you are out of the country. Do you think CRA is going to lose any opportunity of clawing money out of you that they feel is "owed"? Think again.

No it isn't too late to declare it retroactively. I've done it after being out of the country more than a year.