You have to clear the cache.

After deleting your current picture refresh your account site and sign in again, THEN try uploading the new one. Try that. I think it's how I eventually got mine to work when I changed it. Otherwise the old one keeps popping back up instead of the new one.