No, their lack of money may be the cause of the racism (among other things), but the actual act of not letting them in is racist.