I agree that a lot of the criticism of the Games is unwarrented. However, check when the organising actually started in earnest.
The original advisors who did the Sydney Olympics were retained. After delivery of the master plan at a huge cost, they were kicked out. I don't know if they were later re-engaged.
Until 2003 very little had happened. How many DG's were there before the final one Al-Qatani? Why was the Director of Overall Planning removed after only 5 weeks? He had organised the Japan/Korea World Cup and was a member of the Committee for the German World Cup.
The OCA have a lot to answer for.
Earlier commencement of "serious" planning would have identified many of the issues that eventually resulted in problems.