Sometimes it's just garden-variety corruption, like the scandal that erupted when it was revealed that International Olympic Committee members accepted gifts and bribes from Salt Lake City officials who hoped to bring the games to their otherwise-boring city. On top of being highly illegal, the ploy didn't even work; the 1998 Winter Games ended up being awarded to Nagano, Japan.
When human rights groups reminded the IOC that China treats its citizens like shit after the 2008 Summer Games were awarded to Beijing, they were paid no mind at all -- even after a dissident who criticized the government's push to bring the games to town was sentenced to two years in prison just for voicing his disagreement.
Even that barely scratches the surface when it comes to the atrocities associated with the Olympics. The imprisoned dissident turned out to have every reason to be concerned, seeing as how an estimated 1.5 million residents of the city were displaced or forced into homelessness to make room for the massive structures and complexes needed to house all that sporting activity. The exact same thing is happening in Brazil right now.
Again, I acknowledge that the Olympics make for good television viewing, especially when Bob Costas comes down with pink eye and tries to fight through it like no one will notice. But it would be great if we were to someday figure out how to make them happen without ruining millions of lives every few years.