Yes, the building was a Nazi headquarters, and it was just chock full of violent, racist assholes. They knocked over the small man's ladder and closed in on him from all sides. One thing was for sure: Somebody wasn't walking away from this fight.
And that "somebody" was 20 Nazis.
Because our tiny hero was, at the time, the world's strongest man.
We're guessing the Carrot Top 'fro threw them.
Joseph Greenstein, aka the Mighty Atom, was a renowned circus strongman, and he quite reasonably proceeded to beat the s**t out of every Nazi he could lay his little hands on. He knocked back over a dozen men, breaking arms, noses, legs and presumably the hearts of treasure-hunting femmes fatales everywhere. Joe was arrested and taken to court after the fracas, where he was charged with dozens of counts of battery. The judge, being unable to fathom one man causing so much damage, asked the arresting officer if all of the men involved in the fight were standing before him. Surely, he thought, Joe had accomplices who simply got away.
"Have we ruled out the possibility of jet packs?"