You'd think Thor would have it easy, given his sleeveless look and frequently exposed abs. Nevertheless, the costume gets so hot that Chris Hemsworth has a small air-conditioning unit underneath his armor. Scarlett Johansson has it even worse; her Black Widow costume is basically a giant wetsuit that boils her alive. She's admitted to hallucinating during a particularly intense fight scene on a roof, and forgoing underwear to avoid the heat. In her own words, as soon as the director yells cut, everyone's like "Arrrgh ... god, this thing, get it off me, it's awful!!" (Now guess which part of that interview all the headlines fixated on.)
Keep this in mind when Avengers: The Miami Mission is greenlit and they're all wearing shorts and T-shirts.
Worst of all might be Loki's costuming. Unlike the cheap plastic version you found at the Halloween store, Tom Hiddleston's costume is made of metal and leather and is a ridiculous thing to wear. It weighs upwards of 30 lbs, and the horned headpiece alone weighs about as much as a small child. Hiddleston gets a bit claustrophobic in it, but he's been a good sport. He likes the way it all looks on camera, and has said that he really can't complain, because hey, he gets to be Loki.
Luckily, between scenes, all the Avengers get access to cooling fans, which we're guessing accounted for a significant chunk of the budget.