The Nazi's logic seems airtight. In the Bible, the Ark tends to make the armies that carry it invincible. Sure those armies tended to be made up mostly of Jews, God's chosen people and the Nazi's chosen extinction target, but the God of the Old Testament wasn't one to get bent out of shape over a minor technicality. Of course, as we find out in the climactic scene, Indy knows a secret about the Ark the Nazis don't: It has a habit of melting faces right the hell off their skull.
What Would Have Made More Sense:
The most sensible response for the Army to this whole Ark business is coincidentally the easiest: Do absolutely nothing.
The Fuhrer's army, lacking Indy's superior archaeology skills, misunderstood the location of the Ark to begin with. Without Indy's meddling, the Nazis would have blown millions of dollars from their 1936 budget digging in the wrong place.
But lets say the Army doesn't know the Nazis are geographically challenged. All the more reason not to do shit. Instead of stealing and re-stealing the Ark from the Nazis, Indiana Jones and the U.S. Army should have been rooting for them to find it. Their best case scenario is that the Nazis mission goes exactly according to plan: find it, ship it off to Germany and open it in a lavish pageant in Berlin with the whole Nazi high command in attendance. That was what they had planned to do all along. All the top Nazis in Berlin, including Hitler, front and center at the grand opening of a device that has a reputation for melting the faces of anyone in its vicinity.
"OK, now open it!"