The Problem ...
First off, birds hardly use their noses; instead they rely on their eyes and ears. So no, they can't immediately sense when a baby has been tainted by human stench, whether you pick it up with your hands or try to rub your balls on it.
Second, birds don't really care if humans touch their chicks or not. While touching a bird when it is learning to fly can be detrimental to its ability to fly, it doesn't stop mother birds from feeding or caring for their chicks. They're still bonded to them through good and bad, just like your parents are with you.
Except for the times they left you in the woods, only for the wolves to return you in disgust.
So How'd It Get Started?
Animal welfare organizations have long maintained (even now) that you shouldn't help newly born animals with anything because they're still learning. But, if you've ever met a 7-year-old, you know they're not easily deterred. "But MOM! The baby needs my help getting back to the nest!" So, the moms came up with this urban legend: "If you touch it, its mom will let him starve while watching you with blank, judgmental eyes dear. You see for mommies, it's a thin line between nurturing, and gladly ridding ourselves of a foul smelling accident that ruined her life."
"I'm not sure how we got on to this."