Of course, you know what they didn't -- that mercury is definitely not something you want to handle without protection. You don't want to breathe it, either -- the compound the workers used produced large amounts of mercury vapor, and thanks to the piss-poor ventilation of the workshops, the workers were regularly exposed to lots of the gas. This caused the workers to develop tremors, also known as "hatter shakes," which made their teeth fall out and caused a whole range of vision and hearing problems. And then it got bad.
Over time, mercury poisoning victims are turned into drooling, bumbling caricatures of their former selves. The workers hallucinated, slurred their speech, and displayed signs of shyness and irritability. They had trouble controlling their tempers, resulting in many an impromptu fistfight at the workplace. They picked fights with their colleagues and strangers who visited the workshops, and sometimes they refused to take orders from bosses, giving rise to the phrase "mad as a hatter."
Hatters gonna hat.
The employees were forced to work under these conditions until the combination of tremors and wacky behavior became too much and they either were fired or simply dropped dead from mercury poisoning. And thus "mad hatter disease" was born. But damn it, when you see a dapper fellow strolling along in a bowler hat, all you can think is "It was all worth it."