There's also the hypocritical nature of this argument, which is that trans people using the bathroom is somehow more of an invasion of privacy than people policing which bathroom they belong in. In fact, this will inevitably lead to (in fact, it has already has led to) non-trans people being harassed in bathrooms for looking like they might be trans. There's no floating neon "T" that identifies transgender people. This means that to prevent trans people from using the "wrong" bathroom, you'll have to scrutinize everyone to see if they "look enough" like a male or a female. That or inspect everyone's genitals before they enter, which sounds a tad like an invasion of privacy. Also, in this case, we're talking about schools. Kids are already Sith Lords at being cruel; they don't need more ammunition to bully other children based on how they look.
"What about safety?"
One of the biggest issues people seem to have with the idea of a transgender person in the women's bathroom (because for some reason, they almost never talk about the reverse) is the idea that someone will be attacked or raped or murdered or upper-deckered. And who would want that while they're on the can? But on the other hand, isn't this fear suggesting that up until now, the only thing stopping a raging psychopath from storming into the toilet and trying to wear you like a hat is the sign on the door? As if that sign has some magical anti-pervert force field that takes effect if someone enters while not wearing that "woman" triangle dress. And the fact that there's no evidence of transgender people attacking anyone in bathrooms kind of makes this sound like a lot of bluster for no reason.
Dangerous people absolutely do exist, but by and large, they tend to ignore rules, which is why they're dangerous. No one ever committed a murder because they didn't see a NO MURDERS sign posted. If someone wants to attack you in the bathroom, they're not going to let something like a sign or "social conventions" or "decency" stop them.
Gandalf hasn't been popping out to "YOU SHALL NOT PASS!" criminally determined perverts.