"Actually, if you traveled back in time and encountered yourself, the space-time continuum and the universe itself would collapse."
I'm not talking about all uses of the word "actually" here, I'm talking about when people use it in response to a statement that's clearly meant as a joke. This comes from people who always feel like they have to have the last word, or otherwise "top" what you just said.
"Oh, that is precious. Actually, if those three religious fellows were in England, they'd be in a pub, not a bar."
Why People Do It
I once made a joke about how I'd rather be shot into the core of the sun than watch a full season of anything written by Joss Whedon. It was a quick jab at a group of my friends who would bottle the air if they knew it contained one of his farts. What I got in response was a complete stranger butting in with "Actually, you'd never make it to the center of the sun because the temperature of its atmosphere is 300 times hotter than the surface. You'd disintegrate long before you got there."
Hey, I have to give him this much: His knowledge of high school science trivia must have worked, because I didn't shoot myself into the core of the sun. Of course, he didn't have any ill intentions when he butted into the conversation. He had this fact collecting dust in his head, and when he heard me make fun of something he liked, using that analogy, he saw his opportunity and took it.