According to Joker, this "distinguished gentleman" (as the script calls him) bore a resemblance to his own dad (especially when the gentleman stood up to this "thug"). In the Nolan-verse, who was the last heroic, distinguished gentleman who stood up to a thug? The gentleman who currently wears bat-themed body armor and the guy who serves him tea don't count.
So for now, Doctor Thomas Wayne is our number-one suspect. And if you think that this is a leap, remember that every Batman film since Tim Burton's first has had an "Oh, s**t. YOU'RE HIM" revelation moment. Every. Single. One. (That matters.)
Here, if you found The Dark Knight to be too morally convoluted, Christopher Nolan takes the time to tell you all of the themes of the movie directly: