That sounds crazy looking at the picture up there, but no picture of space really conveys the distances. For instance, once upon a time, our asteroid belt had way more asteroids in it (about a thousand times more, in fact). Even if Han had to fly through that density of asteroids, he'd find that each asteroid has a mind-boggling 400,000 square miles to itself.
It's empty. That's why it's called space.
You could argue that maybe in the particular galaxy Star Wars takes place in they for some reason have superdense asteroid belts, but that's actually impossible -- the whole problem is that over time, the asteroids will disperse. If they were as close as the Star Wars belt, for instance, each time an asteroid bumped into another, they would go flying off into outer space, with nothing to stop them.
This means that actually getting hit by an asteroid in an asteroid belt is less a matter of not paying attention and more a matter of veering off course by a good couple of million miles. And actively trying to find an asteroid. And then somehow intercepting it at the perfect time, with the perfect velocity and the accurate trajectory. You'd need a hell of a space pilot with a hell of a death wish.
And a tolerance for giant space cocks.