This doesn't mean an indie ref's job is easier than a TV guy's (though the lack of crazy old men screeching in our ear every ten seconds is certainly a plus). Most indie shows draw anywhere from five to 500 people -- it's way more intimate than your average WrestleMania. So a silent, lethargic referee who's just there to count pinfalls will quickly stick out in an "oh right, this shit's fake" kind of way.
So when not asking for submissions or scolding muscleheads for breaking rules that they'll re-break 30 seconds later, I focus on being an active background character: making hand motions that subtly direct the action for the audience, intensely watching the action so I can quickly jump in and react when necessary, and constantly shifting position so I'm in the proper place should shoulders hit mat. If I'm not going to guide the action, I should at least make sure to not come across as a creepy stalker just hanging out in the corner.
East Coast Championship Wrestling
Here I am unleashing my greatest power: making women not listen to me.
I say all this because so few do the same.