They can also appear when we feel sexually aroused or when we feel in awe of something, like listening to a moving piece of music, or if you're the type, watching monster trucks smash smaller cars (to each his own).
Ever see the fur on the back of a scared or angry animal suddenly stand straight up?
It's that. Goosebumps are caused by the tiny muscles at the base of your hair follicles contracting, causing our hair to stand on end and covering our skin with tiny bumps.
There is really no reason to have this reaction anymore as it's of no use to us. But it was useful to our ancestors who were a lot hairier.
Goosebumps raise the hairs on an animals for two main reasons. First, if the animal is scared, its hair stands on end to appear bigger which might be enough to make a predator change their mind about going after it.
Above: not bear-food.
The second major way goosebumps used to help us out was by keeping us warm when we would have been freezing our asses off. When our ancestors were feeling a little too chilly, their hair would stand on end, trapping a layer of air close to the skin so it could be used as an insulator. It's an ability that most mammals covered in hair still enjoy.
For... um, warmth.