Although a water heater tank is basically an overgrown Thermos bottle, we inspectors see common installation issues that can significantly reduce safety. Obviously, safety features won't be tested unless there is a failure in the system. Because the water heater is working fine right now doesn't mean it will behave when you most need it to.
Common safety issues include lack of or poorly installed expansion tanks (see expansion tank post elsewhere in plumbing tags) and strangely plumbed Temperature Pressure Relief valves. It is difficult to imagine what may happen if the TPR valve discharges (due to a failure in the thermostat). Think HUGE pressure cooker suddenly with a 3/4 inch hole in it. One cannot use PVC pipe for the TPR discharge pipe because PVC can't take the pressure and these pipes should be six inches from the floor otherwise the super heated spray will either splash up too much or out too much.
Failures include the bottom rusting out and flooding the home and expansion tanks filling with water and rupturing pipes. Yet if the thermostat fails and the TPR valve fails you have a steam rocket. Manufactures recommend replacement very 8 to 12 years. Reasons for replacement include integrity of the tank, tightness of the seals to the plumbing system, and age of the fail safes (thermostat and TPR valve).
Weird things can happen in a water heater's youth too. Expansion tanks can fail in two years. A mouse got caught in my power vent water heater's vent tube and the unit overworked and turned the PVC vent tube pink. After 4 years the tank started leaking from the exterior panels. So that mouse cost me about $1500.
The bottom line is safety. Plumbers recommend not keeping a water heater for more than 15 years.
Other problems caused by old water heaters include steel flakes from rusting tanks settling on your copper pipes and causing galvanic corrosion. I have seen whole lengths of horizontal copper pipes with green dots at the bottom, which eventually leak. Also, when water heaters become filled with scale deposits they become much less efficient in heating the water.
Here is a picture of a water heater with so many metal flakes that they came out in the tub! Such corrosion is what the sacrificial anodes help prevent: they provide a more reactive surface than steel.
Matt doesn't do his homework before he starts with an issue but this is fun and shows a variety of the inside of tanks. Calcium deposits occur in water heaters because those salts are actually less soluble in warm water, unlike other salts.
Water heater tanks are never supposed to be hot. The insulation provided by the manufacturers inside the tank are quite adequate. However, if there are drafting issues, too much sediment or a missing pilot light door, the outside can get hot. Found my first one after 1500 inspections. Replace it immediately and check for poor drafting. Do not want a superheated water discharge from the TPR tube or worse.