A battery is rated with both a voltage and with a capacity. These are both important things to know, but most people only pay attention to the voltage of a battery (for example, people might ask for a 9 volt, never specifying the battery's capacity). Voltage can be thought of as the potential that a battery has to deliver a certain charge. If a battery was a water tower, then the voltage could be thought of as the pressure found at the base--the potential energy. A battery's capacity, on the other hand, is how long that battery can provide its voltage. Capacity is measured in mAh, or the number of amps the battery can provide over a certain number of hours.
The relationship between a battery's voltage and its capacity is fairly simple. Think of two batteries as two streams of water, one big and one small. If a small pipe is placed in each stream (the pipe represents the voltage), both streams (batteries) can provide the necessary flow. However, the larger stream has more water in it, so it has a larger capacity and more voltage can come out of it for a longer time. Voltage is a measure of a battery's power, whereas capacity is the measure of the battery's endurance. Usually, the higher a battery's voltage, the higher its capacity as well.
If a battery is placed in a circuit where it gives out less than its maximum voltage, it won't be using as much of its potential and, therefore, it will last longer. On the other hand, if a battery is putting out above its voltage rating (say if it was plugged into a circuit that would suck more power from the battery than it was meant to put out), the battery will die more quickly. The more voltage that's going out of a battery, the smaller its overall capacity becomes.