Short answer: a card's power draw and its efficiency are two different things, and confusing them costs money. The A2 and P4 draw so little power (60–70 W) that they are suited to 24/7 operation where the bill hurts, but they have lower compute performance. Under full load the V100 draws about 4× more power than the A2, and it still has the highest tok/s per watt in our test, because single-stream token generation is limited mainly by memory bandwidth and the V100's 900 GB/s HBM2 turns the same power budget into clearly more tokens. The Pascal cards have the lowest tok/s per watt: the P40 draws almost 240 W, the P100 about 200 W, and both deliver the fewest tokens per watt of the whole field (while still drawing much more power than the A2 and P4). You buy them for their VRAM and price, not for energy efficiency. Below are our measured figures: how much they really draw, how hot they run and how much useful work comes out of it.
I am writing this for people who plan to keep a card switched on all the time: an inference server, a transcoder, a RAG workstation running in the background. With occasional use, power draw does not matter; with continuous operation it is a budget item that can exceed the price of a cheap card within a year. We measured the power draw of every model we sell, so you do not have to guess from the TDP on the box.
Hard numbers first: how much they really draw
These are not catalogue values. We burned in every card with gpu-burn at the full power limit, with an additional nvidia-smi process sampling power draw every second. The "load" column is the average under full load, "max" is the peak for sizing the power supply, and "Avg. of max temp." is the average of the maximum temperatures of the individual units of a given model (e.g. the P4 from 25 units, the P40 from 41):
| Card | Idle | Load (avg.) | Max (peak) | Avg. of max temp. |
|---|---|---|---|---|
| P4 8 GB | ~13 W | ~64 W | 70 W | ~59 °C |
| A2 16 GB | ~7 W | ~60 W | 62 W | ~68 °C |
| P100 16 GB | ~29 W | ~199 W | 223 W | ~62 °C |
| P40 24 GB | ~21 W | ~237 W | 255 W | ~64 °C |
| V100 16 GB | ~30 W | ~242 W | 256 W | ~65 °C |
| V100 32 GB | ~32 W | ~242 W | 267 W | ~67 °C |
Two things stand out straight away. First, there is a clear split into two leagues: the P4 and A2 at around 60–70 W, and all the rest at around 200–250 W. Second, the A2 hits its 60 W cap: its power limit is configurable between 40 and 60 W, and 60 W is both the default and the maximum, so it does not go higher by design (the 62 W peak is a recorded reading above the cap, with sampling once per second). That is a feature of the card, not a limitation of the test bench.
When sizing the power supply, do not look only at the average draw: take into account the GPU's maximum draw, the platform's requirements and headroom for short load spikes. Two 250 W cards do not mean "a 500 W power supply is enough". But that is a topic for a separate post; here we are talking about the electricity bill, and that is calculated from the average under load.
Why power draw alone misleads: back to the point of the card
If only watts counted, the conclusion would be trivial: take the A2, it draws the least. But a card is not there to draw little power: it is there to compute something. So the question is a different one: how much useful work do you get for that energy?
Let us take LLM token generation (tg128, the same gemma-2-9B model in Q4 on every card; the more tok/s, the better) and set it against the power draw under load. "Tok/s per watt" is a simple quotient: how many tokens per second are generated per watt drawn. Formally it is tok/s/W, i.e. tok/J; we treat this value as a simple relative comparative indicator. One caveat up front: it is a relative indicator, not a measured inference efficiency: the tok/s comes from the LLM benchmark and the power draw from the steady-state burn-in, not from the same run. It serves to compare the cards with each other, not as an absolute count of tokens per joule.
| Card | tg128 (tok/s) | Load (W) | Tok/s per watt (relative) |
|---|---|---|---|
| P4 8 GB | 20.5 | ~64 | ~0.32 |
| A2 16 GB | 19.6 | ~60 | ~0.33 |
| P100 16 GB | 30.2 | ~199 | ~0.15 |
| P40 24 GB | 31.4 | ~237 | ~0.13 |
| V100 16/32 GB | 89.0 | ~242 | ~0.37 |
And this is where it gets interesting. The V100 draws about 4× more than the A2 (and only slightly more than the P40), and at the same time it has the highest tok/s per watt in this comparison. The V100 reaches 89 tok/s where the P40, at almost identical power draw (237 W), delivers 31.4. Against the P40, much of the V100's advantage can plausibly be explained by memory bandwidth (900 vs up to 347 GB/s): single-stream token generation is generally bandwidth-bound, and llama.cpp's quantised MMVQ matrix-vector kernels do not use Tensor Cores. The P100 is a separate case: it has 732 GB/s of HBM2, but as a compute capability 6.0 card it lacks native dp4a. llama.cpp emulates this operation below compute capability 6.1, which may affect performance. Without identifying the build and profiling its kernels, however, we cannot attribute the P100 result exclusively to that software path or rule out other bottlenecks. High power draw does not mean "a poor ratio": here it means "computes a lot".
The opposite pole is Pascal: the P40 and P100. The P40 draws almost as much as the V100 (~237 W), the P100 less (~199 W), but both reach about a 2.4–2.8× lower tok/s per watt than the V100. In our measurements, both Pascal cards draw about 200–240 W and generate ~30 tok/s, while the V100 reaches 89 at a similar draw. The P40 has GDDR5 with up to 347 GB/s bandwidth; the P100's lack of native dp4a and the associated llama.cpp software path may affect its result. The benchmark alone does not isolate these factors. If you buy a P40, you are buying its 24 GB of VRAM and its low price, not energy efficiency, and it is worth knowing that before you put it into continuous operation.
The A2 and P4 land in the middle on the ratio, but they win on something else: absolute power draw. 0.33 tok/s per watt is not a record, but at a power draw of about 60 W the card can run 24/7, and the cost of running it continuously is much lower than for cards drawing 200–250 W. That is a completely different argument from tok/s per watt; more on that in a moment.
An honest methodological note: I calculate this column from the burn-in power draw, and real LLM inference is often lighter (a P40 running llama.cpp alone draws ~175–183 W, not 237). Under a real AI load, Pascal therefore comes out slightly better than the table shows. Treat tokens per watt as a relative ranking, not a count down to the watt: the direction does not change, the proportions may shift.
Three questions, not one, because "efficiency" means something different to everyone
The gap between "few watts" and "many tokens per watt" is not a contradiction in the data. They are two different questions that people merge into one "which is the most economical". Let us separate them:
Do you want the lowest bill at the end of the month? Look at absolute power draw, not efficiency. An A2 (~60 W) or a P4 (~64 W under full load) running 24/7 costs a fraction of what Pascal or Volta does. Assuming ~PLN 1.0–1.3/kWh (business tariffs in Poland, 2026; check your own), an A2 under full load is about 1.44 kWh a day for the card alone (0.060 kW × 24 h), i.e. of the order of PLN 1.4–1.9 a day. A P40 under a real LLM load (~180 W) is already about 4.3 kWh and about PLN 4.3–5.6 a day. Both figures assume load around the clock. A server that mostly waits for requests spends much of the day near idle; note that a card holding a model in VRAM may not drop to the idle values in our table (forum reports say the P40 stays in its highest power state while a CUDA context is open), so measure the waiting state too. The host platform adds its own draw on top, whichever card you choose. Over a year of continuous operation, under these power-draw and electricity-rate assumptions, the difference is about 1,050 kWh, or roughly PLN 1,050–1,370.
Do you want to do the most computation for the electricity you use? Look at tok/s per watt. The V100 wins here: the highest ratio of the whole field, despite the highest absolute power draw. On our indicator the V100 leads, but only about 13–15% ahead of the A2 and P4; its lead over the P100 and P40 is about 2.4× and 2.8× respectively. Because this indicator combines generation throughput with power draw from a separate stress test, it does not establish actual inference energy efficiency. That requires measuring power during the same LLM run.
Do you want to compute the most within a given time window, with electricity secondary? That is no longer a question about energy but about performance, and then you look at tok/s without dividing by watts. But since you are reading a post about power draw, that is probably not what you are after.
The most common mistake I see: someone takes a P40 "because it is cheap and has 24 GB", puts it into 24/7 operation and after a quarter is surprised by the bill. The P40 is an excellent card for its memory capacity, but in energy terms it is expensive to run. If your model fits on an A2 or a V100 and the card is going to run non-stop, calculate the electricity cost before you are tempted by the cheapest price tag.
What follows: choosing for the workload profile
- A light workload where low absolute power draw matters most (transcoder, light inference, edge): A2 or P4. 60–70 W of GPU power draw, much easier to cool than 200–250 W cards. Lower compute performance, but entirely sufficient for low-intensity tasks, and that is their proper place.
- An inference server under full load where tok/s per watt matters: V100. It draws a lot, but has the highest tok/s per watt in our benchmark. On our relative tok/s-per-watt indicator it is far ahead of the P40 and P100 and roughly level with the A2 and P4, while generating more than four times as many tokens per second as the A2 and P4. Actual continuous-inference energy efficiency needs to be compared using power measured during the same LLM workload.
- Large VRAM for little money, electricity secondary: P40. You take it for its 24 GB and its price, knowingly paying more for electricity. Sensible if you use it occasionally or capacity matters more than the bill.
- FP64/HPC on a budget: P100 (4.7 TFLOPS FP64; the V100 gives 7 TFLOPS if the budget allows), but that is a different subject (double precision), not energy efficiency; in this LLM test it is as "expensive" in energy terms as the P40.
How can you check on your own system before you trust our figures? One command in the background during your real workload:
nvidia-smi --query-gpu=power.draw,utilization.gpu --format=csv -l 1
Leave it running for a few minutes under a task typical for you and look at the average. That is your real power draw, much closer to the truth than the TDP on the card, because it depends on the model, the context and how hard you actually load the card.
One final thought
The "250 W TDP" label does not tell you how much the card will actually draw during your workload. Nor does it tell you how much you will pay for electricity or whether the card is energy-efficient: for that you need real measurements of power draw and performance. The V100 draws the most, and yet it has the highest tok/s per watt in our benchmark. The A2 draws the least and is suited to running non-stop. The P40 draws a lot and is expensive in energy terms, and yet it is often the best choice when its VRAM is what counts. Three different truths from one power-draw table. Choose the one that fits how you will really use the card.
Sources
- Agile Store's own measurements. Power draw and temperature:
gpu-burnburn-in at the full power limit, power draw sampled every 1 s bynvidia-smi --query-gpu=power.draw, value averaged over the whole run. Performance (tg128):llama-bench -ngl 99 -p 512 -n 128 -r 3on the gemma-2-9B Q4 model, driver 580.173.02 / CUDA 12.0. The "tokens per watt" indicator = tg128 ÷ average power draw under full load. - Indicative electricity rate used (~PLN 1.0–1.3/kWh, business tariffs in Poland, 2026): to be confirmed against the reader's actual contract; the annual amounts illustrate the method and are not a guarantee.