Benchmarks: Bielik and PLLuM on the P40/P100/V100 - real tokens/s with prefill time

The short answer: cheap VRAM does not mean "fast". We measured Bielik and PLLuM (two Polish-language LLMs) on used Teslas, and the differences are large. Token generation on the P40 is acceptable (~28 tok/s), but prefill (processing the prompt before the first word appears) is slow in absolute terms: the P40 manages ~590 tok/s prompt eval, and the V100 (Volta) ~1960, more than three times faster. In our test the P100 shows no advantage over the P40, despite its faster HBM2 memory. Below you will find our measured figures together with the conditions, plus a ready-made template so that you can verify them on your own card.

I write this from the perspective of someone who sells these cards and at the same time is tired of benchmarks that show only one figure: "how many tokens per second it generates". That is half the truth, and the more flattering half. A customer buys a Tesla P40 24GB, starts Bielik, types a short "hi, how are you" and sees an acceptable speed. Then they paste in a page of text to summarise, or build a RAG pipeline with a long context, and suddenly they wait longer before the model even starts writing. This is not a fault. It is prefill. And most guides say nothing about it.

Two phases you need to distinguish

Every request to an LLM has two phases with completely different performance characteristics.

Prefill (prompt processing / prompt eval). The model has to process your entire prompt (all the input tokens) before it generates the first token of the response. In a typical interactive scenario (batch = 1) this phase is much more compute-bound: it places a load on the compute units (matrix multiplication). Here raw GPU compute power gives the advantage, and on Volta additionally the Tensor Cores, provided the backend's compute path uses them. The longer the context (system prompt + documents + conversation history), the longer prefill takes. In llama.cpp you will see this as prompt eval time and a tok/s metric (pp).

Generation (decode / eval / token generation). After prefill, the model produces the response token by token. In the same batch = 1 scenario this phase is usually limited by memory bandwidth: what matters is how quickly the card reads the model weights from VRAM. Here bandwidth sets the ceiling: the V100's HBM2 (900 GB/s) gives it a clear advantage over the P40's GDDR5 (346 GB/s), which is still enough for reasonable speeds on 7-13B models. A ceiling is not a guarantee, though: in our test the P100 (732 GB/s) generated no faster than the P40. A plausible explanation is llama.cpp's less favourable compute path on the P100, including its lack of hardware __dp4a support (see below).

The key conclusion: VRAM decides whether the model will FIT. Memory bandwidth decides how fast it GENERATES. Compute power decides how fast it processes the PROMPT (and on Volta the Tensor Cores help with this). The P40 wins on capacity (24 GB) and loses on prefill. This is not a hidden defect: it is the physics of the Pascal architecture, which has no Tensor Cores at all.

Why these particular cards differ so much

Short, established facts:

• P40 (Pascal): 24 GB of GDDR5, no Tensor Cores, weak FP16 (it computes natively in FP32/INT8). The most VRAM for the least money. Prefill in our test was clearly slower; the P40 has neither Tensor Cores nor the hardware matrix acceleration available in Volta.
• P100 (Pascal): 16 GB of HBM2, no Tensor Cores, but full-rate FP16. Better bandwidth than the P40 and comparable generation, while prefill, surprisingly, came out even lower than on the P40 in our test. A plausible explanation is how the backend selects compute paths for each card's hardware capabilities. The P40 supports the hardware __dp4a INT8 dot-product instruction (compute capability 6.1), while the P100 does not (6.0). In current llama.cpp code this instruction accelerates some quantised kernels; on the P100 the dot product is emulated, and dense-model prefill may use dequantisation and floating-point computation instead of MMQ kernels. This can limit P100 performance despite its fast HBM2 memory. The P100 also has fewer CUDA cores (3584 vs 3840), but core counts alone cannot explain the result. Without the build identifier and profiling, we cannot determine each factor's contribution.
• V100 (Volta): 16 or 32 GB of HBM2, first-generation Tensor Cores. This is the card that in our test genuinely speeds up prefill compared with the Pascal cards: ~3.3× compared with the P40 and ~4.7× compared with the P100. How much of this advantage comes from the Volta architecture itself and how much from the Tensor Cores, the benchmark does not settle; it shows the result, not its breakdown. More expensive, but if a long context or the time to the first token is what hurts, the difference is qualitative, not cosmetic.

Results table: our measurements

Below are real figures from our test bench. Treat them as a point of reference, not as gospel, and be sure to measure your own configuration: the driver and the llama.cpp build can shift the result, so the best benchmark is the one on your card. You will find a ready-made procedure and command below, in the section "How to measure it yourself".

Card	VRAM	Model / quant	Prefill (pp512, tok/s)	Generation (tg128, tok/s)
P40 24GB	24 GB GDDR5	Bielik 11B Q4_K_M	593.6	28.5
P40 24GB	24 GB GDDR5	PLLuM 12B Q4_K_M	612.2	27.1
P100 16GB	16 GB HBM2	Bielik 11B Q4_K_M	419.0	27.2
V100 16GB	16 GB HBM2	Bielik 11B Q4_K_M	1963.0	77.7
V100 32GB	32 GB HBM2	Bielik 11B Q4_K_M	1963.0	77.7
V100 32GB	32 GB HBM2	Bielik 11B Q5_K_M	1986.1	70.0

Measurement conditions (keep them constant if you are comparing): llama.cpp llama-bench, GGUF, the whole model on the GPU (-ngl 99); prefill measured on a 512-token prompt (-p 512, micro-batch -ub 512), generation of 128 tokens (-n 128), 3 repetitions (-r 3). Every card at its maximum power limit (P40/P100 = 250 W: at a lower, throttled limit prefill can drop by several tens of per cent). NVIDIA driver 580.173.02, CUDA 12.0, cards on PCIe Gen3. The V100 16 GB and 32 GB use the same GV100 chip: on the same Q4_K_M quant they give an identical result (1963 / 77.7), so the silicon decides the speed, not the capacity; the only difference between them is how large a model will fit (32 GB fits larger models). Note: the third V100 row (Q5_K_M) is a different quant: do not compare it directly with the Q4 rows; a higher quant reads more data from VRAM, hence the lower generation speed (70.0 vs 77.7). Bielik-11B-v2.3-Instruct and PLLuM-12B-instruct (the commercial variant); we measured PLLuM on the P40 only.

Two things these figures show, contrary to popular myths:

1. The gap is already there with a short prompt: it is the absolute prompt eval throughput. The time to process a 512-token prompt alone can be approximated as prompt_length ÷ pp: P40 ~0.86 s (512 ÷ 594), V100 ~0.26 s (512 ÷ 1963). This is one component of time to first token, often the dominant one when the model is already loaded and the request is not queued. A longer context (system prompt + RAG documents) requires more computation, including more attention work. Runtime need not grow exactly in proportion to prompt length: GPU utilisation and backend settings also matter. In this comparison we measured 512 tokens only, so run -p 2048 (or your real prompt length) on your card to measure the increase. A longer prompt can mean a longer wait when you paste in a page of text instead of a short question.
2. Generation on the V100 is in a different league: ~78 tok/s vs ~28 on the P40/P100. This roughly matches the memory bandwidth ratio (900 vs 346 GB/s, 2.6×); Tensor Cores play little part here, because generating one token at a time is a matrix-vector operation, not the large matrix multiplication they accelerate. For interactive chat, 28 tok/s is perfectly comfortable; when serving many users or producing long responses, the difference becomes noticeable.

Links: V100 16GB, V100 32GB, P100 16GB.

Verdict

Based on measured figures, not hunches:

• P40 24GB: budget and capacity. The best ratio of VRAM to price. A 13B model will fit with room to spare, and 70B in a lower quant will fit on two cards. Generation at ~28 tok/s is comfortable for chat. The trade-off: prompt eval at ~590 tok/s is ~3× less than the V100, and with a long context you will feel it as a longer "time to first token". If you use short prompts and value plenty of VRAM for little money, it is an excellent choice. More on the P40 in practice: Tesla P40 + Ollama.
• P100 16GB: the weakest result in our llama.cpp (GGUF) test. HBM2 did not translate into an advantage for us: prefill ~419 tok/s (lower than the P40), generation ~27 tok/s (like the P40), and 16 GB limits larger models. It makes sense only at a really good price.
• V100 16/32GB: power when prefill or generation hurts. The V100, with the Volta architecture and Tensor Cores, achieved ~1960 tok/s prompt eval and ~78 tok/s generation in our test. If you are building RAG, feeding in long documents or serving many users, the difference is noticeable here. You pay more and get a faster "time to first token" and higher throughput. If the V100 is to work as part of a multi-card platform, we also offer Dell R740 servers with 1–3× Tesla V100 32 GB.

I break down how much VRAM you really need for a given model in a separate article: how much VRAM for LLaMA 7B/13B/70B.

How to measure it yourself: step by step

Do not take our word for it: verify it on your own machine. It is the only way to know whether your use case will get stuck on prefill.

1. Build llama.cpp with CUDA support using a CUDA 12.x toolkit (CUDA 13 no longer compiles for Pascal and Volta): cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES="60;61;70" (60 = P100, 61 = P40, 70 = V100).
2. Download the Bielik and PLLuM GGUF files in Q4_K_M and Q5_K_M.
3. Set the card to its maximum power limit (sudo nvidia-smi -pm 1 && sudo nvidia-smi -pl <max>), otherwise you will underestimate prefill throughput.
4. Run llama-bench (example below) and record pp and tg for each combination.
5. Repeat with the card under load (a second process) if you are planning a multi-service production setup.
6. Note the CUDA version, the build and the driver: without them you cannot compare results between machines.

llama-bench -m Bielik-11B-v2.3-Instruct.Q4_K_M.gguf -p 512,2048 -n 128 -ngl 99 -r 3

-p sets the prompt lengths (prefill), and -n the number of tokens generated. You will get the result split into pp (prompt processing) and tg (token generation). Those are the two figures that matter.

Sources

• Bielik: SpeakLeash (weights and variants on HuggingFace): https://huggingface.co/speakleash
• PLLuM: CYFRAGOVPL (weights and variants on HuggingFace): https://huggingface.co/CYFRAGOVPL
• Bielik GGUF files (Q4_K_M, Q5_K_M): https://huggingface.co/speakleash/Bielik-11B-v2.3-Instruct-GGUF
• llama-bench (parameters and output format): https://github.com/ggml-org/llama.cpp/blob/master/tools/llama-bench/README.md

---

Yes, we sell these cards (P40 24GB, V100 16GB, V100 32GB, P100 16GB), so treat this article with appropriate caution. But that is exactly why we give the measurement conditions and a template for verifying them yourself: fewer returns, fewer disappointments. We put every card through a burn-in test and issue a certificate. Do you have an unusual scenario, or would you like us to help you choose a card for your context? Write to us.