NVIDIA TESLA P4 8 GB

Tesla P40 vs V100 for a local LLM/RAG - 24 GB of VRAM or the faster Volta?

In our measurements the V100 generated text 2.6–3.1× faster than the P40 with the same model and quant, but the first question is whether the bottleneck of your RAG is generation, or retrieval and context processing. Data from one model, one backend and one methodology for three cards.

28 October 2026

Why the prompt is slow on a P40 - and how to really improve it

The Tesla P40 generates tokens decently, but processing a long prompt (prefill) is slow on it, because the P40 (GP102) has no Tensor Cores and does not offer high FP16 throughput.

21 October 2026

2× GPU for LLMs: how llama.cpp splits a model between P40 and V100

Two cards will let you run a model that does not fit on one, but not because the VRAM adds up into one big pool. llama.cpp and Ollama split the model into layers and distribute them between the cards.

14 October 2026

Whisper in Polish on a GPU - transcription and subtitles locally (GDPR)

Whisper (and preferably faster-whisper) transcribes Polish very well, and locally on a GPU it does so quickly and without sending recordings to an external service. In our test the V100 32GB reached ~15× realtime, and the P40/A2 ~6×.

7 October 2026

RAG on company documents offline - what hardware and how to start (GDPR)

What hardware for local RAG over company documents: the split between embeddings (P4/A2) and the LLM (V100/P40), the stack, GDPR/ACL/OCR and an honest comparison of your own hardware vs the cloud.

30 September 2026

How much power does a used Tesla draw - and which gives the most tokens per watt

How much power used accelerators (P4, A2, P40, P100, V100) really draw and which has the best ratio of tokens per watt, from our own power-draw measurements.

16 September 2026