NVIDIA TESLA V100 16 GB

Tesla P40 vs V100 for a local LLM/RAG - 24 GB of VRAM or the faster Volta?

In our measurements the V100 generated text 2.6–3.1× faster than the P40 with the same model and quant, but the first question is whether the bottleneck of your RAG is generation, or retrieval and context processing. Data from one model, one backend and one methodology for three cards.

28 October 2026

2× GPU for LLMs: how llama.cpp splits a model between P40 and V100

Two cards will let you run a model that does not fit on one, but not because the VRAM adds up into one big pool. llama.cpp and Ollama split the model into layers and distribute them between the cards.

14 October 2026

PLLuM locally on a used Tesla - how to run it and how much VRAM

You can run PLLuM locally on a used Tesla like any GGUF model, through Ollama or llama.cpp. This article covers how much VRAM you need, which card (P40/P100/V100) to choose and how to get it running.

20 August 2026

Benchmarks: Bielik and PLLuM on the P40/P100/V100 - real tokens/s with prefill time

We measured Bielik and PLLuM on used Teslas: real tokens/s with prefill time, not just generation alone. The figures, the measurement conditions and a template for your own verification.

13 August 2026

Why the prompt is slow on a P40 - and how to really improve it

The Tesla P40 generates tokens decently, but processing a long prompt (prefill) is slow on it, because the P40 (GP102) has no Tensor Cores and does not offer high FP16 throughput.

21 October 2026

RAG on company documents offline - what hardware and how to start (GDPR)

What hardware for local RAG over company documents: the split between embeddings (P4/A2) and the LLM (V100/P40), the stack, GDPR/ACL/OCR and an honest comparison of your own hardware vs the cloud.

30 September 2026