NVIDIA TESLA V100 32 GB

Tesla P40 vs V100 for a local LLM/RAG - 24 GB of VRAM or the faster Volta?

In our measurements the V100 generated text 2.6–3.1× faster than the P40 with the same model and quant, but the first question is whether the bottleneck of your RAG is generation, or retrieval and context processing. Data from one model, one backend and one methodology for three cards.

28 October 2026

2× GPU for LLMs: how llama.cpp splits a model between P40 and V100

Two cards will let you run a model that does not fit on one, but not because the VRAM adds up into one big pool. llama.cpp and Ollama split the model into layers and distribute them between the cards.

14 October 2026

Why the prompt is slow on a P40 - and how to really improve it

The Tesla P40 generates tokens decently, but processing a long prompt (prefill) is slow on it, because the P40 (GP102) has no Tensor Cores and does not offer high FP16 throughput.

21 October 2026

PLLuM locally on a used Tesla - how to run it and how much VRAM

You can run PLLuM locally on a used Tesla like any GGUF model, through Ollama or llama.cpp. This article covers how much VRAM you need, which card (P40/P100/V100) to choose and how to get it running.

20 August 2026

Benchmarks: Bielik and PLLuM on the P40/P100/V100 - real tokens/s with prefill time

We measured Bielik and PLLuM on used Teslas: real tokens/s with prefill time, not just generation alone. The figures, the measurement conditions and a template for your own verification.

13 August 2026

V100 or A2 for fine-tuning models? An honest comparison of two used cards

The short answer before I go into detail: if you are planning real fine-tuning, even with LoRA, and compute throughput on large batches matters, go for the V100, ideally the 32GB version. If you are building an economical server on which fine-tuning is an addition to inference, and low power consumption and a modern stack (BF16, TF32) are the priority, the A2 will be the better choice.

16 July 2026