GPUs for a laboratory or university without an H100 budget - how to build an AI/HPC workstation for less

If you conduct research, manage a grant budget or buy equipment for a student research group, sooner or later you hit the same wall: a new data-centre accelerator costs as much as a year's stipend for several PhD students. The answer I arrived at myself after several purchases for my own testing needs is unglamorous: buy used data-centre hardware, choose the card for the specific task rather than for the marketing, and accept that this hardware will not last you a decade. For most laboratory and teaching applications it will be more than enough anyway.

Why used hardware makes sense in research at all

The hardware I am talking about (the Tesla V100, P100 and P40, and the NVIDIA A2) consists of cards that a few years ago stood in the server rooms of cloud providers and computing centres. They reach the secondary market after leasing cycles end or clusters are upgraded. The architecture and performance do not change from the moment they go on sale: you get exactly the same silicon (CUDA cores and, from Volta onwards, Tensor Cores) as new hardware of that generation, only for a fraction of the original price.

This is not an argument of "because it is cheaper", but of "because it fits within the budget at all". A grant for research equipment or the budget of a student research group rarely provides tens of thousands of PLN for a single card. For similar money you can set up several workstations with used data-centre cards and actually teach students or run experiments, instead of waiting a year for a decision to buy one unit of top-end hardware.

To be fair: this is not hardware for production model training at the scale of an AI company. An older architecture means no access to the latest optimisations: no FP8 on any of these cards, and no BF16 or FlashAttention-2 on Pascal and Volta (the A2, as an Ampere card, has both). Current vLLM also needs compute capability 7.5 or higher, so it does not run on the P40, P100 or V100; on those cards LLM serving means llama.cpp or Ollama. Add lower memory bandwidth than the current generation and a shorter realistic period of driver support. But for research work, prototyping, teaching and most training pipelines of the "fits on a single card" kind, these cards are more than adequate.

Matching the card to the task: there is no single best one

The biggest mistake I see with such purchases is buying "whatever is most powerful according to some ranking" instead of matching the card to the real workload. The four cards I describe below have genuinely different strengths, and this is not an exhaustive list. I discuss the models that most often end up in labs and that we happen to have in our current range; if you need a different accelerator (for example, a newer one or one with more VRAM), we can source it to order, with a quotation on request.

V100: the workhorse for deep learning

The V100 (and the 32 GB version) is a Volta card with first-generation Tensor Cores and HBM2 memory with a bandwidth of around 900 GB/s. It is a sensible choice if your main activity is training neural networks: classic deep learning, fine-tuning smaller models, experimenting with architectures. It also has strong FP64 (about 7 TFLOPS), so it is not a card for AI alone: it is also suitable for scientific computing that requires precision.

The 32 GB version matters in practice for larger batches or experiments with longer sequences: you do not have to trim the data size quite so aggressively to fit into memory.

P100: the specialist for HPC and simulations

The P100 is an older card (Pascal), with 16 GB of HBM2 and 732 GB/s of bandwidth. At first glance it is weaker than the V100, but it has one feature that makes it a sensible choice for specific tasks: full FP16 support and strong FP64 (a 1:2 ratio, about 4.7 TFLOPS) for an inexpensive card of this class. If your workload is not network training but numerical simulations, physics calculations, computational chemistry or other HPC tasks where double-precision accuracy matters, not just matrix throughput, the P100 is an inexpensive way into double precision. If you need the highest absolute FP64 in this comparison, that is the V100 (about 7 TFLOPS); see the section above.

P40: plenty of VRAM for reasonable money, for teaching and inference

The P40 has 24 GB of GDDR5 memory: more than any 16 GB card here, and the cheapest way in this comparison to get more than 16 GB. The catch: weak FP16 and weak FP64, so it is not particularly suited to training. Where it excels is teaching (students learn to deploy models and write pipelines, and do not need maximum performance) and inference of larger language models, where memory capacity matters above all, not raw speed. If a student research group wants to set up a workstation for experimenting with local LLMs without breaking the bank, it is the natural candidate.

A2: a low-power inference workstation and edge

The A2 is an Ampere-architecture card with 16 GB, low power consumption and a longer realistic period of support than older generations. The A2 is aimed primarily at inference, edge computing and demonstration workstations, although it can also train smaller models. It is passive and requires directed airflow. Its power limit is configurable from 40 to 60 W. In a desktop, plan for a fan and duct; fan selection and noise depend on the enclosure, ambient temperature and workload.

Table: task → recommended card

Task	Recommended card	Why
Neural network training, deep learning	V100 16/32 GB	1st-gen Tensor Cores, fast HBM2, a good FP32/FP16 balance
Scientific simulations, FP64 computing, HPC	P100 (or V100)	P100 = usually the best FP64-to-price ratio among these models at current used prices; the V100 PCIe has the highest absolute FP64 (7 TFLOPS)
Teaching, LLM inference with a large context	P40 24 GB	Plenty of VRAM for a reasonable price; llama.cpp and Ollama can use paths that avoid slow FP16 arithmetic. Current vLLM requires compute capability 7.5 or higher. Native FP16 runs at 1/64 of FP32, so do not expect the usual mixed-precision training speedup; results depend on the kernels and model
Inference workstation, edge, office setup	A2 16 GB	Low power limit (40-60 W), newer architecture, directed airflow required
Experiments with larger batches/contexts on the V100	V100 32 GB	More memory than the 16 GB version with the same architecture

If you are deciding specifically between the V100 and the A2 for fine-tuning, I cover this in more detail in a separate article: V100 vs A2 for fine-tuning.

Considerations that matter for institutions, not only for private buyers

Buying for a university or a laboratory follows its own rules, which are easy to overlook when looking only at the card's specification.

VAT invoice. Obvious, but worth stating explicitly when buying on the secondary market: not every seller of used hardware issues a proper invoice, and this is often a requirement for settling a grant or for a purchase from a university budget.

Availability of identical units. If you are setting up a laboratory workstation or a teaching room, you usually need several identical units, not one. The secondary market can be unpredictable on this point: batches of the same model run out, so when planning a purchase for several workstations it is worth asking about the availability of more units up front, instead of buying them one at a time as the need arises.

Low cost of entry. This is the main argument of this whole article: the possibility of setting up several workstations instead of one for the same budget.

No vGPU licence needed for CUDA computing. If the task is pure computation (training, inference, simulations) run directly on the host or in a container, rather than GPU virtualisation across multiple virtual machines, you do not need an NVIDIA vGPU licence. This difference has a real effect on the total cost of deployment, and it is easy to forget when budgeting only for the hardware.

Practical points worth considering before you buy

These cards are passive designs, intended to work in server chassis with forced, directional airflow. In an ordinary desktop case without additional cooling (a shroud, additional fans), the card will overheat. If you are setting up a workstation outside a server room, add a suitable chassis or cooling adapter to the budget: this is not an optional extra, but a condition for the card to work (more in how to cool a passive server GPU in a desktop). Two more desktop details: the P40, P100 and V100 PCIe take a CPU 8-pin (EPS) power connector, not a PCIe one, so plan for the correct adapter and never force a PCIe plug in; and the motherboard must have Above 4G Decoding enabled in the BIOS, or the card may not initialise. The A2 needs neither an auxiliary power cable nor a large power supply.

Pascal (P100, P40) and Volta (V100) have a limited remaining software-support window. CUDA 13.0 removed offline compilation and library support for these architectures: CUDA 12.9 is the last toolkit that compiles for them, the R580 driver branch is the last one that supports them (maintained until mid-2028), and PyTorch 2.14 is the last release with a ready-made wheel for them. Both generations share the same timetable. Existing compatible CUDA 12.x stacks keep working, but newer framework releases may drop support for these architectures. If you are buying for many years rather than for a single project or semester, the safest choice in terms of the length of support is Ampere (A2).

When buying a used card for an institution, request a load-test report, for example from gpu-burn, stating the duration, detected computation errors and temperatures. A passing result documents that card's behaviour during that test. Also request available memory data: nvidia-smi -q -d ECC,PAGE_RETIREMENT,ROW_REMAPPER shows supported ECC counters, retired pages and remapped rows. Row remapping is relevant to the A2; unsupported fields may show N/A. Compare readings before and after testing, distinguishing corrected from uncorrectable errors.

I write more about what to look out for in general when buying a used data-centre accelerator here: A used data-centre accelerator - what to watch out for.