How we check every used GPU accelerator before it reaches you - 6 qualification stages and what is on the certificate
- 0
- 0
Short answer: when you buy a used data-centre card from an anonymous seller, you are buying a pig in a poke: you do not know whether the unit is sound or worn out from use in a cryptocurrency mining farm, with failing memory, and you have no way to check after the fact. We put every unit through a six-stage qualification, from a test of the entire available VRAM space, through a burn-in with a thermal watchdog, to a check that its calculations are correct, and we issue a certificate assigned to that unit, with the card's technical fingerprint and the result of every stage. It is a document for this unit, not for the model: you can compare it with the card you are holding. It does not turn used hardware into new hardware with a manufacturer's warranty (there is none, and we say so plainly), but it turns "trust me" into "check for yourself".
Let me start by being straight about it: we sell used data-centre cards from the secondary market. They carry no manufacturer's warranty; the seller's warranty applies, on the terms given at the time of sale. In this world, trust does not come from the brand on the box, because there is no box. It comes from showing exactly what we checked and that it works. This post shows how we do it, so that you know what you are paying for and can verify it yourself.
What you are really afraid of with a used accelerator
The secondary market for data-centre cards has three real traps, and it is worth naming them before I say how we address them:
Ex-mining cards. Hardware that ran for months under full load with poor cooling. The mere fact that "it was used for cryptocurrency mining" does not mean it will fail, but heavy use and thermal cycling increase the risk of problems with solder joints, memory and the power delivery, which you want to catch before you pay.
Failing memory. Individual VRAM cells can throw errors under load even though the card "boots" and passes basic tests. On a compute card this is a serious problem: a memory error can mean a wrong result.
A swapped or misrepresented unit. A "V100 32 GB" at auction that is really 16 GB, a different BIOS, a re-applied label. Without reading the card in software, you have no way to check what you actually received.
An anonymous auction answers none of these questions. Our process is built precisely to answer each of them with something concrete that stays with you on paper.
Six qualification stages: what we check and why
Every unit passes an entry gate (the test conditions) and the same set of six tests, in fail-fast order: the first stage that fails ends the test of that card (there is no point heating up a unit that has already been rejected). The only thing that differs is the time variant of the test (baseline or more thorough, more on that below), and the certificate shows clearly which one was run. Every stage has an explicit pass criterion and a saved artefact: no "passed" without a file.
Entry gate: test conditions. Before we measure anything, we enforce two things: the card's full power limit and ECC enabled (memory error correction). Without them the result would be invalid: a lower limit caps the maximum power draw and changes the thermal profile of the test, and with ECC disabled there is nothing to read. Only when both conditions are met do the actual tests start. We record the state of both as evidence of the test conditions. Then the six tests:
1. Technical fingerprint. We read the unit's permanent identifiers: model, serial number, GPU UUID, VBIOS, PCI ID, full board part number, memory size, and also the card's thermal thresholds (slowdown and shutdown temperature, read with nvidia-smi), and we check that they match the declared model. This is where we catch "16 instead of 32 GB" or a swapped BIOS; the thermal thresholds also calibrate the burn-in watchdog. A model mismatch = stop, the unit is set aside for investigation.
2. Memory: a test of the entire available VRAM space. This is the most important difference from "the card booted". We run cuda_memtest on the largest block of memory that can be allocated (on a 16-gigabyte P100 that is about 16,000 MB, practically all the memory available to the process apart from the driver's overhead), writing and verifying patterns in it (walking-bit, own-address, 8- and 32-bit and random moving inversions, block move, random sequences). This is what actually validates the memory: ECC monitoring (below) is an important signal, but it does not replace a systematic pass over every address. The criterion is merciless: any error = rejection.
3. Burn-in under full load, with a thermal watchdog. gpu-burn loads the card with matrix work at the full power limit, and telemetry records temperature, power draw and throttling signals every second. A watchdog with our test's conservative thresholds oversees this: at 75 °C it warns, at 80 °C it aborts the test and marks the card as overheated (we checked this physically: cutting the cooling really does trigger the abort). These are the test's safety thresholds, deliberately lower than the card's factory slowdown/shutdown thresholds, which we read separately in the fingerprint. Why full load? Because problems most often show up right at the limit, where your real workload will push the card. We will not catch every future failure this way, but full load catches far more than a cursory start-up.
4. ECC: a real reading of the counters, per architecture. Some honesty matters here: the mechanism differs between cards. Pascal, Volta and Turing (P4/P40/P100/V100/T4) use Retired Pages (retiring memory pages); Ampere (A2) uses the Row Remapper (remapping rows). We read the counters appropriate to the architecture after the burn-in. The qualification criterion is precise: no uncorrectable errors and no new retired pages / failed remap during the test (a single correctable error is not the same as damage; we distinguish between these counters). Rising ECC counts are one of the earliest signs that something is happening to the memory.
5. PCIe interface. We check the link speed under load, bus errors (AER) and one specific link replay counter (pcie.replay_counter from nvidia-smi): not a general "retransmission" derived indirectly, just this one unambiguous value. One honest note, so as not to mislead: the A2 natively runs on PCIe x8: that is its correct specification, not a degradation, and that is how we assess it (we compare against the model's native width, not against "x16 from every card").
6. Correctness of calculations (known-answer). gpu-burn compares the results of repeated matrix multiplications with the first result and counts differences exceeding a set tolerance, so it detects inconsistent calculations, but it does not compare them with a known correct answer and exercises only one kind of computation. So we run YOLOv8 detection on a reference image and compare the set of detected objects from the FP32 and FP16 runs: they must match, and they must be the expected objects (the point is not bit-identical results, but matching detections). This is an additional correctness test of the compute path for this workload and this input. To be honest about the scope: it checks the FP16 path; only Volta, Turing and Ampere (V100, T4, A2) have Tensor Cores; Pascal (P4/P40/P100) does not, so we talk about Tensor Cores only for the former. And even more clearly: this is not a formal validation of all of the card's compute units; it is an end-to-end test that catches some errors invisible in a stability test alone, not proof that every operation is correct.
What does this look like in a real result? The figures below come from a real certificate of a qualified P100 unit (serial number and UUID masked). The certificate itself is issued in Polish; this is an English rendering of its fields:
GPU QUALIFICATION CERTIFICATE - Agile Store
Status: PASSED (PASS)
Date: 2026-08-27 13:17
Model: Tesla P100-PCIE-16GB
Serial number: 03247…7810
GPU UUID: GPU-1411d3ac-****-****-****-********ea4
VBIOS: 86.00.41.00.06
Board Part Number: 900-2H400-0000-000
Memory: 16384 MiB
Thermal thresholds: slowdown 82°C / shutdown 85°C (watchdog WARN 75 / ABORT 80)
Conditions: PASS power 250/250 W (full); ECC=Enabled
Fingerprint: PASS matches the declared model
VRAM: PASS cuda_memtest 1.2.3; 16006 MB; 0 errors; 1 pass
Burn-in: PASS gpu-burn 636 s; maxT 61°C; avgP 202.4 W; throttle 0
ECC: PASS Retired Pages; uncorrectable 0/0; double-bit 0
PCIe/AER: PASS gen3 x16 (native); replay 0; 0 AER entries
Inference: PASS YOLO FP16 == FP32; classes ["bus","person"×4]
These are the figures of this particular unit, from its evidence folder, not an averaged profile of the model. For every card we keep a full set of raw logs (including the complete cuda_memtest run and per-second burn-in telemetry) and the scripts to repeat the test.
A per-unit certificate: what it really carries
For every unit we generate a separate document assigned to that unit: one A4 page per card, with a Code128 barcode and a verification QR code. What is on it now:
- Technical fingerprint: model, serial number, GPU UUID, VBIOS, full board part number (Board PN), memory size, PCI ID, plus the card's thermal thresholds (slowdown/shutdown).
- Qualification record: the result of every stage run (conditions, VRAM/memtest, burn-in with maximum temperature and power draw, ECC, PCIe, inference), with the date and time of the test.
- Warranty terms: every point, stated plainly, with no small print.
Why does this matter? Because the GPU UUID and the serial number are unique to the unit. You can read them from the card with one command and compare them with the paper:
nvidia-smi --query-gpu=name,uuid,serial,vbios_version,memory.total,ecc.mode.current,pci.device_id --format=csv
Verify the card yourself in 30 seconds:
- Read the GPU UUID from the command's output and compare it with the certificate.
- The same for the serial number.
- The same for the VBIOS.
- Check the memory size (
memory.totalin the output): it must match the certificate. On cards with GDDR memory (P4, P40, T4, A2), ECC reserves 6.25% of it, so a P40 with ECC enabled shows about 22,900 MiB rather than 24,576 MiB: compare with ECC in the same state as on the certificate (enabled). - If anything does not match, this is not the unit we tested. Write to us.
If the UUID, serial number and VBIOS match, you confirm that the identifiers correspond to the card we tested, not to "an identical one" swapped along the way. The memory size on the certificate settles the "16 or 32 GB" question. The board number is a factory marking that a sticker alone will not change. And for every unit we keep a full set of raw logs and the scripts to repeat the test: nothing is taken on trust.
What this process does not do, because honesty is part of the certificate too
I do not want you to leave this post with the impression that we sell new hardware with five years of support. We do not. So, plainly, what is outside the scope:
- There is no manufacturer's warranty. This is the secondary market: the seller's warranty applies, on the terms given at the time of sale. Qualification is not the same as a manufacturer's warranty; if anyone promises you an "NVIDIA warranty" on a used accelerator from the secondary market, ask for written confirmation and the warranty terms applicable to that model, region and sales channel, including confirmation that coverage extends to the next buyer.
- We do not do full thermal cycling. A burn-in catches thermal instability under load, but it does not simulate many heating/cooling cycles, which can reveal mechanical problems such as micro-cracks in joints. That is a separate, long regime: we do not pretend to use it.
- Baseline vs thorough level. As standard we qualify in the baseline variant (one memtest pass, 10 minutes of burn-in). The more thorough variant (three memory passes, 30 minutes of burn-in) we run on request: we do not pretend that every unit goes through the maximum variant.
- Qualification is a snapshot. It certifies the card's condition at the moment of the test. Used hardware can fail later: that is why there is a warranty, not because the test was weak.
This is not "less certainty". It is the right certainty: you know exactly what was checked, in which variant, and what the test does not cover, instead of buying a promise with nothing behind it.
What you get out of it: three concrete things
- Verifiability. You read the UUID/serial/VBIOS from the card and compare them with the certificate. A match = this is the unit, tested, not swapped.
- A reference point. You have on paper the result of every stage, and the temperature and power draw from the moment of the certification test. If the card starts to behave differently, you know what to compare against.
- Repeatability. For every unit we keep a full set of logs and scripts: the qualification can be repeated, not just read about as "passed".
One final thought
A used data-centre card without documentation is a lottery, and the price often reflects that, because you pay less for more risk. We do not hide that risk; we address it: we put every unit through six stages and give you the fingerprint of that specific card together with the result of each of them. It does not turn used into new. It turns "believe the seller" into "verify for yourself", and that is the only currency of trust that makes sense on the secondary market.
Sources
Agile Store's own process, cert-pipeline: a six-stage per-unit qualification, deployed and validated in our quality-control process. We keep the raw logs, results and scripts to repeat the test per serial number.
Tools and documentation the process relies on: cuda_memtest, gpu-burn, page retirement on Pascal, Volta and Turing cards, row remapping on Ampere and newer cards, nvidia-smi documentation, NVIDIA warranty.
