Whisper in Polish on a GPU - transcription and subtitles locally (GDPR)

Short answer: Whisper (and preferably faster-whisper) transcribes Polish very well, and locally on a GPU it does so quickly and without sending recordings to an external service, which with sensitive data can be a strong argument. In our test (a Polish sample of ~690 s, large-v3) the V100 32GB reached ~15× realtime, and the P40/A2 ~6×. For large-v3 you want cards with spare VRAM (V100 32GB, Tesla P40 24GB); for small/medium a Tesla P4 8GB or an NVIDIA A2 16GB is enough. And yes, we sell these cards; full disclaimer at the end.

I am writing this for everyone who regularly has recordings to transcribe (offices, law firms, local authorities, newsrooms, video creators) and does not want to or cannot upload them to an online service. Without beating about the bush: local Whisper is one of the most "cost-effective" uses of a used Tesla, because the task is continuous, repetitive and privacy-sensitive. Below: what to choose and how to get it running.

Whisper, faster-whisper, whisper.cpp: what is what

Whisper is an open speech-recognition model from OpenAI, very good in many languages, including Polish. "Bare" Whisper on its own is, however, fairly heavy. That is why in practice people use its optimised variants:

• faster-whisper: an implementation based on CTranslate2; in many configurations it significantly speeds up inference and reduces memory use compared with the reference implementation (the exact gain depends on the hardware, batching, version and compute type). It is the default choice on a GPU.
• whisper.cpp: a light implementation in C/C++, excellent on CPUs and weaker hardware, with quantisation. On a powerful GPU you will usually choose faster-whisper, but whisper.cpp can be convenient for simple, portable deployments.
• WhisperX: an add-on that provides precise timestamps and speaker separation (diarisation), useful for subtitles and "who said what" minutes. Diarisation uses pyannote: the first run requires access to the model on Hugging Face (gated) and a token; once it is downloaded, the pipeline can run locally.

For Polish and for a GPU: faster-whisper with the large-v3 model is the quality benchmark. You choose smaller models when speed matters to you or you have a weaker card.

Why locally: GDPR and more

Reason number one is again GDPR and confidentiality. Recordings can contain very sensitive information: personal data, special categories of data or content covered by professional secrecy:

• Recordings of meetings and conferences: names, figures, decisions and personal data come up. Sending this to an external service means passing the data to an external entity.
• Law firms and medicine: conversations covered by professional secrecy. Local processing simplifies the risk model here and limits passing recordings outside.
• Local authorities: transcripts of council and committee sessions, broadcasts. Public data, but huge in volume and cheap to process in-house.
• Video creators: subtitles for material where cost and processing recordings repeatedly in bulk matter.

Beware of mental shortcuts: the cloud is not by definition incompatible with GDPR or with professional secrecy. Compliance depends on, among other things, the legal basis, the controller/processor roles, the data processing agreement (DPA), transfers, security and retention. Local processing simply simplifies this risk model and limits the number of entities to which you entrust recordings.

On top of that comes cost: transcription services charge per minute of recording. With a large and regular volume (hours of recordings a week), your own hardware can lower the unit cost (you pay once for the card, not for every minute), but the break-even point depends on the price of the service, utilisation, energy and the cost of the host. And it works offline, without limits or queues.

Models and their appetite for VRAM

Whisper comes in several sizes. The larger it is, the better the quality (especially with difficult audio: background noise, accents, a poor microphone), but the more VRAM it needs and the slower it is.

Instead of general estimates, we give the measured peak VRAM usage from our test (faster-whisper, a Polish sample of ~690 s), in the configurations we actually ran:

Model	Precision	Measured peak VRAM	Card in the test	Polish quality (general, not measured)
large-v3	float16	4.4 GB	V100 32GB	Best
large-v3	int8	2.8 GB	V100 32GB	Best
large-v3	float16	4.1 GB	A2 16GB	Best
large-v3	int8	1.3 GB	A2 16GB	Best
large-v3	int8	3.4 GB	P40 24GB	Best
large-v3	float32	8.4 GB	P40 24GB	Best
medium	int8	2.0 GB	P4 8GB	Good
small	int8	0.8 GB	P4 8GB	Decent

VRAM usage depends on the compute type (float16/int8/float32), the batch size and the other components of the pipeline: the above are peaks for a single file without batching. One thing is clear, though: faster-whisper in INT8 runs large-v3 using much less memory than bare Whisper; if you are fighting for memory, that is the first step.

Which used Tesla for Whisper

V100 32GB: for large-v3, at volume and fast. Volta with Tensor Cores and fast HBM2 memory is a strong choice for the large models; we measured ~15.2× realtime on large-v3/FP16. 32 GB is plenty of spare capacity for batching (processing many recordings at once) and for WhisperX with diarisation: that is an argument about capacity, not a performance advantage we measured (we did not measure batch scaling in this round). If transcription is a regular part of your production workflow, this is the right card.

Tesla P40 24GB: large on a budget. 24 GB of VRAM at a relatively low purchase cost; it easily fits large-v3 (3.4 GB in INT8 in our test). It is Pascal (CC 6.1): CTranslate2 does not offer an optimised float16 path here, so you run INT8 or FP32, which is still enough to make the P40 a budget card for continuous work.

Tesla P4 8GB: for small/medium, cheap and frugal. 8 GB, 75 W, low-profile. It comfortably handles small/medium (medium int8 was 2.0 GB in our test). Large-v3 in a suitable quantisation mode may fit in 8 GB, but we do not have our own measurement of large-v3 on the P4 in this campaign, so we do not recommend it for that variant without a separate performance test. Ideal as a light, low-power "transcriber" or for edge deployments.

NVIDIA A2 16GB: newer Ampere, economical. Configurable 40–60 W, low-profile, a newer architecture. It comfortably handles small/medium, and large in INT8 also fits thanks to the 16 GB. Good when low power draw and low heat output in 24/7 operation matter to you.

Performance, and the most important sentence about NVENC

The key metric is speed relative to real time (× realtime): how many times faster than the length of the recording the transcription runs. A value of 5× means you transcribe an hour-long recording in 12 minutes. (A note on terminology: in the literature the abbreviation RTF is sometimes defined the other way round, processing time ÷ audio time, where lower = better. We give its inverse, i.e. the speed multiplier; we call it "× realtime" outright to avoid confusion.)

Our measurement (faster-whisper, a Polish sample of ~690 s, 16 kHz mono WAV; × realtime = audio length ÷ time of the transcription alone; full power limit):

Card	Model	Precision	× realtime	VRAM
V100 32GB	large-v3	float16	15.2×	4.4 GB
V100 32GB	large-v3	int8	14.0×	2.8 GB
A2 16GB	large-v3	float16	6.0×	4.1 GB
A2 16GB	large-v3	int8	5.9×	1.3 GB
P40 24GB	large-v3	int8	5.9×	3.4 GB
P40 24GB	large-v3	float32	4.7×	8.4 GB
P4 8GB	medium	int8	7.2×	2.0 GB
P4 8GB	small	int8	16.7×	0.8 GB

Two conclusions from this run: (1) on large-v3 the V100 was clearly faster: ~15.2× realtime against ~5.9× on the P40/A2. The result is consistent with Volta's advantage in FP16/Tensor Core computation, but the benchmark does not isolate the contribution of the individual elements (Tensor Cores, HBM2, CTranslate2 kernels, compute type). (2) The P40 (CC 6.1) has no optimised float16 path in CTranslate2: the documentation indicates an FP16→FP32 fallback, and optimised FP16 starts at CC 7.0 (Volta); that is why we used INT8 or FP32 on the P40. On the A2/V100 float16 works and is optimal. In this configuration the relationships between the cards were clear; with a different model, batch, CTranslate2 version and settings they may change.

And now something worth understanding so that you do not overpay or reject a good card for the wrong reason:

NVENC does not matter for transcription with Whisper itself. Whisper is a compute task, not video transcoding: it runs on the compute cores/Tensor Cores. NVENC is a hardware video encoder, useful for streaming and compressing films, not for speech recognition. This is worth knowing, because some buyers reject the V100 "because it has no NVENC", and that is a double misunderstanding: our V100 (Tesla V100-PCIE-32GB) does have the encoder, three NVENC engines in fact (NVIDIA's official matrix; it is compute cards such as the A100, A30 and H100 that have none), and it is not needed for transcription anyway. What makes the V100 good for Whisper is compute and memory: Tensor Cores and fast HBM2, not the video engine. (If your task is video transcoding, that is another story, but that is not Whisper.)

How we measured

So that the figures can be read honestly:

• Backend: faster-whisper (CTranslate2), the large-v3 model (and medium/small on the P4).
• Sample: one Polish file, 16 kHz mono WAV, ~690.4 s (LibriVox, public domain).
• Transcription parameters: beam_size=5, no VAD filter, otherwise default settings. On large-v3 (V100, A2, P40) the only variable between the cards was compute_type (float16/int8/float32); the P4 ran medium and small instead. All cards got the same file and the same parameters.
• Timing: one warm-up run, then one measured full run of the transcription (we did not average several repeats). The time does not include loading the model: we measured that separately (about 6.5–8.6 s on the P40).
• Hardware: a test bench, driver 580.173.02, full power limit (P40/V100 250 W, A2 configured at 60 W). The V100 variant is the Tesla V100-PCIE-32GB. On the P40 and V100, stay on the R580 driver branch: it is the last one that supports Pascal and Volta.
• What we did not record: the exact versions of the faster-whisper/CTranslate2 libraries for these particular runs, which is why we do not give them here rather than guess.

That is enough as "our test" on a single repeat; do not treat these figures as a fully averaged comparative benchmark across vendors.

How to get it running: faster-whisper

The minimal start really is short. Installation and transcription of one file:

pip install faster-whisper

from faster_whisper import WhisperModel

model = WhisperModel("large-v3", device="cuda", compute_type="float16")
segments, info = model.transcribe("nagranie.mp3", language="pl")

for seg in segments:
    print(f"[{seg.start:.2f} -> {seg.end:.2f}] {seg.text}")

A few tips from practice:

• compute_type="int8_float16" or "int8": saves a lot of VRAM. The effect on quality depends on the model and the material; in this campaign we measured performance and VRAM, not WER/CER. The first move when you are fighting for memory on large.
• language="pl": if you know the material is in Polish, set this parameter: you skip automatic language detection and eliminate the risk of it getting the language wrong.
• Subtitles: segments with start/end are easy to write to the .srt format. For precise timestamps and speaker separation, use WhisperX.
• Batch: with a large volume, process many files at once (faster-whisper has BatchedInferencePipeline); here the 32 GB on the V100 gives spare capacity for a larger batch (we did not measure its scaling).

Typical deployments and cost vs the cloud

So that this is not abstract, a few setups I actually see:

• Law firm / office: a dictaphone recording from a hearing or a meeting → faster-whisper medium/large on a P40 24GB → a completed written record in a dozen or so minutes, without sending the recording outside. With data covered by professional secrecy, local processing greatly simplifies the risk model.
• Local authority: broadcasts of council and committee sessions → batch large-v3 transcription on a V100 32GB at night, when the card would be idle anyway. Hours of recordings a week processed in bulk.
• Creator / newsroom: subtitles for video → medium on a P4 8GB or an A2 16GB, cheaply and at low power, producing .srt files in bulk.

The calculation is simple in outline: transcription services charge per minute of recording, so with a constant volume (several to a dozen or so hours of audio a week) the cost of the service grows linearly, while a used Tesla is a one-off expense. But the full calculation is TCO, not just the price of the card: the host, energy, administration time and the real utilisation of the hardware come on top. The break-even point depends on the price of the service, the volume, energy and the cost of the host: the more minutes and the more sensitive the recordings, the more often using your own hardware makes sense, but you need to calculate that for your own situation. With data covered by professional secrecy, a simplified risk model also enters the calculation, although, as above, the cloud itself is not ruled out legally.

Sources

• OpenAI Whisper: https://github.com/openai/whisper
• faster-whisper (SYSTRAN, CTranslate2): https://github.com/SYSTRAN/faster-whisper
• WhisperX (diarisation, precise timestamps): https://github.com/m-bain/whisperX
• NVIDIA Video Encode/Decode GPU Support Matrix (the P100 and V100 have NVENC; the A100, A30 and H100 have none): https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new

---

An honest disclaimer: we sell used data-centre cards, so I have an interest in this, but I have tried to present the matter honestly, including the fact that with a small, occasional volume a cloud service can be simpler and cheaper than setting up a server. This is used hardware: before shipping we put every unit through a multi-stage qualification that includes, among other things, a VRAM check, a burn-in with a watchdog, ECC, PCIe and computational correctness; we include a report/certificate from this process with the card. Not sure whether a P4, P40 or V100 will be better for your volume of recordings? write to us: we will ask about the type of audio and the scale, and advise you honestly.