lintoai/linto-stt-whisper repository overview

⁠LinTO-STT-Whisper

ASR API based on Whisper models⁠. Supports offline and real-time transcription.

⁠Quick Start

⁠Prerequisites

• Docker⁠
• nvidia-container-toolkit⁠ for GPU
• At least 8GB disk space for the image, plus up to 5GB per model
• Up to 7GB RAM depending on model size

⁠VRAM Usage

Approximate GPU VRAM peak usage by model size and backend:

⁠Pull or Build

# CPU
docker pull lintoai/linto-stt-whisper

# GPU (includes CUDA runtime: cuBLAS + cuDNN for ctranslate2)
docker pull lintoai/linto-stt-whisper-gpu

or

# CPU
docker build -t linto-stt-whisper:latest --build-arg STT_ENGINE=whisper .

# GPU (~1.3GB larger, adds cuBLAS + cuDNN)
docker build -t linto-stt-whisper-gpu:latest --build-arg STT_ENGINE=whisper --build-arg GPU=1 .

⁠Run

HTTP file transcription:

docker run -p 8080:80 --rm \
  -e SERVICE_MODE=http \
  -e MODEL=large-v3 \
  --env-file .env \
  lintoai/linto-stt-whisper

WebSocket streaming:

docker run -p 8080:80 --rm \
  -e SERVICE_MODE=websocket \
  -e MODEL=large-v3-turbo \
  --env-file .env \
  lintoai/linto-stt-whisper

For GPU, use the GPU image with --gpus all -e DEVICE=cuda:

docker run -p 8080:80 --rm \
  -e SERVICE_MODE=http \
  -e MODEL=large-v3 \
  -e DEVICE=cuda \
  --gpus all \
  --env-file .env \
  lintoai/linto-stt-whisper-gpu

Mount cache with -v ~/.cache:/root/.cache to avoid re-downloads.

⁠Whisper Models

The model is downloaded on first transcription and cached. You can specify a model by size name, HuggingFace identifier, or local path.

⁠OpenAI Whisper Models

Multi-lingual: tiny⁠ | base⁠ | small⁠ | medium⁠ | large-v1⁠ | large-v2⁠ | large-v3⁠ | large-v3-turbo⁠

English-only: tiny.en⁠ | base.en⁠ | small.en⁠ | medium.en⁠

HuggingFace models like distil-whisper/distil-large-v2⁠ are also supported (use the identifier directly or download locally).

If you used Whisper locally before, models are in ~/.cache/whisper.

⁠Alignment Model (deprecated)

The ALIGNMENT_MODEL variable for wav2vec word alignment is deprecated and no longer tested. We advise not to use it.

⁠Whisper-Specific Configuration

⁠LANGUAGE

The LANGUAGE variable sets the default recognition language (can be overridden per request).

Values:

• * — automatic language detection
• Language code: fr, en, yue, ...
• BCP-47 code: fr-FR, en-US, yue-HK, ...
• Language name: French, English, Cantonese, ...

Supported languages: af(afrikaans), am(amharic), ar(arabic), as(assamese), az(azerbaijani), ba(bashkir), be(belarusian), bg(bulgarian), bn(bengali), bo(tibetan), br(breton), bs(bosnian), ca(catalan), cs(czech), cy(welsh), da(danish), de(german), el(greek), en(english), es(spanish), et(estonian), eu(basque), fa(persian), fi(finnish), fo(faroese), fr(french), gl(galician), gu(gujarati), ha(hausa), haw(hawaiian), he(hebrew), hi(hindi), hr(croatian), ht(haitian creole), hu(hungarian), hy(armenian), id(indonesian), is(icelandic), it(italian), ja(japanese), jw(javanese), ka(georgian), kk(kazakh), km(khmer), kn(kannada), ko(korean), la(latin), lb(luxembourgish), ln(lingala), lo(lao), lt(lithuanian), lv(latvian), mg(malagasy), mi(maori), mk(macedonian), ml(malayalam), mn(mongolian), mr(marathi), ms(malay), mt(maltese), my(myanmar), ne(nepali), nl(dutch), nn(nynorsk), no(norwegian), oc(occitan), pa(punjabi), pl(polish), ps(pashto), pt(portuguese), ro(romanian), ru(russian), sa(sanskrit), sd(sindhi), si(sinhala), sk(slovak), sl(slovenian), sn(shona), so(somali), sq(albanian), sr(serbian), su(sundanese), sv(swedish), sw(swahili), ta(tamil), te(telugu), tg(tajik), th(thai), tk(turkmen), tl(tagalog), tr(turkish), tt(tatar), uk(ukrainian), ur(urdu), uz(uzbek), vi(vietnamese), yi(yiddish), yo(yoruba), zh(chinese).

Model large-v3 and derivatives also support yue(cantonese).

⁠Streaming Tuning

We recommend running streaming on GPU with large-v3-turbo or smaller models (avoid large-v3 as it is very expensive). Using a VAD on the server side (e.g. silero) is recommended.

How to choose STREAMING_MIN_CHUNK_SIZE and STREAMING_BUFFER_TRIMMING_SEC:

• Low latency (2-5s on a RTX 4090): Set STREAMING_MIN_CHUNK_SIZE=0.5 and STREAMING_BUFFER_TRIMMING_SEC around 10 (compromise between latency and accuracy). Range: 6 to 15 depending on hardware/model.
• High latency (~30s, minimize GPU activity): Set STREAMING_MIN_CHUNK_SIZE=26. STREAMING_BUFFER_TRIMMING_SEC must be lower (6-12). Lower values reduce GPU usage but may degrade accuracy.

⁠License

AGPLv3 (see LICENSE).

⁠Acknowledgments

• Ctranslate2⁠ / Faster-Whisper⁠
• OpenAI Whisper⁠ / Whisper-Timestamped⁠
• HuggingFace Transformers⁠
• Whisper_Streaming⁠