Transcription of audio files with faster-whisper

faster-whisper is a reimplementation of OpenAI's Whisper model using CTranslate2, which is a fast inference engine for Transformer models. ( https://github.com/SYSTRAN/faster-whisper )

There are several GUI available, targeted mostly at desktop computers. However the model runs excellent on a HPC linux environment.

In the example below faster-whisper is installed in a virtual python environment on ALICE and used to transcribe an audiofile in mp3 to tekst.

Install the virtual environment

Load a set of CUDA and a Python version modules:

module load cuDNN/9.1.1.17-CUDA-12.4.0 Python/3.12.3-GCCcore-13.3.0

Remember to load these modules each time before use.

mkdir myproject
cd myproject
python -m venv venv
source venv/bin/activate

Add the python modules

pip install wheel requests
pip install faster-whisper

If the installation goes well, this is sufficient to start transcription of files. See the site for examples.

Example

For our example script we use mutagen to read some information from mp3

pip install mutagen

As a test we use a podcast from https://www.cgdev.org/file/conversation-christine-lagarde-podcastmp3

wget https://traffic.libsyn.com/cgdev/Conversation-Christine-Lagarde-podcast.mp3

Create a python script and name it mp3totekst.py:

#!/usr/bin/env python3
import sys
from faster_whisper import WhisperModel
from mutagen.mp3 import MP3

def main():
    if len(sys.argv) != 2:
        print("Usage: mp3totekst.py <filename.mp3>")
        sys.exit(1)

    filename = sys.argv[1]

    try:
        audio = MP3(filename)
        print(f"Filename: {filename}")
        print(f"Length: {audio.info.length:.2f} seconds")
        print(f"Bitrate: {audio.info.bitrate} bps")

    except Exception as e:
        print(f"Error reading MP3 file: {e}")

    # link to source
    model_size = "large-v3"

    # Run on GPU with FP16
    model = WhisperModel(model_size, device="cuda", compute_type="float16")

    # or run on GPU with INT8
    # model = WhisperModel(model_size, device="cuda", compute_type="int8_float16")
    # or run on CPU with INT8
    # model = WhisperModel(model_size, device="cpu", compute_type="int8")

    segments, info = model.transcribe(filename, beam_size=5)

    print("Detected language '%s' with probability %f" % (info.language, info.language_probability))

    for segment in segments:
        print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

if __name__ == "__main__":
    main()

Usage

Ensure modules are loaded, your venv is active and your python script executable.

Then:

venv) $./mp3totext.py Conversation-Christine-Lagarde-podcast.mp3
Filename: Conversation-Christine-Lagarde-podcast.mp3
Length: 2018.29 seconds
Bitrate: 320000 bps
Detected language 'en' with probability 0.999512
[0.00s -> 9.52s]  Hello, I'm Rajesh Merchandani and thanks for joining me for this edition of the CGD
[9.52s -> 14.90s]  podcast.
(...)

Benchmark

Transcribing the 30 min mp3 file with FP16 on a L4 GPU took about 2 minutes.

On a AMD zen4 cpu with INT8 the same file takes 19 minutes. But it could still be worth to utilize cpu’s for such workloads as cpu’s are cost-efficient.

GPU/CPU	Precision	runtime

GPU/CPU	Precision	runtime
gpu (l4)	FP16	2 min
gpu ( rtx2080ti )	INT8	6.55 min
cpu (zen4)	INT8	19 min

Example batch file

Todo slurm script