public_git/whisper.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/whisper.cpp.git synced 2026-10-04 13:31:18 +02:00
Code Issues Packages Projects Releases Wiki Activity

tests : add script to benchmark whisper.cpp on LibriSpeech corpus (#2999)

Browse Source 
* tests : add script to benchmark whisper.cpp on LibriSpeech corpus

LibriSpeech is a widely-used benchmark dataset for training and
testing speech recognition models.

This adds a set of scripts to measure the recognition accuracy of
whisper.cpp models, following the common benchmark standards.

Signed-off-by: Fujimoto Seiji <fujimoto@ceptord.net>

* Document how to prepare `whisper-cli` and model files

Feedback from Daniel Bevenius.

This adds a short code example how to prepare the `whisper-cli`
command, to make the initial setup step a little bit clearer.

Signed-off-by: Fujimoto Seiji <fujimoto@ceptord.net>

* tests : Simplify how to set up Python environment

Based on a feedback from Georgi Gerganov.

Instead of setting up a virtual environment in Makefile, let users
set up the Python environment. This is better since users may have
their own preferred workflow/toolkit.

Signed-off-by: Fujimoto Seiji <fujimoto@ceptord.net>

---------

Signed-off-by: Fujimoto Seiji <fujimoto@ceptord.net>
This commit is contained in:
Fujimoto Seiji
2025-04-04 19:51:26 +03:00
committed by GitHub
parent e6234cd435
commit 448f3d3b93
11 changed files with 2571 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/librispeech/eval.mk
+39
View File
@@ -0,0 +1,39 @@
PYTHON = python
WHISPER_PREFIX = ../../
WHISPER_MODEL = tiny
WHISPER_CLI = $(WHISPER_PREFIX)build/bin/whisper-cli
WHISPER_FLAGS = --no-prints --language en --output-txt
# You can create eval.conf to override the WHISPER_* variables
# defined above.
-include eval.conf
# This follows the file structure of the LibriSpeech project.
AUDIO_SRCS = $(sort $(wildcard LibriSpeech/*/*/*/*.flac))
TRANS_TXTS = $(addsuffix .txt, $(AUDIO_SRCS))
# We output the evaluation result to this file.
DONE = $(WHISPER_MODEL).txt
all: $(DONE)
$(DONE): $(TRANS_TXTS)
$(PYTHON) eval.py > $@.tmp
mv $@.tmp $@
# Note: This task writes to a temporary file first to
# create the target file atomically.
%.flac.txt: %.flac
$(WHISPER_CLI) $(WHISPER_FLAGS) --model $(WHISPER_PREFIX)models/ggml-$(WHISPER_MODEL).bin --file $^ --output-file $^.tmp
mv $^.tmp.txt $^.txt
archive:
tar -czf $(WHISPER_MODEL).tar.gz --exclude="*.flac" LibriSpeech $(DONE)
clean:
@rm -f $(TRANS_TXTS)
@rm -f $(DONE)
.PHONY: all clean