whisper : map token timestamps to original time when VAD is enabled (#3910)

whisper_full_get_token_data().t0/t1 are in the VAD-processed timeline (silences
removed), so they don't line up with the original audio. Add
whisper_full_get_token_t0/t1 that map them back.

A token inside a speech segment is interpolated within it; a token that falls in a
removed inter-segment silence snaps to the nearest boundary, so it never lands in
the middle of a cut-out gap. Without VAD the raw times are returned unchanged.
This commit is contained in:
Lin Xiaodong
2026-07-01 08:23:50 +02:00
committed by GitHub
parent 909307c8bd
commit 5cad7abb1e
3 changed files with 86 additions and 0 deletions
+12
View File
@@ -50,6 +50,18 @@ int main() {
assert(whisper_full_get_segment_t0(wctx, 0) == 32);
assert(whisper_full_get_segment_t1(wctx, 0) == 1051);
// token times mapped back to the original timeline: ordered, non-negative duration
const int n_tokens = whisper_full_n_tokens(wctx, 0);
assert(n_tokens > 0);
int64_t prev_t0 = -1;
for (int j = 0; j < n_tokens; ++j) {
const int64_t t0 = whisper_full_get_token_t0(wctx, 0, j);
const int64_t t1 = whisper_full_get_token_t1(wctx, 0, j);
assert(t0 >= 0 && t1 >= t0);
assert(t0 >= prev_t0);
prev_t0 = t0;
}
whisper_free(wctx);
return 0;