With flash attention enabled, the encoder self-attention and the decoder
cross-attention view their K/V buffers padded to GGML_PAD(n_ctx, 256)
but only write n_ctx rows, and call ggml_flash_attn_ext with a null
mask. The unwritten pad rows are attended at full weight; their content
is whatever the buffer holds - zeros on a fresh state, or leftovers of
previous whisper_full calls with a different audio_ctx (layer strides
il*n_ctx_pad shift between window sizes, so runs overwrite each other's
pad regions).
On a long-running server this makes transcriptions depend on request
history: e.g. large-v3-turbo with audio_ctx=250 on a 3 s clip produced
5x sentence duplication on a fresh state, 2x after one audio_ctx=500
request, and clean output after full-window requests. The default full
window is affected too (36 unmasked rows out of 1536), just diluted
enough to go unnoticed.
Fix: give both call sites a proper mask input (mirroring the decoder
self-attention KQ_mask) with -INFINITY over [n_ctx, n_ctx_pad), filled
in whisper_encode_internal / whisper_decode_internal. Outputs at reduced
audio_ctx are now deterministic regardless of prior requests; timing is
unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
6ce8e7028c