mirror of
https://github.com/ggml-org/whisper.cpp.git
synced 2026-10-20 00:22:49 +02:00
ruby : add VAD::Context#segments_from_samples, allow Pathname, etc. (#3633)
* ruby : Bump version to 1.3.6 * Fix code in example * Add sample code to transcribe from MemoryView * Define GetVADContext macro * Use GetVADContext * Extract parse_full_args function * Use parse_full_args in ruby_whisper_full_parallel * Free samples after use * Check return value of parse_full_args() * Define GetVADParams macro * Add VAD::Context#segments_from_samples * Add tests for VAD::Context#segments_from_samples * Add signature for VAD::Context#segments_from_samples * Add sample code for VAD::Context#segments_from_samples * Add test for Whisper::Context#transcribe with Pathname * Make Whisper::Context#transcribe and Whisper::VAD::Context#detect accept Pathname * Update signature of Whisper::Context#transcribe * Fix variable name * Don't free memory view * Make parse_full_args return struct * Fallback when failed to get MemoryView * Add num of samples when too long * Check members of MemoryView * Fix a typo * Remove unnecessary include * Fix a typo * Fix a typo * Care the case of MemoryView doesn't fit spec * Add TODO comment * Add optimazation option to compiler flags * Use ALLOC_N instead of malloc * Add description to sample code * Rename and change args: parse_full_args -> parse_samples * Free samples when exception raised * Assign type check result to a variable * Define wrapper function of whisper_full * Change signature of parse_samples for rb_ensure * Ensure release MemoryView * Extract fill_samples function * Free samples memory when filling it failed * Free samples memory when transcription failed * Prepare transcription in wrapper funciton * Change function name * Simplify function boundary
This commit is contained in:
+33
-2
@@ -323,7 +323,24 @@ whisper
|
||||
end
|
||||
```
|
||||
|
||||
The second argument `samples` may be an array, an object with `length` and `each` method, or a MemoryView. If you can prepare audio data as C array and export it as a MemoryView, whispercpp accepts and works with it with zero copy.
|
||||
The second argument `samples` may be an array, an object with `length` and `each` method, or a MemoryView.
|
||||
|
||||
If you can prepare audio data as C array and export it as a MemoryView, whispercpp accepts and works with it with zero copy.
|
||||
|
||||
```ruby
|
||||
require "torchaudio"
|
||||
require "arrow-numo-narray"
|
||||
require "whisper"
|
||||
|
||||
waveform, sample_rate = TorchAudio.load("test/fixtures/jfk.wav")
|
||||
# Convert Torch::Tensor to Arrow::Array via Numo::NArray
|
||||
samples = waveform.squeeze.numo.to_arrow.to_arrow_array
|
||||
|
||||
whisper = Whisper::Context.new("base")
|
||||
whisper
|
||||
# Arrow::Array exports MemoryView
|
||||
.full(Whisper::Params.new, samples)
|
||||
```
|
||||
|
||||
Using VAD separately from ASR
|
||||
-----------------------------
|
||||
@@ -334,13 +351,27 @@ VAD feature itself is useful. You can use it separately from ASR:
|
||||
vad = Whisper::VAD::Context.new("silero-v6.2.0")
|
||||
vad
|
||||
.detect("path/to/audio.wav", Whisper::VAD::Params.new)
|
||||
.each_with_index do |segment, index|
|
||||
.each.with_index do |segment, index|
|
||||
segment => {start_time: st, end_time: ed} # `Segment` responds to `#deconstruct_keys`
|
||||
|
||||
puts "[%{nth}: %{st} --> %{ed}]" % {nth: index + 1, st:, ed:}
|
||||
end
|
||||
```
|
||||
|
||||
You may also low level API `Whisper::VAD::Context#segments_from_samples` as such `Whisper::Context#full`:
|
||||
|
||||
```ruby
|
||||
# Ruby Array
|
||||
reader = WaveFile::Reader.new("path/to/audio.wav", WaveFile::Format.new(:mono, :float, 16000))
|
||||
samples = reader.enum_for(:each_buffer).map(&:samples).flatten
|
||||
|
||||
# Or, object which exports MemoryView
|
||||
waveform, sample_rate = TorchAudio.load("test/fixtures/jfk.wav")
|
||||
samples = waveform.squeeze.numo.to_arrow.to_arrow_array
|
||||
|
||||
segments = vad.segments_from_samples(Whisper::VAD::Params.new, samples)
|
||||
```
|
||||
|
||||
Development
|
||||
-----------
|
||||
|
||||
|
||||
Reference in New Issue
Block a user