doc: added README for how-to with the new contexts, removed SetGate public method
This commit is contained in:
parent
2305a6142a
commit
bf72a5b7e2
|
|
@ -7,8 +7,12 @@ This package provides Go bindings for whisper.cpp. They have been tested on:
|
|||
* Fedora Linux on x86_64
|
||||
|
||||
The "low level" bindings are in the `bindings/go` directory and there is a more
|
||||
Go-style package in the `bindings/go/pkg/whisper` directory. The most simple usage
|
||||
is as follows:
|
||||
Go-style package in the `bindings/go/pkg/whisper` directory.
|
||||
|
||||
Legacy stateless example (single worker). For the recommended stateful API and
|
||||
concurrency-safe usage, see "New high-level API" below. Note: `Model.NewContext()`
|
||||
returns a stateless context for backward compatibility and is not safe for parallel
|
||||
`Process` calls (may return `ErrStatelessBusy`).
|
||||
|
||||
```go
|
||||
import (
|
||||
|
|
@ -100,6 +104,147 @@ Getting help:
|
|||
|
||||
* Follow the discussion for the go bindings [here](https://github.com/ggml-org/whisper.cpp/discussions/312)
|
||||
|
||||
## New high-level API (stateful and stateless contexts)
|
||||
|
||||
The `pkg/whisper` package now exposes two context kinds:
|
||||
|
||||
- StatefulContext: recommended for concurrency. Each context owns its own whisper_state.
|
||||
- StatelessContext: shares the model context. Simpler, but not suitable for parallel `Process` calls.
|
||||
|
||||
### Quick start: stateful context (recommended)
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
whisper "github.com/ggerganov/whisper.cpp/bindings/go/pkg/whisper"
|
||||
)
|
||||
|
||||
func main() {
|
||||
// Load model
|
||||
model, err := whisper.NewModelContext("./models/ggml-small.en.bin")
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
defer model.Close()
|
||||
|
||||
// Configure parameters (optional: provide a config func)
|
||||
params, err := whisper.NewParameters(model, whisper.SAMPLING_GREEDY, func(p *whisper.Parameters) {
|
||||
p.SetThreads(4)
|
||||
p.SetLanguage("en") // or "auto"
|
||||
p.SetTranslate(false)
|
||||
})
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// Create stateful context (safe for running in parallel goroutines)
|
||||
ctx, err := whisper.NewStatefulContext(model, params)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
defer ctx.Close()
|
||||
|
||||
// Your 16-bit mono PCM at 16kHz as float32 samples
|
||||
var samples []float32
|
||||
|
||||
// Process. Callbacks are optional.
|
||||
if err := ctx.Process(samples, nil, nil, nil); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// Read segments
|
||||
for {
|
||||
seg, err := ctx.NextSegment()
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
fmt.Printf("[%v -> %v] %s\n", seg.Start, seg.End, seg.Text)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Quick start: stateless context (single worker)
|
||||
|
||||
```go
|
||||
// Load model as above
|
||||
model, _ := whisper.NewModelContext("./models/ggml-small.en.bin")
|
||||
defer model.Close()
|
||||
|
||||
params, _ := whisper.NewParameters(model, whisper.SAMPLING_GREEDY, nil)
|
||||
ctx, _ := whisper.NewStatelessContext(model, params)
|
||||
defer ctx.Close()
|
||||
|
||||
if err := ctx.Process(samples, nil, nil, nil); err != nil { panic(err) }
|
||||
for {
|
||||
seg, err := ctx.NextSegment()
|
||||
if err != nil { break }
|
||||
fmt.Println(seg.Text)
|
||||
}
|
||||
```
|
||||
|
||||
### Deprecations and migration notes
|
||||
|
||||
- The `Context` interface setters are deprecated (SetThreads, SetLanguage, etc.). Use `Parameters` via `NewParameters` and pass it when creating a context.
|
||||
- `Model.NewContext()` remains for backward compatibility and returns a stateless context by default. Prefer `NewStatefulContext` for concurrency.
|
||||
- Stateless contexts share the model context. A concurrency gate prevents overlapping `Process` calls and will return `ErrStatelessBusy` if another `Process` is in flight.
|
||||
- For parallel processing, create one `StatefulContext` per goroutine.
|
||||
|
||||
## Benchmarks
|
||||
|
||||
Benchmarks live in `pkg/whisper` and compare CPU vs GPU, stateful vs stateless, threads, and callback modes.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Model: `models/ggml-small.en.bin` (or your choice).
|
||||
- Sample: `samples/jfk.wav`.
|
||||
- Build the C libs once (also downloads a model for examples):
|
||||
|
||||
```bash
|
||||
cd bindings/go
|
||||
make examples
|
||||
# optionally: ./build/go-model-download -out models
|
||||
```
|
||||
|
||||
### Run CPU benchmarks
|
||||
|
||||
```bash
|
||||
cd bindings/go/pkg/whisper
|
||||
go test -bench=ContextProcessCPU -benchmem -run=^$ .
|
||||
go test -bench=ContextProcessBigCPU -benchmem -run=^$ .
|
||||
# or run all
|
||||
go test -bench=. -benchmem -run=^$ .
|
||||
```
|
||||
|
||||
### Run GPU benchmarks
|
||||
|
||||
GPU runs toggle `UseGPU` in model params. Ensure your build has GPU backends enabled:
|
||||
|
||||
- CUDA: build the C libs with CUDA
|
||||
|
||||
```bash
|
||||
cd bindings/go
|
||||
GGML_CUDA=1 make whisper
|
||||
```
|
||||
|
||||
- macOS Metal: supported by default in the build flags on Darwin.
|
||||
|
||||
Then run:
|
||||
|
||||
```bash
|
||||
cd bindings/go/pkg/whisper
|
||||
go test -bench=ContextProcessGPU -benchmem -run=^$ .
|
||||
go test -bench=ContextProcessBigGPU -benchmem -run=^$ .
|
||||
```
|
||||
|
||||
### What the benchmarks measure
|
||||
|
||||
- Variants: device (cpu/gpu) x context kind (stateless/stateful) x threads {1,2,4, NumCPU} x callback mode (NoCallback, WithSegmentCallback).
|
||||
- Standard Go benchmark outputs: ns/op, B/op, allocs/op. We also set bytes per op to sample bytes.
|
||||
- Custom metric `ms_process`: wall time per `Process` iteration, reported via `b.ReportMetric`.
|
||||
- When `printTimings` is enabled, model-level timings are printed for NoCallback runs using `model.PrintTimings()`.
|
||||
|
||||
## License
|
||||
|
||||
The license for the Go bindings is the same as the license for the rest of the whisper.cpp project, which is the MIT License. See the `LICENSE` file for more details.
|
||||
|
|
|
|||
Loading…
Reference in New Issue