doc: added README for how-to with the new contexts, removed SetGate public method

This commit is contained in:
ciricc 2025-09-20 16:15:28 +03:00
parent 2305a6142a
commit bf72a5b7e2
1 changed files with 147 additions and 2 deletions

View File

@ -7,8 +7,12 @@ This package provides Go bindings for whisper.cpp. They have been tested on:
* Fedora Linux on x86_64
The "low level" bindings are in the `bindings/go` directory and there is a more
Go-style package in the `bindings/go/pkg/whisper` directory. The most simple usage
is as follows:
Go-style package in the `bindings/go/pkg/whisper` directory.
Legacy stateless example (single worker). For the recommended stateful API and
concurrency-safe usage, see "New high-level API" below. Note: `Model.NewContext()`
returns a stateless context for backward compatibility and is not safe for parallel
`Process` calls (may return `ErrStatelessBusy`).
```go
import (
@ -100,6 +104,147 @@ Getting help:
* Follow the discussion for the go bindings [here](https://github.com/ggml-org/whisper.cpp/discussions/312)
## New high-level API (stateful and stateless contexts)
The `pkg/whisper` package now exposes two context kinds:
- StatefulContext: recommended for concurrency. Each context owns its own whisper_state.
- StatelessContext: shares the model context. Simpler, but not suitable for parallel `Process` calls.
### Quick start: stateful context (recommended)
```go
package main
import (
"fmt"
whisper "github.com/ggerganov/whisper.cpp/bindings/go/pkg/whisper"
)
func main() {
// Load model
model, err := whisper.NewModelContext("./models/ggml-small.en.bin")
if err != nil {
panic(err)
}
defer model.Close()
// Configure parameters (optional: provide a config func)
params, err := whisper.NewParameters(model, whisper.SAMPLING_GREEDY, func(p *whisper.Parameters) {
p.SetThreads(4)
p.SetLanguage("en") // or "auto"
p.SetTranslate(false)
})
if err != nil {
panic(err)
}
// Create stateful context (safe for running in parallel goroutines)
ctx, err := whisper.NewStatefulContext(model, params)
if err != nil {
panic(err)
}
defer ctx.Close()
// Your 16-bit mono PCM at 16kHz as float32 samples
var samples []float32
// Process. Callbacks are optional.
if err := ctx.Process(samples, nil, nil, nil); err != nil {
panic(err)
}
// Read segments
for {
seg, err := ctx.NextSegment()
if err != nil {
break
}
fmt.Printf("[%v -> %v] %s\n", seg.Start, seg.End, seg.Text)
}
}
```
### Quick start: stateless context (single worker)
```go
// Load model as above
model, _ := whisper.NewModelContext("./models/ggml-small.en.bin")
defer model.Close()
params, _ := whisper.NewParameters(model, whisper.SAMPLING_GREEDY, nil)
ctx, _ := whisper.NewStatelessContext(model, params)
defer ctx.Close()
if err := ctx.Process(samples, nil, nil, nil); err != nil { panic(err) }
for {
seg, err := ctx.NextSegment()
if err != nil { break }
fmt.Println(seg.Text)
}
```
### Deprecations and migration notes
- The `Context` interface setters are deprecated (SetThreads, SetLanguage, etc.). Use `Parameters` via `NewParameters` and pass it when creating a context.
- `Model.NewContext()` remains for backward compatibility and returns a stateless context by default. Prefer `NewStatefulContext` for concurrency.
- Stateless contexts share the model context. A concurrency gate prevents overlapping `Process` calls and will return `ErrStatelessBusy` if another `Process` is in flight.
- For parallel processing, create one `StatefulContext` per goroutine.
## Benchmarks
Benchmarks live in `pkg/whisper` and compare CPU vs GPU, stateful vs stateless, threads, and callback modes.
### Prerequisites
- Model: `models/ggml-small.en.bin` (or your choice).
- Sample: `samples/jfk.wav`.
- Build the C libs once (also downloads a model for examples):
```bash
cd bindings/go
make examples
# optionally: ./build/go-model-download -out models
```
### Run CPU benchmarks
```bash
cd bindings/go/pkg/whisper
go test -bench=ContextProcessCPU -benchmem -run=^$ .
go test -bench=ContextProcessBigCPU -benchmem -run=^$ .
# or run all
go test -bench=. -benchmem -run=^$ .
```
### Run GPU benchmarks
GPU runs toggle `UseGPU` in model params. Ensure your build has GPU backends enabled:
- CUDA: build the C libs with CUDA
```bash
cd bindings/go
GGML_CUDA=1 make whisper
```
- macOS Metal: supported by default in the build flags on Darwin.
Then run:
```bash
cd bindings/go/pkg/whisper
go test -bench=ContextProcessGPU -benchmem -run=^$ .
go test -bench=ContextProcessBigGPU -benchmem -run=^$ .
```
### What the benchmarks measure
- Variants: device (cpu/gpu) x context kind (stateless/stateful) x threads {1,2,4, NumCPU} x callback mode (NoCallback, WithSegmentCallback).
- Standard Go benchmark outputs: ns/op, B/op, allocs/op. We also set bytes per op to sample bytes.
- Custom metric `ms_process`: wall time per `Process` iteration, reported via `b.ReportMetric`.
- When `printTimings` is enabled, model-level timings are printed for NoCallback runs using `model.PrintTimings()`.
## License
The license for the Go bindings is the same as the license for the rest of the whisper.cpp project, which is the MIT License. See the `LICENSE` file for more details.