public_git
/
whisper.cpp
mirror of https://github.com/ggml-org/whisper.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

benchmark: add harness artifacts and handoff record

This commit is contained in:
shaihi 2026-03-09 10:13:34 +02:00
parent 30c5194c96
commit 40a46f9ed1
498 changed files with 24244 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

155
THREAD_HANDOFF_2026-03-08.md Normal file
View File

@ -0,0 +1,155 @@
# Whisper Metal Optimization Handoff (2026-03-08)
## Scope and Guardrails
- Repository: `/Users/shaihi/Downloads/whisper_optimization`
- Target: Apple Silicon + Metal backend performance
- Benchmark policy: use only `benchmark/bench.sh`
- Current user constraint: run `short.wav` only unless explicitly asked to run `medium.wav` or `long.wav`
- Success criterion for an optimization: median runtime improvement >= 5% with stable repeated runs and acceptable correctness (WER/CER gates)
## Canonical Benchmark Harness
- Script: `/Users/shaihi/Downloads/whisper_optimization/benchmark/bench.sh`
- Parser/reporting: `/Users/shaihi/Downloads/whisper_optimization/benchmark/parse_results.py`
- Fixed config (from harness):
- Warm-up runs: 1
- Measured runs: 5
- Model: `models/ggml-small.en.bin`
- CLI decode flags: `-l en -tp 0 -tpi 0 -nf -bs 1 -bo 1 -fa`
- Thread/process: `-t 8 -p 1`
- Short-only run command:
- `./benchmark/bench.sh --variant <name> --audio short`
## Measurement Definitions
- Runtime metrics:
- wall clock runtime
- first inference latency
- full transcription runtime
- throughput in audio-seconds per second
- tokens/s if available in logs
- Efficiency in this project:
- primary: lower median runtime
- secondary: higher throughput, lower variability (std dev)
- Correctness:
- WER = Word Error Rate
- CER = Character Error Rate
- reference texts in `/Users/shaihi/Downloads/whisper_optimization/benchmark/references/`
- default thresholds in harness: `MAX_WER=0.02`, `MAX_CER=0.02`
## Current Short Reference Size
- File: `/Users/shaihi/Downloads/whisper_optimization/benchmark/references/short.txt`
- Content: `"The Town Hose Story"`
- Count snapshot (`wc -w -m`):
- words: 4
- characters: 22
## Established Baseline and Latest Opt3 Comparison (short only)
### Baseline (current conditions)
- Variant: `metal-baseline-r2`
- Result dir: `/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_232244_metal-baseline-r2`
- Summary file: `/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_232244_metal-baseline-r2/summary.csv`
- Key metrics:
- init_mean_ms: 509.688
- first_inference_mean_s: 3.5896615666
- runtime_median_s: 3.6510759580
- throughput_mean_audio_s_per_s: 7.7969541540
- runtime_std_dev_s: 0.3323240978
- correctness_pass: True
- notes: encode mean=2179.93 ms; decode mean=664.96 ms
### Optimization #3 (latest tuned run)
- Variant: `metal-opt3-v5`
- Result dir: `/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_232524_metal-opt3-v5`
- Summary file: `/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_232524_metal-opt3-v5/summary.csv`
- Key metrics:
- init_mean_ms: 565.146
- first_inference_mean_s: 3.3695700166
- runtime_median_s: 3.6233550420
- throughput_mean_audio_s_per_s: 8.2722771621
- runtime_std_dev_s: 0.0125376431
- correctness_pass: True
- notes: encode mean=2041.77 ms; decode mean=536.73 ms
### Delta (opt3-v5 vs baseline-r2)
- runtime median: 3.6511s -> 3.6234s (about 0.76% faster)
- throughput mean: 7.797 -> 8.272 (about 6.10% higher)
- decode mean (from notes): 664.96 ms -> 536.73 ms (about 19.3% lower)
- success criterion status: NOT met on median runtime (needs >=5%)
## Optimization #3 Code Changes (currently uncommitted)
- File: `/Users/shaihi/Downloads/whisper_optimization/ggml/src/ggml-metal/ggml-metal-ops.cpp`
- Changes:
- `const int ne11_mm_min = props_dev->supports_gpu_family_apple7 ? 6 : 8;`
- `const int ne21_mm_id_min = props_dev->supports_gpu_family_apple7 ? 24 : 32;`
- Intent:
- allow simdgroup matmul kernel path at slightly smaller matrix thresholds on Apple7+ GPUs to reduce decode-side overhead
## Working Tree Snapshot
- Modified: `/Users/shaihi/Downloads/whisper_optimization/ggml/src/ggml-metal/ggml-metal-ops.cpp`
- Untracked benchmark artifacts exist under `/Users/shaihi/Downloads/whisper_optimization/benchmark/`
## Immediate Task List for Fresh Thread
1. Choose next optimization branch to test next: #1 (decoder input staging/mask overhead) or #2 (Metal command buffer/scheduling overhead).
2. Keep `short`-only runs for dev iterations:
- baseline refresh command: `./benchmark/bench.sh --variant metal-baseline-r3 --audio short`
- candidate command: `./benchmark/bench.sh --variant metal-optX-v1 --audio short`
3. Compare candidate against the refreshed baseline using `summary.csv` only; report median runtime delta first.
4. Keep correctness gates enabled; reject changes with WER/CER threshold failures.
5. If short-only looks strong and user explicitly asks, run medium/long to verify scalability.
## Notes to Carry Forward
- User requested: test short only by default.
- User requested workflow: prove improvement for one optimization first, then ask before moving to others.
## 2026-03-09 Isolation Experiment (User-requested: short+medium+long)
### Isolated options tested (single-line edits only)
- Option A only:
- `/Users/shaihi/Downloads/whisper_optimization/ggml/src/ggml-metal/ggml-metal-ops.cpp`
- `ne11_mm_min: 8 -> props_dev->supports_gpu_family_apple7 ? 6 : 8`
- Option B only:
- `/Users/shaihi/Downloads/whisper_optimization/ggml/src/ggml-metal/ggml-metal-ops.cpp`
- `ne21_mm_id_min: 32 -> props_dev->supports_gpu_family_apple7 ? 24 : 32`
### Benchmark runs executed
- Baseline: `/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_233303_metal-baseline-r3`
- Option A: `/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_235028_metal-optA-r1`
- Option B: `/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260309_000042_metal-optB-r1`
- Consolidated table: `/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260309_isolation_summary.md`
### Result snapshot
- Short:
- Option A and B are effectively neutral/slightly worse on runtime median (-0.27%, -0.13% vs baseline)
- Medium:
- Option A is inconclusive (runtime median slightly worse, very high run variance)
- Option B is strongly faster (+35.65% runtime median improvement vs baseline)
- Long:
- Option A and B both improve runtime median (+31.41%, +34.50% vs baseline)
- Option B has better stability (std dev 0.457s vs Option A 2.080s)
- Correctness:
- All runs pass WER/CER gates (all `correctness_pass=true`)
### Certainty assessment
- Option B is the strongest candidate from this pass.
- Baseline long variance was high in this run set, so final certainty is moderate (not final-proof).
- Recommended confirmation before merging:
- rerun ordered A/B/A baseline sandwich (baseline -> option -> baseline) on medium+long
- keep identical thermal/system conditions as much as possible
### Repository state after experiment
- Source reverted to baseline thresholds.
- Rebuilt so binaries match baseline source.
- No optimization currently left applied in source.
## 2026-03-09 Option B Kept + Run
- Source state: Option B is currently applied in `/Users/shaihi/Downloads/whisper_optimization/ggml/src/ggml-metal/ggml-metal-ops.cpp`
- `ne21_mm_id_min = props_dev->supports_gpu_family_apple7 ? 24 : 32;`
- Build completed after applying Option B.
- Benchmark executed (short only):
- Command: `./benchmark/bench.sh --variant metal-optB-final --audio short`
- Result dir: `/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260309_084640_metal-optB-final`
- Summary: `/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260309_084640_metal-optB-final/summary.csv`
- Key metrics (short):
- runtime_median_s: `1.061812124986318`
- throughput_mean_audio_s_per_s: `28.18097234137864`
- runtime_std_dev_s: `0.00702706432431074`
- correctness_pass: `True`

79
benchmark/README.md Normal file
View File

@ -0,0 +1,79 @@
# Benchmark Harness
This harness is the single source of truth for baseline and optimization performance measurements.
## Layout
```text
benchmark/
bench.sh
audio/
short.wav
medium.wav
long.wav
lock.json
references/
short.txt
medium.txt
long.txt
parse_results.py
results/
```
## Requirements
- `build/bin/whisper-cli` exists (build first with CMake).
- Audio files exist at:
- `benchmark/audio/short.wav` (~30s)
- `benchmark/audio/medium.wav` (~5m)
- `benchmark/audio/long.wav` (~30m)
- Audio files must be 16 kHz, mono, 16-bit WAV.
## Fixed Benchmark Policy
- Warm-up runs: 1
- Measured runs: 5
- Sequential execution only
- Fixed model: `models/ggml-small.en.bin`
- Fixed decode flags:
- `-l en -tp 0 -tpi 0 -nf -bs 1 -bo 1 -fa`
- Fixed threading config:
- `-t 8 -p 1`
## Usage
1. Create the lock file (checksums + durations):
```bash
./benchmark/bench.sh --create-lock
```
2. Run benchmark (after lock exists):
```bash
./benchmark/bench.sh --variant metal-baseline
```
## Outputs
Each run writes to `benchmark/results/<timestamp>_<variant>/`:
- `config.json`: exact benchmark config + environment metadata
- `validated_inputs.json`: lock validation snapshot
- `raw/...`: per-run logs and metadata (`warmup_*.log`, `run_*.log`, `*.meta.json`)
- `runs.csv`: per-run metrics
- `summary.csv`: aggregated metrics
- `summary.json`: detailed aggregates
- `summary.md`: required table format
- `correctness.json`: WER/CER gate report against references
## Correctness Gate
- References are read from `benchmark/references/{short,medium,long}.txt`.
- The parser extracts transcript text from each measured run log and computes:
- WER (word error rate)
- CER (character error rate)
- Default enforcement thresholds from `bench.sh`:
- `MAX_WER=0.02`
- `MAX_CER=0.02`
- If enforcement is enabled and references are missing or thresholds are exceeded, the run exits non-zero.

BIN
benchmark/__pycache__/parse_results.cpython-313.pyc Normal file
View File

Binary file not shown.

25
benchmark/audio/README.md Normal file
View File

@ -0,0 +1,25 @@
# Audio Inputs
Place fixed benchmark inputs in this directory:
- `short.wav` (~30 seconds)
- `medium.wav` (~5 minutes)
- `long.wav` (~30 minutes)
All inputs must be:
- 16 kHz
- mono
- 16-bit PCM WAV
Example conversion:
```bash
ffmpeg -i input.ext -ar 16000 -ac 1 -c:a pcm_s16le output.wav
```
After files are in place, create/update the reproducibility lock:
```bash
../bench.sh --create-lock
```

28
benchmark/audio/lock.json Normal file
View File

@ -0,0 +1,28 @@
{
"audio": {
"long": {
"channels": 1,
"duration_s": 1800.0,
"sample_rate_hz": 16000,
"sample_width_bytes": 2,
"sha256": "fd586cbcf50f12d8eca4908bd38a3cd71a0c146992923d28259376aa0e6f4367"
},
"medium": {
"channels": 1,
"duration_s": 300.0,
"sample_rate_hz": 16000,
"sample_width_bytes": 2,
"sha256": "a918fe0de597908c917d44c4bfc54cfbfad3dbf79a763cdc558b0dbd95fe2aca"
},
"short": {
"channels": 1,
"duration_s": 30.0,
"sample_rate_hz": 16000,
"sample_width_bytes": 2,
"sha256": "bf6376be2d6749925b9fb53a194a55cd0330329f0425705e5b4e67a0dd1ebd3a"
}
},
"model_rel": "models/ggml-small.en.bin",
"model_sha256": "c6138d6d58ecc8322097e0f987c32f1be8bb0a18532a3f88f734d1bbf9c41e5d",
"schema_version": 1
}

BIN
benchmark/audio/long.wav Normal file
View File

Binary file not shown.

BIN
benchmark/audio/medium.wav Normal file
View File

Binary file not shown.

BIN
benchmark/audio/short.wav Normal file
View File

Binary file not shown.

BIN
benchmark/audio/source_mobydick_054.mp3 Normal file
View File

Binary file not shown.

511
benchmark/bench.sh Executable file
View File

@ -0,0 +1,511 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
AUDIO_DIR="${SCRIPT_DIR}/audio"
REFS_DIR="${SCRIPT_DIR}/references"
RESULTS_ROOT="${SCRIPT_DIR}/results"
LOCK_FILE="${AUDIO_DIR}/lock.json"
# Fixed benchmark configuration for reproducibility.
WARMUP_RUNS=1
MEASURED_RUNS=5
THREADS=8
PROCESSORS=1
VARIANT="metal-baseline"
MODEL_REL_DEFAULT="models/ggml-small.en.bin"
MAX_WER=0.02
MAX_CER=0.02
ENFORCE_CORRECTNESS=1
AUDIO_KEYS=()
CLI_BIN="${REPO_ROOT}/build/bin/whisper-cli"
CLI_ARGS=(
"-l" "en"
"-tp" "0"
"-tpi" "0"
"-nf"
"-bs" "1"
"-bo" "1"
"-fa"
)
usage() {
cat <<'EOF'
Usage:
benchmark/bench.sh --create-lock
benchmark/bench.sh [--variant <name>] [--audio <short|medium|long>]...
benchmark/bench.sh [--variant <name>] --all-audio
Description:
--create-lock Validates benchmark/audio/{short,medium,long}.wav and writes benchmark/audio/lock.json
with file hashes + durations. This lock is required for benchmark runs.
--variant Logical variant label in output tables (default: metal-baseline).
--audio Add one audio key to run: short, medium, or long.
If omitted, only short is run (development default).
--all-audio Run short + medium + long.
Notes:
- This harness runs 1 warm-up + 5 measured runs per audio sample.
- Runs are always sequential and use fixed model + fixed CLI flags.
- Audio files must be 16 kHz, mono, 16-bit WAV.
- Correctness is computed via WER/CER against benchmark/references/{short,medium,long}.txt.
EOF
}
MODE="run"
while [[ $# -gt 0 ]]; do
case "$1" in
--create-lock)
MODE="create-lock"
shift
;;
--variant)
VARIANT="$2"
shift 2
;;
--audio)
case "$2" in
short|medium|long)
AUDIO_KEYS+=( "$2" )
;;
*)
echo "Invalid --audio value: $2 (expected short|medium|long)" >&2
exit 2
;;
esac
shift 2
;;
--all-audio)
AUDIO_KEYS=( "short" "medium" "long" )
shift
;;
--help|-h)
usage
exit 0
;;
*)
echo "Unknown argument: $1" >&2
usage >&2
exit 2
;;
esac
done
require_file() {
local path="$1"
if [[ ! -f "$path" ]]; then
echo "Required file not found: $path" >&2
exit 1
fi
}
create_lock_file() {
mkdir -p "${AUDIO_DIR}"
require_file "${REPO_ROOT}/${MODEL_REL_DEFAULT}"
require_file "${AUDIO_DIR}/short.wav"
require_file "${AUDIO_DIR}/medium.wav"
require_file "${AUDIO_DIR}/long.wav"
python3 - "${LOCK_FILE}" "${REPO_ROOT}" "${MODEL_REL_DEFAULT}" "${AUDIO_DIR}" <<'PY'
import contextlib
import hashlib
import json
import os
import sys
import wave
from pathlib import Path
lock_path = Path(sys.argv[1])
repo_root = Path(sys.argv[2])
model_rel = sys.argv[3]
audio_dir = Path(sys.argv[4])
def sha256(path: Path) -> str:
h = hashlib.sha256()
with path.open("rb") as f:
while True:
chunk = f.read(1024 * 1024)
if not chunk:
break
h.update(chunk)
return h.hexdigest()
def inspect_wav(path: Path):
with contextlib.closing(wave.open(str(path), "rb")) as wf:
channels = wf.getnchannels()
sample_rate = wf.getframerate()
sample_width = wf.getsampwidth()
frames = wf.getnframes()
if channels != 1:
raise SystemExit(f"{path}: expected mono (1 channel), got {channels}")
if sample_rate != 16000:
raise SystemExit(f"{path}: expected 16000 Hz, got {sample_rate}")
if sample_width != 2:
raise SystemExit(f"{path}: expected 16-bit PCM (2 bytes), got {sample_width}")
duration_s = frames / float(sample_rate)
return {
"sha256": sha256(path),
"duration_s": duration_s,
"sample_rate_hz": sample_rate,
"channels": channels,
"sample_width_bytes": sample_width,
}
model_path = repo_root / model_rel
if not model_path.is_file():
raise SystemExit(f"Model not found: {model_path}")
lock = {
"schema_version": 1,
"model_rel": model_rel,
"model_sha256": sha256(model_path),
"audio": {
"short": inspect_wav(audio_dir / "short.wav"),
"medium": inspect_wav(audio_dir / "medium.wav"),
"long": inspect_wav(audio_dir / "long.wav"),
},
}
tmp = lock_path.with_suffix(".json.tmp")
tmp.write_text(json.dumps(lock, indent=2, sort_keys=True) + "\n", encoding="utf-8")
os.replace(tmp, lock_path)
print(f"Wrote lock file: {lock_path}")
PY
}
validate_inputs_against_lock() {
require_file "${LOCK_FILE}"
require_file "${CLI_BIN}"
local validation_json="$1"
python3 - "${LOCK_FILE}" "${REPO_ROOT}" "${AUDIO_DIR}" "${MODEL_REL_DEFAULT}" > "${validation_json}" <<'PY'
import contextlib
import hashlib
import json
import sys
import wave
from pathlib import Path
lock_path = Path(sys.argv[1])
repo_root = Path(sys.argv[2])
audio_dir = Path(sys.argv[3])
configured_model_rel = sys.argv[4]
def sha256(path: Path) -> str:
h = hashlib.sha256()
with path.open("rb") as f:
while True:
chunk = f.read(1024 * 1024)
if not chunk:
break
h.update(chunk)
return h.hexdigest()
def inspect_wav(path: Path):
with contextlib.closing(wave.open(str(path), "rb")) as wf:
channels = wf.getnchannels()
sample_rate = wf.getframerate()
sample_width = wf.getsampwidth()
frames = wf.getnframes()
duration_s = frames / float(sample_rate)
return channels, sample_rate, sample_width, duration_s
lock = json.loads(lock_path.read_text(encoding="utf-8"))
if lock.get("model_rel") != configured_model_rel:
raise SystemExit(
f"Lock model_rel mismatch: lock={lock.get('model_rel')} script={configured_model_rel}"
)
model_rel = lock["model_rel"]
model_path = repo_root / model_rel
if not model_path.is_file():
raise SystemExit(f"Model file not found: {model_path}")
actual_model_sha = sha256(model_path)
if actual_model_sha != lock["model_sha256"]:
raise SystemExit(
f"Model checksum mismatch for {model_path}. expected={lock['model_sha256']} actual={actual_model_sha}"
)
validated_audio = {}
for key in ("short", "medium", "long"):
wav_path = audio_dir / f"{key}.wav"
if not wav_path.is_file():
raise SystemExit(f"Audio file missing: {wav_path}")
expected = lock["audio"][key]
actual_sha = sha256(wav_path)
if actual_sha != expected["sha256"]:
raise SystemExit(
f"Checksum mismatch for {wav_path}. expected={expected['sha256']} actual={actual_sha}"
)
channels, sample_rate, sample_width, duration_s = inspect_wav(wav_path)
if channels != 1 or sample_rate != 16000 or sample_width != 2:
raise SystemExit(
f"Format mismatch for {wav_path}: channels={channels}, sample_rate={sample_rate}, sample_width={sample_width}"
)
validated_audio[key] = {
"path": str(wav_path),
"duration_s": duration_s,
"sha256": actual_sha,
}
print(
json.dumps(
{
"model_rel": model_rel,
"model_abs": str(model_path),
"model_sha256": actual_model_sha,
"audio": validated_audio,
},
indent=2,
sort_keys=True,
)
)
PY
}
write_config_file() {
local config_path="$1"
local validation_json="$2"
python3 - "${config_path}" "${validation_json}" "${VARIANT}" "${THREADS}" "${PROCESSORS}" "${WARMUP_RUNS}" "${MEASURED_RUNS}" "${REPO_ROOT}" "${CLI_BIN}" "${CLI_ARGS[@]}" <<'PY'
import datetime as dt
import json
import platform
import subprocess
import sys
from pathlib import Path
config_path = Path(sys.argv[1])
validation_path = Path(sys.argv[2])
variant = sys.argv[3]
threads = int(sys.argv[4])
processors = int(sys.argv[5])
warmup_runs = int(sys.argv[6])
measured_runs = int(sys.argv[7])
repo_root = sys.argv[8]
cli_bin = sys.argv[9]
cli_args = sys.argv[10:]
validation = json.loads(validation_path.read_text(encoding="utf-8"))
def run_cmd(args):
try:
return subprocess.check_output(args, stderr=subprocess.STDOUT, text=True).strip()
except Exception:
return ""
env = {
"git_commit": run_cmd(["git", "-C", repo_root, "rev-parse", "HEAD"]),
"git_short_commit": run_cmd(["git", "-C", repo_root, "rev-parse", "--short", "HEAD"]),
"sw_vers": run_cmd(["sw_vers"]),
"uname": run_cmd(["uname", "-a"]),
"hw_memsize": run_cmd(["sysctl", "-n", "hw.memsize"]),
"cpu_brand_string": run_cmd(["sysctl", "-n", "machdep.cpu.brand_string"]),
"xcodebuild_version": run_cmd(["xcodebuild", "-version"]),
"clang_version": run_cmd(["clang", "--version"]),
"cmake_version": run_cmd(["cmake", "--version"]),
"python_version": platform.python_version(),
}
cfg = {
"created_at_utc": dt.datetime.now(dt.timezone.utc).isoformat(),
"variant": variant,
"run_policy": {
"warmup_runs": warmup_runs,
"measured_runs": measured_runs,
"sequential_execution": True,
},
"model": {
"rel_path": validation["model_rel"],
"abs_path": validation["model_abs"],
"sha256": validation["model_sha256"],
},
"audio": validation["audio"],
"cli": {
"binary": cli_bin,
"args": cli_args,
"threads": threads,
"processors": processors,
},
"environment": env,
}
config_path.write_text(json.dumps(cfg, indent=2, sort_keys=True) + "\n", encoding="utf-8")
PY
}
run_one() {
local log_path="$1"
local meta_path="$2"
local audio_key="$3"
local audio_duration="$4"
local run_kind="$5"
local run_index="$6"
shift 6
local cmd=( "$@" )
python3 - "${log_path}" "${meta_path}" "${audio_key}" "${audio_duration}" "${run_kind}" "${run_index}" "${cmd[@]}" <<'PY'
import datetime as dt
import json
import re
import subprocess
import sys
import time
from pathlib import Path
log_path = Path(sys.argv[1])
meta_path = Path(sys.argv[2])
audio_key = sys.argv[3]
audio_duration_s = float(sys.argv[4])
run_kind = sys.argv[5]
run_index = int(sys.argv[6])
cmd = sys.argv[7:]
segment_re = re.compile(r"^\[\d{2}:\d{2}:\d{2}\.\d{3}\s+-->\s+\d{2}:\d{2}:\d{2}\.\d{3}\]")
start_utc = dt.datetime.now(dt.timezone.utc).isoformat()
start = time.perf_counter()
first_inference_s = None
with log_path.open("w", encoding="utf-8") as logf:
proc = subprocess.Popen(
cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
bufsize=1,
)
assert proc.stdout is not None
for line in proc.stdout:
logf.write(line)
if first_inference_s is None and segment_re.search(line.strip()):
first_inference_s = time.perf_counter() - start
rc = proc.wait()
end = time.perf_counter()
end_utc = dt.datetime.now(dt.timezone.utc).isoformat()
meta = {
"audio_key": audio_key,
"audio_duration_s": audio_duration_s,
"run_kind": run_kind,
"run_index": run_index,
"command": cmd,
"log_path": str(log_path),
"start_utc": start_utc,
"end_utc": end_utc,
"wall_clock_runtime_s": end - start,
"first_inference_latency_s": first_inference_s,
"exit_code": rc,
}
meta_path.write_text(json.dumps(meta, indent=2, sort_keys=True) + "\n", encoding="utf-8")
sys.exit(rc)
PY
}
if [[ "${MODE}" == "create-lock" ]]; then
create_lock_file
exit 0
fi
if [[ "${#AUDIO_KEYS[@]}" -eq 0 ]]; then
AUDIO_KEYS=( "short" )
fi
mkdir -p "${RESULTS_ROOT}"
RUN_ID="$(date '+%Y%m%d_%H%M%S')"
RUN_DIR="${RESULTS_ROOT}/${RUN_ID}_${VARIANT}"
RAW_DIR="${RUN_DIR}/raw"
mkdir -p "${RAW_DIR}"
VALIDATION_JSON="${RUN_DIR}/validated_inputs.json"
validate_inputs_against_lock "${VALIDATION_JSON}"
write_config_file "${RUN_DIR}/config.json" "${VALIDATION_JSON}"
MODEL_PATH="$(python3 - "${VALIDATION_JSON}" <<'PY'
import json
import sys
from pathlib import Path
data = json.loads(Path(sys.argv[1]).read_text(encoding="utf-8"))
print(data["model_abs"])
PY
)"
echo "Benchmark run directory: ${RUN_DIR}"
echo "Variant: ${VARIANT}"
echo "Model: ${MODEL_PATH}"
echo "Warm-up runs: ${WARMUP_RUNS}, measured runs: ${MEASURED_RUNS}"
echo "Threads: ${THREADS}, processors: ${PROCESSORS}"
echo "Fixed CLI flags: ${CLI_ARGS[*]}"
echo "Audio set: ${AUDIO_KEYS[*]}"
for audio_key in "${AUDIO_KEYS[@]}"; do
audio_path="${AUDIO_DIR}/${audio_key}.wav"
audio_duration="$(python3 - "${VALIDATION_JSON}" "${audio_key}" <<'PY'
import json
import sys
from pathlib import Path
data = json.loads(Path(sys.argv[1]).read_text(encoding="utf-8"))
print(data["audio"][sys.argv[2]]["duration_s"])
PY
)"
audio_run_dir="${RAW_DIR}/${audio_key}"
mkdir -p "${audio_run_dir}"
for i in $(seq 1 "${WARMUP_RUNS}"); do
printf -v idx "%02d" "${i}"
echo "[${audio_key}] warm-up ${i}/${WARMUP_RUNS}"
run_one \
"${audio_run_dir}/warmup_${idx}.log" \
"${audio_run_dir}/warmup_${idx}.meta.json" \
"${audio_key}" \
"${audio_duration}" \
"warmup" \
"${i}" \
"${CLI_BIN}" -m "${MODEL_PATH}" -f "${audio_path}" -t "${THREADS}" -p "${PROCESSORS}" "${CLI_ARGS[@]}"
done
for i in $(seq 1 "${MEASURED_RUNS}"); do
printf -v idx "%02d" "${i}"
echo "[${audio_key}] measured ${i}/${MEASURED_RUNS}"
run_one \
"${audio_run_dir}/run_${idx}.log" \
"${audio_run_dir}/run_${idx}.meta.json" \
"${audio_key}" \
"${audio_duration}" \
"measured" \
"${i}" \
"${CLI_BIN}" -m "${MODEL_PATH}" -f "${audio_path}" -t "${THREADS}" -p "${PROCESSORS}" "${CLI_ARGS[@]}"
done
done
PARSE_ARGS=(
"--run-dir" "${RUN_DIR}"
"--refs-dir" "${REFS_DIR}"
"--max-wer" "${MAX_WER}"
"--max-cer" "${MAX_CER}"
)
if [[ "${ENFORCE_CORRECTNESS}" == "1" ]]; then
PARSE_ARGS+=( "--enforce-correctness" )
fi
python3 "${SCRIPT_DIR}/parse_results.py" "${PARSE_ARGS[@]}"
echo "Completed benchmark parsing:"
echo " ${RUN_DIR}/runs.csv"
echo " ${RUN_DIR}/summary.csv"
echo " ${RUN_DIR}/summary.md"
echo " ${RUN_DIR}/correctness.json"

546
benchmark/parse_results.py Executable file
View File

@ -0,0 +1,546 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import csv
from difflib import SequenceMatcher
import json
import re
import statistics
import unicodedata
from pathlib import Path
from typing import Dict, Iterable, List, Optional, Sequence, Tuple
TIMING_PATTERNS = {
"model_load_ms": re.compile(r"load time\s*=\s*([0-9]+(?:\.[0-9]+)?)\s*ms"),
"mel_ms": re.compile(r"mel time\s*=\s*([0-9]+(?:\.[0-9]+)?)\s*ms"),
"sample_ms": re.compile(r"sample time\s*=\s*([0-9]+(?:\.[0-9]+)?)\s*ms"),
"encode_ms": re.compile(r"encode time\s*=\s*([0-9]+(?:\.[0-9]+)?)\s*ms"),
"decode_ms": re.compile(r"decode time\s*=\s*([0-9]+(?:\.[0-9]+)?)\s*ms"),
"batchd_ms": re.compile(r"batchd time\s*=\s*([0-9]+(?:\.[0-9]+)?)\s*ms"),
"prompt_ms": re.compile(r"prompt time\s*=\s*([0-9]+(?:\.[0-9]+)?)\s*ms"),
"full_runtime_ms": re.compile(r"total time\s*=\s*([0-9]+(?:\.[0-9]+)?)\s*ms"),
}
TOKEN_RATE_PATTERNS = [
re.compile(r"tokens?\s*/\s*s(?:ec(?:ond)?)?\s*[:=]\s*([0-9]+(?:\.[0-9]+)?)", re.IGNORECASE),
re.compile(r"tokens?\s+per\s+second\s*[:=]\s*([0-9]+(?:\.[0-9]+)?)", re.IGNORECASE),
]
SEGMENT_LINE_PATTERN = re.compile(
r"^\[\d{2}:\d{2}:\d{2}\.\d{3}\s+-->\s+\d{2}:\d{2}:\d{2}\.\d{3}\]\s*(.*)$"
)
def parse_args() -> argparse.Namespace:
p = argparse.ArgumentParser(description="Parse benchmark logs and compute run statistics.")
p.add_argument("--run-dir", required=True, help="Path to benchmark results run directory.")
p.add_argument(
"--refs-dir",
default=None,
help="Directory containing reference transcripts {short,medium,long}.txt (default: benchmark/references).",
)
p.add_argument("--max-wer", type=float, default=0.02, help="Max allowed median WER per audio.")
p.add_argument("--max-cer", type=float, default=0.02, help="Max allowed median CER per audio.")
p.add_argument(
"--enforce-correctness",
action="store_true",
help="Exit non-zero if any referenced audio exceeds max WER/CER or if references are missing.",
)
return p.parse_args()
def to_float_or_none(value: Optional[float]) -> Optional[float]:
if value is None:
return None
return float(value)
def parse_log_metrics(log_text: str) -> Dict[str, Optional[float]]:
out: Dict[str, Optional[float]] = {k: None for k in TIMING_PATTERNS.keys()}
for key, pat in TIMING_PATTERNS.items():
m = pat.search(log_text)
out[key] = float(m.group(1)) if m else None
token_rate = None
for pat in TOKEN_RATE_PATTERNS:
m = pat.search(log_text)
if m:
token_rate = float(m.group(1))
break
out["tokens_per_second"] = token_rate
return out
def extract_transcript(log_text: str) -> str:
lines: List[str] = []
for raw_line in log_text.splitlines():
line = raw_line.strip()
m = SEGMENT_LINE_PATTERN.match(line)
if not m:
continue
text = m.group(1).strip()
if text:
lines.append(text)
return " ".join(lines).strip()
def normalize_text(text: str) -> str:
text = unicodedata.normalize("NFKC", text).lower()
text = text.replace("_", " ")
text = re.sub(r"[^\w\s']", " ", text, flags=re.UNICODE)
text = re.sub(r"\s+", " ", text).strip()
return text
def levenshtein_distance(a: Sequence[str], b: Sequence[str]) -> int:
if not a:
return len(b)
if not b:
return len(a)
prev = list(range(len(b) + 1))
for i, ca in enumerate(a, start=1):
curr = [i]
for j, cb in enumerate(b, start=1):
cost = 0 if ca == cb else 1
curr.append(
min(
prev[j] + 1, # delete
curr[j - 1] + 1, # insert
prev[j - 1] + cost, # substitute
)
)
prev = curr
return prev[-1]
def error_rate(reference: Sequence[str], hypothesis: Sequence[str]) -> float:
if len(reference) == 0:
return 0.0 if len(hypothesis) == 0 else 1.0
# Exact DP is expensive for long transcripts. Use exact distance for small/medium
# inputs and a fast similarity-based fallback for long inputs.
if len(reference) * max(1, len(hypothesis)) <= 2_000_000:
dist = levenshtein_distance(reference, hypothesis)
return dist / float(len(reference))
ratio = SequenceMatcher(a=list(reference), b=list(hypothesis), autojunk=False).ratio()
return max(0.0, min(1.0, 1.0 - ratio))
def compute_wer_cer(reference_text: str, hypothesis_text: str) -> Tuple[float, float]:
ref_norm = normalize_text(reference_text)
hyp_norm = normalize_text(hypothesis_text)
ref_words = ref_norm.split() if ref_norm else []
hyp_words = hyp_norm.split() if hyp_norm else []
ref_chars = list(ref_norm.replace(" ", ""))
hyp_chars = list(hyp_norm.replace(" ", ""))
wer = error_rate(ref_words, hyp_words)
cer = error_rate(ref_chars, hyp_chars)
return wer, cer
def safe_mean(values: Iterable[float]) -> Optional[float]:
vals = list(values)
return statistics.mean(vals) if vals else None
def safe_median(values: Iterable[float]) -> Optional[float]:
vals = list(values)
return statistics.median(vals) if vals else None
def safe_min(values: Iterable[float]) -> Optional[float]:
vals = list(values)
return min(vals) if vals else None
def safe_max(values: Iterable[float]) -> Optional[float]:
vals = list(values)
return max(vals) if vals else None
def safe_stdev(values: Iterable[float]) -> Optional[float]:
vals = list(values)
if not vals:
return None
if len(vals) == 1:
return 0.0
return statistics.stdev(vals)
def stats_block(values: Iterable[float]) -> Dict[str, Optional[float]]:
vals = list(values)
return {
"mean": safe_mean(vals),
"median": safe_median(vals),
"min": safe_min(vals),
"max": safe_max(vals),
"std_dev": safe_stdev(vals),
}
def fmt(value: Optional[float], suffix: str = "", decimals: int = 3) -> str:
if value is None:
return "NA"
return f"{value:.{decimals}f}{suffix}"
def write_runs_csv(path: Path, rows: List[Dict[str, object]]) -> None:
if not rows:
path.write_text("", encoding="utf-8")
return
fieldnames = [
"variant",
"model",
"audio_key",
"audio_length_s",
"run_kind",
"run_index",
"wall_clock_runtime_s",
"tokens_per_second",
"audio_seconds_per_second",
"model_load_ms",
"first_inference_latency_s",
"full_runtime_ms",
"mel_ms",
"sample_ms",
"encode_ms",
"decode_ms",
"batchd_ms",
"prompt_ms",
"transcript_word_error_rate",
"transcript_char_error_rate",
"reference_present",
"reference_path",
"transcript_path",
"metal_kernel_runtime_ms",
"cpu_orchestration_ms",
"log_path",
]
with path.open("w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=fieldnames)
writer.writeheader()
writer.writerows(rows)
def main() -> int:
args = parse_args()
run_dir = Path(args.run_dir).resolve()
if not run_dir.is_dir():
raise SystemExit(f"Run directory does not exist: {run_dir}")
if args.refs_dir:
refs_dir = Path(args.refs_dir).resolve()
else:
refs_dir = run_dir.parent.parent / "references"
config_path = run_dir / "config.json"
if not config_path.is_file():
raise SystemExit(f"Missing config file: {config_path}")
config = json.loads(config_path.read_text(encoding="utf-8"))
variant = config["variant"]
model = Path(config["model"]["rel_path"]).name
run_meta_paths = sorted((run_dir / "raw").glob("*/*run_*.meta.json"))
if not run_meta_paths:
raise SystemExit(f"No measured run metadata files found under {(run_dir / 'raw')}")
reference_texts: Dict[str, Optional[str]] = {}
reference_paths: Dict[str, Path] = {}
missing_references: List[str] = []
for audio_key in ("short", "medium", "long"):
ref_path = refs_dir / f"{audio_key}.txt"
reference_paths[audio_key] = ref_path
if ref_path.is_file():
reference_texts[audio_key] = ref_path.read_text(encoding="utf-8", errors="replace").strip()
else:
reference_texts[audio_key] = None
missing_references.append(audio_key)
run_rows: List[Dict[str, object]] = []
for meta_path in run_meta_paths:
meta = json.loads(meta_path.read_text(encoding="utf-8"))
if meta.get("run_kind") != "measured":
continue
log_path = Path(meta["log_path"])
log_text = log_path.read_text(encoding="utf-8", errors="replace")
parsed = parse_log_metrics(log_text)
transcript_text = extract_transcript(log_text)
transcript_path = log_path.with_suffix(".transcript.txt")
transcript_path.write_text(transcript_text + "\n", encoding="utf-8")
audio_key = str(meta["audio_key"])
ref_text = reference_texts.get(audio_key)
ref_present = ref_text is not None
wer = None
cer = None
if ref_present:
wer, cer = compute_wer_cer(ref_text, transcript_text)
wall_s = float(meta["wall_clock_runtime_s"])
audio_len_s = float(meta["audio_duration_s"])
audio_s_per_s = audio_len_s / wall_s if wall_s > 0 else None
full_runtime_ms = parsed["full_runtime_ms"]
if full_runtime_ms is None:
full_runtime_ms = wall_s * 1000.0
run_rows.append(
{
"variant": variant,
"model": model,
"audio_key": audio_key,
"audio_length_s": audio_len_s,
"run_kind": meta["run_kind"],
"run_index": int(meta["run_index"]),
"wall_clock_runtime_s": wall_s,
"tokens_per_second": to_float_or_none(parsed["tokens_per_second"]),
"audio_seconds_per_second": to_float_or_none(audio_s_per_s),
"model_load_ms": to_float_or_none(parsed["model_load_ms"]),
"first_inference_latency_s": to_float_or_none(meta.get("first_inference_latency_s")),
"full_runtime_ms": to_float_or_none(full_runtime_ms),
"mel_ms": to_float_or_none(parsed["mel_ms"]),
"sample_ms": to_float_or_none(parsed["sample_ms"]),
"encode_ms": to_float_or_none(parsed["encode_ms"]),
"decode_ms": to_float_or_none(parsed["decode_ms"]),
"batchd_ms": to_float_or_none(parsed["batchd_ms"]),
"prompt_ms": to_float_or_none(parsed["prompt_ms"]),
"transcript_word_error_rate": to_float_or_none(wer),
"transcript_char_error_rate": to_float_or_none(cer),
"reference_present": ref_present,
"reference_path": str(reference_paths[audio_key]),
"transcript_path": str(transcript_path),
"metal_kernel_runtime_ms": None,
"cpu_orchestration_ms": None,
"log_path": str(log_path),
}
)
if not run_rows:
raise SystemExit("No measured runs were parsed.")
runs_csv_path = run_dir / "runs.csv"
write_runs_csv(runs_csv_path, run_rows)
by_audio: Dict[str, List[Dict[str, object]]] = {}
for row in run_rows:
by_audio.setdefault(str(row["audio_key"]), []).append(row)
summary_rows: List[Dict[str, object]] = []
md_lines = [
"| Variant | Model | Audio Length | Runs | Init Mean | First Inference Mean | Runtime Median | Throughput | Std Dev | Notes |",
"|---|---|---:|---:|---:|---:|---:|---:|---:|---|",
]
overall_correctness_pass = True
for audio_key in ("short", "medium", "long"):
rows = sorted(by_audio.get(audio_key, []), key=lambda r: int(r["run_index"]))
if not rows:
continue
audio_length_s = float(rows[0]["audio_length_s"])
wall_values = [float(r["wall_clock_runtime_s"]) for r in rows if r["wall_clock_runtime_s"] is not None]
load_values = [float(r["model_load_ms"]) for r in rows if r["model_load_ms"] is not None]
first_values = [float(r["first_inference_latency_s"]) for r in rows if r["first_inference_latency_s"] is not None]
throughput_values = [float(r["audio_seconds_per_second"]) for r in rows if r["audio_seconds_per_second"] is not None]
token_values = [float(r["tokens_per_second"]) for r in rows if r["tokens_per_second"] is not None]
full_runtime_values = [float(r["full_runtime_ms"]) for r in rows if r["full_runtime_ms"] is not None]
encode_values = [float(r["encode_ms"]) for r in rows if r["encode_ms"] is not None]
decode_values = [float(r["decode_ms"]) for r in rows if r["decode_ms"] is not None]
wer_values = [float(r["transcript_word_error_rate"]) for r in rows if r["transcript_word_error_rate"] is not None]
cer_values = [float(r["transcript_char_error_rate"]) for r in rows if r["transcript_char_error_rate"] is not None]
reference_present = bool(rows[0]["reference_present"])
runtime_stats = stats_block(wall_values)
full_runtime_stats = stats_block(full_runtime_values)
load_stats = stats_block(load_values)
first_stats = stats_block(first_values)
throughput_stats = stats_block(throughput_values)
token_stats = stats_block(token_values)
encode_stats = stats_block(encode_values)
decode_stats = stats_block(decode_values)
wer_stats = stats_block(wer_values)
cer_stats = stats_block(cer_values)
notes_parts: List[str] = []
if token_stats["mean"] is None:
notes_parts.append("tokens/s unavailable")
else:
notes_parts.append(f"tokens/s mean={token_stats['mean']:.3f}")
if encode_stats["mean"] is not None:
notes_parts.append(f"encode mean={encode_stats['mean']:.2f} ms")
if decode_stats["mean"] is not None:
notes_parts.append(f"decode mean={decode_stats['mean']:.2f} ms")
if reference_present:
notes_parts.append(f"wer median={fmt(wer_stats['median'], '', 4)}")
notes_parts.append(f"cer median={fmt(cer_stats['median'], '', 4)}")
else:
notes_parts.append("reference missing")
correctness_pass: Optional[bool]
if not reference_present:
correctness_pass = False if args.enforce_correctness else None
else:
correctness_pass = (
wer_stats["median"] is not None
and cer_stats["median"] is not None
and wer_stats["median"] <= args.max_wer
and cer_stats["median"] <= args.max_cer
)
if correctness_pass is False:
overall_correctness_pass = False
notes = "; ".join(notes_parts)
summary_rows.append(
{
"variant": variant,
"model": model,
"audio_key": audio_key,
"audio_length_s": audio_length_s,
"runs": len(rows),
"model_load_ms": load_stats,
"first_inference_latency_s": first_stats,
"wall_clock_runtime_s": runtime_stats,
"full_runtime_ms": full_runtime_stats,
"throughput_audio_seconds_per_second": throughput_stats,
"tokens_per_second": token_stats,
"encode_ms": encode_stats,
"decode_ms": decode_stats,
"wer": wer_stats,
"cer": cer_stats,
"reference_present": reference_present,
"correctness_pass": correctness_pass,
"notes": notes,
}
)
md_lines.append(
"| "
+ " | ".join(
[
variant,
model,
f"{audio_length_s:.3f}s",
str(len(rows)),
fmt(load_stats["mean"], " ms", decimals=2),
fmt(first_stats["mean"], " s", decimals=3),
fmt(runtime_stats["median"], " s", decimals=3),
fmt(throughput_stats["mean"], " audio-s/s", decimals=3),
fmt(runtime_stats["std_dev"], " s", decimals=3),
notes,
]
)
+ " |"
)
summary_csv_path = run_dir / "summary.csv"
with summary_csv_path.open("w", newline="", encoding="utf-8") as f:
fieldnames = [
"variant",
"model",
"audio_key",
"audio_length_s",
"runs",
"init_mean_ms",
"first_inference_mean_s",
"runtime_median_s",
"throughput_mean_audio_s_per_s",
"runtime_std_dev_s",
"reference_present",
"wer_median",
"cer_median",
"max_wer",
"max_cer",
"correctness_pass",
"notes",
]
writer = csv.DictWriter(f, fieldnames=fieldnames)
writer.writeheader()
for row in summary_rows:
writer.writerow(
{
"variant": row["variant"],
"model": row["model"],
"audio_key": row["audio_key"],
"audio_length_s": row["audio_length_s"],
"runs": row["runs"],
"init_mean_ms": row["model_load_ms"]["mean"],
"first_inference_mean_s": row["first_inference_latency_s"]["mean"],
"runtime_median_s": row["wall_clock_runtime_s"]["median"],
"throughput_mean_audio_s_per_s": row["throughput_audio_seconds_per_second"]["mean"],
"runtime_std_dev_s": row["wall_clock_runtime_s"]["std_dev"],
"reference_present": row["reference_present"],
"wer_median": row["wer"]["median"],
"cer_median": row["cer"]["median"],
"max_wer": args.max_wer,
"max_cer": args.max_cer,
"correctness_pass": row["correctness_pass"],
"notes": row["notes"],
}
)
summary_json_path = run_dir / "summary.json"
summary_json_path.write_text(json.dumps(summary_rows, indent=2) + "\n", encoding="utf-8")
correctness_json_path = run_dir / "correctness.json"
correctness_report = {
"refs_dir": str(refs_dir),
"max_wer": args.max_wer,
"max_cer": args.max_cer,
"enforce_correctness": args.enforce_correctness,
"missing_references": missing_references,
"overall_correctness_pass": overall_correctness_pass and (
not args.enforce_correctness or len(missing_references) == 0
),
"audios": [
{
"audio_key": row["audio_key"],
"reference_present": row["reference_present"],
"wer_median": row["wer"]["median"],
"cer_median": row["cer"]["median"],
"correctness_pass": row["correctness_pass"],
}
for row in summary_rows
],
}
correctness_json_path.write_text(json.dumps(correctness_report, indent=2) + "\n", encoding="utf-8")
summary_md_path = run_dir / "summary.md"
summary_md_path.write_text("\n".join(md_lines) + "\n", encoding="utf-8")
print(f"Wrote: {runs_csv_path}")
print(f"Wrote: {summary_csv_path}")
print(f"Wrote: {summary_json_path}")
print(f"Wrote: {summary_md_path}")
print(f"Wrote: {correctness_json_path}")
if args.enforce_correctness:
if missing_references:
print(
"Correctness gate failed: missing references for "
+ ", ".join(sorted(missing_references))
)
return 3
if not overall_correctness_pass:
print(
f"Correctness gate failed: one or more audios exceeded max WER={args.max_wer} or max CER={args.max_cer}"
)
return 4
return 0
if __name__ == "__main__":
raise SystemExit(main())

23
benchmark/references/README.md Normal file
View File

@ -0,0 +1,23 @@
# Reference Transcripts
This directory contains canonical reference transcripts used by the benchmark correctness gate.
Expected files:
- `short.txt`
- `medium.txt`
- `long.txt`
How they are used:
- `benchmark/parse_results.py` extracts transcript text from each measured run log.
- Text is normalized (case, punctuation, spacing).
- WER and CER are computed against these reference files.
- `benchmark/bench.sh` enforces correctness thresholds by default:
- `MAX_WER=0.02`
- `MAX_CER=0.02`
Notes:
- Keep references fixed once baseline is established.
- If audio inputs change, regenerate references intentionally and document why.

1
benchmark/references/long.txt Normal file
View File

File diff suppressed because one or more lines are too long

1
benchmark/references/medium.txt Normal file
View File

@ -0,0 +1 @@
"The Town Hose Story" As told at the Golden Inn. The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part. It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick. For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men. The Town Hose Story was a very strange and forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy." But the following night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest. Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mist. After weaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record. Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin. End of footnote. For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn. Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put and which are duly answered at the time. Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn. She was somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hole than common. They supposed a swordfish had stabbed her, gentlemen. But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous, though indeed they could not find it after searching the hole as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals. But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased. So much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired. Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that it was a good thing.

1
benchmark/references/short.txt Normal file
View File

@ -0,0 +1 @@
"The Town Hose Story"

1
benchmark/results/.gitkeep Normal file
View File

@ -0,0 +1 @@

62
benchmark/results/20260308_211026_metal-baseline/config.json Normal file
View File

@ -0,0 +1,62 @@
{
"audio": {
"long": {
"duration_s": 1800.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav",
"sha256": "fd586cbcf50f12d8eca4908bd38a3cd71a0c146992923d28259376aa0e6f4367"
},
"medium": {
"duration_s": 300.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav",
"sha256": "a918fe0de597908c917d44c4bfc54cfbfad3dbf79a763cdc558b0dbd95fe2aca"
},
"short": {
"duration_s": 30.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"sha256": "bf6376be2d6749925b9fb53a194a55cd0330329f0425705e5b4e67a0dd1ebd3a"
}
},
"cli": {
"args": [
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"binary": "/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"processors": 1,
"threads": 8
},
"created_at_utc": "2026-03-08T19:10:26.830615+00:00",
"environment": {
"clang_version": "Apple clang version 17.0.0 (clang-1700.6.4.2)\nTarget: arm64-apple-darwin25.3.0\nThread model: posix\nInstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin",
"cmake_version": "cmake version 4.2.3\n\nCMake suite maintained and supported by Kitware (kitware.com/cmake).",
"cpu_brand_string": "Apple M2",
"git_commit": "30c5194c9691e4e9a98b3dea9f19727397d3f46e",
"git_short_commit": "30c5194",
"hw_memsize": "17179869184",
"python_version": "3.13.2",
"sw_vers": "ProductName:\t\tmacOS\nProductVersion:\t\t26.3.1\nBuildVersion:\t\t25D2128",
"uname": "Darwin Shais-MacBook-Air.local 25.3.0 Darwin Kernel Version 25.3.0: Wed Jan 28 20:56:34 PST 2026; root:xnu-12377.91.3~2/RELEASE_ARM64_T8112 arm64",
"xcodebuild_version": "Xcode 26.3\nBuild version 17C529"
},
"model": {
"abs_path": "/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"rel_path": "models/ggml-small.en.bin",
"sha256": "c6138d6d58ecc8322097e0f987c32f1be8bb0a18532a3f88f734d1bbf9c41e5d"
},
"run_policy": {
"measured_runs": 5,
"sequential_execution": true,
"warmup_runs": 1
},
"variant": "metal-baseline"
}

31
benchmark/results/20260308_211026_metal-baseline/correctness.json Normal file
View File

@ -0,0 +1,31 @@
{
"refs_dir": "/Users/shaihi/Downloads/whisper_optimization/benchmark/references",
"max_wer": 0.02,
"max_cer": 0.02,
"enforce_correctness": true,
"missing_references": [],
"overall_correctness_pass": true,
"audios": [
{
"audio_key": "short",
"reference_present": true,
"wer_median": 0.0,
"cer_median": 0.0,
"correctness_pass": true
},
{
"audio_key": "medium",
"reference_present": true,
"wer_median": 0.0,
"cer_median": 0.0,
"correctness_pass": true
},
{
"audio_key": "long",
"reference_present": true,
"wer_median": 0.0,
"cer_median": 0.0,
"correctness_pass": true
}
]
}

190
benchmark/results/20260308_211026_metal-baseline/raw/long/run_01.log Normal file
View File

@ -0,0 +1,190 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.010 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav' (28800000 samples, 1800.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:40.000] The Town Hose Story was a very popular circumstance, with its own particular accompaniments forming what may be called the "secret part" of the tragedy about to be narrated, never reach the ears of Captain Ahab or his mates.
[00:01:41.000 --> 00:01:58.000] The Town Hose Story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."
[00:01:59.000 --> 00:02:28.000] One night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest. Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mess.
[00:02:29.000 --> 00:02:41.000] Interweaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record.
[00:02:41.000 --> 00:02:55.000] Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin.
[00:02:56.000 --> 00:03:13.000] For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn.
[00:03:14.000 --> 00:03:27.000] Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put, and which are duly answered at the time.
[00:03:28.000 --> 00:03:47.000] Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn.
[00:03:48.000 --> 00:03:59.000] Somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hull than common.
[00:03:59.000 --> 00:04:17.000] They supposed a swordfish had stabbed her, gentlemen. But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous,
[00:04:18.000 --> 00:04:31.000] indeed they could not find it after searching the hold as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals.
[00:04:32.000 --> 00:04:52.000] But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased, so much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired.
[00:04:53.000 --> 00:05:13.000] Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that his ship would found her by the way, because his pumps were of the best, and being periodically relieved at them, those six and thirty men of his could easily keep the ship free, never mind if the leak should double on her.
[00:05:14.000 --> 00:05:40.000] In truth, well nigh the whole of this passage being attended by very prosperous breezes, the town-hole had all but certainly arrived in perfect safety at her port without the occurrence of the least fatality, had it not been for the brutal overbearing of Radney, the mate of Vineyarder, and the bitterly provoked vengeance of Steelkilt, a Lakeman in Desperado from Buffalo.
[00:05:41.000 --> 00:05:51.000] "Lakeman, Buffalo! Pray what is a Lakeman, and where is Buffalo?" said Don Sebastian, rising in his swinging mat of grass.
[00:05:51.000 --> 00:06:00.000] On the eastern shore of our Lake Erie, Don, but I crave your courtesy, may be you shall soon hear further of all that.
[00:06:01.000 --> 00:06:24.000] Now, gentlemen, in square sail-brigs and three-masted ships, well nigh as large and stout as any that ever sailed out of your old Caiao to far Manila, this Lakeman, in the landlocked heart of our America, had yet been nurtured by all those agrarian, free-booting impressions popularly connected with the open ocean.
[00:06:25.000 --> 00:06:51.000] Where interflowing aggregate those grand freshwater seas of ours, Erie and Ontario and Huron and Superior and Michigan, possess an ocean-like expansiveness with many of the ocean's noblest traits, with many of its rimmed varieties of races and of climes, they contain round archipelagos of romantic isles, even as the Polynesian waters do.
[00:06:52.000 --> 00:06:57.000] Each part are shored by two great contrasting nations as the Atlantic is.
[00:06:57.000 --> 00:07:06.000] They furnish long maritime approaches to our numerous territorial colonies from the east, dotted all round their banks.
[00:07:06.000 --> 00:07:13.000] Here and there are frowned upon by batteries and by the goat-like craggy guns of lofty Mackinac.
[00:07:14.000 --> 00:07:26.000] They have heard the fleet thunderings of naval victories. At intervals they yield their beaches to wild barbarians whose red painted faces flash from out their peltry wigwams.
[00:07:26.000 --> 00:07:42.000] For leagues and leagues are flanked by ancient and unentered forests, where the gaunt pines stand like serried lines of kings in Gothic genealogies, those same woods harboring wild, Africa beasts of prey,
[00:07:43.000 --> 00:07:47.000] and the creatures whose exported furs give robes to tartar emperors.
[00:07:47.000 --> 00:07:53.000] They mirror the paved capitals of Buffalo and Cleveland, as well as Winnebago villages.
[00:07:53.000 --> 00:08:01.000] They float alike the full-rigged merchant ship, the armed cruiser of the state, the steamer and the beach canoe.
[00:08:01.000 --> 00:08:08.000] They are swept by borean and dis-masting blasts as direful as any that lash the salted wave.
[00:08:09.000 --> 00:08:17.000] They know what shipwrecks are for out of sight of land, however inland they have drowned full many a midnight ship with all its shrieking crew.
[00:08:17.000 --> 00:08:28.000] Thus, gentlemen, though an inlander, steel-kilt was wild ocean-born and wild ocean-nurtured, as much of an audacious mariner as any.
[00:08:29.000 --> 00:08:43.000] And for Radney, though in his infancy he may have laid him down on the lone Nantucket beach, to nurse at his maternal sea, though in afterlife he had long followed our austere Atlantic and your contemplative Pacific,
[00:08:43.000 --> 00:08:53.000] yet he was quite as vengeful and full of social quarrel as the backwoods seamen, fresh from the latitudes of buckhorn-handled buoy knives.
[00:08:54.000 --> 00:09:11.000] Yet was this Nantucketer a man with some good-hearted traits, and this lakeman, a mariner, who though a sort of devil indeed, might yet by inflexible firmness only tempered by that common decency of human recognition which is the meanest slave's right,
[00:09:12.000 --> 00:09:27.000] as treated, this steel-kilt had long been retained harmless and docile. At all events he had proved so thus far, but Radney was doomed and made mad, and steel-kilt, but gentlemen you shall hear.
[00:09:28.000 --> 00:09:42.000] It was not more than a day or two at the furthest after pointing her prow for her island haven that the town-hose leak seemed again increasing, but only so as to require an hour or more at the pumps every day.
[00:09:43.000 --> 00:09:58.000] Let us know that in a settled and civilized ocean like our Atlantic, for example, some skippers think little of pumping their whole way across it, though of a still sleepy night should the officer of the deck happen to forget his duty in that respect,
[00:09:58.000 --> 00:10:07.000] the probability would be that he and his shipmates would never again remember it, on account of all hands gently subsiding to the bottom.
[00:10:08.000 --> 00:10:29.000] Whether in the solitary and savage seas far from you to the westward, gentlemen, is it altogether unusual for ships to keep clanging at their pump handles in full chorus even for a voyage of considerable length, that is if it lie along a tolerably accessible coast, or if any other reasonable retreat is afforded them.
[00:10:30.000 --> 00:10:40.000] It is only when a leaky vessel is in some very out-of-the-way part of those waters, some really landless latitude, that her captain begins to feel a little anxious.
[00:10:40.000 --> 00:10:55.000] Much this way had it been with the town-hoe, so when her leak was found gaining once more, there was in truth some small concern manifested by several of her company, especially by Radney the mate.
[00:10:56.000 --> 00:11:03.000] He commanded the upper sails to be well hoisted, sheeted home anew, and every way expanded to the breeze.
[00:11:03.000 --> 00:11:20.000] Now this Radney, I suppose, was as little of a coward, and as little inclined to any sort of nervous apprehensiveness touching his own person, as any fearless, unthinking creature on land or on sea that you can conveniently imagine, gentlemen.
[00:11:21.000 --> 00:11:31.000] Therefore when he betrayed this solicitude about the safety of the ship, some of the seamen declared that it was only on account of his being a part-owner in her.
[00:11:31.000 --> 00:11:45.000] So when they were working that evening at the pumps, there was on this head no small game-someness slyly going on among them, as they stood with their feet continually overflowed by the rippling clear water.
[00:11:46.000 --> 00:11:56.000] As any mountain-spring, gentlemen, that bubbling from the pumps ran across the deck and poured itself out in steady spouts at the Lee Scupper-holes.
[00:11:56.000 --> 00:12:14.000] Now as you well know, it is not seldom the case in this conventional world of ours, watery or otherwise, that when a person placed in command over his fellow men finds one of them to be very significantly his superior in general pride of manhood,
[00:12:15.000 --> 00:12:28.000] it may against that man he conceives an unconquerable dislike and bitterness, and if he have a chance, he will pull down and pulverize that subaltern's tower and make a little heap of dust of it.
[00:12:29.000 --> 00:12:54.000] Be this conceit of mine as it may, gentlemen, at all events steel-kilt was a tall and noble animal with a head like a Roman, and a flowing golden beard like the tasseled housings of your last viceroy's snorting charger, and a brain and a heart and a soul in him, gentlemen, which had made steel-kilt Charlemagne, had he been born son to Charlemagne's father.
[00:12:55.000 --> 00:13:06.000] In the maddening, the mate was as ugly as a mule, yet as hardy, as stubborn, as malicious. He did not love steel-kilt, and steel-kilt knew it.
[00:13:06.000 --> 00:13:18.000] As spying the mate drawing near as he was toiling at the pump with the rest, the lakeman affected not to notice him, but anod went on with his gay banterings.
[00:13:19.000 --> 00:13:47.000] "Aye, aye, my merry lads, it's a lively leak, this. Hold a cannekin, one of you, and let's have a taste. By the Lord, it's worth bottling. I tell you what, men, old Rad's investment must go for it. He had best cut away his part of the hull and tow it home. The fact is, boys, that swordfish only began the job. He's come back again with a gang of ship carpenters, sawfish and filefish and whatnot.
[00:13:48.000 --> 00:14:17.000] "The whole posse of 'em are now hard at work, cutting and slashing the bottom, making improvements, I suppose. If old Rad were here now, I'd tell him to jump overboard and scatter 'em. They're playing the devil with his estate, I can tell him. But he's a simple old soul, Rad, and a beauty too. Boys, they say the rest of his property is invested in looking-glasses. I wonder if he'd give a poor devil like me the model of his nose."
[00:14:18.000 --> 00:14:28.000] "Damn your eyes! What's that pump stopping for?" roared Radney, pretending not to have heard the sailors talk. Thunder away at it.
[00:14:28.000 --> 00:14:34.000] "Aye, aye, sir," said Steelkilt. "Mary is a cricket. Lively, boys, lively now."
[00:14:34.000 --> 00:14:49.000] And with that the pump clanged like fifty fire engines, the men tossed their hats off to it, and ere long that peculiar gasping of the lungs was heard, which denotes the fullest tensions of life's utmost energies.
[00:14:49.000 --> 00:15:06.000] Quitting the pump at last with the rest of his band, the lakeman went forward all panting, and sat himself down on the windlass, his face fiery red, his eyes bloodshot, and wiping the profuse sweat from his brow.
[00:15:06.000 --> 00:15:33.000] Now what cousining fiend it was, gentlemen, that possessed Radney to meddle with such a man in that corporeally exasperated state, I know not. But so it happened. Intolerably striding along the deck, the mate commanded him to get a broom and sweep down the planks, and also a shovel, and remove some offensive matters consequent upon allowing a pig to run at large.
[00:15:33.000 --> 00:15:45.000] Now, gentlemen, sweeping a ship's deck at sea is a piece of household work, which in all times but raging gales is regularly attended to every evening.
[00:15:45.000 --> 00:16:03.000] It has been known to be done in the case of ships actually foundering at the time. Such, gentlemen, is the inflexibility of sea usages, and the instinctive love of neatness in seaman, some of whom would not willingly drown without first washing their faces.
[00:16:03.000 --> 00:16:18.000] But in all vessels this broom business is the prescriptive province of the boys, if boys there be aboard. Besides, it was the stronger men in the town hoe that had been divided into gangs, taking turns at the pumps.
[00:16:18.000 --> 00:16:35.000] And, being the most athletic seaman of them all, Steelkilt had been regularly assigned captain of one of the gangs. Consequently he should have been freed from any trivial business not connected with truly nautical duties, such being the case with his comrades.
[00:16:35.000 --> 00:16:43.000] I mention all of these particulars so that you may understand exactly how this affair stood between the two men.
[00:16:43.000 --> 00:16:53.000] But there was more than this. The order about the shovel was almost as plainly meant to sting and insult Steelkilt as though Radney had spat in his face.
[00:16:53.000 --> 00:17:05.000] Any man who has gone sailor in a whale ship will understand this, and all this and doubtless much more the lakeman fully comprehended when the mate uttered his command.
[00:17:05.000 --> 00:17:19.000] But as he sat still for a moment, and as he steadfastly looked into the mate's malignant eye and perceived the stacks of powder casks heaped up in him, and the slow match silently burning along towards them,
[00:17:19.000 --> 00:17:41.000] as he instinctively saw all this, that strange forbearance and unwillingness to stir up the deeper passionateness in any already-ireful being, a repugnance most felt when felt at all by really valiant men even when aggrieved, this nameless phantom-feeling gentleman stole over Steelkilt.
[00:17:41.000 --> 00:17:55.000] Therefore, in his ordinary tone, only a little broken by the bodily exhaustion he was temporarily in, he answered him saying that sweeping the deck was not his business, and he would not do it.
[00:17:55.000 --> 00:18:07.000] And then, without at all alluding to the shovel, he pointed to three lads as the customary sweepers, who, not being billeted at the pumps, had done little or nothing all day.
[00:18:07.000 --> 00:18:25.000] To this, Radney replied with an oath, in a most domineering and outrageous manner unconditionally reiterating his command, meanwhile advancing upon these still-seated lakemen with an uplifted Cooper's club hammer which he had snatched from a cask nearby.
[00:18:25.000 --> 00:18:38.000] Heeded and irritated as he was by his spasmodic toil at the pumps, for all his first nameless feeling of forbearance, the sweating Steelkilt could but ill-brooke this bearing in the mate.
[00:18:38.000 --> 00:18:55.000] But somehow, still smothering the conflagration within him, without speaking, he remained doggedly rooted to his seat, till at last the incensed Radney shook the hammer within a few inches of his face, furiously commanding him to do his bidding.
[00:18:55.000 --> 00:19:08.000] Steelkilt rose, and slowly retreating round the windlass, steadily followed by the mate with his menacing hammer, deliberately repeated his intention not to obey.
[00:19:08.000 --> 00:19:20.000] Seeing, however, that his forbearance had not the slightest effect, by an awful and unspeakable intimation with his twisted hand, he warned off the foolish and infatuated man.
[00:19:20.000 --> 00:19:40.000] But it was to no purpose, and in this way the two went once slowly round the windlass, when resolved at last no longer to retreat, bethinking him that he had now foreborn as much as comported with his humor, the lakemen paused on the hatches and thus spoke to the officer.
[00:19:40.000 --> 00:20:01.000] "Mr. Radney, I will not obey you. Take that hammer away, or look to yourself." But the predestinated mate, coming still closer to him, where the lakemen stood fixed, now shook the heavy hammer within an inch of his teeth, meanwhile repeating a string of insufferable maledictions,
[00:20:01.000 --> 00:20:22.000] retreating not the thousandth part of an inch, stabbing him in the eye with the unflinching panyard of his glance, Steelkilt, clenching his right hand behind him and creepingly drawing it back, told his persecutor that if the hammer but grazed his cheek, he, Steelkilt, would murder him.
[00:20:22.000 --> 00:20:37.000] But, gentlemen, the fool had been branded for the slaughter by the gods. Immediately the hammer touched his cheek, the next instant the lower jaw of the mate was stove in his head, he fell on the hatch spouting blood like a whale.
[00:20:37.000 --> 00:21:01.000] Ere the cry could go aft, Steelkilt was shaking one of the backstays leading far aloft, to where two of his comrades were standing their mastheads. They were both canalers. "Canalers!" cried Don Pedro. "We have seen many whale ships in our harbors, but never heard of your canalers. Pardon. Who and what are they?"
[00:21:01.000 --> 00:21:08.000] "Canalers, Don, are the boatmen belonging to our grand eerie canal. You must have heard of it."
[00:21:08.000 --> 00:21:19.000] "Nay, señor. Hereabouts in this dull, warm, most lazy and hereditary land, we know but little of your vigorous north."
[00:21:19.000 --> 00:21:33.000] "Aye? Well, then, Don, refill my cup. Your chicha's very fine, and ere proceeding further I will tell you what our canalers are, for such information may throw sidelight upon my story."
[00:21:33.000 --> 00:21:51.000] For 360 miles, gentlemen, through the entire breadth of the state of New York, through numerous populous cities and most thriving villages, through long, dismal, uninhabited swamps and affluent cultivated fields, unrivaled for fertility,
[00:21:51.000 --> 00:22:04.000] by billiard-room and bar-room, through the holy of holies of great forests, on Roman arches over Indian rivers, through sun and shade, by happy hearts are broken,
[00:22:04.000 --> 00:22:16.000] through all the wide contrasting scenery of those noble Mohawk counties, and especially by rows of snow-white chapels, whose spires stand almost like milestones,
[00:22:16.000 --> 00:22:35.000] flows one continual stream of venetianly corrupt and often lawless life. There's your true Ashanti, gentlemen. There howl your pagans, where you ever find them next door to you, under the long-flung shadow and the snug patronizing lee of churches.
[00:22:35.000 --> 00:22:49.000] For by some curious fatality, as it is often noted of your metropolitan free-booters that they ever encamp around the halls of justice, so sinners, gentlemen, most abound in holiest vicinities.
[00:22:49.000 --> 00:22:58.000] "Is that a friar passing?" said Don Pedro, looking downwards into the crowded piazza with humorous concern.
[00:22:58.000 --> 00:23:09.000] "Well, for our northern friend, dim Isabella's inquisition wanes in Lima," laughed Don Sebastian. "Proceeds in yore."
[00:23:09.000 --> 00:23:28.000] "A moment! Pardon!" cried another of the company. "In the name of all us limis, I but desire to express to you, Sir Sailor, that we have by no means overlooked your delicacy in not substituting present Lima for distant Venice in your corrupt comparison.
[00:23:28.000 --> 00:23:43.000] Oh, no, do not bow and look surprised. You know the proverb along all this coast, 'Corrupt as Lima.' It but bears out your saying, too, churches more plentiful than billiard tables and forever open.
[00:23:43.000 --> 00:24:00.000] And corrupt as Lima! So too Venice, I have been there, the holy city of the blessed evangelist St. Mark. St. Dominic, purge it! Your cup! Thanks, here I refill. Now, you pour out again."
[00:24:00.000 --> 00:24:25.000] Freely depicted in his own vocation, gentlemen, the canaler would make a fine dramatic hero, so abundantly and picturesquely wicked is he. Like Mark Antony, for days and days along his green-turfed flowery Nile, he indolently floats, openly toying with his red-cheeked Cleopatra, ripening his apricot thigh upon the sunny deck.
[00:24:25.000 --> 00:24:37.000] But ashore all this effeminacy is dashed. The brigandish guise which the canaler so proudly sports, his slouched and gaily-ribboned hat betoken his grand features.
[00:24:37.000 --> 00:24:47.000] A terror to the smiling innocence of the villages through which he floats, his swart visage and bold swagger, are not unshunned in cities.
[00:24:47.000 --> 00:25:08.000] Once a vagabond on his own canal, I have received good turns from one of these canalers. I thanked him heartily. Would Fane not be ungrateful? But it is often one of the prime redeeming qualities of your man of violence, that at times he is as stiff an arm to back a poor stranger in a strait as to plunder a wealthy one.
[00:25:08.000 --> 00:25:28.000] In some, gentlemen, what the wildness of this canal life is, is emphatically evinced by this. That our wild whale fishery contains so many of its most finished graduates, and that scarce any race of mankind, except Sydney men, are so much distrusted by our whaling captains.
[00:25:28.000 --> 00:25:50.000] Nor does it at all diminish the curiousness of this matter, that to many thousands of our rural boys and young men born along its line, the probationary life of the grand canal furnishes the sole transition between quietly reaping in a Christian cornfield and recklessly plowing the waters of the most barbaric seas.
[00:25:50.000 --> 00:26:12.000] "I see, I see," impetuously exclaimed Don Pedro, spilling his chicha upon his silvery ruffles. "No need to travel. The world's one Lima. I had thought now that at your temperate north, the generations were cold and holy as the hills. But the story."
[00:26:12.000 --> 00:26:26.000] I left off, gentlemen, where the lakeman shook the backstay. Hardly had he done so, when he was surrounded by the three junior mates and the four harpooners, who all crowded him to the deck.
[00:26:26.000 --> 00:26:35.000] But sliding down the ropes like baleful comets, the two canalers rushed into the uproar, and sought to drag their man out of it toward the folksil.
[00:26:35.000 --> 00:26:53.000] Others of the sailors joined with them in this attempt, and a twisted turmoil ensued. While standing out of harm's way, the valiant captain danced up and down with a whale pike, calling upon his officers to manhandle that atrocious scoundrel and smoke him along to the quarter deck.
[00:26:53.000 --> 00:27:07.000] At intervals he ran close up to the revolving border of the confusion, and prying into the heart of it with his pike, sought to prick out the object of his resentment. But steelkilt and his desperados were too much for them all.
[00:27:07.000 --> 00:27:22.000] They succeeded in gaining the folksil deck, where, hastily slewing about three or four large casks in a line with the windlass, these sea Parisians entrenched themselves behind the barricade.
[00:27:22.000 --> 00:27:34.000] "Come out of that, ye pirates!" roared the captain, now menacing them with a pistol in each hand, just brought to him by the steward. "Come out of that, ye cutthroats!"
[00:27:34.000 --> 00:28:01.000] Steelkilt leaped on the barricade, and, striding up and down there, defied the worst the pistols could do, but gave the captain to understand distinctly that his, Steelkilt's death, would be the signal for a murderous mutiny on the part of all hands. Fearing in his heart, lest this might prove but too true, the captain a little desisted, but still commanded the insurgents instantly to return to their duty.
[00:28:01.000 --> 00:28:19.000] "Will ye promise not to touch us if we do?" demanded their ringleader. "Turn two, turn two! I make no promise. To your duty! Do ye want to sink the ship by knocking off at a time like this? Turn two!" And he once more raised a pistol.
[00:28:19.000 --> 00:28:36.000] "Sink the ship!" cried Steelkilt. "Aye, let her sink! Not a man of us turns two unless ye swear not to raise a rope-yard against us. What say ye men?" turning to his comrades. A fierce cheer was their response.
[00:28:36.000 --> 00:29:04.000] The lakeman now patrolled the barricade, all the while keeping his eye on the captain, and jerking out such sentences as these. "It's not our fault! We didn't want it! I told him to take his hammer away! It was boys' business! He might have known me before this! I told him not to prick the buffalo! I believe I have broken a finger here against his cursed jaw! Ain't those mincing knives down in the fuxal there, men?
[00:29:04.000 --> 00:29:20.000] Look to those hand spikes, my hearties. Captain, by God, look to yourself. Say the word and don't be a fool. Forget it all. We are ready to turn two. Treat us decently and we're your men, but we won't be flogged."
[00:29:20.000 --> 00:29:49.000] "Turn two! I make no promises! Turn two, I say!" "Look ye now!" cried the lakeman, flinging out his arms towards him. "There are a few of us here, and I am one of them, who have shipped for the cruise, de' you see. Now, as you well know, sir, we can claim our discharge as soon as the anchor is down. So we don't want a row! It's not our interest! We want to be peaceable! We are ready to work, but we won't be flogged!"
[00:29:49.000 --> 00:30:00.000] "Turn two!" roared the captain. Steelkilt glanced round him a moment and then said, "I tell you what it is now, captain."
whisper_print_timings: load time = 280.92 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 1002.14 ms
whisper_print_timings: sample time = 3432.77 ms / 1 runs ( 3432.77 ms per run)
whisper_print_timings: encode time = 46745.75 ms / 86 runs ( 543.56 ms per run)
whisper_print_timings: decode time = 58407.11 ms / 5925 runs ( 9.86 ms per run)
whisper_print_timings: batchd time = 19.53 ms / 11 runs ( 1.78 ms per run)
whisper_print_timings: prompt time = 7377.74 ms / 18584 runs ( 0.40 ms per run)
whisper_print_timings: total time = 119378.98 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/long/run_01.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 1800.0,
"audio_key": "long",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:25:48.253694+00:00",
"exit_code": 0,
"first_inference_latency_s": 1.9347949999937555,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_01.log",
"run_index": 1,
"run_kind": "measured",
"start_utc": "2026-03-08T19:23:48.357952+00:00",
"wall_clock_runtime_s": 119.89503466699534
}

1
benchmark/results/20260308_211026_metal-baseline/raw/long/run_01.transcript.txt Normal file
View File

File diff suppressed because one or more lines are too long

190
benchmark/results/20260308_211026_metal-baseline/raw/long/run_02.log Normal file
View File

@ -0,0 +1,190 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.010 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav' (28800000 samples, 1800.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:40.000] The Town Hose Story was a very popular circumstance, with its own particular accompaniments forming what may be called the "secret part" of the tragedy about to be narrated, never reach the ears of Captain Ahab or his mates.
[00:01:41.000 --> 00:01:58.000] The Town Hose Story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."
[00:01:59.000 --> 00:02:28.000] One night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest. Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mess.
[00:02:29.000 --> 00:02:41.000] Interweaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record.
[00:02:41.000 --> 00:02:55.000] Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin.
[00:02:56.000 --> 00:03:13.000] For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn.
[00:03:14.000 --> 00:03:27.000] Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put, and which are duly answered at the time.
[00:03:28.000 --> 00:03:47.000] Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn.
[00:03:48.000 --> 00:03:59.000] Somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hull than common.
[00:03:59.000 --> 00:04:17.000] They supposed a swordfish had stabbed her, gentlemen. But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous,
[00:04:18.000 --> 00:04:31.000] indeed they could not find it after searching the hold as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals.
[00:04:32.000 --> 00:04:52.000] But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased, so much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired.
[00:04:53.000 --> 00:05:13.000] Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that his ship would found her by the way, because his pumps were of the best, and being periodically relieved at them, those six and thirty men of his could easily keep the ship free, never mind if the leak should double on her.
[00:05:14.000 --> 00:05:40.000] In truth, well nigh the whole of this passage being attended by very prosperous breezes, the town-hole had all but certainly arrived in perfect safety at her port without the occurrence of the least fatality, had it not been for the brutal overbearing of Radney, the mate of Vineyarder, and the bitterly provoked vengeance of Steelkilt, a Lakeman in Desperado from Buffalo.
[00:05:41.000 --> 00:05:51.000] "Lakeman, Buffalo! Pray what is a Lakeman, and where is Buffalo?" said Don Sebastian, rising in his swinging mat of grass.
[00:05:51.000 --> 00:06:00.000] On the eastern shore of our Lake Erie, Don, but I crave your courtesy, may be you shall soon hear further of all that.
[00:06:01.000 --> 00:06:24.000] Now, gentlemen, in square sail-brigs and three-masted ships, well nigh as large and stout as any that ever sailed out of your old Caiao to far Manila, this Lakeman, in the landlocked heart of our America, had yet been nurtured by all those agrarian, free-booting impressions popularly connected with the open ocean.
[00:06:25.000 --> 00:06:51.000] Where interflowing aggregate those grand freshwater seas of ours, Erie and Ontario and Huron and Superior and Michigan, possess an ocean-like expansiveness with many of the ocean's noblest traits, with many of its rimmed varieties of races and of climes, they contain round archipelagos of romantic isles, even as the Polynesian waters do.
[00:06:52.000 --> 00:06:57.000] Each part are shored by two great contrasting nations as the Atlantic is.
[00:06:57.000 --> 00:07:06.000] They furnish long maritime approaches to our numerous territorial colonies from the east, dotted all round their banks.
[00:07:06.000 --> 00:07:13.000] Here and there are frowned upon by batteries and by the goat-like craggy guns of lofty Mackinac.
[00:07:14.000 --> 00:07:26.000] They have heard the fleet thunderings of naval victories. At intervals they yield their beaches to wild barbarians whose red painted faces flash from out their peltry wigwams.
[00:07:26.000 --> 00:07:42.000] For leagues and leagues are flanked by ancient and unentered forests, where the gaunt pines stand like serried lines of kings in Gothic genealogies, those same woods harboring wild, Africa beasts of prey,
[00:07:43.000 --> 00:07:47.000] and the creatures whose exported furs give robes to tartar emperors.
[00:07:47.000 --> 00:07:53.000] They mirror the paved capitals of Buffalo and Cleveland, as well as Winnebago villages.
[00:07:53.000 --> 00:08:01.000] They float alike the full-rigged merchant ship, the armed cruiser of the state, the steamer and the beach canoe.
[00:08:01.000 --> 00:08:08.000] They are swept by borean and dis-masting blasts as direful as any that lash the salted wave.
[00:08:09.000 --> 00:08:17.000] They know what shipwrecks are for out of sight of land, however inland they have drowned full many a midnight ship with all its shrieking crew.
[00:08:17.000 --> 00:08:28.000] Thus, gentlemen, though an inlander, steel-kilt was wild ocean-born and wild ocean-nurtured, as much of an audacious mariner as any.
[00:08:29.000 --> 00:08:43.000] And for Radney, though in his infancy he may have laid him down on the lone Nantucket beach, to nurse at his maternal sea, though in afterlife he had long followed our austere Atlantic and your contemplative Pacific,
[00:08:43.000 --> 00:08:53.000] yet he was quite as vengeful and full of social quarrel as the backwoods seamen, fresh from the latitudes of buckhorn-handled buoy knives.
[00:08:54.000 --> 00:09:11.000] Yet was this Nantucketer a man with some good-hearted traits, and this lakeman, a mariner, who though a sort of devil indeed, might yet by inflexible firmness only tempered by that common decency of human recognition which is the meanest slave's right,
[00:09:12.000 --> 00:09:27.000] as treated, this steel-kilt had long been retained harmless and docile. At all events he had proved so thus far, but Radney was doomed and made mad, and steel-kilt, but gentlemen you shall hear.
[00:09:28.000 --> 00:09:42.000] It was not more than a day or two at the furthest after pointing her prow for her island haven that the town-hose leak seemed again increasing, but only so as to require an hour or more at the pumps every day.
[00:09:43.000 --> 00:09:58.000] Let us know that in a settled and civilized ocean like our Atlantic, for example, some skippers think little of pumping their whole way across it, though of a still sleepy night should the officer of the deck happen to forget his duty in that respect,
[00:09:58.000 --> 00:10:07.000] the probability would be that he and his shipmates would never again remember it, on account of all hands gently subsiding to the bottom.
[00:10:08.000 --> 00:10:29.000] Whether in the solitary and savage seas far from you to the westward, gentlemen, is it altogether unusual for ships to keep clanging at their pump handles in full chorus even for a voyage of considerable length, that is if it lie along a tolerably accessible coast, or if any other reasonable retreat is afforded them.
[00:10:30.000 --> 00:10:40.000] It is only when a leaky vessel is in some very out-of-the-way part of those waters, some really landless latitude, that her captain begins to feel a little anxious.
[00:10:40.000 --> 00:10:55.000] Much this way had it been with the town-hoe, so when her leak was found gaining once more, there was in truth some small concern manifested by several of her company, especially by Radney the mate.
[00:10:56.000 --> 00:11:03.000] He commanded the upper sails to be well hoisted, sheeted home anew, and every way expanded to the breeze.
[00:11:03.000 --> 00:11:20.000] Now this Radney, I suppose, was as little of a coward, and as little inclined to any sort of nervous apprehensiveness touching his own person, as any fearless, unthinking creature on land or on sea that you can conveniently imagine, gentlemen.
[00:11:21.000 --> 00:11:31.000] Therefore when he betrayed this solicitude about the safety of the ship, some of the seamen declared that it was only on account of his being a part-owner in her.
[00:11:31.000 --> 00:11:45.000] So when they were working that evening at the pumps, there was on this head no small game-someness slyly going on among them, as they stood with their feet continually overflowed by the rippling clear water.
[00:11:46.000 --> 00:11:56.000] As any mountain-spring, gentlemen, that bubbling from the pumps ran across the deck and poured itself out in steady spouts at the Lee Scupper-holes.
[00:11:56.000 --> 00:12:14.000] Now as you well know, it is not seldom the case in this conventional world of ours, watery or otherwise, that when a person placed in command over his fellow men finds one of them to be very significantly his superior in general pride of manhood,
[00:12:15.000 --> 00:12:28.000] it may against that man he conceives an unconquerable dislike and bitterness, and if he have a chance, he will pull down and pulverize that subaltern's tower and make a little heap of dust of it.
[00:12:29.000 --> 00:12:54.000] Be this conceit of mine as it may, gentlemen, at all events steel-kilt was a tall and noble animal with a head like a Roman, and a flowing golden beard like the tasseled housings of your last viceroy's snorting charger, and a brain and a heart and a soul in him, gentlemen, which had made steel-kilt Charlemagne, had he been born son to Charlemagne's father.
[00:12:55.000 --> 00:13:06.000] In the maddening, the mate was as ugly as a mule, yet as hardy, as stubborn, as malicious. He did not love steel-kilt, and steel-kilt knew it.
[00:13:06.000 --> 00:13:18.000] As spying the mate drawing near as he was toiling at the pump with the rest, the lakeman affected not to notice him, but anod went on with his gay banterings.
[00:13:19.000 --> 00:13:47.000] "Aye, aye, my merry lads, it's a lively leak, this. Hold a cannekin, one of you, and let's have a taste. By the Lord, it's worth bottling. I tell you what, men, old Rad's investment must go for it. He had best cut away his part of the hull and tow it home. The fact is, boys, that swordfish only began the job. He's come back again with a gang of ship carpenters, sawfish and filefish and whatnot.
[00:13:48.000 --> 00:14:17.000] "The whole posse of 'em are now hard at work, cutting and slashing the bottom, making improvements, I suppose. If old Rad were here now, I'd tell him to jump overboard and scatter 'em. They're playing the devil with his estate, I can tell him. But he's a simple old soul, Rad, and a beauty too. Boys, they say the rest of his property is invested in looking-glasses. I wonder if he'd give a poor devil like me the model of his nose."
[00:14:18.000 --> 00:14:28.000] "Damn your eyes! What's that pump stopping for?" roared Radney, pretending not to have heard the sailors talk. Thunder away at it.
[00:14:28.000 --> 00:14:34.000] "Aye, aye, sir," said Steelkilt. "Mary is a cricket. Lively, boys, lively now."
[00:14:34.000 --> 00:14:49.000] And with that the pump clanged like fifty fire engines, the men tossed their hats off to it, and ere long that peculiar gasping of the lungs was heard, which denotes the fullest tensions of life's utmost energies.
[00:14:49.000 --> 00:15:06.000] Quitting the pump at last with the rest of his band, the lakeman went forward all panting, and sat himself down on the windlass, his face fiery red, his eyes bloodshot, and wiping the profuse sweat from his brow.
[00:15:06.000 --> 00:15:33.000] Now what cousining fiend it was, gentlemen, that possessed Radney to meddle with such a man in that corporeally exasperated state, I know not. But so it happened. Intolerably striding along the deck, the mate commanded him to get a broom and sweep down the planks, and also a shovel, and remove some offensive matters consequent upon allowing a pig to run at large.
[00:15:33.000 --> 00:15:45.000] Now, gentlemen, sweeping a ship's deck at sea is a piece of household work, which in all times but raging gales is regularly attended to every evening.
[00:15:45.000 --> 00:16:03.000] It has been known to be done in the case of ships actually foundering at the time. Such, gentlemen, is the inflexibility of sea usages, and the instinctive love of neatness in seaman, some of whom would not willingly drown without first washing their faces.
[00:16:03.000 --> 00:16:18.000] But in all vessels this broom business is the prescriptive province of the boys, if boys there be aboard. Besides, it was the stronger men in the town hoe that had been divided into gangs, taking turns at the pumps.
[00:16:18.000 --> 00:16:35.000] And, being the most athletic seaman of them all, Steelkilt had been regularly assigned captain of one of the gangs. Consequently he should have been freed from any trivial business not connected with truly nautical duties, such being the case with his comrades.
[00:16:35.000 --> 00:16:43.000] I mention all of these particulars so that you may understand exactly how this affair stood between the two men.
[00:16:43.000 --> 00:16:53.000] But there was more than this. The order about the shovel was almost as plainly meant to sting and insult Steelkilt as though Radney had spat in his face.
[00:16:53.000 --> 00:17:05.000] Any man who has gone sailor in a whale ship will understand this, and all this and doubtless much more the lakeman fully comprehended when the mate uttered his command.
[00:17:05.000 --> 00:17:19.000] But as he sat still for a moment, and as he steadfastly looked into the mate's malignant eye and perceived the stacks of powder casks heaped up in him, and the slow match silently burning along towards them,
[00:17:19.000 --> 00:17:41.000] as he instinctively saw all this, that strange forbearance and unwillingness to stir up the deeper passionateness in any already-ireful being, a repugnance most felt when felt at all by really valiant men even when aggrieved, this nameless phantom-feeling gentleman stole over Steelkilt.
[00:17:41.000 --> 00:17:55.000] Therefore, in his ordinary tone, only a little broken by the bodily exhaustion he was temporarily in, he answered him saying that sweeping the deck was not his business, and he would not do it.
[00:17:55.000 --> 00:18:07.000] And then, without at all alluding to the shovel, he pointed to three lads as the customary sweepers, who, not being billeted at the pumps, had done little or nothing all day.
[00:18:07.000 --> 00:18:25.000] To this, Radney replied with an oath, in a most domineering and outrageous manner unconditionally reiterating his command, meanwhile advancing upon these still-seated lakemen with an uplifted Cooper's club hammer which he had snatched from a cask nearby.
[00:18:25.000 --> 00:18:38.000] Heeded and irritated as he was by his spasmodic toil at the pumps, for all his first nameless feeling of forbearance, the sweating Steelkilt could but ill-brooke this bearing in the mate.
[00:18:38.000 --> 00:18:55.000] But somehow, still smothering the conflagration within him, without speaking, he remained doggedly rooted to his seat, till at last the incensed Radney shook the hammer within a few inches of his face, furiously commanding him to do his bidding.
[00:18:55.000 --> 00:19:08.000] Steelkilt rose, and slowly retreating round the windlass, steadily followed by the mate with his menacing hammer, deliberately repeated his intention not to obey.
[00:19:08.000 --> 00:19:20.000] Seeing, however, that his forbearance had not the slightest effect, by an awful and unspeakable intimation with his twisted hand, he warned off the foolish and infatuated man.
[00:19:20.000 --> 00:19:40.000] But it was to no purpose, and in this way the two went once slowly round the windlass, when resolved at last no longer to retreat, bethinking him that he had now foreborn as much as comported with his humor, the lakemen paused on the hatches and thus spoke to the officer.
[00:19:40.000 --> 00:20:01.000] "Mr. Radney, I will not obey you. Take that hammer away, or look to yourself." But the predestinated mate, coming still closer to him, where the lakemen stood fixed, now shook the heavy hammer within an inch of his teeth, meanwhile repeating a string of insufferable maledictions,
[00:20:01.000 --> 00:20:22.000] retreating not the thousandth part of an inch, stabbing him in the eye with the unflinching panyard of his glance, Steelkilt, clenching his right hand behind him and creepingly drawing it back, told his persecutor that if the hammer but grazed his cheek, he, Steelkilt, would murder him.
[00:20:22.000 --> 00:20:37.000] But, gentlemen, the fool had been branded for the slaughter by the gods. Immediately the hammer touched his cheek, the next instant the lower jaw of the mate was stove in his head, he fell on the hatch spouting blood like a whale.
[00:20:37.000 --> 00:21:01.000] Ere the cry could go aft, Steelkilt was shaking one of the backstays leading far aloft, to where two of his comrades were standing their mastheads. They were both canalers. "Canalers!" cried Don Pedro. "We have seen many whale ships in our harbors, but never heard of your canalers. Pardon. Who and what are they?"
[00:21:01.000 --> 00:21:08.000] "Canalers, Don, are the boatmen belonging to our grand eerie canal. You must have heard of it."
[00:21:08.000 --> 00:21:19.000] "Nay, señor. Hereabouts in this dull, warm, most lazy and hereditary land, we know but little of your vigorous north."
[00:21:19.000 --> 00:21:33.000] "Aye? Well, then, Don, refill my cup. Your chicha's very fine, and ere proceeding further I will tell you what our canalers are, for such information may throw sidelight upon my story."
[00:21:33.000 --> 00:21:51.000] For 360 miles, gentlemen, through the entire breadth of the state of New York, through numerous populous cities and most thriving villages, through long, dismal, uninhabited swamps and affluent cultivated fields, unrivaled for fertility,
[00:21:51.000 --> 00:22:04.000] by billiard-room and bar-room, through the holy of holies of great forests, on Roman arches over Indian rivers, through sun and shade, by happy hearts are broken,
[00:22:04.000 --> 00:22:16.000] through all the wide contrasting scenery of those noble Mohawk counties, and especially by rows of snow-white chapels, whose spires stand almost like milestones,
[00:22:16.000 --> 00:22:35.000] flows one continual stream of venetianly corrupt and often lawless life. There's your true Ashanti, gentlemen. There howl your pagans, where you ever find them next door to you, under the long-flung shadow and the snug patronizing lee of churches.
[00:22:35.000 --> 00:22:49.000] For by some curious fatality, as it is often noted of your metropolitan free-booters that they ever encamp around the halls of justice, so sinners, gentlemen, most abound in holiest vicinities.
[00:22:49.000 --> 00:22:58.000] "Is that a friar passing?" said Don Pedro, looking downwards into the crowded piazza with humorous concern.
[00:22:58.000 --> 00:23:09.000] "Well, for our northern friend, dim Isabella's inquisition wanes in Lima," laughed Don Sebastian. "Proceeds in yore."
[00:23:09.000 --> 00:23:28.000] "A moment! Pardon!" cried another of the company. "In the name of all us limis, I but desire to express to you, Sir Sailor, that we have by no means overlooked your delicacy in not substituting present Lima for distant Venice in your corrupt comparison.
[00:23:28.000 --> 00:23:43.000] Oh, no, do not bow and look surprised. You know the proverb along all this coast, 'Corrupt as Lima.' It but bears out your saying, too, churches more plentiful than billiard tables and forever open.
[00:23:43.000 --> 00:24:00.000] And corrupt as Lima! So too Venice, I have been there, the holy city of the blessed evangelist St. Mark. St. Dominic, purge it! Your cup! Thanks, here I refill. Now, you pour out again."
[00:24:00.000 --> 00:24:25.000] Freely depicted in his own vocation, gentlemen, the canaler would make a fine dramatic hero, so abundantly and picturesquely wicked is he. Like Mark Antony, for days and days along his green-turfed flowery Nile, he indolently floats, openly toying with his red-cheeked Cleopatra, ripening his apricot thigh upon the sunny deck.
[00:24:25.000 --> 00:24:37.000] But ashore all this effeminacy is dashed. The brigandish guise which the canaler so proudly sports, his slouched and gaily-ribboned hat betoken his grand features.
[00:24:37.000 --> 00:24:47.000] A terror to the smiling innocence of the villages through which he floats, his swart visage and bold swagger, are not unshunned in cities.
[00:24:47.000 --> 00:25:08.000] Once a vagabond on his own canal, I have received good turns from one of these canalers. I thanked him heartily. Would Fane not be ungrateful? But it is often one of the prime redeeming qualities of your man of violence, that at times he is as stiff an arm to back a poor stranger in a strait as to plunder a wealthy one.
[00:25:08.000 --> 00:25:28.000] In some, gentlemen, what the wildness of this canal life is, is emphatically evinced by this. That our wild whale fishery contains so many of its most finished graduates, and that scarce any race of mankind, except Sydney men, are so much distrusted by our whaling captains.
[00:25:28.000 --> 00:25:50.000] Nor does it at all diminish the curiousness of this matter, that to many thousands of our rural boys and young men born along its line, the probationary life of the grand canal furnishes the sole transition between quietly reaping in a Christian cornfield and recklessly plowing the waters of the most barbaric seas.
[00:25:50.000 --> 00:26:12.000] "I see, I see," impetuously exclaimed Don Pedro, spilling his chicha upon his silvery ruffles. "No need to travel. The world's one Lima. I had thought now that at your temperate north, the generations were cold and holy as the hills. But the story."
[00:26:12.000 --> 00:26:26.000] I left off, gentlemen, where the lakeman shook the backstay. Hardly had he done so, when he was surrounded by the three junior mates and the four harpooners, who all crowded him to the deck.
[00:26:26.000 --> 00:26:35.000] But sliding down the ropes like baleful comets, the two canalers rushed into the uproar, and sought to drag their man out of it toward the folksil.
[00:26:35.000 --> 00:26:53.000] Others of the sailors joined with them in this attempt, and a twisted turmoil ensued. While standing out of harm's way, the valiant captain danced up and down with a whale pike, calling upon his officers to manhandle that atrocious scoundrel and smoke him along to the quarter deck.
[00:26:53.000 --> 00:27:07.000] At intervals he ran close up to the revolving border of the confusion, and prying into the heart of it with his pike, sought to prick out the object of his resentment. But steelkilt and his desperados were too much for them all.
[00:27:07.000 --> 00:27:22.000] They succeeded in gaining the folksil deck, where, hastily slewing about three or four large casks in a line with the windlass, these sea Parisians entrenched themselves behind the barricade.
[00:27:22.000 --> 00:27:34.000] "Come out of that, ye pirates!" roared the captain, now menacing them with a pistol in each hand, just brought to him by the steward. "Come out of that, ye cutthroats!"
[00:27:34.000 --> 00:28:01.000] Steelkilt leaped on the barricade, and, striding up and down there, defied the worst the pistols could do, but gave the captain to understand distinctly that his, Steelkilt's death, would be the signal for a murderous mutiny on the part of all hands. Fearing in his heart, lest this might prove but too true, the captain a little desisted, but still commanded the insurgents instantly to return to their duty.
[00:28:01.000 --> 00:28:19.000] "Will ye promise not to touch us if we do?" demanded their ringleader. "Turn two, turn two! I make no promise. To your duty! Do ye want to sink the ship by knocking off at a time like this? Turn two!" And he once more raised a pistol.
[00:28:19.000 --> 00:28:36.000] "Sink the ship!" cried Steelkilt. "Aye, let her sink! Not a man of us turns two unless ye swear not to raise a rope-yard against us. What say ye men?" turning to his comrades. A fierce cheer was their response.
[00:28:36.000 --> 00:29:04.000] The lakeman now patrolled the barricade, all the while keeping his eye on the captain, and jerking out such sentences as these. "It's not our fault! We didn't want it! I told him to take his hammer away! It was boys' business! He might have known me before this! I told him not to prick the buffalo! I believe I have broken a finger here against his cursed jaw! Ain't those mincing knives down in the fuxal there, men?
[00:29:04.000 --> 00:29:20.000] Look to those hand spikes, my hearties. Captain, by God, look to yourself. Say the word and don't be a fool. Forget it all. We are ready to turn two. Treat us decently and we're your men, but we won't be flogged."
[00:29:20.000 --> 00:29:49.000] "Turn two! I make no promises! Turn two, I say!" "Look ye now!" cried the lakeman, flinging out his arms towards him. "There are a few of us here, and I am one of them, who have shipped for the cruise, de' you see. Now, as you well know, sir, we can claim our discharge as soon as the anchor is down. So we don't want a row! It's not our interest! We want to be peaceable! We are ready to work, but we won't be flogged!"
[00:29:49.000 --> 00:30:00.000] "Turn two!" roared the captain. Steelkilt glanced round him a moment and then said, "I tell you what it is now, captain."
whisper_print_timings: load time = 275.32 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 978.92 ms
whisper_print_timings: sample time = 3435.74 ms / 1 runs ( 3435.74 ms per run)
whisper_print_timings: encode time = 46621.23 ms / 86 runs ( 542.11 ms per run)
whisper_print_timings: decode time = 58503.88 ms / 5925 runs ( 9.87 ms per run)
whisper_print_timings: batchd time = 17.89 ms / 11 runs ( 1.63 ms per run)
whisper_print_timings: prompt time = 7292.03 ms / 18584 runs ( 0.39 ms per run)
whisper_print_timings: total time = 119247.56 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/long/run_02.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 1800.0,
"audio_key": "long",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:27:48.098240+00:00",
"exit_code": 0,
"first_inference_latency_s": 1.8494081250028103,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_02.log",
"run_index": 2,
"run_kind": "measured",
"start_utc": "2026-03-08T19:25:48.317733+00:00",
"wall_clock_runtime_s": 119.78003570900182
}

1
benchmark/results/20260308_211026_metal-baseline/raw/long/run_02.transcript.txt Normal file
View File

File diff suppressed because one or more lines are too long

190
benchmark/results/20260308_211026_metal-baseline/raw/long/run_03.log Normal file
View File

@ -0,0 +1,190 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.011 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav' (28800000 samples, 1800.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:40.000] The Town Hose Story was a very popular circumstance, with its own particular accompaniments forming what may be called the "secret part" of the tragedy about to be narrated, never reach the ears of Captain Ahab or his mates.
[00:01:41.000 --> 00:01:58.000] The Town Hose Story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."
[00:01:59.000 --> 00:02:28.000] One night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest. Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mess.
[00:02:29.000 --> 00:02:41.000] Interweaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record.
[00:02:41.000 --> 00:02:55.000] Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin.
[00:02:56.000 --> 00:03:13.000] For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn.
[00:03:14.000 --> 00:03:27.000] Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put, and which are duly answered at the time.
[00:03:28.000 --> 00:03:47.000] Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn.
[00:03:48.000 --> 00:03:59.000] Somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hull than common.
[00:03:59.000 --> 00:04:17.000] They supposed a swordfish had stabbed her, gentlemen. But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous,
[00:04:18.000 --> 00:04:31.000] indeed they could not find it after searching the hold as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals.
[00:04:32.000 --> 00:04:52.000] But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased, so much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired.
[00:04:53.000 --> 00:05:13.000] Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that his ship would found her by the way, because his pumps were of the best, and being periodically relieved at them, those six and thirty men of his could easily keep the ship free, never mind if the leak should double on her.
[00:05:14.000 --> 00:05:40.000] In truth, well nigh the whole of this passage being attended by very prosperous breezes, the town-hole had all but certainly arrived in perfect safety at her port without the occurrence of the least fatality, had it not been for the brutal overbearing of Radney, the mate of Vineyarder, and the bitterly provoked vengeance of Steelkilt, a Lakeman in Desperado from Buffalo.
[00:05:41.000 --> 00:05:51.000] "Lakeman, Buffalo! Pray what is a Lakeman, and where is Buffalo?" said Don Sebastian, rising in his swinging mat of grass.
[00:05:51.000 --> 00:06:00.000] On the eastern shore of our Lake Erie, Don, but I crave your courtesy, may be you shall soon hear further of all that.
[00:06:01.000 --> 00:06:24.000] Now, gentlemen, in square sail-brigs and three-masted ships, well nigh as large and stout as any that ever sailed out of your old Caiao to far Manila, this Lakeman, in the landlocked heart of our America, had yet been nurtured by all those agrarian, free-booting impressions popularly connected with the open ocean.
[00:06:25.000 --> 00:06:51.000] Where interflowing aggregate those grand freshwater seas of ours, Erie and Ontario and Huron and Superior and Michigan, possess an ocean-like expansiveness with many of the ocean's noblest traits, with many of its rimmed varieties of races and of climes, they contain round archipelagos of romantic isles, even as the Polynesian waters do.
[00:06:52.000 --> 00:06:57.000] Each part are shored by two great contrasting nations as the Atlantic is.
[00:06:57.000 --> 00:07:06.000] They furnish long maritime approaches to our numerous territorial colonies from the east, dotted all round their banks.
[00:07:06.000 --> 00:07:13.000] Here and there are frowned upon by batteries and by the goat-like craggy guns of lofty Mackinac.
[00:07:14.000 --> 00:07:26.000] They have heard the fleet thunderings of naval victories. At intervals they yield their beaches to wild barbarians whose red painted faces flash from out their peltry wigwams.
[00:07:26.000 --> 00:07:42.000] For leagues and leagues are flanked by ancient and unentered forests, where the gaunt pines stand like serried lines of kings in Gothic genealogies, those same woods harboring wild, Africa beasts of prey,
[00:07:43.000 --> 00:07:47.000] and the creatures whose exported furs give robes to tartar emperors.
[00:07:47.000 --> 00:07:53.000] They mirror the paved capitals of Buffalo and Cleveland, as well as Winnebago villages.
[00:07:53.000 --> 00:08:01.000] They float alike the full-rigged merchant ship, the armed cruiser of the state, the steamer and the beach canoe.
[00:08:01.000 --> 00:08:08.000] They are swept by borean and dis-masting blasts as direful as any that lash the salted wave.
[00:08:09.000 --> 00:08:17.000] They know what shipwrecks are for out of sight of land, however inland they have drowned full many a midnight ship with all its shrieking crew.
[00:08:17.000 --> 00:08:28.000] Thus, gentlemen, though an inlander, steel-kilt was wild ocean-born and wild ocean-nurtured, as much of an audacious mariner as any.
[00:08:29.000 --> 00:08:43.000] And for Radney, though in his infancy he may have laid him down on the lone Nantucket beach, to nurse at his maternal sea, though in afterlife he had long followed our austere Atlantic and your contemplative Pacific,
[00:08:43.000 --> 00:08:53.000] yet he was quite as vengeful and full of social quarrel as the backwoods seamen, fresh from the latitudes of buckhorn-handled buoy knives.
[00:08:54.000 --> 00:09:11.000] Yet was this Nantucketer a man with some good-hearted traits, and this lakeman, a mariner, who though a sort of devil indeed, might yet by inflexible firmness only tempered by that common decency of human recognition which is the meanest slave's right,
[00:09:12.000 --> 00:09:27.000] as treated, this steel-kilt had long been retained harmless and docile. At all events he had proved so thus far, but Radney was doomed and made mad, and steel-kilt, but gentlemen you shall hear.
[00:09:28.000 --> 00:09:42.000] It was not more than a day or two at the furthest after pointing her prow for her island haven that the town-hose leak seemed again increasing, but only so as to require an hour or more at the pumps every day.
[00:09:43.000 --> 00:09:58.000] Let us know that in a settled and civilized ocean like our Atlantic, for example, some skippers think little of pumping their whole way across it, though of a still sleepy night should the officer of the deck happen to forget his duty in that respect,
[00:09:58.000 --> 00:10:07.000] the probability would be that he and his shipmates would never again remember it, on account of all hands gently subsiding to the bottom.
[00:10:08.000 --> 00:10:29.000] Whether in the solitary and savage seas far from you to the westward, gentlemen, is it altogether unusual for ships to keep clanging at their pump handles in full chorus even for a voyage of considerable length, that is if it lie along a tolerably accessible coast, or if any other reasonable retreat is afforded them.
[00:10:30.000 --> 00:10:40.000] It is only when a leaky vessel is in some very out-of-the-way part of those waters, some really landless latitude, that her captain begins to feel a little anxious.
[00:10:40.000 --> 00:10:55.000] Much this way had it been with the town-hoe, so when her leak was found gaining once more, there was in truth some small concern manifested by several of her company, especially by Radney the mate.
[00:10:56.000 --> 00:11:03.000] He commanded the upper sails to be well hoisted, sheeted home anew, and every way expanded to the breeze.
[00:11:03.000 --> 00:11:20.000] Now this Radney, I suppose, was as little of a coward, and as little inclined to any sort of nervous apprehensiveness touching his own person, as any fearless, unthinking creature on land or on sea that you can conveniently imagine, gentlemen.
[00:11:21.000 --> 00:11:31.000] Therefore when he betrayed this solicitude about the safety of the ship, some of the seamen declared that it was only on account of his being a part-owner in her.
[00:11:31.000 --> 00:11:45.000] So when they were working that evening at the pumps, there was on this head no small game-someness slyly going on among them, as they stood with their feet continually overflowed by the rippling clear water.
[00:11:46.000 --> 00:11:56.000] As any mountain-spring, gentlemen, that bubbling from the pumps ran across the deck and poured itself out in steady spouts at the Lee Scupper-holes.
[00:11:56.000 --> 00:12:14.000] Now as you well know, it is not seldom the case in this conventional world of ours, watery or otherwise, that when a person placed in command over his fellow men finds one of them to be very significantly his superior in general pride of manhood,
[00:12:15.000 --> 00:12:28.000] it may against that man he conceives an unconquerable dislike and bitterness, and if he have a chance, he will pull down and pulverize that subaltern's tower and make a little heap of dust of it.
[00:12:29.000 --> 00:12:54.000] Be this conceit of mine as it may, gentlemen, at all events steel-kilt was a tall and noble animal with a head like a Roman, and a flowing golden beard like the tasseled housings of your last viceroy's snorting charger, and a brain and a heart and a soul in him, gentlemen, which had made steel-kilt Charlemagne, had he been born son to Charlemagne's father.
[00:12:55.000 --> 00:13:06.000] In the maddening, the mate was as ugly as a mule, yet as hardy, as stubborn, as malicious. He did not love steel-kilt, and steel-kilt knew it.
[00:13:06.000 --> 00:13:18.000] As spying the mate drawing near as he was toiling at the pump with the rest, the lakeman affected not to notice him, but anod went on with his gay banterings.
[00:13:19.000 --> 00:13:47.000] "Aye, aye, my merry lads, it's a lively leak, this. Hold a cannekin, one of you, and let's have a taste. By the Lord, it's worth bottling. I tell you what, men, old Rad's investment must go for it. He had best cut away his part of the hull and tow it home. The fact is, boys, that swordfish only began the job. He's come back again with a gang of ship carpenters, sawfish and filefish and whatnot.
[00:13:48.000 --> 00:14:17.000] "The whole posse of 'em are now hard at work, cutting and slashing the bottom, making improvements, I suppose. If old Rad were here now, I'd tell him to jump overboard and scatter 'em. They're playing the devil with his estate, I can tell him. But he's a simple old soul, Rad, and a beauty too. Boys, they say the rest of his property is invested in looking-glasses. I wonder if he'd give a poor devil like me the model of his nose."
[00:14:18.000 --> 00:14:28.000] "Damn your eyes! What's that pump stopping for?" roared Radney, pretending not to have heard the sailors talk. Thunder away at it.
[00:14:28.000 --> 00:14:34.000] "Aye, aye, sir," said Steelkilt. "Mary is a cricket. Lively, boys, lively now."
[00:14:34.000 --> 00:14:49.000] And with that the pump clanged like fifty fire engines, the men tossed their hats off to it, and ere long that peculiar gasping of the lungs was heard, which denotes the fullest tensions of life's utmost energies.
[00:14:49.000 --> 00:15:06.000] Quitting the pump at last with the rest of his band, the lakeman went forward all panting, and sat himself down on the windlass, his face fiery red, his eyes bloodshot, and wiping the profuse sweat from his brow.
[00:15:06.000 --> 00:15:33.000] Now what cousining fiend it was, gentlemen, that possessed Radney to meddle with such a man in that corporeally exasperated state, I know not. But so it happened. Intolerably striding along the deck, the mate commanded him to get a broom and sweep down the planks, and also a shovel, and remove some offensive matters consequent upon allowing a pig to run at large.
[00:15:33.000 --> 00:15:45.000] Now, gentlemen, sweeping a ship's deck at sea is a piece of household work, which in all times but raging gales is regularly attended to every evening.
[00:15:45.000 --> 00:16:03.000] It has been known to be done in the case of ships actually foundering at the time. Such, gentlemen, is the inflexibility of sea usages, and the instinctive love of neatness in seaman, some of whom would not willingly drown without first washing their faces.
[00:16:03.000 --> 00:16:18.000] But in all vessels this broom business is the prescriptive province of the boys, if boys there be aboard. Besides, it was the stronger men in the town hoe that had been divided into gangs, taking turns at the pumps.
[00:16:18.000 --> 00:16:35.000] And, being the most athletic seaman of them all, Steelkilt had been regularly assigned captain of one of the gangs. Consequently he should have been freed from any trivial business not connected with truly nautical duties, such being the case with his comrades.
[00:16:35.000 --> 00:16:43.000] I mention all of these particulars so that you may understand exactly how this affair stood between the two men.
[00:16:43.000 --> 00:16:53.000] But there was more than this. The order about the shovel was almost as plainly meant to sting and insult Steelkilt as though Radney had spat in his face.
[00:16:53.000 --> 00:17:05.000] Any man who has gone sailor in a whale ship will understand this, and all this and doubtless much more the lakeman fully comprehended when the mate uttered his command.
[00:17:05.000 --> 00:17:19.000] But as he sat still for a moment, and as he steadfastly looked into the mate's malignant eye and perceived the stacks of powder casks heaped up in him, and the slow match silently burning along towards them,
[00:17:19.000 --> 00:17:41.000] as he instinctively saw all this, that strange forbearance and unwillingness to stir up the deeper passionateness in any already-ireful being, a repugnance most felt when felt at all by really valiant men even when aggrieved, this nameless phantom-feeling gentleman stole over Steelkilt.
[00:17:41.000 --> 00:17:55.000] Therefore, in his ordinary tone, only a little broken by the bodily exhaustion he was temporarily in, he answered him saying that sweeping the deck was not his business, and he would not do it.
[00:17:55.000 --> 00:18:07.000] And then, without at all alluding to the shovel, he pointed to three lads as the customary sweepers, who, not being billeted at the pumps, had done little or nothing all day.
[00:18:07.000 --> 00:18:25.000] To this, Radney replied with an oath, in a most domineering and outrageous manner unconditionally reiterating his command, meanwhile advancing upon these still-seated lakemen with an uplifted Cooper's club hammer which he had snatched from a cask nearby.
[00:18:25.000 --> 00:18:38.000] Heeded and irritated as he was by his spasmodic toil at the pumps, for all his first nameless feeling of forbearance, the sweating Steelkilt could but ill-brooke this bearing in the mate.
[00:18:38.000 --> 00:18:55.000] But somehow, still smothering the conflagration within him, without speaking, he remained doggedly rooted to his seat, till at last the incensed Radney shook the hammer within a few inches of his face, furiously commanding him to do his bidding.
[00:18:55.000 --> 00:19:08.000] Steelkilt rose, and slowly retreating round the windlass, steadily followed by the mate with his menacing hammer, deliberately repeated his intention not to obey.
[00:19:08.000 --> 00:19:20.000] Seeing, however, that his forbearance had not the slightest effect, by an awful and unspeakable intimation with his twisted hand, he warned off the foolish and infatuated man.
[00:19:20.000 --> 00:19:40.000] But it was to no purpose, and in this way the two went once slowly round the windlass, when resolved at last no longer to retreat, bethinking him that he had now foreborn as much as comported with his humor, the lakemen paused on the hatches and thus spoke to the officer.
[00:19:40.000 --> 00:20:01.000] "Mr. Radney, I will not obey you. Take that hammer away, or look to yourself." But the predestinated mate, coming still closer to him, where the lakemen stood fixed, now shook the heavy hammer within an inch of his teeth, meanwhile repeating a string of insufferable maledictions,
[00:20:01.000 --> 00:20:22.000] retreating not the thousandth part of an inch, stabbing him in the eye with the unflinching panyard of his glance, Steelkilt, clenching his right hand behind him and creepingly drawing it back, told his persecutor that if the hammer but grazed his cheek, he, Steelkilt, would murder him.
[00:20:22.000 --> 00:20:37.000] But, gentlemen, the fool had been branded for the slaughter by the gods. Immediately the hammer touched his cheek, the next instant the lower jaw of the mate was stove in his head, he fell on the hatch spouting blood like a whale.
[00:20:37.000 --> 00:21:01.000] Ere the cry could go aft, Steelkilt was shaking one of the backstays leading far aloft, to where two of his comrades were standing their mastheads. They were both canalers. "Canalers!" cried Don Pedro. "We have seen many whale ships in our harbors, but never heard of your canalers. Pardon. Who and what are they?"
[00:21:01.000 --> 00:21:08.000] "Canalers, Don, are the boatmen belonging to our grand eerie canal. You must have heard of it."
[00:21:08.000 --> 00:21:19.000] "Nay, señor. Hereabouts in this dull, warm, most lazy and hereditary land, we know but little of your vigorous north."
[00:21:19.000 --> 00:21:33.000] "Aye? Well, then, Don, refill my cup. Your chicha's very fine, and ere proceeding further I will tell you what our canalers are, for such information may throw sidelight upon my story."
[00:21:33.000 --> 00:21:51.000] For 360 miles, gentlemen, through the entire breadth of the state of New York, through numerous populous cities and most thriving villages, through long, dismal, uninhabited swamps and affluent cultivated fields, unrivaled for fertility,
[00:21:51.000 --> 00:22:04.000] by billiard-room and bar-room, through the holy of holies of great forests, on Roman arches over Indian rivers, through sun and shade, by happy hearts are broken,
[00:22:04.000 --> 00:22:16.000] through all the wide contrasting scenery of those noble Mohawk counties, and especially by rows of snow-white chapels, whose spires stand almost like milestones,
[00:22:16.000 --> 00:22:35.000] flows one continual stream of venetianly corrupt and often lawless life. There's your true Ashanti, gentlemen. There howl your pagans, where you ever find them next door to you, under the long-flung shadow and the snug patronizing lee of churches.
[00:22:35.000 --> 00:22:49.000] For by some curious fatality, as it is often noted of your metropolitan free-booters that they ever encamp around the halls of justice, so sinners, gentlemen, most abound in holiest vicinities.
[00:22:49.000 --> 00:22:58.000] "Is that a friar passing?" said Don Pedro, looking downwards into the crowded piazza with humorous concern.
[00:22:58.000 --> 00:23:09.000] "Well, for our northern friend, dim Isabella's inquisition wanes in Lima," laughed Don Sebastian. "Proceeds in yore."
[00:23:09.000 --> 00:23:28.000] "A moment! Pardon!" cried another of the company. "In the name of all us limis, I but desire to express to you, Sir Sailor, that we have by no means overlooked your delicacy in not substituting present Lima for distant Venice in your corrupt comparison.
[00:23:28.000 --> 00:23:43.000] Oh, no, do not bow and look surprised. You know the proverb along all this coast, 'Corrupt as Lima.' It but bears out your saying, too, churches more plentiful than billiard tables and forever open.
[00:23:43.000 --> 00:24:00.000] And corrupt as Lima! So too Venice, I have been there, the holy city of the blessed evangelist St. Mark. St. Dominic, purge it! Your cup! Thanks, here I refill. Now, you pour out again."
[00:24:00.000 --> 00:24:25.000] Freely depicted in his own vocation, gentlemen, the canaler would make a fine dramatic hero, so abundantly and picturesquely wicked is he. Like Mark Antony, for days and days along his green-turfed flowery Nile, he indolently floats, openly toying with his red-cheeked Cleopatra, ripening his apricot thigh upon the sunny deck.
[00:24:25.000 --> 00:24:37.000] But ashore all this effeminacy is dashed. The brigandish guise which the canaler so proudly sports, his slouched and gaily-ribboned hat betoken his grand features.
[00:24:37.000 --> 00:24:47.000] A terror to the smiling innocence of the villages through which he floats, his swart visage and bold swagger, are not unshunned in cities.
[00:24:47.000 --> 00:25:08.000] Once a vagabond on his own canal, I have received good turns from one of these canalers. I thanked him heartily. Would Fane not be ungrateful? But it is often one of the prime redeeming qualities of your man of violence, that at times he is as stiff an arm to back a poor stranger in a strait as to plunder a wealthy one.
[00:25:08.000 --> 00:25:28.000] In some, gentlemen, what the wildness of this canal life is, is emphatically evinced by this. That our wild whale fishery contains so many of its most finished graduates, and that scarce any race of mankind, except Sydney men, are so much distrusted by our whaling captains.
[00:25:28.000 --> 00:25:50.000] Nor does it at all diminish the curiousness of this matter, that to many thousands of our rural boys and young men born along its line, the probationary life of the grand canal furnishes the sole transition between quietly reaping in a Christian cornfield and recklessly plowing the waters of the most barbaric seas.
[00:25:50.000 --> 00:26:12.000] "I see, I see," impetuously exclaimed Don Pedro, spilling his chicha upon his silvery ruffles. "No need to travel. The world's one Lima. I had thought now that at your temperate north, the generations were cold and holy as the hills. But the story."
[00:26:12.000 --> 00:26:26.000] I left off, gentlemen, where the lakeman shook the backstay. Hardly had he done so, when he was surrounded by the three junior mates and the four harpooners, who all crowded him to the deck.
[00:26:26.000 --> 00:26:35.000] But sliding down the ropes like baleful comets, the two canalers rushed into the uproar, and sought to drag their man out of it toward the folksil.
[00:26:35.000 --> 00:26:53.000] Others of the sailors joined with them in this attempt, and a twisted turmoil ensued. While standing out of harm's way, the valiant captain danced up and down with a whale pike, calling upon his officers to manhandle that atrocious scoundrel and smoke him along to the quarter deck.
[00:26:53.000 --> 00:27:07.000] At intervals he ran close up to the revolving border of the confusion, and prying into the heart of it with his pike, sought to prick out the object of his resentment. But steelkilt and his desperados were too much for them all.
[00:27:07.000 --> 00:27:22.000] They succeeded in gaining the folksil deck, where, hastily slewing about three or four large casks in a line with the windlass, these sea Parisians entrenched themselves behind the barricade.
[00:27:22.000 --> 00:27:34.000] "Come out of that, ye pirates!" roared the captain, now menacing them with a pistol in each hand, just brought to him by the steward. "Come out of that, ye cutthroats!"
[00:27:34.000 --> 00:28:01.000] Steelkilt leaped on the barricade, and, striding up and down there, defied the worst the pistols could do, but gave the captain to understand distinctly that his, Steelkilt's death, would be the signal for a murderous mutiny on the part of all hands. Fearing in his heart, lest this might prove but too true, the captain a little desisted, but still commanded the insurgents instantly to return to their duty.
[00:28:01.000 --> 00:28:19.000] "Will ye promise not to touch us if we do?" demanded their ringleader. "Turn two, turn two! I make no promise. To your duty! Do ye want to sink the ship by knocking off at a time like this? Turn two!" And he once more raised a pistol.
[00:28:19.000 --> 00:28:36.000] "Sink the ship!" cried Steelkilt. "Aye, let her sink! Not a man of us turns two unless ye swear not to raise a rope-yard against us. What say ye men?" turning to his comrades. A fierce cheer was their response.
[00:28:36.000 --> 00:29:04.000] The lakeman now patrolled the barricade, all the while keeping his eye on the captain, and jerking out such sentences as these. "It's not our fault! We didn't want it! I told him to take his hammer away! It was boys' business! He might have known me before this! I told him not to prick the buffalo! I believe I have broken a finger here against his cursed jaw! Ain't those mincing knives down in the fuxal there, men?
[00:29:04.000 --> 00:29:20.000] Look to those hand spikes, my hearties. Captain, by God, look to yourself. Say the word and don't be a fool. Forget it all. We are ready to turn two. Treat us decently and we're your men, but we won't be flogged."
[00:29:20.000 --> 00:29:49.000] "Turn two! I make no promises! Turn two, I say!" "Look ye now!" cried the lakeman, flinging out his arms towards him. "There are a few of us here, and I am one of them, who have shipped for the cruise, de' you see. Now, as you well know, sir, we can claim our discharge as soon as the anchor is down. So we don't want a row! It's not our interest! We want to be peaceable! We are ready to work, but we won't be flogged!"
[00:29:49.000 --> 00:30:00.000] "Turn two!" roared the captain. Steelkilt glanced round him a moment and then said, "I tell you what it is now, captain."
whisper_print_timings: load time = 269.53 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 799.48 ms
whisper_print_timings: sample time = 3429.35 ms / 1 runs ( 3429.35 ms per run)
whisper_print_timings: encode time = 46570.10 ms / 86 runs ( 541.51 ms per run)
whisper_print_timings: decode time = 58910.39 ms / 5925 runs ( 9.94 ms per run)
whisper_print_timings: batchd time = 20.71 ms / 11 runs ( 1.88 ms per run)
whisper_print_timings: prompt time = 7425.47 ms / 18584 runs ( 0.40 ms per run)
whisper_print_timings: total time = 119543.45 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/long/run_03.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 1800.0,
"audio_key": "long",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:29:48.016765+00:00",
"exit_code": 0,
"first_inference_latency_s": 1.6381932919975952,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_03.log",
"run_index": 3,
"run_kind": "measured",
"start_utc": "2026-03-08T19:27:48.161863+00:00",
"wall_clock_runtime_s": 119.85825262499566
}

1
benchmark/results/20260308_211026_metal-baseline/raw/long/run_03.transcript.txt Normal file
View File

File diff suppressed because one or more lines are too long

190
benchmark/results/20260308_211026_metal-baseline/raw/long/run_04.log Normal file
View File

@ -0,0 +1,190 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.010 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav' (28800000 samples, 1800.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:40.000] The Town Hose Story was a very popular circumstance, with its own particular accompaniments forming what may be called the "secret part" of the tragedy about to be narrated, never reach the ears of Captain Ahab or his mates.
[00:01:41.000 --> 00:01:58.000] The Town Hose Story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."
[00:01:59.000 --> 00:02:28.000] One night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest. Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mess.
[00:02:29.000 --> 00:02:41.000] Interweaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record.
[00:02:41.000 --> 00:02:55.000] Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin.
[00:02:56.000 --> 00:03:13.000] For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn.
[00:03:14.000 --> 00:03:27.000] Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put, and which are duly answered at the time.
[00:03:28.000 --> 00:03:47.000] Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn.
[00:03:48.000 --> 00:03:59.000] Somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hull than common.
[00:03:59.000 --> 00:04:17.000] They supposed a swordfish had stabbed her, gentlemen. But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous,
[00:04:18.000 --> 00:04:31.000] indeed they could not find it after searching the hold as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals.
[00:04:32.000 --> 00:04:52.000] But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased, so much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired.
[00:04:53.000 --> 00:05:13.000] Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that his ship would found her by the way, because his pumps were of the best, and being periodically relieved at them, those six and thirty men of his could easily keep the ship free, never mind if the leak should double on her.
[00:05:14.000 --> 00:05:40.000] In truth, well nigh the whole of this passage being attended by very prosperous breezes, the town-hole had all but certainly arrived in perfect safety at her port without the occurrence of the least fatality, had it not been for the brutal overbearing of Radney, the mate of Vineyarder, and the bitterly provoked vengeance of Steelkilt, a Lakeman in Desperado from Buffalo.
[00:05:41.000 --> 00:05:51.000] "Lakeman, Buffalo! Pray what is a Lakeman, and where is Buffalo?" said Don Sebastian, rising in his swinging mat of grass.
[00:05:51.000 --> 00:06:00.000] On the eastern shore of our Lake Erie, Don, but I crave your courtesy, may be you shall soon hear further of all that.
[00:06:01.000 --> 00:06:24.000] Now, gentlemen, in square sail-brigs and three-masted ships, well nigh as large and stout as any that ever sailed out of your old Caiao to far Manila, this Lakeman, in the landlocked heart of our America, had yet been nurtured by all those agrarian, free-booting impressions popularly connected with the open ocean.
[00:06:25.000 --> 00:06:51.000] Where interflowing aggregate those grand freshwater seas of ours, Erie and Ontario and Huron and Superior and Michigan, possess an ocean-like expansiveness with many of the ocean's noblest traits, with many of its rimmed varieties of races and of climes, they contain round archipelagos of romantic isles, even as the Polynesian waters do.
[00:06:52.000 --> 00:06:57.000] Each part are shored by two great contrasting nations as the Atlantic is.
[00:06:57.000 --> 00:07:06.000] They furnish long maritime approaches to our numerous territorial colonies from the east, dotted all round their banks.
[00:07:06.000 --> 00:07:13.000] Here and there are frowned upon by batteries and by the goat-like craggy guns of lofty Mackinac.
[00:07:14.000 --> 00:07:26.000] They have heard the fleet thunderings of naval victories. At intervals they yield their beaches to wild barbarians whose red painted faces flash from out their peltry wigwams.
[00:07:26.000 --> 00:07:42.000] For leagues and leagues are flanked by ancient and unentered forests, where the gaunt pines stand like serried lines of kings in Gothic genealogies, those same woods harboring wild, Africa beasts of prey,
[00:07:43.000 --> 00:07:47.000] and the creatures whose exported furs give robes to tartar emperors.
[00:07:47.000 --> 00:07:53.000] They mirror the paved capitals of Buffalo and Cleveland, as well as Winnebago villages.
[00:07:53.000 --> 00:08:01.000] They float alike the full-rigged merchant ship, the armed cruiser of the state, the steamer and the beach canoe.
[00:08:01.000 --> 00:08:08.000] They are swept by borean and dis-masting blasts as direful as any that lash the salted wave.
[00:08:09.000 --> 00:08:17.000] They know what shipwrecks are for out of sight of land, however inland they have drowned full many a midnight ship with all its shrieking crew.
[00:08:17.000 --> 00:08:28.000] Thus, gentlemen, though an inlander, steel-kilt was wild ocean-born and wild ocean-nurtured, as much of an audacious mariner as any.
[00:08:29.000 --> 00:08:43.000] And for Radney, though in his infancy he may have laid him down on the lone Nantucket beach, to nurse at his maternal sea, though in afterlife he had long followed our austere Atlantic and your contemplative Pacific,
[00:08:43.000 --> 00:08:53.000] yet he was quite as vengeful and full of social quarrel as the backwoods seamen, fresh from the latitudes of buckhorn-handled buoy knives.
[00:08:54.000 --> 00:09:11.000] Yet was this Nantucketer a man with some good-hearted traits, and this lakeman, a mariner, who though a sort of devil indeed, might yet by inflexible firmness only tempered by that common decency of human recognition which is the meanest slave's right,
[00:09:12.000 --> 00:09:27.000] as treated, this steel-kilt had long been retained harmless and docile. At all events he had proved so thus far, but Radney was doomed and made mad, and steel-kilt, but gentlemen you shall hear.
[00:09:28.000 --> 00:09:42.000] It was not more than a day or two at the furthest after pointing her prow for her island haven that the town-hose leak seemed again increasing, but only so as to require an hour or more at the pumps every day.
[00:09:43.000 --> 00:09:58.000] Let us know that in a settled and civilized ocean like our Atlantic, for example, some skippers think little of pumping their whole way across it, though of a still sleepy night should the officer of the deck happen to forget his duty in that respect,
[00:09:58.000 --> 00:10:07.000] the probability would be that he and his shipmates would never again remember it, on account of all hands gently subsiding to the bottom.
[00:10:08.000 --> 00:10:29.000] Whether in the solitary and savage seas far from you to the westward, gentlemen, is it altogether unusual for ships to keep clanging at their pump handles in full chorus even for a voyage of considerable length, that is if it lie along a tolerably accessible coast, or if any other reasonable retreat is afforded them.
[00:10:30.000 --> 00:10:40.000] It is only when a leaky vessel is in some very out-of-the-way part of those waters, some really landless latitude, that her captain begins to feel a little anxious.
[00:10:40.000 --> 00:10:55.000] Much this way had it been with the town-hoe, so when her leak was found gaining once more, there was in truth some small concern manifested by several of her company, especially by Radney the mate.
[00:10:56.000 --> 00:11:03.000] He commanded the upper sails to be well hoisted, sheeted home anew, and every way expanded to the breeze.
[00:11:03.000 --> 00:11:20.000] Now this Radney, I suppose, was as little of a coward, and as little inclined to any sort of nervous apprehensiveness touching his own person, as any fearless, unthinking creature on land or on sea that you can conveniently imagine, gentlemen.
[00:11:21.000 --> 00:11:31.000] Therefore when he betrayed this solicitude about the safety of the ship, some of the seamen declared that it was only on account of his being a part-owner in her.
[00:11:31.000 --> 00:11:45.000] So when they were working that evening at the pumps, there was on this head no small game-someness slyly going on among them, as they stood with their feet continually overflowed by the rippling clear water.
[00:11:46.000 --> 00:11:56.000] As any mountain-spring, gentlemen, that bubbling from the pumps ran across the deck and poured itself out in steady spouts at the Lee Scupper-holes.
[00:11:56.000 --> 00:12:14.000] Now as you well know, it is not seldom the case in this conventional world of ours, watery or otherwise, that when a person placed in command over his fellow men finds one of them to be very significantly his superior in general pride of manhood,
[00:12:15.000 --> 00:12:28.000] it may against that man he conceives an unconquerable dislike and bitterness, and if he have a chance, he will pull down and pulverize that subaltern's tower and make a little heap of dust of it.
[00:12:29.000 --> 00:12:54.000] Be this conceit of mine as it may, gentlemen, at all events steel-kilt was a tall and noble animal with a head like a Roman, and a flowing golden beard like the tasseled housings of your last viceroy's snorting charger, and a brain and a heart and a soul in him, gentlemen, which had made steel-kilt Charlemagne, had he been born son to Charlemagne's father.
[00:12:55.000 --> 00:13:06.000] In the maddening, the mate was as ugly as a mule, yet as hardy, as stubborn, as malicious. He did not love steel-kilt, and steel-kilt knew it.
[00:13:06.000 --> 00:13:18.000] As spying the mate drawing near as he was toiling at the pump with the rest, the lakeman affected not to notice him, but anod went on with his gay banterings.
[00:13:19.000 --> 00:13:47.000] "Aye, aye, my merry lads, it's a lively leak, this. Hold a cannekin, one of you, and let's have a taste. By the Lord, it's worth bottling. I tell you what, men, old Rad's investment must go for it. He had best cut away his part of the hull and tow it home. The fact is, boys, that swordfish only began the job. He's come back again with a gang of ship carpenters, sawfish and filefish and whatnot.
[00:13:48.000 --> 00:14:17.000] "The whole posse of 'em are now hard at work, cutting and slashing the bottom, making improvements, I suppose. If old Rad were here now, I'd tell him to jump overboard and scatter 'em. They're playing the devil with his estate, I can tell him. But he's a simple old soul, Rad, and a beauty too. Boys, they say the rest of his property is invested in looking-glasses. I wonder if he'd give a poor devil like me the model of his nose."
[00:14:18.000 --> 00:14:28.000] "Damn your eyes! What's that pump stopping for?" roared Radney, pretending not to have heard the sailors talk. Thunder away at it.
[00:14:28.000 --> 00:14:34.000] "Aye, aye, sir," said Steelkilt. "Mary is a cricket. Lively, boys, lively now."
[00:14:34.000 --> 00:14:49.000] And with that the pump clanged like fifty fire engines, the men tossed their hats off to it, and ere long that peculiar gasping of the lungs was heard, which denotes the fullest tensions of life's utmost energies.
[00:14:49.000 --> 00:15:06.000] Quitting the pump at last with the rest of his band, the lakeman went forward all panting, and sat himself down on the windlass, his face fiery red, his eyes bloodshot, and wiping the profuse sweat from his brow.
[00:15:06.000 --> 00:15:33.000] Now what cousining fiend it was, gentlemen, that possessed Radney to meddle with such a man in that corporeally exasperated state, I know not. But so it happened. Intolerably striding along the deck, the mate commanded him to get a broom and sweep down the planks, and also a shovel, and remove some offensive matters consequent upon allowing a pig to run at large.
[00:15:33.000 --> 00:15:45.000] Now, gentlemen, sweeping a ship's deck at sea is a piece of household work, which in all times but raging gales is regularly attended to every evening.
[00:15:45.000 --> 00:16:03.000] It has been known to be done in the case of ships actually foundering at the time. Such, gentlemen, is the inflexibility of sea usages, and the instinctive love of neatness in seaman, some of whom would not willingly drown without first washing their faces.
[00:16:03.000 --> 00:16:18.000] But in all vessels this broom business is the prescriptive province of the boys, if boys there be aboard. Besides, it was the stronger men in the town hoe that had been divided into gangs, taking turns at the pumps.
[00:16:18.000 --> 00:16:35.000] And, being the most athletic seaman of them all, Steelkilt had been regularly assigned captain of one of the gangs. Consequently he should have been freed from any trivial business not connected with truly nautical duties, such being the case with his comrades.
[00:16:35.000 --> 00:16:43.000] I mention all of these particulars so that you may understand exactly how this affair stood between the two men.
[00:16:43.000 --> 00:16:53.000] But there was more than this. The order about the shovel was almost as plainly meant to sting and insult Steelkilt as though Radney had spat in his face.
[00:16:53.000 --> 00:17:05.000] Any man who has gone sailor in a whale ship will understand this, and all this and doubtless much more the lakeman fully comprehended when the mate uttered his command.
[00:17:05.000 --> 00:17:19.000] But as he sat still for a moment, and as he steadfastly looked into the mate's malignant eye and perceived the stacks of powder casks heaped up in him, and the slow match silently burning along towards them,
[00:17:19.000 --> 00:17:41.000] as he instinctively saw all this, that strange forbearance and unwillingness to stir up the deeper passionateness in any already-ireful being, a repugnance most felt when felt at all by really valiant men even when aggrieved, this nameless phantom-feeling gentleman stole over Steelkilt.
[00:17:41.000 --> 00:17:55.000] Therefore, in his ordinary tone, only a little broken by the bodily exhaustion he was temporarily in, he answered him saying that sweeping the deck was not his business, and he would not do it.
[00:17:55.000 --> 00:18:07.000] And then, without at all alluding to the shovel, he pointed to three lads as the customary sweepers, who, not being billeted at the pumps, had done little or nothing all day.
[00:18:07.000 --> 00:18:25.000] To this, Radney replied with an oath, in a most domineering and outrageous manner unconditionally reiterating his command, meanwhile advancing upon these still-seated lakemen with an uplifted Cooper's club hammer which he had snatched from a cask nearby.
[00:18:25.000 --> 00:18:38.000] Heeded and irritated as he was by his spasmodic toil at the pumps, for all his first nameless feeling of forbearance, the sweating Steelkilt could but ill-brooke this bearing in the mate.
[00:18:38.000 --> 00:18:55.000] But somehow, still smothering the conflagration within him, without speaking, he remained doggedly rooted to his seat, till at last the incensed Radney shook the hammer within a few inches of his face, furiously commanding him to do his bidding.
[00:18:55.000 --> 00:19:08.000] Steelkilt rose, and slowly retreating round the windlass, steadily followed by the mate with his menacing hammer, deliberately repeated his intention not to obey.
[00:19:08.000 --> 00:19:20.000] Seeing, however, that his forbearance had not the slightest effect, by an awful and unspeakable intimation with his twisted hand, he warned off the foolish and infatuated man.
[00:19:20.000 --> 00:19:40.000] But it was to no purpose, and in this way the two went once slowly round the windlass, when resolved at last no longer to retreat, bethinking him that he had now foreborn as much as comported with his humor, the lakemen paused on the hatches and thus spoke to the officer.
[00:19:40.000 --> 00:20:01.000] "Mr. Radney, I will not obey you. Take that hammer away, or look to yourself." But the predestinated mate, coming still closer to him, where the lakemen stood fixed, now shook the heavy hammer within an inch of his teeth, meanwhile repeating a string of insufferable maledictions,
[00:20:01.000 --> 00:20:22.000] retreating not the thousandth part of an inch, stabbing him in the eye with the unflinching panyard of his glance, Steelkilt, clenching his right hand behind him and creepingly drawing it back, told his persecutor that if the hammer but grazed his cheek, he, Steelkilt, would murder him.
[00:20:22.000 --> 00:20:37.000] But, gentlemen, the fool had been branded for the slaughter by the gods. Immediately the hammer touched his cheek, the next instant the lower jaw of the mate was stove in his head, he fell on the hatch spouting blood like a whale.
[00:20:37.000 --> 00:21:01.000] Ere the cry could go aft, Steelkilt was shaking one of the backstays leading far aloft, to where two of his comrades were standing their mastheads. They were both canalers. "Canalers!" cried Don Pedro. "We have seen many whale ships in our harbors, but never heard of your canalers. Pardon. Who and what are they?"
[00:21:01.000 --> 00:21:08.000] "Canalers, Don, are the boatmen belonging to our grand eerie canal. You must have heard of it."
[00:21:08.000 --> 00:21:19.000] "Nay, señor. Hereabouts in this dull, warm, most lazy and hereditary land, we know but little of your vigorous north."
[00:21:19.000 --> 00:21:33.000] "Aye? Well, then, Don, refill my cup. Your chicha's very fine, and ere proceeding further I will tell you what our canalers are, for such information may throw sidelight upon my story."
[00:21:33.000 --> 00:21:51.000] For 360 miles, gentlemen, through the entire breadth of the state of New York, through numerous populous cities and most thriving villages, through long, dismal, uninhabited swamps and affluent cultivated fields, unrivaled for fertility,
[00:21:51.000 --> 00:22:04.000] by billiard-room and bar-room, through the holy of holies of great forests, on Roman arches over Indian rivers, through sun and shade, by happy hearts are broken,
[00:22:04.000 --> 00:22:16.000] through all the wide contrasting scenery of those noble Mohawk counties, and especially by rows of snow-white chapels, whose spires stand almost like milestones,
[00:22:16.000 --> 00:22:35.000] flows one continual stream of venetianly corrupt and often lawless life. There's your true Ashanti, gentlemen. There howl your pagans, where you ever find them next door to you, under the long-flung shadow and the snug patronizing lee of churches.
[00:22:35.000 --> 00:22:49.000] For by some curious fatality, as it is often noted of your metropolitan free-booters that they ever encamp around the halls of justice, so sinners, gentlemen, most abound in holiest vicinities.
[00:22:49.000 --> 00:22:58.000] "Is that a friar passing?" said Don Pedro, looking downwards into the crowded piazza with humorous concern.
[00:22:58.000 --> 00:23:09.000] "Well, for our northern friend, dim Isabella's inquisition wanes in Lima," laughed Don Sebastian. "Proceeds in yore."
[00:23:09.000 --> 00:23:28.000] "A moment! Pardon!" cried another of the company. "In the name of all us limis, I but desire to express to you, Sir Sailor, that we have by no means overlooked your delicacy in not substituting present Lima for distant Venice in your corrupt comparison.
[00:23:28.000 --> 00:23:43.000] Oh, no, do not bow and look surprised. You know the proverb along all this coast, 'Corrupt as Lima.' It but bears out your saying, too, churches more plentiful than billiard tables and forever open.
[00:23:43.000 --> 00:24:00.000] And corrupt as Lima! So too Venice, I have been there, the holy city of the blessed evangelist St. Mark. St. Dominic, purge it! Your cup! Thanks, here I refill. Now, you pour out again."
[00:24:00.000 --> 00:24:25.000] Freely depicted in his own vocation, gentlemen, the canaler would make a fine dramatic hero, so abundantly and picturesquely wicked is he. Like Mark Antony, for days and days along his green-turfed flowery Nile, he indolently floats, openly toying with his red-cheeked Cleopatra, ripening his apricot thigh upon the sunny deck.
[00:24:25.000 --> 00:24:37.000] But ashore all this effeminacy is dashed. The brigandish guise which the canaler so proudly sports, his slouched and gaily-ribboned hat betoken his grand features.
[00:24:37.000 --> 00:24:47.000] A terror to the smiling innocence of the villages through which he floats, his swart visage and bold swagger, are not unshunned in cities.
[00:24:47.000 --> 00:25:08.000] Once a vagabond on his own canal, I have received good turns from one of these canalers. I thanked him heartily. Would Fane not be ungrateful? But it is often one of the prime redeeming qualities of your man of violence, that at times he is as stiff an arm to back a poor stranger in a strait as to plunder a wealthy one.
[00:25:08.000 --> 00:25:28.000] In some, gentlemen, what the wildness of this canal life is, is emphatically evinced by this. That our wild whale fishery contains so many of its most finished graduates, and that scarce any race of mankind, except Sydney men, are so much distrusted by our whaling captains.
[00:25:28.000 --> 00:25:50.000] Nor does it at all diminish the curiousness of this matter, that to many thousands of our rural boys and young men born along its line, the probationary life of the grand canal furnishes the sole transition between quietly reaping in a Christian cornfield and recklessly plowing the waters of the most barbaric seas.
[00:25:50.000 --> 00:26:12.000] "I see, I see," impetuously exclaimed Don Pedro, spilling his chicha upon his silvery ruffles. "No need to travel. The world's one Lima. I had thought now that at your temperate north, the generations were cold and holy as the hills. But the story."
[00:26:12.000 --> 00:26:26.000] I left off, gentlemen, where the lakeman shook the backstay. Hardly had he done so, when he was surrounded by the three junior mates and the four harpooners, who all crowded him to the deck.
[00:26:26.000 --> 00:26:35.000] But sliding down the ropes like baleful comets, the two canalers rushed into the uproar, and sought to drag their man out of it toward the folksil.
[00:26:35.000 --> 00:26:53.000] Others of the sailors joined with them in this attempt, and a twisted turmoil ensued. While standing out of harm's way, the valiant captain danced up and down with a whale pike, calling upon his officers to manhandle that atrocious scoundrel and smoke him along to the quarter deck.
[00:26:53.000 --> 00:27:07.000] At intervals he ran close up to the revolving border of the confusion, and prying into the heart of it with his pike, sought to prick out the object of his resentment. But steelkilt and his desperados were too much for them all.
[00:27:07.000 --> 00:27:22.000] They succeeded in gaining the folksil deck, where, hastily slewing about three or four large casks in a line with the windlass, these sea Parisians entrenched themselves behind the barricade.
[00:27:22.000 --> 00:27:34.000] "Come out of that, ye pirates!" roared the captain, now menacing them with a pistol in each hand, just brought to him by the steward. "Come out of that, ye cutthroats!"
[00:27:34.000 --> 00:28:01.000] Steelkilt leaped on the barricade, and, striding up and down there, defied the worst the pistols could do, but gave the captain to understand distinctly that his, Steelkilt's death, would be the signal for a murderous mutiny on the part of all hands. Fearing in his heart, lest this might prove but too true, the captain a little desisted, but still commanded the insurgents instantly to return to their duty.
[00:28:01.000 --> 00:28:19.000] "Will ye promise not to touch us if we do?" demanded their ringleader. "Turn two, turn two! I make no promise. To your duty! Do ye want to sink the ship by knocking off at a time like this? Turn two!" And he once more raised a pistol.
[00:28:19.000 --> 00:28:36.000] "Sink the ship!" cried Steelkilt. "Aye, let her sink! Not a man of us turns two unless ye swear not to raise a rope-yard against us. What say ye men?" turning to his comrades. A fierce cheer was their response.
[00:28:36.000 --> 00:29:04.000] The lakeman now patrolled the barricade, all the while keeping his eye on the captain, and jerking out such sentences as these. "It's not our fault! We didn't want it! I told him to take his hammer away! It was boys' business! He might have known me before this! I told him not to prick the buffalo! I believe I have broken a finger here against his cursed jaw! Ain't those mincing knives down in the fuxal there, men?
[00:29:04.000 --> 00:29:20.000] Look to those hand spikes, my hearties. Captain, by God, look to yourself. Say the word and don't be a fool. Forget it all. We are ready to turn two. Treat us decently and we're your men, but we won't be flogged."
[00:29:20.000 --> 00:29:49.000] "Turn two! I make no promises! Turn two, I say!" "Look ye now!" cried the lakeman, flinging out his arms towards him. "There are a few of us here, and I am one of them, who have shipped for the cruise, de' you see. Now, as you well know, sir, we can claim our discharge as soon as the anchor is down. So we don't want a row! It's not our interest! We want to be peaceable! We are ready to work, but we won't be flogged!"
[00:29:49.000 --> 00:30:00.000] "Turn two!" roared the captain. Steelkilt glanced round him a moment and then said, "I tell you what it is now, captain."
whisper_print_timings: load time = 257.67 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 960.72 ms
whisper_print_timings: sample time = 3453.00 ms / 1 runs ( 3453.00 ms per run)
whisper_print_timings: encode time = 46440.52 ms / 86 runs ( 540.01 ms per run)
whisper_print_timings: decode time = 58337.10 ms / 5925 runs ( 9.85 ms per run)
whisper_print_timings: batchd time = 19.65 ms / 11 runs ( 1.79 ms per run)
whisper_print_timings: prompt time = 7312.95 ms / 18584 runs ( 0.39 ms per run)
whisper_print_timings: total time = 118916.23 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/long/run_04.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 1800.0,
"audio_key": "long",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:31:47.401819+00:00",
"exit_code": 0,
"first_inference_latency_s": 1.7880880420052563,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_04.log",
"run_index": 4,
"run_kind": "measured",
"start_utc": "2026-03-08T19:29:48.078545+00:00",
"wall_clock_runtime_s": 119.3234943330026
}

1
benchmark/results/20260308_211026_metal-baseline/raw/long/run_04.transcript.txt Normal file
View File

File diff suppressed because one or more lines are too long

190
benchmark/results/20260308_211026_metal-baseline/raw/long/run_05.log Normal file
View File

@ -0,0 +1,190 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.010 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav' (28800000 samples, 1800.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:40.000] The Town Hose Story was a very popular circumstance, with its own particular accompaniments forming what may be called the "secret part" of the tragedy about to be narrated, never reach the ears of Captain Ahab or his mates.
[00:01:41.000 --> 00:01:58.000] The Town Hose Story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."
[00:01:59.000 --> 00:02:28.000] One night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest. Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mess.
[00:02:29.000 --> 00:02:41.000] Interweaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record.
[00:02:41.000 --> 00:02:55.000] Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin.
[00:02:56.000 --> 00:03:13.000] For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn.
[00:03:14.000 --> 00:03:27.000] Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put, and which are duly answered at the time.
[00:03:28.000 --> 00:03:47.000] Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn.
[00:03:48.000 --> 00:03:59.000] Somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hull than common.
[00:03:59.000 --> 00:04:17.000] They supposed a swordfish had stabbed her, gentlemen. But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous,
[00:04:18.000 --> 00:04:31.000] indeed they could not find it after searching the hold as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals.
[00:04:32.000 --> 00:04:52.000] But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased, so much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired.
[00:04:53.000 --> 00:05:13.000] Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that his ship would found her by the way, because his pumps were of the best, and being periodically relieved at them, those six and thirty men of his could easily keep the ship free, never mind if the leak should double on her.
[00:05:14.000 --> 00:05:40.000] In truth, well nigh the whole of this passage being attended by very prosperous breezes, the town-hole had all but certainly arrived in perfect safety at her port without the occurrence of the least fatality, had it not been for the brutal overbearing of Radney, the mate of Vineyarder, and the bitterly provoked vengeance of Steelkilt, a Lakeman in Desperado from Buffalo.
[00:05:41.000 --> 00:05:51.000] "Lakeman, Buffalo! Pray what is a Lakeman, and where is Buffalo?" said Don Sebastian, rising in his swinging mat of grass.
[00:05:51.000 --> 00:06:00.000] On the eastern shore of our Lake Erie, Don, but I crave your courtesy, may be you shall soon hear further of all that.
[00:06:01.000 --> 00:06:24.000] Now, gentlemen, in square sail-brigs and three-masted ships, well nigh as large and stout as any that ever sailed out of your old Caiao to far Manila, this Lakeman, in the landlocked heart of our America, had yet been nurtured by all those agrarian, free-booting impressions popularly connected with the open ocean.
[00:06:25.000 --> 00:06:51.000] Where interflowing aggregate those grand freshwater seas of ours, Erie and Ontario and Huron and Superior and Michigan, possess an ocean-like expansiveness with many of the ocean's noblest traits, with many of its rimmed varieties of races and of climes, they contain round archipelagos of romantic isles, even as the Polynesian waters do.
[00:06:52.000 --> 00:06:57.000] Each part are shored by two great contrasting nations as the Atlantic is.
[00:06:57.000 --> 00:07:06.000] They furnish long maritime approaches to our numerous territorial colonies from the east, dotted all round their banks.
[00:07:06.000 --> 00:07:13.000] Here and there are frowned upon by batteries and by the goat-like craggy guns of lofty Mackinac.
[00:07:14.000 --> 00:07:26.000] They have heard the fleet thunderings of naval victories. At intervals they yield their beaches to wild barbarians whose red painted faces flash from out their peltry wigwams.
[00:07:26.000 --> 00:07:42.000] For leagues and leagues are flanked by ancient and unentered forests, where the gaunt pines stand like serried lines of kings in Gothic genealogies, those same woods harboring wild, Africa beasts of prey,
[00:07:43.000 --> 00:07:47.000] and the creatures whose exported furs give robes to tartar emperors.
[00:07:47.000 --> 00:07:53.000] They mirror the paved capitals of Buffalo and Cleveland, as well as Winnebago villages.
[00:07:53.000 --> 00:08:01.000] They float alike the full-rigged merchant ship, the armed cruiser of the state, the steamer and the beach canoe.
[00:08:01.000 --> 00:08:08.000] They are swept by borean and dis-masting blasts as direful as any that lash the salted wave.
[00:08:09.000 --> 00:08:17.000] They know what shipwrecks are for out of sight of land, however inland they have drowned full many a midnight ship with all its shrieking crew.
[00:08:17.000 --> 00:08:28.000] Thus, gentlemen, though an inlander, steel-kilt was wild ocean-born and wild ocean-nurtured, as much of an audacious mariner as any.
[00:08:29.000 --> 00:08:43.000] And for Radney, though in his infancy he may have laid him down on the lone Nantucket beach, to nurse at his maternal sea, though in afterlife he had long followed our austere Atlantic and your contemplative Pacific,
[00:08:43.000 --> 00:08:53.000] yet he was quite as vengeful and full of social quarrel as the backwoods seamen, fresh from the latitudes of buckhorn-handled buoy knives.
[00:08:54.000 --> 00:09:11.000] Yet was this Nantucketer a man with some good-hearted traits, and this lakeman, a mariner, who though a sort of devil indeed, might yet by inflexible firmness only tempered by that common decency of human recognition which is the meanest slave's right,
[00:09:12.000 --> 00:09:27.000] as treated, this steel-kilt had long been retained harmless and docile. At all events he had proved so thus far, but Radney was doomed and made mad, and steel-kilt, but gentlemen you shall hear.
[00:09:28.000 --> 00:09:42.000] It was not more than a day or two at the furthest after pointing her prow for her island haven that the town-hose leak seemed again increasing, but only so as to require an hour or more at the pumps every day.
[00:09:43.000 --> 00:09:58.000] Let us know that in a settled and civilized ocean like our Atlantic, for example, some skippers think little of pumping their whole way across it, though of a still sleepy night should the officer of the deck happen to forget his duty in that respect,
[00:09:58.000 --> 00:10:07.000] the probability would be that he and his shipmates would never again remember it, on account of all hands gently subsiding to the bottom.
[00:10:08.000 --> 00:10:29.000] Whether in the solitary and savage seas far from you to the westward, gentlemen, is it altogether unusual for ships to keep clanging at their pump handles in full chorus even for a voyage of considerable length, that is if it lie along a tolerably accessible coast, or if any other reasonable retreat is afforded them.
[00:10:30.000 --> 00:10:40.000] It is only when a leaky vessel is in some very out-of-the-way part of those waters, some really landless latitude, that her captain begins to feel a little anxious.
[00:10:40.000 --> 00:10:55.000] Much this way had it been with the town-hoe, so when her leak was found gaining once more, there was in truth some small concern manifested by several of her company, especially by Radney the mate.
[00:10:56.000 --> 00:11:03.000] He commanded the upper sails to be well hoisted, sheeted home anew, and every way expanded to the breeze.
[00:11:03.000 --> 00:11:20.000] Now this Radney, I suppose, was as little of a coward, and as little inclined to any sort of nervous apprehensiveness touching his own person, as any fearless, unthinking creature on land or on sea that you can conveniently imagine, gentlemen.
[00:11:21.000 --> 00:11:31.000] Therefore when he betrayed this solicitude about the safety of the ship, some of the seamen declared that it was only on account of his being a part-owner in her.
[00:11:31.000 --> 00:11:45.000] So when they were working that evening at the pumps, there was on this head no small game-someness slyly going on among them, as they stood with their feet continually overflowed by the rippling clear water.
[00:11:46.000 --> 00:11:56.000] As any mountain-spring, gentlemen, that bubbling from the pumps ran across the deck and poured itself out in steady spouts at the Lee Scupper-holes.
[00:11:56.000 --> 00:12:14.000] Now as you well know, it is not seldom the case in this conventional world of ours, watery or otherwise, that when a person placed in command over his fellow men finds one of them to be very significantly his superior in general pride of manhood,
[00:12:15.000 --> 00:12:28.000] it may against that man he conceives an unconquerable dislike and bitterness, and if he have a chance, he will pull down and pulverize that subaltern's tower and make a little heap of dust of it.
[00:12:29.000 --> 00:12:54.000] Be this conceit of mine as it may, gentlemen, at all events steel-kilt was a tall and noble animal with a head like a Roman, and a flowing golden beard like the tasseled housings of your last viceroy's snorting charger, and a brain and a heart and a soul in him, gentlemen, which had made steel-kilt Charlemagne, had he been born son to Charlemagne's father.
[00:12:55.000 --> 00:13:06.000] In the maddening, the mate was as ugly as a mule, yet as hardy, as stubborn, as malicious. He did not love steel-kilt, and steel-kilt knew it.
[00:13:06.000 --> 00:13:18.000] As spying the mate drawing near as he was toiling at the pump with the rest, the lakeman affected not to notice him, but anod went on with his gay banterings.
[00:13:19.000 --> 00:13:47.000] "Aye, aye, my merry lads, it's a lively leak, this. Hold a cannekin, one of you, and let's have a taste. By the Lord, it's worth bottling. I tell you what, men, old Rad's investment must go for it. He had best cut away his part of the hull and tow it home. The fact is, boys, that swordfish only began the job. He's come back again with a gang of ship carpenters, sawfish and filefish and whatnot.
[00:13:48.000 --> 00:14:17.000] "The whole posse of 'em are now hard at work, cutting and slashing the bottom, making improvements, I suppose. If old Rad were here now, I'd tell him to jump overboard and scatter 'em. They're playing the devil with his estate, I can tell him. But he's a simple old soul, Rad, and a beauty too. Boys, they say the rest of his property is invested in looking-glasses. I wonder if he'd give a poor devil like me the model of his nose."
[00:14:18.000 --> 00:14:28.000] "Damn your eyes! What's that pump stopping for?" roared Radney, pretending not to have heard the sailors talk. Thunder away at it.
[00:14:28.000 --> 00:14:34.000] "Aye, aye, sir," said Steelkilt. "Mary is a cricket. Lively, boys, lively now."
[00:14:34.000 --> 00:14:49.000] And with that the pump clanged like fifty fire engines, the men tossed their hats off to it, and ere long that peculiar gasping of the lungs was heard, which denotes the fullest tensions of life's utmost energies.
[00:14:49.000 --> 00:15:06.000] Quitting the pump at last with the rest of his band, the lakeman went forward all panting, and sat himself down on the windlass, his face fiery red, his eyes bloodshot, and wiping the profuse sweat from his brow.
[00:15:06.000 --> 00:15:33.000] Now what cousining fiend it was, gentlemen, that possessed Radney to meddle with such a man in that corporeally exasperated state, I know not. But so it happened. Intolerably striding along the deck, the mate commanded him to get a broom and sweep down the planks, and also a shovel, and remove some offensive matters consequent upon allowing a pig to run at large.
[00:15:33.000 --> 00:15:45.000] Now, gentlemen, sweeping a ship's deck at sea is a piece of household work, which in all times but raging gales is regularly attended to every evening.
[00:15:45.000 --> 00:16:03.000] It has been known to be done in the case of ships actually foundering at the time. Such, gentlemen, is the inflexibility of sea usages, and the instinctive love of neatness in seaman, some of whom would not willingly drown without first washing their faces.
[00:16:03.000 --> 00:16:18.000] But in all vessels this broom business is the prescriptive province of the boys, if boys there be aboard. Besides, it was the stronger men in the town hoe that had been divided into gangs, taking turns at the pumps.
[00:16:18.000 --> 00:16:35.000] And, being the most athletic seaman of them all, Steelkilt had been regularly assigned captain of one of the gangs. Consequently he should have been freed from any trivial business not connected with truly nautical duties, such being the case with his comrades.
[00:16:35.000 --> 00:16:43.000] I mention all of these particulars so that you may understand exactly how this affair stood between the two men.
[00:16:43.000 --> 00:16:53.000] But there was more than this. The order about the shovel was almost as plainly meant to sting and insult Steelkilt as though Radney had spat in his face.
[00:16:53.000 --> 00:17:05.000] Any man who has gone sailor in a whale ship will understand this, and all this and doubtless much more the lakeman fully comprehended when the mate uttered his command.
[00:17:05.000 --> 00:17:19.000] But as he sat still for a moment, and as he steadfastly looked into the mate's malignant eye and perceived the stacks of powder casks heaped up in him, and the slow match silently burning along towards them,
[00:17:19.000 --> 00:17:41.000] as he instinctively saw all this, that strange forbearance and unwillingness to stir up the deeper passionateness in any already-ireful being, a repugnance most felt when felt at all by really valiant men even when aggrieved, this nameless phantom-feeling gentleman stole over Steelkilt.
[00:17:41.000 --> 00:17:55.000] Therefore, in his ordinary tone, only a little broken by the bodily exhaustion he was temporarily in, he answered him saying that sweeping the deck was not his business, and he would not do it.
[00:17:55.000 --> 00:18:07.000] And then, without at all alluding to the shovel, he pointed to three lads as the customary sweepers, who, not being billeted at the pumps, had done little or nothing all day.
[00:18:07.000 --> 00:18:25.000] To this, Radney replied with an oath, in a most domineering and outrageous manner unconditionally reiterating his command, meanwhile advancing upon these still-seated lakemen with an uplifted Cooper's club hammer which he had snatched from a cask nearby.
[00:18:25.000 --> 00:18:38.000] Heeded and irritated as he was by his spasmodic toil at the pumps, for all his first nameless feeling of forbearance, the sweating Steelkilt could but ill-brooke this bearing in the mate.
[00:18:38.000 --> 00:18:55.000] But somehow, still smothering the conflagration within him, without speaking, he remained doggedly rooted to his seat, till at last the incensed Radney shook the hammer within a few inches of his face, furiously commanding him to do his bidding.
[00:18:55.000 --> 00:19:08.000] Steelkilt rose, and slowly retreating round the windlass, steadily followed by the mate with his menacing hammer, deliberately repeated his intention not to obey.
[00:19:08.000 --> 00:19:20.000] Seeing, however, that his forbearance had not the slightest effect, by an awful and unspeakable intimation with his twisted hand, he warned off the foolish and infatuated man.
[00:19:20.000 --> 00:19:40.000] But it was to no purpose, and in this way the two went once slowly round the windlass, when resolved at last no longer to retreat, bethinking him that he had now foreborn as much as comported with his humor, the lakemen paused on the hatches and thus spoke to the officer.
[00:19:40.000 --> 00:20:01.000] "Mr. Radney, I will not obey you. Take that hammer away, or look to yourself." But the predestinated mate, coming still closer to him, where the lakemen stood fixed, now shook the heavy hammer within an inch of his teeth, meanwhile repeating a string of insufferable maledictions,
[00:20:01.000 --> 00:20:22.000] retreating not the thousandth part of an inch, stabbing him in the eye with the unflinching panyard of his glance, Steelkilt, clenching his right hand behind him and creepingly drawing it back, told his persecutor that if the hammer but grazed his cheek, he, Steelkilt, would murder him.
[00:20:22.000 --> 00:20:37.000] But, gentlemen, the fool had been branded for the slaughter by the gods. Immediately the hammer touched his cheek, the next instant the lower jaw of the mate was stove in his head, he fell on the hatch spouting blood like a whale.
[00:20:37.000 --> 00:21:01.000] Ere the cry could go aft, Steelkilt was shaking one of the backstays leading far aloft, to where two of his comrades were standing their mastheads. They were both canalers. "Canalers!" cried Don Pedro. "We have seen many whale ships in our harbors, but never heard of your canalers. Pardon. Who and what are they?"
[00:21:01.000 --> 00:21:08.000] "Canalers, Don, are the boatmen belonging to our grand eerie canal. You must have heard of it."
[00:21:08.000 --> 00:21:19.000] "Nay, señor. Hereabouts in this dull, warm, most lazy and hereditary land, we know but little of your vigorous north."
[00:21:19.000 --> 00:21:33.000] "Aye? Well, then, Don, refill my cup. Your chicha's very fine, and ere proceeding further I will tell you what our canalers are, for such information may throw sidelight upon my story."
[00:21:33.000 --> 00:21:51.000] For 360 miles, gentlemen, through the entire breadth of the state of New York, through numerous populous cities and most thriving villages, through long, dismal, uninhabited swamps and affluent cultivated fields, unrivaled for fertility,
[00:21:51.000 --> 00:22:04.000] by billiard-room and bar-room, through the holy of holies of great forests, on Roman arches over Indian rivers, through sun and shade, by happy hearts are broken,
[00:22:04.000 --> 00:22:16.000] through all the wide contrasting scenery of those noble Mohawk counties, and especially by rows of snow-white chapels, whose spires stand almost like milestones,
[00:22:16.000 --> 00:22:35.000] flows one continual stream of venetianly corrupt and often lawless life. There's your true Ashanti, gentlemen. There howl your pagans, where you ever find them next door to you, under the long-flung shadow and the snug patronizing lee of churches.
[00:22:35.000 --> 00:22:49.000] For by some curious fatality, as it is often noted of your metropolitan free-booters that they ever encamp around the halls of justice, so sinners, gentlemen, most abound in holiest vicinities.
[00:22:49.000 --> 00:22:58.000] "Is that a friar passing?" said Don Pedro, looking downwards into the crowded piazza with humorous concern.
[00:22:58.000 --> 00:23:09.000] "Well, for our northern friend, dim Isabella's inquisition wanes in Lima," laughed Don Sebastian. "Proceeds in yore."
[00:23:09.000 --> 00:23:28.000] "A moment! Pardon!" cried another of the company. "In the name of all us limis, I but desire to express to you, Sir Sailor, that we have by no means overlooked your delicacy in not substituting present Lima for distant Venice in your corrupt comparison.
[00:23:28.000 --> 00:23:43.000] Oh, no, do not bow and look surprised. You know the proverb along all this coast, 'Corrupt as Lima.' It but bears out your saying, too, churches more plentiful than billiard tables and forever open.
[00:23:43.000 --> 00:24:00.000] And corrupt as Lima! So too Venice, I have been there, the holy city of the blessed evangelist St. Mark. St. Dominic, purge it! Your cup! Thanks, here I refill. Now, you pour out again."
[00:24:00.000 --> 00:24:25.000] Freely depicted in his own vocation, gentlemen, the canaler would make a fine dramatic hero, so abundantly and picturesquely wicked is he. Like Mark Antony, for days and days along his green-turfed flowery Nile, he indolently floats, openly toying with his red-cheeked Cleopatra, ripening his apricot thigh upon the sunny deck.
[00:24:25.000 --> 00:24:37.000] But ashore all this effeminacy is dashed. The brigandish guise which the canaler so proudly sports, his slouched and gaily-ribboned hat betoken his grand features.
[00:24:37.000 --> 00:24:47.000] A terror to the smiling innocence of the villages through which he floats, his swart visage and bold swagger, are not unshunned in cities.
[00:24:47.000 --> 00:25:08.000] Once a vagabond on his own canal, I have received good turns from one of these canalers. I thanked him heartily. Would Fane not be ungrateful? But it is often one of the prime redeeming qualities of your man of violence, that at times he is as stiff an arm to back a poor stranger in a strait as to plunder a wealthy one.
[00:25:08.000 --> 00:25:28.000] In some, gentlemen, what the wildness of this canal life is, is emphatically evinced by this. That our wild whale fishery contains so many of its most finished graduates, and that scarce any race of mankind, except Sydney men, are so much distrusted by our whaling captains.
[00:25:28.000 --> 00:25:50.000] Nor does it at all diminish the curiousness of this matter, that to many thousands of our rural boys and young men born along its line, the probationary life of the grand canal furnishes the sole transition between quietly reaping in a Christian cornfield and recklessly plowing the waters of the most barbaric seas.
[00:25:50.000 --> 00:26:12.000] "I see, I see," impetuously exclaimed Don Pedro, spilling his chicha upon his silvery ruffles. "No need to travel. The world's one Lima. I had thought now that at your temperate north, the generations were cold and holy as the hills. But the story."
[00:26:12.000 --> 00:26:26.000] I left off, gentlemen, where the lakeman shook the backstay. Hardly had he done so, when he was surrounded by the three junior mates and the four harpooners, who all crowded him to the deck.
[00:26:26.000 --> 00:26:35.000] But sliding down the ropes like baleful comets, the two canalers rushed into the uproar, and sought to drag their man out of it toward the folksil.
[00:26:35.000 --> 00:26:53.000] Others of the sailors joined with them in this attempt, and a twisted turmoil ensued. While standing out of harm's way, the valiant captain danced up and down with a whale pike, calling upon his officers to manhandle that atrocious scoundrel and smoke him along to the quarter deck.
[00:26:53.000 --> 00:27:07.000] At intervals he ran close up to the revolving border of the confusion, and prying into the heart of it with his pike, sought to prick out the object of his resentment. But steelkilt and his desperados were too much for them all.
[00:27:07.000 --> 00:27:22.000] They succeeded in gaining the folksil deck, where, hastily slewing about three or four large casks in a line with the windlass, these sea Parisians entrenched themselves behind the barricade.
[00:27:22.000 --> 00:27:34.000] "Come out of that, ye pirates!" roared the captain, now menacing them with a pistol in each hand, just brought to him by the steward. "Come out of that, ye cutthroats!"
[00:27:34.000 --> 00:28:01.000] Steelkilt leaped on the barricade, and, striding up and down there, defied the worst the pistols could do, but gave the captain to understand distinctly that his, Steelkilt's death, would be the signal for a murderous mutiny on the part of all hands. Fearing in his heart, lest this might prove but too true, the captain a little desisted, but still commanded the insurgents instantly to return to their duty.
[00:28:01.000 --> 00:28:19.000] "Will ye promise not to touch us if we do?" demanded their ringleader. "Turn two, turn two! I make no promise. To your duty! Do ye want to sink the ship by knocking off at a time like this? Turn two!" And he once more raised a pistol.
[00:28:19.000 --> 00:28:36.000] "Sink the ship!" cried Steelkilt. "Aye, let her sink! Not a man of us turns two unless ye swear not to raise a rope-yard against us. What say ye men?" turning to his comrades. A fierce cheer was their response.
[00:28:36.000 --> 00:29:04.000] The lakeman now patrolled the barricade, all the while keeping his eye on the captain, and jerking out such sentences as these. "It's not our fault! We didn't want it! I told him to take his hammer away! It was boys' business! He might have known me before this! I told him not to prick the buffalo! I believe I have broken a finger here against his cursed jaw! Ain't those mincing knives down in the fuxal there, men?
[00:29:04.000 --> 00:29:20.000] Look to those hand spikes, my hearties. Captain, by God, look to yourself. Say the word and don't be a fool. Forget it all. We are ready to turn two. Treat us decently and we're your men, but we won't be flogged."
[00:29:20.000 --> 00:29:49.000] "Turn two! I make no promises! Turn two, I say!" "Look ye now!" cried the lakeman, flinging out his arms towards him. "There are a few of us here, and I am one of them, who have shipped for the cruise, de' you see. Now, as you well know, sir, we can claim our discharge as soon as the anchor is down. So we don't want a row! It's not our interest! We want to be peaceable! We are ready to work, but we won't be flogged!"
[00:29:49.000 --> 00:30:00.000] "Turn two!" roared the captain. Steelkilt glanced round him a moment and then said, "I tell you what it is now, captain."
whisper_print_timings: load time = 265.81 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 891.28 ms
whisper_print_timings: sample time = 3466.45 ms / 1 runs ( 3466.45 ms per run)
whisper_print_timings: encode time = 47112.68 ms / 86 runs ( 547.82 ms per run)
whisper_print_timings: decode time = 60712.06 ms / 5925 runs ( 10.25 ms per run)
whisper_print_timings: batchd time = 17.78 ms / 11 runs ( 1.62 ms per run)
whisper_print_timings: prompt time = 7477.49 ms / 18584 runs ( 0.40 ms per run)
whisper_print_timings: total time = 122060.63 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/long/run_05.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 1800.0,
"audio_key": "long",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:33:49.805207+00:00",
"exit_code": 0,
"first_inference_latency_s": 1.739509791994351,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_05.log",
"run_index": 5,
"run_kind": "measured",
"start_utc": "2026-03-08T19:31:47.467600+00:00",
"wall_clock_runtime_s": 122.33720404199994
}

1
benchmark/results/20260308_211026_metal-baseline/raw/long/run_05.transcript.txt Normal file
View File

File diff suppressed because one or more lines are too long

190
benchmark/results/20260308_211026_metal-baseline/raw/long/warmup_01.log Normal file
View File

@ -0,0 +1,190 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.012 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav' (28800000 samples, 1800.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:40.000] The Town Hose Story was a very popular circumstance, with its own particular accompaniments forming what may be called the "secret part" of the tragedy about to be narrated, never reach the ears of Captain Ahab or his mates.
[00:01:41.000 --> 00:01:58.000] The Town Hose Story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."
[00:01:59.000 --> 00:02:28.000] One night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest. Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mess.
[00:02:29.000 --> 00:02:41.000] Interweaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record.
[00:02:41.000 --> 00:02:55.000] Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin.
[00:02:56.000 --> 00:03:13.000] For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn.
[00:03:14.000 --> 00:03:27.000] Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put, and which are duly answered at the time.
[00:03:28.000 --> 00:03:47.000] Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn.
[00:03:48.000 --> 00:03:59.000] Somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hull than common.
[00:03:59.000 --> 00:04:17.000] They supposed a swordfish had stabbed her, gentlemen. But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous,
[00:04:18.000 --> 00:04:31.000] indeed they could not find it after searching the hold as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals.
[00:04:32.000 --> 00:04:52.000] But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased, so much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired.
[00:04:53.000 --> 00:05:13.000] Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that his ship would found her by the way, because his pumps were of the best, and being periodically relieved at them, those six and thirty men of his could easily keep the ship free, never mind if the leak should double on her.
[00:05:14.000 --> 00:05:40.000] In truth, well nigh the whole of this passage being attended by very prosperous breezes, the town-hole had all but certainly arrived in perfect safety at her port without the occurrence of the least fatality, had it not been for the brutal overbearing of Radney, the mate of Vineyarder, and the bitterly provoked vengeance of Steelkilt, a Lakeman in Desperado from Buffalo.
[00:05:41.000 --> 00:05:51.000] "Lakeman, Buffalo! Pray what is a Lakeman, and where is Buffalo?" said Don Sebastian, rising in his swinging mat of grass.
[00:05:51.000 --> 00:06:00.000] On the eastern shore of our Lake Erie, Don, but I crave your courtesy, may be you shall soon hear further of all that.
[00:06:01.000 --> 00:06:24.000] Now, gentlemen, in square sail-brigs and three-masted ships, well nigh as large and stout as any that ever sailed out of your old Caiao to far Manila, this Lakeman, in the landlocked heart of our America, had yet been nurtured by all those agrarian, free-booting impressions popularly connected with the open ocean.
[00:06:25.000 --> 00:06:51.000] Where interflowing aggregate those grand freshwater seas of ours, Erie and Ontario and Huron and Superior and Michigan, possess an ocean-like expansiveness with many of the ocean's noblest traits, with many of its rimmed varieties of races and of climes, they contain round archipelagos of romantic isles, even as the Polynesian waters do.
[00:06:52.000 --> 00:06:57.000] Each part are shored by two great contrasting nations as the Atlantic is.
[00:06:57.000 --> 00:07:06.000] They furnish long maritime approaches to our numerous territorial colonies from the east, dotted all round their banks.
[00:07:06.000 --> 00:07:13.000] Here and there are frowned upon by batteries and by the goat-like craggy guns of lofty Mackinac.
[00:07:14.000 --> 00:07:26.000] They have heard the fleet thunderings of naval victories. At intervals they yield their beaches to wild barbarians whose red painted faces flash from out their peltry wigwams.
[00:07:26.000 --> 00:07:42.000] For leagues and leagues are flanked by ancient and unentered forests, where the gaunt pines stand like serried lines of kings in Gothic genealogies, those same woods harboring wild, Africa beasts of prey,
[00:07:43.000 --> 00:07:47.000] and the creatures whose exported furs give robes to tartar emperors.
[00:07:47.000 --> 00:07:53.000] They mirror the paved capitals of Buffalo and Cleveland, as well as Winnebago villages.
[00:07:53.000 --> 00:08:01.000] They float alike the full-rigged merchant ship, the armed cruiser of the state, the steamer and the beach canoe.
[00:08:01.000 --> 00:08:08.000] They are swept by borean and dis-masting blasts as direful as any that lash the salted wave.
[00:08:09.000 --> 00:08:17.000] They know what shipwrecks are for out of sight of land, however inland they have drowned full many a midnight ship with all its shrieking crew.
[00:08:17.000 --> 00:08:28.000] Thus, gentlemen, though an inlander, steel-kilt was wild ocean-born and wild ocean-nurtured, as much of an audacious mariner as any.
[00:08:29.000 --> 00:08:43.000] And for Radney, though in his infancy he may have laid him down on the lone Nantucket beach, to nurse at his maternal sea, though in afterlife he had long followed our austere Atlantic and your contemplative Pacific,
[00:08:43.000 --> 00:08:53.000] yet he was quite as vengeful and full of social quarrel as the backwoods seamen, fresh from the latitudes of buckhorn-handled buoy knives.
[00:08:54.000 --> 00:09:11.000] Yet was this Nantucketer a man with some good-hearted traits, and this lakeman, a mariner, who though a sort of devil indeed, might yet by inflexible firmness only tempered by that common decency of human recognition which is the meanest slave's right,
[00:09:12.000 --> 00:09:27.000] as treated, this steel-kilt had long been retained harmless and docile. At all events he had proved so thus far, but Radney was doomed and made mad, and steel-kilt, but gentlemen you shall hear.
[00:09:28.000 --> 00:09:42.000] It was not more than a day or two at the furthest after pointing her prow for her island haven that the town-hose leak seemed again increasing, but only so as to require an hour or more at the pumps every day.
[00:09:43.000 --> 00:09:58.000] Let us know that in a settled and civilized ocean like our Atlantic, for example, some skippers think little of pumping their whole way across it, though of a still sleepy night should the officer of the deck happen to forget his duty in that respect,
[00:09:58.000 --> 00:10:07.000] the probability would be that he and his shipmates would never again remember it, on account of all hands gently subsiding to the bottom.
[00:10:08.000 --> 00:10:29.000] Whether in the solitary and savage seas far from you to the westward, gentlemen, is it altogether unusual for ships to keep clanging at their pump handles in full chorus even for a voyage of considerable length, that is if it lie along a tolerably accessible coast, or if any other reasonable retreat is afforded them.
[00:10:30.000 --> 00:10:40.000] It is only when a leaky vessel is in some very out-of-the-way part of those waters, some really landless latitude, that her captain begins to feel a little anxious.
[00:10:40.000 --> 00:10:55.000] Much this way had it been with the town-hoe, so when her leak was found gaining once more, there was in truth some small concern manifested by several of her company, especially by Radney the mate.
[00:10:56.000 --> 00:11:03.000] He commanded the upper sails to be well hoisted, sheeted home anew, and every way expanded to the breeze.
[00:11:03.000 --> 00:11:20.000] Now this Radney, I suppose, was as little of a coward, and as little inclined to any sort of nervous apprehensiveness touching his own person, as any fearless, unthinking creature on land or on sea that you can conveniently imagine, gentlemen.
[00:11:21.000 --> 00:11:31.000] Therefore when he betrayed this solicitude about the safety of the ship, some of the seamen declared that it was only on account of his being a part-owner in her.
[00:11:31.000 --> 00:11:45.000] So when they were working that evening at the pumps, there was on this head no small game-someness slyly going on among them, as they stood with their feet continually overflowed by the rippling clear water.
[00:11:46.000 --> 00:11:56.000] As any mountain-spring, gentlemen, that bubbling from the pumps ran across the deck and poured itself out in steady spouts at the Lee Scupper-holes.
[00:11:56.000 --> 00:12:14.000] Now as you well know, it is not seldom the case in this conventional world of ours, watery or otherwise, that when a person placed in command over his fellow men finds one of them to be very significantly his superior in general pride of manhood,
[00:12:15.000 --> 00:12:28.000] it may against that man he conceives an unconquerable dislike and bitterness, and if he have a chance, he will pull down and pulverize that subaltern's tower and make a little heap of dust of it.
[00:12:29.000 --> 00:12:54.000] Be this conceit of mine as it may, gentlemen, at all events steel-kilt was a tall and noble animal with a head like a Roman, and a flowing golden beard like the tasseled housings of your last viceroy's snorting charger, and a brain and a heart and a soul in him, gentlemen, which had made steel-kilt Charlemagne, had he been born son to Charlemagne's father.
[00:12:55.000 --> 00:13:06.000] In the maddening, the mate was as ugly as a mule, yet as hardy, as stubborn, as malicious. He did not love steel-kilt, and steel-kilt knew it.
[00:13:06.000 --> 00:13:18.000] As spying the mate drawing near as he was toiling at the pump with the rest, the lakeman affected not to notice him, but anod went on with his gay banterings.
[00:13:19.000 --> 00:13:47.000] "Aye, aye, my merry lads, it's a lively leak, this. Hold a cannekin, one of you, and let's have a taste. By the Lord, it's worth bottling. I tell you what, men, old Rad's investment must go for it. He had best cut away his part of the hull and tow it home. The fact is, boys, that swordfish only began the job. He's come back again with a gang of ship carpenters, sawfish and filefish and whatnot.
[00:13:48.000 --> 00:14:17.000] "The whole posse of 'em are now hard at work, cutting and slashing the bottom, making improvements, I suppose. If old Rad were here now, I'd tell him to jump overboard and scatter 'em. They're playing the devil with his estate, I can tell him. But he's a simple old soul, Rad, and a beauty too. Boys, they say the rest of his property is invested in looking-glasses. I wonder if he'd give a poor devil like me the model of his nose."
[00:14:18.000 --> 00:14:28.000] "Damn your eyes! What's that pump stopping for?" roared Radney, pretending not to have heard the sailors talk. Thunder away at it.
[00:14:28.000 --> 00:14:34.000] "Aye, aye, sir," said Steelkilt. "Mary is a cricket. Lively, boys, lively now."
[00:14:34.000 --> 00:14:49.000] And with that the pump clanged like fifty fire engines, the men tossed their hats off to it, and ere long that peculiar gasping of the lungs was heard, which denotes the fullest tensions of life's utmost energies.
[00:14:49.000 --> 00:15:06.000] Quitting the pump at last with the rest of his band, the lakeman went forward all panting, and sat himself down on the windlass, his face fiery red, his eyes bloodshot, and wiping the profuse sweat from his brow.
[00:15:06.000 --> 00:15:33.000] Now what cousining fiend it was, gentlemen, that possessed Radney to meddle with such a man in that corporeally exasperated state, I know not. But so it happened. Intolerably striding along the deck, the mate commanded him to get a broom and sweep down the planks, and also a shovel, and remove some offensive matters consequent upon allowing a pig to run at large.
[00:15:33.000 --> 00:15:45.000] Now, gentlemen, sweeping a ship's deck at sea is a piece of household work, which in all times but raging gales is regularly attended to every evening.
[00:15:45.000 --> 00:16:03.000] It has been known to be done in the case of ships actually foundering at the time. Such, gentlemen, is the inflexibility of sea usages, and the instinctive love of neatness in seaman, some of whom would not willingly drown without first washing their faces.
[00:16:03.000 --> 00:16:18.000] But in all vessels this broom business is the prescriptive province of the boys, if boys there be aboard. Besides, it was the stronger men in the town hoe that had been divided into gangs, taking turns at the pumps.
[00:16:18.000 --> 00:16:35.000] And, being the most athletic seaman of them all, Steelkilt had been regularly assigned captain of one of the gangs. Consequently he should have been freed from any trivial business not connected with truly nautical duties, such being the case with his comrades.
[00:16:35.000 --> 00:16:43.000] I mention all of these particulars so that you may understand exactly how this affair stood between the two men.
[00:16:43.000 --> 00:16:53.000] But there was more than this. The order about the shovel was almost as plainly meant to sting and insult Steelkilt as though Radney had spat in his face.
[00:16:53.000 --> 00:17:05.000] Any man who has gone sailor in a whale ship will understand this, and all this and doubtless much more the lakeman fully comprehended when the mate uttered his command.
[00:17:05.000 --> 00:17:19.000] But as he sat still for a moment, and as he steadfastly looked into the mate's malignant eye and perceived the stacks of powder casks heaped up in him, and the slow match silently burning along towards them,
[00:17:19.000 --> 00:17:41.000] as he instinctively saw all this, that strange forbearance and unwillingness to stir up the deeper passionateness in any already-ireful being, a repugnance most felt when felt at all by really valiant men even when aggrieved, this nameless phantom-feeling gentleman stole over Steelkilt.
[00:17:41.000 --> 00:17:55.000] Therefore, in his ordinary tone, only a little broken by the bodily exhaustion he was temporarily in, he answered him saying that sweeping the deck was not his business, and he would not do it.
[00:17:55.000 --> 00:18:07.000] And then, without at all alluding to the shovel, he pointed to three lads as the customary sweepers, who, not being billeted at the pumps, had done little or nothing all day.
[00:18:07.000 --> 00:18:25.000] To this, Radney replied with an oath, in a most domineering and outrageous manner unconditionally reiterating his command, meanwhile advancing upon these still-seated lakemen with an uplifted Cooper's club hammer which he had snatched from a cask nearby.
[00:18:25.000 --> 00:18:38.000] Heeded and irritated as he was by his spasmodic toil at the pumps, for all his first nameless feeling of forbearance, the sweating Steelkilt could but ill-brooke this bearing in the mate.
[00:18:38.000 --> 00:18:55.000] But somehow, still smothering the conflagration within him, without speaking, he remained doggedly rooted to his seat, till at last the incensed Radney shook the hammer within a few inches of his face, furiously commanding him to do his bidding.
[00:18:55.000 --> 00:19:08.000] Steelkilt rose, and slowly retreating round the windlass, steadily followed by the mate with his menacing hammer, deliberately repeated his intention not to obey.
[00:19:08.000 --> 00:19:20.000] Seeing, however, that his forbearance had not the slightest effect, by an awful and unspeakable intimation with his twisted hand, he warned off the foolish and infatuated man.
[00:19:20.000 --> 00:19:40.000] But it was to no purpose, and in this way the two went once slowly round the windlass, when resolved at last no longer to retreat, bethinking him that he had now foreborn as much as comported with his humor, the lakemen paused on the hatches and thus spoke to the officer.
[00:19:40.000 --> 00:20:01.000] "Mr. Radney, I will not obey you. Take that hammer away, or look to yourself." But the predestinated mate, coming still closer to him, where the lakemen stood fixed, now shook the heavy hammer within an inch of his teeth, meanwhile repeating a string of insufferable maledictions,
[00:20:01.000 --> 00:20:22.000] retreating not the thousandth part of an inch, stabbing him in the eye with the unflinching panyard of his glance, Steelkilt, clenching his right hand behind him and creepingly drawing it back, told his persecutor that if the hammer but grazed his cheek, he, Steelkilt, would murder him.
[00:20:22.000 --> 00:20:37.000] But, gentlemen, the fool had been branded for the slaughter by the gods. Immediately the hammer touched his cheek, the next instant the lower jaw of the mate was stove in his head, he fell on the hatch spouting blood like a whale.
[00:20:37.000 --> 00:21:01.000] Ere the cry could go aft, Steelkilt was shaking one of the backstays leading far aloft, to where two of his comrades were standing their mastheads. They were both canalers. "Canalers!" cried Don Pedro. "We have seen many whale ships in our harbors, but never heard of your canalers. Pardon. Who and what are they?"
[00:21:01.000 --> 00:21:08.000] "Canalers, Don, are the boatmen belonging to our grand eerie canal. You must have heard of it."
[00:21:08.000 --> 00:21:19.000] "Nay, señor. Hereabouts in this dull, warm, most lazy and hereditary land, we know but little of your vigorous north."
[00:21:19.000 --> 00:21:33.000] "Aye? Well, then, Don, refill my cup. Your chicha's very fine, and ere proceeding further I will tell you what our canalers are, for such information may throw sidelight upon my story."
[00:21:33.000 --> 00:21:51.000] For 360 miles, gentlemen, through the entire breadth of the state of New York, through numerous populous cities and most thriving villages, through long, dismal, uninhabited swamps and affluent cultivated fields, unrivaled for fertility,
[00:21:51.000 --> 00:22:04.000] by billiard-room and bar-room, through the holy of holies of great forests, on Roman arches over Indian rivers, through sun and shade, by happy hearts are broken,
[00:22:04.000 --> 00:22:16.000] through all the wide contrasting scenery of those noble Mohawk counties, and especially by rows of snow-white chapels, whose spires stand almost like milestones,
[00:22:16.000 --> 00:22:35.000] flows one continual stream of venetianly corrupt and often lawless life. There's your true Ashanti, gentlemen. There howl your pagans, where you ever find them next door to you, under the long-flung shadow and the snug patronizing lee of churches.
[00:22:35.000 --> 00:22:49.000] For by some curious fatality, as it is often noted of your metropolitan free-booters that they ever encamp around the halls of justice, so sinners, gentlemen, most abound in holiest vicinities.
[00:22:49.000 --> 00:22:58.000] "Is that a friar passing?" said Don Pedro, looking downwards into the crowded piazza with humorous concern.
[00:22:58.000 --> 00:23:09.000] "Well, for our northern friend, dim Isabella's inquisition wanes in Lima," laughed Don Sebastian. "Proceeds in yore."
[00:23:09.000 --> 00:23:28.000] "A moment! Pardon!" cried another of the company. "In the name of all us limis, I but desire to express to you, Sir Sailor, that we have by no means overlooked your delicacy in not substituting present Lima for distant Venice in your corrupt comparison.
[00:23:28.000 --> 00:23:43.000] Oh, no, do not bow and look surprised. You know the proverb along all this coast, 'Corrupt as Lima.' It but bears out your saying, too, churches more plentiful than billiard tables and forever open.
[00:23:43.000 --> 00:24:00.000] And corrupt as Lima! So too Venice, I have been there, the holy city of the blessed evangelist St. Mark. St. Dominic, purge it! Your cup! Thanks, here I refill. Now, you pour out again."
[00:24:00.000 --> 00:24:25.000] Freely depicted in his own vocation, gentlemen, the canaler would make a fine dramatic hero, so abundantly and picturesquely wicked is he. Like Mark Antony, for days and days along his green-turfed flowery Nile, he indolently floats, openly toying with his red-cheeked Cleopatra, ripening his apricot thigh upon the sunny deck.
[00:24:25.000 --> 00:24:37.000] But ashore all this effeminacy is dashed. The brigandish guise which the canaler so proudly sports, his slouched and gaily-ribboned hat betoken his grand features.
[00:24:37.000 --> 00:24:47.000] A terror to the smiling innocence of the villages through which he floats, his swart visage and bold swagger, are not unshunned in cities.
[00:24:47.000 --> 00:25:08.000] Once a vagabond on his own canal, I have received good turns from one of these canalers. I thanked him heartily. Would Fane not be ungrateful? But it is often one of the prime redeeming qualities of your man of violence, that at times he is as stiff an arm to back a poor stranger in a strait as to plunder a wealthy one.
[00:25:08.000 --> 00:25:28.000] In some, gentlemen, what the wildness of this canal life is, is emphatically evinced by this. That our wild whale fishery contains so many of its most finished graduates, and that scarce any race of mankind, except Sydney men, are so much distrusted by our whaling captains.
[00:25:28.000 --> 00:25:50.000] Nor does it at all diminish the curiousness of this matter, that to many thousands of our rural boys and young men born along its line, the probationary life of the grand canal furnishes the sole transition between quietly reaping in a Christian cornfield and recklessly plowing the waters of the most barbaric seas.
[00:25:50.000 --> 00:26:12.000] "I see, I see," impetuously exclaimed Don Pedro, spilling his chicha upon his silvery ruffles. "No need to travel. The world's one Lima. I had thought now that at your temperate north, the generations were cold and holy as the hills. But the story."
[00:26:12.000 --> 00:26:26.000] I left off, gentlemen, where the lakeman shook the backstay. Hardly had he done so, when he was surrounded by the three junior mates and the four harpooners, who all crowded him to the deck.
[00:26:26.000 --> 00:26:35.000] But sliding down the ropes like baleful comets, the two canalers rushed into the uproar, and sought to drag their man out of it toward the folksil.
[00:26:35.000 --> 00:26:53.000] Others of the sailors joined with them in this attempt, and a twisted turmoil ensued. While standing out of harm's way, the valiant captain danced up and down with a whale pike, calling upon his officers to manhandle that atrocious scoundrel and smoke him along to the quarter deck.
[00:26:53.000 --> 00:27:07.000] At intervals he ran close up to the revolving border of the confusion, and prying into the heart of it with his pike, sought to prick out the object of his resentment. But steelkilt and his desperados were too much for them all.
[00:27:07.000 --> 00:27:22.000] They succeeded in gaining the folksil deck, where, hastily slewing about three or four large casks in a line with the windlass, these sea Parisians entrenched themselves behind the barricade.
[00:27:22.000 --> 00:27:34.000] "Come out of that, ye pirates!" roared the captain, now menacing them with a pistol in each hand, just brought to him by the steward. "Come out of that, ye cutthroats!"
[00:27:34.000 --> 00:28:01.000] Steelkilt leaped on the barricade, and, striding up and down there, defied the worst the pistols could do, but gave the captain to understand distinctly that his, Steelkilt's death, would be the signal for a murderous mutiny on the part of all hands. Fearing in his heart, lest this might prove but too true, the captain a little desisted, but still commanded the insurgents instantly to return to their duty.
[00:28:01.000 --> 00:28:19.000] "Will ye promise not to touch us if we do?" demanded their ringleader. "Turn two, turn two! I make no promise. To your duty! Do ye want to sink the ship by knocking off at a time like this? Turn two!" And he once more raised a pistol.
[00:28:19.000 --> 00:28:36.000] "Sink the ship!" cried Steelkilt. "Aye, let her sink! Not a man of us turns two unless ye swear not to raise a rope-yard against us. What say ye men?" turning to his comrades. A fierce cheer was their response.
[00:28:36.000 --> 00:29:04.000] The lakeman now patrolled the barricade, all the while keeping his eye on the captain, and jerking out such sentences as these. "It's not our fault! We didn't want it! I told him to take his hammer away! It was boys' business! He might have known me before this! I told him not to prick the buffalo! I believe I have broken a finger here against his cursed jaw! Ain't those mincing knives down in the fuxal there, men?
[00:29:04.000 --> 00:29:20.000] Look to those hand spikes, my hearties. Captain, by God, look to yourself. Say the word and don't be a fool. Forget it all. We are ready to turn two. Treat us decently and we're your men, but we won't be flogged."
[00:29:20.000 --> 00:29:49.000] "Turn two! I make no promises! Turn two, I say!" "Look ye now!" cried the lakeman, flinging out his arms towards him. "There are a few of us here, and I am one of them, who have shipped for the cruise, de' you see. Now, as you well know, sir, we can claim our discharge as soon as the anchor is down. So we don't want a row! It's not our interest! We want to be peaceable! We are ready to work, but we won't be flogged!"
[00:29:49.000 --> 00:30:00.000] "Turn two!" roared the captain. Steelkilt glanced round him a moment and then said, "I tell you what it is now, captain."
whisper_print_timings: load time = 341.92 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 3530.67 ms
whisper_print_timings: sample time = 4939.29 ms / 1 runs ( 4939.29 ms per run)
whisper_print_timings: encode time = 113504.72 ms / 86 runs ( 1319.82 ms per run)
whisper_print_timings: decode time = 174960.73 ms / 5925 runs ( 29.53 ms per run)
whisper_print_timings: batchd time = 50.56 ms / 11 runs ( 4.60 ms per run)
whisper_print_timings: prompt time = 19145.34 ms / 18584 runs ( 1.03 ms per run)
whisper_print_timings: total time = 319002.38 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/long/warmup_01.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 1800.0,
"audio_key": "long",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:23:48.286049+00:00",
"exit_code": 0,
"first_inference_latency_s": 6.273360041996057,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/warmup_01.log",
"run_index": 1,
"run_kind": "warmup",
"start_utc": "2026-03-08T19:18:28.986782+00:00",
"wall_clock_runtime_s": 319.29833287499787
}

95
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_01.log Normal file
View File

@ -0,0 +1,95 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.010 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav' (4800000 samples, 300.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:29.460] The Town Hose Story was a very strange and
[00:01:30.000 --> 00:01:57.000] forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."
[00:01:58.000 --> 00:02:08.000] But the following night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest.
[00:02:09.000 --> 00:02:29.000] Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mist.
[00:02:30.000 --> 00:02:41.000] After weaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record.
[00:02:41.000 --> 00:02:54.000] Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin.
[00:02:54.000 --> 00:02:56.000] End of footnote.
[00:02:57.000 --> 00:03:13.000] For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn.
[00:03:14.000 --> 00:03:27.000] Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put and which are duly answered at the time.
[00:03:28.000 --> 00:03:46.000] Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn.
[00:03:47.000 --> 00:03:59.000] She was somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hole than common.
[00:03:59.000 --> 00:04:02.000] They supposed a swordfish had stabbed her, gentlemen.
[00:04:03.000 --> 00:04:31.000] But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous, though indeed they could not find it after searching the hole as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals.
[00:04:32.000 --> 00:04:52.000] But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased. So much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired.
[00:04:53.000 --> 00:05:02.000] Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that it was a good thing.
whisper_print_timings: load time = 294.49 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 659.74 ms
whisper_print_timings: sample time = 810.78 ms / 1 runs ( 810.78 ms per run)
whisper_print_timings: encode time = 26512.70 ms / 16 runs ( 1657.04 ms per run)
whisper_print_timings: decode time = 31846.68 ms / 881 runs ( 36.15 ms per run)
whisper_print_timings: batchd time = 50.81 ms / 11 runs ( 4.62 ms per run)
whisper_print_timings: prompt time = 3453.35 ms / 2803 runs ( 1.23 ms per run)
whisper_print_timings: total time = 64042.66 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_01.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 300.0,
"audio_key": "medium",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:13:07.713150+00:00",
"exit_code": 0,
"first_inference_latency_s": 2.772809457994299,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_01.log",
"run_index": 1,
"run_kind": "measured",
"start_utc": "2026-03-08T19:12:03.385147+00:00",
"wall_clock_runtime_s": 64.32755733299564
}

1
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_01.transcript.txt Normal file
View File

@ -0,0 +1 @@
"The Town Hose Story" As told at the Golden Inn. The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part. It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick. For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men. The Town Hose Story was a very strange and forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy." But the following night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest. Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mist. After weaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record. Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin. End of footnote. For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn. Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put and which are duly answered at the time. Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn. She was somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hole than common. They supposed a swordfish had stabbed her, gentlemen. But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous, though indeed they could not find it after searching the hole as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals. But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased. So much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired. Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that it was a good thing.

95
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_02.log Normal file
View File

@ -0,0 +1,95 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.012 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav' (4800000 samples, 300.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:29.460] The Town Hose Story was a very strange and
[00:01:30.000 --> 00:01:57.000] forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."
[00:01:58.000 --> 00:02:08.000] But the following night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest.
[00:02:09.000 --> 00:02:29.000] Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mist.
[00:02:30.000 --> 00:02:41.000] After weaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record.
[00:02:41.000 --> 00:02:54.000] Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin.
[00:02:54.000 --> 00:02:56.000] End of footnote.
[00:02:57.000 --> 00:03:13.000] For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn.
[00:03:14.000 --> 00:03:27.000] Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put and which are duly answered at the time.
[00:03:28.000 --> 00:03:46.000] Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn.
[00:03:47.000 --> 00:03:59.000] She was somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hole than common.
[00:03:59.000 --> 00:04:02.000] They supposed a swordfish had stabbed her, gentlemen.
[00:04:03.000 --> 00:04:31.000] But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous, though indeed they could not find it after searching the hole as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals.
[00:04:32.000 --> 00:04:52.000] But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased. So much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired.
[00:04:53.000 --> 00:05:02.000] Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that it was a good thing.
whisper_print_timings: load time = 317.08 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 622.25 ms
whisper_print_timings: sample time = 824.27 ms / 1 runs ( 824.27 ms per run)
whisper_print_timings: encode time = 32447.71 ms / 16 runs ( 2027.98 ms per run)
whisper_print_timings: decode time = 40943.01 ms / 881 runs ( 46.47 ms per run)
whisper_print_timings: batchd time = 66.66 ms / 11 runs ( 6.06 ms per run)
whisper_print_timings: prompt time = 4672.81 ms / 2803 runs ( 1.67 ms per run)
whisper_print_timings: total time = 80319.90 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_02.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 300.0,
"audio_key": "medium",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:14:28.432239+00:00",
"exit_code": 0,
"first_inference_latency_s": 3.64504562500224,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_02.log",
"run_index": 2,
"run_kind": "measured",
"start_utc": "2026-03-08T19:13:07.767446+00:00",
"wall_clock_runtime_s": 80.66440254100598
}

1
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_02.transcript.txt Normal file
View File

@ -0,0 +1 @@
"The Town Hose Story" As told at the Golden Inn. The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part. It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick. For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men. The Town Hose Story was a very strange and forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy." But the following night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest. Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mist. After weaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record. Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin. End of footnote. For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn. Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put and which are duly answered at the time. Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn. She was somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hole than common. They supposed a swordfish had stabbed her, gentlemen. But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous, though indeed they could not find it after searching the hole as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals. But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased. So much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired. Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that it was a good thing.

95
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_03.log Normal file
View File

@ -0,0 +1,95 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.012 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav' (4800000 samples, 300.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:29.460] The Town Hose Story was a very strange and
[00:01:30.000 --> 00:01:57.000] forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."
[00:01:58.000 --> 00:02:08.000] But the following night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest.
[00:02:09.000 --> 00:02:29.000] Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mist.
[00:02:30.000 --> 00:02:41.000] After weaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record.
[00:02:41.000 --> 00:02:54.000] Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin.
[00:02:54.000 --> 00:02:56.000] End of footnote.
[00:02:57.000 --> 00:03:13.000] For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn.
[00:03:14.000 --> 00:03:27.000] Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put and which are duly answered at the time.
[00:03:28.000 --> 00:03:46.000] Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn.
[00:03:47.000 --> 00:03:59.000] She was somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hole than common.
[00:03:59.000 --> 00:04:02.000] They supposed a swordfish had stabbed her, gentlemen.
[00:04:03.000 --> 00:04:31.000] But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous, though indeed they could not find it after searching the hole as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals.
[00:04:32.000 --> 00:04:52.000] But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased. So much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired.
[00:04:53.000 --> 00:05:02.000] Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that it was a good thing.
whisper_print_timings: load time = 396.58 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 614.12 ms
whisper_print_timings: sample time = 845.26 ms / 1 runs ( 845.26 ms per run)
whisper_print_timings: encode time = 32607.48 ms / 16 runs ( 2037.97 ms per run)
whisper_print_timings: decode time = 40475.29 ms / 881 runs ( 45.94 ms per run)
whisper_print_timings: batchd time = 56.52 ms / 11 runs ( 5.14 ms per run)
whisper_print_timings: prompt time = 4965.88 ms / 2803 runs ( 1.77 ms per run)
whisper_print_timings: total time = 80413.61 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_03.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 300.0,
"audio_key": "medium",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:15:49.175141+00:00",
"exit_code": 0,
"first_inference_latency_s": 3.6369052919981186,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_03.log",
"run_index": 3,
"run_kind": "measured",
"start_utc": "2026-03-08T19:14:28.498589+00:00",
"wall_clock_runtime_s": 80.67616358400119
}

1
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_03.transcript.txt Normal file
View File

@ -0,0 +1 @@
"The Town Hose Story" As told at the Golden Inn. The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part. It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick. For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men. The Town Hose Story was a very strange and forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy." But the following night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest. Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mist. After weaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record. Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin. End of footnote. For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn. Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put and which are duly answered at the time. Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn. She was somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hole than common. They supposed a swordfish had stabbed her, gentlemen. But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous, though indeed they could not find it after searching the hole as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals. But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased. So much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired. Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that it was a good thing.

95
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_04.log Normal file
View File

@ -0,0 +1,95 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.011 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav' (4800000 samples, 300.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:29.460] The Town Hose Story was a very strange and
[00:01:30.000 --> 00:01:57.000] forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."
[00:01:58.000 --> 00:02:08.000] But the following night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest.
[00:02:09.000 --> 00:02:29.000] Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mist.
[00:02:30.000 --> 00:02:41.000] After weaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record.
[00:02:41.000 --> 00:02:54.000] Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin.
[00:02:54.000 --> 00:02:56.000] End of footnote.
[00:02:57.000 --> 00:03:13.000] For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn.
[00:03:14.000 --> 00:03:27.000] Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put and which are duly answered at the time.
[00:03:28.000 --> 00:03:46.000] Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn.
[00:03:47.000 --> 00:03:59.000] She was somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hole than common.
[00:03:59.000 --> 00:04:02.000] They supposed a swordfish had stabbed her, gentlemen.
[00:04:03.000 --> 00:04:31.000] But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous, though indeed they could not find it after searching the hole as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals.
[00:04:32.000 --> 00:04:52.000] But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased. So much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired.
[00:04:53.000 --> 00:05:02.000] Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that it was a good thing.
whisper_print_timings: load time = 334.57 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 620.52 ms
whisper_print_timings: sample time = 863.87 ms / 1 runs ( 863.87 ms per run)
whisper_print_timings: encode time = 32469.40 ms / 16 runs ( 2029.34 ms per run)
whisper_print_timings: decode time = 40726.66 ms / 881 runs ( 46.23 ms per run)
whisper_print_timings: batchd time = 65.71 ms / 11 runs ( 5.97 ms per run)
whisper_print_timings: prompt time = 4584.38 ms / 2803 runs ( 1.64 ms per run)
whisper_print_timings: total time = 80115.46 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_04.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 300.0,
"audio_key": "medium",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:17:09.910160+00:00",
"exit_code": 0,
"first_inference_latency_s": 3.6302462079984252,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_04.log",
"run_index": 4,
"run_kind": "measured",
"start_utc": "2026-03-08T19:15:49.239114+00:00",
"wall_clock_runtime_s": 80.6706331250025
}

1
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_04.transcript.txt Normal file
View File

@ -0,0 +1 @@
"The Town Hose Story" As told at the Golden Inn. The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part. It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick. For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men. The Town Hose Story was a very strange and forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy." But the following night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest. Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mist. After weaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record. Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin. End of footnote. For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn. Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put and which are duly answered at the time. Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn. She was somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hole than common. They supposed a swordfish had stabbed her, gentlemen. But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous, though indeed they could not find it after searching the hole as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals. But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased. So much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired. Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that it was a good thing.

95
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_05.log Normal file
View File

@ -0,0 +1,95 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.012 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav' (4800000 samples, 300.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:29.460] The Town Hose Story was a very strange and
[00:01:30.000 --> 00:01:57.000] forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."
[00:01:58.000 --> 00:02:08.000] But the following night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest.
[00:02:09.000 --> 00:02:29.000] Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mist.
[00:02:30.000 --> 00:02:41.000] After weaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record.
[00:02:41.000 --> 00:02:54.000] Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin.
[00:02:54.000 --> 00:02:56.000] End of footnote.
[00:02:57.000 --> 00:03:13.000] For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn.
[00:03:14.000 --> 00:03:27.000] Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put and which are duly answered at the time.
[00:03:28.000 --> 00:03:46.000] Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn.
[00:03:47.000 --> 00:03:59.000] She was somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hole than common.
[00:03:59.000 --> 00:04:02.000] They supposed a swordfish had stabbed her, gentlemen.
[00:04:03.000 --> 00:04:31.000] But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous, though indeed they could not find it after searching the hole as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals.
[00:04:32.000 --> 00:04:52.000] But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased. So much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired.
[00:04:53.000 --> 00:05:02.000] Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that it was a good thing.
whisper_print_timings: load time = 514.21 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 685.91 ms
whisper_print_timings: sample time = 864.80 ms / 1 runs ( 864.80 ms per run)
whisper_print_timings: encode time = 32602.29 ms / 16 runs ( 2037.64 ms per run)
whisper_print_timings: decode time = 38833.69 ms / 881 runs ( 44.08 ms per run)
whisper_print_timings: batchd time = 57.78 ms / 11 runs ( 5.25 ms per run)
whisper_print_timings: prompt time = 4506.39 ms / 2803 runs ( 1.61 ms per run)
whisper_print_timings: total time = 78526.80 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_05.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 300.0,
"audio_key": "medium",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:18:28.874758+00:00",
"exit_code": 0,
"first_inference_latency_s": 3.8416501249957946,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_05.log",
"run_index": 5,
"run_kind": "measured",
"start_utc": "2026-03-08T19:17:09.970515+00:00",
"wall_clock_runtime_s": 78.9038724589991
}

1
benchmark/results/20260308_211026_metal-baseline/raw/medium/run_05.transcript.txt Normal file
View File

@ -0,0 +1 @@
"The Town Hose Story" As told at the Golden Inn. The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part. It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick. For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men. The Town Hose Story was a very strange and forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy." But the following night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest. Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mist. After weaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record. Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin. End of footnote. For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn. Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put and which are duly answered at the time. Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn. She was somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hole than common. They supposed a swordfish had stabbed her, gentlemen. But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous, though indeed they could not find it after searching the hole as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals. But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased. So much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired. Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that it was a good thing.

95
benchmark/results/20260308_211026_metal-baseline/raw/medium/warmup_01.log Normal file
View File

@ -0,0 +1,95 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.010 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav' (4800000 samples, 300.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:29.460] The Town Hose Story was a very strange and
[00:01:30.000 --> 00:01:57.000] forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."
[00:01:58.000 --> 00:02:08.000] But the following night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest.
[00:02:09.000 --> 00:02:29.000] Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mist.
[00:02:30.000 --> 00:02:41.000] After weaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record.
[00:02:41.000 --> 00:02:54.000] Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin.
[00:02:54.000 --> 00:02:56.000] End of footnote.
[00:02:57.000 --> 00:03:13.000] For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn.
[00:03:14.000 --> 00:03:27.000] Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put and which are duly answered at the time.
[00:03:28.000 --> 00:03:46.000] Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn.
[00:03:47.000 --> 00:03:59.000] She was somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hole than common.
[00:03:59.000 --> 00:04:02.000] They supposed a swordfish had stabbed her, gentlemen.
[00:04:03.000 --> 00:04:31.000] But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous, though indeed they could not find it after searching the hole as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals.
[00:04:32.000 --> 00:04:52.000] But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased. So much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired.
[00:04:53.000 --> 00:05:02.000] Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that it was a good thing.
whisper_print_timings: load time = 270.33 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 125.65 ms
whisper_print_timings: sample time = 833.71 ms / 1 runs ( 833.71 ms per run)
whisper_print_timings: encode time = 22246.27 ms / 16 runs ( 1390.39 ms per run)
whisper_print_timings: decode time = 30292.67 ms / 881 runs ( 34.38 ms per run)
whisper_print_timings: batchd time = 36.57 ms / 11 runs ( 3.32 ms per run)
whisper_print_timings: prompt time = 3564.09 ms / 2803 runs ( 1.27 ms per run)
whisper_print_timings: total time = 57805.10 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/medium/warmup_01.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 300.0,
"audio_key": "medium",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:12:03.332992+00:00",
"exit_code": 0,
"first_inference_latency_s": 2.342593166002189,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/warmup_01.log",
"run_index": 1,
"run_kind": "warmup",
"start_utc": "2026-03-08T19:11:05.087728+00:00",
"wall_clock_runtime_s": 58.24492199999804
}

76
benchmark/results/20260308_211026_metal-baseline/raw/short/run_01.log Normal file
View File

@ -0,0 +1,76 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.010 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav' (480000 samples, 30.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
whisper_print_timings: load time = 271.83 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 16.27 ms
whisper_print_timings: sample time = 7.92 ms / 1 runs ( 7.92 ms per run)
whisper_print_timings: encode time = 1662.36 ms / 1 runs ( 1662.36 ms per run)
whisper_print_timings: decode time = 251.72 ms / 10 runs ( 25.17 ms per run)
whisper_print_timings: batchd time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 2233.08 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/short/run_01.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 30.0,
"audio_key": "short",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:10:54.315673+00:00",
"exit_code": 0,
"first_inference_latency_s": 2.312735416999203,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_01.log",
"run_index": 1,
"run_kind": "measured",
"start_utc": "2026-03-08T19:10:51.696784+00:00",
"wall_clock_runtime_s": 2.618632500001695
}

1
benchmark/results/20260308_211026_metal-baseline/raw/short/run_01.transcript.txt Normal file
View File

@ -0,0 +1 @@
"The Town Hose Story"

76
benchmark/results/20260308_211026_metal-baseline/raw/short/run_02.log Normal file
View File

@ -0,0 +1,76 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.012 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav' (480000 samples, 30.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
whisper_print_timings: load time = 297.72 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 17.15 ms
whisper_print_timings: sample time = 8.54 ms / 1 runs ( 8.54 ms per run)
whisper_print_timings: encode time = 1553.29 ms / 1 runs ( 1553.29 ms per run)
whisper_print_timings: decode time = 256.03 ms / 10 runs ( 25.60 ms per run)
whisper_print_timings: batchd time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 2157.06 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/short/run_02.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 30.0,
"audio_key": "short",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:10:56.982473+00:00",
"exit_code": 0,
"first_inference_latency_s": 2.2354854160003015,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_02.log",
"run_index": 2,
"run_kind": "measured",
"start_utc": "2026-03-08T19:10:54.371752+00:00",
"wall_clock_runtime_s": 2.6105057909953757
}

1
benchmark/results/20260308_211026_metal-baseline/raw/short/run_02.transcript.txt Normal file
View File

@ -0,0 +1 @@
"The Town Hose Story"

76
benchmark/results/20260308_211026_metal-baseline/raw/short/run_03.log Normal file
View File

@ -0,0 +1,76 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.011 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav' (480000 samples, 30.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
whisper_print_timings: load time = 307.39 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 14.35 ms
whisper_print_timings: sample time = 8.08 ms / 1 runs ( 8.08 ms per run)
whisper_print_timings: encode time = 1683.02 ms / 1 runs ( 1683.02 ms per run)
whisper_print_timings: decode time = 237.17 ms / 10 runs ( 23.72 ms per run)
whisper_print_timings: batchd time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 2273.92 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/short/run_03.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 30.0,
"audio_key": "short",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:10:59.647676+00:00",
"exit_code": 0,
"first_inference_latency_s": 2.351175958996464,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_03.log",
"run_index": 3,
"run_kind": "measured",
"start_utc": "2026-03-08T19:10:57.038280+00:00",
"wall_clock_runtime_s": 2.609167916998558
}

1
benchmark/results/20260308_211026_metal-baseline/raw/short/run_03.transcript.txt Normal file
View File

@ -0,0 +1 @@
"The Town Hose Story"

76
benchmark/results/20260308_211026_metal-baseline/raw/short/run_04.log Normal file
View File

@ -0,0 +1,76 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.012 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav' (480000 samples, 30.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
whisper_print_timings: load time = 315.30 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 20.75 ms
whisper_print_timings: sample time = 8.21 ms / 1 runs ( 8.21 ms per run)
whisper_print_timings: encode time = 1728.98 ms / 1 runs ( 1728.98 ms per run)
whisper_print_timings: decode time = 305.62 ms / 10 runs ( 30.56 ms per run)
whisper_print_timings: batchd time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 2410.01 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/short/run_04.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 30.0,
"audio_key": "short",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:11:02.312196+00:00",
"exit_code": 0,
"first_inference_latency_s": 2.487656583005446,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_04.log",
"run_index": 4,
"run_kind": "measured",
"start_utc": "2026-03-08T19:10:59.703634+00:00",
"wall_clock_runtime_s": 2.6083485830022255
}

1
benchmark/results/20260308_211026_metal-baseline/raw/short/run_04.transcript.txt Normal file
View File

@ -0,0 +1 @@
"The Town Hose Story"

76
benchmark/results/20260308_211026_metal-baseline/raw/short/run_05.log Normal file
View File

@ -0,0 +1,76 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.011 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav' (480000 samples, 30.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
whisper_print_timings: load time = 394.88 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 67.06 ms
whisper_print_timings: sample time = 8.15 ms / 1 runs ( 8.15 ms per run)
whisper_print_timings: encode time = 1430.30 ms / 1 runs ( 1430.30 ms per run)
whisper_print_timings: decode time = 241.39 ms / 10 runs ( 24.14 ms per run)
whisper_print_timings: batchd time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 2194.47 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/short/run_05.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 30.0,
"audio_key": "short",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:11:04.982512+00:00",
"exit_code": 0,
"first_inference_latency_s": 2.274417334003374,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_05.log",
"run_index": 5,
"run_kind": "measured",
"start_utc": "2026-03-08T19:11:02.368548+00:00",
"wall_clock_runtime_s": 2.6137177499986137
}

1
benchmark/results/20260308_211026_metal-baseline/raw/short/run_05.transcript.txt Normal file
View File

@ -0,0 +1 @@
"The Town Hose Story"

76
benchmark/results/20260308_211026_metal-baseline/raw/short/warmup_01.log Normal file
View File

@ -0,0 +1,76 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 17.263 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav' (480000 samples, 30.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
whisper_print_timings: load time = 510.43 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 71.73 ms
whisper_print_timings: sample time = 5.64 ms / 1 runs ( 5.64 ms per run)
whisper_print_timings: encode time = 2086.68 ms / 1 runs ( 2086.68 ms per run)
whisper_print_timings: decode time = 385.76 ms / 10 runs ( 38.58 ms per run)
whisper_print_timings: batchd time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 3210.71 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_211026_metal-baseline/raw/short/warmup_01.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 30.0,
"audio_key": "short",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T19:10:51.623838+00:00",
"exit_code": 0,
"first_inference_latency_s": 24.324409958004253,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/warmup_01.log",
"run_index": 1,
"run_kind": "warmup",
"start_utc": "2026-03-08T19:10:26.967586+00:00",
"wall_clock_runtime_s": 24.65598445800424
}

16
benchmark/results/20260308_211026_metal-baseline/runs.csv Normal file
View File

@ -0,0 +1,16 @@
variant,model,audio_key,audio_length_s,run_kind,run_index,wall_clock_runtime_s,tokens_per_second,audio_seconds_per_second,model_load_ms,first_inference_latency_s,full_runtime_ms,mel_ms,sample_ms,encode_ms,decode_ms,batchd_ms,prompt_ms,transcript_word_error_rate,transcript_char_error_rate,reference_present,reference_path,transcript_path,metal_kernel_runtime_ms,cpu_orchestration_ms,log_path
metal-baseline,ggml-small.en.bin,long,1800.0,measured,1,119.89503466699534,,15.013132153466096,280.92,1.9347949999937555,119378.98,1002.14,3432.77,46745.75,58407.11,19.53,7377.74,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/long.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_01.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_01.log
metal-baseline,ggml-small.en.bin,long,1800.0,measured,2,119.78003570900182,,15.027546029231353,275.32,1.8494081250028103,119247.56,978.92,3435.74,46621.23,58503.88,17.89,7292.03,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/long.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_02.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_02.log
metal-baseline,ggml-small.en.bin,long,1800.0,measured,3,119.85825262499566,,15.017739376125544,269.53,1.6381932919975952,119543.45,799.48,3429.35,46570.1,58910.39,20.71,7425.47,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/long.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_03.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_03.log
metal-baseline,ggml-small.en.bin,long,1800.0,measured,4,119.3234943330026,,15.085042640275365,257.67,1.7880880420052563,118916.23,960.72,3453.0,46440.52,58337.1,19.65,7312.95,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/long.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_04.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_04.log
metal-baseline,ggml-small.en.bin,long,1800.0,measured,5,122.33720404199994,,14.713430914949116,265.81,1.739509791994351,122060.63,891.28,3466.45,47112.68,60712.06,17.78,7477.49,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/long.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_05.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_05.log
metal-baseline,ggml-small.en.bin,medium,300.0,measured,1,64.32755733299564,,4.6636311471774246,294.49,2.772809457994299,64042.66,659.74,810.78,26512.7,31846.68,50.81,3453.35,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/medium.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_01.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_01.log
metal-baseline,ggml-small.en.bin,medium,300.0,measured,2,80.66440254100598,,3.7191126513023405,317.08,3.64504562500224,80319.9,622.25,824.27,32447.71,40943.01,66.66,4672.81,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/medium.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_02.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_02.log
metal-baseline,ggml-small.en.bin,medium,300.0,measured,3,80.67616358400119,,3.718570475746974,396.58,3.6369052919981186,80413.61,614.12,845.26,32607.48,40475.29,56.52,4965.88,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/medium.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_03.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_03.log
metal-baseline,ggml-small.en.bin,medium,300.0,measured,4,80.6706331250025,,3.7188254062062156,334.57,3.6302462079984252,80115.46,620.52,863.87,32469.4,40726.66,65.71,4584.38,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/medium.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_04.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_04.log
metal-baseline,ggml-small.en.bin,medium,300.0,measured,5,78.9038724589991,,3.802094759745655,514.21,3.8416501249957946,78526.8,685.91,864.8,32602.29,38833.69,57.78,4506.39,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/medium.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_05.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_05.log
metal-baseline,ggml-small.en.bin,short,30.0,measured,1,2.618632500001695,,11.456361287802157,271.83,2.312735416999203,2233.08,16.27,7.92,1662.36,251.72,0.0,0.0,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/short.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_01.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_01.log
metal-baseline,ggml-small.en.bin,short,30.0,measured,2,2.6105057909953757,,11.492025837859229,297.72,2.2354854160003015,2157.06,17.15,8.54,1553.29,256.03,0.0,0.0,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/short.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_02.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_02.log
metal-baseline,ggml-small.en.bin,short,30.0,measured,3,2.609167916998558,,11.497918476059729,307.39,2.351175958996464,2273.92,14.35,8.08,1683.02,237.17,0.0,0.0,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/short.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_03.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_03.log
metal-baseline,ggml-small.en.bin,short,30.0,measured,4,2.6083485830022255,,11.501530200181225,315.3,2.487656583005446,2410.01,20.75,8.21,1728.98,305.62,0.0,0.0,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/short.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_04.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_04.log
metal-baseline,ggml-small.en.bin,short,30.0,measured,5,2.6137177499986137,,11.477903457638421,394.88,2.274417334003374,2194.47,67.06,8.15,1430.3,241.39,0.0,0.0,0.0,0.0,True,/Users/shaihi/Downloads/whisper_optimization/benchmark/references/short.txt,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_05.transcript.txt,,,/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_05.log
1 variant model audio_key audio_length_s run_kind run_index wall_clock_runtime_s tokens_per_second audio_seconds_per_second model_load_ms first_inference_latency_s full_runtime_ms mel_ms sample_ms encode_ms decode_ms batchd_ms prompt_ms transcript_word_error_rate transcript_char_error_rate reference_present reference_path transcript_path metal_kernel_runtime_ms cpu_orchestration_ms log_path
2 metal-baseline ggml-small.en.bin long 1800.0 measured 1 119.89503466699534 15.013132153466096 280.92 1.9347949999937555 119378.98 1002.14 3432.77 46745.75 58407.11 19.53 7377.74 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/long.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_01.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_01.log
3 metal-baseline ggml-small.en.bin long 1800.0 measured 2 119.78003570900182 15.027546029231353 275.32 1.8494081250028103 119247.56 978.92 3435.74 46621.23 58503.88 17.89 7292.03 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/long.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_02.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_02.log
4 metal-baseline ggml-small.en.bin long 1800.0 measured 3 119.85825262499566 15.017739376125544 269.53 1.6381932919975952 119543.45 799.48 3429.35 46570.1 58910.39 20.71 7425.47 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/long.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_03.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_03.log
5 metal-baseline ggml-small.en.bin long 1800.0 measured 4 119.3234943330026 15.085042640275365 257.67 1.7880880420052563 118916.23 960.72 3453.0 46440.52 58337.1 19.65 7312.95 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/long.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_04.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_04.log
6 metal-baseline ggml-small.en.bin long 1800.0 measured 5 122.33720404199994 14.713430914949116 265.81 1.739509791994351 122060.63 891.28 3466.45 47112.68 60712.06 17.78 7477.49 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/long.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_05.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/long/run_05.log
7 metal-baseline ggml-small.en.bin medium 300.0 measured 1 64.32755733299564 4.6636311471774246 294.49 2.772809457994299 64042.66 659.74 810.78 26512.7 31846.68 50.81 3453.35 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/medium.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_01.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_01.log
8 metal-baseline ggml-small.en.bin medium 300.0 measured 2 80.66440254100598 3.7191126513023405 317.08 3.64504562500224 80319.9 622.25 824.27 32447.71 40943.01 66.66 4672.81 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/medium.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_02.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_02.log
9 metal-baseline ggml-small.en.bin medium 300.0 measured 3 80.67616358400119 3.718570475746974 396.58 3.6369052919981186 80413.61 614.12 845.26 32607.48 40475.29 56.52 4965.88 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/medium.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_03.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_03.log
10 metal-baseline ggml-small.en.bin medium 300.0 measured 4 80.6706331250025 3.7188254062062156 334.57 3.6302462079984252 80115.46 620.52 863.87 32469.4 40726.66 65.71 4584.38 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/medium.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_04.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_04.log
11 metal-baseline ggml-small.en.bin medium 300.0 measured 5 78.9038724589991 3.802094759745655 514.21 3.8416501249957946 78526.8 685.91 864.8 32602.29 38833.69 57.78 4506.39 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/medium.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_05.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/medium/run_05.log
12 metal-baseline ggml-small.en.bin short 30.0 measured 1 2.618632500001695 11.456361287802157 271.83 2.312735416999203 2233.08 16.27 7.92 1662.36 251.72 0.0 0.0 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/short.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_01.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_01.log
13 metal-baseline ggml-small.en.bin short 30.0 measured 2 2.6105057909953757 11.492025837859229 297.72 2.2354854160003015 2157.06 17.15 8.54 1553.29 256.03 0.0 0.0 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/short.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_02.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_02.log
14 metal-baseline ggml-small.en.bin short 30.0 measured 3 2.609167916998558 11.497918476059729 307.39 2.351175958996464 2273.92 14.35 8.08 1683.02 237.17 0.0 0.0 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/short.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_03.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_03.log
15 metal-baseline ggml-small.en.bin short 30.0 measured 4 2.6083485830022255 11.501530200181225 315.3 2.487656583005446 2410.01 20.75 8.21 1728.98 305.62 0.0 0.0 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/short.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_04.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_04.log
16 metal-baseline ggml-small.en.bin short 30.0 measured 5 2.6137177499986137 11.477903457638421 394.88 2.274417334003374 2194.47 67.06 8.15 1430.3 241.39 0.0 0.0 0.0 0.0 True /Users/shaihi/Downloads/whisper_optimization/benchmark/references/short.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_05.transcript.txt /Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_211026_metal-baseline/raw/short/run_05.log

4
benchmark/results/20260308_211026_metal-baseline/summary.csv Normal file
View File

@ -0,0 +1,4 @@
variant,model,audio_key,audio_length_s,runs,init_mean_ms,first_inference_mean_s,runtime_median_s,throughput_mean_audio_s_per_s,runtime_std_dev_s,reference_present,wer_median,cer_median,max_wer,max_cer,correctness_pass,notes
metal-baseline,ggml-small.en.bin,short,30.0,5,317.424,2.3322941418009577,2.6105057909953757,11.485147851908152,0.004198189036346631,True,0.0,0.0,0.02,0.02,True,tokens/s unavailable; encode mean=1611.59 ms; decode mean=258.39 ms; wer median=0.0000; cer median=0.0000
metal-baseline,ggml-small.en.bin,medium,300.0,5,371.386,3.5053313415977754,80.66440254100598,3.924446888035722,7.1522606904299195,True,0.0,0.0,0.02,0.02,True,tokens/s unavailable; encode mean=31327.92 ms; decode mean=38565.07 ms; wer median=0.0000; cer median=0.0000
metal-baseline,ggml-small.en.bin,long,1800.0,5,269.85,1.7899988501987536,119.85825262499566,14.971378222809495,1.1952551580019861,True,0.0,0.0,0.02,0.02,True,tokens/s unavailable; encode mean=46698.06 ms; decode mean=58974.11 ms; wer median=0.0000; cer median=0.0000
1 variant model audio_key audio_length_s runs init_mean_ms first_inference_mean_s runtime_median_s throughput_mean_audio_s_per_s runtime_std_dev_s reference_present wer_median cer_median max_wer max_cer correctness_pass notes
2 metal-baseline ggml-small.en.bin short 30.0 5 317.424 2.3322941418009577 2.6105057909953757 11.485147851908152 0.004198189036346631 True 0.0 0.0 0.02 0.02 True tokens/s unavailable; encode mean=1611.59 ms; decode mean=258.39 ms; wer median=0.0000; cer median=0.0000
3 metal-baseline ggml-small.en.bin medium 300.0 5 371.386 3.5053313415977754 80.66440254100598 3.924446888035722 7.1522606904299195 True 0.0 0.0 0.02 0.02 True tokens/s unavailable; encode mean=31327.92 ms; decode mean=38565.07 ms; wer median=0.0000; cer median=0.0000
4 metal-baseline ggml-small.en.bin long 1800.0 5 269.85 1.7899988501987536 119.85825262499566 14.971378222809495 1.1952551580019861 True 0.0 0.0 0.02 0.02 True tokens/s unavailable; encode mean=46698.06 ms; decode mean=58974.11 ms; wer median=0.0000; cer median=0.0000

242
benchmark/results/20260308_211026_metal-baseline/summary.json Normal file
View File

@ -0,0 +1,242 @@
[
{
"variant": "metal-baseline",
"model": "ggml-small.en.bin",
"audio_key": "short",
"audio_length_s": 30.0,
"runs": 5,
"model_load_ms": {
"mean": 317.424,
"median": 307.39,
"min": 271.83,
"max": 394.88,
"std_dev": 46.2916972037103
},
"first_inference_latency_s": {
"mean": 2.3322941418009577,
"median": 2.312735416999203,
"min": 2.2354854160003015,
"max": 2.487656583005446,
"std_dev": 0.09695128821408004
},
"wall_clock_runtime_s": {
"mean": 2.6120745081992935,
"median": 2.6105057909953757,
"min": 2.6083485830022255,
"max": 2.618632500001695,
"std_dev": 0.004198189036346631
},
"full_runtime_ms": {
"mean": 2253.708,
"median": 2233.08,
"min": 2157.06,
"max": 2410.01,
"std_dev": 97.614459840743
},
"throughput_audio_seconds_per_second": {
"mean": 11.485147851908152,
"median": 11.492025837859229,
"min": 11.456361287802157,
"max": 11.501530200181225,
"std_dev": 0.018438576329723858
},
"tokens_per_second": {
"mean": null,
"median": null,
"min": null,
"max": null,
"std_dev": null
},
"encode_ms": {
"mean": 1611.59,
"median": 1662.36,
"min": 1430.3,
"max": 1728.98,
"std_dev": 120.13035003694947
},
"decode_ms": {
"mean": 258.38599999999997,
"median": 251.72,
"min": 237.17,
"max": 305.62,
"std_dev": 27.477353038456968
},
"wer": {
"mean": 0.0,
"median": 0.0,
"min": 0.0,
"max": 0.0,
"std_dev": 0.0
},
"cer": {
"mean": 0.0,
"median": 0.0,
"min": 0.0,
"max": 0.0,
"std_dev": 0.0
},
"reference_present": true,
"correctness_pass": true,
"notes": "tokens/s unavailable; encode mean=1611.59 ms; decode mean=258.39 ms; wer median=0.0000; cer median=0.0000"
},
{
"variant": "metal-baseline",
"model": "ggml-small.en.bin",
"audio_key": "medium",
"audio_length_s": 300.0,
"runs": 5,
"model_load_ms": {
"mean": 371.386,
"median": 334.57,
"min": 294.49,
"max": 514.21,
"std_dev": 88.38974335294793
},
"first_inference_latency_s": {
"mean": 3.5053313415977754,
"median": 3.6369052919981186,
"min": 2.772809457994299,
"max": 3.8416501249957946,
"std_dev": 0.41896722523467633
},
"wall_clock_runtime_s": {
"mean": 77.04852580840088,
"median": 80.66440254100598,
"min": 64.32755733299564,
"max": 80.67616358400119,
"std_dev": 7.1522606904299195
},
"full_runtime_ms": {
"mean": 76683.686,
"median": 80115.46,
"min": 64042.66,
"max": 80413.61,
"std_dev": 7108.165474233136
},
"throughput_audio_seconds_per_second": {
"mean": 3.924446888035722,
"median": 3.7191126513023405,
"min": 3.718570475746974,
"max": 4.6636311471774246,
"std_dev": 0.41478634619569676
},
"tokens_per_second": {
"mean": null,
"median": null,
"min": null,
"max": null,
"std_dev": null
},
"encode_ms": {
"mean": 31327.916,
"median": 32469.4,
"min": 26512.7,
"max": 32607.48,
"std_dev": 2692.7932789818824
},
"decode_ms": {
"mean": 38565.066,
"median": 40475.29,
"min": 31846.68,
"max": 40943.01,
"std_dev": 3846.587166454181
},
"wer": {
"mean": 0.0,
"median": 0.0,
"min": 0.0,
"max": 0.0,
"std_dev": 0.0
},
"cer": {
"mean": 0.0,
"median": 0.0,
"min": 0.0,
"max": 0.0,
"std_dev": 0.0
},
"reference_present": true,
"correctness_pass": true,
"notes": "tokens/s unavailable; encode mean=31327.92 ms; decode mean=38565.07 ms; wer median=0.0000; cer median=0.0000"
},
{
"variant": "metal-baseline",
"model": "ggml-small.en.bin",
"audio_key": "long",
"audio_length_s": 1800.0,
"runs": 5,
"model_load_ms": {
"mean": 269.85,
"median": 269.53,
"min": 257.67,
"max": 280.92,
"std_dev": 8.905647084855763
},
"first_inference_latency_s": {
"mean": 1.7899988501987536,
"median": 1.7880880420052563,
"min": 1.6381932919975952,
"max": 1.9347949999937555,
"std_dev": 0.11190745128820256
},
"wall_clock_runtime_s": {
"mean": 120.23880427519907,
"median": 119.85825262499566,
"min": 119.3234943330026,
"max": 122.33720404199994,
"std_dev": 1.1952551580019861
},
"full_runtime_ms": {
"mean": 119829.37,
"median": 119378.98,
"min": 118916.23,
"max": 122060.63,
"std_dev": 1268.4094490542118
},
"throughput_audio_seconds_per_second": {
"mean": 14.971378222809495,
"median": 15.017739376125544,
"min": 14.713430914949116,
"max": 15.085042640275365,
"std_dev": 0.14705779745612452
},
"tokens_per_second": {
"mean": null,
"median": null,
"min": null,
"max": null,
"std_dev": null
},
"encode_ms": {
"mean": 46698.056,
"median": 46621.23,
"min": 46440.52,
"max": 47112.68,
"std_dev": 256.31403713023684
},
"decode_ms": {
"mean": 58974.108,
"median": 58503.88,
"min": 58337.1,
"max": 60712.06,
"std_dev": 996.6089295556201
},
"wer": {
"mean": 0.0,
"median": 0.0,
"min": 0.0,
"max": 0.0,
"std_dev": 0.0
},
"cer": {
"mean": 0.0,
"median": 0.0,
"min": 0.0,
"max": 0.0,
"std_dev": 0.0
},
"reference_present": true,
"correctness_pass": true,
"notes": "tokens/s unavailable; encode mean=46698.06 ms; decode mean=58974.11 ms; wer median=0.0000; cer median=0.0000"
}
]

5
benchmark/results/20260308_211026_metal-baseline/summary.md Normal file
View File

@ -0,0 +1,5 @@
| Variant | Model | Audio Length | Runs | Init Mean | First Inference Mean | Runtime Median | Throughput | Std Dev | Notes |
|---|---|---:|---:|---:|---:|---:|---:|---:|---|
| metal-baseline | ggml-small.en.bin | 30.000s | 5 | 317.42 ms | 2.332 s | 2.611 s | 11.485 audio-s/s | 0.004 s | tokens/s unavailable; encode mean=1611.59 ms; decode mean=258.39 ms; wer median=0.0000; cer median=0.0000 |
| metal-baseline | ggml-small.en.bin | 300.000s | 5 | 371.39 ms | 3.505 s | 80.664 s | 3.924 audio-s/s | 7.152 s | tokens/s unavailable; encode mean=31327.92 ms; decode mean=38565.07 ms; wer median=0.0000; cer median=0.0000 |
| metal-baseline | ggml-small.en.bin | 1800.000s | 5 | 269.85 ms | 1.790 s | 119.858 s | 14.971 audio-s/s | 1.195 s | tokens/s unavailable; encode mean=46698.06 ms; decode mean=58974.11 ms; wer median=0.0000; cer median=0.0000 |

22
benchmark/results/20260308_211026_metal-baseline/validated_inputs.json Normal file
View File

@ -0,0 +1,22 @@
{
"audio": {
"long": {
"duration_s": 1800.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav",
"sha256": "fd586cbcf50f12d8eca4908bd38a3cd71a0c146992923d28259376aa0e6f4367"
},
"medium": {
"duration_s": 300.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav",
"sha256": "a918fe0de597908c917d44c4bfc54cfbfad3dbf79a763cdc558b0dbd95fe2aca"
},
"short": {
"duration_s": 30.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"sha256": "bf6376be2d6749925b9fb53a194a55cd0330329f0425705e5b4e67a0dd1ebd3a"
}
},
"model_abs": "/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"model_rel": "models/ggml-small.en.bin",
"model_sha256": "c6138d6d58ecc8322097e0f987c32f1be8bb0a18532a3f88f734d1bbf9c41e5d"
}

62
benchmark/results/20260308_231357_metal-opt3-v1/config.json Normal file
View File

@ -0,0 +1,62 @@
{
"audio": {
"long": {
"duration_s": 1800.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav",
"sha256": "fd586cbcf50f12d8eca4908bd38a3cd71a0c146992923d28259376aa0e6f4367"
},
"medium": {
"duration_s": 300.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav",
"sha256": "a918fe0de597908c917d44c4bfc54cfbfad3dbf79a763cdc558b0dbd95fe2aca"
},
"short": {
"duration_s": 30.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"sha256": "bf6376be2d6749925b9fb53a194a55cd0330329f0425705e5b4e67a0dd1ebd3a"
}
},
"cli": {
"args": [
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"binary": "/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"processors": 1,
"threads": 8
},
"created_at_utc": "2026-03-08T21:13:59.184796+00:00",
"environment": {
"clang_version": "Apple clang version 17.0.0 (clang-1700.6.4.2)\nTarget: arm64-apple-darwin25.3.0\nThread model: posix\nInstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin",
"cmake_version": "cmake version 4.2.3\n\nCMake suite maintained and supported by Kitware (kitware.com/cmake).",
"cpu_brand_string": "Apple M2",
"git_commit": "30c5194c9691e4e9a98b3dea9f19727397d3f46e",
"git_short_commit": "30c5194",
"hw_memsize": "17179869184",
"python_version": "3.13.2",
"sw_vers": "ProductName:\t\tmacOS\nProductVersion:\t\t26.3.1\nBuildVersion:\t\t25D2128",
"uname": "Darwin Shais-MacBook-Air.local 25.3.0 Darwin Kernel Version 25.3.0: Wed Jan 28 20:56:34 PST 2026; root:xnu-12377.91.3~2/RELEASE_ARM64_T8112 arm64",
"xcodebuild_version": "Xcode 26.3\nBuild version 17C529"
},
"model": {
"abs_path": "/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"rel_path": "models/ggml-small.en.bin",
"sha256": "c6138d6d58ecc8322097e0f987c32f1be8bb0a18532a3f88f734d1bbf9c41e5d"
},
"run_policy": {
"measured_runs": 5,
"sequential_execution": true,
"warmup_runs": 1
},
"variant": "metal-opt3-v1"
}

95
benchmark/results/20260308_231357_metal-opt3-v1/raw/medium/run_01.log Normal file
View File

@ -0,0 +1,95 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.203 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav' (4800000 samples, 300.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:29.460] The Town Hose Story was a very strange and
[00:01:30.000 --> 00:01:57.000] forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."
[00:01:58.000 --> 00:02:08.000] But the following night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest.
[00:02:09.000 --> 00:02:29.000] Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mist.
[00:02:30.000 --> 00:02:41.000] After weaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record.
[00:02:41.000 --> 00:02:54.000] Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin.
[00:02:54.000 --> 00:02:56.000] End of footnote.
[00:02:57.000 --> 00:03:13.000] For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn.
[00:03:14.000 --> 00:03:27.000] Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put and which are duly answered at the time.
[00:03:28.000 --> 00:03:46.000] Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn.
[00:03:47.000 --> 00:03:59.000] She was somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hole than common.
[00:03:59.000 --> 00:04:02.000] They supposed a swordfish had stabbed her, gentlemen.
[00:04:03.000 --> 00:04:31.000] But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous, though indeed they could not find it after searching the hole as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals.
[00:04:32.000 --> 00:04:52.000] But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased. So much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired.
[00:04:53.000 --> 00:05:02.000] Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that it was a good thing.
whisper_print_timings: load time = 745.30 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 697.43 ms
whisper_print_timings: sample time = 1419.03 ms / 1 runs ( 1419.03 ms per run)
whisper_print_timings: encode time = 36139.71 ms / 16 runs ( 2258.73 ms per run)
whisper_print_timings: decode time = 52301.55 ms / 881 runs ( 59.37 ms per run)
whisper_print_timings: batchd time = 82.54 ms / 11 runs ( 7.50 ms per run)
whisper_print_timings: prompt time = 4769.20 ms / 2803 runs ( 1.70 ms per run)
whisper_print_timings: total time = 96877.95 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_231357_metal-opt3-v1/raw/medium/run_01.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 300.0,
"audio_key": "medium",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T21:17:45.743008+00:00",
"exit_code": 0,
"first_inference_latency_s": 4.968868041003589,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_231357_metal-opt3-v1/raw/medium/run_01.log",
"run_index": 1,
"run_kind": "measured",
"start_utc": "2026-03-08T21:16:08.093638+00:00",
"wall_clock_runtime_s": 97.64872120800283
}

71
benchmark/results/20260308_231357_metal-opt3-v1/raw/medium/run_02.log Normal file
View File

@ -0,0 +1,71 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.014 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav' (4800000 samples, 300.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:29.460] The Town Hose Story was a very strange and
[00:01:30.000 --> 00:01:57.000] forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."

95
benchmark/results/20260308_231357_metal-opt3-v1/raw/medium/warmup_01.log Normal file
View File

@ -0,0 +1,95 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.012 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav' (4800000 samples, 300.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
[00:00:30.000 --> 00:00:32.000] As told at the Golden Inn.
[00:00:32.000 --> 00:00:46.000] The Cape of Good Hope, and all the watery region about there, is much like some noted four corners of a great highway, where you meet more travelers than in any other part.
[00:00:47.000 --> 00:01:03.000] It was not very long after speaking the Goni that another homeward bound whaleman, the Town Hoe, was encountered. She was manned almost wholly by Polynesians. In the short gam that ensued she gave us strong news of Moby Dick.
[00:01:04.000 --> 00:01:25.000] For some the general interest in the white whale was now wildly heightened by a circumstance of the Town Hose Story, which seemed obscurely to involve with the whale a certain wondrous inverted visitation of one of those so-called "judgments of God" which are at times said to overtake some men.
[00:01:26.000 --> 00:01:29.460] The Town Hose Story was a very strange and
[00:01:30.000 --> 00:01:57.000] forming what may be called the "secret part" of the tragedy about to be narrated, never reached the ears of Captain Ahab or his mates. For that secret part of the story was unknown to the Captain of the Town Hoe himself. It was the private property of three Confederate white seamen of that ship, one of whom, it seems, communicated it to Tashtego with "romish injunctions of secrecy."
[00:01:58.000 --> 00:02:08.000] But the following night Tashtego rambled in his sleep and revealed so much of it in that way that when he was wakened he could not well withhold the rest.
[00:02:09.000 --> 00:02:29.000] Nevertheless, so potent an influence did this thing have on those seamen in the Pequod who came to the full knowledge of it, and by such a strange delicacy, to call it so, were they governed in this matter, that they kept the secret among themselves so that it never transpired a bath the Pequod's main-mist.
[00:02:30.000 --> 00:02:41.000] After weaving in its proper place this darker thread with the story as publicly narrated on the ship, the whole of this strange affair I now proceed to put on lasting record.
[00:02:41.000 --> 00:02:54.000] Footnote. Town Hoe. The ancient whale cry upon first sighting a whale from the masthead, still used by whalemen in hunting the famous Galapagos Terrapin.
[00:02:54.000 --> 00:02:56.000] End of footnote.
[00:02:57.000 --> 00:03:13.000] For my humor's sake I shall preserve the style in which I once narrated it at Lima, to a lounging circle of my Spanish friends, one Saint's Eve, smoking upon the thick-guilt, tiled piazza of the Golden Inn.
[00:03:14.000 --> 00:03:27.000] Of those fine cavaliers the young dons Pedro and Sebastian were on closer terms with me, and hence the interluding questions they occasionally put and which are duly answered at the time.
[00:03:28.000 --> 00:03:46.000] Quote. Some two years prior to my first learning the events, which I am about rehearsing to you, gentlemen, the Town Hoe, sperm-whaler of Nantucket, was cruising in your Pacific here, not very many days sail eastward from the eaves of this good Golden Inn.
[00:03:47.000 --> 00:03:59.000] She was somewhere to the northward of the line. One morning upon handling the pumps, according to daily usage, it was observed that she made more water in her hole than common.
[00:03:59.000 --> 00:04:02.000] They supposed a swordfish had stabbed her, gentlemen.
[00:04:03.000 --> 00:04:31.000] But the captain, having some unusual reason for believing that rare good luck awaited him in those latitudes, and therefore being very averse to quit them, and the leak not being then considered at all dangerous, though indeed they could not find it after searching the hole as low down as was possible in rather heavy weather, the ship still continued her cruisings, the mariners working at the pumps at wide and easy intervals.
[00:04:32.000 --> 00:04:52.000] But no good luck came, more days went by, and not only was the leak yet undiscovered, but it sensibly increased. So much so that, now taking some alarm, the captain, making all sail, stood away for the nearest harbor among the islands, there to have his hull hove out and repaired.
[00:04:53.000 --> 00:05:02.000] Though no small passage was before her, yet if the commonest chance favored, he did not at all fear that it was a good thing.
whisper_print_timings: load time = 385.25 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 635.19 ms
whisper_print_timings: sample time = 1051.10 ms / 1 runs ( 1051.10 ms per run)
whisper_print_timings: encode time = 40170.70 ms / 16 runs ( 2510.67 ms per run)
whisper_print_timings: decode time = 52401.66 ms / 881 runs ( 59.48 ms per run)
whisper_print_timings: batchd time = 127.71 ms / 11 runs ( 11.61 ms per run)
whisper_print_timings: prompt time = 5732.22 ms / 2803 runs ( 2.05 ms per run)
whisper_print_timings: total time = 100997.02 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_231357_metal-opt3-v1/raw/medium/warmup_01.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 300.0,
"audio_key": "medium",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T21:16:08.008385+00:00",
"exit_code": 0,
"first_inference_latency_s": 4.139124041998002,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_231357_metal-opt3-v1/raw/medium/warmup_01.log",
"run_index": 1,
"run_kind": "warmup",
"start_utc": "2026-03-08T21:14:26.654124+00:00",
"wall_clock_runtime_s": 101.35375291699893
}

76
benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_01.log Normal file
View File

@ -0,0 +1,76 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.011 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav' (480000 samples, 30.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
whisper_print_timings: load time = 321.53 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 150.37 ms
whisper_print_timings: sample time = 7.43 ms / 1 runs ( 7.43 ms per run)
whisper_print_timings: encode time = 2139.61 ms / 1 runs ( 2139.61 ms per run)
whisper_print_timings: decode time = 740.87 ms / 10 runs ( 74.09 ms per run)
whisper_print_timings: batchd time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 3401.16 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_01.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 30.0,
"audio_key": "short",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T21:14:10.309865+00:00",
"exit_code": 0,
"first_inference_latency_s": 3.483151624997845,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_01.log",
"run_index": 1,
"run_kind": "measured",
"start_utc": "2026-03-08T21:14:06.668757+00:00",
"wall_clock_runtime_s": 3.640878165999311
}

76
benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_02.log Normal file
View File

@ -0,0 +1,76 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.016 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav' (480000 samples, 30.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
whisper_print_timings: load time = 591.40 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 60.26 ms
whisper_print_timings: sample time = 13.06 ms / 1 runs ( 13.06 ms per run)
whisper_print_timings: encode time = 2427.73 ms / 1 runs ( 2427.73 ms per run)
whisper_print_timings: decode time = 544.04 ms / 10 runs ( 54.40 ms per run)
whisper_print_timings: batchd time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 3703.58 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_02.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 30.0,
"audio_key": "short",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T21:14:14.531205+00:00",
"exit_code": 0,
"first_inference_latency_s": 3.8210939999989932,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_02.log",
"run_index": 2,
"run_kind": "measured",
"start_utc": "2026-03-08T21:14:10.370542+00:00",
"wall_clock_runtime_s": 4.160437375001493
}

76
benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_03.log Normal file
View File

@ -0,0 +1,76 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.012 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav' (480000 samples, 30.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
whisper_print_timings: load time = 588.23 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 74.58 ms
whisper_print_timings: sample time = 13.38 ms / 1 runs ( 13.38 ms per run)
whisper_print_timings: encode time = 2275.01 ms / 1 runs ( 2275.01 ms per run)
whisper_print_timings: decode time = 566.85 ms / 10 runs ( 56.69 ms per run)
whisper_print_timings: batchd time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 3574.86 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_03.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 30.0,
"audio_key": "short",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T21:14:18.710909+00:00",
"exit_code": 0,
"first_inference_latency_s": 3.665108875000442,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_03.log",
"run_index": 3,
"run_kind": "measured",
"start_utc": "2026-03-08T21:14:14.582355+00:00",
"wall_clock_runtime_s": 4.1283463340005255
}

76
benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_04.log Normal file
View File

@ -0,0 +1,76 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.012 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav' (480000 samples, 30.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
whisper_print_timings: load time = 353.44 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 56.85 ms
whisper_print_timings: sample time = 9.04 ms / 1 runs ( 9.04 ms per run)
whisper_print_timings: encode time = 2497.57 ms / 1 runs ( 2497.57 ms per run)
whisper_print_timings: decode time = 629.52 ms / 10 runs ( 62.95 ms per run)
whisper_print_timings: batchd time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 3589.96 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_04.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 30.0,
"audio_key": "short",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T21:14:22.885942+00:00",
"exit_code": 0,
"first_inference_latency_s": 3.6701512500003446,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_04.log",
"run_index": 4,
"run_kind": "measured",
"start_utc": "2026-03-08T21:14:18.770925+00:00",
"wall_clock_runtime_s": 4.114808042002551
}

76
benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_05.log Normal file
View File

@ -0,0 +1,76 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.011 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav' (480000 samples, 30.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
whisper_print_timings: load time = 287.35 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 16.84 ms
whisper_print_timings: sample time = 9.38 ms / 1 runs ( 9.38 ms per run)
whisper_print_timings: encode time = 2437.08 ms / 1 runs ( 2437.08 ms per run)
whisper_print_timings: decode time = 616.53 ms / 10 runs ( 61.65 ms per run)
whisper_print_timings: batchd time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 3392.36 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_05.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 30.0,
"audio_key": "short",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T21:14:26.541383+00:00",
"exit_code": 0,
"first_inference_latency_s": 3.465703707995999,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_231357_metal-opt3-v1/raw/short/run_05.log",
"run_index": 5,
"run_kind": "measured",
"start_utc": "2026-03-08T21:14:22.936840+00:00",
"wall_clock_runtime_s": 3.6043351249973057
}

76
benchmark/results/20260308_231357_metal-opt3-v1/raw/short/warmup_01.log Normal file
View File

@ -0,0 +1,76 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.039 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav' (480000 samples, 30.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
whisper_print_timings: load time = 767.65 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 71.04 ms
whisper_print_timings: sample time = 8.46 ms / 1 runs ( 8.46 ms per run)
whisper_print_timings: encode time = 2617.27 ms / 1 runs ( 2617.27 ms per run)
whisper_print_timings: decode time = 783.30 ms / 10 runs ( 78.33 ms per run)
whisper_print_timings: batchd time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 4447.35 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_231357_metal-opt3-v1/raw/short/warmup_01.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 30.0,
"audio_key": "short",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T21:14:06.608389+00:00",
"exit_code": 0,
"first_inference_latency_s": 6.773231832994497,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_231357_metal-opt3-v1/raw/short/warmup_01.log",
"run_index": 1,
"run_kind": "warmup",
"start_utc": "2026-03-08T21:13:59.716590+00:00",
"wall_clock_runtime_s": 6.891371249999793
}

22
benchmark/results/20260308_231357_metal-opt3-v1/validated_inputs.json Normal file
View File

@ -0,0 +1,22 @@
{
"audio": {
"long": {
"duration_s": 1800.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav",
"sha256": "fd586cbcf50f12d8eca4908bd38a3cd71a0c146992923d28259376aa0e6f4367"
},
"medium": {
"duration_s": 300.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav",
"sha256": "a918fe0de597908c917d44c4bfc54cfbfad3dbf79a763cdc558b0dbd95fe2aca"
},
"short": {
"duration_s": 30.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"sha256": "bf6376be2d6749925b9fb53a194a55cd0330329f0425705e5b4e67a0dd1ebd3a"
}
},
"model_abs": "/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"model_rel": "models/ggml-small.en.bin",
"model_sha256": "c6138d6d58ecc8322097e0f987c32f1be8bb0a18532a3f88f734d1bbf9c41e5d"
}

62
benchmark/results/20260308_231832_metal-opt3-v1/config.json Normal file
View File

@ -0,0 +1,62 @@
{
"audio": {
"long": {
"duration_s": 1800.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/long.wav",
"sha256": "fd586cbcf50f12d8eca4908bd38a3cd71a0c146992923d28259376aa0e6f4367"
},
"medium": {
"duration_s": 300.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/medium.wav",
"sha256": "a918fe0de597908c917d44c4bfc54cfbfad3dbf79a763cdc558b0dbd95fe2aca"
},
"short": {
"duration_s": 30.0,
"path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"sha256": "bf6376be2d6749925b9fb53a194a55cd0330329f0425705e5b4e67a0dd1ebd3a"
}
},
"cli": {
"args": [
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"binary": "/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"processors": 1,
"threads": 8
},
"created_at_utc": "2026-03-08T21:18:33.644142+00:00",
"environment": {
"clang_version": "Apple clang version 17.0.0 (clang-1700.6.4.2)\nTarget: arm64-apple-darwin25.3.0\nThread model: posix\nInstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin",
"cmake_version": "cmake version 4.2.3\n\nCMake suite maintained and supported by Kitware (kitware.com/cmake).",
"cpu_brand_string": "Apple M2",
"git_commit": "30c5194c9691e4e9a98b3dea9f19727397d3f46e",
"git_short_commit": "30c5194",
"hw_memsize": "17179869184",
"python_version": "3.13.2",
"sw_vers": "ProductName:\t\tmacOS\nProductVersion:\t\t26.3.1\nBuildVersion:\t\t25D2128",
"uname": "Darwin Shais-MacBook-Air.local 25.3.0 Darwin Kernel Version 25.3.0: Wed Jan 28 20:56:34 PST 2026; root:xnu-12377.91.3~2/RELEASE_ARM64_T8112 arm64",
"xcodebuild_version": "Xcode 26.3\nBuild version 17C529"
},
"model": {
"abs_path": "/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"rel_path": "models/ggml-small.en.bin",
"sha256": "c6138d6d58ecc8322097e0f987c32f1be8bb0a18532a3f88f734d1bbf9c41e5d"
},
"run_policy": {
"measured_runs": 5,
"sequential_execution": true,
"warmup_runs": 1
},
"variant": "metal-opt3-v1"
}

17
benchmark/results/20260308_231832_metal-opt3-v1/correctness.json Normal file
View File

@ -0,0 +1,17 @@
{
"refs_dir": "/Users/shaihi/Downloads/whisper_optimization/benchmark/references",
"max_wer": 0.02,
"max_cer": 0.02,
"enforce_correctness": true,
"missing_references": [],
"overall_correctness_pass": true,
"audios": [
{
"audio_key": "short",
"reference_present": true,
"wer_median": 0.0,
"cer_median": 0.0,
"correctness_pass": true
}
]
}

76
benchmark/results/20260308_231832_metal-opt3-v1/raw/short/run_01.log Normal file
View File

@ -0,0 +1,76 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.012 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav' (480000 samples, 30.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
whisper_print_timings: load time = 498.80 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 90.04 ms
whisper_print_timings: sample time = 9.40 ms / 1 runs ( 9.40 ms per run)
whisper_print_timings: encode time = 2277.09 ms / 1 runs ( 2277.09 ms per run)
whisper_print_timings: decode time = 493.05 ms / 10 runs ( 49.31 ms per run)
whisper_print_timings: batchd time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 3410.10 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_231832_metal-opt3-v1/raw/short/run_01.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 30.0,
"audio_key": "short",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T21:18:42.113477+00:00",
"exit_code": 0,
"first_inference_latency_s": 3.4916511670016916,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_231832_metal-opt3-v1/raw/short/run_01.log",
"run_index": 1,
"run_kind": "measured",
"start_utc": "2026-03-08T21:18:38.496172+00:00",
"wall_clock_runtime_s": 3.6170433750012307
}

1
benchmark/results/20260308_231832_metal-opt3-v1/raw/short/run_01.transcript.txt Normal file
View File

@ -0,0 +1 @@
"The Town Hose Story"

76
benchmark/results/20260308_231832_metal-opt3-v1/raw/short/run_02.log Normal file
View File

@ -0,0 +1,76 @@
whisper_init_from_file_with_params_no_state: loading model from '/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.013 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 12713.12 MB
whisper_init_with_params_no_state: devices = 3
whisper_init_with_params_no_state: backends = 3
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 768
whisper_model_load: n_text_head = 12
whisper_model_load: n_text_layer = 12
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 3 (small)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: MTL0 total size = 487.00 MB
whisper_model_load: model size = 487.00 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: use fusion = true
ggml_metal_init: use concurrency = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size = 18.87 MB
whisper_init_state: kv cross size = 56.62 MB
whisper_init_state: kv pad size = 4.72 MB
whisper_init_state: compute buffer (conv) = 23.38 MB
whisper_init_state: compute buffer (encode) = 39.45 MB
whisper_init_state: compute buffer (cross) = 79.93 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 8 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
main: processing '/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav' (480000 samples, 30.0 sec), 8 threads, 1 processors, 1 beams + best of 1, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:28.720] "The Town Hose Story"
whisper_print_timings: load time = 575.81 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 73.88 ms
whisper_print_timings: sample time = 8.93 ms / 1 runs ( 8.93 ms per run)
whisper_print_timings: encode time = 2347.38 ms / 1 runs ( 2347.38 ms per run)
whisper_print_timings: decode time = 570.94 ms / 10 runs ( 57.09 ms per run)
whisper_print_timings: batchd time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 3632.84 ms
ggml_metal_free: deallocating

35
benchmark/results/20260308_231832_metal-opt3-v1/raw/short/run_02.meta.json Normal file
View File

@ -0,0 +1,35 @@
{
"audio_duration_s": 30.0,
"audio_key": "short",
"command": [
"/Users/shaihi/Downloads/whisper_optimization/build/bin/whisper-cli",
"-m",
"/Users/shaihi/Downloads/whisper_optimization/models/ggml-small.en.bin",
"-f",
"/Users/shaihi/Downloads/whisper_optimization/benchmark/audio/short.wav",
"-t",
"8",
"-p",
"1",
"-l",
"en",
"-tp",
"0",
"-tpi",
"0",
"-nf",
"-bs",
"1",
"-bo",
"1",
"-fa"
],
"end_utc": "2026-03-08T21:18:46.305542+00:00",
"exit_code": 0,
"first_inference_latency_s": 3.7216445000012754,
"log_path": "/Users/shaihi/Downloads/whisper_optimization/benchmark/results/20260308_231832_metal-opt3-v1/raw/short/run_02.log",
"run_index": 2,
"run_kind": "measured",
"start_utc": "2026-03-08T21:18:42.170887+00:00",
"wall_clock_runtime_s": 4.134414500003913
}

Some files were not shown because too many files have changed in this diff Show More