public_git
/
whisper.cpp
mirror of https://github.com/ggml-org/whisper.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

sycl: coalesce the ssm_conv window loads (llama/26612) 

test-backend-ops perf -o SSM_CONV on an Arc Pro B70, interleaved A/B against
master, 6 reps, us/run:

  ne_a=[515,3328,1,1] ne_b=[4,3328,1,1]   n_t=512     97.68 -> 52.95   1.85x
  ne_a=[937,8192,1,1] ne_b=[4,8192,1,1]   n_t=934    516.16 -> 276.13  1.87x
  ne_a=[4,3328,1,1]   ne_b=[4,3328,1,1]   n_t=1        2.73 -> 2.71    flat

llama-bench on qwen35 27B Q4_K - Medium (48 of its 64 blocks run ssm_conv),
-ngl 99 -fa 1 -ctk f16 -ctv f16, interleaved passes of r=3:

  -b 2048 -ub 2048  pp2048  1045.1 / 1043.5 / 1043.7 -> 1069.5 / 1066.3 / 1065.9  +2.2%
  -b 2048 -ub 512   pp2048   771.8 /  772.7          ->  785.5 /  786.6           +1.8%
  -b 2048 -ub 512   tg128     23.81 /  23.88         ->   23.87 /  23.86          flat
This commit is contained in:
Titaniumtown 2026-08-07 11:09:32 -07:00 committed by Georgi Gerganov
parent 8a5ba01c93
commit eb3296f278
1 changed files with 7 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
ggml/src/ggml-sycl/ssm_conv.cpp
View File

@ -36,9 +36,13 @@ static void kernel_ssm_conv(
return;
}
const int channel = static_cast<int>(idx % d_inner);
const int token = static_cast<int>((idx / d_inner) % n_t);
const int seq = static_cast<int>(idx / (static_cast<size_t>(d_inner) * static_cast<size_t>(n_t)));
// src has the tokens of one channel contiguous, dst has the channels of one
// token contiguous, so either the loads or the store must be strided. Indexing
// token-fastest coalesces the d_conv loads, which measured faster except for
// short, cache-resident rows.
const int token = static_cast<int>(idx % n_t);
const int channel = static_cast<int>((idx / n_t) % d_inner);
const int seq = static_cast<int>(idx / (static_cast<size_t>(n_t) * static_cast<size_t>(d_inner)));
const float *s = src_data
+ static_cast<size_t>(seq) * static_cast<size_t>(src_stride_seq)