mirror of
https://github.com/ggml-org/whisper.cpp.git
synced 2026-10-20 17:23:00 +02:00
metal : fix MUL_MAT+ADD fusion when the residual is itself a MUL_MAT (llama/30100)
* metal : fix MUL_MAT+ADD fusion when the residual is itself a MUL_MAT ggml_metal_op_mul_mat_mma picks the residual of a fused MUL_MAT+ADD as "the ADD operand whose op is not MUL_MAT". When both operands of the ADD are mat-mul outputs (x = W1 @ u + W2 @ v), that test is true for both, so the residual resolves to the fused mat-mul's own, never-written output and the kernel adds whatever that buffer holds. The fusion check (ggml_metal_mul_mat_add_operand) already selects the operand by identity; make the encoder do the same. Clef decision models hit this in their head (proj_option_context @ ctx + proj_option_lexical @ lex, 9 option rows): on Metal, /v1/systemone probabilities collapse toward uniform (billing 0.28 where the CPU backend gives 0.977, Cloudflare_clef-flash Q8_0), deterministic per memory layout, correct with GGML_METAL_FUSION_DISABLE=1. Not a quantization issue: the same file is right on CPU. Add a MUL_MAT_ADD mode to test-backend-ops where the residual is a second mat-mul; on Metal it fails 27 of 28 cases before this change (the one pass is f16 n=2, under the MMA row threshold, so nothing fuses). * Update tests/test-backend-ops.cpp Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
This commit is contained in:
co-authored by
Georgi Gerganov
parent
298d79dd9e
commit
50703d3c3a
@@ -2476,7 +2476,8 @@ static int ggml_metal_op_mul_mat_mma(ggml_metal_op_t ctx, int idx) {
|
||||
|
||||
if (fuse_add) {
|
||||
dst = ctx->node(idx + n_fuse - 1);
|
||||
res = dst->src[0]->op == GGML_OP_MUL_MAT ? dst->src[1] : dst->src[0];
|
||||
// the residual is the other operand of the ADD, by identity: it can itself be a MUL_MAT output
|
||||
res = dst->src[0] == op ? dst->src[1] : dst->src[0];
|
||||
}
|
||||
|
||||
auto pipeline = ggml_metal_library_get_pipeline_mul_mv_mma_auto(lib, op, fuse_add);
|
||||
|
||||
Reference in New Issue
Block a user