public_git/whisper.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/whisper.cpp.git synced 2026-10-12 17:25:39 +02:00
Code Issues Packages Projects Releases Wiki Activity
5,411 Commits 67 Branches 95 Tags
396f68d33e0e471b249fb899a82896f4129fc02e
Commit Graph
5411 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Johannes Gäßler 396f68d33e CUDA: fix 2 broken Volta FA cases (llama/29803) 2026-10-06 10:38:15 +03:00
Johannes Gäßler 3a3598cc24 llama: refer to segment documentation [no ci] (llama/29074) 2026-10-06 10:38:15 +03:00
Yiwei ShaoandMax Krasnyansky 15229ac95d hexagon: shared strided DMA copy for CPY and CONCAT, any-dim CONCAT via DMA (llama/29685) 
* hexagon: shared strided DMA copy for CPY and CONCAT, any-dim CONCAT via DMA

* hex-cpy: various fixes on top of the concat optimizations

Removed CONCAT_DMA_MIN_ROW logic, it was broken with 64-bit DMA.
While it's kinda silly to use DMA for tiny stuff if that tensor gets mapped to an extended buffer the only way to read it is DMA.

Added missing dma_queue_flush() calls.

Added additional guards for conditions we don't support.

---------

Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
 2026-10-06 10:38:15 +03:00
Theera K.tkittichJohannes Gäßler
 		aa538005fc cuda : route sm70 to the Turing MMVQ nwarps table (llama/29753) 
* cuda : route sm70 to the Turing MMVQ nwarps table

Volta (sm_70) has no MMVQ parameter table of its own and falls through
to GENERIC, which launches K-quant batch-1 decode (ncols_dst == 1) at
nwarps=4. sm_70 shares TURING's tuning: the K-quant vec_dot prefers
nwarps=2 there. Route sm_70 to the existing MMVQ_PARAMETERS_TURING
table in both the device and the host table selector.

Measured on one Tesla V100 32GB PCIe (PG500-216, driver 580.178.04,
CUDA 12.0.140) with Qwen3.8-27B Q4_K_M, tg128, interleaved A/B in 6
ABBA blocks with paired per-block deltas: +1.091 t/s = +3.17 %
(t = +49.0, all six per-block deltas positive); perplexity
bit-identical (6.3697 +/- 0.04066 both builds, wiki.test.raw). The
patched build's K-quant mul_mat_vec_q kernels launch at nwarps=2
(cubin EIATTR_MAX_THREADS) while Q4_0/Q8_0 stay at nwarps=4, and the
same measurement on the September master base gave +3.84 % (t = 85).

The tuning originates from the V100-focused fork anyei/llamacpp-v100
(MIT), commit b912d1b1e, which carries a dedicated
MMVQ_PARAMETERS_VOLTA table; a cubin-level comparison confirmed that
routing sm_70 to the existing TURING table is equivalent for the
K-quant batch-1 path this change affects, so this is the minimal
2-line form. https://github.com/anyei/llamacpp-v100/commit/b912d1b1e

Original-patch-by: anyei <angelyoelroblesmercedes@gmail.com>

* Update ggml/src/ggml-cuda/mmvq.cu

---------

Co-authored-by: tkittich <tkittich@gmail.com>
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
 2026-10-06 10:38:15 +03:00
Mike van LammerenandNiklas Wenzel 81ca4f81fa metal : release temporary private transfer buffers (llama/29777) 
* metal : release temporary private transfer buffers

Assisted-by: OpenAI Codex

* metal : fix order and formatting

---------

Co-authored-by: Niklas Wenzel <dev@nikwen.de>
 2026-10-06 10:38:15 +03:00
Masashi Yoshimura 88948bdb83 webgpu: add bfloat16 support for MUL_MAT/MUL_MAT_ID/GET_ROWS- #29358 (llama/29358) 2026-10-06 10:38:14 +03:00
ynankaniandJohannes Gäßler 10872be7f8 CUDA: Handle compute type for NVFP4 on cublass path (llama/29173) 
* CUDA: Handle compute type for NVFP4 on cublass path

Signed-off-by: ynankani <ynankani@nvidia.com>

* Use BF16 compute type for quantized models if HW allows

Signed-off-by: ynankani <ynankani@nvidia.com>

* Set acc prec to bf16 for nvfp4 as it needs atleast bf16 range

Signed-off-by: ynankani <ynankani@nvidia.com>

* Update ggml/src/ggml-cuda/ggml-cuda.cu

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

* preserve op_params for per-expert matmul

Signed-off-by: ynankani <ynankani@nvidia.com>

---------

Signed-off-by: ynankani <ynankani@nvidia.com>
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
 2026-10-06 10:38:14 +03:00
Oliver Simons 3b68f9014c CUDA: Make CCCL configurable + pin it to 3.4.3 for CI jobs (llama/29792) 
Pinning to >= 3.4.3 is required to enable DeviceTopK, which was affected by
a race condition https://github.com/NVIDIA/cccl/pull/10627.

We will relax this for future CTK versions which will bundle CCCL >
3.4.X (CTK 13.5 will bundle CCCL 3.5.0 for example)
 2026-10-06 10:38:14 +03:00
Georgi Gerganov 56500f4f74 meta: clear inactive AllReduce shards with FILL, not SCALE (llama/29793) 2026-10-06 10:38:14 +03:00
uvos 817294a93e HIP: avoid treating CDNA as dgx spark for gqa_ratio 20 in fattn_mma dqk 576 (llama/29572) 2026-10-06 10:38:14 +03:00
Max Krasnyansky b4a085a024 hex-workqueue: fix race condition in seqn getting out of sync with idx_read/write (llama/29785) 2026-10-06 10:38:14 +03:00
Pradeep Rao 0ecc317e30 BLAS : Document AOCL-BLAS build and label the device AOCL-BLAS (llama/29640) 
* BLAS : Document AOCL-BLAS build and label the device AOCL-BLAS

* AOCL-Blas : Add an AOCL-BLAS Quick Start and drop the fixed version path

* AOCL-BLAS doc : Note on ZenDNN
 2026-10-06 10:38:13 +03:00
lhez 67ad86d020 opencl: mark vec subgroup bcast as supproted for Adreno E17 compiler (llama/29698) 2026-10-06 10:38:13 +03:00
Georgi Gerganov 61ea5027d5 metal : use bf16 math for mxfp4 mul-mat (llama/29770) 2026-10-06 10:38:13 +03:00
Masashi Yoshimura d371e376cb webgpu: fix SSM_SCAN binding aliasing (llama/29750) 2026-10-06 10:38:13 +03:00
Adrien Gallouët c1c139084c ggml-opencl : replace alloca() with std::vector (llama/29765) 
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-10-06 10:38:13 +03:00
R0CKSTAR bffb6e4b0f cuda: guard the iq4_nl dequantize row kernel against short rows (llama/29683) 
dequantize_block_iq4_nl writes QK_K values per block, but a row can be shorter than that (an IQ4_NL row is only guaranteed to be a multiple of QK4_NL). Threads whose 32-value sub-block starts at or past k currently read and write past the end of the row. Skip those sub-blocks; for rows that are a multiple of QK_K the check never fires.
 2026-10-06 10:38:13 +03:00
Ehsan BateniandMax Krasnyansky 6773ef8739 Hexagon: optimize ALLREDUCE with support for safe scatter mode (llama/29757) 
* hex-allreduce: add support for safe scatter mode

* hex-allreduce: pare down excessive comments

* hex-allreduce: re-write to remove register spills

---------

Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
 2026-10-06 10:38:12 +03:00
Daniel Kuts 02ea0c2ea8 ggml/gguf : fix integer overflow (llama/29384) 
* ggml: fix integer overflow guard for zero-element tensors

* ggml: validate number of elements in tensor to prevent integer overflow

* ggml: fix error print
 2026-10-06 10:38:12 +03:00
Adrien Gallouët 067bb06bd8 ggml-et : remove useless alloca() (llama/29663) 
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-10-06 10:38:12 +03:00
Aman Gupta fd66b6ecce ggml : add BF16 unary, GLU, binary and scale ops (CPU, CUDA) (llama/29675) 
* ggml : add BF16 unary, GLU, binary and scale ops (CPU, CUDA)

* ggml-cpu : use per-op _bf16 functions for BF16 unary and GLU ops

Assisted-by: Claude Opus 5.5

* CUDA: use ggml_cuda_cast in binbcast and unary kernels to fix the HIP bf16 build

* ggml-openvino : reject BF16 SCALE and mixed-type BF16 ADD/MUL/SUB
 2026-10-06 10:38:12 +03:00
Pascal c1b3fb1094 cpu: accept BF16 in src1 of mul_mat (llama/28937) 
* cpu: accept BF16 in src1 of mul_mat

ggml_conv_1d_dw builds its im2col in F32 when the kernel is BF16, then
calls ggml_mul_mat(im2col, kernel), which puts F32 in src0 and BF16 in
src1. The CPU backend refused that combination, so it was reported as
unsupported on every backend and never compared against anything.

Widen BF16 into the F32 work buffer, next to the existing packing of F32
into vec_dot_type. This is the arithmetic the Metal mat vec kernel
already uses, both operands promoted to float and accumulated in float,
so the two agree exactly rather than approximately.

Cover it with a conv_1d_dw test over F32, F16 and BF16 kernels, plus
three mul_mat cases with BF16 in src1.

* vulkan: reject BF16 in src1 of mul_mat unless src0 is BF16

supports_op only checked the src1 type for non contiguous tensors, so
a contiguous BF16 src1 was accepted and the pipeline lookup asserted.
The only BF16 src1 path is the BF16 x BF16 multiply, every other src0
type now reports the op as unsupported and the scheduler keeps it on
the CPU.

The BF16 kernel case of the conv_1d_dw test needs the f32 x bf16
mat vec variants of the Metal backend, which land separately.
 2026-10-06 10:38:12 +03:00
Pascal d0af734dfd openvino: serve GET_ROWS on a weight view from the base Constant (llama/28381) 
* openvino: serve GET_ROWS on a weight view from the base Constant

Resolve view_src when collecting weight Constants so a view over a
quantized weight no longer becomes a dynamic typed Parameter, and fold
the row offset of the view into the gather indices instead of slicing
the dequantization subgraph.

* openvino: lift the quantized GET_ROWS view rejection

The supports_op rejection of a quantized src0 view with a nonzero
offset keeps the vs0 GET_ROWS cases of #28253 away from OpenVINO.
The weight view now resolves to the base Constant with the row offset
folded into the gather indices, so the rejection goes away.
 2026-10-06 10:38:11 +03:00
R0CKSTAR 1cbf7e79f0 musa : define __CUDA_ARCH__ for device passes (llama/29508) 
The MUSA vendor header never defined __CUDA_ARCH__, so every architecture
test in the shared ggml-cuda sources evaluated to 0.  Kernel bodies gated on
the architecture therefore compiled to nothing, for example the q8_0 -> f16
dequantization kernel in convert.cu, whose NO_DEVICE_CODE fallback expands to
an empty body in host code.

Report the newest architecture like the HIP backend does and exclude the
NVIDIA-only features explicitly, as they are not usable on MUSA.  Define it
for device passes only: CUB uses defined(__CUDA_ARCH__) to detect device
compilation, which is also how nvcc behaves.

Drop the now-redundant defined(__CUDA_ARCH__) checks in the architecture
comparisons: __CUDA_ARCH__ is undefined in host passes for CUDA and MUSA, and
HIP defines it for every pass, so both forms select the same branch.
 2026-10-06 10:38:11 +03:00
Captain-Tripps d03bfd7f3c SYCL: reduce tensor allreduce sync with pinned host buffers (llama/29604) 2026-10-06 10:38:11 +03:00
Aaron Teo 66e8b957e7 ggml-zdnn: impl buffer reset, fix memory leaks (llama/29637) 
cont: fix code style

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
 2026-10-06 10:38:11 +03:00
cqderekandMax Krasnyansky b8051db82a Hexagon f16 activation ops (llama/29209) 
* hexagon: add F16 support for activation ops (SILU/GELU/GELU_QUICK/GEGLU/SWIGLU)

Widens ggml_hexagon_supported_activations() to accept F16 (src0/dst/src1
must agree on type), and adds F16 per-thread worker functions in
act-ops.c mirroring the existing F32 workers, backed by new HVX f16
kernels (hvx_sigmoid_f16_aa, hvx_tanh_f16_aa, hvx_mul_mul_f16_aa,
hvx_min_scalar_f16 family).

SILU, GELU, GELU_QUICK, GEGLU, and SWIGLU are verified correct on-device
(QRD8850) via test-backend-ops CPU-diffed correctness tests. SWIGLU_OAI's
F16 path is code-complete and builds clean on host + all 4 DSP arch
variants (v73/v75/v79/v81), but has no F16 test-case coverage in
test-backend-ops and is therefore unverified on-device in this change.

* hex-ops: align macros

* hex-ops: minor formatting

---------

Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
 2026-10-06 10:38:11 +03:00
Xiang Chen 6e48a365f3 gguf : reject tensor size that wraps after padding (llama/26979) 
GGML_PAD(nbytes, alignment) wraps to 0 when nbytes is within
(alignment - 1) of SIZE_MAX, which silently bypassed the size
overflow guard in gguf_init_from_reader. Reject the tensor before
padding when nbytes + (alignment - 1) would overflow.

Adds a test-gguf handcrafted case (F32, ne = [4, 2^30-1, 2^30+1, 1])
whose ggml_nbytes = 2^64 - 16 lands in the wrap window. Fails on
master, passes with the guard.
 2026-10-06 10:38:11 +03:00
Adrien Gallouët 14423995ba ggml : check row bounds in get_rows_back (llama/29575) 
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-10-06 10:38:10 +03:00
Trivikram Reddy 8d2a2cb471 hexagon: optimize concat op (llama/29673) 
* hex-concat: reduce pkts in gather/transpose hot loop

gather directly into dst buffer, use special instruction for gather sync

* hex-concat: use fastdiv

replace calls to sw divide with fastpath

* hex-concat: optimize DMA-HVX pipeline and add transpose helpers
 2026-10-06 10:38:10 +03:00
Pascal 45093fc8f0 CUDA: bitonic argsort handles rows wider than one block (llama/28957) 
Without CUB (HIP, MUSA) argsort ran the bitonic kernel with one thread
per padded column, so any row above 1024 entries launched an invalid
block configuration. Each thread now owns several columns, every stage
of the network runs all owned columns before the barrier, and the block
is capped at 1024 threads. Shared memory becomes the only bound, which
supports_op checks against the device instead of a fixed 1024.

Rows up to 1024 run the same work as before. Bit-exact with the CUB
path on rows of 2048.
 2026-10-06 10:38:10 +03:00
thelittlefiremanandCarl Philipp Klemm 482baacdad ggml-cuda: HIP: optimize packed byte subtraction (__vsubss4 -> __vsub4) (llama/29478) 
* ggml-cuda: HIP: optimize non-saturating packed byte subtraction (`__vsubss4`)

* CI: ignore 1 spilled vgpr in fattn_vec

---------

Co-authored-by: Carl Philipp Klemm <carl@uvos.xyz>
 2026-10-06 10:38:10 +03:00
Aaron Teo 3b86cabd47 ci: add zdnn backend build but not test (llama/29541) 
* ci: add zdnn backend build but not test

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: attempt to run a ubuntu 26.04 container

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: set shell to bash

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: clean up comments

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ggml-zdnn: fix compiler errors

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* vendor: attempt to ignore warnings from vendored files

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

---------

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
 2026-10-06 10:38:10 +03:00
lhez 9f12c9fe6a opencl: fix get_tensor for q5_K adreno gemm_nonshuffle kernel (llama/29555) 2026-10-06 10:38:10 +03:00
Ruben Ortlam 5439f3d969 vulkan: Tune GDN kernel, fix Intel performance (llama/29476) 
* vulkan: tune GDN shader

* tune for Intel
 2026-10-06 10:38:09 +03:00
Matt Corallo 6c07d8218b vulkan : Load F32 A matrix 2 at a time when its 2-aligned (llama/29254) 
It turns out Intel doesn't particularly like loading F32s one at a
time and we already have the _2aliagned load logic in mul_mat_vec,
so here we use it.

While we do already check all the requirements to load elements 4
at a time across [B]F16 and F32, it turns out [B]F16 loading 4 at a
time is sometimes slower on very specific shapes on Intel BMG.
Loading 4 at a time is a bit faster on F32, but its not material
and I assume might be slower on other platforms.

Note that we also need to validate `a_offset` is 2-aligned in
`mul_mat_vec.comp`, which was missing in the original 2-way-load
patch.

Some selected speedups from `test-backend-ops perf` on a B60.

  MUL_MAT(type_a=f32,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0):                    1704 runs -   767.17 us/run - 117.44 MFLOP/run - 153.08 GFLOPS
  MUL_MAT(type_a=f32,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0):                    2556 runs -   529.81 us/run - 117.44 MFLOP/run - 221.66 GFLOPS

  MUL_MAT(type_a=f32,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0):                    1704 runs -   727.13 us/run - 234.88 MFLOP/run - 323.03 GFLOPS
  MUL_MAT(type_a=f32,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0):                    2130 runs -   528.84 us/run - 234.88 MFLOP/run - 444.15 GFLOPS

  MUL_MAT(type_a=f32,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0):                    1704 runs -   702.19 us/run - 352.32 MFLOP/run - 501.74 GFLOPS
  MUL_MAT(type_a=f32,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0):                    1988 runs -   532.14 us/run - 352.32 MFLOP/run - 662.08 GFLOPS

  MUL_MAT(type_a=f32,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0):                    1278 runs -   919.50 us/run - 469.76 MFLOP/run - 510.89 GFLOPS
  MUL_MAT(type_a=f32,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0):                    1917 runs -   543.69 us/run - 469.76 MFLOP/run - 864.03 GFLOPS

  MUL_MAT(type_a=f32,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0):                    1197 runs -   892.12 us/run - 587.20 MFLOP/run - 658.21 GFLOPS
  MUL_MAT(type_a=f32,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0):                    1881 runs -   575.17 us/run - 587.20 MFLOP/run -   1.02 TFLOPS

  MUL_MAT(type_a=f32,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0):                    1498 runs -   716.40 us/run - 939.52 MFLOP/run -   1.31 TFLOPS
  MUL_MAT(type_a=f32,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0):                    1819 runs -   576.36 us/run - 939.52 MFLOP/run -   1.63 TFLOPS

  MUL_MAT(type_a=f32,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0):                   134 runs -  7467.09 us/run -  60.13 GFLOP/run -   8.05 TFLOPS
  MUL_MAT(type_a=f32,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0):                   134 runs -  7478.12 us/run -  60.13 GFLOP/run -   8.04 TFLOPS
 2026-10-06 10:38:09 +03:00
Ankit Khandelwal 8cc3ca48c3 vulkan: MOE aware mat_mul_id tile selection (llama/29182) 
mut_mul_id selected its matmul tile with total token count.
For MoE dispatch grid the true N per workgroup is per-expert rows.
At pp128 on Sarvam 30B that is 6, not 128, so the picker took the l-tile for ~6 live rows.
Most workers in each group had nothing to do.
This wasted time. The slow part was 55% of the whole job.
 2026-10-06 10:38:09 +03:00
XZiarGeorgi GerganovXZiar
 		deae6f3439 ggml : fix c++ odr by properly using GGML_COMMON_DECL_CPP (llama/29504) 
* fix c++ odr by properly using GGML_COMMON_DECL_CPP

* using actual field rather than macro

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: XZiar <xziar@xziar.xziar>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-10-06 10:38:09 +03:00
Adrien Gallouët b97d419389 ggml : accumulate f16 dot products in f32 on AVX512-FP16 (llama/29545) 
Supersedes #29530

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-10-06 10:38:09 +03:00
Georgi Gerganov 4d4817ed25 ggml : require input tensors to be GGML_OP_NONE (llama/29647) 2026-10-06 10:38:08 +03:00
Aparna M P 5e1914a735 hexagon: add FP32 GELU_ERF and GEGLU_ERF support (llama/29631) 
* hexagon: add FP32 GELU_ERF and GEGLU_ERF support

* hex-erf: reduce register pressure in kernels
 2026-10-06 10:38:08 +03:00
Georgi Gerganov 763b67fd85 ggml : collect all input tensors into graph_inputs (llama/29634) 
graph_inputs was populated while splitting the graph, so it only
contained the inputs that are used as srcs of some node. With pipeline
parallelism (n_copies > 1), each graph input contributes n_copies leafs
to graph_copy, so switching between batches that consume different
inputs (e.g. token batches that do not use the embeddings input vs
image batches that do) changed the graph composition. This shifted the
input copies in graph_copy, making the backend ids comparison report
spurious changes and forcing the scheduler to re-reserve. The
re-reserve could then record smaller input sizes (e.g. out_ids with
n_outputs = 0) and abort later on a graph with an unchanged size via
GGML_SCHED_DEBUG_REALLOC.

Collect the inputs after the split instead, from all input leafs of the
graph, so that the graph composition depends only on which inputs
exist, not on which inputs are used.

Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL
 2026-10-06 10:38:08 +03:00
Aaron Teo 847812906f ggml-zdnn: fix 0-row tensor crash (llama/29636) 
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
 2026-10-06 10:38:08 +03:00
Adrien GallouëtandJohannes Gäßler f449de9463 ggml : speed up model loading (llama/29598) 
* ggml : speed up model loading

A crafted model could hang the server for a very long time, try with:

    llama-cli -hf angt/test-gguf-1Mkv -hff model.gguf

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Avoid empty keys

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Fix

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

---------

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
 2026-10-06 10:38:08 +03:00
bri-prism b30ef05f7b metal: FWHT perf optimizations (llama/29602) 
Assisted-by: Claude Code
 2026-10-06 10:38:08 +03:00
Andrew Lee eecee67b76 vulkan : reuse descriptor sets when bindings are constant (llama/29280) 
* vulkan : reuse descriptor sets when bindings are constant

* vulkan : bump buffer_destroy_count before destroying the buffer
 2026-10-06 10:38:07 +03:00
Linux User 97690105c4 vulkan: include functional header (llama/29597) 
Fixes the compile error: `no template named 'function' in namespace 'std'`
 2026-10-06 10:38:07 +03:00
Ravi Panchumarthy 830dc664df ggml-openvino: mark unaligned batch-stride views unsupported (llama/29603) 2026-10-06 10:38:07 +03:00
jbooth 81b951e012 webgpu: Handle unaligned writes in ggml_backend_webgpu_buffer_set_tensor (llama/29471) 
* Fix:  Handle unaligned writes in ggml_backend_webgpu_buffer_set_tensor

* Clang formatting
 2026-10-06 10:38:07 +03:00
Georgi Gerganov ee86a150bd tests : refactor test-recurrent-state-rollback (llama/29426) 
* tests : use llama_context_ptr in test-recurrent-state-rollback

Replace raw llama_context pointers with llama_context_ptr and drop the
manual llama_free calls and cleanup lambda.

Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL

* tests : run test-recurrent-state-rollback over all dummy models

Add a --models DIR mode that mirrors test-save-load-state: iterate every
dummy model, report PASS/FAIL/SKIP in a table and fail only when a model
fails. Register a single ctest entry with ARGS --models instead of the four
per-model registrations.

Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL

* cont : fix typo

* metal : allow fusing 0-element nodes to keep graph packing shape-independent

The fusion packing in ggml_metal_fusion_max excluded 0-element tensors and
the topk_moe/moe_reduce checks rejected n_tokens == 0, so graphs decoding
batches with no outputs packed differently from the worst-case reserved
graph. The Metal optimizer then reordered the nodes differently and
ggml_gallocr_needs_realloc failed on the layout mismatch, forcing an
unexpected graph re-reserve (caught by GGML_SCHED_DEBUG_REALLOC).

Treat empty tensors like their non-empty counterparts: match them in the
pattern sequence and only reject genuinely malformed shapes. Fused kernels
dispatch zero threadgroups for empty graphs, which is a legal no-op.

Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL

* tests : run test_multi_seq_split_replay as a separate test

test_multi_seq_split_replay was invoked at the end of test_rollback,
so its result was folded into the rollback status and it only ran when
the rollback part passed.

Give it its own test_status return, run both tests independently over
both cache fills via a shared run_tests helper, and report them as
separate rollback / split replay columns in the --models table with
per-test summaries. The exit code fails when either test fails.

Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL

* tests : loosen the split replay nmse bound to 1e-4

test-generate-models seeds its weights from std::random_device, and some
generated lfm2 models drift up to ~1.7e-5 nmse on the split replay due to
rounding noise, tripping the previous 1e-5 bound. Raise the bound to 1e-4
so the random generations stop flaking.

Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL

* tests : reuse run_tests_for_model in single-model mode

The single-model path duplicated the model init and the non-recurrent
check from run_tests_for_model; route it through the shared helper
instead. Model load failures now return FAIL rather than SKIP so that
--model with a broken file still exits non-zero, and the helper loads
with model_only like the --models loop does since the tests create
their own contexts.

Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL
 2026-10-06 10:38:07 +03:00