mirror of
https://github.com/ggml-org/whisper.cpp.git
synced 2026-10-12 17:25:39 +02:00
ynankaniandJohannes Gäßler
10872be7f8
* CUDA: Handle compute type for NVFP4 on cublass path Signed-off-by: ynankani <ynankani@nvidia.com> * Use BF16 compute type for quantized models if HW allows Signed-off-by: ynankani <ynankani@nvidia.com> * Set acc prec to bf16 for nvfp4 as it needs atleast bf16 range Signed-off-by: ynankani <ynankani@nvidia.com> * Update ggml/src/ggml-cuda/ggml-cuda.cu Co-authored-by: Johannes Gäßler <johannesg@5d6.de> * preserve op_params for per-expert matmul Signed-off-by: ynankani <ynankani@nvidia.com> --------- Signed-off-by: ynankani <ynankani@nvidia.com> Co-authored-by: Johannes Gäßler <johannesg@5d6.de>