public_git/whisper.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/whisper.cpp.git synced 2026-10-06 22:41:59 +02:00
Code Issues Packages Projects Releases Wiki Activity

llama: add default load-mode auto, which avoids mmap on iGPUs (llama/26081)

Browse Source 
* llama: add new default load-mode auto which picks mmap unless a non-Metal iGPU is used

* Update ggml/src/ggml-hexagon/ggml-hexagon.cpp

Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>

* set mmap_support to false on OpenCL backend

* fix order of load modes

* use -1 for auto

* resolve load mode auto earlier to correctly pick gpu host or cpu memory

* add load mode auto to llama-bench

* bump virtgpu api version, regenerate docs

---------

Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com>
Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
This commit is contained in:
Ruben Ortlam
2026-08-14 22:16:06 +03:00
committed by Georgi Gerganov
co-authored by Max Krasnyansky Piotr Wilkin Georgi Gerganov
parent 0f60f8e691
commit d9589688cb
23 changed files with 31 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-virtgpu/virtgpu-forward-device.cpp
+3 -1
View File
@@ -144,7 +144,8 @@ void apir_device_get_props(virtgpu * gpu,
bool * async,
bool * host_buffer,
bool * buffer_from_host_ptr,
bool * events) {
bool * events,
bool * mmap_support) {
apir_encoder * encoder;
apir_decoder * decoder;
ApirForwardReturnCode ret;
@@ -157,6 +158,7 @@ void apir_device_get_props(virtgpu * gpu,
apir_decode_bool_t(decoder, host_buffer);
apir_decode_bool_t(decoder, buffer_from_host_ptr);
apir_decode_bool_t(decoder, events);
apir_decode_bool_t(decoder, mmap_support);
remote_call_finish(gpu, encoder, decoder);