amritahs-ibm 124eec1664 llamafile : ppc64le MMA INT8 implementation (llama/10912) ... This change upstreams llamafile's cpu matrix multiplication kernels for ppc64le using MMA builtins for quantised int8 datatype. This change results in 10% - 70% improvement in total speed(ie all tokens/total time), across various batch sizes. The patch is tested with Meta-Lllama-3-8B, Mistral-7B, Llama-2-7B-chat-hf models on a IBM POWER10 machine. Signed-off-by: Amrita H S <amritahs@linux.vnet.ibm.com>		2025-01-14 10:38:01 +02:00
..
include	GGUF: C++ refactor, backend support, misc fixes (llama/11030)	2025-01-14 10:38:01 +02:00
src	llamafile : ppc64le MMA INT8 implementation (llama/10912)	2025-01-14 10:38:01 +02:00
.gitignore	whisper : reorganize source code + improve CMake (#2256)	2024-06-26 19:34:09 +03:00
CMakeLists.txt	GGUF: C++ refactor, backend support, misc fixes (llama/11030)	2025-01-14 10:38:01 +02:00