public_git/whisper.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/whisper.cpp.git synced 2026-10-01 03:59:20 +02:00
Code Issues Packages Projects Releases Wiki Activity

rpc : cache and reuse compute graphs (llama/15405)

Browse Source 
Store the last computed graph and reuse it when possible.
Also do not return response from GRAPH_COMPUTE and assume it always
completes successfully. If this this is not the case, the server closes
the connection. This saves us a network round trip to the server.
This commit is contained in:
Radoslav Gerganov
2025-12-12 17:53:11 +02:00
committed by Georgi Gerganov
parent f92d542d4d
commit d26d1c8b85
2 changed files with 92 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/include/ggml-rpc.h
+1 -1
View File
@@ -8,7 +8,7 @@ extern "C" {
#endif
#define RPC_PROTO_MAJOR_VERSION 3
#define RPC_PROTO_MINOR_VERSION 0
#define RPC_PROTO_MINOR_VERSION 5
#define RPC_PROTO_PATCH_VERSION 0
#define GGML_RPC_MAX_SERVERS 16