LLMClient implementation wrapping llama.cpp for local GGUF model inference. More...
| Header: | #include <LocalLlamaClient> |
| Since: | 1.0 |
| Inherits: | LLMClient |
Loads a GGUF model file via llama.cpp, runs inference locally on the CPU or GPU, and delivers results through the LLMClient interface. Supports both one-shot and streaming chat, and allows runtime model loading and unloading to manage memory.
See also LLMClient and GGUFMetadata.