LocalLlamaClient

LLMClient implementation wrapping llama.cpp for local GGUF model inference.

Detailed description

Loads a GGUF model file via llama.cpp, runs inference locally on the CPU or GPU, and delivers results through the LLMClient interface. Supports both one-shot and streaming chat, and allows runtime model loading and unloading to manage memory.

See also LLMClient, GGUFMetadata.