LLMManager
Node-level LLM management with client factory and LRU caching.
| Inherits | QObject |
| Group | libllm |
Declared in src/libs/libllm/llm/LLMManager.hpp.
Detailed description
Wraps an LLMRegistry and creates LLMClient instances on demand for configured models. Maintains an LRU cache of loaded clients to bound memory usage, automatically evicting the least-recently-used client when the cache limit is exceeded.
See also LLMRegistry, LLMClient.
Member function documentation
buildClient
[static] QSharedPointer<LLMClient> LLMManager::buildClient(const LLMProvider &provider, const LLMModel &model)
The client of model at provider, or null when the provider cannot serve it. The chat path and the online driver factory both build their clients here (llm-openai-compatible §OCP-A2). An online provider's wire protocol picks the client (§OCP-W1). An empty base URL means Anthropic's only for the Anthropic protocol: an OpenAI-form provider with none cannot serve.
newClient
[static] std::unique_ptr<LLMClient> LLMManager::newClient(const LLMProvider &provider, const LLMModel &model)
The client of model at provider, or null when the provider cannot serve it, for a caller that owns it alone: the online driver factory gives it to DriverManager (llm-openai-compatible §OCP-A2). buildClient() is the same client for a shared owner.