LLMManager

Node-level LLM management with client factory and LRU caching.

Inherits QObject
Group libllm

Declared in src/libs/libllm/llm/LLMManager.hpp.

Detailed description

Wraps an LLMRegistry and creates LLMClient instances on demand for configured models. Maintains an LRU cache of loaded clients to bound memory usage, automatically evicting the least-recently-used client when the cache limit is exceeded.

See also LLMRegistry, LLMClient.

Member function documentation

buildClient

[static] QSharedPointer<LLMClient> LLMManager::buildClient(const LLMProvider &provider, const LLMModel &model)

The client of model at provider, or null when the provider cannot serve it. The chat path and the online driver factory both build their clients here (llm-openai-compatible §OCP-A2). An online provider's wire protocol picks the client (§OCP-W1). An empty base URL means Anthropic's only for the Anthropic protocol: an OpenAI-form provider with none cannot serve.

newClient

[static] std::unique_ptr<LLMClient> LLMManager::newClient(const LLMProvider &provider, const LLMModel &model)

The client of model at provider, or null when the provider cannot serve it, for a caller that owns it alone: the online driver factory gives it to DriverManager (llm-openai-compatible §OCP-A2). buildClient() is the same client for a shared owner.

In this section