LLMClient

Abstract asynchronous interface for LLM inference.

Inherits octomy::llm::driver::ILLMDriver
Inherited By AndroidPlatformLLMClient, ClaudeCodeClient, LocalLlamaClient, OnlineAPIClient, and OpenAIChatClient
Group libllm

Declared in src/libs/libllm/llm/LLMClient.hpp.

Detailed description

Defines the contract for sending chat messages to an LLM and receiving responses either as a single completed reply or as a stream of tokens. Concrete implementations include OnlineAPIClient, LocalLlamaClient, ClaudeCodeClient and AndroidPlatformLLMClient. A model on a peer node is reached through RemoteNodeDriver, which implements the driver interface instead of this one.

See also OnlineAPIClient, LocalLlamaClient, RemoteNodeDriver.

In this section