Abstract asynchronous interface for LLM inference. More...
| Header: | #include <LLMClient> |
| Since: | 1.0 |
| Inherits: | octomy::llm::driver::ILLMDriver |
| Inherited By: | AndroidPlatformLLMClient, ClaudeCodeClient, LocalLlamaClient, and OnlineAPIClient |
Defines the contract for sending chat messages to an LLM and receiving responses either as a single completed reply or as a stream of tokens. Concrete implementations include OnlineAPIClient, LocalLlamaClient, ClaudeCodeClient and AndroidPlatformLLMClient. A model on a peer node is reached through RemoteNodeDriver, which implements the driver interface instead of this one.
See also OnlineAPIClient, LocalLlamaClient, and RemoteNodeDriver.