OnlineTransport

What the two online clients share: the limits of a request and the map of its requests (llm-openai-compatible §OCP-W1).

Declared in src/libs/libllm/llm/clients/OnlineTransport.hpp.

Detailed description

The Anthropic client and the OpenAI Chat client differ in the wire format only. Both keep a map from a request id to its `WebRequest`, both end a request with no total timeout, and both name the limit that ended one. This class holds those parts, so the two clients cannot differ in them. It is a plain class that its client owns, and each `WebRequest` has the client as its parent.

Member function documentation

abort

void OnlineTransport::abort(const QString &requestId)

Abort the request of requestId, and forget it.

add

void OnlineTransport::add(const QString &requestId, WebRequest *request)

Keep request under requestId.

applyLimits

[static] void OnlineTransport::applyLimits(WebRequest *request, bool selfHosted)

Give request the limits of an LLM request (§OCP-W6): the header limit, which is longer when selfHosted, because such a server can queue the request, the stall limit, and no total limit, because a long answer that streams is alive.

ids

QStringList OnlineTransport::ids() const

The ids of the requests that run.

limitFailure

[static] QString OnlineTransport::limitFailure(const WebRequest *request, const QString &provider, bool selfHosted)

Returns the user's text when a limit of applyLimits() ended request, naming provider, with the header limit of selfHosted. Empty when no limit ended it.

release

void OnlineTransport::release(const QString &requestId)

Forget the request of requestId, and delete it later.

request

WebRequest *OnlineTransport::request(const QString &requestId) const

The request of requestId, or null.