Configure an LLM provider

Set up an LLM provider so OctoMY can use AI features like the OPAL chat overlay.

Prerequisites

• OctoMY hub running
• An API key from your chosen provider (Anthropic, OpenAI, Google, or a custom endpoint). You can skip this if you use a provider that needs no key, such as Claude Code (local).

Steps

1. Open LLM settings

Open the OPAL chat overlay and click the hamburger menu (☰) in the top right, then select LLM settings. This opens the provider and model browser. It works the same way in every app, standalone construct included — you do not need the hub.

2. Add a provider

Click Add provider. The provider editor opens.

Fill in:

3. Set credentials

Click Edit next to the credentials status. This opens the credential editor where you can:

1. Paste your API key
2. Toggle visibility with the Show/Hide button
3. Click Test connection to verify the key works
4. Click Save to store it

The credential status shows:

• "No credentials set" — no API key entered yet
• "Credentials set" — key saved but not tested
• "Credentials valid" — key tested successfully

Note: If your node has no identity, a warning will appear that credentials cannot be encrypted. Deliver an identity first for secure storage.

4. Test the connection

Click Test connection in the provider editor. If successful, you'll see how many models are available and the provider status changes to "Valid" (green) in the registry.

4. Fetch models

Click Fetch models to import the provider's available models into your registry. Models appear in the list below the provider form.

5. Set a default model

Return to LLM settings. Double-click a model, or select it and click Set default. The default model is used by the OPAL chat overlay.

6. Verify readiness

Open the chat overlay (click the OPAL button in the header bar). If everything is configured correctly, the readiness banner will be hidden and the input field will be enabled.

Provider types

Online API

For cloud-hosted models (Anthropic Claude, OpenAI GPT, Google Gemini). Requires an API key and internet connection.

Local llama

For local inference using llama.cpp. Requires a GGUF model file on disk. No internet needed after setup.

Remote node

For using another OctoMY node's LLM capabilities. The remote node must be paired and running.

Claude Code (local)

If you already have Claude Code installed on this machine, it appears in the provider list on its own, named Claude Code (local). There is nothing to add and no API key to paste — OctoMY looks for the claude command and enables the row when it finds it.

To use it, select it in the provider list, set its one model as your default model, and start chatting.

Three things are worth knowing:

• It spends your Claude Code subscription. A chat turn here draws on the same usage allowance as running claude in a terminal. When that allowance runs out, OPAL says so and tells you when it resets. Waiting is the only fix — adding funds does not lift a subscription limit.
• It cannot edit your files. OPAL runs Claude Code in a read-only mode, in an empty working directory, so a chat turn never touches your project.
• Replies arrive whole. Other providers stream a reply word by word. This one delivers the finished answer in one piece, so expect a pause and then the whole reply at once.

If you installed Claude Code while OctoMY was running, restart the app. The check runs at startup.

Tips

• Keep your API key secure. OctoMY encrypts credentials at rest using your node's identity key.
• Test connection before chatting. The validation status (green/orange/grey) in the registry shows whether your provider is verified.
• Fetch models regularly. Providers add new models over time. Use Fetch models to stay current.