LLM Strategy

OctoMY™'s approach to integrating Large Language Models safely and effectively.

Did You Know?

OctoMY™'s LLM strategy was designed with a specific threat model in mind: "what if the AI hallucinates and tries to drive the robot off a cliff?" The answer is the embedded/strategic barrier - even if the AI generates a perfect-sounding command to "maximize velocity toward coordinates (cliff)", the embedded layer's safety monitors will refuse to execute it.

Context

In 2025, ignoring LLMs is not an option. OctoMY™ integrates modern AI technology constructively across:

• Development tooling - Accelerating the development process
• Runtime features - User-facing AI capabilities
• Server-side services - Backend AI integration

Target users

Core principles

1. Local-first

All node-side LLM runs locally. No internet contact except via Zoo (hole-punching) or Hub.

Agent with local LLM
    │
    ├─► Local inference (no internet)
    │
    └─► Hub connection (if external LLM needed)
            │
            └─► External API (Claude, etc.)

Why: Privacy, latency, offline operation, reduced attack surface.

2. Hub as gateway

External services (LLMs, APIs, models) are accessed only through a trusted Hub node.

Why: Single security boundary, centralized access control, simplified node requirements.

3. Permission-centric

LLMs operate within explicit, user-granted permissions via OPAL (Operator Permission and Access Layer).

Why: Human authority over AI actions, auditability, safety.

4. Deterministic identity

Agent personalities are RNG-derived from cryptographic keys, ensuring consistency across contexts.

Why: Recognizable robot identity, consistent interaction experience.

The embedded/strategic barrier

OctoMY™ enforces strict separation between hardware control and AI:

LLMs can request "navigate to kitchen" but cannot command "set motor voltage to 12V".

See Embedded/Strategic Barrier for details.

Implementation phases

Phase 1: Development acceleration

Shorten the LLM-fueled development feedback loop.

Components:

• uiautomation - Tool for UI testing and screenshot capture
• mocknode - Standardized test harness mimicking any node type
• Documentation generation - Automated screenshot capture

Value: Benefits developers immediately. LLM can invoke tools directly, creating closed feedback loops.

Phase 2: Automated testing

Maintain quality as development accelerates.

Components:

• uitest - Extends uiautomation for regression testing
• Visual regression - Screenshot comparison
• LLM-aided test authoring - Beyond unit tests

Value: While Phase 1 lives in the moment, Phase 2 secures the legacy.

Phase 3: OPAL runtime layer

Enable LLM-driven agent management with proper access control.

Components:

• Permission system
• Command API (structured JSON)
• Self-documenting endpoints
• User consent UI

Value: Safe, auditable LLM control of robots.

Phase 4: Convenience features

User-facing LLM helpers that depend on OPAL.

Components:

• Autocomplete/suggestions
• Natural language commands
• Personality-driven interactions

Value: Accessible AI for end users.

LLM deployment options

Local LLM (agent)

• Runs on agent hardware
• No network required
• Limited by hardware capabilities
• Full privacy

External LLM (via hub)

• Powerful models available
• Requires Hub and internet
• Hub manages API keys
• Permissions still enforced

Personality and LLM

When LLMs interact through an agent, personality provides consistency:

Agent Key → Deterministic RNG → Personality Traits
                               ├── Name
                               ├── Voice characteristics
                               ├── Visual identity
                               └── LLM persona prompt

The personality generates a prompt fragment:

You are [Name], a [traits] robot assistant.
You speak with [voice characteristics].
Your responses should reflect [personality aspects].

This ensures the same robot has a consistent identity regardless of which LLM backend is used.