The LLM (Large Language Model) Manager Subsystem provides a powerful, unified abstraction layer for interacting with various LLM providers. Its primary goal is to decouple the agent's logic from the specific implementation details and APIs of any single LLM provider.
The LLM Manager provides a single, consistent interface for all supported LLM providers. This means an agent can be switched from using OpenAI's GPT-4 to Anthropic's Claude by changing only the model identifier in its configuration. The agent's core prompting logic does not need to change.
This abstraction handles:
To optimize performance and reduce costs, the LLM Manager integrates directly with the Memory Manager Subsystem. It can be configured to automatically cache LLM responses. When a prompt is repeated, the cached response can be served instantly, bypassing a costly API call to the LLM provider.
The subsystem includes built-in hooks for tracking token usage (both prompt and completion tokens) for every LLM call. This data is essential for monitoring costs, analyzing agent performance, and enforcing usage limits.
The LLM Manager uses the same connector model as other SRE subsystems, allowing for easy extension to new providers. Common connectors include:
OpenAIConnectorAnthropicConnectorGoogleAIConnector (for Gemini and Vertex AI)AWSBedrockConnectorGroqConnectorPerplexityConnector