Bring Your Own LLM (BYOC)

Enterprise customers can connect their own LLM infrastructure to AtlasAI instead of using the default Atlas Cloud AI models. This enables full control over model selection, data routing, and compliance with data sovereignty requirements.

Supported Providers

Configuration

Azure OpenAI

1. Navigate to Settings → AI → LLM Provider
2. Select Azure OpenAI
3. Enter your configuration:
   • Endpoint: https://<resource-name>.openai.azure.com/
   • API Key: Your Azure OpenAI API key
   • Deployment Name: The name of your model deployment
   • API Version: 2024-02-15-preview or later
4. Click Test Connection to verify
5. Click Save

AWS Bedrock

1. Navigate to Settings → AI → LLM Provider
2. Select AWS Bedrock
3. Enter your configuration:
   • Region: Your AWS region (e.g., us-east-1)
   • Authentication: IAM Role ARN or Access Key / Secret Key
   • Model ID: e.g., anthropic.claude-3-sonnet-20240229-v1:0
4. Click Test Connection to verify
5. Click Save

Self-Hosted (OpenAI-Compatible)

For models served via vLLM, TGI, Ollama, or any OpenAI-compatible API:

1. Navigate to Settings → AI → LLM Provider
2. Select Custom OpenAI-Compatible
3. Enter your configuration:
   • Base URL: e.g., http://gpu-server.internal:8000/v1
   • API Key: (optional, depends on your server)
   • Model Name: e.g., meta-llama/Llama-3-70b-chat
4. Click Test Connection to verify
5. Click Save

Data Flow with BYOC

When BYOC is configured, the data flow changes:

• Without BYOC: Incident context → Atlas Cloud AI Gateway → Atlas AI Models → Response
• With BYOC: Incident context → Your LLM endpoint → Response (no data leaves your network if self-hosted)

The AI Gateway in the Control Plane routes inference requests to your configured provider instead of the default Atlas models. If your provider is unavailable, the system can optionally fall back to Atlas Cloud models (configurable).

Model Requirements

For best results, the connected model should:

• Support function calling / tool use (required for runbook generation)
• Have a context window of at least 32K tokens (128K recommended)
• Support structured output / JSON mode (recommended for RCA)
• Have reasonable latency (< 30 seconds for a typical RCA prompt)

Fallback Configuration

Under Settings → AI → LLM Provider → Fallback, you can configure behavior when your BYOC provider is unavailable:

Cost Considerations

When using BYOC, you are responsible for the inference costs on your LLM provider. AtlasAI does not charge additional fees for AI features when BYOC is configured — you only pay your standard AtlasAI subscription plus your LLM provider costs.