Custom Models

Connect your own AI models or agents to Zencoder.

Overview

Zencoder supports custom agents, allowing you to connect your own models, private deployments, or third-party inference endpoints. Custom models appear in the model selector alongside built-in options.

Use Cases

• Local models: Run open-source models via Ollama or vLLM on your machine
• Private cloud: Use Azure OpenAI, Vertex AI, or AWS Bedrock endpoints
• Air-gapped environments: Point to inference servers inside your network
• Fine-tuned models: Use models you’ve trained on your own data

Configuration

Custom models are configured via settings.json. This file works at two levels:

• Machine-wide: ~/.zencoder/settings.json
• Per project: .zencoder/settings.json (overrides machine-wide)

Example: Ollama (Local)

{
  "providers": {
    "ollama-local": {
      "mode": "direct",
      "type": "openai-compatible",
      "baseUrl": "http://localhost:11434/v1",
      "apiKey": "not-needed",
      "models": {
        "llama3-70b": {
          "name": "llama3:70b",
          "displayName": "Llama 3 70B (Local)",
          "capabilities": [],
          "options": {
            "temperature": 0.7,
            "maxOutputTokens": 4096
          }
        }
      }
    }
  }
}

Example: Azure OpenAI

{
  "providers": {
    "azure-gpt4": {
      "mode": "direct",
      "type": "openai-compatible",
      "baseUrl": "https://your-resource.openai.azure.com/v1",
      "apiKey": "${AZURE_OPENAI_KEY}",
      "models": {
        "gpt4-deployment": {
          "name": "gpt-4",
          "displayName": "Azure GPT-4",
          "capabilities": [],
          "options": {
            "temperature": 0.2,
            "maxOutputTokens": 4096
          }
        }
      }
    }
  }
}

Hiding Default Models

Set useDefaultProviders to false to only show your custom models in the selector:

{
  "useDefaultProviders": false,
  "providers": {
    "my-provider": { ... }
  }
}

Requirements

Custom endpoints must support:

Requirement	Description
OpenAI-compatible API	/chat/completions endpoint
Streaming	Server-sent events for real-time responses
Tool use (optional)	Function calling for enhanced capabilities

Full Reference

See Custom Models Configuration for the complete settings reference, and Private Deployments for air-gapped setup.

Selecting a Custom Model in Chat

In the chat composer, click the preset dropdown (the gear icon) and select Custom. Then use the model dropdown next to Zencoder to choose your model.

Next Steps

Custom Models Config

Full configuration reference

Private Deployments

Air-gapped and on-premise setup