> ## Documentation Index
> Fetch the complete documentation index at: https://developer.kallglot.com/llms.txt
> Use this file to discover all available pages before exploring further.

# AI Voice Agents

> Build intelligent voice agents that understand and respond naturally

# AI Voice Agents

Build AI-powered voice agents that handle conversations autonomously, understand context, and can take actions through function calling.

## Overview

AI agents in Kallglot can:

* Understand natural language in 25+ languages
* Respond with natural-sounding speech
* Follow custom instructions via system prompts
* Access knowledge bases for contextual answers
* Call external functions/tools to take actions
* Escalate to human agents when needed

## Public HTTP API (`/v1/agents`)

Configuration lives in the **Agents** endpoints—sessions only **reference** an agent id.

| Method   | Path                    | Purpose                           |
| -------- | ----------------------- | --------------------------------- |
| `GET`    | `/v1/agents`            | Cursor-paginated list             |
| `GET`    | `/v1/agents/{agent_id}` | Fetch prompts, tools, voice, etc. |
| `POST`   | `/v1/agents`            | Create a new agent                |
| `PATCH`  | `/v1/agents/{agent_id}` | Update fields (partial)           |
| `DELETE` | `/v1/agents/{agent_id}` | Delete an agent                   |

Creating a voice session uses `mode: ai_agent` plus the agent reference—**not** an inline system prompt:

```json theme={null}
{
  "mode": "ai_agent",
  "source_language": "en",
  "target_language": "en",
  "ai_agent": {
    "id": "agt_01HABCDEF",
    "overrides": {}
  }
}
```

Remove `overrides` entirely when you do not need per-session deltas (empty objects are uncommon in clients).

See [Error codes](/errors) for `agent_not_found` (Agents API) versus `ai_agent_not_found` (sessions referencing a missing AI agent).

## Creating an AI Agent Session

<Note>
  Kallglot does not currently provide official SDKs. The helper functions in this guide, such as `createKallglotSession(...)`, are placeholders for direct HTTP requests to the API. The JavaScript object below illustrates **how you might structure agent behavior** (tools, prompt text, knowledge base wiring); mirror the JSON shape above when calling `POST /v1/sessions`.
</Note>

```javascript theme={null}
const session = await createKallglotSession({
  mode: 'ai_agent',
  source_language: 'en',
  target_language: 'en',

  ai_agent: {
    // Custom system prompt
    system_prompt: `You are a helpful customer service agent for Acme Corp.
You assist customers with order inquiries, returns, and general questions.
Be friendly, professional, and concise.
If you cannot help with something, offer to transfer to a human agent.`,

    // Voice selection
    voice: 'alloy',

    // Temperature for response generation
    temperature: 0.7,

    // Knowledge base for context
    knowledge_base_id: 'kb_01ABC123',

    // Function calling
    tools: [
      {
        name: 'lookup_order',
        description: 'Look up an order by order number or customer email',
        parameters: {
          type: 'object',
          properties: {
            order_number: {
              type: 'string',
              description: 'The order number (e.g., ORD-12345)'
            },
            email: {
              type: 'string',
              description: 'Customer email address'
            }
          }
        }
      },
      {
        name: 'initiate_return',
        description: 'Start a return process for an order',
        parameters: {
          type: 'object',
          properties: {
            order_number: { type: 'string' },
            reason: { type: 'string' },
            items: {
              type: 'array',
              items: { type: 'string' }
            }
          },
          required: ['order_number', 'reason']
        }
      },
      {
        name: 'transfer_to_human',
        description: 'Transfer the call to a human agent',
        parameters: {
          type: 'object',
          properties: {
            reason: { type: 'string' },
            department: {
              type: 'string',
              enum: ['support', 'sales', 'billing']
            }
          }
        }
      }
    ]
  }
});
```

## System Prompts

The system prompt defines your agent's personality, capabilities, and behavior.

### Best Practices

```text theme={null}
You are [Role] for [Company].

## Your Capabilities
- [Capability 1]
- [Capability 2]

## Guidelines
- Be [tone] and [style]
- [Specific instruction]
- [Boundary]

## Escalation
Transfer to a human agent when:
- [Condition 1]
- [Condition 2]
```

### Example: E-commerce Support Agent

```text theme={null}
You are a customer support agent for TechGadgets, an electronics retailer.

## Your Capabilities
- Look up order status and tracking information
- Process returns and exchanges for orders within 30 days
- Answer questions about products and warranties
- Help with account issues

## Guidelines
- Be friendly and empathetic
- Keep responses concise (2-3 sentences when possible)
- Always verify the customer's identity before discussing order details
- Never share personal information from other customers

## Escalation
Transfer to a human agent when:
- Customer requests to speak with a person
- Issue involves a refund over $500
- Customer is upset or frustrated after 2 failed resolution attempts
- Technical issues you cannot resolve
```

## Handling Function Calls

When the AI agent needs to call a function, you'll receive a webhook:

```javascript theme={null}
app.post('/webhooks/kallglot/tool_call', async (req, res) => {
  const { session_id, tool } = req.body.data;

  let result;

  switch (tool.name) {
    case 'lookup_order':
      result = await lookupOrder(tool.arguments);
      break;

    case 'initiate_return':
      result = await initiateReturn(tool.arguments);
      break;

    case 'transfer_to_human':
      result = await transferToHuman(session_id, tool.arguments);
      break;

    default:
      result = { error: 'Unknown function' };
  }

  // Send the result back to the agent
  await submitKallglotToolResult(session_id, {
    tool_call_id: tool.call_id,
    result: JSON.stringify(result)
  });

  res.status(200).send('OK');
});

async function lookupOrder({ order_number, email }) {
  const order = await db.orders.findOne({
    $or: [
      { order_number },
      { customer_email: email }
    ]
  });

  if (!order) {
    return {
      found: false,
      message: 'No order found with that information'
    };
  }

  return {
    found: true,
    order_number: order.order_number,
    status: order.status,
    items: order.items,
    tracking_number: order.tracking_number,
    estimated_delivery: order.estimated_delivery
  };
}
```

## Knowledge Bases

Connect a knowledge base to give your agent access to company information:

### Creating a Knowledge Base

```javascript theme={null}
const kb = await createKnowledgeBase({
  name: 'Product Documentation',
  description: 'Product specs, FAQs, and troubleshooting guides'
});

// Add documents
await addKnowledgeBaseDocuments(kb.id, [
  {
    type: 'url',
    url: 'https://your-site.com/products'
  },
  {
    type: 'file',
    file: fs.createReadStream('product-manual.pdf'),
    filename: 'product-manual.pdf'
  },
  {
    type: 'text',
    content: 'FAQ: How to reset your device...',
    metadata: { category: 'troubleshooting' }
  }
]);
```

### Using the Knowledge Base

```javascript theme={null}
const session = await createKallglotSession({
  mode: 'ai_agent',
  ai_agent: {
    knowledge_base_id: kb.id,
    // Agent will automatically search the KB when relevant
  }
});
```

## Voice Configuration

### Available Voices

| Voice ID  | Style               | Best For                 |
| --------- | ------------------- | ------------------------ |
| `alloy`   | Neutral, clear      | General purpose          |
| `echo`    | Warm, friendly      | Customer service         |
| `fable`   | Expressive          | Storytelling, engagement |
| `onyx`    | Deep, authoritative | Professional, formal     |
| `nova`    | Energetic, upbeat   | Sales, marketing         |
| `shimmer` | Calm, soothing      | Support, helpdesk        |

### Voice Parameters

```javascript theme={null}
ai_agent: {
  voice: 'echo',
  voice_settings: {
    speed: 1.0,        // 0.5 - 2.0
    pitch: 0,          // -12 to 12 semitones
    stability: 0.75,   // 0 - 1 (higher = more consistent)
    similarity: 0.85   // 0 - 1 (higher = more similar to original voice)
  }
}
```

## Multilingual Agents

AI agents can handle conversations in multiple languages:

```javascript theme={null}
const session = await createKallglotSession({
  mode: 'ai_agent',
  source_language: 'auto',  // Detect customer's language
  target_language: 'auto',  // Respond in same language

  ai_agent: {
    system_prompt: `You are a multilingual support agent.
Respond in the same language the customer uses.
You are fluent in English, German, French, and Spanish.`,

    // Voice that works well for multiple languages
    voice: 'alloy',

    multilingual: {
      enabled: true,
      supported_languages: ['en', 'de', 'fr', 'es'],
      fallback_language: 'en'
    }
  }
});
```

## Escalation to Human Agents

Handle handoffs gracefully:

```javascript theme={null}
async function transferToHuman(sessionId, { reason, department }) {
  // 1. Get session summary
  const session = await retrieveKallglotSession(sessionId);
  const transcript = await getKallglotTranscript(sessionId);

  // 2. Find available agent
  const agent = await findAvailableAgent(department);

  if (!agent) {
    return {
      success: false,
      message: 'No agents available. Please try again later.'
    };
  }

  // 3. Create a new session for the human agent
  const humanSession = await createKallglotSession({
    mode: 'bidirectional_translation',
    source_language: session.source_language,
    target_language: agent.language
  });

  // 4. Transfer the call
  await transferKallglotSession(sessionId, {
    target_session_id: humanSession.id,
    context: {
      summary: transcript.summary,
      sentiment: transcript.sentiment,
      reason_for_transfer: reason
    }
  });

  return {
    success: true,
    message: 'Transferring you to an agent now.'
  };
}
```

## Conversation Context

Access conversation context during the session:

```javascript theme={null}
// Get current conversation state
const context = await getKallglotSessionContext(sessionId);

console.log(context);
// {
//   turn_count: 5,
//   customer_sentiment: 'positive',
//   topics_discussed: ['order_status', 'shipping'],
//   entities: {
//     order_number: 'ORD-12345',
//     product: 'Wireless Headphones'
//   },
//   intent_history: [
//     { intent: 'check_order', confidence: 0.95 },
//     { intent: 'ask_shipping', confidence: 0.88 }
//   ]
// }
```

## Error Handling

Handle agent errors gracefully:

```javascript theme={null}
session.on('error', async (error) => {
  console.error('Agent error:', error);

  if (error.code === 'llm_timeout') {
    // LLM took too long - provide fallback response
    await session.speak("I'm sorry, I'm having trouble processing that. Let me transfer you to a colleague.");
    await transferToHuman(session.id, { reason: 'technical_issue' });
  }
});

session.on('no_match', async (data) => {
  // Agent couldn't understand the customer
  if (data.consecutive_failures >= 3) {
    await session.speak("I'm having trouble understanding. Would you like to speak with a person?");
  }
});
```

## Analytics

Track agent performance:

```javascript theme={null}
// After session ends
const analysis = await requestKallglotAnalysis(sessionId, {
  analyses: ['quality', 'sentiment', 'summary']
});

// Log metrics
await analytics.track('ai_agent_session', {
  session_id: sessionId,
  duration: session.duration,
  turns: context.turn_count,
  resolution: analysis.results.summary.outcome,
  sentiment: analysis.results.sentiment.label,
  escalated: session.end_reason === 'transfer',
  quality_score: analysis.results.quality.score
});
```

## Best Practices

<AccordionGroup>
  <Accordion title="Keep system prompts focused">
    Avoid overly long system prompts. Focus on the most important behaviors and let the agent handle edge cases naturally.
  </Accordion>

  <Accordion title="Test with real conversations">
    Test your agent with actual customer conversations, not just scripted tests. Real conversations are more varied.
  </Accordion>

  <Accordion title="Monitor and iterate">
    Regularly review transcripts and analytics. Update your system prompt and tools based on actual conversations.
  </Accordion>

  <Accordion title="Set clear escalation paths">
    Always provide a way for customers to reach a human. Make the escalation trigger easy for the agent to understand.
  </Accordion>

  <Accordion title="Handle silence gracefully">
    If the customer is silent, have the agent prompt them gently rather than repeating or hanging up.
  </Accordion>
</AccordionGroup>