Building a Customer Support Bot from Scratch

Customer support is one of the most impactful use cases for AI. But a production support bot needs more than just an LLM — it needs a knowledge base, memory, confidence scoring, human escalation, and a full audit trail. Here’s how to build one with ElsiumAI.

Architecture Overview

Our support bot has four layers:

1. Knowledge Base (RAG) — ingests your docs, help articles, and FAQs into a vector store
2. Agent — takes user questions, retrieves relevant context, and generates answers
3. Safety Layer — confidence scoring, approval gates, and human escalation
4. Observability — audit trail, cost tracking, and performance monitoring

Setup

npm install @elsium-ai/agents @elsium-ai/rag @elsium-ai/observe @elsium-ai/app

Step 1: Build the Knowledge Base

First, set up a RAG pipeline that ingests your support docs into a vector store. ElsiumAI supports multiple document loaders and chunking strategies out of the box.

import { rag, createOpenAIEmbeddings, createPgVectorStore } from '@elsium-ai/rag'
import { mmrRerank, recursiveChunker, htmlLoader, textLoader } from '@elsium-ai/rag'

const knowledgeBase = rag({
  embeddings: createOpenAIEmbeddings({ model: 'text-embedding-3-small' }),
  store: createPgVectorStore({ connectionString: env('DATABASE_URL') }),
  chunker: recursiveChunker({ chunkSize: 512 }),
  reranker: mmrRerank({ lambda: 0.7 }),
  loaders: [htmlLoader(), textLoader()],
})

// Ingest your help center
await knowledgeBase.ingest('./docs/help-center/')
await knowledgeBase.ingest('./docs/faq.md')

The recursiveChunker splits documents into overlapping chunks so context isn’t lost at boundaries. The mmrRerank reranker ensures retrieved results are both relevant and diverse — avoiding five near-identical passages.

Step 2: Define the Support Agent

The agent is the brain of your support bot. It takes user messages, queries the knowledge base for relevant context, and generates a response grounded in your docs.

import { defineAgent, createMemory, createApprovalGate } from '@elsium-ai/agents'
import { createConfidenceScorer } from '@elsium-ai/agents'

const supportAgent = defineAgent({
  name: 'support',
  system: `You are a helpful customer support agent for Acme Corp.
Answer questions using ONLY the knowledge base context provided.
If you are not confident in your answer, say so honestly.
Never make up information.`,
  memory: createMemory(),
  tools: [rag({ store: knowledgeBase })],
  approvalGate: createApprovalGate({ threshold: 'high-risk' }),
})

Key features here:

• createMemory() gives the agent conversation history — users can ask follow-up questions naturally
• The rag() tool lets the agent search the knowledge base as part of its reasoning
• createApprovalGate flags high-risk responses (refund requests, account changes) for human review

Step 3: Add Confidence Scoring

Not every response should be sent automatically. Use confidence scoring to decide when the bot should answer directly vs. escalate to a human.

import { createConfidenceScorer } from '@elsium-ai/agents'

const scorer = createConfidenceScorer({
  thresholds: {
    auto: 0.85,      // Above 85%: send directly
    review: 0.60,    // 60-85%: queue for human review
    escalate: 0.0,   // Below 60%: escalate immediately
  },
})

const response = await supportAgent.run(userMessage)
const confidence = scorer.score(response)

if (confidence.level === 'auto') {
  return response              // Send to customer
} else if (confidence.level === 'review') {
  await queueForReview(response) // Human reviews before sending
} else {
  await escalateToHuman(userId)  // Transfer to live agent
}

Step 4: Add Audit Trail

For compliance, every interaction needs to be tracked — what the user asked, what the bot answered, which knowledge base articles were used, and whether a human was involved.

import { createAuditTrail } from '@elsium-ai/observe'

const audit = createAuditTrail({
  hashChain: true,    // Tamper-proof chain of events
  retention: '7y',    // Compliance retention period
})

// Attach to the agent
const supportAgent = defineAgent({
  name: 'support',
  system: 'You are a helpful support agent.',
  memory: createMemory(),
  tools: [rag({ store: knowledgeBase })],
  audit,
})

The hashChain option creates a cryptographic chain of events — each entry references the hash of the previous one. This makes it impossible to tamper with or delete individual records without detection.

Step 5: Serve with Streaming

Users expect real-time responses. Use SSE streaming to send tokens as they’re generated, giving the user instant feedback.

import { createApp } from '@elsium-ai/app'
import { sseHeaders, streamResponse } from '@elsium-ai/app'

const app = createApp({ port: 3000 })

app.post('/chat', async (req, res) => {
  const { message, userId } = req.body

  res.writeHead(200, sseHeaders())

  const stream = await supportAgent.run(message, {
    userId,
    stream: true,
  })

  await streamResponse(stream, res)
})

Step 6: Test Your Bot

Before deploying, write deterministic tests that verify your bot handles common scenarios correctly.

import { createTestAgent, mockRAG } from '@elsium-ai/testing'

const testAgent = createTestAgent({
  agent: supportAgent,
  tools: [mockRAG({
    'How do I reset my password?': 'Go to Settings > Security > Reset Password.',
    'What is the refund policy?': 'Full refund within 30 days of purchase.',
  })],
})

// Test normal question
const result = await testAgent.run('How do I reset my password?')
assert(result.content.includes('Settings'))

// Test unknown question triggers escalation
const unknown = await testAgent.run('Can you hack into another account?')
assert(unknown.confidence.level === 'escalate')

Production Checklist

What’s Next

From here you can extend your support bot with:

• Sentiment analysis — detect frustrated customers and prioritize their tickets
• Multi-language support — use the gateway to route to the best model per language
• Ticket classification — auto-tag and route tickets to the right team
• Analytics dashboard — track resolution rates, common topics, and escalation patterns with @elsium-ai/observe

Check out the @elsium-ai/agents docs and @elsium-ai/rag docs for the full API reference, or explore 30 more products you can build with ElsiumAI.