botbook/src/chapter-11-features/multi-agent-orchestration.md

Multi-Agent Orchestration

General Bots supports sophisticated multi-agent systems where multiple specialized bots collaborate to handle complex tasks. This chapter covers the architecture, keywords, and best practices for building multi-agent solutions.

Overview

Multi-agent orchestration enables:

• Task specialization - Each bot focuses on what it does best
• Collaborative problem-solving - Bots work together on complex tasks
• Scalable architectures - Add new specialists without modifying existing bots
• Resilient systems - Failures are isolated and handled gracefully

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Multi-Agent System                        │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌──────────┐    A2A Protocol    ┌──────────┐               │
│  │          │◄──────────────────►│          │               │
│  │ Sales    │                    │ Support  │               │
│  │ Bot      │    ┌──────────┐    │ Bot      │               │
│  │          │◄──►│          │◄──►│          │               │
│  └──────────┘    │ Billing  │    └──────────┘               │
│                  │ Bot      │                                │
│  ┌──────────┐    │          │    ┌──────────┐               │
│  │          │◄──►└──────────┘◄──►│          │               │
│  │ Research │                    │ Analytics│               │
│  │ Bot      │                    │ Bot      │               │
│  │          │                    │          │               │
│  └──────────┘                    └──────────┘               │
│                                                              │
└─────────────────────────────────────────────────────────────┘

Core Keywords

ADD BOT

Adds a bot to the current session with optional triggers and tools.

' Add a bot with keyword triggers
ADD BOT "billing-bot" TRIGGER ON "billing,invoice,payment"

' Add a bot with tool access
ADD BOT "analyst-bot" TOOLS "calculate,forecast,report"

' Add a bot with scheduled tasks
ADD BOT "monitor-bot" SCHEDULE "0 */1 * * *"

DELEGATE TO BOT

Sends a task to another bot and optionally waits for response.

' Fire-and-forget delegation
DELEGATE "Process this order" TO BOT "order-processor"

' Get response from delegation
result = DELEGATE "Calculate total for items" TO BOT "calculator-bot"
TALK "Total: " + result

' Delegation with timeout
result = DELEGATE "Analyze report" TO BOT "analyst-bot" TIMEOUT 60

BROADCAST TO BOTS

Sends a message to all bots in the session.

' Notify all bots of an event
BROADCAST "New customer signup: " + customerid TO BOTS

' Emergency shutdown signal
BROADCAST "SHUTDOWN" TO BOTS

TRANSFER CONVERSATION

Hands off the entire conversation to another bot.

' Transfer to specialist
TALK "Let me connect you with our billing specialist."
TRANSFER CONVERSATION TO "billing-bot"

' Transfer with context
SET CONTEXT "issue" AS "refund request"
SET CONTEXT "amount" AS "$150"
TRANSFER CONVERSATION TO "refunds-bot"

A2A Protocol

The Agent-to-Agent (A2A) protocol handles all inter-bot communication.

Message Types

Type	Description	Use Case
Request	Ask bot to perform task	Task delegation
Response	Reply to a request	Return results
Broadcast	Message to all bots	Notifications
Delegate	Hand off task	Specialization
Collaborate	Joint task	Team work

Message Structure

' A2A messages contain:
' - from_agent: Source bot ID
' - to_agent: Target bot ID  
' - message_type: Request, Response, etc.
' - payload: The actual content
' - correlation_id: Links request/response
' - timestamp: When sent

Configuration

name,value
a2a-enabled,true
a2a-timeout,30
a2a-max-hops,5
a2a-retry-count,3
a2a-queue-size,100

Option	Default	Description
a2a-enabled	true	Enable A2A communication
a2a-timeout	30	Default timeout (seconds)
a2a-max-hops	5	Maximum delegation chain depth
a2a-retry-count	3	Retries on failure
a2a-queue-size	100	Max pending messages

Memory Management

User Memory (Cross-Bot)

User memory is accessible across all bots, enabling seamless personalization.

' In any bot - store user preference
SET USER MEMORY "language", "pt-BR"
SET USER MEMORY "timezone", "America/Sao_Paulo"

' In any other bot - retrieve preference
language = GET USER MEMORY("language")
TALK "Olá!" IF language = "pt-BR"

Bot Memory (Per-Bot)

Bot memory is isolated to each bot for bot-specific state.

' In sales-bot
SET BOT MEMORY "deals_closed", dealscount

' In support-bot (different memory space)
SET BOT MEMORY "tickets_resolved", ticketcount

Session Memory (Temporary)

Session memory is shared within a conversation session.

' Store in session
SET "current_topic", "billing"

' Available to all bots in session
topic = GET "current_topic"

Memory Scope Comparison

Memory Type	Scope	Persistence	Use Case
User Memory	Per user, all bots	Permanent	Preferences, profile
Bot Memory	Per bot, all users	Permanent	Bot state, counters
Session Memory	Per session	Session lifetime	Current context

Model Routing

Different bots can use different models optimized for their tasks.

USE MODEL Keyword

' In customer service bot - use quality model
USE MODEL "quality"

' In quick-answer bot - use fast model
USE MODEL "fast"

' In code helper bot - use code model
USE MODEL "code"

' Let system decide
USE MODEL "auto"

Model Routing Strategies

Strategy	Description
manual	Explicit model selection only
auto	System chooses based on query
load-balanced	Distribute for throughput
fallback	Try models in order

Configuration

name,value
model-routing-strategy,auto
model-default,fast
model-fast,DeepSeek-R3-Distill-Qwen-1.5B-Q3_K_M.gguf
model-quality,gpt-4
model-code,codellama-7b.gguf

Hybrid RAG Search

Multi-agent systems benefit from shared knowledge bases with advanced search.

Configuration

name,value
rag-hybrid-enabled,true
rag-dense-weight,0.7
rag-sparse-weight,0.3
rag-reranker-enabled,true

How It Works

1. Dense Search - Semantic/vector similarity (0.7 weight)
2. Sparse Search - BM25 keyword matching (0.3 weight)
3. Fusion - Reciprocal Rank Fusion combines results
4. Reranking - Optional LLM reranking for quality

' Hybrid search is automatic when enabled
USE KB "company-knowledge"
result = FIND "customer refund policy"
' Returns results using both semantic and keyword matching

Code Sandbox

Bots can execute code in isolated sandboxes for data processing.

Supported Languages

' Python for data science
result = RUN PYTHON "
import pandas as pd
df = pd.DataFrame({'a': [1,2,3]})
print(df.sum().to_json())
"

' JavaScript for JSON processing
result = RUN JAVASCRIPT "
const data = {items: [1,2,3]};
console.log(JSON.stringify(data.items.map(x => x * 2)));
"

' Bash for system tasks
result = RUN BASH "ls -la /data"

Sandbox Configuration

name,value
sandbox-runtime,lxc
sandbox-timeout,30
sandbox-memory-mb,512
sandbox-cpu-percent,50
sandbox-network,false

Runtimes

Runtime	Security	Performance	Requirements
LXC	High	Excellent	LXC installed
Docker	High	Good	Docker daemon
Firecracker	Highest	Good	Firecracker
Process	Low	Best	None (fallback)

Agent Reflection

Bots can self-analyze and improve through reflection.

Enable Reflection

' Enable self-reflection
BOT REFLECTION true

' Monitor specific metrics
BOT REFLECTION ON "conversation_quality"
BOT REFLECTION ON "response_accuracy"

Get Insights

' Retrieve reflection analysis
insights = BOT REFLECTION INSIGHTS()

PRINT "Quality Score: " + insights.qualityScore
PRINT "Issues: " + insights.issuesCount

FOR EACH suggestion IN insights.suggestions
    PRINT "Suggestion: " + suggestion
NEXT suggestion

Reflection Metrics

Metric	Description
conversation_quality	Overall conversation effectiveness
response_accuracy	Correctness of responses
user_satisfaction	Estimated user satisfaction
tone_appropriateness	Tone matches context
resolution_rate	Issues successfully resolved

SSE Streaming

Real-time streaming for responsive multi-agent UIs.

Enable Streaming

name,value
sse-enabled,true
sse-heartbeat,30
sse-max-connections,1000

Client Integration

// Connect to SSE endpoint
const eventSource = new EventSource('/api/chat/stream?session=' + sessionId);

eventSource.onmessage = (event) => {
    const data = JSON.parse(event.data);
    
    if (data.type === 'token') {
        // Streaming token
        appendToMessage(data.content);
    } else if (data.type === 'bot_switch') {
        // Different bot responding
        showBotIndicator(data.botName);
    } else if (data.type === 'complete') {
        // Response complete
        finalizeMessage();
    }
};

Patterns and Best Practices

Router Pattern

A central router bot directs queries to specialists.

' router-bot/start.bas
HEAR userquery

' Classify the query
category = LLM "Classify into: billing, technical, sales, general. Query: " + userquery

SWITCH category
    CASE "billing"
        result = DELEGATE userquery TO BOT "billing-bot"
    CASE "technical"
        result = DELEGATE userquery TO BOT "tech-bot"
    CASE "sales"
        result = DELEGATE userquery TO BOT "sales-bot"
    CASE ELSE
        result = LLM userquery
END SWITCH

TALK result

Expert Panel Pattern

Multiple bots provide perspectives on complex questions.

' Get input from multiple experts
question = "Should we expand into the European market?"

marketAnalysis = DELEGATE question TO BOT "market-analyst"
financialView = DELEGATE question TO BOT "finance-expert"
riskAssessment = DELEGATE question TO BOT "risk-assessor"

' Synthesize responses
synthesis = LLM "Synthesize these expert opinions into a recommendation:
Market: " + marketAnalysis + "
Finance: " + financialView + "
Risk: " + riskAssessment

BEGIN TALK
**Expert Panel Summary**

📊 **Market Analysis:** {marketAnalysis}

💰 **Financial View:** {financialView}

⚠️ **Risk Assessment:** {riskAssessment}

📋 **Recommendation:** {synthesis}
END TALK

Escalation Pattern

Automatic escalation when bot can't handle query.

' First-line support bot
confidence = LLM "Rate your confidence (0-100) in answering: " + userquery

IF confidence < 50 THEN
    ' Escalate to specialist
    TALK "Let me connect you with a specialist who can better help."
    SET CONTEXT "escalation_reason" AS "low_confidence"
    SET CONTEXT "original_query" AS userquery
    TRANSFER CONVERSATION TO "senior-support-bot"
ELSE
    ' Handle normally
    response = LLM userquery
    TALK response
END IF

Supervisor Pattern

A supervisor bot monitors and coordinates workers.

' supervisor-bot/monitor.bas
SET SCHEDULE "*/5 * * * *"  ' Run every 5 minutes

' Check all worker bots
workers = ["processor-1", "processor-2", "processor-3"]

FOR EACH worker IN workers
    status = DELEGATE "HEALTH_CHECK" TO BOT worker TIMEOUT 10
    
    IF status = "" OR status = "ERROR" THEN
        ' Worker unresponsive
        SEND MAIL admin, "Bot Alert", worker + " is unresponsive"
        DELEGATE "RESTART" TO BOT "bot-manager"
    END IF
NEXT worker

Database Schema

Multi-agent systems use several database tables:

a2a_messages

Stores inter-agent communication.

Column	Type	Description
id	UUID	Message ID
from_agent	VARCHAR	Sender bot ID
to_agent	VARCHAR	Recipient bot ID
message_type	VARCHAR	Request, Response, etc.
payload	JSONB	Message content
correlation_id	UUID	Links request/response
status	VARCHAR	pending, delivered, failed
created_at	TIMESTAMP	When created

user_memory

Stores cross-bot user data.

Column	Type	Description
user_id	UUID	User identifier
key	VARCHAR	Memory key
value	JSONB	Stored value
memory_type	VARCHAR	preference, fact, context
ttl	TIMESTAMP	Optional expiration

agent_reflections

Stores reflection analysis results.

Column	Type	Description
id	UUID	Reflection ID
bot_id	UUID	Bot that was analyzed
conversation_id	UUID	Analyzed conversation
quality_score	FLOAT	Overall quality (0-1)
insights	JSONB	Analysis details
created_at	TIMESTAMP	When analyzed

Troubleshooting

Bot Not Responding to Delegation

1. Check bot is registered: LIST BOTS
2. Verify A2A is enabled: a2a-enabled,true
3. Check timeout is sufficient
4. Review bot logs for errors

Memory Not Sharing Between Bots

1. Ensure using SET USER MEMORY not SET BOT MEMORY
2. Check user-memory-enabled,true
3. Verify same user identity across bots

Circular Delegation Detected

1. Review delegation chains
2. Increase a2a-max-hops if legitimately deep
3. Add guards to prevent loops:

hops = GET "delegation_hops"
IF hops > 3 THEN
    TALK "I'll handle this directly."
    ' Don't delegate further
ELSE
    SET "delegation_hops", hops + 1
    DELEGATE task TO BOT "specialist"
END IF

See Also

• ADD BOT Keyword
• DELEGATE TO BOT Keyword
• Memory Management
• Model Routing
• Code Sandbox
• SSE Streaming