Add balanced documentation structure

Documentation organized with equilibrium: - Small (50-100 lines): Index files - Medium (250-400 lines): Guides - Large (450-600 lines): Complete references Structure: - docs/api/ - REST endpoints, WebSocket - docs/guides/ - Getting started, deployment, templates - docs/reference/ - BASIC language, configuration, architecture Updated README.md to point to new docs location.
2025-12-04 12:44:18 -03:00 · 2025-12-04 12:44:18 -03:00 · d1301c9cd8
commit d1301c9cd8
parent 78b359ab06
11 changed files with 3614 additions and 138 deletions
--- a/README.md
+++ b/README.md
@ -6,30 +6,9 @@

 ## Quick Links

- **[Complete Documentation](docs/src/SUMMARY.md)** - Full documentation index
- **[Quick Start Guide](docs/QUICK_START.md)** - Get started in minutes
- **[Changelog](CHANGELOG.md)** - Version history
-
-## Documentation Structure
-
-All documentation has been organized into the **[docs/](docs/)** directory:
-
-### Core Documentation
- **[Introduction & Getting Started](docs/src/chapter-01/README.md)**
- **[Package System](docs/src/chapter-02/README.md)**
- **[Knowledge Base Reference](docs/src/chapter-03/README.md)**
- **[User Interface](docs/src/chapter-04-gbui/README.md)**
- **[BASIC Dialogs](docs/src/chapter-06-gbdialog/README.md)**
- **[Architecture Reference](docs/src/chapter-07-gbapp/README.md)**
- **[Configuration](docs/src/chapter-08-config/README.md)**
- **[REST API Reference](docs/src/chapter-10-api/README.md)**
- **[Security & Authentication](docs/src/chapter-12-auth/README.md)**
-
-### Technical References
- **[KB & Tools System](docs/src/chapter-03/kb-and-tools.md)** - Core system architecture
- **[Semantic Cache](docs/src/chapter-03/caching.md)** - LLM caching with 70% cost reduction
- **[Universal Messaging](docs/src/chapter-06-gbdialog/universal-messaging.md)** - Multi-channel communication
- **[External Services](docs/src/appendix-external-services/README.md)** - Service integrations
+- **[Getting Started](docs/guides/getting-started.md)** - Installation and first bot
+- **[API Reference](docs/api/README.md)** - REST and WebSocket endpoints
+- **[BASIC Language](docs/reference/basic-language.md)** - Dialog scripting reference

 ## What is General Bots?

@ -39,45 +18,49 @@ General Bots is a **self-hosted AI automation platform** that provides:
 - **MCP + LLM Tools Generation** - Instant tool creation from code/functions
 - **Semantic Caching** - Intelligent response caching (70% cost reduction)
 - **Web Automation Engine** - Browser automation + AI intelligence
- **External Data APIs** - Integrated services via connectors
 - **Enterprise Data Connectors** - CRM, ERP, database native integrations
 - **Git-like Version Control** - Full history with rollback capabilities
- **Contract Analysis** - Legal document review and summary

-## Command-Line Options
+## Quick Start
+
+### Prerequisites
+
+- **Rust** (1.75+) - [Install from rustup.rs](https://rustup.rs/)
+- **Git** - [Download from git-scm.com](https://git-scm.com/downloads)
+
+### Installation

 ```bash
-# Run with default settings (console UI enabled)
+git clone https://github.com/GeneralBots/BotServer
+cd BotServer
 cargo run
-
-# Run without console UI
-cargo run -- --noconsole
-
-# Run in desktop mode (Tauri)
-cargo run -- --desktop
-
-# Run without any UI
-cargo run -- --noui
-
-# Specify tenant
-cargo run -- --tenant <tenant_name>
-
-# LXC container mode
-cargo run -- --container
 ```

-### Default Behavior
- **Console UI is enabled by default** - Shows real-time system status, logs, and file browser
- **Minimal UI is served by default** at `http://localhost:8080` - Lightweight, fast-loading interface
- Full suite UI available at `http://localhost:8080/suite` - Complete multi-application interface
- Use `--noconsole` to disable the terminal UI and run as a background service
- The HTTP server always runs on port 8080 unless in desktop mode
+On first run, BotServer automatically sets up PostgreSQL, S3 storage, Redis cache, and downloads AI models.

+The server will be available at `http://localhost:8080`.
+
+## Documentation
+
+```
+docs/
+├── api/                        # API documentation
+│   ├── README.md               # API overview
+│   ├── rest-endpoints.md       # HTTP endpoints
+│   └── websocket.md            # Real-time communication
+├── guides/                     # How-to guides
+│   ├── getting-started.md      # Quick start
+│   ├── deployment.md           # Production setup
+│   └── templates.md            # Using templates
+└── reference/                  # Technical reference
+    ├── basic-language.md       # BASIC keywords
+    ├── configuration.md        # Config options
+    └── architecture.md         # System design
+```

 ## Key Features

 ### 4 Essential Keywords
-General Bots provides a minimal, focused system for managing Knowledge Bases and Tools:

 ```basic
 USE KB "kb-name"        ' Load knowledge base into vector database
@ -86,66 +69,32 @@ USE TOOL "tool-name"    ' Make tool available to LLM
 CLEAR TOOLS             ' Remove all tools from session
 ```

-### Strategic Advantages
- **vs ChatGPT/Claude**: Automates entire business processes, not just chat
- **vs n8n/Make**: Simpler approach with little programming needed
- **vs Microsoft 365**: User control, not locked systems
- **vs Salesforce**: Open-source AI orchestration connecting all systems
+### Example Bot

-## Quick Start
+```basic
+' customer-support.bas
+USE KB "support-docs"
+USE TOOL "create-ticket"
+USE TOOL "check-order"

-### Prerequisites
- **Rust** (latest stable) - [Install from rustup.rs](https://rustup.rs/)
- **Git** (latest stable) - [Download from git-scm.com](https://git-scm.com/downloads)
+SET CONTEXT "support" AS "You are a helpful customer support agent."

-### Installation
-
-```bash
-# Clone the repository
-git clone https://github.com/GeneralBots/BotServer
-cd BotServer
-
-# Run the server (auto-installs dependencies)
-cargo run
+TALK "Welcome! How can I help you today?"
 ```

-On first run, BotServer automatically:
- Installs required components (PostgreSQL, S3-compatible storage, Cache, LLM)
- Sets up database with migrations
- Downloads AI models
- Uploads template bots
- Starts HTTP server at `http://127.0.0.1:8080`
-
-### Management Commands
-```bash
-botserver start              # Start all components
-botserver stop               # Stop all components
-botserver restart            # Restart all components
-botserver list               # List available components
-botserver status <component> # Check component status
-```
-
-## Current Status
-
-**Version:** 6.0.8  
-**Build Status:** SUCCESS  
-**Production Ready:** YES  
-**Compilation:** 0 errors  
-
-## Deployment
-
-General Bots supports deployment via **LXC containers** for isolated, lightweight virtualization:
+## Command-Line Options

 ```bash
-# Deploy with LXC container isolation
-cargo run -- --container
+cargo run                    # Default: console UI + web server
+cargo run -- --noconsole     # Background service mode
+cargo run -- --desktop       # Desktop application (Tauri)
+cargo run -- --tenant <name> # Specify tenant
+cargo run -- --container     # LXC container mode
 ```

-See [Container Deployment](docs/src/chapter-07-gbapp/containers.md) for detailed LXC setup instructions.
-
 ## Environment Variables

-General Bots uses minimal environment configuration. Only Directory service variables are required:
+Only directory service variables are required:

 | Variable | Purpose |
 |----------|---------|
@ -153,23 +102,32 @@ General Bots uses minimal environment configuration. Only Directory service vari
 | `DIRECTORY_CLIENT_ID` | OAuth client ID |
 | `DIRECTORY_CLIENT_SECRET` | OAuth client secret |

-All service credentials (database, storage, cache) are managed automatically by the Directory service. Application configuration is done through `config.csv` files in each bot's `.gbot` folder.
+All service credentials are managed automatically. See [Configuration](docs/reference/configuration.md) for details.

-See [Environment Variables](docs/src/appendix-env-vars/README.md) for details.
+## Current Status
+
+**Version:** 6.0.8  
+**Build Status:** SUCCESS  
+**Production Ready:** YES
+
+## Deployment
+
+See [Deployment Guide](docs/guides/deployment.md) for:
+
+- Single server setup
+- Docker Compose
+- LXC containers
+- Kubernetes
+- Reverse proxy configuration

 ## Contributing

-We welcome contributions! Please read:
- **[Contributing Guidelines](docs/src/chapter-13-community/README.md)**
- **[Development Setup](docs/src/chapter-13-community/setup.md)**
- **[Testing Guide](docs/src/chapter-13-community/testing.md)**
+We welcome contributions! Please read our contributing guidelines before submitting PRs.

 ## Security

 Security issues should be reported to: **security@pragmatismo.com.br**

-See **[Security Policy](docs/src/chapter-12-auth/security-policy.md)** for our security guidelines.
-
 ## License

 General Bot Copyright (c) pragmatismo.com.br. All rights reserved.  
@ -177,37 +135,12 @@ Licensed under the **AGPL-3.0**.

 According to our dual licensing model, this program can be used either under the terms of the GNU Affero General Public License, version 3, or under a proprietary license.

-See [LICENSE](LICENSE) for details.
+## Support

-## Key Facts
-
- LLM Orchestrator AGPL licensed (contribute back for custom-label SaaS)
- True community governance
- No single corporate control
- 5+ years of stability
- Never changed license
- Enterprise-grade
- Hosted locally or multicloud
-
-## Support & Resources
-
- **Documentation:** [docs.pragmatismo.com.br](https://docs.pragmatismo.com.br)
- **GitHub:** [github.com/GeneralBots/BotServer](https://github.com/GeneralBots/BotServer)
+- **GitHub Issues:** [github.com/GeneralBots/BotServer/issues](https://github.com/GeneralBots/BotServer/issues)
 - **Stack Overflow:** Tag questions with `generalbots`
 - **Video Tutorial:** [7 AI General Bots LLM Templates](https://www.youtube.com/watch?v=KJgvUPXi3Fw)

-## Demo
-
-See conversational data analytics in action:
-
-```basic
-TALK "General Bots Labs presents FISCAL DATA SHOW BY BASIC"
-result = GET "https://api.fiscaldata.treasury.gov/services/api/..."
-data = SELECT YEAR(record_date) as Yr, SUM(...) AS Amount FROM data
-img = CHART "bar", data
-SEND FILE img
-```
-
 ## Contributors

 <a href="https://github.com/generalbots/botserver/graphs/contributors">
@ -216,8 +149,6 @@ SEND FILE img

 ---

-**General Bots Code Name:** [Guaribas](https://en.wikipedia.org/wiki/Guaribas) (a city in Brazil, state of Piauí)
+**General Bots Code Name:** [Guaribas](https://en.wikipedia.org/wiki/Guaribas)

-> "No one should have to do work that can be done by a machine." - Roberto Mangabeira Unger
-
-<a href="https://stackoverflow.com/questions/ask?tags=generalbots">Ask a question</a> | <a href="https://github.com/GeneralBots/BotBook">Read the Docs</a>
+> "No one should have to do work that can be done by a machine." - Roberto Mangabeira Unger
--- a/docs/README.md
+++ b/docs/README.md
@ -0,0 +1,62 @@
+# General Bots Documentation
+
+Welcome to the General Bots documentation. This guide covers everything you need to build, deploy, and manage AI-powered bots.
+
+## Quick Navigation
+
+| Section | Description |
+|---------|-------------|
+| [Getting Started](guides/getting-started.md) | Installation and first bot |
+| [API Reference](api/README.md) | REST endpoints and WebSocket |
+| [BASIC Language](reference/basic-language.md) | Dialog scripting reference |
+| [Configuration](reference/configuration.md) | Environment and settings |
+
+## Documentation Structure
+
+```
+docs/
+├── api/                    # API documentation
+│   ├── README.md           # API overview
+│   ├── rest-endpoints.md   # HTTP endpoints
+│   └── websocket.md        # Real-time communication
+├── guides/                 # How-to guides
+│   ├── getting-started.md  # Quick start
+│   ├── deployment.md       # Production setup
+│   └── templates.md        # Using templates
+└── reference/              # Technical reference
+    ├── basic-language.md   # BASIC keywords
+    ├── configuration.md    # Config options
+    └── architecture.md     # System design
+```
+
+## Core Concepts
+
+### Knowledge Bases (KB)
+Store documents, FAQs, and data that the AI can search and reference:
+```basic
+USE KB "company-docs"
+```
+
+### Tools
+Functions the AI can call to perform actions:
+```basic
+USE TOOL "send-email"
+USE TOOL "create-ticket"
+```
+
+### Dialogs
+BASIC scripts that define conversation flows and automation:
+```basic
+TALK "Hello! How can I help?"
+answer = HEAR
+```
+
+## Quick Links
+
+- **[GitHub Repository](https://github.com/GeneralBots/BotServer)**
+- **[Issue Tracker](https://github.com/GeneralBots/BotServer/issues)**
+- **[Contributing Guide](../CONTRIBUTING.md)**
+
+## Version
+
+This documentation covers **General Bots v6.x**.
--- a/docs/api/README.md
+++ b/docs/api/README.md
@ -0,0 +1,104 @@
+# API Reference
+
+General Bots exposes a REST API and WebSocket interface for integration with external systems.
+
+## Base URL
+
+```
+http://localhost:8080/api
+```
+
+## Authentication
+
+All API requests require authentication via Bearer token:
+
+```bash
+curl -H "Authorization: Bearer <token>" \
+     http://localhost:8080/api/sessions
+```
+
+## Endpoints Overview
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/api/sessions` | GET | List active sessions |
+| `/api/sessions` | POST | Create new session |
+| `/api/sessions/:id` | GET | Get session details |
+| `/api/sessions/:id/messages` | POST | Send message |
+| `/api/drive/*` | * | File storage operations |
+| `/api/tasks/*` | * | Task management |
+| `/api/email/*` | * | Email operations |
+| `/api/calendar/*` | * | Calendar/CalDAV |
+| `/api/meet/*` | * | Video meetings |
+| `/api/kb/*` | * | Knowledge base search |
+| `/api/analytics/*` | * | Analytics dashboard |
+
+## WebSocket
+
+Real-time communication via WebSocket:
+
+```
+ws://localhost:8080/ws
+```
+
+### Connection
+
+```javascript
+const ws = new WebSocket('ws://localhost:8080/ws');
+ws.onmessage = (event) => {
+    const data = JSON.parse(event.data);
+    console.log('Received:', data);
+};
+```
+
+### Message Format
+
+```json
+{
+    "type": "message",
+    "session_id": "uuid",
+    "content": "Hello bot",
+    "timestamp": "2024-01-01T00:00:00Z"
+}
+```
+
+## Response Format
+
+All responses follow a consistent structure:
+
+### Success
+
+```json
+{
+    "success": true,
+    "data": { ... }
+}
+```
+
+### Error
+
+```json
+{
+    "success": false,
+    "error": {
+        "code": "NOT_FOUND",
+        "message": "Resource not found"
+    }
+}
+```
+
+## Rate Limiting
+
+API requests are rate limited per IP:
+
+| Endpoint Type | Requests/Second | Burst |
+|--------------|-----------------|-------|
+| Standard API | 100 | 200 |
+| Auth endpoints | 10 | 20 |
+| LLM endpoints | 5 | 10 |
+
+## Detailed Documentation
+
+- [REST Endpoints](rest-endpoints.md) - Complete endpoint reference
+- [WebSocket API](websocket.md) - Real-time communication
+- [HTMX Integration](htmx.md) - Frontend patterns
--- a/docs/api/rest-endpoints.md
+++ b/docs/api/rest-endpoints.md
@ -0,0 +1,599 @@
+# REST API Endpoints
+
+Complete reference for all General Bots REST API endpoints.
+
+## Sessions
+
+### List Sessions
+
+```http
+GET /api/sessions
+```
+
+**Query Parameters:**
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `limit` | integer | Max results (default: 50) |
+| `offset` | integer | Pagination offset |
+| `status` | string | Filter by status: `active`, `closed` |
+
+**Response:**
+
+```json
+{
+    "sessions": [
+        {
+            "id": "550e8400-e29b-41d4-a716-446655440000",
+            "user_id": "user@example.com",
+            "bot_id": "default",
+            "status": "active",
+            "created_at": "2024-01-01T10:00:00Z",
+            "last_activity": "2024-01-01T10:30:00Z"
+        }
+    ],
+    "total": 1,
+    "limit": 50,
+    "offset": 0
+}
+```
+
+### Create Session
+
+```http
+POST /api/sessions
+```
+
+**Request Body:**
+
+```json
+{
+    "bot_id": "default",
+    "user_id": "user@example.com",
+    "metadata": {
+        "channel": "web",
+        "language": "en"
+    }
+}
+```
+
+### Get Session
+
+```http
+GET /api/sessions/:id
+```
+
+### Send Message
+
+```http
+POST /api/sessions/:id/messages
+```
+
+**Request Body:**
+
+```json
+{
+    "content": "Hello, I need help",
+    "type": "text"
+}
+```
+
+**Response:**
+
+```json
+{
+    "id": "msg-uuid",
+    "session_id": "session-uuid",
+    "role": "assistant",
+    "content": "Hello! How can I assist you today?",
+    "timestamp": "2024-01-01T10:31:00Z"
+}
+```
+
+---
+
+## Drive (File Storage)
+
+### List Files
+
+```http
+GET /api/drive
+GET /api/drive/:path
+```
+
+**Query Parameters:**
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `recursive` | boolean | Include subdirectories |
+| `type` | string | Filter: `file`, `folder` |
+
+### Upload File
+
+```http
+POST /api/drive/:path
+Content-Type: multipart/form-data
+```
+
+### Download File
+
+```http
+GET /api/drive/:path/download
+```
+
+### Delete File
+
+```http
+DELETE /api/drive/:path
+```
+
+---
+
+## Tasks
+
+### List Tasks
+
+```http
+GET /api/tasks
+```
+
+**Query Parameters:**
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `status` | string | `pending`, `completed`, `cancelled` |
+| `priority` | string | `low`, `medium`, `high` |
+| `due_before` | datetime | Filter by due date |
+
+### Create Task
+
+```http
+POST /api/tasks
+```
+
+**Request Body:**
+
+```json
+{
+    "title": "Follow up with client",
+    "description": "Send proposal document",
+    "due_date": "2024-01-15T17:00:00Z",
+    "priority": "high",
+    "assignee": "user@example.com"
+}
+```
+
+### Update Task
+
+```http
+PUT /api/tasks/:id
+```
+
+### Delete Task
+
+```http
+DELETE /api/tasks/:id
+```
+
+---
+
+## Email
+
+### List Emails
+
+```http
+GET /api/email
+GET /api/email/:folder
+```
+
+**Folders:** `inbox`, `sent`, `drafts`, `trash`
+
+### Send Email
+
+```http
+POST /api/email/send
+```
+
+**Request Body:**
+
+```json
+{
+    "to": ["recipient@example.com"],
+    "cc": [],
+    "bcc": [],
+    "subject": "Meeting Tomorrow",
+    "body": "Hi, let's meet at 3pm.",
+    "html": false,
+    "attachments": []
+}
+```
+
+### Get Email
+
+```http
+GET /api/email/:id
+```
+
+### Delete Email
+
+```http
+DELETE /api/email/:id
+```
+
+---
+
+## Calendar
+
+### List Events
+
+```http
+GET /api/calendar/events
+```
+
+**Query Parameters:**
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `start` | datetime | Range start |
+| `end` | datetime | Range end |
+| `calendar_id` | string | Specific calendar |
+
+### Create Event
+
+```http
+POST /api/calendar/events
+```
+
+**Request Body:**
+
+```json
+{
+    "title": "Team Meeting",
+    "start_time": "2024-01-15T14:00:00Z",
+    "end_time": "2024-01-15T15:00:00Z",
+    "location": "Conference Room A",
+    "attendees": ["team@example.com"],
+    "recurrence": null
+}
+```
+
+### Export Calendar (iCal)
+
+```http
+GET /api/calendar/export.ics
+```
+
+### Import Calendar
+
+```http
+POST /api/calendar/import
+Content-Type: text/calendar
+```
+
+---
+
+## Meet (Video Conferencing)
+
+### Create Room
+
+```http
+POST /api/meet/rooms
+```
+
+**Request Body:**
+
+```json
+{
+    "name": "Team Standup",
+    "scheduled_start": "2024-01-15T09:00:00Z",
+    "max_participants": 10
+}
+```
+
+**Response:**
+
+```json
+{
+    "room_id": "room-uuid",
+    "join_url": "https://meet.example.com/room-uuid",
+    "token": "participant-token"
+}
+```
+
+### Join Room
+
+```http
+POST /api/meet/rooms/:id/join
+```
+
+### List Participants
+
+```http
+GET /api/meet/rooms/:id/participants
+```
+
+---
+
+## Knowledge Base
+
+### Search
+
+```http
+GET /api/kb/search
+```
+
+**Query Parameters:**
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `q` | string | Search query (required) |
+| `limit` | integer | Max results (default: 10) |
+| `threshold` | float | Similarity threshold (0-1) |
+| `collection` | string | Specific KB collection |
+
+**Response:**
+
+```json
+{
+    "results": [
+        {
+            "id": "doc-uuid",
+            "content": "Relevant document content...",
+            "score": 0.95,
+            "metadata": {
+                "source": "company-docs",
+                "title": "Employee Handbook"
+            }
+        }
+    ],
+    "query": "vacation policy",
+    "total": 5
+}
+```
+
+### List Collections
+
+```http
+GET /api/kb/collections
+```
+
+### Reindex Collection
+
+```http
+POST /api/kb/reindex
+```
+
+---
+
+## Analytics
+
+### Dashboard Stats
+
+```http
+GET /api/analytics/stats
+```
+
+**Query Parameters:**
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `time_range` | string | `day`, `week`, `month`, `year` |
+
+**Response:**
+
+```json
+{
+    "total_messages": 15420,
+    "active_sessions": 45,
+    "avg_response_time_ms": 230,
+    "error_rate": 0.02,
+    "top_queries": [
+        {"query": "password reset", "count": 120}
+    ]
+}
+```
+
+### Message Trends
+
+```http
+GET /api/analytics/messages/trend
+```
+
+---
+
+## Paper (Documents)
+
+### List Documents
+
+```http
+GET /api/paper
+```
+
+### Create Document
+
+```http
+POST /api/paper
+```
+
+**Request Body:**
+
+```json
+{
+    "title": "Meeting Notes",
+    "content": "# Meeting Notes\n\n...",
+    "type": "note"
+}
+```
+
+### Get Document
+
+```http
+GET /api/paper/:id
+```
+
+### Update Document
+
+```http
+PUT /api/paper/:id
+```
+
+### Delete Document
+
+```http
+DELETE /api/paper/:id
+```
+
+---
+
+## Designer (Bot Builder)
+
+### List Dialogs
+
+```http
+GET /api/designer/dialogs
+```
+
+### Create Dialog
+
+```http
+POST /api/designer/dialogs
+```
+
+**Request Body:**
+
+```json
+{
+    "name": "greeting",
+    "content": "TALK \"Hello!\"\nanswer = HEAR"
+}
+```
+
+### Validate Dialog
+
+```http
+POST /api/designer/dialogs/:id/validate
+```
+
+**Response:**
+
+```json
+{
+    "valid": true,
+    "errors": [],
+    "warnings": ["Line 15: Consider using END IF"]
+}
+```
+
+### Deploy Dialog
+
+```http
+POST /api/designer/dialogs/:id/deploy
+```
+
+---
+
+## Sources (Templates)
+
+### List Templates
+
+```http
+GET /api/sources/templates
+```
+
+**Query Parameters:**
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `category` | string | Filter by category |
+
+### Get Template
+
+```http
+GET /api/sources/templates/:id
+```
+
+### Use Template
+
+```http
+POST /api/sources/templates/:id/use
+```
+
+---
+
+## Admin
+
+### System Stats
+
+```http
+GET /api/admin/stats
+```
+
+**Response:**
+
+```json
+{
+    "uptime_seconds": 86400,
+    "memory_used_mb": 512,
+    "active_connections": 23,
+    "database_size_mb": 1024,
+    "cache_hit_rate": 0.85
+}
+```
+
+### Health Check
+
+```http
+GET /api/health
+```
+
+**Response:**
+
+```json
+{
+    "status": "healthy",
+    "components": {
+        "database": "ok",
+        "cache": "ok",
+        "storage": "ok",
+        "llm": "ok"
+    }
+}
+```
+
+---
+
+## Error Codes
+
+| Code | HTTP Status | Description |
+|------|-------------|-------------|
+| `BAD_REQUEST` | 400 | Invalid request parameters |
+| `UNAUTHORIZED` | 401 | Missing or invalid auth token |
+| `FORBIDDEN` | 403 | Insufficient permissions |
+| `NOT_FOUND` | 404 | Resource not found |
+| `CONFLICT` | 409 | Resource already exists |
+| `RATE_LIMITED` | 429 | Too many requests |
+| `INTERNAL_ERROR` | 500 | Server error |
+
+---
+
+## Pagination
+
+List endpoints support pagination:
+
+```http
+GET /api/tasks?limit=20&offset=40
+```
+
+Response includes pagination info:
+
+```json
+{
+    "data": [...],
+    "pagination": {
+        "total": 150,
+        "limit": 20,
+        "offset": 40,
+        "has_more": true
+    }
+}
+```
--- a/docs/api/websocket.md
+++ b/docs/api/websocket.md
@ -0,0 +1,362 @@
+# WebSocket API
+
+Real-time bidirectional communication with General Bots.
+
+## Connection
+
+### Endpoint
+
+```
+ws://localhost:8080/ws
+wss://your-domain.com/ws  (production)
+```
+
+### Authentication
+
+Include the auth token as a query parameter or in the first message:
+
+```javascript
+// Option 1: Query parameter
+const ws = new WebSocket('ws://localhost:8080/ws?token=<auth_token>');
+
+// Option 2: First message
+ws.onopen = () => {
+    ws.send(JSON.stringify({
+        type: 'auth',
+        token: '<auth_token>'
+    }));
+};
+```
+
+## Message Format
+
+All messages are JSON objects with a `type` field:
+
+```json
+{
+    "type": "message_type",
+    "payload": { ... },
+    "timestamp": "2024-01-01T10:00:00Z"
+}
+```
+
+## Client Messages
+
+### Send Chat Message
+
+```json
+{
+    "type": "message",
+    "session_id": "550e8400-e29b-41d4-a716-446655440000",
+    "content": "Hello, I need help with my order"
+}
+```
+
+### Start Typing
+
+```json
+{
+    "type": "typing_start",
+    "session_id": "550e8400-e29b-41d4-a716-446655440000"
+}
+```
+
+### Stop Typing
+
+```json
+{
+    "type": "typing_stop",
+    "session_id": "550e8400-e29b-41d4-a716-446655440000"
+}
+```
+
+### Subscribe to Session
+
+```json
+{
+    "type": "subscribe",
+    "session_id": "550e8400-e29b-41d4-a716-446655440000"
+}
+```
+
+### Unsubscribe from Session
+
+```json
+{
+    "type": "unsubscribe",
+    "session_id": "550e8400-e29b-41d4-a716-446655440000"
+}
+```
+
+### Ping (Keep-Alive)
+
+```json
+{
+    "type": "ping"
+}
+```
+
+## Server Messages
+
+### Chat Response
+
+```json
+{
+    "type": "message",
+    "session_id": "550e8400-e29b-41d4-a716-446655440000",
+    "message_id": "msg-uuid",
+    "role": "assistant",
+    "content": "I'd be happy to help! What's your order number?",
+    "timestamp": "2024-01-01T10:00:01Z"
+}
+```
+
+### Streaming Response
+
+For LLM responses, content streams in chunks:
+
+```json
+{
+    "type": "stream_start",
+    "session_id": "550e8400-e29b-41d4-a716-446655440000",
+    "message_id": "msg-uuid"
+}
+```
+
+```json
+{
+    "type": "stream_chunk",
+    "message_id": "msg-uuid",
+    "content": "I'd be happy to "
+}
+```
+
+```json
+{
+    "type": "stream_chunk",
+    "message_id": "msg-uuid",
+    "content": "help! What's your "
+}
+```
+
+```json
+{
+    "type": "stream_end",
+    "message_id": "msg-uuid",
+    "content": "I'd be happy to help! What's your order number?"
+}
+```
+
+### Bot Typing Indicator
+
+```json
+{
+    "type": "bot_typing",
+    "session_id": "550e8400-e29b-41d4-a716-446655440000",
+    "is_typing": true
+}
+```
+
+### Tool Execution
+
+When the bot calls a tool:
+
+```json
+{
+    "type": "tool_call",
+    "session_id": "550e8400-e29b-41d4-a716-446655440000",
+    "tool": "search_orders",
+    "arguments": {"order_id": "12345"}
+}
+```
+
+```json
+{
+    "type": "tool_result",
+    "session_id": "550e8400-e29b-41d4-a716-446655440000",
+    "tool": "search_orders",
+    "result": {"status": "shipped", "tracking": "1Z999..."}
+}
+```
+
+### Error
+
+```json
+{
+    "type": "error",
+    "code": "SESSION_NOT_FOUND",
+    "message": "Session does not exist or has expired"
+}
+```
+
+### Pong (Keep-Alive Response)
+
+```json
+{
+    "type": "pong",
+    "timestamp": "2024-01-01T10:00:00Z"
+}
+```
+
+### Session Events
+
+```json
+{
+    "type": "session_created",
+    "session_id": "550e8400-e29b-41d4-a716-446655440000"
+}
+```
+
+```json
+{
+    "type": "session_closed",
+    "session_id": "550e8400-e29b-41d4-a716-446655440000",
+    "reason": "user_disconnect"
+}
+```
+
+## JavaScript Client Example
+
+```javascript
+class BotClient {
+    constructor(url, token) {
+        this.url = url;
+        this.token = token;
+        this.ws = null;
+        this.handlers = {};
+    }
+
+    connect() {
+        this.ws = new WebSocket(`${this.url}?token=${this.token}`);
+        
+        this.ws.onopen = () => {
+            console.log('Connected to bot');
+            this.emit('connected');
+        };
+        
+        this.ws.onmessage = (event) => {
+            const data = JSON.parse(event.data);
+            this.emit(data.type, data);
+        };
+        
+        this.ws.onclose = () => {
+            console.log('Disconnected');
+            this.emit('disconnected');
+            // Auto-reconnect after 3 seconds
+            setTimeout(() => this.connect(), 3000);
+        };
+        
+        this.ws.onerror = (error) => {
+            console.error('WebSocket error:', error);
+            this.emit('error', error);
+        };
+    }
+
+    send(type, payload) {
+        if (this.ws?.readyState === WebSocket.OPEN) {
+            this.ws.send(JSON.stringify({ type, ...payload }));
+        }
+    }
+
+    sendMessage(sessionId, content) {
+        this.send('message', { session_id: sessionId, content });
+    }
+
+    subscribe(sessionId) {
+        this.send('subscribe', { session_id: sessionId });
+    }
+
+    on(event, handler) {
+        this.handlers[event] = this.handlers[event] || [];
+        this.handlers[event].push(handler);
+    }
+
+    emit(event, data) {
+        (this.handlers[event] || []).forEach(h => h(data));
+    }
+
+    disconnect() {
+        this.ws?.close();
+    }
+}
+
+// Usage
+const client = new BotClient('ws://localhost:8080/ws', 'auth-token');
+
+client.on('connected', () => {
+    client.subscribe('session-uuid');
+});
+
+client.on('message', (data) => {
+    console.log('Bot:', data.content);
+});
+
+client.on('stream_chunk', (data) => {
+    process.stdout.write(data.content);
+});
+
+client.on('error', (data) => {
+    console.error('Error:', data.message);
+});
+
+client.connect();
+client.sendMessage('session-uuid', 'Hello!');
+```
+
+## Meet WebSocket
+
+Video conferencing uses a separate WebSocket endpoint:
+
+```
+ws://localhost:8080/ws/meet
+```
+
+### Join Room
+
+```json
+{
+    "type": "join",
+    "room_id": "room-uuid",
+    "participant_name": "John"
+}
+```
+
+### Leave Room
+
+```json
+{
+    "type": "leave",
+    "room_id": "room-uuid"
+}
+```
+
+### Signaling (WebRTC)
+
+```json
+{
+    "type": "signal",
+    "room_id": "room-uuid",
+    "target_id": "participant-uuid",
+    "signal": { /* WebRTC signal data */ }
+}
+```
+
+## Connection Limits
+
+| Limit | Value |
+|-------|-------|
+| Max connections per IP | 100 |
+| Max message size | 64 KB |
+| Idle timeout | 5 minutes |
+| Ping interval | 30 seconds |
+
+## Error Codes
+
+| Code | Description |
+|------|-------------|
+| `AUTH_FAILED` | Invalid or expired token |
+| `SESSION_NOT_FOUND` | Session doesn't exist |
+| `RATE_LIMITED` | Too many messages |
+| `MESSAGE_TOO_LARGE` | Exceeds 64 KB limit |
+| `INVALID_FORMAT` | Malformed JSON |
+| `SUBSCRIPTION_FAILED` | Cannot subscribe to session |
--- a/docs/guides/deployment.md
+++ b/docs/guides/deployment.md
@ -0,0 +1,465 @@
+# Deployment Guide
+
+Deploy General Bots in production environments with security, scalability, and reliability.
+
+## Deployment Options
+
+| Method | Best For | Complexity |
+|--------|----------|------------|
+| Single Server | Small teams, development | Low |
+| Docker Compose | Medium deployments | Medium |
+| LXC Containers | Isolated multi-tenant | Medium |
+| Kubernetes | Large scale, high availability | High |
+
+## Single Server Deployment
+
+### Requirements
+
+- **CPU**: 4+ cores
+- **RAM**: 16GB minimum
+- **Disk**: 100GB SSD
+- **OS**: Ubuntu 22.04 LTS / Debian 12
+
+### Installation
+
+```bash
+# Install Rust
+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
+source ~/.cargo/env
+
+# Clone and build
+git clone https://github.com/GeneralBots/BotServer
+cd BotServer
+cargo build --release
+
+# Run as service
+sudo cp target/release/botserver /usr/local/bin/
+sudo cp scripts/botserver.service /etc/systemd/system/
+sudo systemctl enable botserver
+sudo systemctl start botserver
+```
+
+### Systemd Service
+
+```ini
+# /etc/systemd/system/botserver.service
+[Unit]
+Description=General Bots Server
+After=network.target postgresql.service
+
+[Service]
+Type=simple
+User=botserver
+Group=botserver
+WorkingDirectory=/opt/botserver
+ExecStart=/usr/local/bin/botserver --noconsole
+Restart=always
+RestartSec=5
+Environment=RUST_LOG=info
+
+[Install]
+WantedBy=multi-user.target
+```
+
+## Docker Deployment
+
+### Docker Compose
+
+```yaml
+# docker-compose.yml
+version: '3.8'
+
+services:
+  botserver:
+    image: generalbots/botserver:latest
+    ports:
+      - "8080:8080"
+    environment:
+      - DATABASE_URL=postgres://bot:password@postgres/botserver
+      - REDIS_URL=redis://redis:6379
+      - S3_ENDPOINT=http://minio:9000
+    depends_on:
+      - postgres
+      - redis
+      - minio
+    volumes:
+      - ./templates:/app/templates
+      - ./data:/app/data
+    restart: unless-stopped
+
+  postgres:
+    image: postgres:15
+    environment:
+      - POSTGRES_USER=bot
+      - POSTGRES_PASSWORD=password
+      - POSTGRES_DB=botserver
+    volumes:
+      - postgres_data:/var/lib/postgresql/data
+    restart: unless-stopped
+
+  redis:
+    image: redis:7-alpine
+    volumes:
+      - redis_data:/data
+    restart: unless-stopped
+
+  minio:
+    image: minio/minio
+    command: server /data --console-address ":9001"
+    environment:
+      - MINIO_ROOT_USER=minioadmin
+      - MINIO_ROOT_PASSWORD=minioadmin
+    volumes:
+      - minio_data:/data
+    ports:
+      - "9001:9001"
+    restart: unless-stopped
+
+  qdrant:
+    image: qdrant/qdrant
+    volumes:
+      - qdrant_data:/qdrant/storage
+    restart: unless-stopped
+
+volumes:
+  postgres_data:
+  redis_data:
+  minio_data:
+  qdrant_data:
+```
+
+### Start Services
+
+```bash
+docker-compose up -d
+docker-compose logs -f botserver
+```
+
+## LXC Container Deployment
+
+LXC provides lightweight isolation for multi-tenant deployments.
+
+### Create Container
+
+```bash
+# Create container
+lxc launch ubuntu:22.04 botserver
+
+# Configure resources
+lxc config set botserver limits.cpu 4
+lxc config set botserver limits.memory 8GB
+
+# Enter container
+lxc exec botserver -- bash
+```
+
+### Inside Container
+
+```bash
+# Install dependencies
+apt update && apt install -y curl build-essential
+
+# Install Rust and build
+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
+source ~/.cargo/env
+git clone https://github.com/GeneralBots/BotServer
+cd BotServer
+cargo build --release
+
+# Run
+./target/release/botserver --container
+```
+
+### Container Networking
+
+```bash
+# Forward port from host
+lxc config device add botserver http proxy \
+  listen=tcp:0.0.0.0:8080 \
+  connect=tcp:127.0.0.1:8080
+```
+
+## Reverse Proxy Setup
+
+### Nginx
+
+```nginx
+# /etc/nginx/sites-available/botserver
+upstream botserver {
+    server 127.0.0.1:8080;
+    keepalive 32;
+}
+
+server {
+    listen 80;
+    server_name bot.example.com;
+    return 301 https://$server_name$request_uri;
+}
+
+server {
+    listen 443 ssl http2;
+    server_name bot.example.com;
+
+    ssl_certificate /etc/letsencrypt/live/bot.example.com/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/bot.example.com/privkey.pem;
+
+    # Security headers
+    add_header X-Frame-Options DENY;
+    add_header X-Content-Type-Options nosniff;
+    add_header X-XSS-Protection "1; mode=block";
+
+    location / {
+        proxy_pass http://botserver;
+        proxy_http_version 1.1;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+    }
+
+    # WebSocket support
+    location /ws {
+        proxy_pass http://botserver;
+        proxy_http_version 1.1;
+        proxy_set_header Upgrade $http_upgrade;
+        proxy_set_header Connection "upgrade";
+        proxy_set_header Host $host;
+        proxy_read_timeout 86400;
+    }
+}
+```
+
+### Enable Site
+
+```bash
+sudo ln -s /etc/nginx/sites-available/botserver /etc/nginx/sites-enabled/
+sudo nginx -t
+sudo systemctl reload nginx
+```
+
+## SSL Certificates
+
+### Let's Encrypt
+
+```bash
+sudo apt install certbot python3-certbot-nginx
+sudo certbot --nginx -d bot.example.com
+```
+
+### Auto-Renewal
+
+```bash
+sudo certbot renew --dry-run
+```
+
+## Environment Configuration
+
+### Production Environment
+
+```bash
+# /opt/botserver/.env
+RUST_LOG=warn,botserver=info
+
+# Directory Service (required)
+DIRECTORY_URL=https://auth.example.com
+DIRECTORY_CLIENT_ID=your-client-id
+DIRECTORY_CLIENT_SECRET=your-secret
+
+# Optional overrides
+DATABASE_URL=postgres://user:pass@localhost/botserver
+REDIS_URL=redis://localhost:6379
+S3_ENDPOINT=https://s3.example.com
+
+# Rate limiting
+RATE_LIMIT_ENABLED=true
+RATE_LIMIT_API_RPS=100
+```
+
+## Database Setup
+
+### PostgreSQL Production Config
+
+```sql
+-- Create database and user
+CREATE USER botserver WITH PASSWORD 'secure_password';
+CREATE DATABASE botserver OWNER botserver;
+GRANT ALL PRIVILEGES ON DATABASE botserver TO botserver;
+
+-- Enable extensions
+\c botserver
+CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
+CREATE EXTENSION IF NOT EXISTS "pg_trgm";
+```
+
+### Connection Pooling (PgBouncer)
+
+```ini
+# /etc/pgbouncer/pgbouncer.ini
+[databases]
+botserver = host=localhost dbname=botserver
+
+[pgbouncer]
+listen_addr = 127.0.0.1
+listen_port = 6432
+auth_type = md5
+pool_mode = transaction
+max_client_conn = 1000
+default_pool_size = 20
+```
+
+## Monitoring
+
+### Health Check Endpoint
+
+```bash
+curl http://localhost:8080/api/health
+```
+
+### Prometheus Metrics
+
+```yaml
+# prometheus.yml
+scrape_configs:
+  - job_name: 'botserver'
+    static_configs:
+      - targets: ['localhost:8080']
+    metrics_path: /metrics
+```
+
+### Log Aggregation
+
+```bash
+# Stream logs to file
+journalctl -u botserver -f >> /var/log/botserver/app.log
+
+# Logrotate config
+# /etc/logrotate.d/botserver
+/var/log/botserver/*.log {
+    daily
+    rotate 14
+    compress
+    delaycompress
+    missingok
+    notifempty
+}
+```
+
+## Backup Strategy
+
+### Database Backup
+
+```bash
+#!/bin/bash
+# /opt/botserver/scripts/backup.sh
+DATE=$(date +%Y%m%d_%H%M%S)
+BACKUP_DIR=/backups/botserver
+
+# PostgreSQL
+pg_dump -U botserver botserver | gzip > $BACKUP_DIR/db_$DATE.sql.gz
+
+# File storage
+tar -czf $BACKUP_DIR/files_$DATE.tar.gz /opt/botserver/data
+
+# Retain 30 days
+find $BACKUP_DIR -mtime +30 -delete
+```
+
+### Cron Schedule
+
+```bash
+# Daily backup at 2 AM
+0 2 * * * /opt/botserver/scripts/backup.sh
+```
+
+## Security Checklist
+
+- [ ] Run as non-root user
+- [ ] Enable firewall (only ports 80, 443)
+- [ ] Configure SSL/TLS
+- [ ] Set secure file permissions
+- [ ] Enable rate limiting
+- [ ] Configure authentication
+- [ ] Regular security updates
+- [ ] Audit logging enabled
+- [ ] Backup encryption
+- [ ] Network isolation for database
+
+## Scaling
+
+### Horizontal Scaling
+
+```yaml
+# docker-compose.scale.yml
+services:
+  botserver:
+    deploy:
+      replicas: 3
+    
+  nginx:
+    image: nginx
+    ports:
+      - "80:80"
+    volumes:
+      - ./nginx-lb.conf:/etc/nginx/nginx.conf
+```
+
+### Load Balancer Config
+
+```nginx
+upstream botserver_cluster {
+    least_conn;
+    server botserver1:8080;
+    server botserver2:8080;
+    server botserver3:8080;
+}
+```
+
+## Troubleshooting
+
+### Check Service Status
+
+```bash
+sudo systemctl status botserver
+journalctl -u botserver -n 100
+```
+
+### Database Connection Issues
+
+```bash
+psql -U botserver -h localhost -d botserver -c "SELECT 1"
+```
+
+### Memory Issues
+
+```bash
+# Check memory usage
+free -h
+cat /proc/meminfo | grep -E "MemTotal|MemFree|Cached"
+
+# Increase swap if needed
+sudo fallocate -l 4G /swapfile
+sudo chmod 600 /swapfile
+sudo mkswap /swapfile
+sudo swapon /swapfile
+```
+
+## Updates
+
+### Rolling Update
+
+```bash
+# Build new version
+cd /opt/botserver
+git pull
+cargo build --release
+
+# Graceful restart
+sudo systemctl reload botserver
+```
+
+### Zero-Downtime with Docker
+
+```bash
+docker-compose pull
+docker-compose up -d --no-deps --build botserver
+```
--- a/docs/guides/getting-started.md
+++ b/docs/guides/getting-started.md
@ -0,0 +1,248 @@
+# Getting Started with General Bots
+
+This guide will help you install, configure, and run your first General Bots instance.
+
+## Prerequisites
+
+- **Rust** (1.75 or later) - [Install from rustup.rs](https://rustup.rs/)
+- **Git** - [Download from git-scm.com](https://git-scm.com/downloads)
+- **8GB RAM** minimum (16GB recommended)
+- **10GB disk space** for dependencies and data
+
+## Installation
+
+### 1. Clone the Repository
+
+```bash
+git clone https://github.com/GeneralBots/BotServer
+cd BotServer
+```
+
+### 2. Run the Server
+
+```bash
+cargo run
+```
+
+On first run, General Bots automatically:
+
+1. Downloads and compiles dependencies
+2. Sets up PostgreSQL database
+3. Configures S3-compatible storage (MinIO)
+4. Initializes Redis cache
+5. Downloads default LLM models
+6. Creates template bots
+7. Starts the HTTP server
+
+The server will be available at `http://localhost:8080`.
+
+## First Steps
+
+### Access the Web Interface
+
+Open your browser to:
+
+- **Minimal UI**: `http://localhost:8080` - Lightweight chat interface
+- **Full Suite**: `http://localhost:8080/suite` - Complete application suite
+
+### Create Your First Bot
+
+1. Navigate to the `templates/` directory
+2. Copy the `template.gbai` folder:
+
+```bash
+cp -r templates/template.gbai templates/mybot.gbai
+```
+
+3. Edit the configuration in `mybot.gbai/mybot.gbot/config.csv`:
+
+```csv
+name,value
+theme-title,My First Bot
+theme-color1,#1565C0
+```
+
+4. Add knowledge to `mybot.gbai/mybot.gbkb/`:
+
+```markdown
+# Company FAQ
+
+## What are your hours?
+We are open Monday to Friday, 9 AM to 5 PM.
+
+## How do I contact support?
+Email support@example.com or call 1-800-EXAMPLE.
+```
+
+5. Create a dialog in `mybot.gbai/mybot.gbdialog/start.bas`:
+
+```basic
+' Welcome dialog
+USE KB "mybot.gbkb"
+
+TALK "Welcome to My Bot!"
+TALK "How can I help you today?"
+
+SET CONTEXT "assistant" AS "You are a helpful assistant for My Company."
+```
+
+6. Restart the server to load your new bot.
+
+## Command-Line Options
+
+```bash
+# Default: console UI + web server
+cargo run
+
+# Disable console UI (background service)
+cargo run -- --noconsole
+
+# Desktop application mode (Tauri)
+cargo run -- --desktop
+
+# Specify tenant
+cargo run -- --tenant mycompany
+
+# LXC container mode
+cargo run -- --container
+
+# Disable all UI
+cargo run -- --noui
+```
+
+## Project Structure
+
+```
+mybot.gbai/
+├── mybot.gbot/           # Bot configuration
+│   └── config.csv        # Theme and settings
+├── mybot.gbkb/           # Knowledge base
+│   └── faq.md            # FAQ documents
+├── mybot.gbdialog/       # Dialog scripts
+│   └── start.bas         # Main dialog
+└── mybot.gbdrive/        # File storage
+    └── templates/        # Document templates
+```
+
+## Essential BASIC Keywords
+
+### Knowledge Base
+
+```basic
+USE KB "knowledge-name"     ' Load knowledge base
+CLEAR KB                    ' Remove from session
+```
+
+### Tools
+
+```basic
+USE TOOL "tool-name"        ' Make tool available
+CLEAR TOOLS                 ' Remove all tools
+```
+
+### Conversation
+
+```basic
+TALK "message"              ' Send message to user
+answer = HEAR               ' Wait for user input
+WAIT 5                      ' Wait 5 seconds
+```
+
+### Data
+
+```basic
+SAVE "table.csv", field1, field2    ' Save to storage
+data = GET "https://api.example.com" ' HTTP request
+SEND FILE "document.pdf"             ' Send file to user
+```
+
+## Environment Variables
+
+General Bots requires minimal configuration. Only directory service variables are needed:
+
+```bash
+export DIRECTORY_URL="https://zitadel.example.com"
+export DIRECTORY_CLIENT_ID="your-client-id"
+export DIRECTORY_CLIENT_SECRET="your-secret"
+```
+
+All other services (database, storage, cache) are configured automatically.
+
+## Testing Your Bot
+
+### Via Web Interface
+
+1. Open `http://localhost:8080`
+2. Type a message in the chat box
+3. The bot responds using your knowledge base and dialogs
+
+### Via API
+
+```bash
+# Create a session
+curl -X POST http://localhost:8080/api/sessions \
+  -H "Content-Type: application/json" \
+  -d '{"bot_id": "mybot"}'
+
+# Send a message
+curl -X POST http://localhost:8080/api/sessions/{session_id}/messages \
+  -H "Content-Type: application/json" \
+  -d '{"content": "What are your hours?"}'
+```
+
+### Via WebSocket
+
+```javascript
+const ws = new WebSocket('ws://localhost:8080/ws');
+
+ws.onopen = () => {
+    ws.send(JSON.stringify({
+        type: 'message',
+        session_id: 'your-session-id',
+        content: 'Hello!'
+    }));
+};
+
+ws.onmessage = (event) => {
+    const data = JSON.parse(event.data);
+    console.log('Bot:', data.content);
+};
+```
+
+## Common Issues
+
+### Port Already in Use
+
+```bash
+# Find process using port 8080
+lsof -i :8080
+
+# Kill the process or use a different port
+cargo run -- --port 8081
+```
+
+### Database Connection Failed
+
+Ensure PostgreSQL is running:
+
+```bash
+botserver status postgres
+botserver restart postgres
+```
+
+### LLM Not Responding
+
+Check your LLM configuration in the admin panel or verify API keys are set.
+
+## Next Steps
+
+- **[API Reference](../api/README.md)** - Integrate with external systems
+- **[BASIC Language](../reference/basic-language.md)** - Complete keyword reference
+- **[Templates](templates.md)** - Pre-built bot templates
+- **[Deployment](deployment.md)** - Production setup guide
+
+## Getting Help
+
+- **GitHub Issues**: [github.com/GeneralBots/BotServer/issues](https://github.com/GeneralBots/BotServer/issues)
+- **Stack Overflow**: Tag questions with `generalbots`
+- **Documentation**: [docs.pragmatismo.com.br](https://docs.pragmatismo.com.br)
--- a/docs/guides/templates.md
+++ b/docs/guides/templates.md
@ -0,0 +1,308 @@
+# Using Templates
+
+Templates are pre-built bot configurations that accelerate development. General Bots includes templates for common use cases like CRM, HR, IT support, and more.
+
+## Template Structure
+
+Each template follows the `.gbai` package format:
+
+```
+template-name.gbai/
+├── template-name.gbot/       # Configuration
+│   └── config.csv            # Bot settings
+├── template-name.gbkb/       # Knowledge base
+│   └── *.md                  # Documentation files
+├── template-name.gbdialog/   # Dialog scripts
+│   ├── start.bas             # Entry point
+│   └── *.bas                 # Tool scripts
+└── template-name.gbdrive/    # File storage
+    └── templates/            # Document templates
+```
+
+## Available Templates
+
+### Business Operations
+
+| Template | Description | Path |
+|----------|-------------|------|
+| CRM Contacts | Contact management | `crm/contacts.gbai` |
+| Sales Pipeline | Deal tracking | `crm/sales-pipeline.gbai` |
+| HR Employees | Employee management | `hr/employees.gbai` |
+| IT Helpdesk | Ticket system | `it/helpdesk.gbai` |
+
+### Productivity
+
+| Template | Description | Path |
+|----------|-------------|------|
+| Office | Document automation | `productivity/office.gbai` |
+| Reminder | Task reminders | `productivity/reminder.gbai` |
+| Analytics | Data dashboards | `platform/analytics.gbai` |
+
+### Integration
+
+| Template | Description | Path |
+|----------|-------------|------|
+| API Client | External API calls | `integration/api-client.gbai` |
+| Public APIs | 70+ free APIs | `integration/public-apis.gbai` |
+
+### Compliance
+
+| Template | Description | Path |
+|----------|-------------|------|
+| HIPAA Medical | Healthcare compliance | `compliance/hipaa-medical.gbai` |
+| Privacy | GDPR/LGPD | `compliance/privacy.gbai` |
+
+## Using a Template
+
+### 1. Copy the Template
+
+```bash
+cp -r templates/crm/contacts.gbai templates/mycrm.gbai
+```
+
+### 2. Rename Internal Folders
+
+```bash
+cd templates/mycrm.gbai
+mv contacts.gbot mycrm.gbot
+mv contacts.gbkb mycrm.gbkb
+mv contacts.gbdialog mycrm.gbdialog
+mv contacts.gbdrive mycrm.gbdrive
+```
+
+### 3. Update Configuration
+
+Edit `mycrm.gbot/config.csv`:
+
+```csv
+name,value
+theme-title,My CRM Bot
+theme-color1,#2196F3
+theme-color2,#E3F2FD
+prompt-history,2
+```
+
+### 4. Customize Knowledge Base
+
+Edit files in `mycrm.gbkb/`:
+
+```markdown
+# My Company CRM Guide
+
+## Adding Contacts
+To add a new contact, say "add contact" and provide:
+- Name
+- Email
+- Phone number
+- Company
+
+## Searching Contacts
+Say "find contact [name]" to search.
+```
+
+### 5. Modify Dialogs
+
+Edit `mycrm.gbdialog/start.bas`:
+
+```basic
+' My CRM Bot - Start Script
+
+USE KB "mycrm.gbkb"
+USE TOOL "add-contact"
+USE TOOL "search-contact"
+
+SET CONTEXT "crm" AS "You are a CRM assistant for My Company."
+
+TALK "Welcome to My Company CRM!"
+TALK "I can help you manage contacts and deals."
+
+ADD SUGGESTION "add" AS "Add new contact"
+ADD SUGGESTION "search" AS "Find a contact"
+ADD SUGGESTION "list" AS "Show all contacts"
+```
+
+### 6. Restart Server
+
+```bash
+cargo run
+```
+
+## Creating Custom Templates
+
+### Step 1: Create Package Structure
+
+```bash
+mkdir -p templates/mytemplate.gbai/{mytemplate.gbot,mytemplate.gbkb,mytemplate.gbdialog,mytemplate.gbdrive}
+```
+
+### Step 2: Create Configuration
+
+```csv
+# mytemplate.gbot/config.csv
+name,value
+theme-title,My Template
+theme-color1,#1565C0
+theme-color2,#E3F2FD
+theme-logo,https://example.com/logo.svg
+prompt-history,2
+prompt-compact,4
+```
+
+### Step 3: Create Start Dialog
+
+```basic
+' mytemplate.gbdialog/start.bas
+
+' Load knowledge base
+USE KB "mytemplate.gbkb"
+
+' Register tools
+USE TOOL "my-action"
+
+' Set AI context
+SET CONTEXT "assistant" AS "You are a helpful assistant."
+
+' Welcome message
+BEGIN TALK
+    **Welcome to My Template!**
+    
+    I can help you with:
+    • Feature 1
+    • Feature 2
+    • Feature 3
+END TALK
+
+' Add quick suggestions
+CLEAR SUGGESTIONS
+ADD SUGGESTION "help" AS "Show help"
+ADD SUGGESTION "action" AS "Do something"
+```
+
+### Step 4: Create Tools
+
+```basic
+' mytemplate.gbdialog/my-action.bas
+
+PARAM item_name AS STRING LIKE "Example" DESCRIPTION "Name of the item"
+PARAM quantity AS INTEGER LIKE 1 DESCRIPTION "How many items"
+
+DESCRIPTION "Performs an action with the specified item."
+
+' Validate input
+IF item_name = "" THEN
+    TALK "Please provide an item name."
+    item_name = HEAR
+END IF
+
+' Process
+let result = item_name + " x " + quantity
+
+' Save record
+SAVE "items.csv", item_name, quantity, result
+
+' Respond
+TALK "✅ Created: " + result
+
+RETURN result
+```
+
+### Step 5: Add Knowledge Base
+
+```markdown
+# mytemplate.gbkb/guide.md
+
+# My Template Guide
+
+## Overview
+This template helps you accomplish specific tasks.
+
+## Features
+
+### Feature 1
+Description of feature 1.
+
+### Feature 2
+Description of feature 2.
+
+## FAQ
+
+### How do I get started?
+Just say "help" to see available commands.
+
+### How do I contact support?
+Email support@example.com.
+```
+
+## Template Best Practices
+
+### Configuration
+
+- Use clear, descriptive `theme-title`
+- Choose accessible color combinations
+- Set appropriate `prompt-history` (2-4 recommended)
+
+### Knowledge Base
+
+- Write clear, concise documentation
+- Use headers for organization
+- Include FAQ section
+- Keep files under 50KB each
+
+### Dialogs
+
+- Always include `start.bas`
+- Use `DESCRIPTION` for all tools
+- Validate user input
+- Provide helpful error messages
+- Add suggestions for common actions
+
+### File Organization
+
+```
+mytemplate.gbai/
+├── mytemplate.gbot/
+│   └── config.csv
+├── mytemplate.gbkb/
+│   ├── guide.md           # Main documentation
+│   ├── faq.md             # Frequently asked questions
+│   └── troubleshooting.md # Common issues
+├── mytemplate.gbdialog/
+│   ├── start.bas          # Entry point (required)
+│   ├── tool-1.bas         # First tool
+│   ├── tool-2.bas         # Second tool
+│   └── jobs.bas           # Scheduled tasks
+└── mytemplate.gbdrive/
+    └── templates/
+        └── report.docx    # Document templates
+```
+
+## Sharing Templates
+
+### Export
+
+```bash
+tar -czf mytemplate.tar.gz templates/mytemplate.gbai
+```
+
+### Import
+
+```bash
+tar -xzf mytemplate.tar.gz -C templates/
+```
+
+### Version Control
+
+Templates work well with Git:
+
+```bash
+cd templates/mytemplate.gbai
+git init
+git add .
+git commit -m "Initial template"
+```
+
+## Next Steps
+
+- [BASIC Language Reference](../reference/basic-language.md) - Complete keyword list
+- [API Reference](../api/README.md) - Integrate with external systems
+- [Deployment Guide](deployment.md) - Production setup
--- a/docs/reference/architecture.md
+++ b/docs/reference/architecture.md
@ -0,0 +1,530 @@
+# Architecture Reference
+
+System architecture and design overview for General Bots.
+
+## High-Level Architecture
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                         Clients                                  │
+│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐        │
+│  │   Web    │  │  Mobile  │  │   API    │  │ WhatsApp │        │
+│  │  (HTMX)  │  │   App    │  │ Clients  │  │ Telegram │        │
+│  └────┬─────┘  └────┬─────┘  └────┬─────┘  └────┬─────┘        │
+└───────┼─────────────┼─────────────┼─────────────┼───────────────┘
+        │             │             │             │
+        └─────────────┴──────┬──────┴─────────────┘
+                             │
+                    ┌────────▼────────┐
+                    │   HTTP Server   │
+                    │     (Axum)      │
+                    └────────┬────────┘
+                             │
+        ┌────────────────────┼────────────────────┐
+        │                    │                    │
+┌───────▼───────┐  ┌─────────▼─────────┐  ┌──────▼──────┐
+│  REST API     │  │    WebSocket      │  │   Static    │
+│  Handlers     │  │    Handlers       │  │   Files     │
+└───────┬───────┘  └─────────┬─────────┘  └─────────────┘
+        │                    │
+        └────────┬───────────┘
+                 │
+        ┌────────▼────────┐
+        │   App State     │
+        │  (Shared Arc)   │
+        └────────┬────────┘
+                 │
+   ┌─────────────┼─────────────┬─────────────┐
+   │             │             │             │
+┌──▼──┐     ┌────▼────┐   ┌────▼────┐   ┌───▼───┐
+│ DB  │     │  Cache  │   │ Storage │   │  LLM  │
+│(PG) │     │ (Redis) │   │  (S3)   │   │(Multi)│
+└─────┘     └─────────┘   └─────────┘   └───────┘
+```
+
+## Core Components
+
+### HTTP Server (Axum)
+
+The main entry point for all requests.
+
+```rust
+// Main router structure
+Router::new()
+    .nest("/api", api_routes())
+    .nest("/ws", websocket_routes())
+    .nest_service("/", static_files())
+    .layer(middleware_stack())
+```
+
+**Responsibilities:**
+- Request routing
+- Middleware execution
+- CORS handling
+- Rate limiting
+- Authentication
+
+### App State
+
+Shared application state passed to all handlers.
+
+```rust
+pub struct AppState {
+    pub db_pool: Pool<ConnectionManager<PgConnection>>,
+    pub redis: RedisPool,
+    pub s3_client: S3Client,
+    pub llm_client: LlmClient,
+    pub qdrant: QdrantClient,
+    pub config: AppConfig,
+}
+```
+
+**Contains:**
+- Database connection pool
+- Redis cache connection
+- S3 storage client
+- LLM provider clients
+- Vector database client
+- Application configuration
+
+### Request Flow
+
+```
+Request → Router → Middleware → Handler → Response
+                      │
+                      ├── Auth Check
+                      ├── Rate Limit
+                      ├── Logging
+                      └── Error Handling
+```
+
+## Module Structure
+
+```
+botserver/src/
+├── main.rs              # Entry point
+├── lib.rs               # Library exports
+├── core/                # Core infrastructure
+│   ├── shared/          # Shared types and state
+│   │   ├── state.rs     # AppState definition
+│   │   ├── models.rs    # Common models
+│   │   └── schema.rs    # Database schema
+│   ├── urls.rs          # URL constants
+│   ├── secrets/         # Vault integration
+│   └── rate_limit.rs    # Rate limiting
+├── basic/               # BASIC language
+│   ├── compiler/        # Compiler/parser
+│   ├── runtime/         # Execution engine
+│   └── keywords/        # Keyword implementations
+├── llm/                 # LLM integration
+│   ├── mod.rs           # Provider abstraction
+│   ├── openai.rs        # OpenAI client
+│   ├── anthropic.rs     # Anthropic client
+│   └── prompt_manager/  # Prompt management
+├── multimodal/          # Media processing
+│   ├── vision.rs        # Image analysis
+│   ├── audio.rs         # Speech processing
+│   └── document.rs      # Document parsing
+├── security/            # Authentication
+│   ├── auth.rs          # Auth middleware
+│   ├── zitadel.rs       # Zitadel client
+│   └── jwt.rs           # Token handling
+├── analytics/           # Analytics module
+├── calendar/            # Calendar/CalDAV
+├── designer/            # Bot builder
+├── drive/               # File storage
+├── email/               # Email (IMAP/SMTP)
+├── meet/                # Video conferencing
+├── paper/               # Document editor
+├── research/            # KB search
+├── sources/             # Templates
+└── tasks/               # Task management
+```
+
+## Data Flow
+
+### Chat Message Flow
+
+```
+User Input
+    │
+    ▼
+┌──────────────┐
+│  WebSocket   │
+│   Handler    │
+└──────┬───────┘
+       │
+       ▼
+┌──────────────┐     ┌──────────────┐
+│   Session    │────▶│   Message    │
+│   Manager    │     │   History    │
+└──────┬───────┘     └──────────────┘
+       │
+       ▼
+┌──────────────┐
+│  KB Search   │◀────── Vector DB (Qdrant)
+│  (Context)   │
+└──────┬───────┘
+       │
+       ▼
+┌──────────────┐
+│ Tool Check   │◀────── Registered Tools
+│              │
+└──────┬───────┘
+       │
+       ▼
+┌──────────────┐
+│  LLM Call    │◀────── Semantic Cache (Redis)
+│              │
+└──────┬───────┘
+       │
+       ▼
+┌──────────────┐
+│  Response    │
+│  Streaming   │
+└──────┬───────┘
+       │
+       ▼
+User Response
+```
+
+### Tool Execution Flow
+
+```
+LLM Response (tool_call)
+    │
+    ▼
+┌──────────────┐
+│ Tool Router  │
+└──────┬───────┘
+       │
+       ▼
+┌──────────────┐
+│BASIC Runtime │
+└──────┬───────┘
+       │
+       ├──▶ Database Operations
+       ├──▶ HTTP Requests
+       ├──▶ File Operations
+       └──▶ Email/Notifications
+       │
+       ▼
+┌──────────────┐
+│ Tool Result  │
+└──────┬───────┘
+       │
+       ▼
+LLM (continue or respond)
+```
+
+## Database Schema
+
+### Core Tables
+
+```sql
+-- User sessions
+CREATE TABLE user_sessions (
+    id UUID PRIMARY KEY,
+    user_id VARCHAR(255),
+    bot_id VARCHAR(100),
+    status VARCHAR(20),
+    metadata JSONB,
+    created_at TIMESTAMPTZ,
+    updated_at TIMESTAMPTZ
+);
+
+-- Message history
+CREATE TABLE message_history (
+    id UUID PRIMARY KEY,
+    session_id UUID REFERENCES user_sessions(id),
+    role VARCHAR(20),
+    content TEXT,
+    tokens_used INTEGER,
+    created_at TIMESTAMPTZ
+);
+
+-- Bot configurations
+CREATE TABLE bot_configs (
+    id UUID PRIMARY KEY,
+    bot_id VARCHAR(100) UNIQUE,
+    config JSONB,
+    created_at TIMESTAMPTZ,
+    updated_at TIMESTAMPTZ
+);
+
+-- Knowledge base documents
+CREATE TABLE kb_documents (
+    id UUID PRIMARY KEY,
+    collection VARCHAR(100),
+    content TEXT,
+    embedding_id VARCHAR(100),
+    metadata JSONB,
+    created_at TIMESTAMPTZ
+);
+```
+
+### Indexes
+
+```sql
+CREATE INDEX idx_sessions_user ON user_sessions(user_id);
+CREATE INDEX idx_sessions_bot ON user_sessions(bot_id);
+CREATE INDEX idx_messages_session ON message_history(session_id);
+CREATE INDEX idx_kb_collection ON kb_documents(collection);
+```
+
+## Caching Strategy
+
+### Cache Layers
+
+```
+┌─────────────────────────────────────────┐
+│            Semantic Cache               │
+│  (LLM responses by query similarity)    │
+└───────────────────┬─────────────────────┘
+                    │
+┌───────────────────▼─────────────────────┐
+│            Session Cache                │
+│    (Active sessions, user context)      │
+└───────────────────┬─────────────────────┘
+                    │
+┌───────────────────▼─────────────────────┐
+│            Data Cache                   │
+│   (KB results, config, templates)       │
+└─────────────────────────────────────────┘
+```
+
+### Cache Keys
+
+| Pattern | TTL | Description |
+|---------|-----|-------------|
+| `session:{id}` | 30m | Active session data |
+| `semantic:{hash}` | 24h | LLM response cache |
+| `kb:{collection}:{hash}` | 1h | KB search results |
+| `config:{bot_id}` | 5m | Bot configuration |
+| `user:{id}` | 15m | User preferences |
+
+### Semantic Cache
+
+```rust
+// Query similarity check
+let cache_key = compute_embedding_hash(query);
+if let Some(cached) = redis.get(&cache_key).await? {
+    if similarity(query, cached.query) > 0.95 {
+        return cached.response;
+    }
+}
+```
+
+## LLM Integration
+
+### Provider Abstraction
+
+```rust
+pub trait LlmProvider: Send + Sync {
+    async fn complete(&self, request: CompletionRequest) 
+        -> Result<CompletionResponse>;
+    
+    async fn complete_stream(&self, request: CompletionRequest) 
+        -> Result<impl Stream<Item = StreamChunk>>;
+    
+    async fn embed(&self, text: &str) 
+        -> Result<Vec<f32>>;
+}
+```
+
+### Supported Providers
+
+| Provider | Models | Features |
+|----------|--------|----------|
+| OpenAI | GPT-4, GPT-3.5 | Streaming, Functions, Vision |
+| Anthropic | Claude 3 | Streaming, Long context |
+| Groq | Llama, Mixtral | Fast inference |
+| Ollama | Any local | Self-hosted |
+
+### Request Flow
+
+```rust
+// 1. Build messages with context
+let messages = build_messages(history, kb_context, system_prompt);
+
+// 2. Add tools if registered
+let tools = get_registered_tools(session);
+
+// 3. Check semantic cache
+if let Some(cached) = semantic_cache.get(&messages).await? {
+    return Ok(cached);
+}
+
+// 4. Call LLM
+let response = llm.complete(CompletionRequest {
+    messages,
+    tools,
+    temperature: config.temperature,
+    max_tokens: config.max_tokens,
+}).await?;
+
+// 5. Cache response
+semantic_cache.set(&messages, &response).await?;
+```
+
+## BASIC Runtime
+
+### Compilation Pipeline
+
+```
+Source Code (.bas)
+       │
+       ▼
+┌──────────────┐
+│    Lexer     │──▶ Tokens
+└──────────────┘
+       │
+       ▼
+┌──────────────┐
+│    Parser    │──▶ AST
+└──────────────┘
+       │
+       ▼
+┌──────────────┐
+│   Analyzer   │──▶ Validated AST
+└──────────────┘
+       │
+       ▼
+┌──────────────┐
+│   Runtime    │──▶ Execution
+└──────────────┘
+```
+
+### Execution Context
+
+```rust
+pub struct RuntimeContext {
+    pub session: Session,
+    pub variables: HashMap<String, Value>,
+    pub tools: Vec<Tool>,
+    pub kb: Vec<KnowledgeBase>,
+    pub state: Arc<AppState>,
+}
+```
+
+## Security Architecture
+
+### Authentication Flow
+
+```
+Client Request
+       │
+       ▼
+┌──────────────┐
+│   Extract    │
+│    Token     │
+└──────┬───────┘
+       │
+       ▼
+┌──────────────┐     ┌──────────────┐
+│   Validate   │────▶│   Zitadel    │
+│     JWT      │     │   (OIDC)     │
+└──────┬───────┘     └──────────────┘
+       │
+       ▼
+┌──────────────┐
+│  Check       │
+│  Permissions │
+└──────┬───────┘
+       │
+       ▼
+Handler Execution
+```
+
+### Security Layers
+
+1. **Transport**: TLS 1.3 (rustls)
+2. **Authentication**: JWT/OAuth 2.0 (Zitadel)
+3. **Authorization**: Role-based access control
+4. **Rate Limiting**: Per-IP token bucket
+5. **Input Validation**: Type-safe parameters
+6. **Output Sanitization**: HTML escaping
+
+## Deployment Architecture
+
+### Single Instance
+
+```
+┌─────────────────────────────────────┐
+│           Single Server             │
+│  ┌─────────────────────────────┐   │
+│  │        BotServer            │   │
+│  └─────────────────────────────┘   │
+│  ┌────────┐ ┌────────┐ ┌───────┐   │
+│  │PostgreSQL│ │ Redis │ │ MinIO │   │
+│  └────────┘ └────────┘ └───────┘   │
+└─────────────────────────────────────┘
+```
+
+### Clustered
+
+```
+┌─────────────────────────────────────────────────────┐
+│                   Load Balancer                      │
+└───────────────────────┬─────────────────────────────┘
+                        │
+        ┌───────────────┼───────────────┐
+        │               │               │
+   ┌────▼────┐    ┌─────▼────┐    ┌────▼────┐
+   │BotServer│    │BotServer │    │BotServer│
+   │   #1    │    │    #2    │    │   #3    │
+   └────┬────┘    └─────┬────┘    └────┬────┘
+        │               │               │
+        └───────────────┼───────────────┘
+                        │
+   ┌────────────────────┼────────────────────┐
+   │                    │                    │
+┌──▼───┐          ┌─────▼────┐         ┌────▼────┐
+│  PG  │          │  Redis   │         │   S3    │
+│Cluster│         │  Cluster │         │ Cluster │
+└──────┘          └──────────┘         └─────────┘
+```
+
+## Performance Characteristics
+
+| Operation | Latency | Throughput |
+|-----------|---------|------------|
+| REST API | < 10ms | 10,000 req/s |
+| WebSocket message | < 5ms | 50,000 msg/s |
+| KB search | < 50ms | 1,000 req/s |
+| LLM call (cached) | < 20ms | 5,000 req/s |
+| LLM call (uncached) | 500-3000ms | 50 req/s |
+
+## Monitoring Points
+
+| Metric | Description |
+|--------|-------------|
+| `http_requests_total` | Total HTTP requests |
+| `http_request_duration` | Request latency |
+| `ws_connections` | Active WebSocket connections |
+| `llm_requests_total` | LLM API calls |
+| `llm_cache_hits` | Semantic cache hit rate |
+| `db_pool_size` | Active DB connections |
+| `memory_usage` | Process memory |
+
+## Extension Points
+
+### Adding a New Module
+
+1. Create module in `src/`
+2. Define routes in `mod.rs`
+3. Register in `lib.rs`
+4. Add to router in `main.rs`
+
+### Adding a New LLM Provider
+
+1. Implement `LlmProvider` trait
+2. Add provider enum variant
+3. Register in provider factory
+
+### Adding a New BASIC Keyword
+
+1. Add keyword to lexer
+2. Implement AST node
+3. Add runtime handler
+4. Update documentation
--- a/docs/reference/basic-language.md
+++ b/docs/reference/basic-language.md
@ -0,0 +1,525 @@
+# BASIC Language Reference
+
+Complete reference for General Bots BASIC dialog scripting language.
+
+## Overview
+
+General Bots BASIC is a domain-specific language for creating conversational AI dialogs. It provides keywords for:
+
+- User interaction (TALK, HEAR)
+- Knowledge base management (USE KB, CLEAR KB)
+- Tool registration (USE TOOL, CLEAR TOOLS)
+- Data operations (SAVE, GET, POST)
+- File handling (SEND FILE, DOWNLOAD)
+- Flow control (IF/THEN/ELSE, FOR/NEXT)
+
+## Conversation Keywords
+
+### TALK
+
+Send a message to the user.
+
+```basic
+TALK "Hello, how can I help you?"
+TALK "Your order number is: " + ordernumber
+```
+
+#### Multi-line Messages
+
+```basic
+BEGIN TALK
+    **Welcome!**
+    
+    I can help you with:
+    • Orders
+    • Shipping
+    • Returns
+END TALK
+```
+
+### HEAR
+
+Wait for and capture user input.
+
+```basic
+answer = HEAR
+name = HEAR AS NAME
+email = HEAR AS EMAIL
+choice = HEAR AS "Option A", "Option B", "Option C"
+confirmed = HEAR AS BOOLEAN
+```
+
+#### Input Types
+
+| Type | Description | Example |
+|------|-------------|---------|
+| `STRING` | Free text | `answer = HEAR` |
+| `NAME` | Person name | `name = HEAR AS NAME` |
+| `EMAIL` | Email address | `email = HEAR AS EMAIL` |
+| `PHONE` | Phone number | `phone = HEAR AS PHONE` |
+| `INTEGER` | Whole number | `count = HEAR AS INTEGER` |
+| `NUMBER` | Decimal number | `amount = HEAR AS NUMBER` |
+| `BOOLEAN` | Yes/No | `confirm = HEAR AS BOOLEAN` |
+| `DATE` | Date value | `date = HEAR AS DATE` |
+| Options | Multiple choice | `choice = HEAR AS "A", "B", "C"` |
+
+### WAIT
+
+Pause execution for specified seconds.
+
+```basic
+WAIT 5
+TALK "Processing..."
+WAIT 2
+TALK "Done!"
+```
+
+## Knowledge Base Keywords
+
+### USE KB
+
+Load a knowledge base into the current session.
+
+```basic
+USE KB "company-docs"
+USE KB "product-catalog.gbkb"
+```
+
+### CLEAR KB
+
+Remove knowledge base from session.
+
+```basic
+CLEAR KB "company-docs"
+CLEAR KB  ' Clear all KBs
+```
+
+## Tool Keywords
+
+### USE TOOL
+
+Register a tool for the AI to call.
+
+```basic
+USE TOOL "create-ticket"
+USE TOOL "send-email"
+USE TOOL "search-orders"
+```
+
+### CLEAR TOOLS
+
+Remove all registered tools.
+
+```basic
+CLEAR TOOLS
+```
+
+## Context Keywords
+
+### SET CONTEXT
+
+Define AI behavior context.
+
+```basic
+SET CONTEXT "assistant" AS "You are a helpful customer service agent for Acme Corp."
+```
+
+### System Prompt
+
+Define detailed AI instructions.
+
+```basic
+BEGIN SYSTEM PROMPT
+    You are a professional assistant.
+    Always be polite and helpful.
+    If you don't know something, say so.
+    Never make up information.
+END SYSTEM PROMPT
+```
+
+## Suggestion Keywords
+
+### ADD SUGGESTION
+
+Add a quick-reply button for users.
+
+```basic
+ADD SUGGESTION "help" AS "Show help"
+ADD SUGGESTION "order" AS "Track my order"
+ADD SUGGESTION "contact" AS "Contact support"
+```
+
+### CLEAR SUGGESTIONS
+
+Remove all suggestions.
+
+```basic
+CLEAR SUGGESTIONS
+```
+
+## Data Keywords
+
+### SAVE
+
+Save data to storage.
+
+```basic
+SAVE "contacts.csv", name, email, phone
+SAVE "orders.csv", orderid, product, quantity, total
+```
+
+### GET
+
+HTTP GET request.
+
+```basic
+data = GET "https://api.example.com/users"
+weather = GET "https://api.weather.com/current?city=" + city
+```
+
+### POST
+
+HTTP POST request.
+
+```basic
+result = POST "https://api.example.com/orders", orderdata
+```
+
+### PUT
+
+HTTP PUT request.
+
+```basic
+result = PUT "https://api.example.com/users/" + userid, userdata
+```
+
+### DELETE HTTP
+
+HTTP DELETE request.
+
+```basic
+result = DELETE HTTP "https://api.example.com/users/" + userid
+```
+
+### SET HEADER
+
+Set HTTP header for requests.
+
+```basic
+SET HEADER "Authorization" = "Bearer " + token
+SET HEADER "Content-Type" = "application/json"
+data = GET "https://api.example.com/protected"
+```
+
+### CLEAR HEADERS
+
+Remove all custom headers.
+
+```basic
+CLEAR HEADERS
+```
+
+## File Keywords
+
+### SEND FILE
+
+Send a file to the user.
+
+```basic
+SEND FILE "report.pdf"
+SEND FILE filepath
+```
+
+### DOWNLOAD
+
+Download a file from URL.
+
+```basic
+file = DOWNLOAD "https://example.com/document.pdf"
+SEND FILE file
+```
+
+### DELETE FILE
+
+Delete a file from storage.
+
+```basic
+DELETE FILE "old-report.pdf"
+```
+
+## Email Keywords
+
+### SEND MAIL
+
+Send an email.
+
+```basic
+SEND MAIL "recipient@example.com", "Subject Line", "Email body text"
+SEND MAIL email, subject, body
+```
+
+## Memory Keywords
+
+### SET BOT MEMORY
+
+Store a value in bot memory (persists across sessions).
+
+```basic
+SET BOT MEMORY "last_order", orderid
+SET BOT MEMORY "user_preference", preference
+```
+
+### GET BOT MEMORY
+
+Retrieve a value from bot memory.
+
+```basic
+lastorder = GET BOT MEMORY("last_order")
+pref = GET BOT MEMORY("user_preference")
+```
+
+## Schedule Keywords
+
+### SET SCHEDULE
+
+Define when a job should run (cron format).
+
+```basic
+SET SCHEDULE "0 9 * * *"     ' Daily at 9 AM
+SET SCHEDULE "0 0 * * 1"     ' Weekly on Monday
+SET SCHEDULE "0 8 1 * *"     ' Monthly on the 1st at 8 AM
+```
+
+#### Cron Format
+
+```
+┌───────────── minute (0-59)
+│ ┌───────────── hour (0-23)
+│ │ ┌───────────── day of month (1-31)
+│ │ │ ┌───────────── month (1-12)
+│ │ │ │ ┌───────────── day of week (0-6, Sun=0)
+│ │ │ │ │
+* * * * *
+```
+
+## Flow Control
+
+### IF/THEN/ELSE
+
+Conditional execution.
+
+```basic
+IF status = "active" THEN
+    TALK "Your account is active."
+ELSE IF status = "pending" THEN
+    TALK "Your account is pending approval."
+ELSE
+    TALK "Your account is inactive."
+END IF
+```
+
+### FOR/NEXT
+
+Loop through a range.
+
+```basic
+FOR i = 1 TO 10
+    TALK "Item " + i
+NEXT
+```
+
+### FOR EACH
+
+Loop through a collection.
+
+```basic
+FOR EACH item IN items
+    TALK item.name + ": $" + item.price
+END FOR
+```
+
+## Variables
+
+### Declaration
+
+```basic
+let name = "John"
+let count = 42
+let price = 19.99
+let active = TRUE
+```
+
+### String Operations
+
+```basic
+let greeting = "Hello, " + name + "!"
+let upper = UCASE(text)
+let lower = LCASE(text)
+let length = LEN(text)
+let part = MID(text, 1, 5)
+```
+
+### Array Operations
+
+```basic
+let items = SPLIT(text, ",")
+let first = items[0]
+let count = LEN(items)
+```
+
+## Tool Definition
+
+Tools are BASIC files that the AI can call.
+
+### Structure
+
+```basic
+' tool-name.bas
+
+PARAM parametername AS TYPE LIKE "example" DESCRIPTION "What this parameter is"
+PARAM optionalparam AS STRING DESCRIPTION "Optional parameter"
+
+DESCRIPTION "What this tool does. Called when user wants to [action]."
+
+' Implementation
+IF parametername = "" THEN
+    TALK "Please provide the parameter."
+    parametername = HEAR
+END IF
+
+let result = "processed: " + parametername
+
+SAVE "records.csv", parametername, result
+
+TALK "✅ Done: " + result
+
+RETURN result
+```
+
+### Parameter Types
+
+| Type | Description |
+|------|-------------|
+| `STRING` | Text value |
+| `INTEGER` | Whole number |
+| `NUMBER` | Decimal number |
+| `BOOLEAN` | True/False |
+| `DATE` | Date value |
+| `EMAIL` | Email address |
+| `PHONE` | Phone number |
+
+## Comments
+
+```basic
+' This is a single-line comment
+
+REM This is also a comment
+
+' Multi-line comments use multiple single-line comments
+' Line 1
+' Line 2
+```
+
+## Built-in Functions
+
+### String Functions
+
+| Function | Description | Example |
+|----------|-------------|---------|
+| `LEN(s)` | String length | `LEN("hello")` → `5` |
+| `UCASE(s)` | Uppercase | `UCASE("hello")` → `"HELLO"` |
+| `LCASE(s)` | Lowercase | `LCASE("HELLO")` → `"hello"` |
+| `TRIM(s)` | Remove whitespace | `TRIM("  hi  ")` → `"hi"` |
+| `MID(s,start,len)` | Substring | `MID("hello",2,3)` → `"ell"` |
+| `LEFT(s,n)` | Left characters | `LEFT("hello",2)` → `"he"` |
+| `RIGHT(s,n)` | Right characters | `RIGHT("hello",2)` → `"lo"` |
+| `SPLIT(s,delim)` | Split to array | `SPLIT("a,b,c",",")` → `["a","b","c"]` |
+| `REPLACE(s,old,new)` | Replace text | `REPLACE("hello","l","x")` → `"hexxo"` |
+
+### Date Functions
+
+| Function | Description | Example |
+|----------|-------------|---------|
+| `NOW()` | Current datetime | `NOW()` |
+| `TODAY()` | Current date | `TODAY()` |
+| `YEAR(d)` | Extract year | `YEAR(date)` → `2024` |
+| `MONTH(d)` | Extract month | `MONTH(date)` → `12` |
+| `DAY(d)` | Extract day | `DAY(date)` → `15` |
+| `DATEADD(d,n,unit)` | Add to date | `DATEADD(date,7,"days")` |
+| `DATEDIFF(d1,d2,unit)` | Date difference | `DATEDIFF(date1,date2,"days")` |
+
+### Math Functions
+
+| Function | Description | Example |
+|----------|-------------|---------|
+| `ABS(n)` | Absolute value | `ABS(-5)` → `5` |
+| `ROUND(n,d)` | Round number | `ROUND(3.456,2)` → `3.46` |
+| `FLOOR(n)` | Round down | `FLOOR(3.7)` → `3` |
+| `CEILING(n)` | Round up | `CEILING(3.2)` → `4` |
+| `MIN(a,b)` | Minimum | `MIN(5,3)` → `3` |
+| `MAX(a,b)` | Maximum | `MAX(5,3)` → `5` |
+| `SUM(arr)` | Sum of array | `SUM(numbers)` |
+| `AVG(arr)` | Average | `AVG(numbers)` |
+
+### Conversion Functions
+
+| Function | Description | Example |
+|----------|-------------|---------|
+| `STR(n)` | Number to string | `STR(42)` → `"42"` |
+| `VAL(s)` | String to number | `VAL("42")` → `42` |
+| `INT(n)` | To integer | `INT(3.7)` → `3` |
+
+## Complete Example
+
+```basic
+' customer-support.bas - Main support dialog
+
+' Setup
+USE KB "support-docs"
+USE TOOL "create-ticket"
+USE TOOL "check-order"
+USE TOOL "request-refund"
+
+SET CONTEXT "support" AS "You are a helpful customer support agent for Acme Store."
+
+' Welcome
+BEGIN TALK
+    **Welcome to Acme Support!**
+    
+    I can help you with:
+    • Order tracking
+    • Returns and refunds
+    • Product questions
+END TALK
+
+' Quick actions
+CLEAR SUGGESTIONS
+ADD SUGGESTION "order" AS "Track my order"
+ADD SUGGESTION "return" AS "Request a return"
+ADD SUGGESTION "help" AS "Other questions"
+
+BEGIN SYSTEM PROMPT
+    Be friendly and professional.
+    Always verify order numbers before making changes.
+    For refunds over $100, escalate to human support.
+    If asked about competitors, politely redirect to our products.
+END SYSTEM PROMPT
+```
+
+## Keyword Quick Reference
+
+| Category | Keywords |
+|----------|----------|
+| Conversation | `TALK`, `HEAR`, `WAIT` |
+| Knowledge | `USE KB`, `CLEAR KB` |
+| Tools | `USE TOOL`, `CLEAR TOOLS` |
+| Context | `SET CONTEXT`, `SYSTEM PROMPT` |
+| Suggestions | `ADD SUGGESTION`, `CLEAR SUGGESTIONS` |
+| Data | `SAVE`, `GET`, `POST`, `PUT`, `DELETE HTTP` |
+| HTTP | `SET HEADER`, `CLEAR HEADERS` |
+| Files | `SEND FILE`, `DOWNLOAD`, `DELETE FILE` |
+| Email | `SEND MAIL` |
+| Memory | `SET BOT MEMORY`, `GET BOT MEMORY` |
+| Schedule | `SET SCHEDULE` |
+| Flow | `IF/THEN/ELSE/END IF`, `FOR/NEXT`, `FOR EACH` |
+| Tools | `PARAM`, `DESCRIPTION`, `RETURN` |
--- a/docs/reference/configuration.md
+++ b/docs/reference/configuration.md
@ -0,0 +1,342 @@
+# Configuration Reference
+
+Complete reference for General Bots configuration options.
+
+## Configuration Files
+
+### Bot Configuration (`config.csv`)
+
+Located in each bot's `.gbot` folder:
+
+```
+mybot.gbai/
+└── mybot.gbot/
+    └── config.csv
+```
+
+Format: CSV with `name,value` columns.
+
+```csv
+name,value
+theme-title,My Bot
+theme-color1,#1565C0
+theme-color2,#E3F2FD
+```
+
+## Theme Settings
+
+| Setting | Description | Default | Example |
+|---------|-------------|---------|---------|
+| `theme-title` | Bot display name | Bot ID | `My Company Bot` |
+| `theme-color1` | Primary color | `#1565C0` | `#2196F3` |
+| `theme-color2` | Secondary/background | `#E3F2FD` | `#FFFFFF` |
+| `theme-logo` | Logo URL | Default logo | `https://example.com/logo.svg` |
+| `theme-favicon` | Favicon URL | Default | `https://example.com/favicon.ico` |
+
+### Color Examples
+
+```csv
+name,value
+theme-color1,#1565C0
+theme-color2,#E3F2FD
+```
+
+| Scheme | Primary | Secondary |
+|--------|---------|-----------|
+| Blue | `#1565C0` | `#E3F2FD` |
+| Green | `#2E7D32` | `#E8F5E9` |
+| Purple | `#7B1FA2` | `#F3E5F5` |
+| Orange | `#EF6C00` | `#FFF3E0` |
+| Red | `#C62828` | `#FFEBEE` |
+| Dark | `#212121` | `#424242` |
+
+## Prompt Settings
+
+| Setting | Description | Default | Range |
+|---------|-------------|---------|-------|
+| `prompt-history` | Messages in context | `2` | 1-10 |
+| `prompt-compact` | Compact mode threshold | `4` | 2-20 |
+| `prompt-max-tokens` | Max response tokens | `2048` | 256-8192 |
+| `prompt-temperature` | Response creativity | `0.7` | 0.0-2.0 |
+
+```csv
+name,value
+prompt-history,2
+prompt-compact,4
+prompt-max-tokens,2048
+prompt-temperature,0.7
+```
+
+### History Settings
+
+- `prompt-history=1`: Minimal context, faster responses
+- `prompt-history=2`: Balanced (recommended)
+- `prompt-history=5`: More context, slower responses
+
+## LLM Settings
+
+| Setting | Description | Default |
+|---------|-------------|---------|
+| `llm-provider` | LLM provider | `openai` |
+| `llm-model` | Model name | `gpt-4` |
+| `llm-api-key` | API key (or use env) | - |
+| `llm-endpoint` | Custom endpoint | Provider default |
+
+```csv
+name,value
+llm-provider,openai
+llm-model,gpt-4-turbo
+```
+
+### Supported Providers
+
+| Provider | Models |
+|----------|--------|
+| `openai` | `gpt-4`, `gpt-4-turbo`, `gpt-3.5-turbo` |
+| `anthropic` | `claude-3-opus`, `claude-3-sonnet` |
+| `groq` | `llama-3-70b`, `mixtral-8x7b` |
+| `ollama` | Any local model |
+
+## Feature Flags
+
+| Setting | Description | Default |
+|---------|-------------|---------|
+| `feature-voice` | Enable voice input/output | `false` |
+| `feature-file-upload` | Allow file uploads | `true` |
+| `feature-suggestions` | Show quick replies | `true` |
+| `feature-typing` | Show typing indicator | `true` |
+| `feature-history` | Show chat history | `true` |
+
+```csv
+name,value
+feature-voice,true
+feature-file-upload,true
+feature-suggestions,true
+```
+
+## Security Settings
+
+| Setting | Description | Default |
+|---------|-------------|---------|
+| `auth-required` | Require authentication | `false` |
+| `auth-provider` | Auth provider | `zitadel` |
+| `allowed-domains` | Allowed email domains | `*` |
+| `rate-limit` | Requests per minute | `60` |
+
+```csv
+name,value
+auth-required,true
+auth-provider,zitadel
+allowed-domains,example.com,company.org
+rate-limit,30
+```
+
+## Environment Variables
+
+### Required
+
+| Variable | Description |
+|----------|-------------|
+| `DIRECTORY_URL` | Zitadel/Auth instance URL |
+| `DIRECTORY_CLIENT_ID` | OAuth client ID |
+| `DIRECTORY_CLIENT_SECRET` | OAuth client secret |
+
+### Optional Overrides
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `DATABASE_URL` | PostgreSQL connection | Auto-configured |
+| `REDIS_URL` | Redis connection | Auto-configured |
+| `S3_ENDPOINT` | S3/MinIO endpoint | Auto-configured |
+| `S3_ACCESS_KEY` | S3 access key | Auto-configured |
+| `S3_SECRET_KEY` | S3 secret key | Auto-configured |
+| `QDRANT_URL` | Vector DB URL | Auto-configured |
+
+### LLM API Keys
+
+| Variable | Description |
+|----------|-------------|
+| `OPENAI_API_KEY` | OpenAI API key |
+| `ANTHROPIC_API_KEY` | Anthropic API key |
+| `GROQ_API_KEY` | Groq API key |
+
+### Server Settings
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `PORT` | HTTP server port | `8080` |
+| `HOST` | Bind address | `0.0.0.0` |
+| `RUST_LOG` | Log level | `info` |
+| `WORKERS` | Thread pool size | CPU cores |
+
+### Example `.env` File
+
+```bash
+# Authentication (required)
+DIRECTORY_URL=https://auth.example.com
+DIRECTORY_CLIENT_ID=abc123
+DIRECTORY_CLIENT_SECRET=secret
+
+# LLM Provider
+OPENAI_API_KEY=sk-...
+
+# Optional overrides
+DATABASE_URL=postgres://user:pass@localhost/botserver
+REDIS_URL=redis://localhost:6379
+
+# Server
+PORT=8080
+RUST_LOG=info,botserver=debug
+```
+
+## Rate Limiting
+
+### Environment Variables
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `RATE_LIMIT_ENABLED` | Enable rate limiting | `true` |
+| `RATE_LIMIT_API_RPS` | API requests/second | `100` |
+| `RATE_LIMIT_API_BURST` | API burst limit | `200` |
+| `RATE_LIMIT_AUTH_RPS` | Auth requests/second | `10` |
+| `RATE_LIMIT_AUTH_BURST` | Auth burst limit | `20` |
+| `RATE_LIMIT_LLM_RPS` | LLM requests/second | `5` |
+| `RATE_LIMIT_LLM_BURST` | LLM burst limit | `10` |
+
+```bash
+RATE_LIMIT_ENABLED=true
+RATE_LIMIT_API_RPS=100
+RATE_LIMIT_LLM_RPS=5
+```
+
+## Logging
+
+### Log Levels
+
+| Level | Description |
+|-------|-------------|
+| `error` | Errors only |
+| `warn` | Warnings and errors |
+| `info` | General information (default) |
+| `debug` | Detailed debugging |
+| `trace` | Very verbose |
+
+### Module-Specific Logging
+
+```bash
+# General info, debug for botserver
+RUST_LOG=info,botserver=debug
+
+# Quiet except errors, debug for specific module
+RUST_LOG=error,botserver::llm=debug
+
+# Full trace for development
+RUST_LOG=trace
+```
+
+## Database Configuration
+
+### Connection Pool
+
+| Setting | Description | Default |
+|---------|-------------|---------|
+| `DB_POOL_MIN` | Minimum connections | `2` |
+| `DB_POOL_MAX` | Maximum connections | `10` |
+| `DB_TIMEOUT` | Connection timeout (sec) | `30` |
+
+### PostgreSQL Tuning
+
+```sql
+-- Recommended settings for production
+ALTER SYSTEM SET shared_buffers = '256MB';
+ALTER SYSTEM SET effective_cache_size = '1GB';
+ALTER SYSTEM SET max_connections = 200;
+```
+
+## Storage Configuration
+
+### S3/MinIO Settings
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `S3_ENDPOINT` | Endpoint URL | Auto |
+| `S3_REGION` | AWS region | `us-east-1` |
+| `S3_BUCKET` | Default bucket | `botserver` |
+| `S3_ACCESS_KEY` | Access key | Auto |
+| `S3_SECRET_KEY` | Secret key | Auto |
+
+## Cache Configuration
+
+### Redis Settings
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `REDIS_URL` | Connection URL | Auto |
+| `CACHE_TTL` | Default TTL (seconds) | `3600` |
+| `SEMANTIC_CACHE_ENABLED` | Enable LLM caching | `true` |
+| `SEMANTIC_CACHE_THRESHOLD` | Similarity threshold | `0.95` |
+
+## Complete Example
+
+### config.csv
+
+```csv
+name,value
+theme-title,Acme Support Bot
+theme-color1,#1565C0
+theme-color2,#E3F2FD
+theme-logo,https://acme.com/logo.svg
+prompt-history,2
+prompt-compact,4
+prompt-temperature,0.7
+llm-provider,openai
+llm-model,gpt-4-turbo
+feature-voice,false
+feature-file-upload,true
+feature-suggestions,true
+auth-required,true
+rate-limit,30
+```
+
+### .env
+
+```bash
+# Auth
+DIRECTORY_URL=https://auth.acme.com
+DIRECTORY_CLIENT_ID=bot-client
+DIRECTORY_CLIENT_SECRET=supersecret
+
+# LLM
+OPENAI_API_KEY=sk-...
+
+# Server
+PORT=8080
+RUST_LOG=info
+
+# Rate limiting
+RATE_LIMIT_ENABLED=true
+RATE_LIMIT_API_RPS=100
+```
+
+## Configuration Precedence
+
+1. **Environment variables** (highest priority)
+2. **Bot config.csv**
+3. **Default values** (lowest priority)
+
+Environment variables always override config.csv settings.
+
+## Validation
+
+On startup, General Bots validates configuration and logs warnings for:
+
+- Missing required settings
+- Invalid values
+- Deprecated options
+- Security concerns (e.g., weak rate limits)
+
+Check logs for configuration issues:
+
+```bash
+RUST_LOG=info cargo run 2>&1 | grep -i config
+```