GeneralBots/botserver
2
1
Fork 0
Code Issues Pull requests Projects Releases 323 Packages Wiki Activity Actions

- New docds.

Browse source
This commit is contained in:
Rodrigo Rodriguez (Pragmatismo) 2025-11-22 16:12:32 -03:00
parent da4c80a3f2
commit 94107f828e
45 changed files with 6287 additions and 58 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
docs/src/SUMMARY.md
View file

@ -20,6 +20,7 @@
- [.gbot Bot Configuration](./chapter-02/gbot.md) - [.gbot Bot Configuration](./chapter-02/gbot.md)
- [.gbtheme UI Theming](./chapter-02/gbtheme.md) - [.gbtheme UI Theming](./chapter-02/gbtheme.md)
- [.gbdrive File Storage](./chapter-02/gbdrive.md) - [.gbdrive File Storage](./chapter-02/gbdrive.md)
- [Bot Templates](./chapter-02/templates.md)
# Part III - Knowledge Base # Part III - Knowledge Base
@ -59,12 +60,10 @@
- [LLM](./chapter-05/keyword-llm.md) - [LLM](./chapter-05/keyword-llm.md)
- [GET_BOT_MEMORY](./chapter-05/keyword-get-bot-memory.md) - [GET_BOT_MEMORY](./chapter-05/keyword-get-bot-memory.md)
- [SET_BOT_MEMORY](./chapter-05/keyword-set-bot-memory.md) - [SET_BOT_MEMORY](./chapter-05/keyword-set-bot-memory.md)
- [USE_KB](./chapter-05/keyword-use-kb.md) - [USE_KB](./chapter-05/keyword-use-kb.md)
- [CLEAR_KB](./chapter-05/keyword-clear-kb.md)
- [ADD_WEBSITE](./chapter-05/keyword-add-website.md) - [ADD_WEBSITE](./chapter-05/keyword-add-website.md)
- [USE_TOOL](./chapter-05/keyword-use-tool.md) - [USE_TOOL](./chapter-05/keyword-use-tool.md)
- [REMOVE_TOOL](./chapter-05/keyword-remove-tool.md)
- [CLEAR_TOOLS](./chapter-05/keyword-clear-tools.md) - [CLEAR_TOOLS](./chapter-05/keyword-clear-tools.md)
- [GET](./chapter-05/keyword-get.md) - [GET](./chapter-05/keyword-get.md)
- [FIND](./chapter-05/keyword-find.md) - [FIND](./chapter-05/keyword-find.md)
@ -73,7 +72,7 @@
- [SET_SCHEDULE](./chapter-05/keyword-set-schedule.md) - [SET_SCHEDULE](./chapter-05/keyword-set-schedule.md)
- [CREATE_SITE](./chapter-05/keyword-create-site.md) - [CREATE_SITE](./chapter-05/keyword-create-site.md)
- [CREATE_DRAFT](./chapter-05/keyword-create-draft.md) - [CREATE_DRAFT](./chapter-05/keyword-create-draft.md)
- [WEBSITE OF](./chapter-05/keyword-website-of.md) - [CREATE_TASK](./chapter-05/keyword-create-task.md)
- [PRINT](./chapter-05/keyword-print.md) - [PRINT](./chapter-05/keyword-print.md)
- [WAIT](./chapter-05/keyword-wait.md) - [WAIT](./chapter-05/keyword-wait.md)
- [FORMAT](./chapter-05/keyword-format.md) - [FORMAT](./chapter-05/keyword-format.md)
@ -81,6 +80,14 @@
- [LAST](./chapter-05/keyword-last.md) - [LAST](./chapter-05/keyword-last.md)
- [FOR EACH](./chapter-05/keyword-for-each.md) - [FOR EACH](./chapter-05/keyword-for-each.md)
- [EXIT FOR](./chapter-05/keyword-exit-for.md) - [EXIT FOR](./chapter-05/keyword-exit-for.md)
- [ADD_MEMBER](./chapter-05/keyword-add-member.md)
- [ADD_SUGGESTION](./chapter-05/keyword-add-suggestion.md)
- [CLEAR_SUGGESTIONS](./chapter-05/keyword-clear-suggestions.md)
- [BOOK](./chapter-05/keyword-book.md)
- [REMEMBER](./chapter-05/keyword-remember.md)
- [SAVE_FROM_UNSTRUCTURED](./chapter-05/keyword-save-from-unstructured.md)
- [SEND_MAIL](./chapter-05/keyword-send-mail.md)
- [WEATHER](./chapter-05/keyword-weather.md)
# Part VI - Extending BotServer # Part VI - Extending BotServer
@ -93,6 +100,7 @@
- [Service Layer](./chapter-06/services.md) - [Service Layer](./chapter-06/services.md)
- [Creating Custom Keywords](./chapter-06/custom-keywords.md) - [Creating Custom Keywords](./chapter-06/custom-keywords.md)
- [Adding Dependencies](./chapter-06/dependencies.md) - [Adding Dependencies](./chapter-06/dependencies.md)
- [Prompt Manager](./chapter-06/prompt-manager.md)
# Part VII - Bot Configuration # Part VII - Bot Configuration

369
docs/src/chapter-02/templates.md Normal file
View file

@ -0,0 +1,369 @@
# Bot Templates
BotServer comes with pre-built templates for common use cases. Each template is a complete `.gbai` package with dialogs, configurations, and knowledge bases ready to use.
## Available Templates
### Core Templates
#### default.gbai
The foundation template that all bots inherit from. Contains:
- Basic conversation handling
- Session management
- Error handling
- Standard responses
- Core dialog flows
#### template.gbai
A minimal starting point for custom bots with:
- Skeleton structure
- Basic configuration
- Example dialogs
- Placeholder knowledge base
### Business Templates
#### crm.gbai
Customer Relationship Management bot featuring:
- Contact management
- Lead tracking
- Customer inquiries
- Follow-up scheduling
- Sales pipeline integration
- Customer data lookup
#### erp.gbai
Enterprise Resource Planning assistant with:
- Inventory queries
- Order processing
- Supply chain info
- Resource allocation
- Business metrics
- Report generation
#### bi.gbai
Business Intelligence bot providing:
- Data analysis
- Report generation
- Dashboard queries
- KPI tracking
- Trend analysis
- Executive summaries
#### store.gbai
E-commerce assistant offering:
- Product catalog
- Order status
- Shopping cart help
- Payment processing
- Shipping information
- Return handling
### Communication Templates
#### announcements.gbai
Broadcast messaging system for:
- Company announcements
- News distribution
- Event notifications
- Alert broadcasting
- Multi-channel delivery
- Scheduled messages
#### broadcast.gbai
Mass communication bot with:
- Bulk messaging
- Audience segmentation
- Campaign management
- Delivery tracking
- Response collection
- Analytics reporting
#### whatsapp.gbai
WhatsApp-optimized bot featuring:
- WhatsApp Business API integration
- Media handling
- Quick replies
- List messages
- Location sharing
- Contact cards
#### reminder.gbai
Automated reminder system for:
- Task reminders
- Appointment notifications
- Deadline alerts
- Recurring reminders
- Calendar integration
- Follow-up scheduling
### AI & Automation Templates
#### ai-search.gbai
Advanced search assistant with:
- Semantic search
- Multi-source queries
- Result ranking
- Context understanding
- Query refinement
- Search analytics
#### llm-server.gbai
LLM gateway bot providing:
- Model selection
- Prompt management
- Token optimization
- Response caching
- Rate limiting
- Cost tracking
#### llm-tools.gbai
AI tools collection featuring:
- Text generation
- Summarization
- Translation
- Code generation
- Image description
- Sentiment analysis
#### crawler.gbai
Web scraping bot with:
- Site crawling
- Data extraction
- Content indexing
- Change monitoring
- Structured data parsing
- API integration
#### talk-to-data.gbai
Data conversation interface offering:
- Natural language queries
- Database access
- Data visualization
- Export capabilities
- Statistical analysis
- Report generation
### Industry Templates
#### edu.gbai
Education assistant providing:
- Course information
- Student support
- Assignment help
- Schedule queries
- Resource access
- Grade lookup
#### law.gbai
Legal information bot with:
- Legal term definitions
- Document templates
- Case lookup
- Regulation queries
- Compliance checking
- Disclaimer management
#### marketing.gbai
Marketing automation bot featuring:
- Lead generation
- Campaign management
- Content distribution
- Social media integration
- Analytics tracking
- A/B testing
### Integration Templates
#### api-client.gbai
REST API integration bot with:
- API endpoint management
- Authentication handling
- Request formatting
- Response parsing
- Error handling
- Rate limiting
#### public-apis.gbai
Public API aggregator providing:
- Weather information
- News feeds
- Stock prices
- Currency conversion
- Maps/directions
- Public data access
#### backup.gbai
Backup management bot offering:
- Scheduled backups
- Data archiving
- Restore operations
- Backup verification
- Storage management
- Disaster recovery
## Using Templates
### Quick Start
1. **Copy template to your workspace**:
```bash
cp -r templates/crm.gbai mybot.gbai
```
2. **Customize configuration**:
```bash
cd mybot.gbai/mybot.gbot
vim config.csv
```
3. **Modify dialogs**:
```bash
cd ../mybot.gbdialog
vim start.bas
```
4. **Add knowledge base**:
```bash
cd ../mybot.gbkb
# Add your documents
```
### Template Structure
Every template follows this structure:
```
template-name.gbai/
├── template-name.gbdialog/
│ ├── start.bas # Entry point
│ ├── menu.bas # Menu system
│ └── tools/ # Tool definitions
├── template-name.gbot/
│ └── config.csv # Configuration
├── template-name.gbkb/
│ ├── docs/ # Documentation
│ └── data/ # Reference data
└── template-name.gbtheme/
└── style.css # Optional theming
```
## Customization Guide
### Extending Templates
Templates are designed to be extended:
1. **Inherit from template**:
```basic
INCLUDE "template://default/common.bas"
```
2. **Override functions**:
```basic
FUNCTION handle_greeting()
' Custom greeting logic
TALK "Welcome to MyBot!"
END FUNCTION
```
3. **Add new features**:
```basic
' Add to existing template
FUNCTION new_feature()
' Your custom code
END FUNCTION
```
### Combining Templates
Mix features from multiple templates:
```basic
' Use CRM contact management
INCLUDE "template://crm/contacts.bas"
' Add marketing automation
INCLUDE "template://marketing/campaigns.bas"
' Integrate with APIs
INCLUDE "template://api-client/rest.bas"
```
## Best Practices
### Template Selection
1. **Start with the right template**: Choose based on primary use case
2. **Combine when needed**: Mix templates for complex requirements
3. **Keep core intact**: Don't modify template originals
4. **Document changes**: Track customizations
### Customization Tips
1. **Configuration first**: Adjust config.csv before code
2. **Test incrementally**: Verify each change
3. **Preserve structure**: Maintain template organization
4. **Version control**: Track template modifications
### Performance Considerations
1. **Remove unused features**: Delete unnecessary dialogs
2. **Optimize knowledge base**: Index only needed content
3. **Configure appropriately**: Adjust settings for scale
4. **Monitor resource usage**: Track memory and CPU
## Template Development
### Creating Custom Templates
1. **Start from template.gbai**: Use as foundation
2. **Define clear purpose**: Document template goals
3. **Include examples**: Provide sample data
4. **Write documentation**: Explain usage
5. **Test thoroughly**: Verify all features
### Template Guidelines
- Keep templates focused on specific use cases
- Include comprehensive examples
- Provide clear documentation
- Use consistent naming conventions
- Include error handling
- Make configuration obvious
- Test across channels
## Contributing Templates
To contribute a new template:
1. Create template in `templates/` directory
2. Include README with description
3. Add example configuration
4. Provide sample knowledge base
5. Include test cases
6. Submit pull request
## Template Updates
Templates are versioned and updated regularly:
- Bug fixes
- Security patches
- Feature additions
- Performance improvements
- Documentation updates
Check for updates:
```bash
git pull
diff templates/template-name.gbai
```
## Support
For template-specific help:
- Check template README
- Review example code
- Consult documentation
- Ask in community forums
- Report issues on GitHub

195
docs/src/chapter-05/keyword-add-kb.md
View file

@ -1 +1,196 @@
# USE_KB # USE_KB
Load and activate a knowledge base collection for the current conversation.
## Syntax
```basic
USE_KB kb_name
```
## Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `kb_name` | String | Name of the knowledge base collection to load |
## Description
The `USE_KB` keyword loads a knowledge base collection into the current session's context, making its documents searchable via `FIND` and available to the LLM for context-aware responses. Knowledge bases are vector collections stored in Qdrant containing indexed documents, FAQs, or other reference materials.
Multiple knowledge bases can be active simultaneously, allowing the bot to access diverse information sources.
## Examples
### Load Single Knowledge Base
```basic
USE_KB "product-docs"
answer = FIND "installation guide"
TALK answer
```
### Load Multiple Knowledge Bases
```basic
USE_KB "company-policies"
USE_KB "hr-handbook"
USE_KB "benefits-guide"
question = HEAR "What's the vacation policy?"
answer = FIND question
TALK answer
```
### Conditional KB Loading
```basic
department = HEAR "Which department are you from?"
IF department = "engineering" THEN
USE_KB "technical-docs"
USE_KB "api-reference"
ELSE IF department = "sales" THEN
USE_KB "product-catalog"
USE_KB "pricing-guide"
ELSE
USE_KB "general-info"
END IF
```
### Dynamic KB Selection
```basic
topic = DETECT_TOPIC(user_message)
kb_name = "kb_" + topic
USE_KB kb_name
```
## Knowledge Base Types
Common KB collections include:
- **Documentation**: Product manuals, guides, tutorials
- **FAQs**: Frequently asked questions and answers
- **Policies**: Company policies, procedures, guidelines
- **Products**: Catalogs, specifications, pricing
- **Support**: Troubleshooting guides, known issues
- **Legal**: Terms, contracts, compliance documents
## KB Naming Convention
Knowledge bases follow this naming pattern:
- Format: `category_subcategory_version`
- Examples: `docs_api_v2`, `support_faq_current`, `products_2024`
## Loading Behavior
When `USE_KB` is called:
1. Checks if KB exists in Qdrant
2. Loads vector embeddings into memory
3. Adds to session's active KB list
4. Makes content searchable
5. Updates context for LLM
## Memory Management
- KBs remain loaded for entire session
- Use `CLEAR_KB` to unload specific KB
- Maximum 10 KBs active simultaneously
- Automatically cleared on session end
## Error Handling
```basic
TRY
USE_KB "special-docs"
TALK "Knowledge base loaded successfully"
CATCH "kb_not_found"
TALK "That knowledge base doesn't exist"
USE_KB "default-docs" ' Fallback
CATCH "kb_error"
LOG "Failed to load KB"
TALK "Having trouble accessing documentation"
END TRY
```
## Performance Considerations
- First load may take 1-2 seconds
- Subsequent queries are cached
- Large KBs (>10,000 documents) may impact response time
- Consider loading only necessary KBs
## KB Content Management
### Creating Knowledge Bases
KBs are created from document collections in `.gbkb` packages:
```
mybot.gbkb/
├── docs/ # Becomes "docs" KB
├── faqs/ # Becomes "faqs" KB
└── policies/ # Becomes "policies" KB
```
### Updating Knowledge Bases
- KBs are indexed during bot deployment
- Updates require re-indexing
- Use version suffixes for updates
## Best Practices
1. **Load Relevant KBs Early**: Load at conversation start
2. **Use Descriptive Names**: Make KB purpose clear
3. **Limit Active KBs**: Don't load unnecessary collections
4. **Clear When Done**: Remove KBs when changing context
5. **Handle Missing KBs**: Always have fallback options
6. **Version Your KBs**: Track KB updates with versions
## Integration with Other Keywords
- **[FIND](./keyword-find.md)**: Search within loaded KBs
- **[CLEAR_KB](./keyword-clear-kb.md)**: Unload knowledge bases
- **[ADD_WEBSITE](./keyword-add-website.md)**: Create KB from website
- **[LLM](./keyword-llm.md)**: Use KB context in responses
## Advanced Usage
### KB Metadata
```basic
kb_info = GET_KB_INFO("product-docs")
TALK "KB contains " + kb_info.doc_count + " documents"
TALK "Last updated: " + kb_info.update_date
```
### Conditional Loading Based on Language
```basic
language = GET_USER_LANGUAGE()
USE_KB "docs_" + language ' docs_en, docs_es, docs_fr
```
### KB Search with Filters
```basic
USE_KB "all-products"
' Search only recent products
results = FIND_WITH_FILTER "wireless", "year >= 2023"
```
## Troubleshooting
### KB Not Loading
- Verify KB name is correct
- Check if KB was properly indexed
- Ensure Qdrant service is running
- Review bot logs for errors
### Slow Performance
- Reduce number of active KBs
- Optimize KB content (remove duplicates)
- Check Qdrant server resources
- Consider KB partitioning
### Incorrect Results
- Verify KB contains expected content
- Check document quality
- Review indexing settings
- Test with specific queries
## Implementation
Located in `src/basic/keywords/use_kb.rs`
The implementation connects to Qdrant vector database and manages KB collections per session.

110
docs/src/chapter-05/keyword-add-member.md Normal file
View file

@ -0,0 +1,110 @@
# ADD_MEMBER
Add users to groups or teams within the bot system.
## Syntax
```basic
ADD_MEMBER group_id, user_email, role
```
## Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `group_id` | String | Unique identifier or name of the group |
| `user_email` | String | Email address of the user to add |
| `role` | String | Role to assign to the user in the group (e.g., "member", "admin", "moderator") |
## Description
The `ADD_MEMBER` keyword manages group membership by adding users to specified groups with defined roles. This keyword:
- Validates user existence in the system
- Checks group permissions
- Assigns appropriate role-based access
- Sends notification to the new member
- Updates group analytics
## Examples
### Add Regular Member
```basic
ADD_MEMBER "engineering-team", "developer@company.com", "member"
```
### Add Group Administrator
```basic
ADD_MEMBER "project-alpha", "manager@company.com", "admin"
```
### Dynamic Group Assignment
```basic
group_name = "support-" + DEPARTMENT
user = GET_USER_EMAIL()
ADD_MEMBER group_name, user, "agent"
```
### Bulk Member Addition
```basic
team_members = ["alice@example.com", "bob@example.com", "carol@example.com"]
FOR EACH member IN team_members
ADD_MEMBER "dev-team", member, "developer"
NEXT
```
## Return Value
Returns a membership object containing:
- `membership_id`: Unique identifier for the membership
- `group_id`: Group identifier
- `user_id`: User identifier
- `role`: Assigned role
- `joined_at`: Timestamp of when the user was added
- `status`: "active", "pending", or "error"
## Error Handling
- Validates email format before processing
- Checks if user already exists in the group
- Verifies current user has permission to add members
- Returns error if group doesn't exist
- Handles role validation against allowed roles
## Permissions
The user executing this command must have one of the following:
- Group admin privileges
- System administrator role
- Explicit "add_member" permission for the group
## Related Database Tables
- `users`: User information
- `groups`: Group definitions
- `group_members`: Membership associations
- `roles`: Role definitions and permissions
## Best Practices
1. **Verify Permissions**: Always check if the executing user has rights to add members
2. **Validate Roles**: Ensure the role exists and is valid for the group type
3. **Send Notifications**: Notify both the new member and group admins
4. **Audit Trail**: Log all membership changes for compliance
5. **Handle Duplicates**: Check for existing membership before adding
## See Also
- [CREATE_TASK](./keyword-create-task.md) - Assign tasks to group members
- [SEND_MAIL](./keyword-send-mail.md) - Send group notifications
- [SET_USER](./keyword-set-user.md) - Set user context
## Implementation
Located in `src/basic/keywords/add_member.rs`
The implementation uses:
- Database transactions for atomic operations
- Email validation via regex
- Role-based access control (RBAC)
- Async notification dispatch

176
docs/src/chapter-05/keyword-add-suggestion.md Normal file
View file

@ -0,0 +1,176 @@
# ADD_SUGGESTION
Add conversational suggestions or quick reply options for user interactions.
## Syntax
```basic
ADD_SUGGESTION text AS value
```
## Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `text` | String | Display text shown to the user |
| `value` | String | Value sent when suggestion is clicked |
## Description
The `ADD_SUGGESTION` keyword adds quick reply buttons or suggestion chips to the conversation interface. These provide users with:
- Quick action buttons
- Common response options
- Guided conversation paths
- Menu-like interactions
- Context-aware suggestions
Suggestions appear as clickable elements in supported channels (web, WhatsApp, Teams, etc.).
## Examples
### Basic Suggestions
```basic
ADD_SUGGESTION "Yes" AS "confirm"
ADD_SUGGESTION "No" AS "decline"
ADD_SUGGESTION "Maybe later" AS "postpone"
answer = HEAR "Would you like to proceed?"
```
### Dynamic Suggestions from Data
```basic
departments = ["Sales", "Support", "Billing", "Technical"]
FOR EACH dept IN departments
ADD_SUGGESTION dept AS dept
NEXT
selection = HEAR "Which department do you need?"
```
### Context-Based Suggestions
```basic
IF user_type = "new" THEN
ADD_SUGGESTION "Get Started" AS "onboarding"
ADD_SUGGESTION "View Tutorial" AS "tutorial"
ADD_SUGGESTION "Contact Support" AS "help"
ELSE
ADD_SUGGESTION "Check Status" AS "status"
ADD_SUGGESTION "New Request" AS "create"
ADD_SUGGESTION "View History" AS "history"
END IF
```
### Menu System
```basic
' Main menu
CLEAR_SUGGESTIONS
ADD_SUGGESTION "Products" AS "menu_products"
ADD_SUGGESTION "Services" AS "menu_services"
ADD_SUGGESTION "Support" AS "menu_support"
ADD_SUGGESTION "About Us" AS "menu_about"
choice = HEAR "What would you like to know about?"
IF choice = "menu_products" THEN
CLEAR_SUGGESTIONS
ADD_SUGGESTION "Pricing" AS "product_pricing"
ADD_SUGGESTION "Features" AS "product_features"
ADD_SUGGESTION "Compare" AS "product_compare"
ADD_SUGGESTION "Back" AS "menu_main"
END IF
```
## Channel Support
| Channel | Display Type | Limitations |
|---------|-------------|-------------|
| Web | Buttons/Chips | Up to 10 suggestions |
| WhatsApp | Reply Buttons | Max 3 buttons |
| Teams | Hero Cards | Unlimited |
| Slack | Block Actions | Up to 5 per block |
| SMS | Numbered List | Text-only fallback |
## Suggestion Persistence
Suggestions remain active until:
- User clicks one
- `CLEAR_SUGGESTIONS` is called
- New suggestions replace them
- Conversation ends
- Timeout occurs (configurable)
## Styling and Appearance
Suggestions adapt to channel capabilities:
- **Rich Channels**: Styled buttons with icons
- **Basic Channels**: Numbered text options
- **Voice**: Read as options list
## Return Value
The keyword itself returns `true` if suggestion was added successfully, `false` otherwise.
When user clicks a suggestion, the value is returned to the next `HEAR` command.
## Memory Management
Suggestions are stored in session cache:
- Each session maintains its own suggestion list
- Cleared automatically on session end
- Can store up to 50 suggestions per session
## Best Practices
1. **Clear Before Adding**: Always clear old suggestions when changing context
2. **Limit Options**: Keep to 3-5 suggestions for better UX
3. **Descriptive Text**: Make suggestion text clear and actionable
4. **Meaningful Values**: Use values that are easy to handle in code
5. **Provide Escape**: Always include a "Back" or "Cancel" option
6. **Mobile First**: Consider mobile screen sizes
## Advanced Features
### Suggestion Groups
```basic
' Group related suggestions
ADD_SUGGESTION "Small ($10)" AS "size:small"
ADD_SUGGESTION "Medium ($15)" AS "size:medium"
ADD_SUGGESTION "Large ($20)" AS "size:large"
```
### Conditional Display
```basic
IF has_permission("admin") THEN
ADD_SUGGESTION "Admin Panel" AS "admin"
END IF
```
### Localized Suggestions
```basic
language = GET_USER_LANGUAGE()
IF language = "es" THEN
ADD_SUGGESTION "Sí" AS "yes"
ADD_SUGGESTION "No" AS "no"
ELSE
ADD_SUGGESTION "Yes" AS "yes"
ADD_SUGGESTION "No" AS "no"
END IF
```
## Error Handling
- Silently fails if suggestion limit exceeded
- Logs warning if value is too long
- Falls back to text input if channel doesn't support suggestions
## Related Keywords
- [CLEAR_SUGGESTIONS](./keyword-clear-suggestions.md) - Remove all suggestions
- [HEAR](./keyword-hear.md) - Wait for user input (including suggestion clicks)
- [TALK](./keyword-talk.md) - Send message with suggestions
- [FORMAT](./keyword-format.md) - Format suggestion text
## Implementation
Located in `src/basic/keywords/add_suggestion.rs`
Uses Redis cache for storage when available, falls back to in-memory storage.

241
docs/src/chapter-05/keyword-add-tool.md
View file

@ -1 +1,242 @@
# USE_TOOL # USE_TOOL
Add and activate a tool (custom dialog script) for the current conversation.
## Syntax
```basic
USE_TOOL "tool-name.bas"
```
## Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `tool-name.bas` | String | Path to the tool's BASIC script file |
## Description
The `USE_TOOL` keyword dynamically loads a tool definition from a `.bas` file and makes its functionality available in the current conversation. Tools are reusable dialog scripts that extend the bot's capabilities with custom functions, API integrations, or specialized workflows.
Once loaded, the tool's keywords and functions become available for use in the conversation until the session ends or the tool is explicitly cleared.
## Examples
### Load a Simple Tool
```basic
USE_TOOL "weather.bas"
' Now weather functions are available
result = GET_WEATHER("New York")
TALK result
```
### Load Multiple Tools
```basic
USE_TOOL "calculator.bas"
USE_TOOL "translator.bas"
USE_TOOL "scheduler.bas"
' All three tools are now active
sum = CALCULATE("150 + 250")
translated = TRANSLATE(sum, "Spanish")
SCHEDULE_REMINDER(translated, "tomorrow")
```
### Conditional Tool Loading
```basic
task = HEAR "What would you like to do?"
IF task CONTAINS "email" THEN
USE_TOOL "email-composer.bas"
ELSE IF task CONTAINS "calendar" THEN
USE_TOOL "calendar-manager.bas"
ELSE IF task CONTAINS "document" THEN
USE_TOOL "document-processor.bas"
END IF
```
### Tool with Parameters
```basic
' Some tools accept configuration
USE_TOOL "api-client.bas"
CONFIGURE_API("https://api.example.com", api_key)
response = CALL_API("GET", "/users")
```
## Tool Structure
Tools are BASIC scripts that define:
- **Functions**: Reusable operations
- **Keywords**: Custom commands
- **Integrations**: API connections
- **Workflows**: Multi-step processes
Example tool file (`calculator.bas`):
```basic
FUNCTION CALCULATE(expression)
result = EVAL(expression)
RETURN result
END FUNCTION
FUNCTION PERCENTAGE(value, percent)
RETURN value * percent / 100
END FUNCTION
```
## Tool Discovery
Tools are discovered from:
1. `.gbdialog/tools/` directory in bot package
2. System tools directory (`/opt/tools/`)
3. User tools directory (`~/.gbtools/`)
4. Inline tool definitions
## Return Value
Returns a status object:
- `success`: Boolean indicating if tool loaded
- `tool_name`: Name of the loaded tool
- `functions_added`: List of new functions available
- `error`: Error message if loading failed
## Tool Compilation
When a tool is loaded:
1. Script is parsed and validated
2. Functions are compiled to MCP format
3. OpenAI function format generated
4. Tool registered in session context
5. Functions become callable
## Session Scope
- Tools are session-specific
- Don't affect other conversations
- Automatically unloaded on session end
- Can be manually removed with `CLEAR_TOOLS`
## Error Handling
```basic
TRY
USE_TOOL "advanced-tool.bas"
TALK "Tool loaded successfully"
CATCH "tool_not_found"
TALK "Tool file doesn't exist"
CATCH "compilation_error"
TALK "Tool has syntax errors"
CATCH "permission_denied"
TALK "Not authorized to use this tool"
END TRY
```
## Best Practices
1. **Load Tools Early**: Load at conversation start when possible
2. **Check Dependencies**: Ensure required services are available
3. **Handle Failures**: Always have fallback behavior
4. **Document Tools**: Include usage comments in tool files
5. **Version Tools**: Use version numbers in tool names
6. **Test Thoroughly**: Validate tools before deployment
7. **Limit Tool Count**: Don't load too many tools at once
## Tool Management
### List Active Tools
```basic
tools = GET_ACTIVE_TOOLS()
FOR EACH tool IN tools
TALK "Active tool: " + tool.name
NEXT
```
### Check Tool Status
```basic
IF IS_TOOL_ACTIVE("calculator.bas") THEN
result = CALCULATE("2+2")
ELSE
USE_TOOL "calculator.bas"
END IF
```
### Tool Versioning
```basic
' Load specific version
USE_TOOL "reporter-v2.bas"
' Check version
version = GET_TOOL_VERSION("reporter")
IF version < 2 THEN
CLEAR_TOOLS()
USE_TOOL "reporter-v2.bas"
END IF
```
## Advanced Features
### Tool Chaining
```basic
USE_TOOL "data-fetcher.bas"
data = FETCH_DATA(source)
USE_TOOL "data-processor.bas"
processed = PROCESS_DATA(data)
USE_TOOL "report-generator.bas"
report = GENERATE_REPORT(processed)
```
### Dynamic Tool Creation
```basic
' Create tool from template
CREATE_TOOL_FROM_TEMPLATE("custom-api", api_config)
USE_TOOL "custom-api.bas"
```
### Tool Permissions
```basic
IF HAS_PERMISSION("admin-tools") THEN
USE_TOOL "admin-console.bas"
ELSE
TALK "Admin tools require elevated permissions"
END IF
```
## Performance Considerations
- Tool compilation happens once per session
- Compiled tools are cached
- Large tools may increase memory usage
- Consider lazy loading for complex tools
## Troubleshooting
### Tool Not Loading
- Check file path is correct
- Verify `.bas` extension
- Ensure file has read permissions
- Check for syntax errors in tool
### Functions Not Available
- Confirm tool loaded successfully
- Check function names match exactly
- Verify no naming conflicts
- Review tool compilation logs
### Performance Issues
- Limit number of active tools
- Use lighter tool versions
- Consider tool optimization
- Check for infinite loops in tools
## Related Keywords
- [CLEAR_TOOLS](./keyword-clear-tools.md) - Remove all tools
- [GET](./keyword-get.md) - Often used within tools
- [LLM](./keyword-llm.md) - Tools can enhance LLM capabilities
- [FORMAT](./keyword-format.md) - Format tool outputs
## Implementation
Located in `src/basic/keywords/use_tool.rs`
Integrates with the tool compiler to dynamically load and compile BASIC tool scripts into callable functions.

86
docs/src/chapter-05/keyword-book.md Normal file
View file

@ -0,0 +1,86 @@
# BOOK
Schedule meetings and calendar events with attendees.
## Syntax
```basic
BOOK attendees, subject, duration
```
## Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `attendees` | String/Array | Email address(es) of attendees. Can be a single email or array of emails |
| `subject` | String | Meeting subject/title |
| `duration` | String | Duration of the meeting (e.g., "30m", "1h", "90m") |
## Description
The `BOOK` keyword creates calendar events and schedules meetings with specified attendees. It integrates with the calendar engine to:
- Find available time slots for all attendees
- Create calendar invitations
- Send meeting notifications
- Handle timezone conversions
- Support recurring meetings
## Examples
### Single Attendee Meeting
```basic
BOOK "john@example.com", "Project Review", "30m"
```
### Multiple Attendees
```basic
attendees = ["alice@example.com", "bob@example.com", "carol@example.com"]
BOOK attendees, "Team Standup", "15m"
```
### Dynamic Meeting Scheduling
```basic
email = GET_USER_EMAIL()
BOOK email, "1-on-1 Discussion", "45m"
```
## Integration Points
- **Calendar Engine**: Uses `calendar_engine` module for availability checking
- **Email Notifications**: Sends invitations via email integration
- **User Sessions**: Respects user timezone preferences
- **Meeting Rooms**: Can optionally reserve physical/virtual rooms
## Return Value
Returns a meeting object containing:
- `meeting_id`: Unique identifier for the meeting
- `start_time`: Scheduled start time
- `end_time`: Scheduled end time
- `meeting_link`: Virtual meeting URL (if applicable)
- `status`: Booking status ("confirmed", "tentative", "failed")
## Error Handling
- Returns error if no common availability found
- Validates email addresses before sending invites
- Checks for calendar permissions
- Handles timezone mismatches gracefully
## Best Practices
1. **Check Availability First**: Consider using availability checks for critical meetings
2. **Provide Context**: Include meaningful subject lines
3. **Respect Working Hours**: System checks business hours by default
4. **Handle Conflicts**: Implement retry logic for failed bookings
## See Also
- [CREATE_TASK](./keyword-create-task.md) - Create tasks instead of meetings
- [SET_SCHEDULE](./keyword-set-schedule.md) - Schedule recurring events
- [SEND_MAIL](./keyword-send-mail.md) - Send email notifications
## Implementation
Located in `src/basic/keywords/book.rs`

146
docs/src/chapter-05/keyword-clear-kb.md Normal file
View file

@ -0,0 +1,146 @@
# CLEAR_KB
Clear knowledge base collections from the current session context.
## Syntax
```basic
CLEAR_KB kb_name
```
or
```basic
CLEAR_KB
```
## Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `kb_name` | String | Optional. Name of specific knowledge base to remove. If omitted, clears all KBs |
## Description
The `CLEAR_KB` keyword removes knowledge base collections from the current session's context. This is useful for:
- Switching between different knowledge domains
- Reducing context size for performance
- Preventing irrelevant KB interference
- Managing memory usage in long-running sessions
When called without parameters, it clears all active knowledge bases. When called with a specific KB name, it only removes that collection.
## Examples
### Clear Specific Knowledge Base
```basic
USE_KB "product-docs"
USE_KB "user-manual"
' Later, remove just one
CLEAR_KB "product-docs"
' user-manual remains active
```
### Clear All Knowledge Bases
```basic
USE_KB "docs"
USE_KB "faqs"
USE_KB "policies"
' Clear everything
CLEAR_KB
' No KBs active now
```
### Conditional KB Management
```basic
IF topic = "technical" THEN
CLEAR_KB
USE_KB "engineering-docs"
ELSE
CLEAR_KB "engineering-docs"
USE_KB "general-docs"
END IF
```
### Knowledge Base Rotation
```basic
' Use KB for initial query
USE_KB "current-year-data"
answer = FIND "quarterly results"
TALK answer
' Switch to historical data
CLEAR_KB "current-year-data"
USE_KB "historical-data"
historical = FIND "previous year results"
TALK historical
```
## Return Value
Returns a boolean indicating success:
- `true`: Knowledge base(s) successfully cleared
- `false`: Operation failed or KB not found
## Session Context
The `CLEAR_KB` command affects only the current user session. Other sessions maintain their own KB contexts independently.
## Memory Management
Clearing knowledge bases:
- Removes vector collection references from session
- Frees up context window space
- Does NOT delete the actual KB data (only removes from active context)
- Reduces memory footprint of the session
## Error Handling
- Silently succeeds if specified KB is not currently loaded
- Returns false if KB name is invalid format
- Logs warning if clearing fails due to system constraints
## Performance Considerations
1. **Context Size**: Clearing unused KBs reduces token usage in LLM calls
2. **Query Speed**: Fewer active KBs means faster semantic search
3. **Memory**: Each KB consumes memory for index caching
4. **Relevance**: Too many KBs can introduce irrelevant results
## Best Practices
1. **Clear Before Switching**: Always clear old KBs before loading new ones for different domains
2. **Periodic Cleanup**: In long conversations, periodically clear unused KBs
3. **Domain Separation**: Don't mix unrelated knowledge domains
4. **Check Before Clear**: Optionally check if KB is loaded before clearing
Example of good practice:
```basic
' Clean switch between domains
CLEAR_KB
IF customer_type = "enterprise" THEN
USE_KB "enterprise-docs"
USE_KB "sla-policies"
ELSE
USE_KB "standard-docs"
END IF
```
## Related Keywords
- [USE_KB](./keyword-use-kb.md) - Load knowledge base collections
- [ADD_WEBSITE](./keyword-add-website.md) - Add website content to KB
- [FIND](./keyword-find.md) - Search within loaded KBs
- [LLM](./keyword-llm.md) - Query LLM with KB context
## Implementation Details
Located in `src/basic/keywords/clear_kb.rs`
The implementation:
- Maintains KB references in session state
- Uses HashSet for efficient KB tracking
- Integrates with Qdrant vector store
- Handles concurrent access safely
- Updates session metrics for monitoring

199
docs/src/chapter-05/keyword-clear-suggestions.md Normal file
View file

@ -0,0 +1,199 @@
# CLEAR_SUGGESTIONS
Remove all active suggestions from the current conversation interface.
## Syntax
```basic
CLEAR_SUGGESTIONS
```
## Parameters
None - this keyword takes no parameters.
## Description
The `CLEAR_SUGGESTIONS` keyword removes all quick reply buttons and suggestion chips from the conversation interface. Use this to:
- Reset the suggestion list before showing new options
- Clean up the interface after user selection
- Remove outdated suggestions when context changes
- Prevent confusion from lingering options
- Force text input instead of button selection
## Examples
### Basic Clear
```basic
' Show initial options
ADD_SUGGESTION "Yes" AS "yes"
ADD_SUGGESTION "No" AS "no"
answer = HEAR "Do you agree?"
' Clear after response
CLEAR_SUGGESTIONS
TALK "Thank you for your response"
```
### Menu Navigation
```basic
' Main menu
CLEAR_SUGGESTIONS
ADD_SUGGESTION "Products" AS "products"
ADD_SUGGESTION "Support" AS "support"
choice = HEAR "Select an option:"
' Clear and show submenu
CLEAR_SUGGESTIONS
IF choice = "products" THEN
ADD_SUGGESTION "View All" AS "all"
ADD_SUGGESTION "Categories" AS "categories"
ADD_SUGGESTION "Back" AS "main"
END IF
```
### Conditional Clearing
```basic
IF conversation_stage = "complete" THEN
CLEAR_SUGGESTIONS
TALK "Thank you for using our service!"
ELSE
' Keep suggestions active
TALK "Please select an option above"
END IF
```
### Error Recovery
```basic
retry_count = 0
DO
CLEAR_SUGGESTIONS
ADD_SUGGESTION "Retry" AS "retry"
ADD_SUGGESTION "Cancel" AS "cancel"
ADD_SUGGESTION "Help" AS "help"
action = HEAR "Operation failed. What would you like to do?"
retry_count = retry_count + 1
LOOP WHILE action = "retry" AND retry_count < 3
CLEAR_SUGGESTIONS
```
## Behavior
When `CLEAR_SUGGESTIONS` is called:
1. All suggestions are removed from UI immediately
2. Session cache is cleared of suggestion data
3. Channel-specific cleanup occurs
4. Interface returns to text input mode
5. No error if no suggestions exist
## Channel-Specific Behavior
| Channel | Effect |
|---------|--------|
| Web | Removes button/chip elements |
| WhatsApp | Clears reply keyboard |
| Teams | Removes card actions |
| Slack | Updates message blocks |
| Voice | Stops listing options |
| SMS | No visible effect |
## Performance
- Instant execution (< 1ms)
- Minimal memory impact
- No network calls required
- Thread-safe operation
## Best Practices
1. **Clear Before Adding**: Always clear before adding new suggestion sets
```basic
CLEAR_SUGGESTIONS
ADD_SUGGESTION "Option 1" AS "opt1"
ADD_SUGGESTION "Option 2" AS "opt2"
```
2. **Clear After Selection**: Remove suggestions once user has chosen
```basic
choice = HEAR "Select:"
CLEAR_SUGGESTIONS
TALK "You selected: " + choice
```
3. **Clear on Context Change**: Remove irrelevant suggestions
```basic
IF topic_changed THEN
CLEAR_SUGGESTIONS
END IF
```
4. **Clear on Error**: Reset interface on failures
```basic
ON ERROR
CLEAR_SUGGESTIONS
TALK "Something went wrong. Please type your request."
END ON
```
## Common Patterns
### Wizard/Multi-Step Flow
```basic
' Step 1
CLEAR_SUGGESTIONS
ADD_SUGGESTION "Personal" AS "personal"
ADD_SUGGESTION "Business" AS "business"
account_type = HEAR "Account type?"
' Step 2
CLEAR_SUGGESTIONS
ADD_SUGGESTION "Basic" AS "basic"
ADD_SUGGESTION "Premium" AS "premium"
plan = HEAR "Select plan:"
' Complete
CLEAR_SUGGESTIONS
TALK "Setup complete!"
```
### Dynamic Menu System
```basic
FUNCTION show_menu(menu_name)
CLEAR_SUGGESTIONS
menu_items = GET_MENU_ITEMS(menu_name)
FOR EACH item IN menu_items
ADD_SUGGESTION item.label AS item.value
NEXT
END FUNCTION
```
## Error Handling
- Never throws errors
- Safe to call multiple times
- No-op if no suggestions exist
- Handles concurrent calls gracefully
## Memory Management
Calling `CLEAR_SUGGESTIONS`:
- Frees suggestion cache memory
- Removes Redis entries if cached
- Cleans up UI resources
- Prevents memory leaks in long conversations
## Related Keywords
- [ADD_SUGGESTION](./keyword-add-suggestion.md) - Add quick reply options
- [HEAR](./keyword-hear.md) - Get user input from suggestions
- [TALK](./keyword-talk.md) - Display messages with suggestions
## Implementation
Located in `src/basic/keywords/add_suggestion.rs`
Shares implementation with `ADD_SUGGESTION` for efficient suggestion management.

190
docs/src/chapter-05/keyword-create-task.md Normal file
View file

@ -0,0 +1,190 @@
# CREATE_TASK
Create and assign tasks within the task management system.
## Syntax
```basic
CREATE_TASK title, description, assignee, due_date, priority
```
## Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `title` | String | Task title/name |
| `description` | String | Detailed task description |
| `assignee` | String | Email or user ID of the assignee |
| `due_date` | String | Due date in format "YYYY-MM-DD" or relative like "tomorrow", "next week" |
| `priority` | String | Task priority: "low", "medium", "high", "urgent" |
## Description
The `CREATE_TASK` keyword creates tasks in the task engine system with:
- Automatic assignment to users or groups
- Due date tracking and reminders
- Priority-based organization
- Integration with calendar system
- Email notifications to assignees
- Progress tracking capabilities
## Examples
### Basic Task Creation
```basic
CREATE_TASK "Review proposal", "Review and provide feedback on Q4 proposal", "john@example.com", "2024-01-15", "high"
```
### Task with Current User
```basic
user_email = GET_USER_EMAIL()
CREATE_TASK "Follow up", "Contact customer about renewal", user_email, "tomorrow", "medium"
```
### Bulk Task Creation
```basic
team = ["alice@example.com", "bob@example.com", "carol@example.com"]
FOR EACH member IN team
CREATE_TASK "Complete training", "Finish security awareness training", member, "next week", "medium"
NEXT
```
### Task from User Input
```basic
task_info = HEAR "What task should I create?"
CREATE_TASK task_info, "User requested task", "support@example.com", "today", "high"
TALK "Task created and assigned to support team"
```
## Return Value
Returns a task object containing:
- `task_id`: Unique task identifier
- `status`: Task status ("created", "assigned", "in_progress", "completed")
- `created_at`: Creation timestamp
- `url`: Link to task in web interface
- `reminder_set`: Whether reminder was configured
## Task Statuses
Tasks progress through these statuses:
1. `created` - Initial creation
2. `assigned` - Assigned to user
3. `in_progress` - Work started
4. `blocked` - Waiting on dependency
5. `completed` - Task finished
6. `cancelled` - Task cancelled
## Integration Points
### Calendar Integration
Tasks automatically appear in assignee's calendar if:
- Due date is specified
- Calendar integration is enabled
- User has calendar permissions
### Email Notifications
Sends notifications for:
- Task assignment
- Due date reminders
- Status changes
- Comments added
### Task Dependencies
Can link tasks together:
```basic
parent_task = CREATE_TASK "Project", "Main project", "pm@example.com", "next month", "high"
subtask = CREATE_TASK "Research", "Initial research", "analyst@example.com", "next week", "medium"
LINK_TASKS parent_task.task_id, subtask.task_id
```
## Priority Levels
| Priority | Description | SLA |
|----------|-------------|-----|
| `urgent` | Immediate attention required | 4 hours |
| `high` | Important, time-sensitive | 1 day |
| `medium` | Standard priority | 3 days |
| `low` | Non-urgent | 1 week |
## Date Formats
Supports multiple date formats:
- Absolute: `"2024-01-15"`, `"01/15/2024"`
- Relative: `"today"`, `"tomorrow"`, `"next week"`, `"in 3 days"`
- Natural: `"Monday"`, `"next Friday"`, `"end of month"`
## Error Handling
- Validates assignee exists in system
- Checks date is in the future
- Verifies priority is valid
- Returns error if task creation fails
- Handles permission issues gracefully
## Permissions
User must have one of:
- Task creation permission
- Project member status
- Admin privileges
- Delegation rights from assignee
## Best Practices
1. **Clear Titles**: Use descriptive, action-oriented titles
2. **Detailed Descriptions**: Include acceptance criteria
3. **Realistic Dates**: Set achievable deadlines
4. **Appropriate Priority**: Don't mark everything as urgent
5. **Valid Assignees**: Verify user can handle the task
6. **Follow Up**: Check task status periodically
## Advanced Usage
### Task Templates
```basic
template = GET_TASK_TEMPLATE("customer_onboarding")
CREATE_TASK template.title, template.description, assigned_user, due_date, template.priority
```
### Conditional Creation
```basic
IF urgency = "high" AND department = "support" THEN
CREATE_TASK "Urgent Support", issue_description, "support-lead@example.com", "today", "urgent"
ELSE
CREATE_TASK "Support Request", issue_description, "support@example.com", "tomorrow", "medium"
END IF
```
### Task with Attachments
```basic
task = CREATE_TASK "Review document", "Please review attached", reviewer, deadline, "high"
ATTACH_FILE task.task_id, "proposal.pdf"
```
## Related Keywords
- [BOOK](./keyword-book.md) - Schedule meetings instead of tasks
- [SET_SCHEDULE](./keyword-set-schedule.md) - Create recurring tasks
- [SEND_MAIL](./keyword-send-mail.md) - Send task notifications
- [ADD_MEMBER](./keyword-add-member.md) - Add users to task groups
## Database Tables
Tasks are stored in:
- `tasks` - Main task records
- `task_assignments` - User assignments
- `task_comments` - Task discussions
- `task_attachments` - Related files
- `task_history` - Status changes
## Implementation
Located in `src/basic/keywords/create_task.rs`
Integrates with:
- Task engine module for task management
- Calendar engine for scheduling
- Email module for notifications
- Storage module for attachments

259
docs/src/chapter-05/keyword-remember.md Normal file
View file

@ -0,0 +1,259 @@
# REMEMBER
Store information in bot's long-term memory for future conversations.
## Syntax
```basic
REMEMBER key, value
```
## Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `key` | String | Memory key/identifier |
| `value` | String | Information to remember |
## Description
The `REMEMBER` keyword stores information persistently across conversations. Unlike session variables that expire, remembered data persists:
- Across multiple conversations
- Between user sessions
- After bot restarts
- Indefinitely until explicitly forgotten
This enables bots to build user profiles, track preferences, and maintain context over time.
## Examples
### Store User Preferences
```basic
name = HEAR "What's your name?"
REMEMBER "user_name", name
TALK "Nice to meet you, " + name + ". I'll remember that!"
```
### Remember User Choices
```basic
language = HEAR "Preferred language? (English/Spanish/French)"
REMEMBER "preferred_language", language
TALK "I'll communicate in " + language + " from now on"
```
### Build User Profile
```basic
REMEMBER "signup_date", TODAY()
REMEMBER "user_tier", "premium"
REMEMBER "last_contact", NOW()
REMEMBER "interaction_count", interaction_count + 1
```
### Store Complex Information
```basic
' Store JSON-like data
preferences = "{theme: 'dark', notifications: true, timezone: 'EST'}"
REMEMBER "user_preferences", preferences
' Store lists
interests = "technology, science, sports"
REMEMBER "interests", interests
```
## Retrieval
Use `RECALL` to retrieve remembered information:
```basic
' In a future conversation
name = RECALL("user_name")
IF name != "" THEN
TALK "Welcome back, " + name + "!"
ELSE
TALK "Hello! I don't think we've met before."
END IF
```
## Memory Scope
Memories are scoped by:
- User ID (each user has separate memory)
- Bot ID (memories don't cross bots)
- Optional namespace (for organizing memories)
## Memory Management
### List All Memories
```basic
memories = GET_ALL_MEMORIES()
FOR EACH memory IN memories
PRINT memory.key + ": " + memory.value
NEXT
```
### Forget Specific Memory
```basic
FORGET "temporary_data"
```
### Clear All Memories
```basic
CLEAR_ALL_MEMORIES()
TALK "Memory wiped clean!"
```
### Memory with Expiration
```basic
' Remember for 30 days
REMEMBER_TEMP "trial_status", "active", 30
```
## Use Cases
### Customer Service
```basic
' Remember customer issues
issue = HEAR "What problem are you experiencing?"
REMEMBER "last_issue", issue
REMEMBER "issue_date", TODAY()
' In follow-up conversation
last_issue = RECALL("last_issue")
IF last_issue != "" THEN
TALK "Following up on your issue: " + last_issue
END IF
```
### Personal Assistant
```basic
' Remember important dates
birthday = HEAR "When is your birthday?"
REMEMBER "birthday", birthday
' Check on birthday
IF TODAY() = RECALL("birthday") THEN
TALK "Happy Birthday! 🎉"
END IF
```
### Learning System
```basic
' Track user corrections
correction = HEAR "Actually, that's not correct..."
REMEMBER "correction_" + topic, correction
corrections_count = RECALL("total_corrections") + 1
REMEMBER "total_corrections", corrections_count
```
### Preferences Tracking
```basic
' Remember communication preferences
IF time_of_day = "morning" AND response_time < 5 THEN
REMEMBER "active_time", "morning"
END IF
preferred_channel = GET_CHANNEL()
REMEMBER "preferred_channel", preferred_channel
```
## Performance Considerations
- Memories are indexed for fast retrieval
- Large values (>1MB) should be stored as files
- Frequently accessed memories are cached
- Memory operations are asynchronous
## Privacy and Security
### Data Protection
- Memories are encrypted at rest
- PII should be marked as sensitive
- Comply with data retention policies
- Support user data deletion requests
### GDPR Compliance
```basic
' Allow users to export their data
IF request = "export_my_data" THEN
data = EXPORT_USER_MEMORIES()
SEND_MAIL user_email, "Your Data", data
END IF
' Allow users to delete their data
IF request = "forget_me" THEN
DELETE_USER_MEMORIES()
TALK "Your data has been deleted"
END IF
```
## Best Practices
1. **Use descriptive keys**: Make memory keys self-documenting
2. **Validate before storing**: Check data quality
3. **Handle missing memories**: Always check if memory exists
4. **Organize with namespaces**: Group related memories
5. **Clean up old data**: Remove outdated memories
6. **Respect privacy**: Ask permission for sensitive data
7. **Document memories**: Keep track of what's stored
## Advanced Features
### Structured Memory
```basic
' Store structured data
user_profile = CREATE_MAP()
user_profile["name"] = name
user_profile["age"] = age
user_profile["interests"] = interests
REMEMBER "profile", JSON_STRINGIFY(user_profile)
' Retrieve and parse
profile_json = RECALL("profile")
profile = JSON_PARSE(profile_json)
```
### Memory Search
```basic
' Search memories by pattern
matching_memories = SEARCH_MEMORIES("pref_*")
FOR EACH mem IN matching_memories
PRINT mem.key + ": " + mem.value
NEXT
```
### Memory Analytics
```basic
' Track memory usage
stats = GET_MEMORY_STATS()
PRINT "Total memories: " + stats.count
PRINT "Storage used: " + stats.size_mb + "MB"
PRINT "Oldest memory: " + stats.oldest_date
```
## Error Handling
```basic
TRY
REMEMBER "important_data", data
CATCH "storage_full"
' Clean up old memories
DELETE_OLD_MEMORIES(30) ' Delete memories older than 30 days
RETRY
CATCH "invalid_data"
LOG "Cannot store invalid data"
END TRY
```
## Related Keywords
- [GET_BOT_MEMORY](./keyword-get-bot-memory.md) - Session-scoped memory
- [SET_BOT_MEMORY](./keyword-set-bot-memory.md) - Temporary memory
- [SET_USER](./keyword-set-user.md) - Set user context
- [SET_CONTEXT](./keyword-set-context.md) - Set conversation context
## Implementation
Located in `src/basic/keywords/remember.rs`
Uses persistent storage (PostgreSQL) with caching layer (Redis) for performance.

255
docs/src/chapter-05/keyword-save-from-unstructured.md Normal file
View file

@ -0,0 +1,255 @@
# SAVE_FROM_UNSTRUCTURED
Extract and save structured data from unstructured text content.
## Syntax
```basic
SAVE_FROM_UNSTRUCTURED text, schema, destination
```
## Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `text` | String | Unstructured text to process |
| `schema` | String | Schema name or definition for extraction |
| `destination` | String | Where to save extracted data (table, file, variable) |
## Description
The `SAVE_FROM_UNSTRUCTURED` keyword uses AI to extract structured information from free-form text like emails, documents, conversations, or web content. It:
- Identifies relevant data points
- Validates against schema
- Transforms to structured format
- Saves to specified destination
- Handles various text formats
## Examples
### Extract Contact Information
```basic
email_text = "Hi, I'm John Smith from Acme Corp. You can reach me at john@acme.com or 555-1234."
SAVE_FROM_UNSTRUCTURED email_text, "contact_schema", "contacts_table"
' Extracts: name, company, email, phone
```
### Process Invoice Data
```basic
invoice_text = GET_FILE_CONTENT("invoice.pdf")
SAVE_FROM_UNSTRUCTURED invoice_text, "invoice_schema", "invoices_db"
' Extracts: invoice_number, date, amount, items, tax
```
### Extract Meeting Notes
```basic
transcript = "Meeting on Jan 15. Present: Alice, Bob. Discussed Q1 targets of $1M. Action: Bob to prepare report by Friday."
SAVE_FROM_UNSTRUCTURED transcript, "meeting_schema", meeting_data
' Extracts: date, attendees, topics, actions, deadlines
```
### Parse Customer Feedback
```basic
review = HEAR "Please tell us about your experience"
SAVE_FROM_UNSTRUCTURED review, "feedback_schema", "reviews_table"
' Extracts: sentiment, rating, issues, suggestions
```
## Schema Definition
### Predefined Schemas
Common schemas available out-of-the-box:
- `contact_schema` - Name, email, phone, company
- `address_schema` - Street, city, state, zip, country
- `invoice_schema` - Number, date, items, amounts
- `meeting_schema` - Date, attendees, agenda, actions
- `feedback_schema` - Sentiment, topics, ratings
- `resume_schema` - Skills, experience, education
- `product_schema` - Name, price, features, specs
### Custom Schema
```basic
' Define custom extraction schema
schema = {
"fields": [
{"name": "customer_id", "type": "string", "required": true},
{"name": "issue_type", "type": "enum", "values": ["billing", "technical", "other"]},
{"name": "priority", "type": "enum", "values": ["low", "medium", "high"]},
{"name": "description", "type": "string"}
]
}
SAVE_FROM_UNSTRUCTURED support_ticket, schema, "tickets_table"
```
## Destinations
### Database Table
```basic
SAVE_FROM_UNSTRUCTURED text, "schema", "table_name"
' Saves to database table
```
### Variable
```basic
SAVE_FROM_UNSTRUCTURED text, "schema", extracted_data
' Saves to variable for further processing
```
### File
```basic
SAVE_FROM_UNSTRUCTURED text, "schema", "output.json"
' Saves as JSON file
```
### Multiple Destinations
```basic
' Extract once, save multiple places
data = EXTRACT_STRUCTURED(text, "schema")
SAVE_TO_DB data, "table"
SAVE_TO_FILE data, "backup.json"
SEND_TO_API data, "https://api.example.com/data"
```
## Return Value
Returns extraction result object:
- `success`: Boolean indicating success
- `extracted_count`: Number of records extracted
- `data`: Extracted structured data
- `confidence`: Extraction confidence score (0-1)
- `errors`: Any validation errors
## Advanced Features
### Batch Processing
```basic
documents = GET_ALL_FILES("inbox/*.txt")
FOR EACH doc IN documents
SAVE_FROM_UNSTRUCTURED doc.content, "contract_schema", "contracts_table"
NEXT
```
### Validation Rules
```basic
schema_with_rules = {
"fields": [...],
"validation": {
"email": "must be valid email",
"phone": "must be 10 digits",
"amount": "must be positive number"
}
}
SAVE_FROM_UNSTRUCTURED text, schema_with_rules, destination
```
### Confidence Threshold
```basic
result = SAVE_FROM_UNSTRUCTURED text, schema, destination, min_confidence=0.8
IF result.confidence < 0.8 THEN
' Manual review needed
SEND_FOR_REVIEW text, result
END IF
```
### Multi-Language Support
```basic
' Process text in different languages
spanish_text = "Nombre: Juan, Teléfono: 555-1234"
SAVE_FROM_UNSTRUCTURED spanish_text, "contact_schema", "contacts", language="es"
```
## Use Cases
### CRM Data Entry
```basic
' Auto-populate CRM from emails
email = GET_LATEST_EMAIL()
SAVE_FROM_UNSTRUCTURED email.body, "lead_schema", "crm_leads"
TALK "New lead added to CRM"
```
### Document Processing
```basic
' Process uploaded documents
document = UPLOAD_FILE()
text = EXTRACT_TEXT(document)
SAVE_FROM_UNSTRUCTURED text, detect_schema(document.type), "documents_db"
```
### Form Automation
```basic
' Convert free-text to form submission
user_input = HEAR "Describe your issue"
SAVE_FROM_UNSTRUCTURED user_input, "ticket_schema", "support_tickets"
ticket_id = GET_LAST_INSERT_ID()
TALK "Ticket #" + ticket_id + " created"
```
### Data Migration
```basic
' Migrate unstructured data to structured database
old_notes = GET_LEGACY_NOTES()
FOR EACH note IN old_notes
SAVE_FROM_UNSTRUCTURED note, "modern_schema", "new_database"
NEXT
```
## Error Handling
```basic
TRY
SAVE_FROM_UNSTRUCTURED text, schema, destination
CATCH "invalid_schema"
LOG "Schema validation failed"
CATCH "extraction_failed"
LOG "Could not extract required fields"
' Fall back to manual processing
CREATE_MANUAL_TASK text
CATCH "save_failed"
LOG "Database save failed"
' Save to backup location
SAVE_TO_FILE text, "failed_extractions.log"
END TRY
```
## Performance Considerations
- Large texts are processed in chunks
- Extraction is cached for identical inputs
- Schema validation happens before AI processing
- Batch operations are optimized
- Background processing for large datasets
## Best Practices
1. **Define clear schemas**: Be specific about expected fields
2. **Validate important data**: Add validation rules for critical fields
3. **Set confidence thresholds**: Require human review for low confidence
4. **Handle errors gracefully**: Have fallback procedures
5. **Test with samples**: Verify extraction accuracy
6. **Monitor performance**: Track extraction success rates
7. **Document schemas**: Keep schema documentation updated
8. **Version schemas**: Track schema changes over time
## Limitations
- Accuracy depends on text quality
- Complex nested structures may need preprocessing
- Very long texts may need chunking
- Ambiguous data requires human review
- Processing time increases with text length
## Related Keywords
- [GET](./keyword-get.md) - Fetch unstructured content
- [FIND](./keyword-find.md) - Search extracted data
- [FORMAT](./keyword-format.md) - Format extracted data
- [LLM](./keyword-llm.md) - Process with language model
## Implementation
Located in `src/basic/keywords/save_from_unstructured.rs`
Uses LLM for intelligent extraction with schema validation and multiple storage backends.

265
docs/src/chapter-05/keyword-send-mail.md Normal file
View file

@ -0,0 +1,265 @@
# SEND_MAIL
Send email messages from within bot conversations.
## Syntax
```basic
SEND_MAIL to, subject, body
```
or with attachments:
```basic
SEND_MAIL to, subject, body, attachments
```
## Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `to` | String/Array | Email recipient(s). Single email or array of emails |
| `subject` | String | Email subject line |
| `body` | String | Email body content (HTML supported) |
| `attachments` | Array | Optional. File paths or URLs to attach |
## Description
The `SEND_MAIL` keyword sends emails through the configured SMTP service. It supports:
- Single or multiple recipients
- HTML formatted messages
- File attachments
- CC and BCC recipients (via extended syntax)
- Email templates
- Delivery tracking
- Bounce handling
## Examples
### Simple Email
```basic
SEND_MAIL "user@example.com", "Welcome", "Thanks for signing up!"
```
### Multiple Recipients
```basic
recipients = ["alice@example.com", "bob@example.com"]
SEND_MAIL recipients, "Team Update", "Meeting tomorrow at 3pm"
```
### HTML Email
```basic
body = "<h1>Invoice</h1><p>Amount due: <b>$100</b></p>"
SEND_MAIL customer_email, "Invoice #123", body
```
### Email with Attachments
```basic
files = ["report.pdf", "data.xlsx"]
SEND_MAIL manager_email, "Monthly Report", "Please find attached reports", files
```
### Template-Based Email
```basic
' Load email template
template = LOAD_TEMPLATE("welcome_email")
body = FORMAT(template, customer_name, account_id)
SEND_MAIL customer_email, "Welcome to Our Service", body
```
### Conditional Email
```basic
IF order_total > 1000 THEN
subject = "Large Order Alert"
body = "New order over $1000: Order #" + order_id
SEND_MAIL "sales@example.com", subject, body
END IF
```
## Advanced Usage
### Email with CC and BCC
```basic
' Using extended parameters
email_data = CREATE_MAP()
email_data["to"] = "primary@example.com"
email_data["cc"] = ["manager@example.com", "team@example.com"]
email_data["bcc"] = "archive@example.com"
email_data["subject"] = "Project Status"
email_data["body"] = status_report
SEND_MAIL_EXTENDED email_data
```
### Bulk Email with Personalization
```basic
customers = GET_CUSTOMER_LIST()
FOR EACH customer IN customers
subject = "Special Offer for " + customer.name
body = FORMAT("Hi {name}, your discount code is {code}",
customer.name, customer.discount_code)
SEND_MAIL customer.email, subject, body
WAIT 1 ' Rate limiting
NEXT
```
### Email with Dynamic Attachments
```basic
' Generate report
report = GENERATE_REPORT(date_range)
report_file = SAVE_TEMP_FILE(report, "report.pdf")
' Send with attachment
SEND_MAIL recipient, "Daily Report", "See attached", [report_file]
' Cleanup
DELETE_TEMP_FILE(report_file)
```
## Configuration
Email settings in `config.csv`:
```csv
smtpHost,smtp.gmail.com
smtpPort,587
smtpUser,bot@example.com
smtpPassword,app-password
smtpFrom,Bot <noreply@example.com>
smtpUseTls,true
```
## Return Value
Returns an email status object:
- `message_id`: Unique message identifier
- `status`: "sent", "queued", "failed"
- `timestamp`: Send timestamp
- `error`: Error message if failed
## Error Handling
Common errors and solutions:
| Error | Cause | Solution |
|-------|-------|----------|
| Authentication failed | Invalid credentials | Check SMTP password |
| Connection timeout | Network issue | Verify SMTP host/port |
| Invalid recipient | Bad email format | Validate email addresses |
| Attachment not found | Missing file | Check file paths |
| Size limit exceeded | Large attachments | Compress or link files |
## Email Validation
Emails are validated before sending:
- Format check (RFC 5322)
- Domain verification
- Blacklist checking
- Bounce history
## Delivery Tracking
Track email delivery:
```basic
result = SEND_MAIL recipient, subject, body
IF result.status = "sent" THEN
LOG "Email sent: " + result.message_id
TRACK_DELIVERY result.message_id
ELSE
LOG "Email failed: " + result.error
RETRY_EMAIL result
END IF
```
## Templates
Use predefined email templates:
```basic
' Templates stored in mybot.gbai/templates/
template = LOAD_EMAIL_TEMPLATE("invoice")
body = FILL_TEMPLATE(template, invoice_data)
SEND_MAIL customer_email, "Invoice", body
```
## Security Considerations
1. **Never hardcode credentials** - Use environment variables
2. **Validate recipients** - Prevent email injection
3. **Sanitize content** - Escape HTML in user input
4. **Rate limit sends** - Prevent spam
5. **Use authentication** - Enable SMTP auth
6. **Encrypt connections** - Use TLS/SSL
7. **Monitor bounces** - Handle invalid addresses
8. **Implement unsubscribe** - Honor opt-outs
## Best Practices
1. **Use templates**: Maintain consistent formatting
2. **Test emails**: Send test emails before production
3. **Handle failures**: Implement retry logic
4. **Log sends**: Keep audit trail
5. **Personalize content**: Use recipient name
6. **Mobile-friendly**: Use responsive HTML
7. **Plain text fallback**: Include text version
8. **Unsubscribe link**: Always provide opt-out
9. **SPF/DKIM**: Configure domain authentication
10. **Monitor reputation**: Track delivery rates
## Rate Limiting
Prevent overwhelming SMTP server:
```basic
email_queue = []
FOR EACH email IN email_queue
SEND_MAIL email.to, email.subject, email.body
WAIT 2 ' 2 second delay between sends
NEXT
```
## Attachments
Supported attachment types:
- Documents: PDF, DOC, DOCX, TXT
- Spreadsheets: XLS, XLSX, CSV
- Images: PNG, JPG, GIF
- Archives: ZIP, RAR
- Any file under size limit
Size limits:
- Single file: 10MB default
- Total attachments: 25MB default
## Integration
Integrates with:
- SMTP servers (Gmail, Outlook, SendGrid)
- Email tracking services
- Template engines
- File storage (MinIO)
- Analytics systems
## Performance
- Async sending for non-blocking operation
- Queue management for bulk sends
- Connection pooling for efficiency
- Automatic retry on failure
## Monitoring
Track email metrics:
- Send rate
- Delivery rate
- Bounce rate
- Open rate (if tracking enabled)
- Click rate (for links)
## Implementation
Located in `src/basic/keywords/send_mail.rs`
Uses the email module for SMTP operations and supports multiple email providers.

273
docs/src/chapter-05/keyword-set-schedule.md
View file

@ -1 +1,274 @@
# SET_SCHEDULE # SET_SCHEDULE
Schedule recurring tasks or events to run at specified intervals.
## Syntax
```basic
SET_SCHEDULE schedule_name, cron_expression, task
```
## Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `schedule_name` | String | Unique identifier for the schedule |
| `cron_expression` | String | Cron expression or time interval |
| `task` | String | Task to execute (dialog name or command) |
## Description
The `SET_SCHEDULE` keyword creates recurring scheduled tasks that execute at specified times or intervals. It supports both cron expressions for complex scheduling and simple interval syntax for common patterns.
Schedules persist across bot restarts and run independently of user sessions, making them ideal for automated tasks, reminders, reports, and maintenance operations.
## Examples
### Daily Reminder
```basic
SET_SCHEDULE "daily_standup", "0 9 * * MON-FRI", "send_standup_reminder.bas"
TALK "Daily standup reminder scheduled for 9 AM on weekdays"
```
### Hourly Task
```basic
SET_SCHEDULE "check_inventory", "every 1 hour", "inventory_check.bas"
TALK "Inventory will be checked every hour"
```
### Weekly Report
```basic
SET_SCHEDULE "weekly_report", "0 10 * * MON", "generate_weekly_report.bas"
TALK "Weekly reports will be generated every Monday at 10 AM"
```
### Monthly Billing
```basic
SET_SCHEDULE "monthly_billing", "0 0 1 * *", "process_billing.bas"
TALK "Billing will process on the 1st of each month"
```
## Schedule Formats
### Cron Expression Format
Standard cron format with five fields:
```
* * * * *
│ │ │ │ │
│ │ │ │ └─── Day of week (0-7, MON-SUN)
│ │ │ └───── Month (1-12)
│ │ └─────── Day of month (1-31)
│ └───────── Hour (0-23)
└─────────── Minute (0-59)
```
Examples:
- `"0 9 * * *"` - Every day at 9:00 AM
- `"*/15 * * * *"` - Every 15 minutes
- `"0 0 * * SUN"` - Every Sunday at midnight
- `"0 8,12,16 * * *"` - At 8 AM, noon, and 4 PM daily
### Simple Interval Format
Human-readable intervals:
- `"every 30 minutes"`
- `"every 2 hours"`
- `"every day"`
- `"every week"`
- `"every month"`
### Relative Time Format
- `"in 5 minutes"` - One-time schedule
- `"tomorrow at 3pm"`
- `"next Monday"`
- `"end of month"`
## Task Types
### Dialog Execution
```basic
SET_SCHEDULE "greeting", "0 8 * * *", "morning_greeting.bas"
```
### Function Call
```basic
SET_SCHEDULE "backup", "0 2 * * *", "BACKUP_DATABASE()"
```
### Email Task
```basic
SET_SCHEDULE "reminder", "0 9 * * MON", "SEND_MAIL admin@example.com, 'Weekly Tasks', 'Please review weekly tasks'"
```
### Multi-Step Task
```basic
task = "FETCH_DATA(); PROCESS_DATA(); SEND_REPORT()"
SET_SCHEDULE "data_pipeline", "*/30 * * * *", task
```
## Schedule Management
### List Active Schedules
```basic
schedules = GET_SCHEDULES()
FOR EACH schedule IN schedules
TALK schedule.name + ": " + schedule.expression + " - " + schedule.task
NEXT
```
### Cancel Schedule
```basic
CANCEL_SCHEDULE "daily_reminder"
TALK "Daily reminder has been cancelled"
```
### Update Schedule
```basic
' Cancel old and create new
CANCEL_SCHEDULE "report"
SET_SCHEDULE "report", "0 10 * * *", "new_report.bas"
```
### Check Schedule Status
```basic
status = GET_SCHEDULE_STATUS("backup")
TALK "Last run: " + status.last_run
TALK "Next run: " + status.next_run
```
## Advanced Features
### Conditional Scheduling
```basic
IF is_production THEN
SET_SCHEDULE "prod_backup", "0 1 * * *", "backup_prod.bas"
ELSE
SET_SCHEDULE "dev_backup", "0 3 * * SUN", "backup_dev.bas"
END IF
```
### Dynamic Scheduling
```basic
hour = USER_PREFERENCE("reminder_hour")
cron = "0 " + hour + " * * *"
SET_SCHEDULE "user_reminder", cron, "send_reminder.bas"
```
### Schedule with Parameters
```basic
' Pass parameters to scheduled task
task_with_params = "PROCESS_REPORT('daily', 'sales', true)"
SET_SCHEDULE "daily_sales", "0 6 * * *", task_with_params
```
### Error Handling in Schedules
```basic
task = "TRY; RUN_TASK(); CATCH; LOG_ERROR(); END TRY"
SET_SCHEDULE "safe_task", "*/10 * * * *", task
```
## Timezone Handling
Schedules use the bot's configured timezone:
```basic
' Set timezone in config.csv
' timezone = "America/New_York"
SET_SCHEDULE "ny_task", "0 9 * * *", "task.bas"
' Runs at 9 AM New York time
```
## Return Value
Returns schedule object:
- `id`: Unique schedule identifier
- `name`: Schedule name
- `created`: Creation timestamp
- `next_run`: Next execution time
- `active`: Schedule status
## Persistence
- Schedules stored in database
- Survive bot restarts
- Automatic recovery after crashes
- Execution history tracked
## Best Practices
1. **Use descriptive names**: Make schedule purpose clear
2. **Test cron expressions**: Verify timing is correct
3. **Handle failures**: Include error handling in tasks
4. **Avoid overlaps**: Ensure tasks complete before next run
5. **Monitor execution**: Check logs for failures
6. **Clean up old schedules**: Remove unused schedules
7. **Document schedules**: Keep list of active schedules
## Common Use Cases
### Daily Reports
```basic
SET_SCHEDULE "daily_summary", "0 18 * * *", "generate_daily_summary.bas"
```
### Data Synchronization
```basic
SET_SCHEDULE "sync_data", "*/15 * * * *", "sync_with_external_api.bas"
```
### Maintenance Tasks
```basic
SET_SCHEDULE "cleanup", "0 3 * * *", "cleanup_old_files.bas"
SET_SCHEDULE "optimize", "0 4 * * SUN", "optimize_database.bas"
```
### User Reminders
```basic
SET_SCHEDULE "medication", "0 8,14,20 * * *", "send_medication_reminder.bas"
```
### Business Processes
```basic
SET_SCHEDULE "payroll", "0 10 L * *", "process_payroll.bas" ' Last day of month
SET_SCHEDULE "invoicing", "0 9 1 * *", "generate_invoices.bas" ' First of month
```
## Limitations
- Maximum 100 active schedules per bot
- Minimum interval: 1 minute
- Tasks timeout after 5 minutes
- No sub-second precision
- Schedules run in bot context (not user)
## Troubleshooting
### Schedule Not Running
- Verify cron expression syntax
- Check bot is active
- Review task for errors
- Check timezone settings
### Task Failures
- Check task dialog exists
- Verify permissions
- Review error logs
- Test task manually first
### Performance Issues
- Limit concurrent schedules
- Optimize task execution
- Use appropriate intervals
- Monitor resource usage
## Related Keywords
- [CREATE_TASK](./keyword-create-task.md) - Create one-time tasks
- [WAIT](./keyword-wait.md) - Delay execution
- [ON](./keyword-on.md) - Event-based triggers
- [SEND_MAIL](./keyword-send-mail.md) - Often used in scheduled tasks
## Implementation
Located in `src/basic/keywords/set_schedule.rs`
Uses cron parser for expression validation and tokio scheduler for execution management.

258
docs/src/chapter-05/keyword-set.md
View file

@ -1 +1,259 @@
# SET # SET
Assign values to variables in BASIC dialogs.
## Syntax
```basic
SET variable = value
```
or simply:
```basic
variable = value
```
## Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `variable` | Identifier | Variable name to assign to |
| `value` | Any | Value to assign (string, number, boolean, array, object) |
## Description
The `SET` keyword assigns values to variables within BASIC dialog scripts. Variables are dynamically typed and can hold any type of value. The `SET` keyword is optional - you can use direct assignment with `=`.
Variables are scoped to the current dialog execution and persist throughout the conversation session until explicitly changed or the session ends.
## Examples
### Basic Assignment
```basic
SET name = "John Doe"
SET age = 25
SET is_premium = true
SET score = 98.5
```
### Direct Assignment (without SET)
```basic
name = "Jane Smith"
count = 0
message = "Welcome!"
```
### Array Assignment
```basic
SET colors = ["red", "green", "blue"]
SET numbers = [1, 2, 3, 4, 5]
SET mixed = ["text", 123, true]
```
### Object/Map Assignment
```basic
SET user = {
"name": "Alice",
"email": "alice@example.com",
"age": 30,
"active": true
}
```
### Dynamic Values
```basic
SET current_time = NOW()
SET user_input = HEAR "What's your name?"
SET calculated = price * quantity * tax_rate
SET formatted = FORMAT("Hello, {0}!", username)
```
## Variable Types
BASIC supports these variable types:
- **String**: Text values
- **Number**: Integers and decimals
- **Boolean**: true/false
- **Array**: Ordered lists
- **Object**: Key-value maps
- **Null**: Empty/undefined
## Variable Naming
Valid variable names:
- Start with letter or underscore
- Contain letters, numbers, underscores
- Case-sensitive
- No reserved keywords
Examples:
```basic
SET userName = "John"
SET user_name = "John"
SET _private = true
SET value123 = 456
SET firstName = "Jane"
```
Invalid names:
```basic
' These will cause errors
SET 123name = "error" ' Starts with number
SET user-name = "error" ' Contains hyphen
SET if = "error" ' Reserved keyword
```
## Variable Scope
### Session Variables
Regular variables exist for the session:
```basic
SET session_data = "persists during conversation"
```
### Global Variables
Use special prefixes for broader scope:
```basic
SET $global_var = "accessible across dialogs"
SET @bot_var = "bot-level variable"
```
### Temporary Variables
```basic
SET _temp = "temporary use"
' Prefix with underscore for temporary/internal use
```
## Type Conversion
Variables automatically convert types when needed:
```basic
SET text = "123"
SET number = text + 0 ' Converts to number: 123
SET back_to_text = number + "" ' Converts to string: "123"
SET boolean = number > 100 ' Converts to boolean: true
```
## Operations on Variables
### String Operations
```basic
SET full_name = first_name + " " + last_name
SET uppercase = UPPER(name)
SET length = LEN(message)
SET substring = MID(text, 1, 5)
```
### Numeric Operations
```basic
SET sum = a + b
SET difference = a - b
SET product = a * b
SET quotient = a / b
SET remainder = a MOD b
SET power = a ^ b
```
### Array Operations
```basic
SET first = colors[0]
SET last = colors[LEN(colors) - 1]
colors[1] = "yellow" ' Modify array element
SET combined = array1 + array2 ' Concatenate
```
### Object/Map Operations
```basic
SET email = user["email"]
SET age = user.age
user["status"] = "active"
user.last_login = NOW()
```
## Conditional Assignment
```basic
SET status = IF(score >= 70, "pass", "fail")
SET discount = IF(is_member, 0.2, 0.1)
SET greeting = IF(hour < 12, "Good morning", "Good afternoon")
```
## Common Patterns
### Counter Variables
```basic
SET counter = 0
FOR i = 1 TO 10
counter = counter + 1
NEXT
```
### Flag Variables
```basic
SET is_complete = false
' ... process ...
SET is_complete = true
```
### Accumulator Variables
```basic
SET total = 0
FOR EACH item IN cart
total = total + item.price
NEXT
```
### State Variables
```basic
SET state = "initial"
' ... logic ...
SET state = "processing"
' ... more logic ...
SET state = "complete"
```
## Best Practices
1. **Use descriptive names**: `customer_email` instead of `e`
2. **Initialize variables**: Set initial values before use
3. **Use consistent naming**: camelCase or snake_case
4. **Avoid global pollution**: Use local variables when possible
5. **Clean up large variables**: Set to null when done
6. **Document complex variables**: Add comments
7. **Validate before use**: Check if variable exists
## Error Handling
```basic
' Check if variable exists
IF EXISTS(user_data) THEN
SET name = user_data.name
ELSE
SET name = "Guest"
END IF
' Safe assignment with default
SET value = GET_VALUE_OR_DEFAULT(config.setting, "default")
```
## Memory Management
```basic
' Clear large variables when done
SET big_data = LOAD_FILE("large.json")
' ... use big_data ...
SET big_data = null ' Free memory
```
## Related Keywords
- [GET](./keyword-get.md) - Retrieve data from external sources
- [HEAR](./keyword-hear.md) - Get user input into variable
- [FORMAT](./keyword-format.md) - Format values for assignment
- [SET_BOT_MEMORY](./keyword-set-bot-memory.md) - Persistent storage
- [SET_USER](./keyword-set-user.md) - Set user context
## Implementation Notes
Variables are stored in the BASIC engine's scope map and persist for the duration of the dialog execution. The `SET` keyword is syntactic sugar - the parser treats both `SET x = y` and `x = y` identically.

264
docs/src/chapter-05/keyword-weather.md Normal file
View file

@ -0,0 +1,264 @@
# WEATHER
Get current weather information for any location.
## Syntax
```basic
WEATHER location
```
## Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `location` | String | City name, coordinates, or zip code |
## Description
The `WEATHER` keyword retrieves real-time weather information from weather services. It provides:
- Current temperature
- Weather conditions
- Humidity and pressure
- Wind speed and direction
- Feels-like temperature
- Forecast data
## Examples
### Basic Weather Query
```basic
weather = WEATHER "New York"
TALK weather
```
### Weather with User Location
```basic
city = HEAR "What city are you in?"
current_weather = WEATHER city
TALK "The weather in " + city + " is: " + current_weather
```
### Conditional Weather Responses
```basic
weather_data = WEATHER "London"
temp = EXTRACT_TEMP(weather_data)
IF temp < 10 THEN
TALK "It's cold in London (" + temp + "°C). Bundle up!"
ELSE IF temp > 25 THEN
TALK "It's warm in London (" + temp + "°C). Perfect for outdoor activities!"
ELSE
TALK "Mild weather in London: " + temp + "°C"
END IF
```
### Multiple Location Weather
```basic
cities = ["Tokyo", "Paris", "Sydney", "Cairo"]
FOR EACH city IN cities
weather = WEATHER city
TALK city + ": " + weather
WAIT 1
NEXT
```
## Return Format
The keyword returns a formatted string with weather details:
```
"New York: 72°F (22°C), Partly Cloudy, Humidity: 65%, Wind: 10 mph NW"
```
For programmatic access, use the extended version:
```basic
data = WEATHER_DATA "San Francisco"
' Returns object with properties:
' data.temperature
' data.condition
' data.humidity
' data.wind_speed
' data.wind_direction
' data.pressure
' data.feels_like
```
## Location Formats
Supported location formats:
- City name: `"Paris"`
- City, Country: `"Paris, France"`
- Coordinates: `"48.8566, 2.3522"`
- Zip/Postal code: `"10001"` (US), `"SW1A 1AA"` (UK)
- Airport code: `"LAX"`
## Configuration
Configure weather service in `config.csv`:
```csv
weatherApiKey,your-api-key
weatherProvider,openweather
weatherUnits,metric
weatherLanguage,en
```
Supported providers:
- OpenWeather
- WeatherAPI
- AccuWeather
- DarkSky (legacy)
## Error Handling
```basic
TRY
weather = WEATHER location
TALK weather
CATCH "location_not_found"
TALK "I couldn't find weather for " + location
CATCH "api_error"
TALK "Weather service is temporarily unavailable"
CATCH "rate_limit"
TALK "Too many weather requests. Please try again later."
END TRY
```
## Advanced Usage
### Weather Alerts
```basic
alerts = WEATHER_ALERTS "Miami"
IF alerts != "" THEN
TALK "⚠️ Weather Alert: " + alerts
END IF
```
### Forecast
```basic
' Get 5-day forecast
forecast = WEATHER_FORECAST "Seattle", 5
FOR EACH day IN forecast
TALK day.date + ": " + day.high + "/" + day.low + " - " + day.condition
NEXT
```
### Historical Weather
```basic
' Get weather for specific date
historical = WEATHER_HISTORY "Chicago", "2024-01-15"
TALK "Weather on Jan 15: " + historical
```
### Weather Comparison
```basic
city1_weather = WEATHER_DATA "Los Angeles"
city2_weather = WEATHER_DATA "New York"
temp_diff = city1_weather.temperature - city2_weather.temperature
TALK "LA is " + ABS(temp_diff) + " degrees " +
IF(temp_diff > 0, "warmer", "cooler") + " than NYC"
```
## Caching
Weather data is cached to reduce API calls:
- Current weather: 10 minutes
- Forecast: 1 hour
- Historical: 24 hours
Force refresh with:
```basic
fresh_weather = WEATHER location, refresh=true
```
## Localization
Weather descriptions are localized based on user language:
```basic
SET_LANGUAGE "es"
weather = WEATHER "Madrid"
' Returns: "Madrid: 25°C, Parcialmente nublado..."
```
## Units
Temperature units based on configuration or override:
```basic
' Force Fahrenheit
weather_f = WEATHER "Toronto", units="imperial"
' Force Celsius
weather_c = WEATHER "Toronto", units="metric"
```
## Integration Examples
### Travel Assistant
```basic
destination = HEAR "Where are you traveling to?"
weather = WEATHER destination
TALK "Current weather at " + destination + ": " + weather
IF weather CONTAINS "rain" THEN
TALK "Don't forget an umbrella!"
ELSE IF weather CONTAINS "snow" THEN
TALK "Pack warm clothes and boots!"
END IF
```
### Event Planning
```basic
event_date = HEAR "When is your event?"
location = HEAR "Where will it be held?"
forecast = WEATHER_FORECAST location, event_date
IF forecast CONTAINS "rain" OR forecast CONTAINS "storm" THEN
TALK "Consider an indoor venue - rain is expected"
ELSE
TALK "Weather looks good for an outdoor event!"
END IF
```
### Agriculture Bot
```basic
farm_location = "Iowa"
weather_data = WEATHER_DATA farm_location
IF weather_data.temperature < 32 THEN
TALK "Frost warning! Protect sensitive crops"
END IF
IF weather_data.humidity < 30 THEN
TALK "Low humidity - increase irrigation"
END IF
```
## Performance Tips
1. **Cache responses**: Avoid repeated API calls
2. **Batch requests**: Get multiple locations at once
3. **Use coordinates**: More accurate than city names
4. **Handle timeouts**: Set reasonable timeout values
5. **Rate limit**: Respect API limits
## Limitations
- Requires valid API key
- Subject to rate limits
- Accuracy depends on provider
- Some locations may not be available
- Historical data may be limited
## Related Keywords
- [GET](./keyword-get.md) - Fetch data from URLs
- [SET_SCHEDULE](./keyword-set-schedule.md) - Schedule weather updates
- [FORMAT](./keyword-format.md) - Format weather display
## Implementation
Located in `src/basic/keywords/weather.rs`
Integrates with multiple weather API providers and includes fallback mechanisms.

20
docs/src/chapter-05/keywords.md
View file

@ -18,12 +18,10 @@ The source code for each keyword lives in `src/basic/keywords/`. Only the keywor
- [LLM](./keyword-llm.md) - [LLM](./keyword-llm.md)
- [GET_BOT_MEMORY](./keyword-get-bot-memory.md) - [GET_BOT_MEMORY](./keyword-get-bot-memory.md)
- [SET_BOT_MEMORY](./keyword-set-bot-memory.md) - [SET_BOT_MEMORY](./keyword-set-bot-memory.md)
- [SET_KB](./keyword-set-kb.md) - [USE_KB](./keyword-use-kb.md)
- [USE_KB](./keyword-add-kb.md) - [CLEAR_KB](./keyword-clear-kb.md)
- [ADD_WEBSITE](./keyword-add-website.md) - [ADD_WEBSITE](./keyword-add-website.md)
- [USE_TOOL](./keyword-add-tool.md) - [USE_TOOL](./keyword-use-tool.md)
- [LIST_TOOLS](./keyword-list-tools.md)
- [REMOVE_TOOL](./keyword-remove-tool.md)
- [CLEAR_TOOLS](./keyword-clear-tools.md) - [CLEAR_TOOLS](./keyword-clear-tools.md)
- [GET](./keyword-get.md) - [GET](./keyword-get.md)
- [FIND](./keyword-find.md) - [FIND](./keyword-find.md)
@ -32,11 +30,19 @@ The source code for each keyword lives in `src/basic/keywords/`. Only the keywor
- [SET_SCHEDULE](./keyword-set-schedule.md) - [SET_SCHEDULE](./keyword-set-schedule.md)
- [CREATE_SITE](./keyword-create-site.md) - [CREATE_SITE](./keyword-create-site.md)
- [CREATE_DRAFT](./keyword-create-draft.md) - [CREATE_DRAFT](./keyword-create-draft.md)
- [WEBSITE_OF](./keyword-website-of.md)
- [PRINT](./keyword-print.md) - [PRINT](./keyword-print.md)
- [WAIT](./keyword-wait.md) - [WAIT](./keyword-wait.md)
- [FORMAT](./keyword-format.md) - [FORMAT](./keyword-format.md)
- [FIRST](./keyword-first.md) - [FIRST](./keyword-first.md)
- [LAST](./keyword-last.md) - [LAST](./keyword-last.md)
- [FOR EACH](./keyword-for-each.md) - [FOR EACH ... NEXT](./keyword-for-each.md)
- [EXIT FOR](./keyword-exit-for.md) - [EXIT FOR](./keyword-exit-for.md)
- [ADD_MEMBER](./keyword-add-member.md)
- [ADD_SUGGESTION](./keyword-add-suggestion.md)
- [CLEAR_SUGGESTIONS](./keyword-clear-suggestions.md)
- [BOOK](./keyword-book.md)
- [CREATE_TASK](./keyword-create-task.md)
- [REMEMBER](./keyword-remember.md)
- [SAVE_FROM_UNSTRUCTURED](./keyword-save-from-unstructured.md)
- [SEND_MAIL](./keyword-send-mail.md)
- [WEATHER](./keyword-weather.md)

497
docs/src/chapter-06/architecture.md
View file

@ -1,69 +1,472 @@
# Architecture Overview # Architecture Overview
## Auto Bootstrap Process BotServer follows a modular architecture designed for scalability, maintainability, and extensibility. Each module handles specific responsibilities and communicates through well-defined interfaces.
The Auto Bootstrap process is responsible for initializing and configuring the entire BotServer environment after installation. It ensures that all system components are installed, configured, and started automatically, and that bots are created from predefined templates. ## Core Architecture
### 1. Bootstrap Initialization ```
BotServer
├── Core Engine
│ ├── BASIC Interpreter
│ ├── Session Manager
│ ├── Context Manager
│ └── Bootstrap System
├── AI & NLP Layer
│ ├── LLM Integration
│ ├── Prompt Management
│ ├── Knowledge Base
│ └── Vector Search
├── Communication Layer
│ ├── Multi-Channel Support
│ ├── WebSocket Server
│ ├── REST API
│ └── Event Bus
├── Storage Layer
│ ├── PostgreSQL
│ ├── MinIO/S3
│ ├── Redis Cache
│ └── Qdrant Vector DB
└── Services Layer
├── Authentication
├── Email Service
├── Calendar Engine
└── Task Engine
```
The process begins with the `BootstrapManager`, which is instantiated with an installation mode (`Local` or `Container`) and an optional tenant name. It initializes the `PackageManager`, which detects the operating system and sets up the base installation path (e.g., `/opt/gbo` or `botserver-stack`). ## Module Organization
### 2. Component Registration and Installation ### Core Modules
The `PackageManager` registers all system components such as: #### `auth/`
Authentication and authorization system handling:
- User management
- Group/organization management
- Role-based access control (RBAC)
- JWT token management
- OAuth integration
- Two-factor authentication
- **tables** (PostgreSQL database) #### `automation/`
- **cache** (Valkey/Redis) Workflow automation engine providing:
- **drive** (MinIO object storage) - Process automation
- **llm** (local LLM server) - Scheduled tasks
- **email**, **proxy**, **directory**, **alm**, **dns**, **meeting**, **table_editor**, **doc_editor**, **desktop**, **devtools**, **bot**, **system**, **vector_db**, **host** - Event-driven automation
- Workflow orchestration
- Integration with external systems
Each component has a `ComponentConfig` defining: #### `basic/`
- Ports and dependencies BASIC dialect interpreter and runtime:
- Download URLs and binaries - Keyword implementation
- Pre/post-install commands - Script compilation
- Variable management
- Flow control
- Tool integration
- Error handling
#### `bootstrap/`
System initialization and startup:
- Component verification
- Service startup sequencing
- Database migrations
- Template deployment
- Health checks
- Configuration loading
#### `bot/`
Bot instance management:
- Bot lifecycle (create, mount, unmount)
- Conversation handling
- User input processing
- Response generation
- Multi-bot coordination
- Session isolation
### Communication Modules
#### `channels/`
Multi-channel messaging adapters:
- Web interface
- WhatsApp Business API
- Microsoft Teams
- Slack
- Instagram
- SMS
- Voice
#### `meet/`
Real-time communication features:
- Video conferencing
- Voice calls
- Screen sharing
- Recording
- Transcription
- Meeting scheduling
#### `web_server/`
HTTP server and web interface:
- Static file serving
- WebSocket handling
- REST API routing
- CORS management
- Request/response processing
### AI & Knowledge Modules
#### `llm/`
Large Language Model integration:
- Model selection
- Prompt formatting
- Token management
- Response streaming
- Cost tracking
- Model fallbacks
#### `llm_models/`
Specific model implementations:
- OpenAI (GPT-3.5, GPT-4)
- Anthropic (Claude)
- Google (Gemini)
- Meta (Llama)
- Local models
- Custom models
#### `prompt_manager/`
Centralized prompt management:
- Prompt templates
- Variable substitution
- Model-specific optimization
- Version control
- A/B testing
- Performance tracking
#### `context/`
Conversation context management:
- Context window optimization
- History management
- Context compression
- Relevance filtering
- Multi-turn tracking
### Storage & Data Modules
#### `drive/`
File and document management:
- File upload/download
- Document processing
- Version control
- Sharing permissions
- Quota management
- Search indexing
#### `drive_monitor/`
Storage monitoring and sync:
- Change detection
- Auto-sync
- Conflict resolution
- Backup management
- Storage analytics
#### `package_manager/`
Bot package management:
- Package loading
- Dependency resolution
- Version management
- Hot reload
- Package validation
### Processing Modules
#### `engines/`
Various processing engines:
- Rule engine
- Workflow engine
- Event processor
- Message queue
- Job scheduler
#### `calendar_engine/`
Calendar and scheduling:
- Event management
- Availability checking
- Meeting coordination
- Reminder system
- Timezone handling
#### `task_engine/`
Task management system:
- Task creation
- Assignment
- Status tracking
- Dependencies
- Notifications
#### `email/`
Email integration:
- SMTP sending
- IMAP receiving
- Template management
- Tracking
- Bounce handling
### Utility Modules
#### `session/`
User session management:
- Session creation
- State persistence
- Session timeout
- Concurrent sessions
- Session recovery
#### `config/`
Configuration management:
- Config loading
- Environment variables - Environment variables
- Execution commands - Hot reload
- Validation
- Defaults
During bootstrap, required components (`tables`, `drive`, `cache`) are installed and started automatically. #### `shared/`
For example: Shared utilities and models:
- The **tables** component generates secure database credentials, writes them to `.env`, and applies SQL migrations to initialize the schema. - Database models
- The **drive** component creates secure credentials and stores them encrypted in the database. - Common types
- Helper functions
- Constants
- Error types
### 3. Bot Configuration #### `compliance/`
Regulatory compliance:
- GDPR compliance
- Data retention
- Audit logging
- Privacy controls
- Consent management
After components are installed, the bootstrap process updates the bot configuration in the database. #### `nvidia/`
The method `update_bot_config()` ensures each component’s configuration is linked to a bot record in the `bot_configuration` table. GPU acceleration support:
If no bot exists, a new UUID is generated to associate configuration entries. - CUDA integration
- Model inference
- Batch processing
- Performance optimization
### 4. Template-Based Bot Creation #### `ui_tree/`
UI component tree management:
- Virtual DOM
- Component lifecycle
- State management
- Event handling
- Rendering optimization
The method `create_bots_from_templates()` scans the `templates/` directory for folders ending in `.gbai` (e.g., `default.gbai`, `announcements.gbai`). #### `web_automation/`
Each `.gbai` folder represents a bot template. Web scraping and automation:
- Browser automation
- Content extraction
- Form filling
- Screenshot capture
- Change monitoring
For each template: ## Data Flow
- The folder name is converted into a human-readable bot name (e.g., `default.gbai` → “Default”).
- If the bot doesn’t exist in the `bots` table, a new record is inserted with:
- Default LLM provider (`openai`)
- Default configuration (`{"model": "gpt-4", "temperature": 0.7}`)
- Context provider (`database`)
- Active status (`true`)
This automatically creates bots from templates during bootstrap. ### Request Processing Pipeline
### 5. Template Upload to MinIO 1. **Channel Adapter** receives user input
2. **Session Manager** identifies/creates session
3. **Context Manager** loads conversation history
4. **BASIC Interpreter** executes dialog script
5. **LLM Integration** processes natural language
6. **Knowledge Base** provides relevant information
7. **Response Generator** formats output
8. **Channel Adapter** delivers response
After bots are created, the method `upload_templates_to_minio()` uploads all template files recursively to a MinIO bucket (S3-compatible storage). ### Storage Architecture
This makes templates accessible for runtime bot operations and ensures persistence across environments.
### 6. Summary #### Primary Database (PostgreSQL)
- User accounts
- Bot configurations
- Session data
- Conversation history
- System metadata
The Auto Bootstrap process performs the following steps automatically: #### Object Storage (MinIO/S3)
1. Detects environment and installation mode. - File uploads
2. Registers and installs required components. - Document storage
3. Initializes the database and applies migrations. - Media files
4. Updates bot configuration records. - Backups
5. Creates bots from `.gbai` templates. - Logs
6. Uploads templates to MinIO for storage.
This process ensures that after installation, the system is fully operational with preconfigured bots derived from templates, ready to serve requests immediately. #### Cache Layer (Redis)
- Session cache
- Frequently accessed data
- Rate limiting
- Temporary storage
- Pub/sub messaging
#### Vector Database (Qdrant)
- Document embeddings
- Semantic search index
- Knowledge base vectors
- Similarity matching
## Security Architecture
### Authentication Flow
1. User provides credentials
2. Auth module validates
3. JWT token issued
4. Token verified on each request
5. Session established
6. Permissions checked
### Data Protection
- Encryption at rest (database, files)
- Encryption in transit (TLS/SSL)
- Sensitive data masking
- PII detection
- Secure key management
### Access Control
- Role-based permissions
- Resource-level authorization
- API rate limiting
- IP allowlisting
- Audit logging
## Deployment Architecture
### Container Structure
- Main application container
- PostgreSQL database
- MinIO storage
- Redis cache
- Qdrant vector DB
- Nginx reverse proxy
### Scaling Strategy
- Horizontal scaling for web servers
- Read replicas for database
- Distributed cache
- Load balancing
- Auto-scaling policies
### High Availability
- Multi-zone deployment
- Database replication
- Storage redundancy
- Health monitoring
- Automatic failover
## Performance Optimization
### Caching Strategy
- Response caching
- Query result caching
- Static asset caching
- API response caching
- Knowledge base caching
### Async Processing
- Background jobs
- Message queues
- Event-driven architecture
- Non-blocking I/O
- Worker pools
### Resource Management
- Connection pooling
- Memory management
- Token optimization
- Query optimization
- Lazy loading
## Monitoring & Observability
### Metrics Collection
- System metrics
- Application metrics
- Business metrics
- User analytics
- Performance tracking
### Logging
- Structured logging
- Log aggregation
- Error tracking
- Audit trails
- Debug logging
### Health Checks
- Liveness probes
- Readiness probes
- Dependency checks
- Performance monitoring
- Alert system
## Extension Points
### Plugin System
- Custom keywords
- External tools
- API integrations
- Custom channels
- Model providers
### Webhook Support
- Incoming webhooks
- Outgoing webhooks
- Event subscriptions
- Callback handling
- Retry mechanisms
### API Integration
- REST API
- GraphQL (planned)
- WebSocket
- gRPC (planned)
- OpenAPI spec
## Development Workflow
### Local Development
1. Clone repository
2. Install dependencies
3. Configure environment
4. Run migrations
5. Start services
6. Load templates
### Testing Strategy
- Unit tests
- Integration tests
- End-to-end tests
- Load testing
- Security testing
### CI/CD Pipeline
- Automated testing
- Code quality checks
- Security scanning
- Build process
- Deployment automation
## Future Architecture Plans
### Planned Enhancements
- Microservices migration
- Kubernetes native
- Multi-region support
- Edge deployment
- Serverless functions
### Performance Goals
- Sub-100ms response time
- 10,000+ concurrent users
- 99.99% uptime
- Elastic scaling
- Global CDN
### Feature Roadmap
- GraphQL API
- Real-time collaboration
- Advanced analytics
- Machine learning pipeline
- Blockchain integration

266
docs/src/chapter-06/prompt-manager.md Normal file
View file

@ -0,0 +1,266 @@
# Prompt Manager
The Prompt Manager module provides centralized management of LLM prompts, templates, and system instructions used throughout BotServer.
## Overview
Located in `src/prompt_manager/`, this module maintains a library of reusable prompts that can be:
- Versioned and updated without code changes
- Customized per bot instance
- Composed dynamically based on context
- Optimized for different LLM models
## Architecture
```
src/prompt_manager/
├── mod.rs # Main module interface
├── prompts.csv # Default prompt library
└── templates/ # Complex prompt templates
├── system.md # System instructions
├── tools.md # Tool-use prompts
└── context.md # Context formatting
```
## Prompt Library Format
The `prompts.csv` file stores prompts in a structured format:
```csv
id,category,name,content,model,version
1,system,default,"You are a helpful assistant...",gpt-4,1.0
2,tools,function_call,"To use a tool, follow this format...",any,1.0
3,context,kb_search,"Search the knowledge base for: {query}",any,1.0
```
### Fields
| Field | Description |
|-------|-------------|
| `id` | Unique identifier |
| `category` | Prompt category (system, tools, context, etc.) |
| `name` | Prompt name for retrieval |
| `content` | The actual prompt text with placeholders |
| `model` | Target model or "any" for universal |
| `version` | Version for tracking changes |
## Usage in BASIC
Prompts are automatically loaded and can be referenced in dialogs:
```basic
' Use a predefined prompt
response = LLM "Use prompt: customer_service"
' Override with custom prompt
response = LLM "You are a technical support agent. Help with: " + issue
```
## Rust API
### Loading Prompts
```rust
use crate::prompt_manager::PromptManager;
let manager = PromptManager::new();
manager.load_from_csv("prompts.csv")?;
```
### Retrieving Prompts
```rust
// Get a specific prompt
let prompt = manager.get_prompt("system", "default")?;
// Get prompt with variable substitution
let mut vars = HashMap::new();
vars.insert("query", "user question");
let formatted = manager.format_prompt("context", "kb_search", vars)?;
```
### Dynamic Composition
```rust
// Compose multiple prompts
let system = manager.get_prompt("system", "default")?;
let tools = manager.get_prompt("tools", "available")?;
let context = manager.get_prompt("context", "current")?;
let full_prompt = manager.compose(vec![system, tools, context])?;
```
## Prompt Categories
### System Prompts
Define the AI assistant's role and behavior:
- `default`: Standard helpful assistant
- `professional`: Business-focused responses
- `technical`: Developer-oriented assistance
- `creative`: Creative writing and ideation
### Tool Prompts
Instructions for tool usage:
- `function_call`: How to invoke functions
- `parameter_format`: Parameter formatting rules
- `error_handling`: Tool error responses
### Context Prompts
Templates for providing context:
- `kb_search`: Knowledge base query format
- `conversation_history`: Previous message format
- `user_context`: User information format
### Guardrail Prompts
Safety and compliance instructions:
- `content_filter`: Inappropriate content handling
- `pii_protection`: Personal data protection
- `compliance`: Regulatory compliance rules
## Custom Prompts
Bots can override default prompts by providing their own:
```
mybot.gbai/
└── mybot.gbot/
├── config.csv
└── prompts.csv # Custom prompts override defaults
```
## Model-Specific Optimization
Prompts can be optimized for different models:
```csv
id,category,name,content,model,version
1,system,default,"You are Claude...",claude-3,1.0
2,system,default,"You are GPT-4...",gpt-4,1.0
3,system,default,"You are a helpful assistant",llama-3,1.0
```
The manager automatically selects the best match for the current model.
## Variables and Placeholders
Prompts support variable substitution using `{variable}` syntax:
```
"Search for {query} in {collection} and return {limit} results"
```
Variables are replaced at runtime:
```rust
let vars = hashmap!{
"query" => "pricing information",
"collection" => "docs",
"limit" => "5"
};
let prompt = manager.format_prompt("search", "template", vars)?;
```
## Prompt Versioning
Track prompt evolution:
```csv
id,category,name,content,model,version
1,system,default,"Original prompt...",gpt-4,1.0
2,system,default,"Updated prompt...",gpt-4,1.1
3,system,default,"Latest prompt...",gpt-4,2.0
```
The manager uses the latest version by default but can retrieve specific versions:
```rust
let prompt = manager.get_prompt_version("system", "default", "1.0")?;
```
## Performance Optimization
### Caching
Frequently used prompts are cached in memory:
```rust
manager.cache_prompt("system", "default");
```
### Token Counting
Estimate token usage before sending:
```rust
let tokens = manager.estimate_tokens(prompt, "gpt-4")?;
if tokens > MAX_TOKENS {
prompt = manager.compress_prompt(prompt, MAX_TOKENS)?;
}
```
### Compression
Automatically compress prompts while maintaining meaning:
```rust
let compressed = manager.compress_prompt(original, target_tokens)?;
```
## Best Practices
1. **Modularity**: Keep prompts focused on single responsibilities
2. **Versioning**: Always version prompts for rollback capability
3. **Testing**: Test prompts across different models
4. **Documentation**: Document the purpose and expected output
5. **Variables**: Use placeholders for dynamic content
6. **Optimization**: Tailor prompts to specific model capabilities
## Integration with BASIC
The Prompt Manager is automatically available in BASIC dialogs:
```basic
' Load custom prompt library
LOAD_PROMPTS "custom_prompts.csv"
' Use a prompt by name
response = LLM PROMPT("customer_greeting")
' Use prompt with variables
vars = CREATE_MAP()
vars["name"] = customer_name
vars["issue"] = support_ticket
response = LLM FORMAT_PROMPT("support_response", vars)
```
## Monitoring and Analytics
Track prompt performance:
```rust
// Log prompt usage
manager.log_usage("system", "default", response_quality);
// Get analytics
let stats = manager.get_prompt_stats("system", "default")?;
println!("Success rate: {}%", stats.success_rate);
println!("Avg response time: {}ms", stats.avg_latency);
```
## Error Handling
Handle missing or invalid prompts gracefully:
```rust
match manager.get_prompt("custom", "missing") {
Ok(prompt) => use_prompt(prompt),
Err(PromptError::NotFound) => use_default(),
Err(PromptError::Invalid) => log_and_fallback(),
Err(e) => return Err(e),
}
```
## Future Enhancements
- **A/B Testing**: Compare prompt effectiveness
- **Auto-optimization**: ML-based prompt improvement
- **Multi-language**: Prompt localization support
- **Prompt Chains**: Complex multi-step prompts
- **Visual Editor**: Web-based prompt management UI

432
docs/src/chapter-07/answer-modes.md
View file

@ -1 +1,433 @@
# Answer Modes # Answer Modes
Configure how the bot formulates and delivers responses to users across different scenarios and contexts.
## Overview
Answer modes control the style, length, format, and approach of bot responses. Each mode is optimized for specific use cases and can be switched dynamically based on context or user preferences.
## Available Answer Modes
### Default Mode
Standard conversational responses with balanced detail:
```csv
answerMode,default
```
Characteristics:
- Natural, conversational tone
- Moderate response length
- Includes relevant context
- Friendly and approachable
- Suitable for general interactions
### Simple Mode
Concise, straightforward answers:
```csv
answerMode,simple
```
Characteristics:
- Brief, to-the-point responses
- Minimal elaboration
- Direct answers only
- No unnecessary context
- Ideal for quick queries
Example responses:
- Default: "I'd be happy to help you reset your password. First, click on the 'Forgot Password' link on the login page, then enter your email address. You'll receive a reset link within a few minutes."
- Simple: "Click 'Forgot Password' on login page. Enter email. Check inbox for reset link."
### Detailed Mode
Comprehensive, thorough explanations:
```csv
answerMode,detailed
```
Characteristics:
- Extended explanations
- Multiple examples
- Step-by-step breakdowns
- Additional context and background
- Best for complex topics
### Technical Mode
Precise, technical language for professional users:
```csv
answerMode,technical
```
Characteristics:
- Technical terminology
- Code examples when relevant
- API references
- Detailed specifications
- Assumes technical knowledge
### Educational Mode
Teaching-focused responses with explanations:
```csv
answerMode,educational
```
Characteristics:
- Explains concepts thoroughly
- Uses analogies and examples
- Breaks down complex ideas
- Includes "why" not just "how"
- Patient and encouraging tone
### Professional Mode
Formal business communication:
```csv
answerMode,professional
```
Characteristics:
- Formal language
- Business appropriate
- Structured responses
- No casual expressions
- Suitable for corporate settings
### Friendly Mode
Warm, personable interactions:
```csv
answerMode,friendly
```
Characteristics:
- Casual, warm tone
- Uses emojis appropriately
- Personal touches
- Encouraging language
- Builds rapport
## Mode Selection
### Static Configuration
Set a default mode in config.csv:
```csv
answerMode,professional
```
### Dynamic Switching
Change modes during conversation:
```basic
IF user_type = "developer" THEN
SET_ANSWER_MODE "technical"
ELSE IF user_type = "student" THEN
SET_ANSWER_MODE "educational"
ELSE
SET_ANSWER_MODE "default"
END IF
```
### Context-Based Selection
Automatically adjust based on query:
```basic
query_type = ANALYZE_QUERY(user_input)
IF query_type = "quick_fact" THEN
SET_ANSWER_MODE "simple"
ELSE IF query_type = "how_to" THEN
SET_ANSWER_MODE "detailed"
END IF
```
## Mode Customization
### Custom Answer Modes
Define custom modes for specific needs:
```csv
customAnswerModes,"support,sales,onboarding"
answerMode.support.style,"empathetic"
answerMode.support.length,"moderate"
answerMode.support.examples,"true"
```
### Mode Parameters
Fine-tune each mode:
| Parameter | Description | Values |
|-----------|-------------|---------|
| `style` | Communication style | formal, casual, technical |
| `length` | Response length | brief, moderate, extensive |
| `examples` | Include examples | true, false |
| `formatting` | Text formatting | plain, markdown, html |
| `confidence` | Show confidence level | true, false |
| `sources` | Cite sources | true, false |
## Response Formatting
### Plain Text Mode
Simple text without formatting:
```csv
answerMode,simple
responseFormat,plain
```
Output: "Your order has been confirmed. Order number: 12345"
### Markdown Mode
Rich formatting with markdown:
```csv
answerMode,detailed
responseFormat,markdown
```
Output:
```markdown
## Order Confirmation
Your order has been **successfully confirmed**.
**Order Details:**
- Order Number: `12345`
- Status: ✅ Confirmed
- Delivery: 2-3 business days
```
### Structured Mode
JSON or structured data:
```csv
answerMode,technical
responseFormat,json
```
Output:
```json
{
"status": "confirmed",
"order_id": "12345",
"delivery_estimate": "2-3 days"
}
```
## Language Adaptation
### Complexity Levels
Adjust language complexity:
```csv
answerMode,default
languageLevel,intermediate
```
Levels:
- `basic`: Simple vocabulary, short sentences
- `intermediate`: Standard language
- `advanced`: Complex vocabulary, nuanced expression
- `expert`: Domain-specific terminology
### Tone Variations
| Mode | Tone | Example |
|------|------|---------|
| Professional | Formal | "I shall process your request immediately." |
| Friendly | Warm | "Sure thing! I'll get that done for you right away! 😊" |
| Technical | Precise | "Executing request. ETA: 2.3 seconds." |
| Educational | Patient | "Let me explain how this works step by step..." |
## Use Case Examples
### Customer Support
```csv
answerMode,support
emphathy,high
solutionFocused,true
escalationAware,true
```
Responses include:
- Acknowledgment of issue
- Empathetic language
- Clear solutions
- Escalation options
### Sales Assistant
```csv
answerMode,sales
enthusiasm,high
benefitsFocused,true
objectionHandling,true
```
Responses include:
- Product benefits
- Value propositions
- Addressing concerns
- Call-to-action
### Technical Documentation
```csv
answerMode,technical
codeExamples,true
apiReferences,true
errorCodes,true
```
Responses include:
- Code snippets
- API endpoints
- Error handling
- Implementation details
### Educational Tutor
```csv
answerMode,educational
scaffolding,true
examples,multiple
encouragement,true
```
Responses include:
- Step-by-step learning
- Multiple examples
- Concept reinforcement
- Positive feedback
## Performance Considerations
### Response Time vs Quality
| Mode | Response Time | Quality | Best For |
|------|--------------|---------|----------|
| Simple | Fastest | Basic | Quick queries |
| Default | Fast | Good | General use |
| Detailed | Moderate | High | Complex topics |
| Technical | Slower | Precise | Expert users |
### Token Usage
Approximate token consumption:
- Simple: 50-100 tokens
- Default: 100-200 tokens
- Detailed: 200-500 tokens
- Educational: 300-600 tokens
## Mode Combinations
Combine modes for specific scenarios:
```csv
answerMode,professional+detailed
```
Common combinations:
- `friendly+simple`: Casual quick help
- `professional+detailed`: Business documentation
- `technical+educational`: Developer training
- `support+empathetic`: Crisis handling
## Adaptive Modes
### User Preference Learning
System learns user preferences:
```basic
IF user_history.preferred_length = "short" THEN
SET_ANSWER_MODE "simple"
ELSE IF user_history.technical_level = "high" THEN
SET_ANSWER_MODE "technical"
END IF
```
### Feedback-Based Adjustment
Adjust based on user feedback:
```basic
IF user_feedback = "too_long" THEN
SWITCH_TO_SHORTER_MODE()
ELSE IF user_feedback = "need_more_detail" THEN
SWITCH_TO_DETAILED_MODE()
END IF
```
## Testing Answer Modes
### A/B Testing
Test different modes:
```csv
abTestEnabled,true
abTestModes,"simple,detailed"
abTestSplit,50
```
### Quality Metrics
Monitor mode effectiveness:
- User satisfaction scores
- Completion rates
- Follow-up questions
- Time to resolution
- Engagement metrics
## Best Practices
1. **Match user expectations**: Technical users want precision
2. **Consider context**: Urgent issues need simple mode
3. **Be consistent**: Don't switch modes mid-conversation without reason
4. **Test thoroughly**: Each mode should be tested with real queries
5. **Monitor feedback**: Adjust modes based on user response
6. **Document choices**: Explain why specific modes are used
7. **Provide options**: Let users choose their preferred mode
## Troubleshooting
### Response Too Long
- Switch to simple mode
- Reduce max tokens
- Enable summarization
### Response Too Technical
- Use educational mode
- Add examples
- Simplify language level
### Lack of Detail
- Switch to detailed mode
- Enable examples
- Add context inclusion
### Inconsistent Tone
- Lock mode for session
- Define clear mode parameters
- Test mode transitions

264
docs/src/chapter-07/config-csv.md
View file

@ -1 +1,265 @@
# config.csv Format # config.csv Format
The `config.csv` file is the central configuration for each bot instance. Located in the `.gbot` package directory, it controls all bot behavior, integrations, and system settings.
## File Location
```
mybot.gbai/
└── mybot.gbot/
└── config.csv
```
## Format
Configuration uses simple CSV format with two columns: `key` and `value`.
```csv
key,value
botId,00000000-0000-0000-0000-000000000000
title,My Bot Name
description,Bot description here
```
## Core Settings
### Bot Identity
| Key | Description | Default | Example |
|-----|-------------|---------|---------|
| `botId` | Unique bot identifier (UUID) | Generated | `00000000-0000-0000-0000-000000000000` |
| `title` | Bot display name | Required | `Customer Support Bot` |
| `description` | Bot description | Empty | `Handles customer inquiries` |
| `logoUrl` | Bot avatar/logo URL | Empty | `https://example.com/logo.png` |
| `welcomeMessage` | Initial greeting | Empty | `Hello! How can I help you today?` |
### LLM Configuration
| Key | Description | Default | Example |
|-----|-------------|---------|---------|
| `llmModel` | Model to use | `gpt-4` | `gpt-4`, `claude-3`, `llama-3` |
| `llmApiKey` | API key for LLM service | Required | `sk-...` |
| `llmEndpoint` | Custom LLM endpoint | Provider default | `https://api.openai.com/v1` |
| `llmTemperature` | Response creativity (0-1) | `0.7` | `0.3` for factual, `0.9` for creative |
| `llmMaxTokens` | Max response length | `2000` | `4000` |
| `answerMode` | Response strategy | `default` | `simple`, `detailed`, `technical` |
### Knowledge Base
| Key | Description | Default | Example |
|-----|-------------|---------|---------|
| `qdrantUrl` | Vector database URL | `http://localhost:6333` | `http://qdrant:6333` |
| `qdrantApiKey` | Qdrant API key | Empty | `your-api-key` |
| `embeddingModel` | Model for embeddings | `text-embedding-ada-002` | `all-MiniLM-L6-v2` |
| `chunkSize` | Text chunk size | `1000` | `500` |
| `chunkOverlap` | Overlap between chunks | `200` | `100` |
| `topK` | Number of search results | `5` | `10` |
### Storage Configuration
| Key | Description | Default | Example |
|-----|-------------|---------|---------|
| `minioEndpoint` | MinIO/S3 endpoint | `localhost:9000` | `minio.example.com` |
| `minioAccessKey` | Storage access key | Required | `minioadmin` |
| `minioSecretKey` | Storage secret key | Required | `minioadmin` |
| `minioBucket` | Default bucket name | `botserver` | `my-bot-files` |
| `minioUseSsl` | Use HTTPS for MinIO | `false` | `true` |
### Database
| Key | Description | Default | Example |
|-----|-------------|---------|---------|
| `databaseUrl` | PostgreSQL connection | Required | `postgresql://user:pass@localhost/botdb` |
| `maxConnections` | Connection pool size | `10` | `25` |
| `connectionTimeout` | Timeout in seconds | `30` | `60` |
### Email Integration
| Key | Description | Default | Example |
|-----|-------------|---------|---------|
| `smtpHost` | SMTP server | Empty | `smtp.gmail.com` |
| `smtpPort` | SMTP port | `587` | `465` |
| `smtpUser` | Email username | Empty | `bot@example.com` |
| `smtpPassword` | Email password | Empty | `app-specific-password` |
| `smtpFrom` | From address | Empty | `noreply@example.com` |
| `smtpUseTls` | Use TLS | `true` | `false` |
### Calendar Integration
| Key | Description | Default | Example |
|-----|-------------|---------|---------|
| `calendarEnabled` | Enable calendar features | `false` | `true` |
| `calendarProvider` | Calendar service | `google` | `microsoft`, `caldav` |
| `calendarApiKey` | Calendar API key | Empty | `your-api-key` |
| `workingHoursStart` | Business hours start | `09:00` | `08:30` |
| `workingHoursEnd` | Business hours end | `17:00` | `18:00` |
| `timezone` | Default timezone | `UTC` | `America/New_York` |
### Authentication
| Key | Description | Default | Example |
|-----|-------------|---------|---------|
| `authEnabled` | Require authentication | `false` | `true` |
| `authProvider` | Auth provider | `local` | `oauth`, `saml`, `ldap` |
| `authClientId` | OAuth client ID | Empty | `client-id` |
| `authClientSecret` | OAuth secret | Empty | `client-secret` |
| `authCallbackUrl` | OAuth callback | Empty | `https://bot.example.com/auth/callback` |
| `jwtSecret` | JWT signing secret | Generated | `your-secret-key` |
| `sessionTimeout` | Session duration (min) | `1440` | `60` |
### Channel Configuration
| Key | Description | Default | Example |
|-----|-------------|---------|---------|
| `webEnabled` | Enable web interface | `true` | `false` |
| `whatsappEnabled` | Enable WhatsApp | `false` | `true` |
| `whatsappToken` | WhatsApp API token | Empty | `EAAI...` |
| `whatsappPhoneId` | WhatsApp phone ID | Empty | `123456789` |
| `teamsEnabled` | Enable MS Teams | `false` | `true` |
| `teamsAppId` | Teams app ID | Empty | `app-id` |
| `teamsAppPassword` | Teams app password | Empty | `app-password` |
| `slackEnabled` | Enable Slack | `false` | `true` |
| `slackToken` | Slack bot token | Empty | `xoxb-...` |
### Security
| Key | Description | Default | Example |
|-----|-------------|---------|---------|
| `corsOrigins` | Allowed CORS origins | `*` | `https://example.com` |
| `rateLimitPerMinute` | API rate limit | `60` | `100` |
| `maxFileSize` | Max upload size (MB) | `10` | `50` |
| `allowedFileTypes` | Permitted file types | `pdf,doc,txt` | `*` |
| `encryptionKey` | Data encryption key | Generated | `base64-key` |
| `requireHttps` | Force HTTPS | `false` | `true` |
### Monitoring
| Key | Description | Default | Example |
|-----|-------------|---------|---------|
| `metricsEnabled` | Enable metrics | `false` | `true` |
| `metricsEndpoint` | Metrics endpoint | `/metrics` | `/admin/metrics` |
| `loggingLevel` | Log level | `info` | `debug`, `warn`, `error` |
| `logToFile` | Log to file | `false` | `true` |
| `logFilePath` | Log file location | `./logs` | `/var/log/botserver` |
| `sentryDsn` | Sentry error tracking | Empty | `https://...@sentry.io/...` |
### Advanced Features
| Key | Description | Default | Example |
|-----|-------------|---------|---------|
| `webAutomationEnabled` | Enable web scraping | `false` | `true` |
| `ocrEnabled` | Enable OCR | `false` | `true` |
| `speechEnabled` | Enable speech | `false` | `true` |
| `translationEnabled` | Enable translation | `false` | `true` |
| `cacheEnabled` | Enable Redis cache | `false` | `true` |
| `cacheUrl` | Redis URL | `redis://localhost:6379` | `redis://cache:6379` |
## Environment Variable Override
Any config value can be overridden using environment variables:
```bash
# Override LLM model
export BOT_LLM_MODEL=gpt-4-turbo
# Override database URL
export BOT_DATABASE_URL=postgresql://prod@db/botserver
```
## Multiple Bots Configuration
Each bot has its own `config.csv`. The system loads all bot configurations on startup:
```
templates/
├── support.gbai/
│ └── support.gbot/
│ └── config.csv # Support bot config
├── sales.gbai/
│ └── sales.gbot/
│ └── config.csv # Sales bot config
└── default.gbai/
└── default.gbot/
└── config.csv # Default bot config
```
## Configuration Validation
The system validates configuration on startup:
- Required fields must be present
- UUIDs must be valid format
- URLs must be reachable
- API keys are tested
- File paths must exist
## Hot Reload
Changes to `config.csv` can be reloaded without restart:
1. Edit the file
2. Call `/api/admin/reload-config` endpoint
3. Or use the admin UI reload button
## Security Best Practices
1. **Never commit API keys** - Use environment variables
2. **Encrypt sensitive values** - Use `encryptionKey` setting
3. **Rotate credentials regularly** - Update keys monthly
4. **Use strong JWT secrets** - At least 32 characters
5. **Restrict CORS origins** - Don't use `*` in production
6. **Enable HTTPS** - Set `requireHttps=true`
7. **Set rate limits** - Prevent abuse
8. **Monitor access** - Enable logging and metrics
## Troubleshooting
### Bot Won't Start
- Check required fields are set
- Verify database connection
- Ensure bot ID is unique
### LLM Not Responding
- Verify API key is valid
- Check endpoint URL
- Test rate limits
### Storage Issues
- Verify MinIO is running
- Check access credentials
- Test bucket permissions
### Authentication Problems
- Verify JWT secret matches
- Check session timeout
- Test OAuth callback URL
## Example Configuration
Complete example for a production bot:
```csv
key,value
botId,a1b2c3d4-e5f6-7890-abcd-ef1234567890
title,Customer Support Assistant
description,24/7 automated customer support
welcomeMessage,Hello! I'm here to help with any questions.
llmModel,gpt-4
llmApiKey,${LLM_API_KEY}
llmTemperature,0.3
answerMode,detailed
databaseUrl,${DATABASE_URL}
minioEndpoint,storage.example.com
minioAccessKey,${MINIO_ACCESS}
minioSecretKey,${MINIO_SECRET}
minioBucket,support-bot
minioUseSsl,true
authEnabled,true
authProvider,oauth
authClientId,${OAUTH_CLIENT_ID}
authClientSecret,${OAUTH_CLIENT_SECRET}
corsOrigins,https://app.example.com
requireHttps,true
loggingLevel,info
metricsEnabled,true
cacheEnabled,true
cacheUrl,redis://cache:6379
```

493
docs/src/chapter-07/context-config.md
View file

@ -1 +1,494 @@
# Context Configuration # Context Configuration
Configure how BotServer manages conversation context, memory, and state across user interactions.
## Overview
Context configuration determines how the bot maintains conversation state, manages memory, processes historical interactions, and provides relevant responses based on accumulated knowledge. Proper context configuration is crucial for coherent, contextually-aware conversations.
## Context Providers
### Database Context Provider
Default provider using PostgreSQL for context storage:
```csv
contextProvider,database
contextConnectionPool,10
contextQueryTimeout,5000
```
Features:
- Persistent context storage
- Fast retrieval with indexes
- Supports complex queries
- Scales with database
- ACID compliance
### Memory Context Provider
In-memory context for maximum performance:
```csv
contextProvider,memory
contextMaxMemoryMB,512
contextEvictionPolicy,lru
```
Features:
- Ultra-fast access
- No persistence overhead
- Ideal for stateless bots
- Limited by RAM
- Lost on restart
### Hybrid Context Provider
Combines database and memory for optimal performance:
```csv
contextProvider,hybrid
contextCacheTTL,3600
contextCacheSize,1000
```
Features:
- Memory cache with DB backing
- Best of both approaches
- Automatic synchronization
- Configurable cache policies
- Failover support
## Context Window Management
### Window Size Configuration
Control how much history is maintained:
```csv
contextWindowSize,10
contextMaxTokens,4000
contextPruningStrategy,sliding
```
| Parameter | Description | Default | Range |
|-----------|-------------|---------|-------|
| `contextWindowSize` | Number of messages to keep | 10 | 1-100 |
| `contextMaxTokens` | Maximum context tokens | 4000 | 500-32000 |
| `contextPruningStrategy` | How to prune context | sliding | sliding, summary, selective |
### Pruning Strategies
**Sliding Window**: Keep last N messages
```csv
contextPruningStrategy,sliding
contextWindowSize,10
```
**Summarization**: Compress older messages
```csv
contextPruningStrategy,summary
contextSummaryThreshold,20
contextSummaryRatio,0.3
```
**Selective**: Keep important messages only
```csv
contextPruningStrategy,selective
contextImportanceThreshold,0.7
contextKeepSystemMessages,true
```
## Memory Types
### Short-Term Memory
Temporary context for current session:
```csv
shortTermMemoryEnabled,true
shortTermMemoryDuration,3600
shortTermMemorySize,100
```
Stores:
- Current conversation
- Temporary variables
- Session state
- Recent interactions
### Long-Term Memory
Persistent memory across sessions:
```csv
longTermMemoryEnabled,true
longTermMemoryRetention,365
longTermMemoryCompression,true
```
Stores:
- User preferences
- Historical interactions
- Learned patterns
- Important facts
### Working Memory
Active context for processing:
```csv
workingMemorySize,5
workingMemoryRefresh,message
workingMemoryScope,conversation
```
Contains:
- Current topic
- Active variables
- Immediate context
- Processing state
## Context Enhancement
### Semantic Context
Add semantically relevant information:
```csv
semanticContextEnabled,true
semanticSearchTopK,5
semanticThreshold,0.75
```
Features:
- Retrieves related past conversations
- Adds relevant knowledge base entries
- Includes similar resolved issues
- Enhances response relevance
### Entity Tracking
Track entities throughout conversation:
```csv
entityTrackingEnabled,true
entityTypes,"person,product,location,date"
entityResolution,true
```
Tracks:
- Named entities
- References (pronouns)
- Relationships
- Entity state changes
### Topic Detection
Identify and track conversation topics:
```csv
topicDetectionEnabled,true
topicChangeThreshold,0.6
topicHistorySize,5
```
Benefits:
- Context switching awareness
- Relevant response generation
- Topic-based memory retrieval
- Conversation flow management
## State Management
### Session State
Maintain state within sessions:
```csv
sessionStateEnabled,true
sessionTimeout,1800
sessionStorage,redis
```
Stores:
- User authentication
- Current dialog position
- Variable values
- Temporary flags
### Global State
Shared state across sessions:
```csv
globalStateEnabled,true
globalStateNamespace,bot
globalStateSyncInterval,60
```
Contains:
- System-wide settings
- Shared resources
- Global counters
- Feature flags
### User State
Per-user persistent state:
```csv
userStateEnabled,true
userStateFields,"preferences,history,profile"
userStateEncrypted,true
```
Includes:
- User preferences
- Interaction history
- Personalization data
- Custom attributes
## Context Injection
### System Context
Always-present system information:
```csv
systemContextEnabled,true
systemContextTemplate,"You are {bot_name}. Current time: {time}. User: {user_name}"
```
### Dynamic Context
Conditionally injected context:
```csv
dynamicContextEnabled,true
dynamicContextRules,"business_hours,user_tier,location"
```
Examples:
- Business hours notice
- User tier benefits
- Location-based info
- Seasonal messages
### Tool Context
Context for active tools:
```csv
toolContextEnabled,true
toolContextAutoLoad,true
toolContextFormat,structured
```
## Performance Optimization
### Context Caching
Cache frequently accessed context:
```csv
contextCacheEnabled,true
contextCacheProvider,redis
contextCacheTTL,300
contextCacheMaxSize,1000
```
Benefits:
- Reduced database queries
- Faster response times
- Lower latency
- Resource efficiency
### Lazy Loading
Load context on demand:
```csv
lazyLoadingEnabled,true
lazyLoadThreshold,0.8
preloadCommonContext,true
```
### Context Compression
Compress stored context:
```csv
contextCompressionEnabled,true
contextCompressionLevel,6
contextCompressionThreshold,1000
```
## Multi-Turn Conversations
### Conversation Threading
Track conversation threads:
```csv
threadingEnabled,true
threadTimeout,300
threadMaxDepth,10
```
### Context Carryover
Maintain context across interactions:
```csv
contextCarryoverEnabled,true
carryoverFields,"topic,intent,entities"
carryoverDuration,600
```
### Dialog State
Track dialog flow state:
```csv
dialogStateEnabled,true
dialogStateProvider,memory
dialogStateTimeout,1800
```
## Context Rules
### Inclusion Rules
Define what to include in context:
```csv
contextIncludeUserMessages,true
contextIncludeSystemMessages,true
contextIncludeErrors,false
contextIncludeDebug,false
```
### Exclusion Rules
Define what to exclude:
```csv
contextExcludePII,true
contextExcludePasswords,true
contextExcludeSensitive,true
```
### Transformation Rules
Transform context before use:
```csv
contextMaskPII,true
contextNormalizeText,true
contextTranslate,false
```
## Monitoring
### Context Metrics
Track context performance:
```csv
contextMetricsEnabled,true
contextMetricsInterval,60
contextMetricsRetention,7
```
Key metrics:
- Context size
- Retrieval time
- Cache hit rate
- Memory usage
- Pruning frequency
### Debug Logging
Debug context operations:
```csv
contextDebugEnabled,false
contextDebugLevel,info
contextDebugOutput,file
```
## Best Practices
1. **Right-size context window**: Balance completeness vs performance
2. **Use appropriate provider**: Database for persistence, memory for speed
3. **Enable caching**: Significantly improves performance
4. **Prune strategically**: Keep relevant, remove redundant
5. **Monitor metrics**: Track and optimize based on usage
6. **Secure sensitive data**: Encrypt and mask PII
7. **Test context switching**: Ensure smooth topic transitions
8. **Document configuration**: Explain choices and trade-offs
## Common Configurations
### High-Performance Chat
```csv
contextProvider,memory
contextWindowSize,5
contextCacheEnabled,true
lazyLoadingEnabled,true
```
### Long Conversations
```csv
contextProvider,database
contextWindowSize,50
contextPruningStrategy,summary
contextCompressionEnabled,true
```
### Privacy-Focused
```csv
contextProvider,memory
contextExcludePII,true
contextMaskSensitive,true
contextEncryption,true
```
### Multi-User Support
```csv
contextProvider,hybrid
userStateEnabled,true
sessionIsolation,strict
contextNamespacing,true
```
## Troubleshooting
### Context Loss
- Check session timeout settings
- Verify database connectivity
- Review memory limits
- Check pruning settings
### Slow Context Retrieval
- Enable caching
- Optimize queries
- Reduce window size
- Use lazy loading
### Memory Issues
- Reduce context window
- Enable compression
- Increase pruning frequency
- Switch to database provider
### Inconsistent Responses
- Check context carryover
- Verify entity tracking
- Review pruning strategy
- Test context injection

418
docs/src/chapter-07/llm-config.md
View file

@ -1 +1,419 @@
# LLM Configuration # LLM Configuration
Configure Large Language Model providers, models, and parameters for optimal bot performance.
## Overview
BotServer supports multiple LLM providers with flexible configuration options. Each bot can use different models and settings based on requirements for performance, cost, and capabilities.
## Supported Providers
### OpenAI
The most popular provider with GPT models:
```csv
llmProvider,openai
llmModel,gpt-4
llmApiKey,sk-...
llmEndpoint,https://api.openai.com/v1
```
Available models:
- `gpt-4` - Most capable, higher cost
- `gpt-4-turbo` - Faster, more affordable GPT-4
- `gpt-3.5-turbo` - Fast and cost-effective
- `gpt-3.5-turbo-16k` - Extended context window
### Anthropic (Claude)
Advanced models with strong reasoning:
```csv
llmProvider,anthropic
llmModel,claude-3-opus-20240229
llmApiKey,sk-ant-...
llmEndpoint,https://api.anthropic.com
```
Available models:
- `claude-3-opus` - Most capable Claude model
- `claude-3-sonnet` - Balanced performance
- `claude-3-haiku` - Fast and efficient
- `claude-2.1` - Previous generation
### Google (Gemini)
Google's multimodal AI models:
```csv
llmProvider,google
llmModel,gemini-pro
llmApiKey,AIza...
llmProject,my-project-id
```
Available models:
- `gemini-pro` - Text generation
- `gemini-pro-vision` - Multimodal (text + images)
- `gemini-ultra` - Most advanced (limited access)
### Local Models
Self-hosted open-source models:
```csv
llmProvider,local
llmModel,llama-3-70b
llmEndpoint,http://localhost:8000
```
Supported local models:
- Llama 3 (8B, 70B)
- Mistral (7B, 8x7B)
- Falcon (7B, 40B)
- Vicuna
- Alpaca
## Model Selection Guide
### By Use Case
| Use Case | Recommended Model | Reasoning |
|----------|------------------|-----------|
| Customer Support | gpt-3.5-turbo | Fast, cost-effective, good quality |
| Technical Documentation | gpt-4 | Accurate, detailed responses |
| Creative Writing | claude-3-opus | Strong creative capabilities |
| Code Generation | gpt-4 | Best code understanding |
| Multilingual | gemini-pro | Excellent language support |
| Privacy-Sensitive | Local Llama 3 | Data stays on-premise |
| High Volume | gpt-3.5-turbo | Lowest cost per token |
### By Requirements
**Need accuracy?** → GPT-4 or Claude-3-opus
**Need speed?** → GPT-3.5-turbo or Claude-3-haiku
**Need low cost?** → Local models or GPT-3.5
**Need privacy?** → Local models only
**Need vision?** → Gemini-pro-vision or GPT-4V
## Temperature Settings
Temperature controls response creativity and randomness:
```csv
llmTemperature,0.7
```
### Temperature Guide
| Value | Use Case | Behavior |
|-------|----------|----------|
| 0.0 | Factual Q&A | Deterministic, same output |
| 0.2 | Technical docs | Very focused, minimal variation |
| 0.5 | Customer service | Balanced consistency |
| 0.7 | General chat | Natural variation (default) |
| 0.9 | Creative tasks | High creativity |
| 1.0 | Brainstorming | Maximum randomness |
### Examples by Domain
**Legal/Medical**: 0.1-0.3 (high accuracy required)
**Education**: 0.3-0.5 (clear, consistent)
**Sales/Marketing**: 0.6-0.8 (engaging, varied)
**Entertainment**: 0.8-1.0 (creative, surprising)
## Token Management
### Context Window Sizes
| Model | Max Tokens | Recommended |
|-------|------------|-------------|
| GPT-3.5 | 4,096 | 3,000 |
| GPT-3.5-16k | 16,384 | 12,000 |
| GPT-4 | 8,192 | 6,000 |
| GPT-4-32k | 32,768 | 25,000 |
| Claude-3 | 200,000 | 150,000 |
| Gemini-Pro | 32,768 | 25,000 |
### Token Allocation
```csv
llmMaxTokens,2000
llmContextTokens,2000
llmResponseTokens,1000
```
Best practices:
- Reserve 25% for system prompt
- Allocate 50% for context/history
- Keep 25% for response
### Cost Optimization
Monitor and control token usage:
```csv
llmTokenLimit,1000000
llmCostLimit,100
llmRequestLimit,10000
```
Tips for reducing costs:
1. Use smaller models when possible
2. Implement response caching
3. Compress conversation history
4. Set appropriate max tokens
5. Use temperature 0 for consistent caching
## Advanced Parameters
### Sampling Parameters
Fine-tune response generation:
```csv
llmTopP,0.9
llmTopK,50
llmFrequencyPenalty,0.5
llmPresencePenalty,0.5
```
| Parameter | Effect | Range | Default |
|-----------|--------|-------|---------|
| `topP` | Nucleus sampling threshold | 0-1 | 1.0 |
| `topK` | Top tokens to consider | 1-100 | None |
| `frequencyPenalty` | Reduce repetition | -2 to 2 | 0 |
| `presencePenalty` | Encourage new topics | -2 to 2 | 0 |
### Stop Sequences
Control when generation stops:
```csv
llmStopSequences,"Human:,Assistant:,###"
```
Common stop sequences:
- Conversation markers: `"Human:", "User:", "AI:"`
- Section dividers: `"###", "---", "==="`
- Custom tokens: `"[END]", "</response>"`
## System Prompts
Define bot personality and behavior:
```csv
llmSystemPrompt,"You are a helpful customer service agent for ACME Corp. Be friendly, professional, and concise. Always verify customer information before making changes."
```
### System Prompt Templates
**Professional Assistant**:
```
You are a professional assistant. Provide accurate, helpful responses.
Be concise but thorough. Maintain a formal, respectful tone.
```
**Technical Support**:
```
You are a technical support specialist. Help users troubleshoot issues.
Ask clarifying questions. Provide step-by-step solutions.
```
**Sales Representative**:
```
You are a friendly sales representative. Help customers find products.
Be enthusiastic but not pushy. Focus on customer needs.
```
## Model Fallbacks
Configure backup models for reliability:
```csv
llmProvider,openai
llmModel,gpt-4
llmFallbackModel,gpt-3.5-turbo
llmFallbackOnError,true
llmFallbackOnRateLimit,true
```
Fallback strategies:
1. **Error fallback**: Use backup on API errors
2. **Rate limit fallback**: Switch when rate limited
3. **Cost fallback**: Use cheaper model at limits
4. **Load balancing**: Distribute across models
## Response Caching
Improve performance and reduce costs:
```csv
llmCacheEnabled,true
llmCacheTTL,3600
llmCacheKey,message_hash
llmCacheOnlyDeterministic,true
```
Cache strategies:
- Cache identical queries (temperature=0)
- Cache by semantic similarity
- Cache common questions/FAQs
- Invalidate cache on knowledge updates
## Streaming Configuration
Enable real-time response streaming:
```csv
llmStreamEnabled,true
llmStreamChunkSize,10
llmStreamTimeout,30000
```
Benefits:
- Faster perceived response time
- Better user experience
- Allows interruption
- Progressive rendering
## Error Handling
Configure error behavior:
```csv
llmRetryAttempts,3
llmRetryDelay,1000
llmTimeoutSeconds,30
llmErrorMessage,"I'm having trouble processing that. Please try again."
```
Error types and handling:
- **Timeout**: Retry with shorter prompt
- **Rate limit**: Wait and retry or fallback
- **Invalid request**: Log and return error
- **Service unavailable**: Use fallback model
## Monitoring and Logging
Track LLM performance:
```csv
llmLogRequests,true
llmLogResponses,false
llmMetricsEnabled,true
llmLatencyAlert,5000
```
Key metrics to monitor:
- Response latency
- Token usage
- Cost per conversation
- Error rates
- Cache hit rates
## Multi-Model Strategies
Use different models for different tasks:
```csv
llmModelRouting,true
llmSimpleQueries,gpt-3.5-turbo
llmComplexQueries,gpt-4
llmCreativeTasks,claude-3-opus
```
Routing logic:
1. Analyze query complexity
2. Check user tier/permissions
3. Consider cost budget
4. Route to appropriate model
## Best Practices
### Development vs Production
**Development**:
```csv
llmModel,gpt-3.5-turbo
llmLogResponses,true
llmCacheEnabled,false
llmMockMode,true
```
**Production**:
```csv
llmModel,gpt-4
llmLogResponses,false
llmCacheEnabled,true
llmMockMode,false
```
### Cost Management
1. Start with smaller models
2. Use caching aggressively
3. Implement token limits
4. Monitor usage daily
5. Set up cost alerts
6. Use fallback models
7. Compress contexts
### Performance Optimization
1. Enable streaming for long responses
2. Use appropriate temperature
3. Set reasonable max tokens
4. Implement response caching
5. Use connection pooling
6. Consider edge deployment
### Security
1. Never commit API keys
2. Use environment variables
3. Rotate keys regularly
4. Implement rate limiting
5. Validate all inputs
6. Sanitize outputs
7. Log security events
## Troubleshooting
### Common Issues
**Slow responses**: Lower max tokens, enable streaming
**High costs**: Use cheaper models, enable caching
**Inconsistent output**: Lower temperature, add examples
**Rate limits**: Implement backoff, use multiple keys
**Timeouts**: Increase timeout, reduce prompt size
### Debugging
Enable debug mode:
```csv
llmDebugMode,true
llmVerboseLogging,true
llmTraceRequests,true
```
## Migration Guide
### Switching Providers
1. Update `llmProvider` and `llmModel`
2. Set appropriate API key
3. Adjust token limits for new model
4. Test with sample queries
5. Update system prompts if needed
6. Monitor for behavior changes
### Upgrading Models
1. Test new model in development
2. Compare outputs with current model
3. Adjust temperature/parameters
4. Update fallback configuration
5. Gradual rollout to production
6. Monitor metrics closely

294
docs/src/chapter-07/parameters.md
View file

@ -1 +1,295 @@
# Bot Parameters # Bot Parameters
Comprehensive reference for all bot configuration parameters available in `config.csv`.
## Parameter Categories
Bot parameters are organized into functional groups for easier management and understanding.
## Core Bot Settings
### Identity Parameters
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `botId` | UUID | Yes | Generated | Unique bot identifier |
| `title` | String | Yes | None | Bot display name |
| `description` | String | No | Empty | Bot description |
| `version` | String | No | "1.0" | Bot version |
| `author` | String | No | Empty | Bot creator |
| `language` | String | No | "en" | Default language |
| `timezone` | String | No | "UTC" | Bot timezone |
### Behavior Parameters
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `welcomeMessage` | String | No | Empty | Initial greeting |
| `fallbackMessage` | String | No | "I don't understand" | Default error response |
| `goodbyeMessage` | String | No | "Goodbye!" | Session end message |
| `typingDelay` | Number | No | 1000 | Typing indicator delay (ms) |
| `responseTimeout` | Number | No | 30000 | Response timeout (ms) |
| `maxRetries` | Number | No | 3 | Maximum retry attempts |
| `debugMode` | Boolean | No | false | Enable debug logging |
## LLM Parameters
### Model Configuration
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `llmProvider` | String | Yes | "openai" | LLM provider (openai, anthropic, google, local) |
| `llmModel` | String | Yes | "gpt-4" | Model name |
| `llmApiKey` | String | Yes* | None | API key (*not required for local) |
| `llmEndpoint` | String | No | Provider default | Custom API endpoint |
| `llmOrganization` | String | No | Empty | Organization ID (OpenAI) |
| `llmProject` | String | No | Empty | Project ID (Google) |
### Response Control
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `llmTemperature` | Float | No | 0.7 | Creativity (0.0-1.0) |
| `llmMaxTokens` | Number | No | 2000 | Max response tokens |
| `llmTopP` | Float | No | 1.0 | Nucleus sampling |
| `llmFrequencyPenalty` | Float | No | 0.0 | Reduce repetition |
| `llmPresencePenalty` | Float | No | 0.0 | Encourage new topics |
| `llmStopSequences` | String | No | Empty | Stop generation sequences |
| `llmSystemPrompt` | String | No | Default | System instruction |
### Cost Management
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `llmCostLimit` | Number | No | 100 | Monthly cost limit ($) |
| `llmTokenLimit` | Number | No | 1000000 | Monthly token limit |
| `llmRequestLimit` | Number | No | 10000 | Daily request limit |
| `llmCacheEnabled` | Boolean | No | true | Enable response caching |
| `llmCacheTTL` | Number | No | 3600 | Cache duration (seconds) |
## Knowledge Base Parameters
### Vector Database
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `vectorDbUrl` | String | No | "http://localhost:6333" | Qdrant URL |
| `vectorDbApiKey` | String | No | Empty | Qdrant API key |
| `vectorDbCollection` | String | No | Bot name | Default collection |
| `embeddingModel` | String | No | "text-embedding-ada-002" | Embedding model |
| `embeddingDimension` | Number | No | 1536 | Vector dimension |
### Search Configuration
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `searchTopK` | Number | No | 5 | Results to return |
| `searchThreshold` | Float | No | 0.7 | Minimum similarity |
| `searchRerank` | Boolean | No | false | Enable reranking |
| `chunkSize` | Number | No | 1000 | Text chunk size |
| `chunkOverlap` | Number | No | 200 | Chunk overlap |
## Storage Parameters
### Object Storage
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `storageProvider` | String | No | "minio" | Storage provider |
| `storageEndpoint` | String | Yes | "localhost:9000" | S3/MinIO endpoint |
| `storageAccessKey` | String | Yes | None | Access key |
| `storageSecretKey` | String | Yes | None | Secret key |
| `storageBucket` | String | No | "botserver" | Default bucket |
| `storageRegion` | String | No | "us-east-1" | AWS region |
| `storageUseSsl` | Boolean | No | false | Use HTTPS |
### File Handling
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `maxFileSize` | Number | No | 10 | Max file size (MB) |
| `allowedFileTypes` | String | No | "pdf,doc,txt,csv" | Allowed extensions |
| `fileRetention` | Number | No | 90 | Days to keep files |
| `autoDeleteTemp` | Boolean | No | true | Auto-delete temp files |
## Communication Parameters
### Email Settings
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `emailEnabled` | Boolean | No | false | Enable email |
| `smtpHost` | String | No* | Empty | SMTP server |
| `smtpPort` | Number | No | 587 | SMTP port |
| `smtpUser` | String | No* | Empty | Email username |
| `smtpPassword` | String | No* | Empty | Email password |
| `smtpFrom` | String | No* | Empty | From address |
| `smtpUseTls` | Boolean | No | true | Use TLS |
| `smtpUseStarttls` | Boolean | No | true | Use STARTTLS |
### Channel Configuration
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `webEnabled` | Boolean | No | true | Web interface |
| `webPort` | Number | No | 8080 | Web port |
| `whatsappEnabled` | Boolean | No | false | WhatsApp integration |
| `whatsappToken` | String | No* | Empty | WhatsApp token |
| `teamsEnabled` | Boolean | No | false | Teams integration |
| `teamsAppId` | String | No* | Empty | Teams app ID |
| `slackEnabled` | Boolean | No | false | Slack integration |
| `slackToken` | String | No* | Empty | Slack token |
## Security Parameters
### Authentication
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `authRequired` | Boolean | No | false | Require authentication |
| `authProvider` | String | No | "local" | Auth provider |
| `jwtSecret` | String | Yes* | Generated | JWT secret |
| `jwtExpiration` | Number | No | 86400 | Token expiration (s) |
| `sessionTimeout` | Number | No | 3600 | Session timeout (s) |
| `maxSessions` | Number | No | 100 | Max concurrent sessions |
### Access Control
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `corsOrigins` | String | No | "*" | Allowed origins |
| `ipWhitelist` | String | No | Empty | Allowed IPs |
| `ipBlacklist` | String | No | Empty | Blocked IPs |
| `rateLimitPerMinute` | Number | No | 60 | Requests per minute |
| `rateLimitPerHour` | Number | No | 1000 | Requests per hour |
| `requireHttps` | Boolean | No | false | Force HTTPS |
### Data Protection
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `encryptData` | Boolean | No | true | Encrypt stored data |
| `encryptionKey` | String | Yes* | Generated | Encryption key |
| `maskPii` | Boolean | No | true | Mask personal data |
| `auditLogging` | Boolean | No | true | Enable audit logs |
| `dataRetention` | Number | No | 365 | Data retention (days) |
## Performance Parameters
### Caching
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `cacheEnabled` | Boolean | No | true | Enable caching |
| `cacheProvider` | String | No | "redis" | Cache provider |
| `cacheUrl` | String | No | "redis://localhost:6379" | Cache URL |
| `cacheTtl` | Number | No | 3600 | Default TTL (s) |
| `cacheMaxSize` | Number | No | 100 | Max cache size (MB) |
### Resource Limits
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `maxCpu` | Number | No | 2 | CPU cores limit |
| `maxMemory` | Number | No | 2048 | Memory limit (MB) |
| `maxConnections` | Number | No | 100 | DB connections |
| `maxWorkers` | Number | No | 4 | Worker threads |
| `queueSize` | Number | No | 1000 | Task queue size |
## Monitoring Parameters
### Logging
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `logLevel` | String | No | "info" | Log level |
| `logToFile` | Boolean | No | true | Log to file |
| `logFilePath` | String | No | "./logs" | Log directory |
| `logRotation` | String | No | "daily" | Rotation schedule |
| `logRetention` | Number | No | 30 | Keep logs (days) |
### Metrics
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `metricsEnabled` | Boolean | No | false | Enable metrics |
| `metricsEndpoint` | String | No | "/metrics" | Metrics endpoint |
| `sentryDsn` | String | No | Empty | Sentry DSN |
| `datadogApiKey` | String | No | Empty | Datadog API key |
| `prometheusPort` | Number | No | 9090 | Prometheus port |
## Feature Flags
### Experimental Features
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `betaFeatures` | Boolean | No | false | Enable beta features |
| `webAutomation` | Boolean | No | false | Web scraping |
| `ocrEnabled` | Boolean | No | false | OCR support |
| `speechEnabled` | Boolean | No | false | Speech I/O |
| `visionEnabled` | Boolean | No | false | Image analysis |
| `codeExecution` | Boolean | No | false | Code running |
## Environment-Specific Parameters
### Development
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `devMode` | Boolean | No | false | Development mode |
| `hotReload` | Boolean | No | false | Hot reload |
| `mockServices` | Boolean | No | false | Use mock services |
| `verboseErrors` | Boolean | No | false | Detailed errors |
### Production
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `prodMode` | Boolean | No | true | Production mode |
| `clustering` | Boolean | No | false | Enable clustering |
| `loadBalancing` | Boolean | No | false | Load balancing |
| `autoScale` | Boolean | No | false | Auto-scaling |
## Parameter Validation
Parameters are validated on startup:
1. Required parameters must be present
2. Types are checked and coerced
3. Ranges are enforced
4. Dependencies verified
5. Conflicts detected
## Environment Variable Override
Any parameter can be overridden via environment:
```bash
BOT_TITLE="My Bot" BOT_LLM_MODEL="gpt-4-turbo" botserver
```
## Dynamic Parameter Updates
Some parameters can be updated at runtime:
- Log level
- Rate limits
- Cache settings
- Feature flags
Use the admin API to update:
```
POST /api/admin/config
{
"logLevel": "debug",
"rateLimitPerMinute": 120
}
```
## Best Practices
1. **Start with defaults**: Most parameters have sensible defaults
2. **Override only what's needed**: Don't set everything
3. **Use environment variables**: For sensitive values
4. **Document custom values**: Explain why changed
5. **Test configuration**: Validate before production
6. **Monitor performance**: Adjust based on metrics
7. **Version control**: Track configuration changes

339
docs/src/chapter-09/web-automation.md
View file

@ -1 +1,340 @@
# Web Automation # Web Automation
The web automation module enables BotServer to interact with websites, extract content, and perform automated browser tasks.
## Overview
Web automation features allow bots to:
- Crawl and index website content
- Extract structured data from web pages
- Automate form submissions
- Capture screenshots
- Monitor website changes
- Perform headless browser operations
## Configuration
Enable web automation in `config.csv`:
```csv
webAutomationEnabled,true
browserTimeout,30000
maxCrawlDepth,3
userAgent,BotServer/1.0
```
## Features
### Website Crawling
The `ADD_WEBSITE` keyword triggers web crawling:
```basic
ADD_WEBSITE "https://example.com"
```
This will:
1. Launch headless browser
2. Navigate to the URL
3. Extract text content
4. Follow internal links (respecting robots.txt)
5. Index content in vector database
6. Make content searchable via `FIND` keyword
### Content Extraction
Extract specific data from web pages:
```basic
url = "https://news.example.com"
content = GET url
headlines = EXTRACT_CSS content, "h2.headline"
```
### Form Automation
Submit forms programmatically:
```basic
NAVIGATE "https://example.com/contact"
FILL_FIELD "name", customer_name
FILL_FIELD "email", customer_email
FILL_FIELD "message", inquiry_text
CLICK_BUTTON "submit"
result = GET_PAGE_TEXT()
```
### Screenshot Capture
Capture visual representations:
```basic
NAVIGATE "https://example.com/dashboard"
screenshot = CAPTURE_SCREENSHOT()
SAVE_FILE screenshot, "dashboard.png"
```
### Change Monitoring
Monitor websites for updates:
```basic
SET_MONITOR "https://example.com/status", "hourly"
ON "website_changed" DO
changes = GET_CHANGES()
SEND_MAIL admin_email, "Website Updated", changes
END ON
```
## Crawler Configuration
### Crawl Rules
Control crawler behavior:
| Setting | Description | Default |
|---------|-------------|---------|
| `maxDepth` | Maximum crawl depth | 3 |
| `maxPages` | Maximum pages to crawl | 100 |
| `crawlDelay` | Delay between requests (ms) | 1000 |
| `respectRobots` | Honor robots.txt | true |
| `followRedirects` | Follow HTTP redirects | true |
| `includeImages` | Extract image URLs | false |
| `includePDFs` | Process PDF links | true |
### Selector Strategies
Extract content using CSS selectors:
```basic
' Extract specific elements
titles = EXTRACT_CSS page, "h1, h2, h3"
paragraphs = EXTRACT_CSS page, "p"
links = EXTRACT_CSS page, "a[href]"
images = EXTRACT_CSS page, "img[src]"
```
Or XPath expressions:
```basic
' XPath extraction
prices = EXTRACT_XPATH page, "//span[@class='price']"
reviews = EXTRACT_XPATH page, "//div[@class='review-text']"
```
## Browser Automation
### Navigation
Control browser navigation:
```basic
NAVIGATE "https://example.com"
WAIT_FOR_ELEMENT "#content"
SCROLL_TO_BOTTOM()
BACK()
FORWARD()
REFRESH()
```
### Interaction
Interact with page elements:
```basic
CLICK "#login-button"
TYPE "#username", user_credentials
SELECT "#country", "USA"
CHECK "#agree-terms"
UPLOAD_FILE "#document", "report.pdf"
```
### Waiting Strategies
Wait for specific conditions:
```basic
WAIT_FOR_ELEMENT "#results"
WAIT_FOR_TEXT "Loading complete"
WAIT_FOR_URL "success"
WAIT_SECONDS 3
```
## Data Processing
### Structured Data Extraction
Extract structured data from pages:
```basic
products = EXTRACT_TABLE "#product-list"
FOR EACH product IN products
SAVE_TO_DB product.name, product.price, product.stock
NEXT
```
### Content Cleaning
Clean extracted content:
```basic
raw_text = GET_PAGE_TEXT()
clean_text = REMOVE_HTML(raw_text)
clean_text = REMOVE_SCRIPTS(clean_text)
clean_text = NORMALIZE_WHITESPACE(clean_text)
```
## Performance Optimization
### Caching
Cache crawled content:
```basic
IF NOT IN_CACHE(url) THEN
content = CRAWL_URL(url)
CACHE_SET(url, content, "1 hour")
ELSE
content = CACHE_GET(url)
END IF
```
### Parallel Processing
Process multiple URLs concurrently:
```basic
urls = ["url1", "url2", "url3"]
results = PARALLEL_CRAWL(urls, max_workers=5)
```
## Security Considerations
### Authentication
Handle authenticated sessions:
```basic
LOGIN "https://example.com/login", username, password
cookie = GET_COOKIE("session")
' Use cookie for subsequent requests
NAVIGATE "https://example.com/dashboard"
```
### Rate Limiting
Respect rate limits:
```basic
CONFIGURE_CRAWLER(
rate_limit = 10, ' requests per second
user_agent = "BotServer/1.0",
timeout = 30000
)
```
### Content Filtering
Filter inappropriate content:
```basic
content = CRAWL_URL(url)
IF CONTAINS_INAPPROPRIATE(content) THEN
LOG_WARNING "Inappropriate content detected"
SKIP_URL(url)
END IF
```
## Error Handling
Handle common web automation errors:
```basic
TRY
content = CRAWL_URL(url)
CATCH "timeout"
LOG "Page load timeout: " + url
RETRY_WITH_DELAY(5000)
CATCH "404"
LOG "Page not found: " + url
MARK_AS_BROKEN(url)
CATCH "blocked"
LOG "Access blocked, might need CAPTCHA"
USE_PROXY()
END TRY
```
## Integration with Knowledge Base
Automatically index crawled content:
```basic
ADD_WEBSITE "https://docs.example.com"
' Content is automatically indexed
' Later, search the indexed content
answer = FIND "installation guide"
TALK answer
```
## Monitoring and Logging
Track automation activities:
```basic
START_MONITORING()
result = CRAWL_URL(url)
metrics = GET_METRICS()
LOG "Pages crawled: " + metrics.page_count
LOG "Time taken: " + metrics.duration
LOG "Data extracted: " + metrics.data_size
```
## Best Practices
1. **Respect robots.txt**: Always honor website crawling rules
2. **Use appropriate delays**: Don't overwhelm servers
3. **Handle errors gracefully**: Implement retry logic
4. **Cache when possible**: Reduce redundant requests
5. **Monitor performance**: Track crawling metrics
6. **Secure credentials**: Never hardcode passwords
7. **Test selectors**: Verify CSS/XPath selectors work
8. **Clean data**: Remove unnecessary HTML/scripts
9. **Set timeouts**: Prevent infinite waiting
10. **Log activities**: Maintain audit trail
## Limitations
- JavaScript-heavy sites may require additional configuration
- CAPTCHA-protected sites need manual intervention
- Some sites block automated access
- Large-scale crawling requires distributed setup
- Dynamic content may need special handling
## Troubleshooting
### Page Not Loading
- Check network connectivity
- Verify URL is accessible
- Increase timeout values
- Check for JavaScript requirements
### Content Not Found
- Verify CSS selectors are correct
- Check if content is dynamically loaded
- Wait for elements to appear
- Use browser developer tools to test
### Access Denied
- Check user agent settings
- Verify authentication credentials
- Respect rate limits
- Consider using proxies
## Implementation
The web automation module is located in `src/web_automation/` and uses:
- Headless browser engine for rendering
- HTML parsing for content extraction
- Request throttling for rate limiting
- Vector database for content indexing

1
docs/src/chapter-12/admin-api.md Normal file
View file

@ -0,0 +1 @@
# Admin API

1
docs/src/chapter-12/ai-api.md Normal file
View file

@ -0,0 +1 @@
# AI API

1
docs/src/chapter-12/analytics-api.md Normal file
View file

@ -0,0 +1 @@
# Analytics API

1
docs/src/chapter-12/calendar-api.md Normal file
View file

@ -0,0 +1 @@
# Calendar API

1
docs/src/chapter-12/calls-api.md Normal file
View file

@ -0,0 +1 @@
# Calls API

1
docs/src/chapter-12/conversations-api.md Normal file
View file

@ -0,0 +1 @@
# Conversations API

1
docs/src/chapter-12/document-processing.md Normal file
View file

@ -0,0 +1 @@
# Document Processing API

1
docs/src/chapter-12/email-api.md Normal file
View file

@ -0,0 +1 @@
# Email API

1
docs/src/chapter-12/examples.md Normal file
View file

@ -0,0 +1 @@
# Example Integrations

1
docs/src/chapter-12/group-membership.md Normal file
View file

@ -0,0 +1 @@
# Group Membership API

1
docs/src/chapter-12/groups-api.md Normal file
View file

@ -0,0 +1 @@
# Groups API

1
docs/src/chapter-12/ml-api.md Normal file
View file

@ -0,0 +1 @@
# ML API

1
docs/src/chapter-12/monitoring-api.md Normal file
View file

@ -0,0 +1 @@
# Monitoring API

1
docs/src/chapter-12/notifications-api.md Normal file
View file

@ -0,0 +1 @@
# Notifications API

1
docs/src/chapter-12/reports-api.md Normal file
View file

@ -0,0 +1 @@
# Reports API

1
docs/src/chapter-12/storage-api.md Normal file
View file

@ -0,0 +1 @@
# Storage API

1
docs/src/chapter-12/tasks-api.md Normal file
View file

@ -0,0 +1 @@
# Tasks API

1
docs/src/chapter-12/user-security.md Normal file
View file

@ -0,0 +1 @@
# User Security API

1
docs/src/chapter-12/users-api.md Normal file
View file

@ -0,0 +1 @@
# Users API

1
docs/src/chapter-12/whiteboard-api.md Normal file
View file

@ -0,0 +1 @@
# Whiteboard API