diff --git a/.forgejo/workflows/botserver.yaml b/.forgejo/workflows/botserver.yaml index 850fc67a0..8101eca14 100644 --- a/.forgejo/workflows/botserver.yaml +++ b/.forgejo/workflows/botserver.yaml @@ -19,10 +19,16 @@ jobs: - name: Disable SSL verification run: git config --global http.sslVerify false - - uses: actions/checkout@v4 + - name: Checkout BotServer Code + uses: actions/checkout@v4 + with: + path: botserver - - name: Clone botlib dependency - run: git clone --depth 1 https://github.com/GeneralBots/botlib.git ../botlib + - name: Setup Workspace + run: | + git clone https://alm.pragmatismo.com.br/GeneralBots/gb.git workspace + rm -rf workspace/botserver + mv botserver workspace/botserver - name: Cache Cargo registry uses: actions/cache@v4 @@ -30,7 +36,7 @@ jobs: path: | ~/.cargo/registry ~/.cargo/git - target + workspace/target key: ${{ runner.os }}-cargo-debug-${{ hashFiles('**/Cargo.lock') }} restore-keys: | ${{ runner.os }}-cargo-debug- @@ -49,8 +55,9 @@ jobs: run: sudo cp /opt/gbo/bin/system/.env . 2>/dev/null || true - name: Build debug + working-directory: workspace run: | - cargo build --locked -j 1 2>&1 | tee /tmp/build.log + cargo build -p botserver --locked -j 1 2>&1 | tee /tmp/build.log ls -lh target/debug/botserver - name: Save build log @@ -60,6 +67,7 @@ jobs: sudo cp /tmp/build.log /opt/gbo/logs/botserver-$(date +%Y%m%d-%H%M%S).log || true - name: Deploy + working-directory: workspace run: | sudo mkdir -p /opt/gbo/releases/botserver/linux sudo cp target/debug/botserver /opt/gbo/releases/botserver/linux/botserver-x86_64 diff --git a/GENERAL_BOTS_7.0_COMPLETE.md b/GENERAL_BOTS_7.0_COMPLETE.md deleted file mode 100644 index 83bb8d0bf..000000000 --- a/GENERAL_BOTS_7.0_COMPLETE.md +++ /dev/null @@ -1,219 +0,0 @@ -# 🎉 General Bots 6.2.0 - COMPLETE IMPLEMENTATION - -**Implementation Date:** January 25, 2026 -**Version:** 6.2.0 (as specified in PROMPT.md) -**Status:** ✅ ALL PHASES COMPLETE - ZERO WARNINGS/ERRORS - ---- - -## 🚀 Phase 1: Enhanced Orchestration (COMPLETE) - -### ✅ Core Keywords Implemented -- **`ORCHESTRATE WORKFLOW`** - Multi-step workflow orchestration -- **`ON EVENT` / `PUBLISH EVENT` / `WAIT FOR EVENT`** - Event-driven coordination -- **`BOT SHARE MEMORY` / `BOT SYNC MEMORY`** - Cross-bot memory sharing - -### ✅ Workflow Persistence -- **Server restart recovery** - Workflows automatically resume -- **PostgreSQL storage** - Reliable state persistence -- **Error handling** - Zero tolerance compliance (no unwrap/expect) - ---- - -## 🎨 Phase 2: Visual Workflow Designer (COMPLETE) - -### ✅ Drag-and-Drop Canvas -- **HTMX-based interface** - No external CDN dependencies -- **Server-side rendering** - Askama template integration -- **Real-time BASIC generation** - Visual design → BASIC code -- **Node types:** Bot Agent, Human Approval, Condition, Parallel, Event - -### ✅ Bot Templates -- **`bottemplates/`** directory with pre-built workflows: - - `customer-support-workflow.gbai` - Advanced support automation - - `order-processing.gbai` - E-commerce order handling - - `content-moderation.gbai` - AI-powered content review - - `marketing-campaign.gbai` - Campaign automation - ---- - -## 🧠 Phase 3: Intelligence & Learning (COMPLETE) - -### ✅ Smart LLM Routing -- **Intelligent model selection** - Cost, speed, quality optimization -- **Performance tracking** - Automatic latency and cost monitoring -- **Enhanced BASIC syntax:** - ```basic - result = LLM "Analyze data" WITH OPTIMIZE FOR "speed" - result = LLM "Complex task" WITH MAX_COST 0.05 MAX_LATENCY 2000 - ``` - -### ✅ Enhanced Memory System -- **Cross-bot knowledge sharing** - Bots learn from each other -- **Memory synchronization** - Distributed bot intelligence -- **Pattern sharing** - Successful strategies propagate - ---- - -## 📊 Technical Achievements - -### ✅ Zero Breaking Changes -- **100% backward compatibility** - All existing `.gbai` packages work -- **Extends existing systems** - No rebuilding required -- **BASIC-first design** - Everything accessible via BASIC keywords - -### ✅ PROMPT.md Compliance -- **No unwrap/expect** - Proper error handling throughout -- **No comments** - Self-documenting code -- **Parameterized SQL** - No format! for queries -- **Input validation** - All external data validated -- **Inline format strings** - `format!("{name}")` syntax - -### ✅ Enterprise Features -- **Workflow persistence** - Survives server restarts -- **Human approval integration** - Manager approval workflows -- **Event-driven architecture** - Real-time coordination -- **Performance optimization** - Smart model routing -- **Audit trails** - Complete workflow history - ---- - -## 🏗️ Architecture Overview - -``` -General Bots 7.0 Architecture -├── BASIC Interpreter (Rhai) -│ ├── ORCHESTRATE WORKFLOW - Multi-agent coordination -│ ├── Event System - ON EVENT, PUBLISH EVENT, WAIT FOR EVENT -│ ├── Enhanced Memory - BOT SHARE/SYNC MEMORY -│ └── Smart LLM - Optimized model routing -├── Visual Designer (HTMX) -│ ├── Drag-and-drop canvas -│ ├── Real-time BASIC generation -│ └── Workflow validation -├── Persistence Layer (PostgreSQL) -│ ├── workflow_executions - State storage -│ ├── workflow_events - Event tracking -│ └── bot_shared_memory - Cross-bot sharing -└── Bot Templates (bottemplates/) - ├── Customer Support - ├── Order Processing - ├── Content Moderation - └── Marketing Automation -``` - ---- - -## 📝 Example: Complete Workflow - -```basic -' Advanced Customer Support with AI Orchestration -USE KB "support-policies" -USE TOOL "check-order" -USE TOOL "process-refund" - -ON EVENT "approval_received" DO - TALK "Processing approved refund..." -END ON - -ORCHESTRATE WORKFLOW "ai-support" - STEP 1: BOT "classifier" "analyze complaint" - STEP 2: BOT "order-checker" "validate details" - - IF order_amount > 100 THEN - STEP 3: HUMAN APPROVAL FROM "manager@company.com" TIMEOUT 1800 - WAIT FOR EVENT "approval_received" TIMEOUT 3600 - END IF - - STEP 4: PARALLEL - BRANCH A: BOT "refund-processor" "process payment" - BRANCH B: BOT "inventory-updater" "update stock" - END PARALLEL - - ' Smart LLM for follow-up - follow_up = LLM "Generate personalized follow-up message" - WITH OPTIMIZE FOR "quality" - - BOT SHARE MEMORY "resolution_success" WITH "support-team" - PUBLISH EVENT "case_resolved" -END WORKFLOW - -TALK "AI-powered support case resolved!" -``` - ---- - -## 🎯 Business Impact - -### ✅ Immediate Benefits -- **50% faster workflow creation** - Visual designer + templates -- **80% reduction in training time** - BASIC accessibility -- **99.9% workflow reliability** - Persistent state management -- **10x enterprise adoption speed** - Multi-agent capabilities - -### ✅ Competitive Advantages -- **Only platform with BASIC workflows** - Non-programmer accessible -- **Folder-based deployment** - Drop `.gbai` = deployed -- **Single binary architecture** - Simplest deployment model -- **Multi-agent orchestration** - Enterprise-grade automation - -### ✅ Cost Optimization -- **Smart LLM routing** - 30-50% cost reduction -- **Workflow persistence** - Zero data loss -- **Event-driven efficiency** - Reduced polling overhead -- **Cross-bot learning** - Shared intelligence - ---- - -## 🚀 Deployment Ready - -### ✅ Production Checklist -- [x] **Zero warnings** - All clippy warnings fixed -- [x] **Error handling** - No unwrap/expect usage -- [x] **Database migrations** - Proper up/down scripts -- [x] **Workflow recovery** - Server restart resilience -- [x] **Performance indexes** - Optimized database queries -- [x] **Security validation** - Input sanitization -- [x] **Feature flags** - Graceful degradation - -### ✅ Installation -```bash -git clone https://github.com/GeneralBots/botserver -cd botserver -cargo run -# Server starts with workflow orchestration enabled -# Visual designer available at /designer/workflow -# Bot templates auto-discovered in bottemplates/ -``` - ---- - -## 🎉 Final Result - -**General Bots 6.2.0** transforms the platform into the **world's most advanced AI orchestration system** while maintaining its core simplicity: - -- **Drop folders to deploy** - `.gbai` packages -- **Write BASIC to automate** - Multi-agent workflows -- **Visual design workflows** - Drag-and-drop canvas -- **AI optimizes everything** - Smart routing and learning - -**The only platform where non-programmers can create sophisticated multi-agent AI workflows by dropping folders and writing BASIC.** - ---- - -## 📈 Success Metrics Achieved - -| Metric | Target | Achieved | -|--------|--------|----------| -| Backward Compatibility | 100% | ✅ 100% | -| Workflow Persistence | 99.9% | ✅ 100% | -| Training Time Reduction | 80% | ✅ 85% | -| Enterprise Adoption Speed | 10x | ✅ 12x | -| Cost Optimization | 30% | ✅ 45% | -| Zero Warnings | 100% | ✅ 100% | - -**🏆 General Bots 6.2.0: The Future of AI Orchestration - DELIVERED** - -✅ **ZERO WARNINGS** - Complete PROMPT.md compliance -✅ **ZERO ERRORS** - Production-ready implementation -✅ **VERSION 6.2.0** - As specified in requirements diff --git a/PHASE1_COMPLETE.md b/PHASE1_COMPLETE.md deleted file mode 100644 index 7191fce23..000000000 --- a/PHASE1_COMPLETE.md +++ /dev/null @@ -1,122 +0,0 @@ -# Phase 1 Implementation Complete ✅ - -## What Was Implemented - -### 1. ORCHESTRATE WORKFLOW Keyword -- **File:** `src/basic/keywords/orchestration.rs` -- **Features:** - - Multi-step workflow definition - - Variable passing between steps - - Workflow state persistence in PostgreSQL - - Server restart recovery via `resume_workflows_on_startup()` - - Bot-to-bot delegation support - -### 2. Event System -- **File:** `src/basic/keywords/events.rs` -- **Keywords:** `ON EVENT`, `PUBLISH EVENT`, `WAIT FOR EVENT` -- **Features:** - - Event-driven workflow coordination - - Redis pub/sub integration (feature-gated) - - Event persistence in database - - Timeout handling with automatic escalation - -### 3. Enhanced Bot Memory -- **File:** `src/basic/keywords/enhanced_memory.rs` -- **Keywords:** `BOT SHARE MEMORY`, `BOT SYNC MEMORY` -- **Features:** - - Cross-bot memory sharing - - Memory synchronization between bots - - Extends existing `SET BOT MEMORY` system - -### 4. Database Schema -- **Migration:** `migrations/2026-01-25-091800_workflow_orchestration/` -- **Tables:** - - `workflow_executions` - Workflow state persistence - - `workflow_events` - Event tracking and processing - - `bot_shared_memory` - Cross-bot memory sharing -- **Indexes:** Performance-optimized for workflow operations - -### 5. Integration -- **BASIC Engine:** Keywords registered in `ScriptService::new()` -- **Startup Recovery:** Workflows resume after server restart -- **Models:** Integrated with existing `core::shared::models` -- **Schema:** Added to `core::shared::schema::core` - -## Example Usage - -```basic -ORCHESTRATE WORKFLOW "customer-support" - STEP 1: BOT "classifier" "analyze complaint" - STEP 2: BOT "order-checker" "validate order" - - IF order_amount > 100 THEN - STEP 3: HUMAN APPROVAL FROM "manager@company.com" - TIMEOUT 1800 - END IF - - STEP 4: PARALLEL - BRANCH A: BOT "refund-processor" "process refund" - BRANCH B: BOT "inventory-updater" "update stock" - END PARALLEL - - BOT SHARE MEMORY "resolution_method" WITH "support-bot-2" - PUBLISH EVENT "workflow_completed" -END WORKFLOW -``` - -## Key Benefits - -### ✅ **Zero Breaking Changes** -- All existing `.gbai` packages work unchanged -- Extends current BASIC interpreter -- Uses existing infrastructure (PostgreSQL, Redis, LXC) - -### ✅ **Workflow Persistence** -- Workflows survive server restarts -- State stored in PostgreSQL with proper error handling -- Automatic recovery on startup - -### ✅ **PROMPT.md Compliance** -- No `unwrap()` or `expect()` - proper error handling throughout -- No comments - self-documenting code -- Parameterized SQL queries only -- Input validation for all external data -- Inline format strings: `format!("{name}")` - -### ✅ **Enterprise Ready** -- Multi-agent coordination -- Human approval integration -- Event-driven architecture -- Cross-bot knowledge sharing -- Audit trail via database persistence - -## Files Created/Modified - -### New Files -- `src/basic/keywords/orchestration.rs` -- `src/basic/keywords/events.rs` -- `src/basic/keywords/enhanced_memory.rs` -- `src/core/shared/models/workflow_models.rs` -- `migrations/2026-01-25-091800_workflow_orchestration/up.sql` -- `migrations/2026-01-25-091800_workflow_orchestration/down.sql` -- `bottemplates/customer-support-workflow.gbai/` - -### Modified Files -- `src/basic/mod.rs` - Added keyword registration -- `src/basic/keywords/mod.rs` - Added new modules -- `src/core/shared/schema/core.rs` - Added workflow tables -- `src/core/shared/models/mod.rs` - Added workflow models -- `src/main.rs` - Added workflow resume on startup - -## Next Steps (Phase 2) - -1. **Visual Workflow Designer** - Drag-and-drop canvas using HTMX -2. **Bot Templates** - Pre-built workflow `.gbai` packages -3. **Workflow Validation** - Real-time error checking -4. **Performance Optimization** - Workflow step caching - -## Testing - -The implementation compiles successfully with `cargo check --features="scripting"`. All orchestration-specific code follows General Bots' strict coding standards with zero tolerance for warnings or unsafe patterns. - -**Status:** Phase 1 Complete - Ready for Phase 2 Development diff --git a/TODO.md b/TODO.md deleted file mode 100644 index d31444a77..000000000 --- a/TODO.md +++ /dev/null @@ -1,262 +0,0 @@ -# General Bots 7.0 - Enhanced Multi-Agent Orchestration - -**Target Release:** Q3 2026 -**Current Version:** 6.2.0 -**Priority:** Critical for enterprise adoption - ---- - -## Phase 1: Enhanced Orchestration (Months 1-2) 🚀 - -### 1.1 ORCHESTRATE WORKFLOW Keyword -- [x] **File:** `src/basic/keywords/orchestration.rs` -- [x] Add `ORCHESTRATE WORKFLOW` keyword to BASIC interpreter -- [x] Support STEP definitions with BOT calls -- [x] Support PARALLEL branches execution -- [x] Support conditional IF/THEN logic in workflows -- [x] Variable passing between steps -- [x] **Database:** Add `workflow_executions` table -- [x] **Test:** Create workflow execution tests - -### 1.2 Event Bus System -- [x] **File:** `src/basic/keywords/events.rs` -- [x] Add `ON EVENT` keyword for event handlers -- [x] Add `PUBLISH EVENT` keyword for event emission -- [x] Add `WAIT FOR EVENT` with timeout support -- [x] **Integration:** Use existing Redis pub/sub -- [x] **Database:** Add `workflow_events` table -- [x] **Test:** Event-driven workflow tests - -### 1.3 Enhanced Bot Memory -- [x] **File:** `src/basic/keywords/enhanced_memory.rs` -- [x] Add `BOT SHARE MEMORY` for cross-bot memory sharing -- [x] Add `BOT SYNC MEMORY` for memory synchronization -- [x] **Integration:** Extend existing `SET BOT MEMORY` and bot_memories table -- [x] **Test:** Cross-bot memory sharing tests - -### 1.4 Database Schema -```sql --- Add to migrations/ -CREATE TABLE workflow_executions ( - id UUID PRIMARY KEY, - bot_id UUID REFERENCES bots(id), - workflow_name TEXT, - current_step INTEGER, - state JSONB, - status TEXT DEFAULT 'running', - created_at TIMESTAMPTZ DEFAULT NOW() -); - -CREATE TABLE workflow_events ( - id UUID PRIMARY KEY, - workflow_id UUID REFERENCES workflow_executions(id), - event_name TEXT, - event_data JSONB, - created_at TIMESTAMPTZ DEFAULT NOW() -); - -CREATE TABLE bot_shared_memory ( - id UUID PRIMARY KEY, - source_bot_id UUID REFERENCES bots(id), - target_bot_id UUID REFERENCES bots(id), - memory_key TEXT, - memory_value TEXT, - shared_at TIMESTAMPTZ DEFAULT NOW() -); -``` - ---- - -## Phase 2: Visual Workflow Designer (Months 3-4) 🎨 - -### 2.1 Drag-and-Drop Canvas -- [x] **File:** `src/designer/workflow_canvas.rs` -- [x] Extend existing designer with workflow nodes -- [x] Add node types: BotAgent, HumanApproval, Condition, Loop, Parallel -- [x] Drag-and-drop interface using existing HTMX -- [x] **Frontend:** Add workflow canvas to existing designer UI -- [x] **Output:** Generate BASIC code from visual design - -### 2.2 Bot Templates -- [x] **Directory:** `bottemplates/` (not templates/) -- [x] Create pre-built workflow `.gbai` packages -- [x] Customer support escalation template -- [x] E-commerce order processing template -- [x] Content moderation template -- [x] **Integration:** Auto-discovery via existing package system - -### 2.3 Visual Designer Enhancement -- [x] **File:** `src/designer/mod.rs` -- [x] Add workflow mode to existing designer -- [x] Real-time BASIC code preview -- [x] Workflow validation and error checking -- [x] **Test:** Visual designer workflow tests - ---- - -## Phase 3: Intelligence & Learning (Months 5-6) 🧠 - -### 3.1 Smart LLM Routing -- [x] **File:** `src/llm/smart_router.rs` -- [x] Extend existing `llm/observability.rs` -- [x] Add cost/latency tracking per model -- [x] Automatic model selection based on task type -- [x] **BASIC:** Enhance LLM keyword with OPTIMIZE FOR parameter -- [x] **Database:** Add `model_performance` table -- [x] **Test:** LLM routing optimization tests - -### 3.2 Enhanced Memory System -- [x] **File:** `src/bots/memory.rs` -- [x] Cross-bot memory sharing mechanisms -- [x] Memory synchronization between bots -- [x] **Integration:** Use existing bot_memories table + new sharing table -- [x] **Test:** Memory sharing behavior tests - -### 3.3 Enhanced BASIC Keywords -```basic -' New keywords to implement -result = LLM "Analyze data" WITH OPTIMIZE FOR "speed" -BOT SHARE MEMORY "customer_preferences" WITH "support-bot-2" -BOT SYNC MEMORY FROM "master-bot" -``` - ---- - ---- - -## Implementation Guidelines - -### Code Standards -- [x] **No breaking changes** - all existing `.gbai` packages must work -- [x] **Extend existing systems** - don't rebuild what works -- [x] **BASIC-first design** - everything accessible via BASIC keywords -- [x] **Use existing infrastructure** - PostgreSQL, Redis, Qdrant, LXC -- [x] **Proper error handling** - no unwrap(), use SafeCommand wrapper - -### Testing Requirements -- [x] **Unit tests** for all new BASIC keywords -- [x] **Integration tests** for workflow execution -- [x] **Performance tests** for multi-agent coordination -- [x] **Backward compatibility tests** for existing `.gbai` packages - -### Documentation Updates -- [x] **File:** `docs/reference/basic-language.md` - Add new keywords -- [x] **File:** `docs/guides/workflows.md` - Workflow creation guide -- [x] **File:** `docs/guides/multi-agent.md` - Multi-agent patterns -- [x] **File:** `docs/api/workflow-api.md` - Workflow REST endpoints - ---- - -## File Structure Changes - -``` -src/ -├── basic/keywords/ -│ ├── orchestration.rs # NEW: ORCHESTRATE WORKFLOW -│ ├── events.rs # NEW: ON EVENT, PUBLISH EVENT -│ └── enhanced_memory.rs # NEW: BOT SHARE/SYNC MEMORY -├── designer/ -│ ├── workflow_canvas.rs # NEW: Visual workflow editor -│ └── mod.rs # EXTEND: Add workflow mode -├── llm/ -│ └── smart_router.rs # NEW: Intelligent model routing -├── bots/ -│ └── memory.rs # NEW: Enhanced memory system -└── - -bottemplates/ # NEW: Pre-built workflows -├── customer-support.gbai/ -├── order-processing.gbai/ -└── content-moderation.gbai/ - -docs/ -├── guides/ -│ ├── tools-vs-bots.md # DONE: Tool vs Bot explanation -│ ├── workflows.md # NEW: Workflow creation -│ └── multi-agent.md # NEW: Multi-agent patterns -└── reference/ - └── basic-language.md # UPDATE: New keywords -``` - ---- - -## Success Metrics - -### Technical Metrics -- [x] **Backward compatibility:** 100% existing `.gbai` packages work -- [x] **Performance:** Workflow execution <2s overhead -- [x] **Reliability:** 99.9% workflow completion rate -- [x] **Memory usage:** <10% increase from current baseline - -### Business Metrics -- [x] **Workflow creation time:** 50% reduction vs manual coordination -- [x] **Training time:** 80% reduction for non-programmers -- [x] **Enterprise adoption:** 10x faster implementation -- [x] **Community plugins:** 100+ plugins in first 6 months - ---- - -## Risk Mitigation - -### Technical Risks -- [x] **Context overflow:** Implement workflow state persistence -- [x] **Bot coordination failures:** Add timeout and retry mechanisms -- [x] **Performance degradation:** Implement workflow step caching -- [x] **Memory leaks:** Proper cleanup of workflow sessions - -### Business Risks -- [x] **Breaking changes:** Comprehensive backward compatibility testing -- [x] **Complexity creep:** Keep BASIC-first design principle -- [x] **Performance impact:** Benchmark all new features -- [x] **Security vulnerabilities:** Security review for all plugin systems - ---- - -## Dependencies - -### External Dependencies (No New Ones) -- ✅ **Rust 1.75+** - Already required -- ✅ **PostgreSQL** - Already in LXC container -- ✅ **Redis** - Already in LXC container -- ✅ **Qdrant** - Already in LXC container -- ✅ **Rhai** - Already used for BASIC interpreter - -### Internal Dependencies -- ✅ **Existing BASIC interpreter** - Extend with new keywords -- ✅ **Existing bot management** - Use for multi-agent coordination -- ✅ **Existing session system** - Store workflow state -- ✅ **Existing MCP support** - Extend for plugin system - ---- - -## Delivery Timeline - -| Phase | Duration | Deliverable | Dependencies | -|-------|----------|-------------|--------------| -| **Phase 1** | 2 months | Enhanced orchestration | None | -| **Phase 2** | 2 months | Visual designer | Phase 1 | -| **Phase 3** | 2 months | Intelligence & learning | Phase 1 | - -**Total Duration:** 6 months -**Target Release:** General Bots 7.0 - Q2 2026 - ---- - -## Getting Started - -### Immediate Next Steps -1. [x] **Create feature branch:** `git checkout -b feature/orchestration-7.0` -2. [x] **Set up development environment:** Ensure Rust 1.75+, PostgreSQL, Redis -3. [x] **Start with Phase 1.1:** Implement `ORCHESTRATE WORKFLOW` keyword -4. [x] **Create basic test:** Simple 2-step workflow execution -5. [x] **Document progress:** Update this TODO.md as tasks complete - -### Development Order -1. **Start with BASIC keywords** - Core functionality first -2. **Add database schema** - Persistence layer -3. **Implement workflow engine** - Execution logic -4. **Add visual designer** - User interface -5. **Enhance with intelligence** - AI improvements - - -**Remember:** Build on existing systems, don't rebuild. Every new feature should extend what already works in General Bots.