Update botserver CI workflow to use workspace structure
This commit is contained in:
parent
9087bb17cd
commit
fe8579e7b2
4 changed files with 13 additions and 608 deletions
|
|
@ -19,10 +19,16 @@ jobs:
|
||||||
- name: Disable SSL verification
|
- name: Disable SSL verification
|
||||||
run: git config --global http.sslVerify false
|
run: git config --global http.sslVerify false
|
||||||
|
|
||||||
- uses: actions/checkout@v4
|
- name: Checkout BotServer Code
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
path: botserver
|
||||||
|
|
||||||
- name: Clone botlib dependency
|
- name: Setup Workspace
|
||||||
run: git clone --depth 1 https://github.com/GeneralBots/botlib.git ../botlib
|
run: |
|
||||||
|
git clone https://alm.pragmatismo.com.br/GeneralBots/gb.git workspace
|
||||||
|
rm -rf workspace/botserver
|
||||||
|
mv botserver workspace/botserver
|
||||||
|
|
||||||
- name: Cache Cargo registry
|
- name: Cache Cargo registry
|
||||||
uses: actions/cache@v4
|
uses: actions/cache@v4
|
||||||
|
|
@ -30,7 +36,7 @@ jobs:
|
||||||
path: |
|
path: |
|
||||||
~/.cargo/registry
|
~/.cargo/registry
|
||||||
~/.cargo/git
|
~/.cargo/git
|
||||||
target
|
workspace/target
|
||||||
key: ${{ runner.os }}-cargo-debug-${{ hashFiles('**/Cargo.lock') }}
|
key: ${{ runner.os }}-cargo-debug-${{ hashFiles('**/Cargo.lock') }}
|
||||||
restore-keys: |
|
restore-keys: |
|
||||||
${{ runner.os }}-cargo-debug-
|
${{ runner.os }}-cargo-debug-
|
||||||
|
|
@ -49,8 +55,9 @@ jobs:
|
||||||
run: sudo cp /opt/gbo/bin/system/.env . 2>/dev/null || true
|
run: sudo cp /opt/gbo/bin/system/.env . 2>/dev/null || true
|
||||||
|
|
||||||
- name: Build debug
|
- name: Build debug
|
||||||
|
working-directory: workspace
|
||||||
run: |
|
run: |
|
||||||
cargo build --locked -j 1 2>&1 | tee /tmp/build.log
|
cargo build -p botserver --locked -j 1 2>&1 | tee /tmp/build.log
|
||||||
ls -lh target/debug/botserver
|
ls -lh target/debug/botserver
|
||||||
|
|
||||||
- name: Save build log
|
- name: Save build log
|
||||||
|
|
@ -60,6 +67,7 @@ jobs:
|
||||||
sudo cp /tmp/build.log /opt/gbo/logs/botserver-$(date +%Y%m%d-%H%M%S).log || true
|
sudo cp /tmp/build.log /opt/gbo/logs/botserver-$(date +%Y%m%d-%H%M%S).log || true
|
||||||
|
|
||||||
- name: Deploy
|
- name: Deploy
|
||||||
|
working-directory: workspace
|
||||||
run: |
|
run: |
|
||||||
sudo mkdir -p /opt/gbo/releases/botserver/linux
|
sudo mkdir -p /opt/gbo/releases/botserver/linux
|
||||||
sudo cp target/debug/botserver /opt/gbo/releases/botserver/linux/botserver-x86_64
|
sudo cp target/debug/botserver /opt/gbo/releases/botserver/linux/botserver-x86_64
|
||||||
|
|
|
||||||
|
|
@ -1,219 +0,0 @@
|
||||||
# 🎉 General Bots 6.2.0 - COMPLETE IMPLEMENTATION
|
|
||||||
|
|
||||||
**Implementation Date:** January 25, 2026
|
|
||||||
**Version:** 6.2.0 (as specified in PROMPT.md)
|
|
||||||
**Status:** ✅ ALL PHASES COMPLETE - ZERO WARNINGS/ERRORS
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚀 Phase 1: Enhanced Orchestration (COMPLETE)
|
|
||||||
|
|
||||||
### ✅ Core Keywords Implemented
|
|
||||||
- **`ORCHESTRATE WORKFLOW`** - Multi-step workflow orchestration
|
|
||||||
- **`ON EVENT` / `PUBLISH EVENT` / `WAIT FOR EVENT`** - Event-driven coordination
|
|
||||||
- **`BOT SHARE MEMORY` / `BOT SYNC MEMORY`** - Cross-bot memory sharing
|
|
||||||
|
|
||||||
### ✅ Workflow Persistence
|
|
||||||
- **Server restart recovery** - Workflows automatically resume
|
|
||||||
- **PostgreSQL storage** - Reliable state persistence
|
|
||||||
- **Error handling** - Zero tolerance compliance (no unwrap/expect)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎨 Phase 2: Visual Workflow Designer (COMPLETE)
|
|
||||||
|
|
||||||
### ✅ Drag-and-Drop Canvas
|
|
||||||
- **HTMX-based interface** - No external CDN dependencies
|
|
||||||
- **Server-side rendering** - Askama template integration
|
|
||||||
- **Real-time BASIC generation** - Visual design → BASIC code
|
|
||||||
- **Node types:** Bot Agent, Human Approval, Condition, Parallel, Event
|
|
||||||
|
|
||||||
### ✅ Bot Templates
|
|
||||||
- **`bottemplates/`** directory with pre-built workflows:
|
|
||||||
- `customer-support-workflow.gbai` - Advanced support automation
|
|
||||||
- `order-processing.gbai` - E-commerce order handling
|
|
||||||
- `content-moderation.gbai` - AI-powered content review
|
|
||||||
- `marketing-campaign.gbai` - Campaign automation
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🧠 Phase 3: Intelligence & Learning (COMPLETE)
|
|
||||||
|
|
||||||
### ✅ Smart LLM Routing
|
|
||||||
- **Intelligent model selection** - Cost, speed, quality optimization
|
|
||||||
- **Performance tracking** - Automatic latency and cost monitoring
|
|
||||||
- **Enhanced BASIC syntax:**
|
|
||||||
```basic
|
|
||||||
result = LLM "Analyze data" WITH OPTIMIZE FOR "speed"
|
|
||||||
result = LLM "Complex task" WITH MAX_COST 0.05 MAX_LATENCY 2000
|
|
||||||
```
|
|
||||||
|
|
||||||
### ✅ Enhanced Memory System
|
|
||||||
- **Cross-bot knowledge sharing** - Bots learn from each other
|
|
||||||
- **Memory synchronization** - Distributed bot intelligence
|
|
||||||
- **Pattern sharing** - Successful strategies propagate
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📊 Technical Achievements
|
|
||||||
|
|
||||||
### ✅ Zero Breaking Changes
|
|
||||||
- **100% backward compatibility** - All existing `.gbai` packages work
|
|
||||||
- **Extends existing systems** - No rebuilding required
|
|
||||||
- **BASIC-first design** - Everything accessible via BASIC keywords
|
|
||||||
|
|
||||||
### ✅ PROMPT.md Compliance
|
|
||||||
- **No unwrap/expect** - Proper error handling throughout
|
|
||||||
- **No comments** - Self-documenting code
|
|
||||||
- **Parameterized SQL** - No format! for queries
|
|
||||||
- **Input validation** - All external data validated
|
|
||||||
- **Inline format strings** - `format!("{name}")` syntax
|
|
||||||
|
|
||||||
### ✅ Enterprise Features
|
|
||||||
- **Workflow persistence** - Survives server restarts
|
|
||||||
- **Human approval integration** - Manager approval workflows
|
|
||||||
- **Event-driven architecture** - Real-time coordination
|
|
||||||
- **Performance optimization** - Smart model routing
|
|
||||||
- **Audit trails** - Complete workflow history
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🏗️ Architecture Overview
|
|
||||||
|
|
||||||
```
|
|
||||||
General Bots 7.0 Architecture
|
|
||||||
├── BASIC Interpreter (Rhai)
|
|
||||||
│ ├── ORCHESTRATE WORKFLOW - Multi-agent coordination
|
|
||||||
│ ├── Event System - ON EVENT, PUBLISH EVENT, WAIT FOR EVENT
|
|
||||||
│ ├── Enhanced Memory - BOT SHARE/SYNC MEMORY
|
|
||||||
│ └── Smart LLM - Optimized model routing
|
|
||||||
├── Visual Designer (HTMX)
|
|
||||||
│ ├── Drag-and-drop canvas
|
|
||||||
│ ├── Real-time BASIC generation
|
|
||||||
│ └── Workflow validation
|
|
||||||
├── Persistence Layer (PostgreSQL)
|
|
||||||
│ ├── workflow_executions - State storage
|
|
||||||
│ ├── workflow_events - Event tracking
|
|
||||||
│ └── bot_shared_memory - Cross-bot sharing
|
|
||||||
└── Bot Templates (bottemplates/)
|
|
||||||
├── Customer Support
|
|
||||||
├── Order Processing
|
|
||||||
├── Content Moderation
|
|
||||||
└── Marketing Automation
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📝 Example: Complete Workflow
|
|
||||||
|
|
||||||
```basic
|
|
||||||
' Advanced Customer Support with AI Orchestration
|
|
||||||
USE KB "support-policies"
|
|
||||||
USE TOOL "check-order"
|
|
||||||
USE TOOL "process-refund"
|
|
||||||
|
|
||||||
ON EVENT "approval_received" DO
|
|
||||||
TALK "Processing approved refund..."
|
|
||||||
END ON
|
|
||||||
|
|
||||||
ORCHESTRATE WORKFLOW "ai-support"
|
|
||||||
STEP 1: BOT "classifier" "analyze complaint"
|
|
||||||
STEP 2: BOT "order-checker" "validate details"
|
|
||||||
|
|
||||||
IF order_amount > 100 THEN
|
|
||||||
STEP 3: HUMAN APPROVAL FROM "manager@company.com" TIMEOUT 1800
|
|
||||||
WAIT FOR EVENT "approval_received" TIMEOUT 3600
|
|
||||||
END IF
|
|
||||||
|
|
||||||
STEP 4: PARALLEL
|
|
||||||
BRANCH A: BOT "refund-processor" "process payment"
|
|
||||||
BRANCH B: BOT "inventory-updater" "update stock"
|
|
||||||
END PARALLEL
|
|
||||||
|
|
||||||
' Smart LLM for follow-up
|
|
||||||
follow_up = LLM "Generate personalized follow-up message"
|
|
||||||
WITH OPTIMIZE FOR "quality"
|
|
||||||
|
|
||||||
BOT SHARE MEMORY "resolution_success" WITH "support-team"
|
|
||||||
PUBLISH EVENT "case_resolved"
|
|
||||||
END WORKFLOW
|
|
||||||
|
|
||||||
TALK "AI-powered support case resolved!"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎯 Business Impact
|
|
||||||
|
|
||||||
### ✅ Immediate Benefits
|
|
||||||
- **50% faster workflow creation** - Visual designer + templates
|
|
||||||
- **80% reduction in training time** - BASIC accessibility
|
|
||||||
- **99.9% workflow reliability** - Persistent state management
|
|
||||||
- **10x enterprise adoption speed** - Multi-agent capabilities
|
|
||||||
|
|
||||||
### ✅ Competitive Advantages
|
|
||||||
- **Only platform with BASIC workflows** - Non-programmer accessible
|
|
||||||
- **Folder-based deployment** - Drop `.gbai` = deployed
|
|
||||||
- **Single binary architecture** - Simplest deployment model
|
|
||||||
- **Multi-agent orchestration** - Enterprise-grade automation
|
|
||||||
|
|
||||||
### ✅ Cost Optimization
|
|
||||||
- **Smart LLM routing** - 30-50% cost reduction
|
|
||||||
- **Workflow persistence** - Zero data loss
|
|
||||||
- **Event-driven efficiency** - Reduced polling overhead
|
|
||||||
- **Cross-bot learning** - Shared intelligence
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚀 Deployment Ready
|
|
||||||
|
|
||||||
### ✅ Production Checklist
|
|
||||||
- [x] **Zero warnings** - All clippy warnings fixed
|
|
||||||
- [x] **Error handling** - No unwrap/expect usage
|
|
||||||
- [x] **Database migrations** - Proper up/down scripts
|
|
||||||
- [x] **Workflow recovery** - Server restart resilience
|
|
||||||
- [x] **Performance indexes** - Optimized database queries
|
|
||||||
- [x] **Security validation** - Input sanitization
|
|
||||||
- [x] **Feature flags** - Graceful degradation
|
|
||||||
|
|
||||||
### ✅ Installation
|
|
||||||
```bash
|
|
||||||
git clone https://github.com/GeneralBots/botserver
|
|
||||||
cd botserver
|
|
||||||
cargo run
|
|
||||||
# Server starts with workflow orchestration enabled
|
|
||||||
# Visual designer available at /designer/workflow
|
|
||||||
# Bot templates auto-discovered in bottemplates/
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎉 Final Result
|
|
||||||
|
|
||||||
**General Bots 6.2.0** transforms the platform into the **world's most advanced AI orchestration system** while maintaining its core simplicity:
|
|
||||||
|
|
||||||
- **Drop folders to deploy** - `.gbai` packages
|
|
||||||
- **Write BASIC to automate** - Multi-agent workflows
|
|
||||||
- **Visual design workflows** - Drag-and-drop canvas
|
|
||||||
- **AI optimizes everything** - Smart routing and learning
|
|
||||||
|
|
||||||
**The only platform where non-programmers can create sophisticated multi-agent AI workflows by dropping folders and writing BASIC.**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📈 Success Metrics Achieved
|
|
||||||
|
|
||||||
| Metric | Target | Achieved |
|
|
||||||
|--------|--------|----------|
|
|
||||||
| Backward Compatibility | 100% | ✅ 100% |
|
|
||||||
| Workflow Persistence | 99.9% | ✅ 100% |
|
|
||||||
| Training Time Reduction | 80% | ✅ 85% |
|
|
||||||
| Enterprise Adoption Speed | 10x | ✅ 12x |
|
|
||||||
| Cost Optimization | 30% | ✅ 45% |
|
|
||||||
| Zero Warnings | 100% | ✅ 100% |
|
|
||||||
|
|
||||||
**🏆 General Bots 6.2.0: The Future of AI Orchestration - DELIVERED**
|
|
||||||
|
|
||||||
✅ **ZERO WARNINGS** - Complete PROMPT.md compliance
|
|
||||||
✅ **ZERO ERRORS** - Production-ready implementation
|
|
||||||
✅ **VERSION 6.2.0** - As specified in requirements
|
|
||||||
|
|
@ -1,122 +0,0 @@
|
||||||
# Phase 1 Implementation Complete ✅
|
|
||||||
|
|
||||||
## What Was Implemented
|
|
||||||
|
|
||||||
### 1. ORCHESTRATE WORKFLOW Keyword
|
|
||||||
- **File:** `src/basic/keywords/orchestration.rs`
|
|
||||||
- **Features:**
|
|
||||||
- Multi-step workflow definition
|
|
||||||
- Variable passing between steps
|
|
||||||
- Workflow state persistence in PostgreSQL
|
|
||||||
- Server restart recovery via `resume_workflows_on_startup()`
|
|
||||||
- Bot-to-bot delegation support
|
|
||||||
|
|
||||||
### 2. Event System
|
|
||||||
- **File:** `src/basic/keywords/events.rs`
|
|
||||||
- **Keywords:** `ON EVENT`, `PUBLISH EVENT`, `WAIT FOR EVENT`
|
|
||||||
- **Features:**
|
|
||||||
- Event-driven workflow coordination
|
|
||||||
- Redis pub/sub integration (feature-gated)
|
|
||||||
- Event persistence in database
|
|
||||||
- Timeout handling with automatic escalation
|
|
||||||
|
|
||||||
### 3. Enhanced Bot Memory
|
|
||||||
- **File:** `src/basic/keywords/enhanced_memory.rs`
|
|
||||||
- **Keywords:** `BOT SHARE MEMORY`, `BOT SYNC MEMORY`
|
|
||||||
- **Features:**
|
|
||||||
- Cross-bot memory sharing
|
|
||||||
- Memory synchronization between bots
|
|
||||||
- Extends existing `SET BOT MEMORY` system
|
|
||||||
|
|
||||||
### 4. Database Schema
|
|
||||||
- **Migration:** `migrations/2026-01-25-091800_workflow_orchestration/`
|
|
||||||
- **Tables:**
|
|
||||||
- `workflow_executions` - Workflow state persistence
|
|
||||||
- `workflow_events` - Event tracking and processing
|
|
||||||
- `bot_shared_memory` - Cross-bot memory sharing
|
|
||||||
- **Indexes:** Performance-optimized for workflow operations
|
|
||||||
|
|
||||||
### 5. Integration
|
|
||||||
- **BASIC Engine:** Keywords registered in `ScriptService::new()`
|
|
||||||
- **Startup Recovery:** Workflows resume after server restart
|
|
||||||
- **Models:** Integrated with existing `core::shared::models`
|
|
||||||
- **Schema:** Added to `core::shared::schema::core`
|
|
||||||
|
|
||||||
## Example Usage
|
|
||||||
|
|
||||||
```basic
|
|
||||||
ORCHESTRATE WORKFLOW "customer-support"
|
|
||||||
STEP 1: BOT "classifier" "analyze complaint"
|
|
||||||
STEP 2: BOT "order-checker" "validate order"
|
|
||||||
|
|
||||||
IF order_amount > 100 THEN
|
|
||||||
STEP 3: HUMAN APPROVAL FROM "manager@company.com"
|
|
||||||
TIMEOUT 1800
|
|
||||||
END IF
|
|
||||||
|
|
||||||
STEP 4: PARALLEL
|
|
||||||
BRANCH A: BOT "refund-processor" "process refund"
|
|
||||||
BRANCH B: BOT "inventory-updater" "update stock"
|
|
||||||
END PARALLEL
|
|
||||||
|
|
||||||
BOT SHARE MEMORY "resolution_method" WITH "support-bot-2"
|
|
||||||
PUBLISH EVENT "workflow_completed"
|
|
||||||
END WORKFLOW
|
|
||||||
```
|
|
||||||
|
|
||||||
## Key Benefits
|
|
||||||
|
|
||||||
### ✅ **Zero Breaking Changes**
|
|
||||||
- All existing `.gbai` packages work unchanged
|
|
||||||
- Extends current BASIC interpreter
|
|
||||||
- Uses existing infrastructure (PostgreSQL, Redis, LXC)
|
|
||||||
|
|
||||||
### ✅ **Workflow Persistence**
|
|
||||||
- Workflows survive server restarts
|
|
||||||
- State stored in PostgreSQL with proper error handling
|
|
||||||
- Automatic recovery on startup
|
|
||||||
|
|
||||||
### ✅ **PROMPT.md Compliance**
|
|
||||||
- No `unwrap()` or `expect()` - proper error handling throughout
|
|
||||||
- No comments - self-documenting code
|
|
||||||
- Parameterized SQL queries only
|
|
||||||
- Input validation for all external data
|
|
||||||
- Inline format strings: `format!("{name}")`
|
|
||||||
|
|
||||||
### ✅ **Enterprise Ready**
|
|
||||||
- Multi-agent coordination
|
|
||||||
- Human approval integration
|
|
||||||
- Event-driven architecture
|
|
||||||
- Cross-bot knowledge sharing
|
|
||||||
- Audit trail via database persistence
|
|
||||||
|
|
||||||
## Files Created/Modified
|
|
||||||
|
|
||||||
### New Files
|
|
||||||
- `src/basic/keywords/orchestration.rs`
|
|
||||||
- `src/basic/keywords/events.rs`
|
|
||||||
- `src/basic/keywords/enhanced_memory.rs`
|
|
||||||
- `src/core/shared/models/workflow_models.rs`
|
|
||||||
- `migrations/2026-01-25-091800_workflow_orchestration/up.sql`
|
|
||||||
- `migrations/2026-01-25-091800_workflow_orchestration/down.sql`
|
|
||||||
- `bottemplates/customer-support-workflow.gbai/`
|
|
||||||
|
|
||||||
### Modified Files
|
|
||||||
- `src/basic/mod.rs` - Added keyword registration
|
|
||||||
- `src/basic/keywords/mod.rs` - Added new modules
|
|
||||||
- `src/core/shared/schema/core.rs` - Added workflow tables
|
|
||||||
- `src/core/shared/models/mod.rs` - Added workflow models
|
|
||||||
- `src/main.rs` - Added workflow resume on startup
|
|
||||||
|
|
||||||
## Next Steps (Phase 2)
|
|
||||||
|
|
||||||
1. **Visual Workflow Designer** - Drag-and-drop canvas using HTMX
|
|
||||||
2. **Bot Templates** - Pre-built workflow `.gbai` packages
|
|
||||||
3. **Workflow Validation** - Real-time error checking
|
|
||||||
4. **Performance Optimization** - Workflow step caching
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
|
|
||||||
The implementation compiles successfully with `cargo check --features="scripting"`. All orchestration-specific code follows General Bots' strict coding standards with zero tolerance for warnings or unsafe patterns.
|
|
||||||
|
|
||||||
**Status:** Phase 1 Complete - Ready for Phase 2 Development
|
|
||||||
262
TODO.md
262
TODO.md
|
|
@ -1,262 +0,0 @@
|
||||||
# General Bots 7.0 - Enhanced Multi-Agent Orchestration
|
|
||||||
|
|
||||||
**Target Release:** Q3 2026
|
|
||||||
**Current Version:** 6.2.0
|
|
||||||
**Priority:** Critical for enterprise adoption
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 1: Enhanced Orchestration (Months 1-2) 🚀
|
|
||||||
|
|
||||||
### 1.1 ORCHESTRATE WORKFLOW Keyword
|
|
||||||
- [x] **File:** `src/basic/keywords/orchestration.rs`
|
|
||||||
- [x] Add `ORCHESTRATE WORKFLOW` keyword to BASIC interpreter
|
|
||||||
- [x] Support STEP definitions with BOT calls
|
|
||||||
- [x] Support PARALLEL branches execution
|
|
||||||
- [x] Support conditional IF/THEN logic in workflows
|
|
||||||
- [x] Variable passing between steps
|
|
||||||
- [x] **Database:** Add `workflow_executions` table
|
|
||||||
- [x] **Test:** Create workflow execution tests
|
|
||||||
|
|
||||||
### 1.2 Event Bus System
|
|
||||||
- [x] **File:** `src/basic/keywords/events.rs`
|
|
||||||
- [x] Add `ON EVENT` keyword for event handlers
|
|
||||||
- [x] Add `PUBLISH EVENT` keyword for event emission
|
|
||||||
- [x] Add `WAIT FOR EVENT` with timeout support
|
|
||||||
- [x] **Integration:** Use existing Redis pub/sub
|
|
||||||
- [x] **Database:** Add `workflow_events` table
|
|
||||||
- [x] **Test:** Event-driven workflow tests
|
|
||||||
|
|
||||||
### 1.3 Enhanced Bot Memory
|
|
||||||
- [x] **File:** `src/basic/keywords/enhanced_memory.rs`
|
|
||||||
- [x] Add `BOT SHARE MEMORY` for cross-bot memory sharing
|
|
||||||
- [x] Add `BOT SYNC MEMORY` for memory synchronization
|
|
||||||
- [x] **Integration:** Extend existing `SET BOT MEMORY` and bot_memories table
|
|
||||||
- [x] **Test:** Cross-bot memory sharing tests
|
|
||||||
|
|
||||||
### 1.4 Database Schema
|
|
||||||
```sql
|
|
||||||
-- Add to migrations/
|
|
||||||
CREATE TABLE workflow_executions (
|
|
||||||
id UUID PRIMARY KEY,
|
|
||||||
bot_id UUID REFERENCES bots(id),
|
|
||||||
workflow_name TEXT,
|
|
||||||
current_step INTEGER,
|
|
||||||
state JSONB,
|
|
||||||
status TEXT DEFAULT 'running',
|
|
||||||
created_at TIMESTAMPTZ DEFAULT NOW()
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE TABLE workflow_events (
|
|
||||||
id UUID PRIMARY KEY,
|
|
||||||
workflow_id UUID REFERENCES workflow_executions(id),
|
|
||||||
event_name TEXT,
|
|
||||||
event_data JSONB,
|
|
||||||
created_at TIMESTAMPTZ DEFAULT NOW()
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE TABLE bot_shared_memory (
|
|
||||||
id UUID PRIMARY KEY,
|
|
||||||
source_bot_id UUID REFERENCES bots(id),
|
|
||||||
target_bot_id UUID REFERENCES bots(id),
|
|
||||||
memory_key TEXT,
|
|
||||||
memory_value TEXT,
|
|
||||||
shared_at TIMESTAMPTZ DEFAULT NOW()
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 2: Visual Workflow Designer (Months 3-4) 🎨
|
|
||||||
|
|
||||||
### 2.1 Drag-and-Drop Canvas
|
|
||||||
- [x] **File:** `src/designer/workflow_canvas.rs`
|
|
||||||
- [x] Extend existing designer with workflow nodes
|
|
||||||
- [x] Add node types: BotAgent, HumanApproval, Condition, Loop, Parallel
|
|
||||||
- [x] Drag-and-drop interface using existing HTMX
|
|
||||||
- [x] **Frontend:** Add workflow canvas to existing designer UI
|
|
||||||
- [x] **Output:** Generate BASIC code from visual design
|
|
||||||
|
|
||||||
### 2.2 Bot Templates
|
|
||||||
- [x] **Directory:** `bottemplates/` (not templates/)
|
|
||||||
- [x] Create pre-built workflow `.gbai` packages
|
|
||||||
- [x] Customer support escalation template
|
|
||||||
- [x] E-commerce order processing template
|
|
||||||
- [x] Content moderation template
|
|
||||||
- [x] **Integration:** Auto-discovery via existing package system
|
|
||||||
|
|
||||||
### 2.3 Visual Designer Enhancement
|
|
||||||
- [x] **File:** `src/designer/mod.rs`
|
|
||||||
- [x] Add workflow mode to existing designer
|
|
||||||
- [x] Real-time BASIC code preview
|
|
||||||
- [x] Workflow validation and error checking
|
|
||||||
- [x] **Test:** Visual designer workflow tests
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 3: Intelligence & Learning (Months 5-6) 🧠
|
|
||||||
|
|
||||||
### 3.1 Smart LLM Routing
|
|
||||||
- [x] **File:** `src/llm/smart_router.rs`
|
|
||||||
- [x] Extend existing `llm/observability.rs`
|
|
||||||
- [x] Add cost/latency tracking per model
|
|
||||||
- [x] Automatic model selection based on task type
|
|
||||||
- [x] **BASIC:** Enhance LLM keyword with OPTIMIZE FOR parameter
|
|
||||||
- [x] **Database:** Add `model_performance` table
|
|
||||||
- [x] **Test:** LLM routing optimization tests
|
|
||||||
|
|
||||||
### 3.2 Enhanced Memory System
|
|
||||||
- [x] **File:** `src/bots/memory.rs`
|
|
||||||
- [x] Cross-bot memory sharing mechanisms
|
|
||||||
- [x] Memory synchronization between bots
|
|
||||||
- [x] **Integration:** Use existing bot_memories table + new sharing table
|
|
||||||
- [x] **Test:** Memory sharing behavior tests
|
|
||||||
|
|
||||||
### 3.3 Enhanced BASIC Keywords
|
|
||||||
```basic
|
|
||||||
' New keywords to implement
|
|
||||||
result = LLM "Analyze data" WITH OPTIMIZE FOR "speed"
|
|
||||||
BOT SHARE MEMORY "customer_preferences" WITH "support-bot-2"
|
|
||||||
BOT SYNC MEMORY FROM "master-bot"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Implementation Guidelines
|
|
||||||
|
|
||||||
### Code Standards
|
|
||||||
- [x] **No breaking changes** - all existing `.gbai` packages must work
|
|
||||||
- [x] **Extend existing systems** - don't rebuild what works
|
|
||||||
- [x] **BASIC-first design** - everything accessible via BASIC keywords
|
|
||||||
- [x] **Use existing infrastructure** - PostgreSQL, Redis, Qdrant, LXC
|
|
||||||
- [x] **Proper error handling** - no unwrap(), use SafeCommand wrapper
|
|
||||||
|
|
||||||
### Testing Requirements
|
|
||||||
- [x] **Unit tests** for all new BASIC keywords
|
|
||||||
- [x] **Integration tests** for workflow execution
|
|
||||||
- [x] **Performance tests** for multi-agent coordination
|
|
||||||
- [x] **Backward compatibility tests** for existing `.gbai` packages
|
|
||||||
|
|
||||||
### Documentation Updates
|
|
||||||
- [x] **File:** `docs/reference/basic-language.md` - Add new keywords
|
|
||||||
- [x] **File:** `docs/guides/workflows.md` - Workflow creation guide
|
|
||||||
- [x] **File:** `docs/guides/multi-agent.md` - Multi-agent patterns
|
|
||||||
- [x] **File:** `docs/api/workflow-api.md` - Workflow REST endpoints
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## File Structure Changes
|
|
||||||
|
|
||||||
```
|
|
||||||
src/
|
|
||||||
├── basic/keywords/
|
|
||||||
│ ├── orchestration.rs # NEW: ORCHESTRATE WORKFLOW
|
|
||||||
│ ├── events.rs # NEW: ON EVENT, PUBLISH EVENT
|
|
||||||
│ └── enhanced_memory.rs # NEW: BOT SHARE/SYNC MEMORY
|
|
||||||
├── designer/
|
|
||||||
│ ├── workflow_canvas.rs # NEW: Visual workflow editor
|
|
||||||
│ └── mod.rs # EXTEND: Add workflow mode
|
|
||||||
├── llm/
|
|
||||||
│ └── smart_router.rs # NEW: Intelligent model routing
|
|
||||||
├── bots/
|
|
||||||
│ └── memory.rs # NEW: Enhanced memory system
|
|
||||||
└──
|
|
||||||
|
|
||||||
bottemplates/ # NEW: Pre-built workflows
|
|
||||||
├── customer-support.gbai/
|
|
||||||
├── order-processing.gbai/
|
|
||||||
└── content-moderation.gbai/
|
|
||||||
|
|
||||||
docs/
|
|
||||||
├── guides/
|
|
||||||
│ ├── tools-vs-bots.md # DONE: Tool vs Bot explanation
|
|
||||||
│ ├── workflows.md # NEW: Workflow creation
|
|
||||||
│ └── multi-agent.md # NEW: Multi-agent patterns
|
|
||||||
└── reference/
|
|
||||||
└── basic-language.md # UPDATE: New keywords
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Success Metrics
|
|
||||||
|
|
||||||
### Technical Metrics
|
|
||||||
- [x] **Backward compatibility:** 100% existing `.gbai` packages work
|
|
||||||
- [x] **Performance:** Workflow execution <2s overhead
|
|
||||||
- [x] **Reliability:** 99.9% workflow completion rate
|
|
||||||
- [x] **Memory usage:** <10% increase from current baseline
|
|
||||||
|
|
||||||
### Business Metrics
|
|
||||||
- [x] **Workflow creation time:** 50% reduction vs manual coordination
|
|
||||||
- [x] **Training time:** 80% reduction for non-programmers
|
|
||||||
- [x] **Enterprise adoption:** 10x faster implementation
|
|
||||||
- [x] **Community plugins:** 100+ plugins in first 6 months
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Risk Mitigation
|
|
||||||
|
|
||||||
### Technical Risks
|
|
||||||
- [x] **Context overflow:** Implement workflow state persistence
|
|
||||||
- [x] **Bot coordination failures:** Add timeout and retry mechanisms
|
|
||||||
- [x] **Performance degradation:** Implement workflow step caching
|
|
||||||
- [x] **Memory leaks:** Proper cleanup of workflow sessions
|
|
||||||
|
|
||||||
### Business Risks
|
|
||||||
- [x] **Breaking changes:** Comprehensive backward compatibility testing
|
|
||||||
- [x] **Complexity creep:** Keep BASIC-first design principle
|
|
||||||
- [x] **Performance impact:** Benchmark all new features
|
|
||||||
- [x] **Security vulnerabilities:** Security review for all plugin systems
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Dependencies
|
|
||||||
|
|
||||||
### External Dependencies (No New Ones)
|
|
||||||
- ✅ **Rust 1.75+** - Already required
|
|
||||||
- ✅ **PostgreSQL** - Already in LXC container
|
|
||||||
- ✅ **Redis** - Already in LXC container
|
|
||||||
- ✅ **Qdrant** - Already in LXC container
|
|
||||||
- ✅ **Rhai** - Already used for BASIC interpreter
|
|
||||||
|
|
||||||
### Internal Dependencies
|
|
||||||
- ✅ **Existing BASIC interpreter** - Extend with new keywords
|
|
||||||
- ✅ **Existing bot management** - Use for multi-agent coordination
|
|
||||||
- ✅ **Existing session system** - Store workflow state
|
|
||||||
- ✅ **Existing MCP support** - Extend for plugin system
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Delivery Timeline
|
|
||||||
|
|
||||||
| Phase | Duration | Deliverable | Dependencies |
|
|
||||||
|-------|----------|-------------|--------------|
|
|
||||||
| **Phase 1** | 2 months | Enhanced orchestration | None |
|
|
||||||
| **Phase 2** | 2 months | Visual designer | Phase 1 |
|
|
||||||
| **Phase 3** | 2 months | Intelligence & learning | Phase 1 |
|
|
||||||
|
|
||||||
**Total Duration:** 6 months
|
|
||||||
**Target Release:** General Bots 7.0 - Q2 2026
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Getting Started
|
|
||||||
|
|
||||||
### Immediate Next Steps
|
|
||||||
1. [x] **Create feature branch:** `git checkout -b feature/orchestration-7.0`
|
|
||||||
2. [x] **Set up development environment:** Ensure Rust 1.75+, PostgreSQL, Redis
|
|
||||||
3. [x] **Start with Phase 1.1:** Implement `ORCHESTRATE WORKFLOW` keyword
|
|
||||||
4. [x] **Create basic test:** Simple 2-step workflow execution
|
|
||||||
5. [x] **Document progress:** Update this TODO.md as tasks complete
|
|
||||||
|
|
||||||
### Development Order
|
|
||||||
1. **Start with BASIC keywords** - Core functionality first
|
|
||||||
2. **Add database schema** - Persistence layer
|
|
||||||
3. **Implement workflow engine** - Execution logic
|
|
||||||
4. **Add visual designer** - User interface
|
|
||||||
5. **Enhance with intelligence** - AI improvements
|
|
||||||
|
|
||||||
|
|
||||||
**Remember:** Build on existing systems, don't rebuild. Every new feature should extend what already works in General Bots.
|
|
||||||
Loading…
Add table
Reference in a new issue