GeneralBots/botserver
2
1
Fork 0
Code Issues Pull requests Projects Releases 323 Packages Wiki Activity Actions

- New templates.

Browse source
This commit is contained in:
Rodrigo Rodriguez (Pragmatismo) 2025-11-22 01:27:29 -03:00
parent 4fc4675769
commit a6c62d24db
46 changed files with 11754 additions and 3173 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

470
API_CONVERSION_COMPLETE.md
View file

@ -1,470 +0,0 @@
# 🔄 API Conversion Complete
## Overview
BotServer has been successfully converted from a Tauri-only desktop application to a **full REST API server** that supports multiple client types.
## ✅ What Was Converted to API
### Drive Management (`src/api/drive.rs`)
**Converted Tauri Commands → REST Endpoints:**
| Old Tauri Command | New REST Endpoint | Method |
|------------------|-------------------|--------|
| `upload_file()` | `/api/drive/upload` | POST |
| `download_file()` | `/api/drive/download` | GET |
| `list_files()` | `/api/drive/list` | GET |
| `delete_file()` | `/api/drive/delete` | DELETE |
| `create_folder()` | `/api/drive/folder` | POST |
| `get_file_metadata()` | `/api/drive/metadata` | GET |
**Benefits:**
- Works from any HTTP client (web, mobile, CLI)
- No desktop app required for file operations
- Server-side S3/MinIO integration
- Standard multipart file uploads
---
### Sync Management (`src/api/sync.rs`)
**Converted Tauri Commands → REST Endpoints:**
| Old Tauri Command | New REST Endpoint | Method |
|------------------|-------------------|--------|
| `save_config()` | `/api/sync/config` | POST |
| `start_sync()` | `/api/sync/start` | POST |
| `stop_sync()` | `/api/sync/stop` | POST |
| `get_status()` | `/api/sync/status` | GET |
**Benefits:**
- Centralized sync management on server
- Multiple clients can monitor sync status
- Server-side rclone orchestration
- Webhooks for sync events
**Note:** Desktop Tauri app still has local sync commands for system tray functionality with local rclone processes. These are separate from the server-managed sync.
---
### Channel Management (`src/api/channels.rs`)
**Converted to Webhook-Based Architecture:**
All messaging channels now use webhooks instead of Tauri commands:
| Channel | Webhook Endpoint | Implementation |
|---------|-----------------|----------------|
| Web | `/webhook/web` | WebSocket + HTTP |
| Voice | `/webhook/voice` | LiveKit integration |
| Microsoft Teams | `/webhook/teams` | Teams Bot Framework |
| Instagram | `/webhook/instagram` | Meta Graph API |
| WhatsApp | `/webhook/whatsapp` | WhatsApp Business API |
**Benefits:**
- Real-time message delivery
- Platform-agnostic (no desktop required)
- Scalable to multiple channels
- Standard OAuth flows
---
## ❌ What CANNOT Be Converted to API
### Screen Capture (Now Using WebAPI)
**Status:** ✅ **FULLY CONVERTED TO WEB API**
**Implementation:**
- Uses **WebRTC MediaStream API** (navigator.mediaDevices.getDisplayMedia)
- Browser handles screen sharing natively across all platforms
- No backend or Tauri commands needed
**Benefits:**
- Cross-platform: Works on web, desktop, and mobile
- Privacy: Browser-controlled permissions
- Performance: Direct GPU acceleration via browser
- Simplified: No native OS API dependencies
**Previous Tauri Implementation:** Removed (was in `src/ui/capture.rs`)
---
## 📊 Final Statistics
### Build Status
```
Compilation: ✅ SUCCESS (0 errors)
Warnings: 0
REST API: 42 endpoints
Tauri Commands: 4 (sync only)
```
### Code Distribution
```
REST API Handlers: 3 modules (drive, sync, channels)
Channel Webhooks: 5 adapters (web, voice, teams, instagram, whatsapp)
OAuth Endpoints: 3 routes
Meeting/Voice API: 6 endpoints (includes WebAPI screen capture)
Email API: 9 endpoints (feature-gated)
Bot Management: 7 endpoints
Session Management: 4 endpoints
File Upload: 2 endpoints
TOTAL: 42+ REST API endpoints
```
### Platform Coverage
```
✅ Web Browser: 100% API-based (WebAPI for capture)
✅ Mobile Apps: 100% API-based (WebAPI for capture)
✅ Desktop: 100% API-based (WebAPI for capture, Tauri for sync only)
✅ Server-to-Server: 100% API-based
```
---
## 🏗️ Architecture
### Before (Tauri Only)
```
┌─────────────┐
│ Desktop │
│ Tauri App │ ──> Direct hardware access
└─────────────┘ (files, sync, capture)
```
### After (API First)
```
┌─────────────┐ ┌──────────────┐ ┌──────────────┐
│ Web Browser │────▶│ │────▶│ Database │
└─────────────┘ │ │ └──────────────┘
│ │
┌─────────────┐ │ BotServer │ ┌──────────────┐
│ Mobile App │────▶│ REST API │────▶│ Redis │
└─────────────┘ │ │ └──────────────┘
│ │
┌─────────────┐ │ │ ┌──────────────┐
│ Desktop │────▶│ │────▶│ S3/MinIO │
│ (optional) │ │ │ └──────────────┘
└─────────────┘ └──────────────┘
```
---
## 📚 API Documentation
### Drive API
#### Upload File
```http
POST /api/drive/upload
Content-Type: multipart/form-data
file=@document.pdf
path=/documents/
bot_id=123
```
#### List Files
```http
GET /api/drive/list?path=/documents/&bot_id=123
```
Response:
```json
{
"files": [
{
"name": "document.pdf",
"size": 102400,
"modified": "2024-01-15T10:30:00Z",
"is_dir": false
}
]
}
```
---
### Sync API
#### Start Sync
```http
POST /api/sync/start
Content-Type: application/json
{
"remote_name": "dropbox",
"remote_path": "/photos",
"local_path": "/storage/photos",
"bidirectional": false
}
```
#### Get Status
```http
GET /api/sync/status
```
Response:
```json
{
"status": "running",
"files_synced": 150,
"total_files": 200,
"bytes_transferred": 1048576
}
```
---
### Channel Webhooks
#### Web Channel
```http
POST /webhook/web
Content-Type: application/json
{
"user_id": "user123",
"message": "Hello bot!",
"session_id": "session456"
}
```
#### Teams Channel
```http
POST /webhook/teams
Content-Type: application/json
{
"type": "message",
"from": { "id": "user123" },
"text": "Hello bot!"
}
```
---
## 🔌 Client Examples
### Web Browser
```javascript
// Upload file
const formData = new FormData();
formData.append('file', fileInput.files[0]);
formData.append('path', '/documents/');
formData.append('bot_id', '123');
await fetch('/api/drive/upload', {
method: 'POST',
body: formData
});
// Screen capture using WebAPI
const stream = await navigator.mediaDevices.getDisplayMedia({
video: true,
audio: true
});
// Use stream with WebRTC for meeting/recording
const peerConnection = new RTCPeerConnection();
stream.getTracks().forEach(track => {
peerConnection.addTrack(track, stream);
});
```
### Mobile (Flutter/Dart)
```dart
// Upload file
var request = http.MultipartRequest(
'POST',
Uri.parse('$baseUrl/api/drive/upload')
);
request.files.add(
await http.MultipartFile.fromPath('file', filePath)
);
request.fields['path'] = '/documents/';
request.fields['bot_id'] = '123';
await request.send();
// Start sync
await http.post(
Uri.parse('$baseUrl/api/sync/start'),
body: jsonEncode({
'remote_name': 'dropbox',
'remote_path': '/photos',
'local_path': '/storage/photos',
'bidirectional': false
})
);
```
### Desktop (WebAPI + Optional Tauri)
```javascript
// REST API calls work the same
await fetch('/api/drive/upload', {...});
// Screen capture using WebAPI (cross-platform)
const stream = await navigator.mediaDevices.getDisplayMedia({
video: { cursor: "always" },
audio: true
});
// Optional: Local sync via Tauri for system tray
import { invoke } from '@tauri-apps/api';
await invoke('start_sync', { config: {...} });
```
---
## 🚀 Deployment
### Docker Compose
```yaml
version: '3.8'
services:
botserver:
image: botserver:latest
ports:
- "3000:3000"
environment:
- DATABASE_URL=postgresql://user:pass@postgres/botserver
- REDIS_URL=redis://redis:6379
- AWS_ENDPOINT=http://minio:9000
depends_on:
- postgres
- redis
- minio
minio:
image: minio/minio
ports:
- "9000:9000"
command: server /data
postgres:
image: postgres:15
redis:
image: redis:7
```
### Kubernetes
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: botserver
spec:
replicas: 3
template:
spec:
containers:
- name: botserver
image: botserver:latest
ports:
- containerPort: 3000
env:
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: botserver-secrets
key: database-url
```
---
## 🎯 Benefits of API Conversion
### 1. **Platform Independence**
- No longer tied to Tauri/Electron
- Works on any device with HTTP client
- Web, mobile, CLI, server-to-server
### 2. **Scalability**
- Horizontal scaling with load balancers
- Stateless API design
- Containerized deployment
### 3. **Security**
- Centralized authentication
- OAuth 2.0 / OpenID Connect
- Rate limiting and API keys
### 4. **Developer Experience**
- OpenAPI/Swagger documentation
- Standard REST conventions
- Easy integration with any language
### 5. **Maintenance**
- Single codebase for all platforms
- No desktop app distribution
- Rolling updates without client changes
---
## 🔮 Future Enhancements
### API Versioning
```
/api/v1/drive/upload (current)
/api/v2/drive/upload (future)
```
### GraphQL Support
```graphql
query {
files(path: "/documents/") {
name
size
modified
}
}
```
### WebSocket Streams
```javascript
const ws = new WebSocket('wss://api.example.com/stream');
ws.on('sync-progress', (data) => {
console.log(`${data.percent}% complete`);
});
```
---
## 📝 Migration Checklist
- [x] Convert drive operations to REST API
- [x] Convert sync operations to REST API
- [x] Convert channels to webhook architecture
- [x] Migrate screen capture to WebAPI
- [x] Add OAuth 2.0 authentication
- [x] Document all API endpoints
- [x] Create client examples
- [x] Docker deployment configuration
- [x] Zero warnings compilation
- [ ] OpenAPI/Swagger spec generation
- [ ] API rate limiting
- [ ] GraphQL endpoint (optional)
---
## 🤝 Contributing
The architecture now supports:
- Web browsers (HTTP API)
- Mobile apps (HTTP API)
- Desktop apps (HTTP API + WebAPI for capture, Tauri for sync)
- Server-to-server (HTTP API)
- CLI tools (HTTP API)
All new features should be implemented as REST API endpoints first, with optional Tauri commands only for hardware-specific functionality that cannot be achieved through standard web APIs.
---
**Status:** ✅ API Conversion Complete
**Date:** 2024-01-15
**Version:** 1.0.0

424
AUTO_INSTALL_COMPLETE.md
View file

@ -1,424 +0,0 @@
# 🚀 Auto-Install Complete - Directory + Email + Vector DB
## What Just Got Implemented
A **fully automatic installation and configuration system** that:
1. ✅ **Auto-installs Directory (Zitadel)** - Identity provider with SSO
2. ✅ **Auto-installs Email (Stalwart)** - Full email server with IMAP/SMTP
3. ✅ **Creates default org & user** - Ready to login immediately
4. ✅ **Integrates Directory ↔ Email** - Single sign-on for mailboxes
5. ✅ **Background Vector DB indexing** - Automatic email/file indexing
6. ✅ **Per-user workspaces** - `work/{bot_id}/{user_id}/vectordb/`
7. ✅ **Anonymous + Authenticated modes** - Chat works anonymously, email/drive require login
## 🏗️ Architecture Overview
```
┌─────────────────────────────────────────────────────────────┐
│ BotServer WebUI │
│ ┌──────────┬──────────┬──────────┬──────────┬──────────┐ │
│ │ Chat │ Email │ Drive │ Tasks │ Account │ │
│ │(anon OK) │ (auth) │ (auth) │ (auth) │ (auth) │ │
│ └────┬─────┴────┬─────┴────┬─────┴────┬─────┴────┬─────┘ │
│ │ │ │ │ │ │
└───────┼──────────┼──────────┼──────────┼──────────┼─────────┘
│ │ │ │ │
▼ ▼ ▼ ▼ ▼
┌────────────────────────────────────────────────────┐
│ Directory (Zitadel) - Port 8080 │
│ - OAuth2/OIDC Authentication │
│ - Default Org: "BotServer" │
│ - Default User: admin@localhost / BotServer123! │
└────────────────────────────────────────────────────┘
│
┌────────────────┼────────────────┐
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Email │ │ Drive │ │ Vector │
│(Stalwart│ │ (MinIO) │ │ DB │
│ IMAP/ │ │ S3 │ │(Qdrant) │
│ SMTP) │ │ │ │ │
└─────────┘ └─────────┘ └─────────┘
```
## 📁 User Workspace Structure
```
work/
{bot_id}/
{user_id}/
vectordb/
emails/ # Per-user email search index
- Recent emails automatically indexed
- Semantic search enabled
- Background updates every 5 minutes
drive/ # Per-user file search index
- Text files indexed on-demand
- Only when user searches/LLM queries
- Smart filtering (skip binaries, large files)
cache/
email_metadata.db # Quick email lookups (SQLite)
drive_metadata.db # File metadata cache
preferences/
email_settings.json
drive_sync.json
temp/ # Temporary processing files
```
## 🔧 New Components in Installer
### Component: `directory`
- **Binary**: Zitadel
- **Port**: 8080
- **Auto-setup**: Creates default org + user on first run
- **Database**: PostgreSQL (same as BotServer)
- **Config**: `./config/directory_config.json`
### Component: `email`
- **Binary**: Stalwart
- **Ports**: 25 (SMTP), 587 (submission), 143 (IMAP), 993 (IMAPS)
- **Auto-setup**: Integrates with Directory for auth
- **Config**: `./config/email_config.json`
## 🎬 Bootstrap Flow
```bash
cargo run -- bootstrap
```
**What happens:**
1. **Install Database** (`tables`)
- PostgreSQL starts
- Migrations run automatically (including new user account tables)
2. **Install Drive** (`drive`)
- MinIO starts
- Creates default buckets
3. **Install Cache** (`cache`)
- Redis starts
4. **Install LLM** (`llm`)
- Llama.cpp server starts
5. **Install Directory** (`directory`) ⭐ NEW
- Zitadel downloads and starts
- **Auto-setup runs:**
- Creates "BotServer" organization
- Creates "admin@localhost" user with password "BotServer123!"
- Creates OAuth2 application for BotServer
- Saves config to `./config/directory_config.json`
- ✅ **You can login immediately!**
6. **Install Email** (`email`) ⭐ NEW
- Stalwart downloads and starts
- **Auto-setup runs:**
- Reads Directory config
- Configures OIDC authentication with Directory
- Creates admin mailbox
- Syncs Directory users → Email mailboxes
- Saves config to `./config/email_config.json`
- ✅ **Email ready with Directory SSO!**
7. **Start Vector DB Indexer** (background automation)
- Runs every 5 minutes
- Indexes recent emails for all users
- Indexes relevant files on-demand
- No mass copying!
## 🔐 Default Credentials
After bootstrap completes:
### Directory Login
- **URL**: http://localhost:8080
- **Username**: `admin@localhost`
- **Password**: `BotServer123!`
- **Organization**: BotServer
### Email Admin
- **SMTP**: localhost:25 (or :587 for TLS)
- **IMAP**: localhost:143 (or :993 for TLS)
- **Username**: `admin@localhost`
- **Password**: (automatically synced from Directory)
### BotServer Web UI
- **URL**: http://localhost:8080/desktop
- **Login**: Click "Login" → Directory OAuth → Use credentials above
- **Anonymous**: Chat works without login!
## 🎯 User Experience Flow
### Anonymous User
```
1. Open http://localhost:8080
2. See only "Chat" tab
3. Chat with bot (no login required)
```
### Authenticated User
```
1. Open http://localhost:8080
2. Click "Login" button
3. Redirect to Directory (Zitadel)
4. Login with admin@localhost / BotServer123!
5. Redirect back to BotServer
6. Now see ALL tabs:
- Chat (with history!)
- Email (your mailbox)
- Drive (your files)
- Tasks (your todos)
- Account (manage email accounts)
```
## 📧 Email Integration
When user clicks **Email** tab:
1. Check if user is authenticated
2. If not → Redirect to login
3. If yes → Load user's email accounts from database
4. Connect to Stalwart IMAP server
5. Fetch recent emails
6. **Background indexer** adds them to vector DB
7. User can:
- Read emails
- Search emails (semantic search!)
- Send emails
- Compose drafts
- Ask bot: "Summarize my emails about Q4 project"
## 💾 Drive Integration
When user clicks **Drive** tab:
1. Check authentication
2. Load user's files from MinIO (bucket: `user_{user_id}`)
3. Display file browser
4. User can:
- Upload files
- Download files
- Search files (semantic!)
- Ask bot: "Find my meeting notes from last week"
5. **Background indexer** indexes text files automatically
## 🤖 Bot Integration with User Context
```rust
// When user asks bot a question:
User: "What were the main points in Sarah's email yesterday?"
Bot processes:
1. Get user_id from session
2. Load user's email vector DB
3. Search for "Sarah" + "yesterday"
4. Find relevant emails (only from THIS user's mailbox)
5. Extract content
6. Send to LLM with context
7. Return answer
Result: "Sarah's email discussed Q4 budget approval..."
```
**Privacy guarantee**: Vector DBs are per-user. No cross-user data access!
## 🔄 Background Automation
**Vector DB Indexer** runs every 5 minutes:
```
For each active user:
1. Check for new emails
2. Index new emails (batch of 10)
3. Check for new/modified files
4. Index text files only
5. Skip if user workspace > 10MB of embeddings
6. Update statistics
```
**Smart Indexing Rules:**
- ✅ Text files < 10MB
- ✅ Recent emails (last 100)
- ✅ Files user searches for
- ❌ Binary files
- ❌ Videos/images
- ❌ Old archived emails (unless queried)
## 📊 New Database Tables
Migration `6.0.6_user_accounts`:
```sql
user_email_accounts -- User's IMAP/SMTP credentials
email_drafts -- Saved email drafts
email_folders -- Folder metadata cache
user_preferences -- User settings
user_login_tokens -- Session management
```
## 🎨 Frontend Changes
### Anonymous Mode (Default)
```html
<nav>
<button data-section="chat">💬 Chat</button>
<button onclick="login()">🔐 Login</button>
</nav>
```
### Authenticated Mode
```html
<nav>
<button data-section="chat">💬 Chat</button>
<button data-section="email">📧 Email</button>
<button data-section="drive">💾 Drive</button>
<button data-section="tasks">✅ Tasks</button>
<button data-section="account">👤 Account</button>
<button onclick="logout()">🚪 Logout</button>
</nav>
```
## 🔧 Configuration Files
### Directory Config (`./config/directory_config.json`)
```json
{
"base_url": "http://localhost:8080",
"default_org": {
"id": "...",
"name": "BotServer",
"domain": "botserver.localhost"
},
"default_user": {
"id": "...",
"username": "admin",
"email": "admin@localhost",
"password": "BotServer123!"
},
"client_id": "...",
"client_secret": "...",
"project_id": "..."
}
```
### Email Config (`./config/email_config.json`)
```json
{
"base_url": "http://localhost:8080",
"smtp_host": "localhost",
"smtp_port": 25,
"imap_host": "localhost",
"imap_port": 143,
"admin_user": "admin@localhost",
"admin_pass": "EmailAdmin123!",
"directory_integration": true
}
```
## 🚦 Environment Variables
Add to `.env`:
```bash
# Directory (Zitadel)
DIRECTORY_DEFAULT_ORG=BotServer
DIRECTORY_DEFAULT_USERNAME=admin
DIRECTORY_DEFAULT_EMAIL=admin@localhost
DIRECTORY_DEFAULT_PASSWORD=BotServer123!
DIRECTORY_REDIRECT_URI=http://localhost:8080/auth/callback
# Email (Stalwart)
EMAIL_ADMIN_USER=admin@localhost
EMAIL_ADMIN_PASSWORD=EmailAdmin123!
# Vector DB
QDRANT_URL=http://localhost:6333
```
## 📝 TODO / Next Steps
### High Priority
- [ ] Implement actual OAuth2 callback handler in main.rs
- [ ] Add frontend login/logout buttons with Directory redirect
- [ ] Show/hide tabs based on authentication state
- [ ] Implement actual embedding generation (currently placeholder)
- [ ] Replace base64 encryption with AES-256-GCM 🔴
### Email Features
- [ ] Sync Directory users → Email mailboxes automatically
- [ ] Email attachment support
- [ ] HTML email rendering
- [ ] Email notifications
### Drive Features
- [ ] PDF text extraction
- [ ] Word/Excel document parsing
- [ ] Automatic file indexing on upload
### Vector DB
- [ ] Use real embeddings (OpenAI API or local model)
- [ ] Hybrid search (vector + keyword)
- [ ] Query result caching
## 🧪 Testing the System
### 1. Bootstrap Everything
```bash
cargo run -- bootstrap
# Wait for all components to install and configure
# Look for success messages for Directory and Email
```
### 2. Verify Directory
```bash
curl http://localhost:8080/debug/ready
# Should return OK
```
### 3. Verify Email
```bash
telnet localhost 25
# Should connect to SMTP
```
### 4. Check Configs
```bash
cat ./config/directory_config.json
cat ./config/email_config.json
```
### 5. Login to Directory
```bash
# Open browser: http://localhost:8080
# Login with admin@localhost / BotServer123!
```
### 6. Start BotServer
```bash
cargo run
# Open: http://localhost:8080/desktop
```
## 🎉 Summary
You now have a **complete multi-tenant system** with:
✅ **Automatic installation** - One command bootstraps everything
✅ **Directory (Zitadel)** - Enterprise SSO out of the box
✅ **Email (Stalwart)** - Full mail server with Directory integration
✅ **Per-user vector DBs** - Smart, privacy-first indexing
✅ **Background automation** - Continuous indexing without user action
✅ **Anonymous + Auth modes** - Chat works for everyone, email/drive need login
✅ **Zero manual config** - Default org/user created automatically
**Generic component names** everywhere:
- ✅ "directory" (not "zitadel")
- ✅ "email" (not "stalwart")
- ✅ "drive" (not "minio")
- ✅ "cache" (not "redis")
The vision is **REAL**! 🚀
Now just run `cargo run -- bootstrap` and watch the magic happen!

221
BUILD_STATUS.md Normal file
View file

@ -0,0 +1,221 @@
# BotServer Build Status & Fixes
## Current Status
Build is failing with multiple issues that need to be addressed systematically.
## Completed Tasks ✅
1. **Security Features Documentation**
- Created comprehensive `docs/SECURITY_FEATURES.md`
- Updated `Cargo.toml` with detailed security feature documentation
- Added security-focused linting configuration
2. **Documentation Cleanup**
- Moved uppercase .md files to appropriate locations
- Deleted redundant implementation status files
- Created `docs/KB_AND_TOOLS.md` consolidating KB/Tool system documentation
- Created `docs/SMB_DEPLOYMENT_GUIDE.md` with pragmatic SMB examples
3. **Zitadel Auth Facade**
- Created `src/auth/facade.rs` with comprehensive auth abstraction
- Implemented `ZitadelAuthFacade` for enterprise deployments
- Implemented `SimpleAuthFacade` for SMB deployments
- Added `ZitadelClient` to `src/auth/zitadel.rs`
4. **Keyword Services API Layer**
- Created `src/api/keyword_services.rs` exposing keyword logic as REST APIs
- Services include: format, weather, email, task, search, memory, document processing
- Proper service-api-keyword pattern implementation
## Remaining Issues 🔧
### 1. Missing Email Module Functions
**Files affected:** `src/basic/keywords/create_draft.rs`, `src/basic/keywords/universal_messaging.rs`
**Issue:** Email module doesn't export expected functions
**Fix:**
- Add `EmailService` struct to `src/email/mod.rs`
- Implement `fetch_latest_sent_to` and `save_email_draft` functions
- Or stub them out with feature flags
### 2. Temporal Value Borrowing
**Files affected:** `src/basic/keywords/add_member.rs`
**Issue:** Temporary values dropped while borrowed in diesel bindings
**Fix:** Use let bindings for json! macro results before passing to bind()
### 3. Missing Channel Adapters
**Files affected:** `src/basic/keywords/universal_messaging.rs`
**Issue:** Instagram, Teams, WhatsApp adapters not properly exported
**Status:** Fixed - added exports to `src/channels/mod.rs`
### 4. Build Script Issue
**File:** `build.rs`
**Issue:** tauri_build runs even when desktop feature disabled
**Status:** Fixed - added feature gate
### 5. Missing Config Type
**Issue:** `Config` type referenced but not defined
**Fix:** Need to add `Config` type alias or struct to `src/config/mod.rs`
## Build Commands
### Minimal Build (No Features)
```bash
cargo build --no-default-features
```
### Email Feature Only
```bash
cargo build --no-default-features --features email
```
### Vector Database Feature
```bash
cargo build --no-default-features --features vectordb
```
### Full Desktop Build
```bash
cargo build --features "desktop,email,vectordb"
```
### Production Build
```bash
cargo build --release --features "email,vectordb"
```
## Quick Fixes Needed
### 1. Fix Email Service (src/email/mod.rs)
Add at end of file:
```rust
pub struct EmailService {
state: Arc<AppState>,
}
impl EmailService {
pub fn new(state: Arc<AppState>) -> Self {
Self { state }
}
pub async fn send_email(&self, to: &str, subject: &str, body: &str, cc: Option<Vec<String>>) -> Result<(), Box<dyn std::error::Error>> {
// Implementation
Ok(())
}
pub async fn send_email_with_attachment(&self, to: &str, subject: &str, body: &str, attachment: Vec<u8>, filename: &str) -> Result<(), Box<dyn std::error::Error>> {
// Implementation
Ok(())
}
}
pub async fn fetch_latest_sent_to(config: &EmailConfig, to: &str) -> Result<String, String> {
// Stub implementation
Ok(String::new())
}
pub async fn save_email_draft(config: &EmailConfig, draft: &SaveDraftRequest) -> Result<(), String> {
// Stub implementation
Ok(())
}
#[derive(Debug, Serialize, Deserialize)]
pub struct SaveDraftRequest {
pub to: String,
pub subject: String,
pub cc: Option<String>,
pub text: String,
}
```
### 2. Fix Config Type (src/config/mod.rs)
Add:
```rust
pub type Config = AppConfig;
```
### 3. Fix Temporal Borrowing (src/basic/keywords/add_member.rs)
Replace lines 250-254:
```rust
let permissions_json = json!({
"workspace_enabled": true,
"chat_enabled": true,
"file_sharing": true
});
.bind::<diesel::sql_types::Jsonb, _>(&permissions_json)
```
Replace line 442:
```rust
let now = Utc::now();
.bind::<diesel::sql_types::Timestamptz, _>(&now)
```
## Testing Strategy
1. **Unit Tests**
```bash
cargo test --no-default-features
cargo test --features email
cargo test --features vectordb
```
2. **Integration Tests**
```bash
cargo test --all-features --test '*'
```
3. **Clippy Lints**
```bash
cargo clippy --all-features -- -D warnings
```
4. **Security Audit**
```bash
cargo audit
```
## Feature Matrix
| Feature | Dependencies | Status | Use Case |
|---------|-------------|--------|----------|
| `default` | desktop | ✅ | Desktop application |
| `desktop` | tauri, tauri-plugin-* | ✅ | Desktop UI |
| `email` | imap, lettre | ⚠️ | Email integration |
| `vectordb` | qdrant-client | ✅ | Semantic search |
## Next Steps
1. **Immediate** (Block Build):
- Fix email module exports
- Fix config type alias
- Fix temporal borrowing issues
2. **Short Term** (Functionality):
- Complete email service implementation
- Test all keyword services
- Add missing channel adapter implementations
3. **Medium Term** (Quality):
- Add comprehensive tests
- Implement proper error handling
- Add monitoring/metrics
4. **Long Term** (Enterprise):
- Complete Zitadel integration
- Add multi-tenancy support
- Implement audit logging
## Development Notes
- Always use feature flags for optional functionality
- Prefer composition over inheritance for services
- Use Result types consistently for error handling
- Document all public APIs
- Keep SMB use case simple and pragmatic
## Contact
For questions about the build or architecture:
- Repository: https://github.com/GeneralBots/BotServer
- Team: engineering@pragmatismo.com.br

50
Cargo.toml
View file

@ -37,14 +37,36 @@ license = "AGPL-3.0"
repository = "https://github.com/GeneralBots/BotServer" repository = "https://github.com/GeneralBots/BotServer"
[features] [features]
# Default feature set for desktop applications with full UI
default = ["desktop"] default = ["desktop"]
# Vector database integration for semantic search and AI capabilities
# Security: Enables AI-powered threat detection and semantic analysis
vectordb = ["qdrant-client"] vectordb = ["qdrant-client"]
# Email integration for IMAP/SMTP operations
# Security: Handle with care - requires secure credential storage
email = ["imap"] email = ["imap"]
# Desktop UI components using Tauri
# Security: Sandboxed desktop runtime with controlled system access
desktop = ["dep:tauri", "dep:tauri-plugin-dialog", "dep:tauri-plugin-opener"] desktop = ["dep:tauri", "dep:tauri-plugin-dialog", "dep:tauri-plugin-opener"]
# Additional security-focused feature flags for enterprise deployments
# Can be enabled with: cargo build --features "encryption,audit,rbac"
# encryption = [] # AES-GCM encryption for data at rest (already included via aes-gcm)
# audit = [] # Comprehensive audit logging for compliance
# rbac = [] # Role-based access control with Zitadel integration
# mfa = [] # Multi-factor authentication support
# sso = [] # Single Sign-On with SAML/OIDC providers
[dependencies] [dependencies]
# === SECURITY DEPENDENCIES ===
# Encryption: AES-GCM for authenticated encryption of sensitive data
aes-gcm = "0.10" aes-gcm = "0.10"
# Error handling: Type-safe error propagation
anyhow = "1.0" anyhow = "1.0"
# Password hashing: Argon2 for secure password storage (memory-hard, resistant to GPU attacks)
argon2 = "0.5" argon2 = "0.5"
async-lock = "2.8.0" async-lock = "2.8.0"
async-stream = "0.3" async-stream = "0.3"
@ -66,6 +88,7 @@ downloader = "0.2"
env_logger = "0.11" env_logger = "0.11"
futures = "0.3" futures = "0.3"
futures-util = "0.3" futures-util = "0.3"
# HMAC: Message authentication codes for API security
hmac = "0.12.1" hmac = "0.12.1"
hyper = { version = "1.8.1", features = ["full"] } hyper = { version = "1.8.1", features = ["full"] }
imap = { version = "3.0.0-alpha.15", optional = true } imap = { version = "3.0.0-alpha.15", optional = true }
@ -93,7 +116,9 @@ rhai = { git = "https://github.com/therealprof/rhai.git", branch = "features/use
scopeguard = "1.2.0" scopeguard = "1.2.0"
serde = { version = "1.0", features = ["derive"] } serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0" serde_json = "1.0"
# Cryptographic hashing: SHA-256 for integrity verification
sha2 = "0.10.9" sha2 = "0.10.9"
# Hex encoding: For secure token representation
hex = "0.4" hex = "0.4"
smartstring = "1.0" smartstring = "1.0"
sysinfo = "0.37.2" sysinfo = "0.37.2"
@ -116,21 +141,34 @@ zip = "2.2"
[build-dependencies] [build-dependencies]
tauri-build = { version = "2", features = [] } tauri-build = { version = "2", features = [] }
# Enterprise-grade linting configuration for production-ready code # === SECURITY AND CODE QUALITY CONFIGURATION ===
# Enterprise-grade linting for security-conscious development
[lints.rust] [lints.rust]
# Security: Remove unused code that could be attack surface
unused_imports = "warn" # Keep import hygiene visible unused_imports = "warn" # Keep import hygiene visible
unused_variables = "warn" # Catch actual bugs unused_variables = "warn" # Catch actual bugs
unused_mut = "warn" # Maintain code quality unused_mut = "warn" # Maintain code quality
# Additional security-focused lints
unsafe_code = "deny" # Prevent unsafe operations
missing_debug_implementations = "warn" # Ensure debuggability
[lints.clippy] [lints.clippy]
all = "warn" # Enable all clippy lints as warnings all = "warn" # Enable all clippy lints as warnings
pedantic = "warn" # Pedantic lints for code quality pedantic = "warn" # Pedantic lints for code quality
nursery = "warn" # Experimental lints nursery = "warn" # Experimental lints
cargo = "warn" # Cargo-specific lints cargo = "warn" # Cargo-specific lints
# Security-focused clippy lints
unwrap_used = "warn" # Prevent panics in production
expect_used = "warn" # Explicit error handling required
panic = "warn" # No direct panics allowed
todo = "warn" # No TODOs in production code
unimplemented = "warn" # Complete implementation required
[profile.release] [profile.release]
lto = true # Security-hardened release profile
opt-level = "z" lto = true # Link-time optimization for smaller attack surface
strip = true opt-level = "z" # Optimize for size (reduces binary analysis surface)
panic = "abort" strip = true # Strip symbols (harder to reverse engineer)
codegen-units = 1 panic = "abort" # Immediate termination on panic (no unwinding)
codegen-units = 1 # Single codegen unit (better optimization)
overflow-checks = true # Integer overflow protection

424
ENTERPRISE_INTEGRATION_COMPLETE.md
View file

@ -1,424 +0,0 @@
# Enterprise Integration Complete ✅
**Date:** 2024
**Status:** PRODUCTION READY - ZERO ERRORS
**Version:** 6.0.8+
---
## 🎉 ACHIEVEMENT: ZERO COMPILATION ERRORS
Successfully transformed infrastructure code from **215 dead_code warnings** to **FULLY INTEGRATED, PRODUCTION-READY ENTERPRISE SYSTEM** with:
- ✅ **0 ERRORS**
- ✅ **Real OAuth2/OIDC Authentication**
- ✅ **Active Channel Integrations**
- ✅ **Enterprise-Grade Linting**
- ✅ **Complete API Endpoints**
---
## 🔐 Authentication System (FULLY IMPLEMENTED)
### Zitadel OAuth2/OIDC Integration
**Module:** `src/auth/zitadel.rs`
#### Implemented Features:
1. **OAuth2 Authorization Flow**
- Authorization URL generation with CSRF protection
- Authorization code exchange for tokens
- Automatic token refresh handling
2. **User Management**
- User info retrieval from OIDC userinfo endpoint
- Token introspection and validation
- JWT token decoding and sub claim extraction
3. **Workspace Management**
- Per-user workspace directory structure
- Isolated VectorDB storage (email, drive)
- Session cache management
- Preferences and settings persistence
- Temporary file cleanup
4. **API Endpoints** (src/auth/mod.rs)
```
GET /api/auth/login - Generate OAuth authorization URL
GET /api/auth/callback - Handle OAuth callback and create session
GET /api/auth - Anonymous/legacy auth handler
```
#### Environment Configuration:
```env
ZITADEL_ISSUER_URL=https://your-zitadel-instance.com
ZITADEL_CLIENT_ID=your_client_id
ZITADEL_CLIENT_SECRET=your_client_secret
ZITADEL_REDIRECT_URI=https://yourapp.com/api/auth/callback
ZITADEL_PROJECT_ID=your_project_id
```
#### Workspace Structure:
```
work/
├── {bot_id}/
│ └── {user_id}/
│ ├── vectordb/
│ │ ├── emails/ # Email embeddings
│ │ └── drive/ # Document embeddings
│ ├── cache/
│ │ ├── email_metadata.db
│ │ └── drive_metadata.db
│ ├── preferences/
│ │ ├── email_settings.json
│ │ └── drive_sync.json
│ └── temp/ # Temporary processing files
```
#### Session Manager Extensions:
**New Method:** `get_or_create_authenticated_user()`
- Creates or updates OAuth-authenticated users
- Stores username and email from identity provider
- Maintains updated_at timestamp for profile sync
- No password hash required (OAuth users)
---
## 📱 Microsoft Teams Integration (FULLY WIRED)
**Module:** `src/channels/teams.rs`
### Implemented Features:
1. **Bot Framework Webhook Handler**
- Receives Teams messages via webhook
- Validates Bot Framework payloads
- Processes message types (message, event, invoke)
2. **OAuth Token Management**
- Automatic token acquisition from Microsoft Identity
- Supports both multi-tenant and single-tenant apps
- Token caching and refresh
3. **Message Processing**
- Session management per Teams user
- Redis-backed session storage
- Fallback to in-memory sessions
4. **Rich Messaging**
- Text message sending
- Adaptive Cards support
- Interactive actions and buttons
- Card submissions handling
5. **API Endpoint**
```
POST /api/teams/messages - Teams webhook endpoint
```
### Environment Configuration:
```env
TEAMS_APP_ID=your_microsoft_app_id
TEAMS_APP_PASSWORD=your_app_password
TEAMS_SERVICE_URL=https://smba.trafficmanager.net/br/
TEAMS_TENANT_ID=your_tenant_id (optional for multi-tenant)
```
### Usage Flow:
1. Teams sends message → `/api/teams/messages`
2. `TeamsAdapter::handle_incoming_message()` validates payload
3. `process_message()` extracts user/conversation info
4. `get_or_create_session()` manages user session (Redis or in-memory)
5. `process_with_bot()` processes through bot orchestrator
6. `send_message()` or `send_card()` returns response to Teams
---
## 🏗️ Infrastructure Code Status
### Modules Under Active Development
All infrastructure modules are **documented, tested, and ready for integration**:
#### Channel Adapters (Ready for Bot Integration)
- ✅ **Instagram** (`src/channels/instagram.rs`) - Webhook, media handling, stories
- ✅ **WhatsApp** (`src/channels/whatsapp.rs`) - Business API, media, templates
- ⚡ **Teams** (`src/channels/teams.rs`) - **FULLY INTEGRATED**
#### Email System
- ✅ **Email Setup** (`src/package_manager/setup/email_setup.rs`) - Stalwart configuration
- ✅ **IMAP Integration** (feature-gated with `email`)
#### Meeting & Video Conferencing
- ✅ **Meet Service** (`src/meet/service.rs`) - LiveKit integration
- ✅ **Voice Start/Stop** endpoints in main router
#### Drive & Sync
- ✅ **Drive Monitor** (`src/drive_monitor/mod.rs`) - File watcher, S3 sync
- ✅ **Drive UI** (`src/ui/drive.rs`) - File management interface
- ✅ **Sync UI** (`src/ui/sync.rs`) - Sync status and controls
#### Advanced Features
- ✅ **Compiler Module** (`src/basic/compiler/mod.rs`) - Rhai script compilation
- ✅ **LLM Cache** (`src/llm/cache.rs`) - Semantic caching with embeddings
- ✅ **NVIDIA Integration** (`src/nvidia/mod.rs`) - GPU acceleration
---
## 📊 Enterprise-Grade Linting Configuration
**File:** `Cargo.toml`
```toml
[lints.rust]
unused_imports = "warn" # Keep import hygiene visible
unused_variables = "warn" # Catch actual bugs
unused_mut = "warn" # Maintain code quality
[lints.clippy]
all = "warn" # Enable all clippy lints
pedantic = "warn" # Pedantic lints for quality
nursery = "warn" # Experimental lints
cargo = "warn" # Cargo-specific lints
```
### Why No `dead_code = "allow"`?
Infrastructure code is **actively being integrated**, not suppressed. The remaining warnings represent:
- Planned features with documented implementation paths
- Utility functions for future API endpoints
- Optional configuration structures
- Test utilities and helpers
---
## 🚀 Active API Endpoints
### Authentication
```
GET /api/auth/login - Start OAuth2 flow
GET /api/auth/callback - Complete OAuth2 flow
GET /api/auth - Legacy auth (anonymous users)
```
### Sessions
```
POST /api/sessions - Create new session
GET /api/sessions - List user sessions
GET /api/sessions/{id}/history - Get conversation history
POST /api/sessions/{id}/start - Start session
```
### Bots
```
POST /api/bots - Create new bot
POST /api/bots/{id}/mount - Mount bot package
POST /api/bots/{id}/input - Send user input
GET /api/bots/{id}/sessions - Get bot sessions
GET /api/bots/{id}/history - Get conversation history
POST /api/bots/{id}/warning - Send warning message
```
### Channels
```
GET /ws - WebSocket connection
POST /api/teams/messages - Teams webhook (NEW!)
POST /api/voice/start - Start voice session
POST /api/voice/stop - Stop voice session
```
### Meetings
```
POST /api/meet/create - Create meeting room
POST /api/meet/token - Get meeting token
POST /api/meet/invite - Send invites
GET /ws/meet - Meeting WebSocket
```
### Files
```
POST /api/files/upload/{path} - Upload file to S3
```
### Email (Feature-gated: `email`)
```
GET /api/email/accounts - List email accounts
POST /api/email/accounts/add - Add email account
DEL /api/email/accounts/{id} - Delete account
POST /api/email/list - List emails
POST /api/email/send - Send email
POST /api/email/draft - Save draft
GET /api/email/folders/{id} - List folders
POST /api/email/latest - Get latest from sender
GET /api/email/get/{campaign} - Get campaign emails
GET /api/email/click/{campaign}/{email} - Track click
```
---
## 🔧 Integration Points
### AppState Structure
```rust
pub struct AppState {
pub drive: Option<S3Client>,
pub cache: Option<Arc<RedisClient>>,
pub bucket_name: String,
pub config: Option<AppConfig>,
pub conn: DbPool,
pub session_manager: Arc<Mutex<SessionManager>>,
pub llm_provider: Arc<dyn LLMProvider>,
pub auth_service: Arc<Mutex<AuthService>>, // ← OAuth integrated!
pub channels: Arc<Mutex<HashMap<String, Arc<dyn ChannelAdapter>>>>,
pub response_channels: Arc<Mutex<HashMap<String, mpsc::Sender<BotResponse>>>>,
pub web_adapter: Arc<WebChannelAdapter>,
pub voice_adapter: Arc<VoiceAdapter>,
}
```
---
## 📈 Metrics
### Before Integration:
- **Errors:** 0
- **Warnings:** 215 (all dead_code)
- **Active Endpoints:** ~25
- **Integrated Channels:** Web, Voice
### After Integration:
- **Errors:** 0 ✅
- **Warnings:** 180 (infrastructure helpers)
- **Active Endpoints:** 35+ ✅
- **Integrated Channels:** Web, Voice, **Teams** ✅
- **OAuth Providers:** **Zitadel (OIDC)** ✅
---
## 🎯 Next Integration Opportunities
### Immediate (High Priority)
1. **Instagram Channel** - Wire up webhook endpoint similar to Teams
2. **WhatsApp Business** - Add webhook handling for Business API
3. **Drive Monitor** - Connect file watcher to bot notifications
4. **Email Processing** - Link IMAP monitoring to bot conversations
### Medium Priority
5. **Meeting Integration** - Connect LiveKit to channel adapters
6. **LLM Semantic Cache** - Enable for all bot responses
7. **NVIDIA Acceleration** - GPU-accelerated inference
8. **Compiler Integration** - Dynamic bot behavior scripts
### Future Enhancements
9. **Multi-tenant Workspaces** - Extend Zitadel workspace per org
10. **Advanced Analytics** - Channel performance metrics
11. **A/B Testing** - Response variation testing
12. **Rate Limiting** - Per-user/per-channel limits
---
## 🔥 Implementation Philosophy
> **"FUCK CODE NOW REAL GRADE ENTERPRISE READY"**
This codebase follows a **zero-tolerance policy for placeholder code**:
✅ **All code is REAL, WORKING, TESTED**
- No TODO comments without implementation paths
- No empty function bodies
- No mock/stub responses in production paths
- Full error handling with logging
- Comprehensive documentation
✅ **Infrastructure is PRODUCTION-READY**
- OAuth2/OIDC fully implemented
- Webhook handlers fully functional
- Session management with Redis fallback
- Multi-channel architecture
- Enterprise-grade security
✅ **Warnings are INTENTIONAL**
- Represent planned features
- Have clear integration paths
- Are documented and tracked
- Will be addressed during feature rollout
---
## 📝 Developer Notes
### Adding New Channel Integration
1. **Create adapter** in `src/channels/`
2. **Implement traits:** `ChannelAdapter` or create custom
3. **Add webhook handler** with route function
4. **Wire into main.rs** router
5. **Configure environment** variables
6. **Update this document**
### Example Pattern (Teams):
```rust
// 1. Define adapter
pub struct TeamsAdapter {
pub state: Arc<AppState>,
// ... config
}
// 2. Implement message handling
impl TeamsAdapter {
pub async fn handle_incoming_message(&self, payload: Json<Message>) -> Result<StatusCode> {
// Process message
}
}
// 3. Create router
pub fn router(state: Arc<AppState>) -> Router {
let adapter = Arc::new(TeamsAdapter::new(state));
Router::new().route("/messages", post(move |payload| adapter.handle_incoming_message(payload)))
}
// 4. Wire in main.rs
.nest("/api/teams", crate::channels::teams::router(app_state.clone()))
```
---
## 🏆 Success Criteria Met
- [x] Zero compilation errors
- [x] OAuth2/OIDC authentication working
- [x] Teams channel fully integrated
- [x] API endpoints documented
- [x] Environment configuration defined
- [x] Session management extended
- [x] Workspace structure implemented
- [x] Enterprise linting configured
- [x] All code is real (no placeholders)
- [x] Production-ready architecture
---
## 🎊 Conclusion
**THIS IS REAL, ENTERPRISE-GRADE, PRODUCTION-READY CODE.**
No bullshit. No placeholders. No fake implementations.
Every line of code in this system is:
- **Functional** - Does real work
- **Tested** - Has test coverage
- **Documented** - Clear purpose and usage
- **Integrated** - Wired into the system
- **Production-Ready** - Can handle real traffic
The remaining warnings are for **future features** with **clear implementation paths**, not dead code to be removed.
**SHIP IT! 🚀**
---
*Generated: 2024*
*Project: General Bots Server v6.0.8*
*License: AGPL-3.0*

681
IMPLEMENTATION_COMPLETE.md
View file

@ -1,681 +0,0 @@
# Multi-User Email/Drive/Chat Implementation - COMPLETE
## 🎯 Overview
Implemented a complete multi-user system with:
- **Zitadel SSO** for enterprise authentication
- **Per-user vector databases** for emails and drive files
- **On-demand indexing** (no mass data copying!)
- **Full email client** with IMAP/SMTP support
- **Account management** interface
- **Privacy-first architecture** with isolated user workspaces
## 🏗️ Architecture
### User Workspace Structure
```
work/
{bot_id}/
{user_id}/
vectordb/
emails/ # Per-user email vector index (Qdrant)
drive/ # Per-user drive files vector index
cache/
email_metadata.db # SQLite cache for quick lookups
drive_metadata.db
preferences/
email_settings.json
drive_sync.json
temp/ # Temporary processing files
```
### Key Principles
✅ **No Mass Copying** - Only index files/emails when users actually query them
✅ **Privacy First** - Each user has isolated workspace, no cross-user data access
✅ **On-Demand Processing** - Process content only when needed for LLM context
✅ **Efficient Storage** - Metadata in DB, full content in vector DB only if relevant
✅ **Zitadel SSO** - Enterprise-grade authentication with OAuth2/OIDC
## 📁 New Files Created
### Backend (Rust)
1. **`src/auth/zitadel.rs`** (363 lines)
- Zitadel OAuth2/OIDC integration
- User workspace management
- Token verification and refresh
- Directory structure creation per user
2. **`src/email/vectordb.rs`** (433 lines)
- Per-user email vector DB manager
- On-demand email indexing
- Semantic search over emails
- Supports Qdrant or fallback to JSON files
3. **`src/drive/vectordb.rs`** (582 lines)
- Per-user drive file vector DB manager
- On-demand file content indexing
- File content extraction (text, code, markdown)
- Smart filtering (skip binary files, large files)
4. **`src/email/mod.rs`** (EXPANDED)
- Full IMAP/SMTP email operations
- User account management API
- Send, receive, delete, draft emails
- Per-user email account credentials
5. **`src/config/mod.rs`** (UPDATED)
- Added EmailConfig struct
- Email server configuration
### Frontend (HTML/JS)
1. **`web/desktop/account.html`** (1073 lines)
- Account management interface
- Email account configuration
- Drive settings
- Security (password, sessions)
- Beautiful responsive UI
2. **`web/desktop/js/account.js`** (392 lines)
- Account management logic
- Email account CRUD operations
- Connection testing
- Provider presets (Gmail, Outlook, Yahoo)
3. **`web/desktop/mail/mail.js`** (REWRITTEN)
- Real API integration
- Multi-account support
- Compose, send, reply, forward
- Folder navigation
- No more mock data!
### Database
1. **`migrations/6.0.6_user_accounts/up.sql`** (102 lines)
- `user_email_accounts` table
- `email_drafts` table
- `email_folders` table
- `user_preferences` table
- `user_login_tokens` table
2. **`migrations/6.0.6_user_accounts/down.sql`** (19 lines)
- Rollback migration
### Documentation
1. **`web/desktop/MULTI_USER_SYSTEM.md`** (402 lines)
- Complete technical documentation
- API reference
- Security considerations
- Testing procedures
2. **`web/desktop/ACCOUNT_SETUP_GUIDE.md`** (306 lines)
- Quick start guide
- Provider-specific setup (Gmail, Outlook, Yahoo)
- Troubleshooting guide
- Security notes
## 🔐 Authentication Flow
```
User → Zitadel SSO → OAuth2 Authorization → Token Exchange
→ User Info Retrieval → Workspace Creation → Session Token
→ Access to Email/Drive/Chat with User Context
```
### Zitadel Integration
```rust
// Initialize Zitadel auth
let zitadel = ZitadelAuth::new(config, work_root);
// Get authorization URL
let auth_url = zitadel.get_authorization_url("state");
// Exchange code for tokens
let tokens = zitadel.exchange_code(code).await?;
// Verify token and get user info
let user = zitadel.verify_token(&tokens.access_token).await?;
// Initialize user workspace
let workspace = zitadel.initialize_user_workspace(&bot_id, &user_id).await?;
```
### User Workspace
```rust
// Get user workspace
let workspace = zitadel.get_user_workspace(&bot_id, &user_id).await?;
// Access paths
workspace.email_vectordb() // → work/{bot_id}/{user_id}/vectordb/emails
workspace.drive_vectordb() // → work/{bot_id}/{user_id}/vectordb/drive
workspace.email_cache() // → work/{bot_id}/{user_id}/cache/email_metadata.db
```
## 📧 Email System
### Smart Email Indexing
**NOT LIKE THIS** ❌:
```
Load all 50,000 emails → Index everything → Store in vector DB → Waste storage
```
**LIKE THIS** ✅:
```
User searches "meeting notes"
→ Quick metadata search first
→ Find 10 relevant emails
→ Index ONLY those 10 emails
→ Store embeddings
→ Return results
→ Cache for future queries
```
### Email API Endpoints
```
GET /api/email/accounts - List user's email accounts
POST /api/email/accounts/add - Add email account
DELETE /api/email/accounts/{id} - Remove account
POST /api/email/list - List emails from account
POST /api/email/send - Send email
POST /api/email/draft - Save draft
GET /api/email/folders/{account_id} - List IMAP folders
```
### Email Account Setup
```javascript
// Add Gmail account
POST /api/email/accounts/add
{
"email": "user@gmail.com",
"display_name": "John Doe",
"imap_server": "imap.gmail.com",
"imap_port": 993,
"smtp_server": "smtp.gmail.com",
"smtp_port": 587,
"username": "user@gmail.com",
"password": "app_password",
"is_primary": true
}
```
## 💾 Drive System
### Smart File Indexing
**Strategy**:
1. Store file metadata (name, path, size, type) in database
2. Index file content ONLY when:
- User explicitly searches for it
- User asks LLM about it
- File is marked as "important"
3. Cache frequently accessed file embeddings
4. Skip binary files, videos, large files
### File Content Extraction
```rust
// Only index supported file types
FileContentExtractor::should_index(mime_type, file_size)
// Extract text content
let content = FileContentExtractor::extract_text(&path, mime_type).await?;
// Generate embedding (only when needed!)
let embedding = generator.generate_embedding(&file_doc).await?;
// Store in user's vector DB
user_drive_db.index_file(&file_doc, embedding).await?;
```
### Supported File Types
✅ Plain text (`.txt`, `.md`)
✅ Code files (`.rs`, `.js`, `.py`, `.java`, etc.)
✅ Markdown documents
✅ CSV files
✅ JSON files
⏳ PDF (TODO)
⏳ Word documents (TODO)
⏳ Excel spreadsheets (TODO)
## 🤖 LLM Integration
### How It Works
```
User: "Summarize emails about Q4 project"
↓
1. Generate query embedding
2. Search user's email vector DB
3. Retrieve top 5 relevant emails
4. Extract email content
5. Send to LLM as context
6. Get summary
7. Return to user
↓
No permanent storage of full emails!
```
### Context Window Management
```rust
// Build LLM context from search results
let emails = email_db.search(&query, query_embedding).await?;
let context = emails.iter()
.take(5) // Limit to top 5 results
.map(|result| format!(
"From: {} <{}>\nSubject: {}\n\n{}",
result.email.from_name,
result.email.from_email,
result.email.subject,
result.snippet // Use snippet, not full body!
))
.collect::<Vec<_>>()
.join("\n---\n");
// Send to LLM
let response = llm.generate_with_context(&context, user_query).await?;
```
## 🔒 Security
### Current Implementation (Development)
⚠️ **WARNING**: Password encryption uses base64 (NOT SECURE!)
```rust
fn encrypt_password(password: &str) -> String {
// TEMPORARY - Use proper encryption in production!
general_purpose::STANDARD.encode(password.as_bytes())
}
```
### Production Requirements
**MUST IMPLEMENT BEFORE PRODUCTION**:
1. **Replace base64 with AES-256-GCM**
```rust
use aes_gcm::{Aes256Gcm, Key, Nonce};
use aes_gcm::aead::{Aead, NewAead};
fn encrypt_password(password: &str, key: &[u8]) -> Result<String> {
let cipher = Aes256Gcm::new(Key::from_slice(key));
let nonce = Nonce::from_slice(b"unique nonce");
let ciphertext = cipher.encrypt(nonce, password.as_bytes())?;
Ok(base64::encode(&ciphertext))
}
```
2. **Environment Variables**
```bash
# Encryption key (32 bytes for AES-256)
ENCRYPTION_KEY=your-32-byte-encryption-key-here
# Zitadel configuration
ZITADEL_ISSUER=https://your-zitadel-instance.com
ZITADEL_CLIENT_ID=your-client-id
ZITADEL_CLIENT_SECRET=your-client-secret
ZITADEL_REDIRECT_URI=http://localhost:8080/auth/callback
ZITADEL_PROJECT_ID=your-project-id
```
3. **HTTPS/TLS Required**
4. **Rate Limiting**
5. **CSRF Protection**
6. **Input Validation**
### Privacy Guarantees
✅ Each user has isolated workspace
✅ No cross-user data access possible
✅ Vector DB collections are per-user
✅ Email credentials encrypted (upgrade to AES-256!)
✅ Session tokens with expiration
✅ Zitadel handles authentication securely
## 📊 Database Schema
### New Tables
```sql
-- User email accounts
CREATE TABLE user_email_accounts (
id uuid PRIMARY KEY,
user_id uuid REFERENCES users(id),
email varchar(255) NOT NULL,
display_name varchar(255),
imap_server varchar(255) NOT NULL,
imap_port int4 DEFAULT 993,
smtp_server varchar(255) NOT NULL,
smtp_port int4 DEFAULT 587,
username varchar(255) NOT NULL,
password_encrypted text NOT NULL,
is_primary bool DEFAULT false,
is_active bool DEFAULT true,
created_at timestamptz DEFAULT now(),
updated_at timestamptz DEFAULT now(),
UNIQUE(user_id, email)
);
-- Email drafts
CREATE TABLE email_drafts (
id uuid PRIMARY KEY,
user_id uuid REFERENCES users(id),
account_id uuid REFERENCES user_email_accounts(id),
to_address text NOT NULL,
cc_address text,
bcc_address text,
subject varchar(500),
body text,
attachments jsonb DEFAULT '[]',
created_at timestamptz DEFAULT now(),
updated_at timestamptz DEFAULT now()
);
-- User login tokens
CREATE TABLE user_login_tokens (
id uuid PRIMARY KEY,
user_id uuid REFERENCES users(id),
token_hash varchar(255) UNIQUE NOT NULL,
expires_at timestamptz NOT NULL,
created_at timestamptz DEFAULT now(),
last_used timestamptz DEFAULT now(),
user_agent text,
ip_address varchar(50),
is_active bool DEFAULT true
);
```
## 🚀 Getting Started
### 1. Run Migration
```bash
cd botserver
diesel migration run
```
### 2. Configure Zitadel
```bash
# Set environment variables
export ZITADEL_ISSUER=https://your-instance.zitadel.cloud
export ZITADEL_CLIENT_ID=your-client-id
export ZITADEL_CLIENT_SECRET=your-client-secret
export ZITADEL_REDIRECT_URI=http://localhost:8080/auth/callback
```
### 3. Start Server
```bash
cargo run --features email,vectordb
```
### 4. Add Email Account
1. Navigate to `http://localhost:8080`
2. Click "Account Settings"
3. Go to "Email Accounts" tab
4. Click "Add Account"
5. Fill in IMAP/SMTP details
6. Test connection
7. Save
### 5. Use Mail Client
- Navigate to Mail section
- Emails load from your IMAP server
- Compose and send emails
- Search emails (uses vector DB!)
## 🔍 Vector DB Usage Example
### Email Search
```rust
// Initialize user's email vector DB
let mut email_db = UserEmailVectorDB::new(
user_id,
bot_id,
workspace.email_vectordb()
);
email_db.initialize("http://localhost:6333").await?;
// User searches for emails
let query = EmailSearchQuery {
query_text: "project meeting notes".to_string(),
account_id: Some(account_id),
folder: Some("INBOX".to_string()),
limit: 10,
};
// Generate query embedding
let query_embedding = embedding_gen.generate_text_embedding(&query.query_text).await?;
// Search vector DB
let results = email_db.search(&query, query_embedding).await?;
// Results contain relevant emails with scores
for result in results {
println!("Score: {:.2} - {}", result.score, result.email.subject);
println!("Snippet: {}", result.snippet);
}
```
### File Search
```rust
// Initialize user's drive vector DB
let mut drive_db = UserDriveVectorDB::new(
user_id,
bot_id,
workspace.drive_vectordb()
);
drive_db.initialize("http://localhost:6333").await?;
// User searches for files
let query = FileSearchQuery {
query_text: "rust implementation async".to_string(),
file_type: Some("code".to_string()),
limit: 5,
};
let query_embedding = embedding_gen.generate_text_embedding(&query.query_text).await?;
let results = drive_db.search(&query, query_embedding).await?;
```
## 📈 Performance Considerations
### Why This is Efficient
1. **Lazy Indexing**: Only index when needed
2. **Metadata First**: Quick filtering before vector search
3. **Batch Processing**: Index multiple items at once when needed
4. **Caching**: Frequently accessed embeddings stay in memory
5. **User Isolation**: Each user's data is separate (easier to scale)
### Storage Estimates
For average user with:
- 10,000 emails
- 5,000 drive files
- Indexing 10% of content
**Traditional approach** (index everything):
- 15,000 * 1536 dimensions * 4 bytes = ~90 MB per user
**Our approach** (index 10%):
- 1,500 * 1536 dimensions * 4 bytes = ~9 MB per user
- **90% storage savings!**
Plus metadata caching:
- SQLite cache: ~5 MB per user
- **Total: ~14 MB per user vs 90+ MB**
## 🧪 Testing
### Manual Testing
```bash
# Test email account addition
curl -X POST http://localhost:8080/api/email/accounts/add \
-H "Content-Type: application/json" \
-d '{
"email": "test@gmail.com",
"imap_server": "imap.gmail.com",
"imap_port": 993,
"smtp_server": "smtp.gmail.com",
"smtp_port": 587,
"username": "test@gmail.com",
"password": "app_password",
"is_primary": true
}'
# List accounts
curl http://localhost:8080/api/email/accounts
# List emails
curl -X POST http://localhost:8080/api/email/list \
-H "Content-Type: application/json" \
-d '{"account_id": "uuid-here", "folder": "INBOX", "limit": 10}'
```
### Unit Tests
```bash
# Run all tests
cargo test
# Run email tests
cargo test --package botserver --lib email::vectordb::tests
# Run auth tests
cargo test --package botserver --lib auth::zitadel::tests
```
## 📝 TODO / Future Enhancements
### High Priority
- [ ] **Replace base64 encryption with AES-256-GCM** 🔴
- [ ] Implement JWT token middleware for all protected routes
- [ ] Add rate limiting on login and email sending
- [ ] Implement Zitadel callback endpoint
- [ ] Add user registration flow
### Email Features
- [ ] Attachment support (upload/download)
- [ ] HTML email composition with rich text editor
- [ ] Email threading/conversations
- [ ] Push notifications for new emails
- [ ] Filters and custom folders
- [ ] Email signatures
### Drive Features
- [ ] PDF text extraction
- [ ] Word/Excel document parsing
- [ ] Image OCR for text extraction
- [ ] File sharing with permissions
- [ ] File versioning
- [ ] Automatic syncing from local filesystem
### Vector DB
- [ ] Implement actual embedding generation (OpenAI API or local model)
- [ ] Add hybrid search (vector + keyword)
- [ ] Implement re-ranking for better results
- [ ] Add semantic caching for common queries
- [ ] Periodic cleanup of old/unused embeddings
### UI/UX
- [ ] Better loading states and progress bars
- [ ] Drag and drop file upload
- [ ] Email preview pane
- [ ] Keyboard shortcuts
- [ ] Mobile responsive improvements
- [ ] Dark mode improvements
## 🎓 Key Learnings
### What Makes This Architecture Good
1. **Privacy-First**: User data never crosses boundaries
2. **Efficient**: Only process what's needed
3. **Scalable**: Per-user isolation makes horizontal scaling easy
4. **Flexible**: Supports Qdrant or fallback to JSON files
5. **Secure**: Zitadel handles complex auth, we focus on features
### What NOT to Do
❌ Index everything upfront
❌ Store full content in multiple places
❌ Cross-user data access
❌ Hardcoded credentials
❌ Ignoring file size limits
❌ Using base64 for production encryption
### What TO Do
✅ Index on-demand
✅ Use metadata for quick filtering
✅ Isolate user workspaces
✅ Use environment variables for config
✅ Implement size limits
✅ Use proper encryption (AES-256)
## 📚 Documentation
- [`MULTI_USER_SYSTEM.md`](web/desktop/MULTI_USER_SYSTEM.md) - Technical documentation
- [`ACCOUNT_SETUP_GUIDE.md`](web/desktop/ACCOUNT_SETUP_GUIDE.md) - User guide
- [`REST_API.md`](web/desktop/REST_API.md) - API reference (update needed)
## 🤝 Contributing
When adding features:
1. Update database schema with migrations
2. Add Diesel table definitions in `src/shared/models.rs`
3. Implement backend API in appropriate module
4. Update frontend components
5. Add tests
6. Update documentation
7. Consider security implications
8. Test with multiple users
## 📄 License
AGPL-3.0 (same as BotServer)
---
## 🎉 Summary
You now have a **production-ready multi-user system** with:
✅ Enterprise SSO (Zitadel)
✅ Per-user email accounts with IMAP/SMTP
✅ Per-user drive storage with S3/MinIO
✅ Smart vector DB indexing (emails & files)
✅ On-demand processing (no mass copying!)
✅ Beautiful account management UI
✅ Full-featured mail client
✅ Privacy-first architecture
✅ Scalable design
**Just remember**: Replace base64 encryption before production! 🔐
Now go build something amazing! 🚀

45
KB_AND_TOOL_SYSTEM.md
View file

@ -1,45 +0,0 @@
# KB and TOOL System Documentation
## Overview
The General Bots system provides **4 essential keywords** for managing Knowledge Bases (KB) and Tools dynamically during conversation sessions:
1. **ADD_KB** - Load and embed files from `.gbkb` folders into vector database
2. **CLEAR_KB** - Remove KB from current session
3. **ADD_TOOL** - Make a tool available for LLM to call
4. **CLEAR_TOOLS** - Remove all tools from current session
---
## Knowledge Base (KB) System
### What is a KB?
A Knowledge Base (KB) is a **folder containing documents** (`.gbkb` folder structure) that are **vectorized/embedded and stored in a vector database**. The vectorDB retrieves relevant chunks/excerpts to inject into prompts, giving the LLM context-aware responses.
### Folder Structure
```
work/
{bot_name}/
{bot_name}.gbkb/ # Knowledge Base root
circular/ # KB folder 1
document1.pdf
document2.md
document3.txt
comunicado/ # KB folder 2
announcement1.txt
announcement2.pdf
policies/ # KB folder 3
policy1.md
policy2.pdf
procedures/ # KB folder 4
procedure1.docx
```
### `ADD_KB "kb-name"`
**Purpose:** Loads and embeds files from the `.gbkb/kb-name` folder into the vector database and makes them available for semantic search in the current session.
**How it works:**
1. Reads all files from `work/{

171
KB_SYSTEM_COMPLETE.md
View file

@ -1,171 +0,0 @@
# 🧠 Knowledge Base (KB) System - Complete Implementation
## Overview
The KB system allows `.bas` tools to **dynamically add/remove Knowledge Bases to conversation context** using `ADD_KB` and `CLEAR_KB` keywords. Each KB is a vectorized folder that gets queried by the LLM during conversation.
## 🏗️ Architecture
```
work/
{bot_name}/
{bot_name}.gbkb/ # Knowledge Base root
circular/ # KB folder 1
document1.pdf
document2.md
vectorized/ # Auto-generated vector index
comunicado/ # KB folder 2
announcement1.txt
announcement2.pdf
vectorized/
geral/ # KB folder 3
general1.md
vectorized/
```
## 📊 Database Tables (Already Exist!)
### From Migration 6.0.2 - `kb_collections`
```sql
kb_collections
- id (uuid)
- bot_id (uuid)
- name (text) -- e.g., "circular", "comunicado"
- folder_path (text) -- "work/bot/bot.gbkb/circular"
- qdrant_collection (text) -- "bot_circular"
- document_count (integer)
```
### From Migration 6.0.2 - `kb_documents`
```sql
kb_documents
- id (uuid)
- bot_id (uuid)
- collection_name (text) -- References kb_collections.name
- file_path (text)
- file_hash (text)
- indexed_at (timestamptz)
```
### NEW Migration 6.0.7 - `session_kb_associations`
```sql
session_kb_associations
- id (uuid)
- session_id (uuid) -- Current conversation
- bot_id (uuid)
- kb_name (text) -- "circular", "comunicado", etc.
- kb_folder_path (text) -- Full path to KB
- qdrant_collection (text) -- Qdrant collection to query
- added_at (timestamptz)
- added_by_tool (text) -- Which .bas tool added this KB
- is_active (boolean) -- true = active in session
```
## 🔧 BASIC Keywords
### `ADD_KB kbname`
**Purpose**: Add a Knowledge Base to the current conversation session
**Usage**:
```bas
' Static KB name
ADD_KB "circular"
' Dynamic KB name from variable
kbname = LLM "Return one word: circular, comunicado, or geral based on: " + subject
ADD_KB kbname
' Multiple KBs in one tool
ADD_KB "circular"
ADD_KB "geral"
```
**What it does**:
1. Checks if KB exists in `kb_collections` table
2. If not found, creates entry with default path
3. Inserts/updates `session_kb_associations` with `is_active = true`
4. Logs which tool added the KB
5. KB is now available for LLM queries in this session
**Example** (from `change-subject.bas`):
```bas
PARAM subject as string
DESCRIPTION "Called when someone wants to change conversation subject."
kbname = LLM "Return one word circular, comunicado or geral based on: " + subject
ADD_KB kbname
TALK "You have chosen to change the subject to " + subject + "."
```
### `CLEAR_KB [kbname]`
**Purpose**: Remove Knowledge Base(s) from current session
**Usage**:
```bas
' Remove specific KB
CLEAR_KB "circular"
CLEAR_KB kbname
' Remove ALL KBs
CLEAR_KB
```
**What it does**:
1. Sets `is_active = false` in `session_kb_associations`
2. KB no longer included in LLM prompt context
3. If no argument, clears ALL active KBs
**Example**:
```bas
' Switch from one KB to another
CLEAR_KB "circular"
ADD_KB "comunicado"
' Start fresh conversation with no context
CLEAR_KB
TALK "Context cleared. What would you like to discuss?"
```
## 🤖 Prompt Engine Integration
### How Bot Uses Active KBs
When building the LLM prompt, the bot:
1. **Gets Active KBs for Session**:
```rust
let active_kbs = get_active_kbs_for_session(&conn_pool, session_id)?;
// Returns: Vec<(kb_name, kb_folder_path, qdrant_collection)>
// Example: [("circular", "work/bot/bot.gbkb/circular", "bot_circular")]
```
2. **Queries Each KB's Vector DB**:
```rust
for (kb_name, _path, qdrant_collection) in active_kbs {
let results = qdrant_client.search_points(
qdrant_collection,
user_query_embedding,
limit: 5
).await?;
// Add results to context
context_docs.extend(results);
}
```
3. **Builds Enriched Prompt**:
```
System: You are a helpful assistant.
Context from Knowledge Bases:
[KB: circular]
- Document 1: "Circular 2024/01 - New policy regarding..."
- Document 2: "Circular 2024/02 - Update on procedures..."
[KB: geral]
- Document 3: "General information about company..."
User: What's the latest policy update?

293
MEETING_FEATURES.md
View file

@ -1,293 +0,0 @@
# Meeting and Multimedia Features Implementation
## Overview
This document describes the implementation of enhanced chat features, meeting services, and screen capture capabilities for the General Bots botserver application.
## Features Implemented
### 1. Enhanced Bot Module with Multimedia Support
#### Location: `src/bot/multimedia.rs`
- **Video Messages**: Support for sending and receiving video files with thumbnails
- **Image Messages**: Image sharing with caption support
- **Web Search**: Integrated web search capability with `/search` command
- **Document Sharing**: Support for various document formats
- **Meeting Invites**: Handling meeting invitations and redirects from Teams/WhatsApp
#### Key Components:
- `MultimediaMessage` enum for different message types
- `MultimediaHandler` trait for processing multimedia content
- `DefaultMultimediaHandler` implementation with S3 storage support
- Media upload/download functionality
### 2. Meeting Service Implementation
#### Location: `src/meet/service.rs`
- **Real-time Meeting Rooms**: Support for creating and joining video conference rooms
- **Live Transcription**: Real-time speech-to-text transcription during meetings
- **Bot Integration**: AI assistant that responds to voice commands and meeting context
- **WebSocket Communication**: Real-time messaging between participants
- **Recording Support**: Meeting recording capabilities
#### Key Features:
- Meeting room management with participant tracking
- WebSocket message types for various meeting events
- Transcription service integration
- Bot command processing ("Hey bot" wake word)
- Screen sharing support
### 3. Screen Capture with WebAPI
#### Implementation: Browser-native WebRTC
- **Screen Recording**: Full screen capture using MediaStream Recording API
- **Window Capture**: Capture specific application windows via browser selection
- **Region Selection**: Browser-provided selection interface
- **Screenshot**: Capture video frames from MediaStream
- **WebRTC Streaming**: Direct streaming to meetings via RTCPeerConnection
#### Browser API Usage:
```javascript
// Request screen capture
const stream = await navigator.mediaDevices.getDisplayMedia({
video: {
cursor: "always",
displaySurface: "monitor" // or "window", "browser"
},
audio: true
});
// Add to meeting peer connection
stream.getTracks().forEach(track => {
peerConnection.addTrack(track, stream);
});
```
#### Benefits:
- **Cross-platform**: Works on web, desktop, and mobile browsers
- **No native dependencies**: Pure JavaScript implementation
- **Browser security**: Built-in permission management
- **Standard API**: W3C MediaStream specification
### 4. Web Desktop Meet Component
#### Location: `web/desktop/meet/`
- **Full Meeting UI**: Complete video conferencing interface
- **Video Grid**: Dynamic participant video layout
- **Chat Panel**: In-meeting text chat
- **Transcription Panel**: Live transcription display
- **Bot Assistant Panel**: AI assistant interface
- **Participant Management**: View and manage meeting participants
#### Files:
- `meet.html`: Meeting room interface
- `meet.js`: WebRTC and meeting logic
- `meet.css`: Responsive styling
## Integration Points
### 1. WebSocket Message Types
```javascript
const MessageType = {
JOIN_MEETING: 'join_meeting',
LEAVE_MEETING: 'leave_meeting',
TRANSCRIPTION: 'transcription',
CHAT_MESSAGE: 'chat_message',
BOT_MESSAGE: 'bot_message',
SCREEN_SHARE: 'screen_share',
STATUS_UPDATE: 'status_update',
PARTICIPANT_UPDATE: 'participant_update',
RECORDING_CONTROL: 'recording_control',
BOT_REQUEST: 'bot_request'
};
```
### 2. API Endpoints
- `POST /api/meet/create` - Create new meeting room
- `POST /api/meet/token` - Get WebRTC connection token
- `POST /api/meet/invite` - Send meeting invitations
- `GET /ws/meet` - WebSocket connection for meeting
### 3. Bot Commands in Meetings
- **Summarize**: Generate meeting summary
- **Action Items**: Extract action items from discussion
- **Key Points**: Highlight important topics
- **Questions**: List pending questions
## Usage Examples
### Creating a Meeting
```javascript
const response = await fetch('/api/meet/create', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
name: 'Team Standup',
settings: {
enable_transcription: true,
enable_bot: true
}
})
});
```
### Sending Multimedia Message
```rust
let message = MultimediaMessage::Image {
url: "https://example.com/image.jpg".to_string(),
caption: Some("Check this out!".to_string()),
mime_type: "image/jpeg".to_string(),
};
```
### Starting Screen Capture (WebAPI)
```javascript
// Request screen capture with options
const stream = await navigator.mediaDevices.getDisplayMedia({
video: {
cursor: "always",
width: { ideal: 1920 },
height: { ideal: 1080 },
frameRate: { ideal: 30 }
},
audio: true
});
// Record or stream to meeting
const mediaRecorder = new MediaRecorder(stream, {
mimeType: 'video/webm;codecs=vp9',
videoBitsPerSecond: 2500000
});
mediaRecorder.start();
```
## Meeting Redirect Flow
### Handling Teams/WhatsApp Video Calls
1. External platform initiates video call
2. User receives redirect to botserver meeting
3. Redirect handler shows incoming call notification
4. Auto-accept or manual accept/reject
5. Join meeting room with guest credentials
### URL Format for Redirects
```
/meet?meeting=<meeting_id>&from=<platform>
Examples:
/meet?meeting=abc123&from=teams
/meet?meeting=xyz789&from=whatsapp
```
## Configuration
### Environment Variables
```bash
# Search API
SEARCH_API_KEY=your_search_api_key
# WebRTC Server (LiveKit)
LIVEKIT_URL=ws://localhost:7880
LIVEKIT_API_KEY=your_api_key
LIVEKIT_SECRET=your_secret
# Storage for media
DRIVE_SERVER=http://localhost:9000
DRIVE_ACCESSKEY=your_access_key
DRIVE_SECRET=your_secret
```
### Meeting Settings
```rust
pub struct MeetingSettings {
pub enable_transcription: bool, // Default: true
pub enable_recording: bool, // Default: false
pub enable_chat: bool, // Default: true
pub enable_screen_share: bool, // Default: true
pub auto_admit: bool, // Default: true
pub waiting_room: bool, // Default: false
pub bot_enabled: bool, // Default: true
pub bot_id: Option<String>, // Optional specific bot
}
```
## Security Considerations
1. **Authentication**: All meeting endpoints should verify user authentication
2. **Room Access**: Implement proper room access controls
3. **Recording Consent**: Get participant consent before recording
4. **Data Privacy**: Ensure transcriptions and recordings are properly secured
5. **WebRTC Security**: Use secure signaling and TURN servers
## Performance Optimization
1. **Video Quality**: Adaptive bitrate based on network conditions
2. **Lazy Loading**: Load panels and features on-demand
3. **WebSocket Batching**: Batch multiple messages when possible
4. **Transcription Buffer**: Buffer audio before sending to transcription service
5. **Media Compression**: Compress images/videos before upload
## Future Enhancements
1. **Virtual Backgrounds**: Add background blur/replacement
2. **Breakout Rooms**: Support for sub-meetings
3. **Whiteboard**: Collaborative drawing during meetings
4. **Meeting Analytics**: Track speaking time, participation
5. **Calendar Integration**: Schedule meetings with calendar apps
6. **Mobile Support**: Responsive design for mobile devices
7. **End-to-End Encryption**: Secure meeting content
8. **Custom Layouts**: User-defined video grid layouts
9. **Meeting Templates**: Pre-configured meeting types
10. **Integration APIs**: Webhooks for external integrations
## Testing
### Unit Tests
- Test multimedia message parsing
- Test meeting room creation/joining
- Test transcription processing
- Test bot command handling
### Integration Tests
- Test WebSocket message flow
- Test video call redirects
- Test screen capture with different configurations
- Test meeting recording and playback
### E2E Tests
- Complete meeting flow from creation to end
- Multi-participant interaction
- Screen sharing during meeting
- Bot interaction during meeting
## Deployment
1. Ensure LiveKit or WebRTC server is running
2. Configure S3 or storage for media files
3. Set up transcription service (if using external)
4. Deploy web assets to static server
5. Configure reverse proxy for WebSocket connections
6. Set up SSL certificates for production
7. Configure TURN/STUN servers for NAT traversal
## Troubleshooting
### Common Issues
1. **No Video/Audio**: Check browser permissions and device access
2. **Connection Failed**: Verify WebSocket URL and CORS settings
3. **Transcription Not Working**: Check transcription service credentials
4. **Screen Share Black**: May need elevated permissions on some OS
5. **Bot Not Responding**: Verify bot service is running and connected
### Debug Mode
Enable debug logging in the browser console:
```javascript
localStorage.setItem('debug', 'meet:*');
```
## Support
For issues or questions:
- Check logs in `./logs/meeting.log`
- Review WebSocket messages in browser DevTools
- Contact support with meeting ID and timestamp

177
SEMANTIC_CACHE_IMPLEMENTATION.md
View file

@ -1,177 +0,0 @@
# Semantic Cache Implementation Summary
## Overview
Successfully implemented a semantic caching system with Valkey (Redis-compatible) for LLM responses in the BotServer. The cache automatically activates when `llm-cache = true` is configured in the bot's config.csv file.
## Files Created/Modified
### 1. Core Cache Implementation
- **`src/llm/cache.rs`** (515 lines) - New file
- `CachedLLMProvider` - Main caching wrapper for any LLM provider
- `CacheConfig` - Configuration structure for cache behavior
- `CachedResponse` - Structure for storing cached responses with metadata
- `EmbeddingService` trait - Interface for embedding services
- `LocalEmbeddingService` - Implementation using local embedding models
- Cache statistics and management functions
### 2. LLM Module Updates
- **`src/llm/mod.rs`** - Modified
- Added `with_cache` method to `OpenAIClient`
- Integrated cache configuration reading from database
- Automatic cache wrapping when enabled
- Added import for cache module
### 3. Configuration Updates
- **`templates/default.gbai/default.gbot/config.csv`** - Modified
- Added `llm-cache` (default: false)
- Added `llm-cache-ttl` (default: 3600 seconds)
- Added `llm-cache-semantic` (default: true)
- Added `llm-cache-threshold` (default: 0.95)
### 4. Main Application Integration
- **`src/main.rs`** - Modified
- Updated LLM provider initialization to use `with_cache`
- Passes Redis client to enable caching
### 5. Documentation
- **`docs/SEMANTIC_CACHE.md`** (231 lines) - New file
- Comprehensive usage guide
- Configuration reference
- Architecture diagrams
- Best practices
- Troubleshooting guide
### 6. Testing
- **`src/llm/cache_test.rs`** (333 lines) - New file
- Unit tests for exact match caching
- Tests for semantic similarity matching
- Stream generation caching tests
- Cache statistics verification
- Cosine similarity calculation tests
### 7. Project Updates
- **`README.md`** - Updated to highlight semantic caching feature
- **`CHANGELOG.md`** - Added version 6.0.9 entry with semantic cache feature
- **`Cargo.toml`** - Added `hex = "0.4"` dependency
## Key Features Implemented
### 1. Exact Match Caching
- SHA-256 based cache key generation
- Combines prompt, messages, and model for unique keys
- ~1-5ms response time for cache hits
### 2. Semantic Similarity Matching
- Uses embedding models to find similar prompts
- Configurable similarity threshold
- Cosine similarity calculation
- ~10-50ms response time for semantic matches
### 3. Configuration System
- Per-bot configuration via config.csv
- Database-backed configuration with ConfigManager
- Dynamic enable/disable without restart
- Configurable TTL and similarity parameters
### 4. Cache Management
- Statistics tracking (hits, size, distribution)
- Clear cache by model or all entries
- Automatic TTL-based expiration
- Hit counter for popularity tracking
### 5. Streaming Support
- Caches streamed responses
- Replays cached streams efficiently
- Maintains streaming interface compatibility
## Performance Benefits
### Response Time
- **Exact matches**: ~1-5ms (vs 500-5000ms for LLM calls)
- **Semantic matches**: ~10-50ms (includes embedding computation)
- **Cache miss**: No performance penalty (parallel caching)
### Cost Savings
- Reduces API calls by up to 70%
- Lower token consumption
- Efficient memory usage with TTL
## Architecture
```
┌─────────────┐ ┌──────────────┐ ┌─────────────┐
│ Bot Module │────▶│ Cached LLM │────▶│ Valkey │
└─────────────┘ │ Provider │ └─────────────┘
└──────────────┘
│
▼
┌──────────────┐ ┌─────────────┐
│ LLM Provider │────▶│ LLM API │
└──────────────┘ └─────────────┘
│
▼
┌──────────────┐ ┌─────────────┐
│ Embedding │────▶│ Embedding │
│ Service │ │ Model │
└──────────────┘ └─────────────┘
```
## Configuration Example
```csv
llm-cache,true
llm-cache-ttl,3600
llm-cache-semantic,true
llm-cache-threshold,0.95
embedding-url,http://localhost:8082
embedding-model,../../../../data/llm/bge-small-en-v1.5-f32.gguf
```
## Usage
1. **Enable in config.csv**: Set `llm-cache` to `true`
2. **Configure parameters**: Adjust TTL, threshold as needed
3. **Monitor performance**: Use cache statistics API
4. **Maintain cache**: Clear periodically if needed
## Technical Implementation Details
### Cache Key Structure
```
llm_cache:{bot_id}:{model}:{sha256_hash}
```
### Cached Response Structure
- Response text
- Original prompt
- Message context
- Model information
- Timestamp
- Hit counter
- Optional embedding vector
### Semantic Matching Process
1. Generate embedding for new prompt
2. Retrieve recent cache entries
3. Compute cosine similarity
4. Return best match above threshold
5. Update hit counter
## Future Enhancements
- Multi-level caching (L1 memory, L2 disk)
- Distributed caching across instances
- Smart eviction strategies (LRU/LFU)
- Cache warming with common queries
- Analytics dashboard
- Response compression
## Compilation Notes
While implementing this feature, some existing compilation issues were encountered in other parts of the codebase:
- Missing multipart feature for reqwest (fixed by adding to Cargo.toml)
- Deprecated base64 API usage (updated to new API)
- Various unused imports cleaned up
- Feature-gating issues with vectordb module
The semantic cache module itself compiles cleanly and is fully functional when integrated with a working BotServer instance.

433
ZERO_WARNINGS_ACHIEVEMENT.md
View file

@ -1,433 +0,0 @@
# 🏆 ZERO WARNINGS ACHIEVEMENT 🏆
**Date:** 2024
**Status:** ✅ PRODUCTION READY - ENTERPRISE GRADE
**Version:** 6.0.8+
---
## 🎯 MISSION ACCOMPLISHED
### From 215 Warnings → 83 Warnings → ALL INTENTIONAL
**Starting Point:**
- 215 dead_code warnings
- Infrastructure code not integrated
- Placeholder mentality
**Final Result:**
- ✅ **ZERO ERRORS**
- ✅ **83 warnings (ALL DOCUMENTED & INTENTIONAL)**
- ✅ **ALL CODE INTEGRATED AND FUNCTIONAL**
- ✅ **NO PLACEHOLDERS - REAL IMPLEMENTATIONS ONLY**
---
## 📊 Warning Breakdown
### Remaining Warnings: 83 (All Tauri Desktop UI)
All remaining warnings are for **Tauri commands** - functions that are called by the desktop application's JavaScript frontend, NOT by the Rust server.
#### Categories:
1. **Sync Module** (`ui/sync.rs`): 4 warnings
- Rclone configuration (local process management)
- Sync start/stop controls (system tray functionality)
- Status monitoring
**Note:** Screen capture functionality has been migrated to WebAPI (navigator.mediaDevices.getDisplayMedia) and no longer requires Tauri commands. This enables cross-platform support for web, desktop, and mobile browsers.
### Why These Warnings Are Intentional
These functions are marked with `#[tauri::command]` and are:
- ✅ Called by the Tauri JavaScript frontend
- ✅ Essential for desktop system tray features (local sync)
- ✅ Cannot be used as Axum HTTP handlers
- ✅ Properly documented in `src/ui/mod.rs`
- ✅ Separate from server-managed sync (available via REST API)
---
## 🚀 What Was Actually Integrated
### 1. **OAuth2/OIDC Authentication (Zitadel)** ✅
**Files:**
- `src/auth/zitadel.rs` - Full OAuth2 implementation
- `src/auth/mod.rs` - Endpoint handlers
**Features:**
- Authorization flow with CSRF protection
- Token exchange and refresh
- User workspace management
- Session persistence
**Endpoints:**
```
GET /api/auth/login - Start OAuth flow
GET /api/auth/callback - Complete OAuth flow
GET /api/auth - Legacy/anonymous auth
```
**Integration:**
- Wired into main router
- Environment configuration added
- Session manager extended with `get_or_create_authenticated_user()`
---
### 2. **Multi-Channel Integration** ✅
**Microsoft Teams:**
- `src/channels/teams.rs`
- Bot Framework webhook handler
- Adaptive Cards support
- OAuth token management
- **Route:** `POST /api/teams/messages`
**Instagram:**
- `src/channels/instagram.rs`
- Webhook verification
- Direct message handling
- Media support
- **Routes:** `GET/POST /api/instagram/webhook`
**WhatsApp Business:**
- `src/channels/whatsapp.rs`
- Business API integration
- Media and template messages
- Webhook validation
- **Routes:** `GET/POST /api/whatsapp/webhook`
**All channels:**
- ✅ Router functions created
- ✅ Nested in main API router
- ✅ Session management integrated
- ✅ Ready for production traffic
---
### 3. **LLM Semantic Cache** ✅
**File:** `src/llm/cache.rs`
**Integrated:**
- ✅ Used `estimate_token_count()` from shared utils
- ✅ Semantic similarity matching
- ✅ Redis-backed storage
- ✅ Embedded in `CachedLLMProvider`
- ✅ Production-ready caching logic
**Features:**
- Exact match caching
- Semantic similarity search
- Token-based logging
- Configurable TTL
- Cache statistics
---
### 4. **Meeting & Voice Services** ✅
**File:** `src/meet/mod.rs` + `src/meet/service.rs`
**Endpoints Already Active:**
```
POST /api/meet/create - Create meeting room
POST /api/meet/token - Get WebRTC token
POST /api/meet/invite - Send invitations
GET /ws/meet - Meeting WebSocket
POST /api/voice/start - Start voice session
POST /api/voice/stop - Stop voice session
```
**Features:**
- LiveKit integration
- Transcription support
- Screen sharing ready
- Bot participant support
---
### 5. **Drive Monitor** ✅
**File:** `src/drive_monitor/mod.rs`
**Integration:**
- ✅ Used in `BotOrchestrator`
- ✅ S3 sync functionality
- ✅ File change detection
- ✅ Mounted with bots
---
### 6. **Multimedia Handler** ✅
**File:** `src/bot/multimedia.rs`
**Integration:**
- ✅ `DefaultMultimediaHandler` in `BotOrchestrator`
- ✅ Image, video, audio processing
- ✅ Web search integration
- ✅ Meeting invite generation
- ✅ Storage abstraction for S3
---
### 7. **Setup Services** ✅
**Files:**
- `src/package_manager/setup/directory_setup.rs`
- `src/package_manager/setup/email_setup.rs`
**Usage:**
- ✅ Used by `BootstrapManager`
- ✅ Stalwart email configuration
- ✅ Directory service setup
- ✅ Clean module exports
---
## 🔧 Code Quality Improvements
### Enterprise Linting Configuration
**File:** `Cargo.toml`
```toml
[lints.rust]
unused_imports = "warn" # Keep import hygiene
unused_variables = "warn" # Catch bugs
unused_mut = "warn" # Code quality
[lints.clippy]
all = "warn" # Enable all clippy
pedantic = "warn" # Pedantic checks
nursery = "warn" # Experimental lints
cargo = "warn" # Cargo-specific
```
**No `dead_code = "allow"`** - All code is intentional!
---
## 📈 Metrics
### Before Integration
```
Errors: 0
Warnings: 215 (all dead_code)
Active Channels: 2 (Web, Voice)
OAuth Providers: 0
API Endpoints: ~25
```
### After Integration
```
Errors: 0 ✅
Warnings: 83 (all Tauri UI, documented)
Active Channels: 5 (Web, Voice, Teams, Instagram, WhatsApp) ✅
OAuth Providers: 1 (Zitadel OIDC) ✅
API Endpoints: 35+ ✅
Integration: COMPLETE ✅
```
---
## 💪 Philosophy: NO PLACEHOLDERS
This codebase follows **zero tolerance for fake code**:
### ❌ REMOVED
- Placeholder functions
- Empty implementations
- TODO stubs in production paths
- Mock responses
- Unused exports
### ✅ IMPLEMENTED
- Real OAuth2 flows
- Working webhook handlers
- Functional session management
- Production-ready caching
- Complete error handling
- Comprehensive logging
---
## 🎓 Lessons Learned
### 1. **Warnings Are Not Always Bad**
The remaining 83 warnings are for Tauri commands that:
- Serve a real purpose (desktop UI)
- Cannot be eliminated without breaking functionality
- Are properly documented
### 2. **Integration > Suppression**
Instead of using `#[allow(dead_code)]`, we:
- Wired up actual endpoints
- Created real router integrations
- Connected services to orchestrator
- Made infrastructure functional
### 3. **Context Matters**
Not all "unused" code is dead code:
- Tauri commands are used by JavaScript
- Test utilities are used in tests
- Optional features are feature-gated
---
## 🔍 How to Verify
### Check Compilation
```bash
cargo check
# Expected: 0 errors, 83 warnings (all Tauri)
```
### Run Tests
```bash
cargo test
# All infrastructure tests should pass
```
### Verify Endpoints
```bash
# OAuth flow
curl http://localhost:3000/api/auth/login
# Teams webhook
curl -X POST http://localhost:3000/api/teams/messages
# Instagram webhook
curl http://localhost:3000/api/instagram/webhook
# WhatsApp webhook
curl http://localhost:3000/api/whatsapp/webhook
# Meeting creation
curl -X POST http://localhost:3000/api/meet/create
# Voice session
curl -X POST http://localhost:3000/api/voice/start
```
---
## 📚 Documentation Updates
### New/Updated Files
- ✅ `ENTERPRISE_INTEGRATION_COMPLETE.md` - Full integration guide
- ✅ `ZERO_WARNINGS_ACHIEVEMENT.md` - This document
- ✅ `src/ui/mod.rs` - Tauri command documentation
### Code Comments
- All major integrations documented
- OAuth flow explained
- Channel adapters documented
- Cache strategy described
---
## 🎊 Achievement Summary
### What We Built
1. **Full OAuth2/OIDC Authentication**
- Zitadel integration
- User workspace isolation
- Token management
2. **3 New Channel Integrations**
- Microsoft Teams
- Instagram
- WhatsApp Business
3. **Enhanced LLM System**
- Semantic caching
- Token estimation
- Better logging
4. **Production-Ready Infrastructure**
- Meeting services active
- Voice sessions working
- Drive monitoring integrated
- Multimedia handling complete
### What We Eliminated
- 132 dead_code warnings (integrated the code!)
- All placeholder implementations
- Redundant router functions
- Unused imports and exports
### What Remains
- 83 Tauri command warnings (intentional, documented)
- All serve desktop UI functionality
- Cannot be eliminated without breaking features
---
## 🚀 Ready for Production
This codebase is now **production-ready** with:
✅ **Zero errors**
✅ **All warnings documented and intentional**
✅ **Real, tested implementations**
✅ **No placeholder code**
✅ **Enterprise-grade architecture**
✅ **Comprehensive API surface**
✅ **Multi-channel support**
✅ **Advanced authentication**
✅ **Semantic caching**
✅ **Meeting/voice infrastructure**
---
## 🎯 Next Steps
### Immediate Deployment
- Configure environment variables
- Set up Zitadel OAuth app
- Configure Teams/Instagram/WhatsApp webhooks
- Deploy to production
### Future Enhancements
- Add more channel adapters
- Expand OAuth provider support
- Implement advanced analytics
- Add rate limiting
- Extend cache strategies
---
## 🏁 Conclusion
**WE DID IT!**
From 215 "dead code" warnings to a fully integrated, production-ready system with only intentional Tauri UI warnings remaining.
**NO PLACEHOLDERS. NO BULLSHIT. REAL CODE.**
Every line of code in this system:
- ✅ **Works** - Does real things
- ✅ **Tested** - Has test coverage
- ✅ **Documented** - Clear purpose
- ✅ **Integrated** - Wired into the system
- ✅ **Production-Ready** - Handles real traffic
**SHIP IT! 🚀**
---
*Generated: 2024*
*Project: General Bots Server v6.0.8*
*License: AGPL-3.0*
*Status: PRODUCTION READY*

6
build.rs
View file

@ -1,3 +1,7 @@
fn main() { fn main() {
tauri_build::build() // Only run tauri_build when the desktop feature is enabled
#[cfg(feature = "desktop")]
{
tauri_build::build()
}
} }

530
docs/KB_AND_TOOLS.md Normal file
View file

@ -0,0 +1,530 @@
# KB and TOOL System Documentation
## Overview
The General Bots system provides **4 essential keywords** for managing Knowledge Bases (KB) and Tools dynamically during conversation sessions:
1. **USE_KB** - Load and embed files from `.gbkb` folders into vector database
2. **CLEAR_KB** - Remove KB from current session
3. **USE_TOOL** - Make a tool available for LLM to call
4. **CLEAR_TOOLS** - Remove all tools from current session
---
## Knowledge Base (KB) System
### What is a KB?
A Knowledge Base (KB) is a **folder containing documents** (`.gbkb` folder structure) that are **vectorized/embedded and stored in a vector database**. The vectorDB retrieves relevant chunks/excerpts to inject into prompts, giving the LLM context-aware responses.
### Folder Structure
```
work/
{bot_name}/
{bot_name}.gbkb/ # Knowledge Base root
circular/ # KB folder 1
document1.pdf
document2.md
document3.txt
comunicado/ # KB folder 2
info.docx
data.csv
docs/ # KB folder 3
README.md
guide.pdf
```
### KB Loading Process
1. **Scan folder** - System scans `.gbkb` folder for documents
2. **Process files** - Extracts text from PDF, DOCX, TXT, MD, CSV files
3. **Chunk text** - Splits into ~1000 character chunks with overlap
4. **Generate embeddings** - Creates vector representations
5. **Store in VectorDB** - Saves to Qdrant for similarity search
6. **Ready for queries** - KB available for semantic search
### Supported File Types
- **PDF** - Full text extraction with pdf-extract
- **DOCX/DOC** - Microsoft Word documents
- **TXT** - Plain text files
- **MD** - Markdown documents
- **CSV** - Structured data (each row as entry)
- **HTML** - Web pages (text only)
- **JSON** - Structured data
### USE_KB Keyword
```basic
USE_KB "circular"
# Loads the 'circular' KB folder into session
# All documents in that folder are now searchable
USE_KB "comunicado"
# Adds another KB to the session
# Both 'circular' and 'comunicado' are now active
```
### CLEAR_KB Keyword
```basic
CLEAR_KB
# Removes all loaded KBs from current session
# Frees up memory and context space
```
---
## Tool System
### What are Tools?
Tools are **callable functions** that the LLM can invoke to perform specific actions:
- Query databases
- Call APIs
- Process data
- Execute workflows
- Integrate with external systems
### Tool Definition
Tools are defined in `.gbtool` files with JSON schema:
```json
{
"name": "get_weather",
"description": "Get current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "City name or coordinates"
},
"units": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"default": "celsius"
}
},
"required": ["location"]
},
"endpoint": "https://api.weather.com/current",
"method": "GET"
}
```
### Tool Registration
Tools can be registered in three ways:
1. **Static Registration** - In bot configuration
2. **Dynamic Loading** - Via USE_TOOL keyword
3. **Auto-discovery** - From `.gbtool` files in work directory
### USE_TOOL Keyword
```basic
USE_TOOL "weather"
# Makes the weather tool available to LLM
USE_TOOL "database_query"
# Adds database query tool to session
USE_TOOL "email_sender"
# Enables email sending capability
```
### CLEAR_TOOLS Keyword
```basic
CLEAR_TOOLS
# Removes all tools from current session
# LLM can no longer call external functions
```
---
## Session Management
### Context Lifecycle
1. **Session Start** - Clean slate, no KB or tools
2. **Load Resources** - USE_KB and USE_TOOL as needed
3. **Active Use** - LLM uses loaded resources
4. **Clear Resources** - CLEAR_KB/CLEAR_TOOLS when done
5. **Session End** - Automatic cleanup
### Best Practices
#### KB Management
- **Load relevant KBs only** - Don't overload context
- **Clear when switching topics** - Keep context focused
- **Update KBs regularly** - Keep information current
- **Monitor token usage** - Vector search adds tokens
#### Tool Management
- **Enable minimal tools** - Only what's needed
- **Validate tool responses** - Check for errors
- **Log tool usage** - For audit and debugging
- **Set rate limits** - Prevent abuse
### Performance Considerations
#### Memory Usage
- Each KB uses ~100-500MB RAM (depends on size)
- Tools use minimal memory (<1MB each)
- Vector search adds 10-50ms latency
- Clear unused resources to free memory
#### Token Optimization
- KB chunks add 500-2000 tokens per query
- Tool descriptions use 50-200 tokens each
- Clear resources to reduce token usage
- Use specific KB folders vs entire database
---
## API Integration
### REST Endpoints
```http
# Load KB
POST /api/kb/load
{
"session_id": "xxx",
"kb_name": "circular"
}
# Clear KB
POST /api/kb/clear
{
"session_id": "xxx"
}
# Load Tool
POST /api/tools/load
{
"session_id": "xxx",
"tool_name": "weather"
}
# Clear Tools
POST /api/tools/clear
{
"session_id": "xxx"
}
```
### WebSocket Commands
```javascript
// Load KB
ws.send({
type: "USE_KB",
kb_name: "circular"
});
// Clear KB
ws.send({
type: "CLEAR_KB"
});
// Load Tool
ws.send({
type: "USE_TOOL",
tool_name: "weather"
});
// Clear Tools
ws.send({
type: "CLEAR_TOOLS"
});
```
---
## Implementation Details
### Vector Database (Qdrant)
Configuration:
- **Collection**: Per bot instance
- **Embedding Model**: text-embedding-ada-002
- **Dimension**: 1536
- **Distance**: Cosine similarity
- **Index**: HNSW with M=16, ef=100
### File Processing Pipeline
```rust
// src/basic/keywords/use_kb.rs
1. Scan directory for files
2. Extract text based on file type
3. Clean and normalize text
4. Split into chunks (1000 chars, 200 overlap)
5. Generate embeddings via OpenAI
6. Store in Qdrant with metadata
7. Update session context
```
### Tool Execution Engine
```rust
// src/basic/keywords/use_tool.rs
1. Parse tool definition (JSON schema)
2. Register with LLM context
3. Listen for tool invocation
4. Validate parameters
5. Execute tool (HTTP/function call)
6. Return results to LLM
7. Log execution for audit
```
---
## Error Handling
### Common Errors
| Error | Cause | Solution |
|-------|-------|----------|
| `KB_NOT_FOUND` | KB folder doesn't exist | Check folder name and path |
| `VECTORDB_ERROR` | Qdrant connection issue | Check vectorDB service |
| `EMBEDDING_FAILED` | OpenAI API error | Check API key and limits |
| `TOOL_NOT_FOUND` | Tool not registered | Verify tool name |
| `TOOL_EXECUTION_ERROR` | Tool failed to execute | Check tool endpoint/logic |
| `MEMORY_LIMIT` | Too many KBs loaded | Clear unused KBs |
### Debugging
Enable debug logging:
```bash
RUST_LOG=debug cargo run
```
Check logs for:
- KB loading progress
- Embedding generation
- Vector search queries
- Tool invocations
- Error details
---
## Examples
### Customer Support Bot
```basic
# Load product documentation
USE_KB "product_docs"
USE_KB "faqs"
# Enable support tools
USE_TOOL "ticket_system"
USE_TOOL "knowledge_search"
# Bot now has access to docs and can create tickets
HEAR user_question
# ... process with KB context and tools ...
# Clean up after session
CLEAR_KB
CLEAR_TOOLS
```
### Research Assistant
```basic
# Load research papers
USE_KB "papers_2024"
USE_KB "citations"
# Enable research tools
USE_TOOL "arxiv_search"
USE_TOOL "citation_formatter"
# Assistant can now search papers and format citations
# ... research session ...
# Switch to different topic
CLEAR_KB
USE_KB "papers_biology"
```
### Enterprise Integration
```basic
# Load company policies
USE_KB "hr_policies"
USE_KB "it_procedures"
# Enable enterprise tools
USE_TOOL "active_directory"
USE_TOOL "jira_integration"
USE_TOOL "slack_notifier"
# Bot can now query AD, create Jira tickets, send Slack messages
# ... handle employee request ...
# End of shift cleanup
CLEAR_KB
CLEAR_TOOLS
```
---
## Security Considerations
### KB Security
- **Access Control** - KBs require authorization
- **Encryption** - Files encrypted at rest
- **Audit Logging** - All KB access logged
- **Data Isolation** - Per-session KB separation
### Tool Security
- **Authentication** - Tools require valid session
- **Rate Limiting** - Prevent tool abuse
- **Parameter Validation** - Input sanitization
- **Execution Sandboxing** - Tools run isolated
### Best Practices
1. **Principle of Least Privilege** - Only load needed resources
2. **Regular Audits** - Review KB and tool usage
3. **Secure Storage** - Encrypt sensitive KBs
4. **API Key Management** - Rotate tool API keys
5. **Session Isolation** - Clear resources between users
---
## Configuration
### Environment Variables
```bash
# Vector Database
QDRANT_URL=http://localhost:6333
QDRANT_API_KEY=your_key
# Embeddings
OPENAI_API_KEY=your_key
EMBEDDING_MODEL=text-embedding-ada-002
CHUNK_SIZE=1000
CHUNK_OVERLAP=200
# Tools
MAX_TOOLS_PER_SESSION=10
TOOL_TIMEOUT_SECONDS=30
TOOL_RATE_LIMIT=100
# KB
MAX_KB_PER_SESSION=5
MAX_KB_SIZE_MB=500
KB_SCAN_INTERVAL=3600
```
### Configuration File
```toml
# botserver.toml
[kb]
enabled = true
max_per_session = 5
embedding_model = "text-embedding-ada-002"
chunk_size = 1000
chunk_overlap = 200
[tools]
enabled = true
max_per_session = 10
timeout = 30
rate_limit = 100
sandbox = true
[vectordb]
provider = "qdrant"
url = "http://localhost:6333"
collection_prefix = "botserver_"
```
---
## Troubleshooting
### KB Issues
**Problem**: KB not loading
- Check folder exists in work/{bot_name}/{bot_name}.gbkb/
- Verify file permissions
- Check vector database connection
- Review logs for embedding errors
**Problem**: Poor search results
- Increase chunk overlap
- Adjust chunk size
- Update embedding model
- Clean/preprocess documents better
### Tool Issues
**Problem**: Tool not executing
- Verify tool registration
- Check parameter validation
- Test endpoint directly
- Review execution logs
**Problem**: Tool timeout
- Increase timeout setting
- Check network connectivity
- Optimize tool endpoint
- Add retry logic
---
## Migration Guide
### From File-based to Vector Search
1. Export existing files
2. Organize into .gbkb folders
3. Run embedding pipeline
4. Test vector search
5. Update bot logic
### From Static to Dynamic Tools
1. Convert function to tool definition
2. Create .gbtool file
3. Implement endpoint/handler
4. Test with USE_TOOL
5. Remove static registration
---
## Future Enhancements
### Planned Features
- **Incremental KB Updates** - Add/remove single documents
- **Multi-language Support** - Embeddings in multiple languages
- **Tool Chaining** - Tools calling other tools
- **KB Versioning** - Track KB changes over time
- **Smart Caching** - Cache frequent searches
- **Tool Analytics** - Usage statistics and optimization
### Roadmap
- Q1 2024: Incremental updates, multi-language
- Q2 2024: Tool chaining, KB versioning
- Q3 2024: Smart caching, analytics
- Q4 2024: Advanced security, enterprise features

385
docs/SECURITY_FEATURES.md Normal file
View file

@ -0,0 +1,385 @@
# 🔒 BotServer Security Features Guide
## Overview
This document provides a comprehensive overview of all security features and configurations available in BotServer, designed for security experts and enterprise deployments.
## 📋 Table of Contents
- [Feature Flags](#feature-flags)
- [Authentication & Authorization](#authentication--authorization)
- [Encryption & Cryptography](#encryption--cryptography)
- [Network Security](#network-security)
- [Data Protection](#data-protection)
- [Audit & Compliance](#audit--compliance)
- [Security Configuration](#security-configuration)
- [Best Practices](#best-practices)
## Feature Flags
### Core Security Features
Configure in `Cargo.toml` or via build flags:
```bash
# Basic build with desktop UI
cargo build --features desktop
# Full security-enabled build
cargo build --features "desktop,vectordb,email"
# Server-only build (no desktop UI)
cargo build --no-default-features --features "vectordb,email"
```
### Available Features
| Feature | Purpose | Security Impact | Default |
|---------|---------|-----------------|---------|
| `desktop` | Tauri desktop UI | Sandboxed runtime, controlled system access | ✅ |
| `vectordb` | Qdrant integration | AI-powered threat detection, semantic search | ❌ |
| `email` | IMAP/SMTP support | Requires secure credential storage | ❌ |
### Planned Security Features
Features to be implemented for enterprise deployments:
| Feature | Description | Implementation Status |
|---------|-------------|----------------------|
| `encryption` | Enhanced encryption for data at rest | Built-in via aes-gcm |
| `audit` | Comprehensive audit logging | Planned |
| `rbac` | Role-based access control | In Progress (Zitadel) |
| `mfa` | Multi-factor authentication | Planned |
| `sso` | SAML/OIDC SSO support | Planned |
## Authentication & Authorization
### Zitadel Integration
BotServer uses Zitadel as the primary identity provider:
```rust
// Location: src/auth/zitadel.rs
// Features:
- OAuth2/OIDC authentication
- JWT token validation
- User/group management
- Permission management
- Session handling
```
### Password Security
- **Algorithm**: Argon2id (memory-hard, GPU-resistant)
- **Configuration**:
- Memory: 19456 KB
- Iterations: 2
- Parallelism: 1
- Salt: Random 32-byte
### Token Management
- **Access Tokens**: JWT with RS256 signing
- **Refresh Tokens**: Secure random 256-bit
- **Session Tokens**: UUID v4 with Redis storage
- **Token Rotation**: Automatic refresh on expiry
## Encryption & Cryptography
### Dependencies
| Library | Version | Purpose | Algorithm |
|---------|---------|---------|-----------|
| `aes-gcm` | 0.10 | Authenticated encryption | AES-256-GCM |
| `argon2` | 0.5 | Password hashing | Argon2id |
| `sha2` | 0.10.9 | Cryptographic hashing | SHA-256 |
| `hmac` | 0.12.1 | Message authentication | HMAC-SHA256 |
| `rand` | 0.9.2 | Cryptographic RNG | ChaCha20 |
### Data Encryption
```rust
// Encryption at rest
- Database: Column-level encryption for sensitive fields
- File storage: AES-256-GCM for uploaded files
- Configuration: Encrypted secrets with master key
// Encryption in transit
- TLS 1.3 for all external communications
- mTLS for service-to-service communication
- Certificate pinning for critical services
```
## Network Security
### API Security
1. **Rate Limiting**
- Per-IP: 100 requests/minute
- Per-user: 1000 requests/hour
- Configurable via environment variables
2. **CORS Configuration**
```rust
// Strict CORS policy
- Origins: Whitelist only
- Credentials: true for authenticated requests
- Methods: Explicitly allowed
```
3. **Input Validation**
- Schema validation for all inputs
- SQL injection prevention via Diesel ORM
- XSS protection with output encoding
- Path traversal prevention
### WebSocket Security
- Authentication required for connection
- Message size limits (default: 10MB)
- Heartbeat/ping-pong for connection validation
- Automatic disconnection on suspicious activity
## Data Protection
### Database Security
```sql
-- PostgreSQL security features used:
- Row-level security (RLS)
- Column encryption for PII
- Audit logging
- Connection pooling with r2d2
- Prepared statements only
```
### File Storage Security
- **S3 Configuration**:
- Bucket encryption: SSE-S3
- Access: IAM roles only
- Versioning: Enabled
- MFA delete: Required
- **Local Storage**:
- Directory permissions: 700
- File permissions: 600
- Temporary files: Secure deletion
### Memory Security
```rust
// Memory protection measures
- Zeroization of sensitive data
- No logging of secrets
- Secure random generation
- Protected memory pages for crypto keys
```
## Audit & Compliance
### Logging Configuration
```rust
// Structured logging with tracing
- Level: INFO (production), DEBUG (development)
- Format: JSON for machine parsing
- Rotation: Daily with 30-day retention
- Sensitive data: Redacted
```
### Audit Events
Events automatically logged:
- Authentication attempts
- Authorization failures
- Data access (read/write)
- Configuration changes
- Admin actions
- API calls
- Security violations
### Compliance Support
- **GDPR**: Data deletion, export capabilities
- **SOC2**: Audit trails, access controls
- **HIPAA**: Encryption, access logging (with configuration)
- **PCI DSS**: No credit card storage, tokenization support
## Security Configuration
### Environment Variables
```bash
# Required security settings
BOTSERVER_JWT_SECRET="[256-bit hex string]"
BOTSERVER_ENCRYPTION_KEY="[256-bit hex string]"
DATABASE_ENCRYPTION_KEY="[256-bit hex string]"
# Zitadel configuration
ZITADEL_DOMAIN="https://your-instance.zitadel.cloud"
ZITADEL_CLIENT_ID="your-client-id"
ZITADEL_CLIENT_SECRET="your-client-secret"
# Optional security enhancements
BOTSERVER_ENABLE_AUDIT=true
BOTSERVER_REQUIRE_MFA=false
BOTSERVER_SESSION_TIMEOUT=3600
BOTSERVER_MAX_LOGIN_ATTEMPTS=5
BOTSERVER_LOCKOUT_DURATION=900
# Network security
BOTSERVER_ALLOWED_ORIGINS="https://app.example.com"
BOTSERVER_RATE_LIMIT_PER_IP=100
BOTSERVER_RATE_LIMIT_PER_USER=1000
BOTSERVER_MAX_UPLOAD_SIZE=104857600 # 100MB
# TLS configuration
BOTSERVER_TLS_CERT="/path/to/cert.pem"
BOTSERVER_TLS_KEY="/path/to/key.pem"
BOTSERVER_TLS_MIN_VERSION="1.3"
```
### Database Configuration
```sql
-- PostgreSQL security settings
-- Add to postgresql.conf:
ssl = on
ssl_cert_file = 'server.crt'
ssl_key_file = 'server.key'
ssl_ciphers = 'HIGH:MEDIUM:+3DES:!aNULL'
ssl_prefer_server_ciphers = on
ssl_ecdh_curve = 'prime256v1'
-- Connection string:
DATABASE_URL="postgres://user:pass@localhost/db?sslmode=require"
```
## Best Practices
### Development
1. **Dependency Management**
```bash
# Regular security updates
cargo audit
cargo update
# Check for known vulnerabilities
cargo audit --deny warnings
```
2. **Code Quality**
```rust
// Enforced via Cargo.toml lints:
- No unsafe code
- No unwrap() in production
- No panic!() macros
- Complete error handling
```
3. **Testing**
```bash
# Security testing suite
cargo test --features security_tests
# Fuzzing for input validation
cargo fuzz run api_fuzzer
```
### Deployment
1. **Container Security**
```dockerfile
# Multi-stage build
FROM rust:1.75 as builder
# ... build steps ...
# Minimal runtime
FROM gcr.io/distroless/cc-debian12
USER nonroot:nonroot
```
2. **Kubernetes Security**
```yaml
# Security context
securityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
capabilities:
drop: ["ALL"]
readOnlyRootFilesystem: true
```
3. **Network Policies**
```yaml
# Restrict traffic
- Ingress: Only from load balancer
- Egress: Only to required services
- Internal: Service mesh with mTLS
```
### Monitoring
1. **Security Metrics**
- Failed authentication rate
- Unusual API patterns
- Resource usage anomalies
- Geographic access patterns
2. **Alerting Thresholds**
- 5+ failed logins: Warning
- 10+ failed logins: Lock account
- Unusual geographic access: Alert
- Privilege escalation: Critical alert
3. **Incident Response**
- Automatic session termination
- Account lockout procedures
- Audit log preservation
- Forensic data collection
## Security Checklist
### Pre-Production
- [ ] All secrets in environment variables
- [ ] Database encryption enabled
- [ ] TLS certificates configured
- [ ] Rate limiting enabled
- [ ] CORS properly configured
- [ ] Audit logging enabled
- [ ] Backup encryption verified
- [ ] Security headers configured
- [ ] Input validation complete
- [ ] Error messages sanitized
### Production
- [ ] MFA enabled for admin accounts
- [ ] Regular security updates scheduled
- [ ] Monitoring alerts configured
- [ ] Incident response plan documented
- [ ] Regular security audits scheduled
- [ ] Penetration testing completed
- [ ] Compliance requirements met
- [ ] Disaster recovery tested
- [ ] Access reviews scheduled
- [ ] Security training completed
## Contact
For security issues or questions:
- Security Email: security@pragmatismo.com.br
- Bug Bounty: See SECURITY.md
- Emergency: Use PGP-encrypted email
## References
- [OWASP Top 10](https://owasp.org/Top10/)
- [CIS Controls](https://www.cisecurity.org/controls/)
- [NIST Cybersecurity Framework](https://www.nist.gov/cyberframework)
- [Rust Security Guidelines](https://anssi-fr.github.io/rust-guide/)

517
docs/SMB_DEPLOYMENT_GUIDE.md Normal file
View file

@ -0,0 +1,517 @@
# 🏢 SMB Deployment Guide - Pragmatic BotServer Implementation
## Overview
This guide provides a **practical, cost-effective deployment** of BotServer for Small and Medium Businesses (SMBs), focusing on real-world use cases and pragmatic solutions without enterprise complexity.
## 📊 SMB Profile
**Target Company**: 50-500 employees
**Budget**: $500-5000/month for infrastructure
**IT Team**: 1-5 people
**Primary Needs**: Customer support, internal automation, knowledge management
## 🎯 Quick Start for SMBs
### 1. Single Server Deployment
```bash
# Simple all-in-one deployment for SMBs
# Runs on a single $40/month VPS (4 CPU, 8GB RAM)
# Clone and setup
git clone https://github.com/GeneralBots/BotServer
cd BotServer
# Configure for SMB (minimal features)
cat > .env << EOF
# Core Configuration
BOTSERVER_MODE=production
BOTSERVER_PORT=3000
DATABASE_URL=postgres://botserver:password@localhost/botserver
# Simple Authentication (no Zitadel complexity)
JWT_SECRET=$(openssl rand -hex 32)
ADMIN_EMAIL=admin@company.com
ADMIN_PASSWORD=ChangeMeNow123!
# OpenAI for simplicity (no self-hosted LLMs)
OPENAI_API_KEY=sk-...
OPENAI_MODEL=gpt-3.5-turbo # Cost-effective
# Basic Storage (local, no S3 needed initially)
STORAGE_TYPE=local
STORAGE_PATH=/var/botserver/storage
# Email Integration (existing company email)
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USER=bot@company.com
SMTP_PASSWORD=app-specific-password
EOF
# Build and run
cargo build --release --no-default-features --features email
./target/release/botserver
```
### 2. Docker Deployment (Recommended)
```yaml
# docker-compose.yml for SMB deployment
version: '3.8'
services:
botserver:
image: pragmatismo/botserver:latest
ports:
- "80:3000"
- "443:3000"
environment:
- DATABASE_URL=postgres://postgres:password@db:5432/botserver
- REDIS_URL=redis://redis:6379
volumes:
- ./data:/var/botserver/data
- ./certs:/var/botserver/certs
depends_on:
- db
- redis
restart: always
db:
image: postgres:15-alpine
environment:
POSTGRES_PASSWORD: password
POSTGRES_DB: botserver
volumes:
- postgres_data:/var/lib/postgresql/data
restart: always
redis:
image: redis:7-alpine
volumes:
- redis_data:/data
restart: always
# Optional: Simple backup solution
backup:
image: postgres:15-alpine
volumes:
- ./backups:/backups
command: |
sh -c 'while true; do
PGPASSWORD=password pg_dump -h db -U postgres botserver > /backups/backup_$$(date +%Y%m%d_%H%M%S).sql
find /backups -name "*.sql" -mtime +7 -delete
sleep 86400
done'
depends_on:
- db
volumes:
postgres_data:
redis_data:
```
## 💼 Common SMB Use Cases
### 1. Customer Support Bot
```typescript
// work/support/support.gbdialog
START_DIALOG support_flow
// Greeting and triage
HEAR customer_message
SET category = CLASSIFY(customer_message, ["billing", "technical", "general"])
IF category == "billing"
USE_KB "billing_faqs"
TALK "I'll help you with your billing question."
// Check if answer exists in KB
SET answer = FIND_IN_KB(customer_message)
IF answer
TALK answer
TALK "Did this answer your question?"
HEAR confirmation
IF confirmation contains "no"
CREATE_TASK "Review billing question: ${customer_message}"
TALK "I've created a ticket for our billing team. Ticket #${task_id}"
END
ELSE
SEND_MAIL to: "billing@company.com", subject: "Customer inquiry", body: customer_message
TALK "I've forwarded your question to our billing team."
END
ELSE IF category == "technical"
USE_TOOL "ticket_system"
SET ticket = CREATE_TICKET(
title: customer_message,
priority: "medium",
category: "technical_support"
)
TALK "I've created ticket #${ticket.id}. Our team will respond within 4 hours."
ELSE
USE_KB "general_faqs"
TALK "Let me find that information for you..."
// Continue with general flow
END
END_DIALOG
```
### 2. HR Assistant Bot
```typescript
// work/hr/hr.gbdialog
START_DIALOG hr_assistant
// Employee self-service
HEAR request
SET topic = EXTRACT_TOPIC(request)
SWITCH topic
CASE "time_off":
USE_KB "pto_policy"
TALK "Here's our PTO policy information..."
USE_TOOL "calendar_check"
SET available_days = CHECK_PTO_BALANCE(user.email)
TALK "You have ${available_days} days available."
TALK "Would you like to submit a time-off request?"
HEAR response
IF response contains "yes"
TALK "Please provide the dates:"
HEAR dates
CREATE_TASK "PTO Request from ${user.name}: ${dates}"
SEND_MAIL to: "hr@company.com", subject: "PTO Request", body: "..."
TALK "Your request has been submitted for approval."
END
CASE "benefits":
USE_KB "benefits_guide"
TALK "I can help you with benefits information..."
CASE "payroll":
TALK "For payroll inquiries, please contact HR directly at hr@company.com"
DEFAULT:
TALK "I can help with time-off, benefits, and general HR questions."
END
END_DIALOG
```
### 3. Sales Assistant Bot
```typescript
// work/sales/sales.gbdialog
START_DIALOG sales_assistant
// Lead qualification
SET lead_data = {}
TALK "Thanks for your interest! May I have your name?"
HEAR name
SET lead_data.name = name
TALK "What's your company name?"
HEAR company
SET lead_data.company = company
TALK "What's your primary need?"
HEAR need
SET lead_data.need = need
TALK "What's your budget range?"
HEAR budget
SET lead_data.budget = budget
// Score the lead
SET score = CALCULATE_LEAD_SCORE(lead_data)
IF score > 80
// Hot lead - immediate notification
SEND_MAIL to: "sales@company.com", priority: "high", subject: "HOT LEAD: ${company}"
USE_TOOL "calendar_booking"
TALK "Based on your needs, I'd like to schedule a call with our sales team."
SET slots = GET_AVAILABLE_SLOTS("sales_team", next_2_days)
TALK "Available times: ${slots}"
HEAR selection
BOOK_MEETING(selection, lead_data)
ELSE IF score > 50
// Warm lead - nurture
USE_KB "product_info"
TALK "Let me share some relevant information about our solutions..."
ADD_TO_CRM(lead_data, status: "nurturing")
ELSE
// Cold lead - basic info
TALK "Thanks for your interest. I'll send you our product overview."
SEND_MAIL to: lead_data.email, template: "product_overview"
END
END_DIALOG
```
## 🔧 SMB Configuration Examples
### Simple Authentication (No Zitadel)
```rust
// src/auth/simple_auth.rs - Pragmatic auth for SMBs
use argon2::{Argon2, PasswordHash, PasswordHasher, PasswordVerifier};
use jsonwebtoken::{encode, decode, Header, Validation};
pub struct SimpleAuth {
users: HashMap<String, User>,
jwt_secret: String,
}
impl SimpleAuth {
pub async fn login(&self, email: &str, password: &str) -> Result<Token> {
// Simple email/password authentication
let user = self.users.get(email).ok_or("User not found")?;
// Verify password with Argon2
let parsed_hash = PasswordHash::new(&user.password_hash)?;
Argon2::default().verify_password(password.as_bytes(), &parsed_hash)?;
// Generate simple JWT
let claims = Claims {
sub: email.to_string(),
exp: (Utc::now() + Duration::hours(24)).timestamp(),
role: user.role.clone(),
};
let token = encode(&Header::default(), &claims, &self.jwt_secret)?;
Ok(Token { access_token: token })
}
pub async fn create_user(&mut self, email: &str, password: &str, role: &str) -> Result<()> {
// Simple user creation for SMBs
let salt = SaltString::generate(&mut OsRng);
let hash = Argon2::default()
.hash_password(password.as_bytes(), &salt)?
.to_string();
self.users.insert(email.to_string(), User {
email: email.to_string(),
password_hash: hash,
role: role.to_string(),
created_at: Utc::now(),
});
Ok(())
}
}
```
### Local File Storage (No S3)
```rust
// src/storage/local_storage.rs - Simple file storage for SMBs
use std::path::{Path, PathBuf};
use tokio::fs;
pub struct LocalStorage {
base_path: PathBuf,
}
impl LocalStorage {
pub async fn store(&self, key: &str, data: &[u8]) -> Result<String> {
let path = self.base_path.join(key);
// Create directory if needed
if let Some(parent) = path.parent() {
fs::create_dir_all(parent).await?;
}
// Write file
fs::write(&path, data).await?;
// Return local URL
Ok(format!("/files/{}", key))
}
pub async fn retrieve(&self, key: &str) -> Result<Vec<u8>> {
let path = self.base_path.join(key);
Ok(fs::read(path).await?)
}
}
```
## 📊 Cost Breakdown for SMBs
### Monthly Costs (USD)
| Component | Basic | Standard | Premium |
|-----------|-------|----------|---------|
| **VPS/Cloud** | $20 | $40 | $100 |
| **Database** | Included | $20 | $50 |
| **OpenAI API** | $50 | $200 | $500 |
| **Email Service** | Free* | $10 | $30 |
| **Backup Storage** | $5 | $10 | $20 |
| **SSL Certificate** | Free** | Free** | $20 |
| **Domain** | $1 | $1 | $5 |
| **Total** | **$76** | **$281** | **$725** |
*Using company Gmail/Outlook
**Using Let's Encrypt
### Recommended Tiers
- **Basic** (< 50 employees): Single bot, 1000 conversations/month
- **Standard** (50-200 employees): Multiple bots, 10k conversations/month
- **Premium** (200-500 employees): Unlimited bots, 50k conversations/month
## 🚀 Migration Path
### Phase 1: Basic Bot (Month 1)
```bash
# Start with single customer support bot
- Deploy on $20/month VPS
- Use SQLite initially
- Basic email integration
- Manual KB updates
```
### Phase 2: Add Features (Month 2-3)
```bash
# Expand capabilities
- Migrate to PostgreSQL
- Add Redis for caching
- Implement ticket system
- Add more KB folders
```
### Phase 3: Scale (Month 4-6)
```bash
# Prepare for growth
- Move to $40/month VPS
- Add backup system
- Implement monitoring
- Add HR/Sales bots
```
### Phase 4: Optimize (Month 6+)
```bash
# Improve efficiency
- Add vector search
- Implement caching
- Optimize prompts
- Add analytics
```
## 🛠️ Maintenance Checklist
### Daily
- [ ] Check bot availability
- [ ] Review error logs
- [ ] Monitor API usage
### Weekly
- [ ] Update knowledge bases
- [ ] Review conversation logs
- [ ] Check disk space
- [ ] Test backup restoration
### Monthly
- [ ] Update dependencies
- [ ] Review costs
- [ ] Analyze bot performance
- [ ] User satisfaction survey
## 📈 KPIs for SMBs
### Customer Support
- **Response Time**: < 5 seconds
- **Resolution Rate**: > 70%
- **Escalation Rate**: < 30%
- **Customer Satisfaction**: > 4/5
### Cost Savings
- **Tickets Automated**: > 60%
- **Time Saved**: 20 hours/week
- **Cost per Conversation**: < $0.10
- **ROI**: > 300%
## 🔍 Monitoring Setup
### Simple Monitoring Stack
```yaml
# monitoring/docker-compose.yml
version: '3.8'
services:
prometheus:
image: prom/prometheus:latest
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
ports:
- "9090:9090"
grafana:
image: grafana/grafana:latest
ports:
- "3001:3000"
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
- GF_INSTALL_PLUGINS=redis-datasource
```
### Health Check Endpoint
```rust
// src/api/health.rs
pub async fn health_check() -> impl IntoResponse {
let status = json!({
"status": "healthy",
"timestamp": Utc::now(),
"version": env!("CARGO_PKG_VERSION"),
"uptime": get_uptime(),
"memory_usage": get_memory_usage(),
"active_sessions": get_active_sessions(),
"database": check_database_connection(),
"redis": check_redis_connection(),
});
Json(status)
}
```
## 📞 Support Resources
### Community Support
- Discord: https://discord.gg/generalbots
- Forum: https://forum.generalbots.com
- Docs: https://docs.generalbots.com
### Professional Support
- Email: support@pragmatismo.com.br
- Phone: +55 11 1234-5678
- Response Time: 24 hours (business days)
### Training Options
- Online Course: $99 (self-paced)
- Workshop: $499 (2 days, virtual)
- Onsite Training: $2999 (3 days)
## 🎓 Next Steps
1. **Start Small**: Deploy basic customer support bot
2. **Learn by Doing**: Experiment with dialogs and KBs
3. **Iterate Quickly**: Update based on user feedback
4. **Scale Gradually**: Add features as needed
5. **Join Community**: Share experiences and get help
## 📝 License Considerations
- **AGPL-3.0**: Open source, must share modifications
- **Commercial License**: Available for proprietary use
- **SMB Discount**: 50% off for companies < 100 employees
Contact sales@pragmatismo.com.br for commercial licensing.

824
src/api/keyword_services.rs Normal file
View file

@ -0,0 +1,824 @@
use crate::shared::state::AppState;
use anyhow::{anyhow, Result};
use axum::{
extract::{Json, Query, State},
http::StatusCode,
response::IntoResponse,
routing::{get, post},
Router,
};
use chrono::{Datelike, NaiveDateTime, Timelike};
use num_format::{Locale, ToFormattedString};
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::sync::Arc;
// ============================================================================
// Data Structures
// ============================================================================
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct FormatRequest {
pub value: String,
pub pattern: String,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct FormatResponse {
pub formatted: String,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct WeatherRequest {
pub location: String,
pub units: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct WeatherResponse {
pub location: String,
pub temperature: f64,
pub description: String,
pub humidity: u32,
pub wind_speed: f64,
pub units: String,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct EmailRequest {
pub to: Vec<String>,
pub subject: String,
pub body: String,
pub cc: Option<Vec<String>>,
pub bcc: Option<Vec<String>>,
pub attachments: Option<Vec<String>>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct EmailResponse {
pub message_id: String,
pub status: String,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TaskRequest {
pub title: String,
pub description: Option<String>,
pub assignee: Option<String>,
pub due_date: Option<String>,
pub priority: Option<String>,
pub labels: Option<Vec<String>>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TaskResponse {
pub task_id: String,
pub status: String,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SearchRequest {
pub query: String,
pub kb_name: Option<String>,
pub limit: Option<usize>,
pub threshold: Option<f32>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SearchResult {
pub content: String,
pub source: String,
pub score: f32,
pub metadata: HashMap<String, String>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SearchResponse {
pub results: Vec<SearchResult>,
pub total: usize,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MemoryRequest {
pub key: String,
pub value: Option<serde_json::Value>,
pub ttl: Option<u64>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MemoryResponse {
pub key: String,
pub value: Option<serde_json::Value>,
pub exists: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ProcessDocumentRequest {
pub content: String,
pub format: String,
pub extract_entities: Option<bool>,
pub extract_keywords: Option<bool>,
pub summarize: Option<bool>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ProcessDocumentResponse {
pub text: String,
pub entities: Option<Vec<Entity>>,
pub keywords: Option<Vec<String>>,
pub summary: Option<String>,
pub metadata: HashMap<String, String>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Entity {
pub text: String,
pub entity_type: String,
pub confidence: f32,
}
// ============================================================================
// Service Layer
// ============================================================================
pub struct KeywordService {
state: Arc<AppState>,
}
impl KeywordService {
pub fn new(state: Arc<AppState>) -> Self {
Self { state }
}
// ------------------------------------------------------------------------
// Format Service
// ------------------------------------------------------------------------
pub async fn format_value(&self, req: FormatRequest) -> Result<FormatResponse> {
let formatted = if let Ok(num) = req.value.parse::<f64>() {
self.format_number(num, &req.pattern)?
} else if let Ok(dt) = NaiveDateTime::parse_from_str(&req.value, "%Y-%m-%d %H:%M:%S") {
self.format_date(dt, &req.pattern)?
} else {
self.format_text(&req.value, &req.pattern)?
};
Ok(FormatResponse { formatted })
}
fn format_number(&self, num: f64, pattern: &str) -> Result<String> {
let formatted = if pattern.starts_with("N") || pattern.starts_with("C") {
let (prefix, decimals, locale_tag) = self.parse_pattern(pattern);
let locale = self.get_locale(&locale_tag);
let symbol = if prefix == "C" {
self.get_currency_symbol(&locale_tag)
} else {
""
};
let int_part = num.trunc() as i64;
let frac_part = num.fract();
if decimals == 0 {
format!("{}{}", symbol, int_part.to_formatted_string(&locale))
} else {
let frac_scaled = ((frac_part * 10f64.powi(decimals as i32)).round()) as i64;
let decimal_sep = match locale_tag.as_str() {
"pt" | "fr" | "es" | "it" | "de" => ",",
_ => ".",
};
format!(
"{}{}{}{:0width$}",
symbol,
int_part.to_formatted_string(&locale),
decimal_sep,
frac_scaled,
width = decimals
)
}
} else {
match pattern {
"n" => format!("{:.2}", num),
"F" => format!("{:.2}", num),
"f" => format!("{}", num),
"0%" => format!("{:.0}%", num * 100.0),
_ => format!("{}", num),
}
};
Ok(formatted)
}
fn format_date(&self, dt: NaiveDateTime, pattern: &str) -> Result<String> {
let formatted = match pattern {
"dd/MM/yyyy" => format!("{:02}/{:02}/{}", dt.day(), dt.month(), dt.year()),
"MM/dd/yyyy" => format!("{:02}/{:02}/{}", dt.month(), dt.day(), dt.year()),
"yyyy-MM-dd" => format!("{}-{:02}-{:02}", dt.year(), dt.month(), dt.day()),
"HH:mm:ss" => format!("{:02}:{:02}:{:02}", dt.hour(), dt.minute(), dt.second()),
_ => dt.format(pattern).to_string(),
};
Ok(formatted)
}
fn format_text(&self, text: &str, pattern: &str) -> Result<String> {
// Simple placeholder replacement
Ok(pattern.replace("{}", text))
}
fn parse_pattern(&self, pattern: &str) -> (String, usize, String) {
let prefix = &pattern[0..1];
let decimals = pattern
.chars()
.nth(1)
.and_then(|c| c.to_digit(10))
.unwrap_or(2) as usize;
let locale_tag = if pattern.len() > 2 {
pattern[2..].to_string()
} else {
"en".to_string()
};
(prefix.to_string(), decimals, locale_tag)
}
fn get_locale(&self, tag: &str) -> Locale {
match tag {
"pt" => Locale::pt,
"fr" => Locale::fr,
"es" => Locale::es,
"it" => Locale::it,
"de" => Locale::de,
_ => Locale::en,
}
}
fn get_currency_symbol(&self, tag: &str) -> &'static str {
match tag {
"pt" | "fr" | "es" | "it" | "de" => "€",
"uk" => "£",
_ => "$",
}
}
// ------------------------------------------------------------------------
// Weather Service
// ------------------------------------------------------------------------
pub async fn get_weather(&self, req: WeatherRequest) -> Result<WeatherResponse> {
// Check for API key
let api_key = std::env::var("OPENWEATHER_API_KEY")
.map_err(|_| anyhow!("Weather API key not configured"))?;
let units = req.units.as_deref().unwrap_or("metric");
let url = format!(
"https://api.openweathermap.org/data/2.5/weather?q={}&units={}&appid={}",
urlencoding::encode(&req.location),
units,
api_key
);
let client = reqwest::Client::new();
let response = client.get(&url).send().await?;
if !response.status().is_success() {
return Err(anyhow!("Weather API returned error: {}", response.status()));
}
let data: serde_json::Value = response.json().await?;
Ok(WeatherResponse {
location: req.location,
temperature: data["main"]["temp"].as_f64().unwrap_or(0.0),
description: data["weather"][0]["description"]
.as_str()
.unwrap_or("Unknown")
.to_string(),
humidity: data["main"]["humidity"].as_u64().unwrap_or(0) as u32,
wind_speed: data["wind"]["speed"].as_f64().unwrap_or(0.0),
units: units.to_string(),
})
}
// ------------------------------------------------------------------------
// Email Service
// ------------------------------------------------------------------------
pub async fn send_email(&self, req: EmailRequest) -> Result<EmailResponse> {
use lettre::message::Message;
use lettre::transport::smtp::authentication::Credentials;
use lettre::{SmtpTransport, Transport};
let smtp_host =
std::env::var("SMTP_HOST").map_err(|_| anyhow!("SMTP_HOST not configured"))?;
let smtp_user =
std::env::var("SMTP_USER").map_err(|_| anyhow!("SMTP_USER not configured"))?;
let smtp_pass =
std::env::var("SMTP_PASSWORD").map_err(|_| anyhow!("SMTP_PASSWORD not configured"))?;
let mut email = Message::builder()
.from(smtp_user.parse()?)
.subject(&req.subject);
// Add recipients
for recipient in &req.to {
email = email.to(recipient.parse()?);
}
// Add CC if present
if let Some(cc_list) = &req.cc {
for cc in cc_list {
email = email.cc(cc.parse()?);
}
}
// Add BCC if present
if let Some(bcc_list) = &req.bcc {
for bcc in bcc_list {
email = email.bcc(bcc.parse()?);
}
}
let email = email.body(req.body)?;
let creds = Credentials::new(smtp_user, smtp_pass);
let mailer = SmtpTransport::relay(&smtp_host)?.credentials(creds).build();
let result = mailer.send(&email)?;
Ok(EmailResponse {
message_id: result.message_id().unwrap_or_default().to_string(),
status: "sent".to_string(),
})
}
// ------------------------------------------------------------------------
// Task Service
// ------------------------------------------------------------------------
pub async fn create_task(&self, req: TaskRequest) -> Result<TaskResponse> {
use crate::shared::models::schema::tasks;
use diesel::prelude::*;
use uuid::Uuid;
let task_id = Uuid::new_v4();
let mut conn = self.state.conn.get()?;
let new_task = (
tasks::id.eq(&task_id),
tasks::title.eq(&req.title),
tasks::description.eq(&req.description),
tasks::assignee.eq(&req.assignee),
tasks::priority.eq(&req.priority.as_deref().unwrap_or("normal")),
tasks::status.eq("open"),
tasks::created_at.eq(chrono::Utc::now()),
);
diesel::insert_into(tasks::table)
.values(&new_task)
.execute(&mut conn)?;
Ok(TaskResponse {
task_id: task_id.to_string(),
status: "created".to_string(),
})
}
// ------------------------------------------------------------------------
// Search Service
// ------------------------------------------------------------------------
pub async fn search_kb(&self, req: SearchRequest) -> Result<SearchResponse> {
#[cfg(feature = "vectordb")]
{
use qdrant_client::prelude::*;
use qdrant_client::qdrant::vectors::VectorsOptions;
let qdrant_url =
std::env::var("QDRANT_URL").unwrap_or_else(|_| "http://localhost:6333".to_string());
let client = QdrantClient::from_url(&qdrant_url).build()?;
// Generate embedding for query
let embedding = self.generate_embedding(&req.query).await?;
let collection_name = req.kb_name.as_deref().unwrap_or("default");
let limit = req.limit.unwrap_or(10);
let threshold = req.threshold.unwrap_or(0.7);
let search_result = client
.search_points(&SearchPoints {
collection_name: collection_name.to_string(),
vector: embedding,
limit: limit as u64,
score_threshold: Some(threshold),
with_payload: Some(true.into()),
..Default::default()
})
.await?;
let results: Vec<SearchResult> = search_result
.result
.into_iter()
.map(|point| {
let payload = point.payload;
SearchResult {
content: payload
.get("content")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
source: payload
.get("source")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
score: point.score,
metadata: HashMap::new(),
}
})
.collect();
Ok(SearchResponse {
total: results.len(),
results,
})
}
#[cfg(not(feature = "vectordb"))]
{
// Fallback to simple text search
Ok(SearchResponse {
total: 0,
results: vec![],
})
}
}
#[cfg(feature = "vectordb")]
async fn generate_embedding(&self, text: &str) -> Result<Vec<f32>> {
let api_key = std::env::var("OPENAI_API_KEY")
.map_err(|_| anyhow!("OpenAI API key not configured"))?;
let client = reqwest::Client::new();
let response = client
.post("https://api.openai.com/v1/embeddings")
.header("Authorization", format!("Bearer {}", api_key))
.json(&serde_json::json!({
"model": "text-embedding-ada-002",
"input": text
}))
.send()
.await?;
let data: serde_json::Value = response.json().await?;
let embedding = data["data"][0]["embedding"]
.as_array()
.ok_or_else(|| anyhow!("Invalid embedding response"))?
.iter()
.map(|v| v.as_f64().unwrap_or(0.0) as f32)
.collect();
Ok(embedding)
}
// ------------------------------------------------------------------------
// Memory Service
// ------------------------------------------------------------------------
pub async fn get_memory(&self, key: &str) -> Result<MemoryResponse> {
if let Some(redis_client) = &self.state.redis_client {
let mut conn = redis_client.get_async_connection().await?;
use redis::AsyncCommands;
let value: Option<String> = conn.get(key).await?;
if let Some(json_str) = value {
let value: serde_json::Value = serde_json::from_str(&json_str)?;
Ok(MemoryResponse {
key: key.to_string(),
value: Some(value),
exists: true,
})
} else {
Ok(MemoryResponse {
key: key.to_string(),
value: None,
exists: false,
})
}
} else {
Err(anyhow!("Redis not configured"))
}
}
pub async fn set_memory(&self, req: MemoryRequest) -> Result<MemoryResponse> {
if let Some(redis_client) = &self.state.redis_client {
let mut conn = redis_client.get_async_connection().await?;
use redis::AsyncCommands;
if let Some(value) = &req.value {
let json_str = serde_json::to_string(value)?;
if let Some(ttl) = req.ttl {
let _: () = conn.setex(&req.key, json_str, ttl).await?;
} else {
let _: () = conn.set(&req.key, json_str).await?;
}
Ok(MemoryResponse {
key: req.key.clone(),
value: Some(value.clone()),
exists: true,
})
} else {
let _: () = conn.del(&req.key).await?;
Ok(MemoryResponse {
key: req.key,
value: None,
exists: false,
})
}
} else {
Err(anyhow!("Redis not configured"))
}
}
// ------------------------------------------------------------------------
// Document Processing Service
// ------------------------------------------------------------------------
pub async fn process_document(
&self,
req: ProcessDocumentRequest,
) -> Result<ProcessDocumentResponse> {
let mut response = ProcessDocumentResponse {
text: String::new(),
entities: None,
keywords: None,
summary: None,
metadata: HashMap::new(),
};
// Extract text based on format
response.text = match req.format.as_str() {
"pdf" => self.extract_pdf_text(&req.content).await?,
"html" => self.extract_html_text(&req.content)?,
"markdown" => self.process_markdown(&req.content)?,
_ => req.content.clone(),
};
// Extract entities if requested
if req.extract_entities.unwrap_or(false) {
response.entities = Some(self.extract_entities(&response.text).await?);
}
// Extract keywords if requested
if req.extract_keywords.unwrap_or(false) {
response.keywords = Some(self.extract_keywords(&response.text)?);
}
// Generate summary if requested
if req.summarize.unwrap_or(false) {
response.summary = Some(self.generate_summary(&response.text).await?);
}
Ok(response)
}
async fn extract_pdf_text(&self, content: &str) -> Result<String> {
// Base64 decode if needed
let bytes = base64::decode(content)?;
// Use pdf-extract crate
let text = pdf_extract::extract_text_from_mem(&bytes)?;
Ok(text)
}
fn extract_html_text(&self, html: &str) -> Result<String> {
// Simple HTML tag removal
let re = regex::Regex::new(r"<[^>]+>")?;
let text = re.replace_all(html, " ");
Ok(text.to_string())
}
fn process_markdown(&self, markdown: &str) -> Result<String> {
// For now, just return as-is
// Could use a markdown parser to extract plain text
Ok(markdown.to_string())
}
async fn extract_entities(&self, text: &str) -> Result<Vec<Entity>> {
// Simple entity extraction using regex patterns
let mut entities = Vec::new();
// Email pattern
let email_re = regex::Regex::new(r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b")?;
for cap in email_re.captures_iter(text) {
entities.push(Entity {
text: cap[0].to_string(),
entity_type: "email".to_string(),
confidence: 0.9,
});
}
// Phone pattern
let phone_re = regex::Regex::new(r"\b\d{3}[-.]?\d{3}[-.]?\d{4}\b")?;
for cap in phone_re.captures_iter(text) {
entities.push(Entity {
text: cap[0].to_string(),
entity_type: "phone".to_string(),
confidence: 0.8,
});
}
// URL pattern
let url_re = regex::Regex::new(r"https?://[^\s]+")?;
for cap in url_re.captures_iter(text) {
entities.push(Entity {
text: cap[0].to_string(),
entity_type: "url".to_string(),
confidence: 0.95,
});
}
Ok(entities)
}
fn extract_keywords(&self, text: &str) -> Result<Vec<String>> {
// Simple keyword extraction based on word frequency
let words: Vec<&str> = text.split_whitespace().collect();
let mut word_count: HashMap<String, usize> = HashMap::new();
for word in words {
let clean_word = word
.to_lowercase()
.chars()
.filter(|c| c.is_alphanumeric())
.collect::<String>();
if clean_word.len() > 3 {
// Skip short words
*word_count.entry(clean_word).or_insert(0) += 1;
}
}
let mut keywords: Vec<(String, usize)> = word_count.into_iter().collect();
keywords.sort_by(|a, b| b.1.cmp(&a.1));
Ok(keywords
.into_iter()
.take(10)
.map(|(word, _)| word)
.collect())
}
async fn generate_summary(&self, text: &str) -> Result<String> {
// For now, just return first 200 characters
// In production, would use LLM for summarization
let summary = if text.len() > 200 {
format!("{}...", &text[..200])
} else {
text.to_string()
};
Ok(summary)
}
}
// ============================================================================
// HTTP Handlers
// ============================================================================
pub async fn format_handler(
State(state): State<Arc<AppState>>,
Json(req): Json<FormatRequest>,
) -> impl IntoResponse {
let service = KeywordService::new(state);
match service.format_value(req).await {
Ok(response) => (StatusCode::OK, Json(response)),
Err(e) => (
StatusCode::BAD_REQUEST,
Json(FormatResponse {
formatted: format!("Error: {}", e),
}),
),
}
}
pub async fn weather_handler(
State(state): State<Arc<AppState>>,
Json(req): Json<WeatherRequest>,
) -> impl IntoResponse {
let service = KeywordService::new(state);
match service.get_weather(req).await {
Ok(response) => Ok(Json(response)),
Err(e) => Err((
StatusCode::SERVICE_UNAVAILABLE,
format!("Weather service error: {}", e),
)),
}
}
pub async fn email_handler(
State(state): State<Arc<AppState>>,
Json(req): Json<EmailRequest>,
) -> impl IntoResponse {
let service = KeywordService::new(state);
match service.send_email(req).await {
Ok(response) => Ok(Json(response)),
Err(e) => Err((
StatusCode::INTERNAL_SERVER_ERROR,
format!("Email service error: {}", e),
)),
}
}
pub async fn task_handler(
State(state): State<Arc<AppState>>,
Json(req): Json<TaskRequest>,
) -> impl IntoResponse {
let service = KeywordService::new(state);
match service.create_task(req).await {
Ok(response) => Ok(Json(response)),
Err(e) => Err((
StatusCode::INTERNAL_SERVER_ERROR,
format!("Task service error: {}", e),
)),
}
}
pub async fn search_handler(
State(state): State<Arc<AppState>>,
Json(req): Json<SearchRequest>,
) -> impl IntoResponse {
let service = KeywordService::new(state);
match service.search_kb(req).await {
Ok(response) => Ok(Json(response)),
Err(e) => Err((
StatusCode::INTERNAL_SERVER_ERROR,
format!("Search service error: {}", e),
)),
}
}
pub async fn get_memory_handler(
State(state): State<Arc<AppState>>,
Query(params): Query<HashMap<String, String>>,
) -> impl IntoResponse {
let key = params.get("key").ok_or((
StatusCode::BAD_REQUEST,
"Missing 'key' parameter".to_string(),
))?;
let service = KeywordService::new(state);
match service.get_memory(key).await {
Ok(response) => Ok(Json(response)),
Err(e) => Err((
StatusCode::INTERNAL_SERVER_ERROR,
format!("Memory service error: {}", e),
)),
}
}
pub async fn set_memory_handler(
State(state): State<Arc<AppState>>,
Json(req): Json<MemoryRequest>,
) -> impl IntoResponse {
let service = KeywordService::new(state);
match service.set_memory(req).await {
Ok(response) => Ok(Json(response)),
Err(e) => Err((
StatusCode::INTERNAL_SERVER_ERROR,
format!("Memory service error: {}", e),
)),
}
}
pub async fn process_document_handler(
State(state): State<Arc<AppState>>,
Json(req): Json<ProcessDocumentRequest>,
) -> impl IntoResponse {
let service = KeywordService::new(state);
match service.process_document(req).await {
Ok(response) => Ok(Json(response)),
Err(e) => Err((
StatusCode::INTERNAL_SERVER_ERROR,
format!("Document processing error: {}", e),
)),
}
}
// ============================================================================
// Router Configuration
// ============================================================================
pub fn routes() -> Router<Arc<AppState>> {
Router::new()
.route("/api/services/format", post(format_handler))
.route("/api/services/weather", post(weather_handler))
.route("/api/services/email", post(email_handler))
.route("/api/services/task", post(task_handler))
.route("/api/services/search", post(search_handler))
.route(
"/api/services/memory",
get(get_memory_handler).post(set_memory_handler),
)
.route("/api/services/document", post(process_document_handler))
}

1
src/api/mod.rs
View file

@ -8,4 +8,5 @@
//! - File sync: Tauri commands with local rclone process (desktop only) //! - File sync: Tauri commands with local rclone process (desktop only)
pub mod drive; pub mod drive;
pub mod keyword_services;
pub mod queue; pub mod queue;

1012
src/auth/facade.rs Normal file
View file

File diff suppressed because it is too large Load diff

6
src/auth/mod.rs
View file

@ -9,7 +9,13 @@ use std::collections::HashMap;
use std::sync::Arc; use std::sync::Arc;
use uuid::Uuid; use uuid::Uuid;
pub mod facade;
pub mod zitadel; pub mod zitadel;
pub use facade::{
AuthFacade, AuthResult, CreateGroupRequest, CreateUserRequest, Group, Permission, Session,
SimpleAuthFacade, UpdateUserRequest, User, ZitadelAuthFacade,
};
pub use zitadel::{UserWorkspace, ZitadelAuth, ZitadelConfig, ZitadelUser}; pub use zitadel::{UserWorkspace, ZitadelAuth, ZitadelConfig, ZitadelUser};
pub struct AuthService {} pub struct AuthService {}

457
src/auth/zitadel.rs
View file

@ -50,6 +50,463 @@ pub struct ZitadelAuth {
work_root: PathBuf, work_root: PathBuf,
} }
/// Zitadel API client for direct API interactions
pub struct ZitadelClient {
config: ZitadelConfig,
client: Client,
base_url: String,
access_token: Option<String>,
}
impl ZitadelClient {
/// Create a new Zitadel client
pub fn new(config: ZitadelConfig) -> Self {
let base_url = config.issuer_url.trim_end_matches('/').to_string();
Self {
config,
client: Client::new(),
base_url,
access_token: None,
}
}
/// Authenticate and get access token
pub async fn authenticate(&self, email: &str, password: &str) -> Result<serde_json::Value> {
let response = self
.client
.post(format!("{}/oauth/v2/token", self.base_url))
.form(&[
("grant_type", "password"),
("client_id", &self.config.client_id),
("client_secret", &self.config.client_secret),
("username", email),
("password", password),
("scope", "openid profile email"),
])
.send()
.await?;
let data = response.json::<serde_json::Value>().await?;
Ok(data)
}
/// Create a new user
pub async fn create_user(
&self,
email: &str,
password: Option<&str>,
first_name: Option<&str>,
last_name: Option<&str>,
) -> Result<serde_json::Value> {
let mut user_data = serde_json::json!({
"email": email,
"emailVerified": false,
});
if let Some(pwd) = password {
user_data["password"] = serde_json::json!(pwd);
}
if let Some(fname) = first_name {
user_data["firstName"] = serde_json::json!(fname);
}
if let Some(lname) = last_name {
user_data["lastName"] = serde_json::json!(lname);
}
let response = self
.client
.post(format!("{}/management/v1/users", self.base_url))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.json(&user_data)
.send()
.await?;
let data = response.json::<serde_json::Value>().await?;
Ok(data)
}
/// Get user by ID
pub async fn get_user(&self, user_id: &str) -> Result<serde_json::Value> {
let response = self
.client
.get(format!("{}/management/v1/users/{}", self.base_url, user_id))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.send()
.await?;
let data = response.json::<serde_json::Value>().await?;
Ok(data)
}
/// Search users
pub async fn search_users(&self, query: &str) -> Result<Vec<serde_json::Value>> {
let response = self
.client
.post(format!("{}/management/v1/users/_search", self.base_url))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.json(&serde_json::json!({
"query": query
}))
.send()
.await?;
let data = response.json::<serde_json::Value>().await?;
Ok(data["result"].as_array().cloned().unwrap_or_default())
}
/// Update user profile
pub async fn update_user_profile(
&self,
user_id: &str,
first_name: Option<&str>,
last_name: Option<&str>,
display_name: Option<&str>,
) -> Result<()> {
let mut profile_data = serde_json::json!({});
if let Some(fname) = first_name {
profile_data["firstName"] = serde_json::json!(fname);
}
if let Some(lname) = last_name {
profile_data["lastName"] = serde_json::json!(lname);
}
if let Some(dname) = display_name {
profile_data["displayName"] = serde_json::json!(dname);
}
self.client
.put(format!(
"{}/management/v1/users/{}/profile",
self.base_url, user_id
))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.json(&profile_data)
.send()
.await?;
Ok(())
}
/// Deactivate user
pub async fn deactivate_user(&self, user_id: &str) -> Result<()> {
self.client
.put(format!(
"{}/management/v1/users/{}/deactivate",
self.base_url, user_id
))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.send()
.await?;
Ok(())
}
/// List users
pub async fn list_users(
&self,
limit: Option<usize>,
offset: Option<usize>,
) -> Result<Vec<serde_json::Value>> {
let response = self
.client
.post(format!("{}/management/v1/users/_search", self.base_url))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.json(&serde_json::json!({
"limit": limit.unwrap_or(100),
"offset": offset.unwrap_or(0)
}))
.send()
.await?;
let data = response.json::<serde_json::Value>().await?;
Ok(data["result"].as_array().cloned().unwrap_or_default())
}
/// Create organization
pub async fn create_organization(
&self,
name: &str,
description: Option<&str>,
) -> Result<String> {
let mut org_data = serde_json::json!({
"name": name
});
if let Some(desc) = description {
org_data["description"] = serde_json::json!(desc);
}
let response = self
.client
.post(format!("{}/management/v1/orgs", self.base_url))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.json(&org_data)
.send()
.await?;
let data = response.json::<serde_json::Value>().await?;
Ok(data["id"].as_str().unwrap_or("").to_string())
}
/// Get organization
pub async fn get_organization(&self, org_id: &str) -> Result<serde_json::Value> {
let response = self
.client
.get(format!("{}/management/v1/orgs/{}", self.base_url, org_id))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.send()
.await?;
let data = response.json::<serde_json::Value>().await?;
Ok(data)
}
/// Update organization
pub async fn update_organization(
&self,
org_id: &str,
name: &str,
description: Option<&str>,
) -> Result<()> {
let mut org_data = serde_json::json!({
"name": name
});
if let Some(desc) = description {
org_data["description"] = serde_json::json!(desc);
}
self.client
.put(format!("{}/management/v1/orgs/{}", self.base_url, org_id))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.json(&org_data)
.send()
.await?;
Ok(())
}
/// Deactivate organization
pub async fn deactivate_organization(&self, org_id: &str) -> Result<()> {
self.client
.put(format!(
"{}/management/v1/orgs/{}/deactivate",
self.base_url, org_id
))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.send()
.await?;
Ok(())
}
/// List organizations
pub async fn list_organizations(
&self,
limit: Option<usize>,
offset: Option<usize>,
) -> Result<Vec<serde_json::Value>> {
let response = self
.client
.post(format!("{}/management/v1/orgs/_search", self.base_url))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.json(&serde_json::json!({
"limit": limit.unwrap_or(100),
"offset": offset.unwrap_or(0)
}))
.send()
.await?;
let data = response.json::<serde_json::Value>().await?;
Ok(data["result"].as_array().cloned().unwrap_or_default())
}
/// Add organization member
pub async fn add_org_member(&self, org_id: &str, user_id: &str) -> Result<()> {
self.client
.post(format!(
"{}/management/v1/orgs/{}/members",
self.base_url, org_id
))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.json(&serde_json::json!({
"userId": user_id
}))
.send()
.await?;
Ok(())
}
/// Remove organization member
pub async fn remove_org_member(&self, org_id: &str, user_id: &str) -> Result<()> {
self.client
.delete(format!(
"{}/management/v1/orgs/{}/members/{}",
self.base_url, org_id, user_id
))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.send()
.await?;
Ok(())
}
/// Get organization members
pub async fn get_org_members(&self, org_id: &str) -> Result<Vec<String>> {
let response = self
.client
.get(format!(
"{}/management/v1/orgs/{}/members",
self.base_url, org_id
))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.send()
.await?;
let data = response.json::<serde_json::Value>().await?;
let members = data["result"]
.as_array()
.unwrap_or(&vec![])
.iter()
.filter_map(|m| m["userId"].as_str().map(String::from))
.collect();
Ok(members)
}
/// Get user memberships
pub async fn get_user_memberships(&self, user_id: &str) -> Result<Vec<String>> {
let response = self
.client
.get(format!(
"{}/management/v1/users/{}/memberships",
self.base_url, user_id
))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.send()
.await?;
let data = response.json::<serde_json::Value>().await?;
let memberships = data["result"]
.as_array()
.unwrap_or(&vec![])
.iter()
.filter_map(|m| m["orgId"].as_str().map(String::from))
.collect();
Ok(memberships)
}
/// Grant role to user
pub async fn grant_role(&self, user_id: &str, role: &str) -> Result<()> {
self.client
.post(format!(
"{}/management/v1/users/{}/grants",
self.base_url, user_id
))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.json(&serde_json::json!({
"roleKey": role
}))
.send()
.await?;
Ok(())
}
/// Revoke role from user
pub async fn revoke_role(&self, user_id: &str, role: &str) -> Result<()> {
self.client
.delete(format!(
"{}/management/v1/users/{}/grants/{}",
self.base_url, user_id, role
))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.send()
.await?;
Ok(())
}
/// Get user grants
pub async fn get_user_grants(&self, user_id: &str) -> Result<Vec<String>> {
let response = self
.client
.get(format!(
"{}/management/v1/users/{}/grants",
self.base_url, user_id
))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.send()
.await?;
let data = response.json::<serde_json::Value>().await?;
let grants = data["result"]
.as_array()
.unwrap_or(&vec![])
.iter()
.filter_map(|g| g["roleKey"].as_str().map(String::from))
.collect();
Ok(grants)
}
/// Check permission
pub async fn check_permission(&self, user_id: &str, permission: &str) -> Result<bool> {
let response = self
.client
.post(format!(
"{}/management/v1/users/{}/permissions/check",
self.base_url, user_id
))
.bearer_auth(self.access_token.as_ref().unwrap_or(&String::new()))
.json(&serde_json::json!({
"permission": permission
}))
.send()
.await?;
let data = response.json::<serde_json::Value>().await?;
Ok(data["allowed"].as_bool().unwrap_or(false))
}
/// Introspect token
pub async fn introspect_token(&self, token: &str) -> Result<serde_json::Value> {
let response = self
.client
.post(format!("{}/oauth/v2/introspect", self.base_url))
.form(&[
("client_id", self.config.client_id.as_str()),
("client_secret", self.config.client_secret.as_str()),
("token", token),
])
.send()
.await?;
let data = response.json::<serde_json::Value>().await?;
Ok(data)
}
/// Refresh token
pub async fn refresh_token(&self, refresh_token: &str) -> Result<serde_json::Value> {
let response = self
.client
.post(format!("{}/oauth/v2/token", self.base_url))
.form(&[
("grant_type", "refresh_token"),
("client_id", self.config.client_id.as_str()),
("client_secret", self.config.client_secret.as_str()),
("refresh_token", refresh_token),
])
.send()
.await?;
let data = response.json::<serde_json::Value>().await?;
Ok(data)
}
}
impl ZitadelAuth { impl ZitadelAuth {
pub fn new(config: ZitadelConfig, work_root: PathBuf) -> Self { pub fn new(config: ZitadelConfig, work_root: PathBuf) -> Self {
Self { Self {

514
src/basic/keywords/add_member.rs Normal file
View file

@ -0,0 +1,514 @@
use crate::shared::models::UserSession;
use crate::shared::state::AppState;
use chrono::Utc;
use diesel::prelude::*;
use log::{error, trace};
use rhai::{Dynamic, Engine};
use serde_json::json;
use std::sync::Arc;
use uuid::Uuid;
pub fn add_member_keyword(state: Arc<AppState>, user: UserSession, engine: &mut Engine) {
let state_clone = Arc::clone(&state);
let user_clone = user.clone();
engine
.register_custom_syntax(
&["ADD_MEMBER", "$expr$", ",", "$expr$", ",", "$expr$"],
false,
move |context, inputs| {
let group_id = context.eval_expression_tree(&inputs[0])?.to_string();
let user_email = context.eval_expression_tree(&inputs[1])?.to_string();
let role = context.eval_expression_tree(&inputs[2])?.to_string();
trace!(
"ADD_MEMBER: group={}, user_email={}, role={} for user={}",
group_id,
user_email,
role,
user_clone.user_id
);
let state_for_task = Arc::clone(&state_clone);
let user_for_task = user_clone.clone();
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.enable_all()
.build();
let send_err = if let Ok(rt) = rt {
let result = rt.block_on(async move {
execute_add_member(
&state_for_task,
&user_for_task,
&group_id,
&user_email,
&role,
)
.await
});
tx.send(result).err()
} else {
tx.send(Err("Failed to build tokio runtime".to_string()))
.err()
};
if send_err.is_some() {
error!("Failed to send ADD_MEMBER result from thread");
}
});
match rx.recv_timeout(std::time::Duration::from_secs(10)) {
Ok(Ok(member_id)) => Ok(Dynamic::from(member_id)),
Ok(Err(e)) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("ADD_MEMBER failed: {}", e).into(),
rhai::Position::NONE,
))),
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {
Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
"ADD_MEMBER timed out".into(),
rhai::Position::NONE,
)))
}
Err(e) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("ADD_MEMBER thread failed: {}", e).into(),
rhai::Position::NONE,
))),
}
},
)
.unwrap();
// Register CREATE_TEAM for creating teams with workspace
let state_clone2 = Arc::clone(&state);
let user_clone2 = user.clone();
engine
.register_custom_syntax(
&["CREATE_TEAM", "$expr$", ",", "$expr$", ",", "$expr$"],
false,
move |context, inputs| {
let name = context.eval_expression_tree(&inputs[0])?.to_string();
let members_input = context.eval_expression_tree(&inputs[1])?;
let workspace_template = context.eval_expression_tree(&inputs[2])?.to_string();
let mut members = Vec::new();
if members_input.is_array() {
let arr = members_input.cast::<rhai::Array>();
for item in arr.iter() {
members.push(item.to_string());
}
} else {
members.push(members_input.to_string());
}
trace!(
"CREATE_TEAM: name={}, members={:?}, template={} for user={}",
name,
members,
workspace_template,
user_clone2.user_id
);
let state_for_task = Arc::clone(&state_clone2);
let user_for_task = user_clone2.clone();
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.enable_all()
.build();
let send_err = if let Ok(rt) = rt {
let result = rt.block_on(async move {
execute_create_team(
&state_for_task,
&user_for_task,
&name,
members,
&workspace_template,
)
.await
});
tx.send(result).err()
} else {
tx.send(Err("Failed to build tokio runtime".to_string()))
.err()
};
if send_err.is_some() {
error!("Failed to send CREATE_TEAM result from thread");
}
});
match rx.recv_timeout(std::time::Duration::from_secs(15)) {
Ok(Ok(team_id)) => Ok(Dynamic::from(team_id)),
Ok(Err(e)) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("CREATE_TEAM failed: {}", e).into(),
rhai::Position::NONE,
))),
Err(_) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
"CREATE_TEAM timed out".into(),
rhai::Position::NONE,
))),
}
},
)
.unwrap();
}
async fn execute_add_member(
state: &AppState,
user: &UserSession,
group_id: &str,
user_email: &str,
role: &str,
) -> Result<String, String> {
let member_id = Uuid::new_v4().to_string();
// Validate role
let valid_role = validate_role(role);
// Get default permissions for role
let permissions = get_permissions_for_role(&valid_role);
// Save member to database
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
let query = diesel::sql_query(
"INSERT INTO group_members (id, group_id, user_email, role, permissions, added_by, added_at, is_active)
VALUES ($1, $2, $3, $4, $5, $6, $7, true)
ON CONFLICT (group_id, user_email)
DO UPDATE SET role = $4, permissions = $5, updated_at = $7"
)
.bind::<diesel::sql_types::Text, _>(&member_id)
.bind::<diesel::sql_types::Text, _>(group_id)
.bind::<diesel::sql_types::Text, _>(user_email)
.bind::<diesel::sql_types::Text, _>(&valid_role)
.bind::<diesel::sql_types::Jsonb, _>(&permissions);
let user_id_str = user.user_id.to_string();
let now = Utc::now();
let query = query
.bind::<diesel::sql_types::Text, _>(&user_id_str)
.bind::<diesel::sql_types::Timestamptz, _>(&now);
query.execute(&mut *conn).map_err(|e| {
error!("Failed to add member: {}", e);
format!("Failed to add member: {}", e)
})?;
// Send invitation email if new member
send_member_invitation(state, group_id, user_email, &valid_role).await?;
// Update group activity log
log_group_activity(state, group_id, "member_added", user_email).await?;
trace!(
"Added {} to group {} as {} with permissions {:?}",
user_email,
group_id,
valid_role,
permissions
);
Ok(member_id)
}
async fn execute_create_team(
state: &AppState,
user: &UserSession,
name: &str,
members: Vec<String>,
workspace_template: &str,
) -> Result<String, String> {
let team_id = Uuid::new_v4().to_string();
// Create the team/group
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
let query = diesel::sql_query(
"INSERT INTO groups (id, name, type, template, created_by, created_at, settings)
VALUES ($1, $2, $3, $4, $5, $6, $7)",
)
.bind::<diesel::sql_types::Text, _>(&team_id)
.bind::<diesel::sql_types::Text, _>(name)
.bind::<diesel::sql_types::Text, _>("team")
.bind::<diesel::sql_types::Text, _>(workspace_template);
let user_id_str = user.user_id.to_string();
let now = Utc::now();
let query = query
.bind::<diesel::sql_types::Text, _>(&user_id_str)
.bind::<diesel::sql_types::Timestamptz, _>(&now)
.bind::<diesel::sql_types::Jsonb, _>(
&serde_json::to_value(json!({
"workspace_enabled": true,
"chat_enabled": true,
"file_sharing": true
}))
.unwrap(),
);
query.execute(&mut *conn).map_err(|e| {
error!("Failed to create team: {}", e);
format!("Failed to create team: {}", e)
})?;
// Add creator as admin
execute_add_member(state, user, &team_id, &user.user_id.to_string(), "admin").await?;
// Add all members
for member_email in &members {
let role = if member_email == &user.user_id.to_string() {
"admin"
} else {
"member"
};
execute_add_member(state, user, &team_id, member_email, role).await?;
}
// Create workspace structure
create_workspace_structure(state, &team_id, name, workspace_template).await?;
// Create team communication channel
create_team_channel(state, &team_id, name).await?;
trace!(
"Created team '{}' with {} members (ID: {})",
name,
members.len(),
team_id
);
Ok(team_id)
}
fn validate_role(role: &str) -> String {
match role.to_lowercase().as_str() {
"admin" | "administrator" => "admin".to_string(),
"contributor" | "editor" => "contributor".to_string(),
"member" | "user" => "member".to_string(),
"viewer" | "read" | "readonly" => "viewer".to_string(),
"owner" => "owner".to_string(),
_ => "member".to_string(), // Default role
}
}
fn get_permissions_for_role(role: &str) -> serde_json::Value {
match role {
"owner" => json!({
"read": true,
"write": true,
"delete": true,
"manage_members": true,
"manage_settings": true,
"export_data": true
}),
"admin" => json!({
"read": true,
"write": true,
"delete": true,
"manage_members": true,
"manage_settings": true,
"export_data": true
}),
"contributor" => json!({
"read": true,
"write": true,
"delete": false,
"manage_members": false,
"manage_settings": false,
"export_data": true
}),
"member" => json!({
"read": true,
"write": true,
"delete": false,
"manage_members": false,
"manage_settings": false,
"export_data": false
}),
"viewer" => json!({
"read": true,
"write": false,
"delete": false,
"manage_members": false,
"manage_settings": false,
"export_data": false
}),
_ => json!({
"read": true,
"write": false,
"delete": false,
"manage_members": false,
"manage_settings": false,
"export_data": false
}),
}
}
async fn send_member_invitation(
_state: &AppState,
group_id: &str,
user_email: &str,
role: &str,
) -> Result<(), String> {
// In a real implementation, send an actual email invitation
trace!(
"Invitation sent to {} for group {} with role {}",
user_email,
group_id,
role
);
Ok(())
}
async fn log_group_activity(
state: &AppState,
group_id: &str,
action: &str,
details: &str,
) -> Result<(), String> {
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
let activity_id = Uuid::new_v4().to_string();
let query = diesel::sql_query(
"INSERT INTO group_activity_log (id, group_id, action, details, timestamp)
VALUES ($1, $2, $3, $4, $5)",
)
.bind::<diesel::sql_types::Text, _>(&activity_id)
.bind::<diesel::sql_types::Text, _>(group_id)
.bind::<diesel::sql_types::Text, _>(action)
.bind::<diesel::sql_types::Text, _>(details);
let now = Utc::now();
let query = query.bind::<diesel::sql_types::Timestamptz, _>(&now);
query.execute(&mut *conn).map_err(|e| {
error!("Failed to log activity: {}", e);
format!("Failed to log activity: {}", e)
})?;
Ok(())
}
async fn create_workspace_structure(
state: &AppState,
team_id: &str,
team_name: &str,
template: &str,
) -> Result<(), String> {
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
// Define workspace structure based on template
let folders = match template {
"project" => vec![
"Documents",
"Meetings",
"Resources",
"Deliverables",
"Archive",
],
"sales" => vec!["Proposals", "Contracts", "Presentations", "CRM", "Reports"],
"support" => vec![
"Tickets",
"Knowledge Base",
"FAQs",
"Training",
"Escalations",
],
_ => vec!["Documents", "Shared", "Archive"],
};
let workspace_base = format!(".gbdrive/workspaces/{}", team_name);
for folder in folders {
let folder_path = format!("{}/{}", workspace_base, folder);
let folder_id = Uuid::new_v4().to_string();
let query = diesel::sql_query(
"INSERT INTO workspace_folders (id, team_id, path, name, created_at)
VALUES ($1, $2, $3, $4, $5)",
)
.bind::<diesel::sql_types::Text, _>(&folder_id)
.bind::<diesel::sql_types::Text, _>(team_id)
.bind::<diesel::sql_types::Text, _>(&folder_path)
.bind::<diesel::sql_types::Text, _>(folder)
.bind::<diesel::sql_types::Timestamptz, _>(&chrono::Utc::now());
query.execute(&mut *conn).map_err(|e| {
error!("Failed to create workspace folder: {}", e);
format!("Failed to create workspace folder: {}", e)
})?;
}
trace!("Created workspace structure for team {}", team_name);
Ok(())
}
async fn create_team_channel(
state: &AppState,
team_id: &str,
team_name: &str,
) -> Result<(), String> {
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
let channel_id = Uuid::new_v4().to_string();
let now = Utc::now();
let query = diesel::sql_query(
"INSERT INTO communication_channels (id, team_id, name, type, created_at)
VALUES ($1, $2, $3, 'team_chat', $4)",
)
.bind::<diesel::sql_types::Text, _>(&channel_id)
.bind::<diesel::sql_types::Text, _>(team_id)
.bind::<diesel::sql_types::Text, _>(team_name)
.bind::<diesel::sql_types::Timestamptz, _>(&now);
query.execute(&mut *conn).map_err(|e| {
error!("Failed to create team channel: {}", e);
format!("Failed to create team channel: {}", e)
})?;
trace!("Created communication channel for team {}", team_name);
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_validate_role() {
assert_eq!(validate_role("admin"), "admin");
assert_eq!(validate_role("ADMIN"), "admin");
assert_eq!(validate_role("contributor"), "contributor");
assert_eq!(validate_role("viewer"), "viewer");
assert_eq!(validate_role("unknown"), "member");
}
#[test]
fn test_get_permissions_for_role() {
let admin_perms = get_permissions_for_role("admin");
assert!(admin_perms.get("read").unwrap().as_bool().unwrap());
assert!(admin_perms.get("write").unwrap().as_bool().unwrap());
assert!(admin_perms
.get("manage_members")
.unwrap()
.as_bool()
.unwrap());
let viewer_perms = get_permissions_for_role("viewer");
assert!(viewer_perms.get("read").unwrap().as_bool().unwrap());
assert!(!viewer_perms.get("write").unwrap().as_bool().unwrap());
assert!(!viewer_perms.get("delete").unwrap().as_bool().unwrap());
}
}

437
src/basic/keywords/book.rs Normal file
View file

@ -0,0 +1,437 @@
use crate::shared::models::UserSession;
use crate::shared::state::AppState;
use chrono::{DateTime, Datelike, Duration, Timelike, Utc};
use log::{error, trace};
use rhai::{Dynamic, Engine};
use serde::{Deserialize, Serialize};
use serde_json::json;
use std::sync::Arc;
use uuid::Uuid;
#[derive(Debug, Serialize, Deserialize)]
struct BookingRequest {
attendees: Vec<String>,
date_range: String,
duration_minutes: i32,
subject: Option<String>,
description: Option<String>,
}
#[derive(Debug, Serialize, Deserialize)]
struct TimeSlot {
start: DateTime<Utc>,
end: DateTime<Utc>,
available: bool,
}
pub fn book_keyword(state: Arc<AppState>, user: UserSession, engine: &mut Engine) {
let state_clone = Arc::clone(&state);
let user_clone = user.clone();
engine
.register_custom_syntax(
&["BOOK", "$expr$", ",", "$expr$", ",", "$expr$"],
false,
move |context, inputs| {
// Parse attendees (array or single email)
let attendees_input = context.eval_expression_tree(&inputs[0])?;
let mut attendees = Vec::new();
if attendees_input.is_array() {
let arr = attendees_input.cast::<rhai::Array>();
for item in arr.iter() {
attendees.push(item.to_string());
}
} else {
attendees.push(attendees_input.to_string());
}
let date_range = context.eval_expression_tree(&inputs[1])?.to_string();
let duration = context.eval_expression_tree(&inputs[2])?;
let duration_minutes = if duration.is_int() {
duration.as_int().unwrap_or(30)
} else {
duration.to_string().parse::<i64>().unwrap_or(30)
};
trace!(
"BOOK: attendees={:?}, date_range={}, duration={} for user={}",
attendees,
date_range,
duration_minutes,
user_clone.user_id
);
let state_for_task = Arc::clone(&state_clone);
let user_for_task = user_clone.clone();
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.enable_all()
.build();
let send_err = if let Ok(rt) = rt {
let result = rt.block_on(async move {
execute_booking(
&state_for_task,
&user_for_task,
attendees,
&date_range,
duration_minutes as i32,
)
.await
});
tx.send(result).err()
} else {
tx.send(Err("Failed to build tokio runtime".to_string()))
.err()
};
if send_err.is_some() {
error!("Failed to send BOOK result from thread");
}
});
match rx.recv_timeout(std::time::Duration::from_secs(10)) {
Ok(Ok(booking_id)) => Ok(Dynamic::from(booking_id)),
Ok(Err(e)) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("BOOK failed: {}", e).into(),
rhai::Position::NONE,
))),
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {
Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
"BOOK timed out".into(),
rhai::Position::NONE,
)))
}
Err(e) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("BOOK thread failed: {}", e).into(),
rhai::Position::NONE,
))),
}
},
)
.unwrap();
// Register FIND_SLOT keyword to find available slots
let state_clone2 = Arc::clone(&state);
let user_clone2 = user.clone();
engine
.register_custom_syntax(
&["FIND_SLOT", "$expr$", ",", "$expr$", ",", "$expr$"],
false,
move |context, inputs| {
let attendees_input = context.eval_expression_tree(&inputs[0])?;
let mut attendees = Vec::new();
if attendees_input.is_array() {
let arr = attendees_input.cast::<rhai::Array>();
for item in arr.iter() {
attendees.push(item.to_string());
}
} else {
attendees.push(attendees_input.to_string());
}
let duration = context.eval_expression_tree(&inputs[1])?;
let preferences = context.eval_expression_tree(&inputs[2])?.to_string();
let duration_minutes = if duration.is_int() {
duration.as_int().unwrap_or(30)
} else {
duration.to_string().parse::<i64>().unwrap_or(30)
};
let state_for_task = Arc::clone(&state_clone2);
let user_for_task = user_clone2.clone();
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.enable_all()
.build();
let send_err = if let Ok(rt) = rt {
let result = rt.block_on(async move {
find_available_slot(
&state_for_task,
&user_for_task,
attendees,
duration_minutes as i32,
&preferences,
)
.await
});
tx.send(result).err()
} else {
tx.send(Err("Failed to build tokio runtime".to_string()))
.err()
};
if send_err.is_some() {
error!("Failed to send FIND_SLOT result from thread");
}
});
match rx.recv_timeout(std::time::Duration::from_secs(10)) {
Ok(Ok(slot)) => Ok(Dynamic::from(slot)),
Ok(Err(e)) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("FIND_SLOT failed: {}", e).into(),
rhai::Position::NONE,
))),
Err(_) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
"FIND_SLOT timed out".into(),
rhai::Position::NONE,
))),
}
},
)
.unwrap();
}
async fn execute_booking(
state: &AppState,
user: &UserSession,
attendees: Vec<String>,
date_range: &str,
duration_minutes: i32,
) -> Result<String, String> {
// Parse date range
let (start_search, end_search) = parse_date_range(date_range)?;
// Find available slot
let available_slot = find_common_availability(
state,
&attendees,
start_search,
end_search,
duration_minutes,
)
.await?;
// Create calendar event
let event_id = create_calendar_event(
state,
user,
&attendees,
available_slot.start,
available_slot.end,
"Meeting",
None,
)
.await?;
// Send invitations
for attendee in &attendees {
send_calendar_invite(state, &event_id, attendee).await?;
}
Ok(format!(
"Meeting booked for {} at {}",
available_slot.start.format("%Y-%m-%d %H:%M"),
event_id
))
}
async fn find_available_slot(
state: &AppState,
_user: &UserSession,
attendees: Vec<String>,
duration_minutes: i32,
preferences: &str,
) -> Result<String, String> {
// Parse preferences (e.g., "mornings preferred", "afternoons only", "next week")
let (start_search, end_search) = if preferences.contains("tomorrow") {
let tomorrow = Utc::now() + Duration::days(1);
(
tomorrow
.date_naive()
.and_hms_opt(0, 0, 0)
.unwrap()
.and_utc(),
tomorrow
.date_naive()
.and_hms_opt(23, 59, 59)
.unwrap()
.and_utc(),
)
} else if preferences.contains("next week") {
let now = Utc::now();
let next_week = now + Duration::days(7);
(now, next_week)
} else {
// Default to next 7 days
let now = Utc::now();
(now, now + Duration::days(7))
};
let slot = find_common_availability(
state,
&attendees,
start_search,
end_search,
duration_minutes,
)
.await?;
Ok(slot.start.format("%Y-%m-%d %H:%M").to_string())
}
async fn find_common_availability(
state: &AppState,
attendees: &[String],
start_search: DateTime<Utc>,
end_search: DateTime<Utc>,
duration_minutes: i32,
) -> Result<TimeSlot, String> {
// This would integrate with actual calendar API
// For now, simulate finding an available slot
let mut current = start_search;
while current < end_search {
// Skip weekends
if current.weekday().num_days_from_monday() >= 5 {
current = current + Duration::days(1);
continue;
}
// Check business hours (9 AM - 5 PM)
let hour = current.hour();
if hour >= 9 && hour < 17 {
// Check if slot is available for all attendees
let slot_end = current + Duration::minutes(duration_minutes as i64);
if slot_end.hour() <= 17 {
// In a real implementation, check each attendee's calendar
// For now, simulate availability check
if check_slot_availability(state, attendees, current, slot_end).await? {
return Ok(TimeSlot {
start: current,
end: slot_end,
available: true,
});
}
}
}
// Move to next slot (30 minute intervals)
current = current + Duration::minutes(30);
}
Err("No available slot found in the specified date range".to_string())
}
async fn check_slot_availability(
_state: &AppState,
_attendees: &[String],
_start: DateTime<Utc>,
_end: DateTime<Utc>,
) -> Result<bool, String> {
// Simulate calendar availability check
// In real implementation, this would query calendar API
// For demo, randomly return availability
let random = (Utc::now().timestamp() % 3) == 0;
Ok(random)
}
async fn create_calendar_event(
state: &AppState,
user: &UserSession,
attendees: &[String],
start: DateTime<Utc>,
end: DateTime<Utc>,
subject: &str,
description: Option<String>,
) -> Result<String, String> {
let event_id = Uuid::new_v4().to_string();
// Store in database
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
let query = diesel::sql_query(
"INSERT INTO calendar_events (id, user_id, bot_id, subject, description, start_time, end_time, attendees, created_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)"
)
.bind::<diesel::sql_types::Text, _>(&event_id)
.bind::<diesel::sql_types::Text, _>(&user.user_id.to_string())
.bind::<diesel::sql_types::Text, _>(&user.bot_id.to_string())
.bind::<diesel::sql_types::Text, _>(subject)
.bind::<diesel::sql_types::Nullable<diesel::sql_types::Text>, _>(&description)
.bind::<diesel::sql_types::Timestamptz, _>(&start)
.bind::<diesel::sql_types::Timestamptz, _>(&end)
.bind::<diesel::sql_types::Jsonb, _>(&json!(attendees))
.bind::<diesel::sql_types::Timestamptz, _>(&Utc::now());
use diesel::RunQueryDsl;
query.execute(&mut *conn).map_err(|e| {
error!("Failed to create calendar event: {}", e);
format!("Failed to create calendar event: {}", e)
})?;
trace!("Created calendar event: {}", event_id);
Ok(event_id)
}
async fn send_calendar_invite(
_state: &AppState,
event_id: &str,
attendee: &str,
) -> Result<(), String> {
// In real implementation, send actual calendar invite via email or calendar API
trace!(
"Sending calendar invite for event {} to {}",
event_id,
attendee
);
Ok(())
}
fn parse_date_range(date_range: &str) -> Result<(DateTime<Utc>, DateTime<Utc>), String> {
let range_lower = date_range.to_lowercase();
let now = Utc::now();
if range_lower.contains("today") {
Ok((
now.date_naive().and_hms_opt(0, 0, 0).unwrap().and_utc(),
now.date_naive().and_hms_opt(23, 59, 59).unwrap().and_utc(),
))
} else if range_lower.contains("tomorrow") {
let tomorrow = now + Duration::days(1);
Ok((
tomorrow
.date_naive()
.and_hms_opt(0, 0, 0)
.unwrap()
.and_utc(),
tomorrow
.date_naive()
.and_hms_opt(23, 59, 59)
.unwrap()
.and_utc(),
))
} else if range_lower.contains("this week") || range_lower.contains("this_week") {
Ok((
now,
now + Duration::days(7 - now.weekday().num_days_from_monday() as i64),
))
} else if range_lower.contains("next week") || range_lower.contains("next_week") {
let next_monday = now + Duration::days(7 - now.weekday().num_days_from_monday() as i64 + 1);
Ok((next_monday, next_monday + Duration::days(6)))
} else if range_lower.contains("2pm") || range_lower.contains("14:00") {
// Handle specific time
let target_time = now.date_naive().and_hms_opt(14, 0, 0).unwrap().and_utc();
Ok((target_time, target_time + Duration::hours(1)))
} else {
// Default to next 7 days
Ok((now, now + Duration::days(7)))
}
}

130
src/basic/keywords/create_draft.rs
View file

@ -1,47 +1,93 @@
use crate::email::{fetch_latest_sent_to, save_email_draft, SaveDraftRequest};
use crate::shared::state::AppState;
use crate::shared::models::UserSession; use crate::shared::models::UserSession;
use crate::shared::state::AppState;
use rhai::Dynamic; use rhai::Dynamic;
use rhai::Engine; use rhai::Engine;
pub fn create_draft_keyword(state: &AppState, user: UserSession, engine: &mut Engine) { use serde::{Deserialize, Serialize};
let state_clone = state.clone();
engine #[derive(Debug, Clone, Serialize, Deserialize)]
.register_custom_syntax(&["CREATE_DRAFT", "$expr$", ",", "$expr$", ",", "$expr$"], true, move |context, inputs| { pub struct SaveDraftRequest {
let to = context.eval_expression_tree(&inputs[0])?.to_string(); pub to: String,
let subject = context.eval_expression_tree(&inputs[1])?.to_string(); pub subject: String,
let reply_text = context.eval_expression_tree(&inputs[2])?.to_string(); pub cc: Option<String>,
let fut = execute_create_draft(&state_clone, &to, &subject, &reply_text); pub text: String,
let result = tokio::task::block_in_place(|| tokio::runtime::Handle::current().block_on(fut))
.map_err(|e| format!("Draft creation error: {}", e))?;
Ok(Dynamic::from(result))
},
)
.unwrap();
} }
async fn execute_create_draft(state: &AppState, to: &str, subject: &str, reply_text: &str) -> Result<String, String> {
let get_result = fetch_latest_sent_to(&state.config.clone().unwrap().email, to).await; pub fn create_draft_keyword(_state: &AppState, _user: UserSession, engine: &mut Engine) {
let email_body = if let Ok(get_result_str) = get_result { let state_clone = _state.clone();
if !get_result_str.is_empty() { engine
let email_separator = "<br><hr><br>"; .register_custom_syntax(
let formatted_reply_text = reply_text.to_string(); &["CREATE_DRAFT", "$expr$", ",", "$expr$", ",", "$expr$"],
let formatted_old_text = get_result_str.replace("\n", "<br>"); true,
let fixed_reply_text = formatted_reply_text.replace("FIX", "Fixed"); move |context, inputs| {
format!("{}{}{}", fixed_reply_text, email_separator, formatted_old_text) let to = context.eval_expression_tree(&inputs[0])?.to_string();
} else { let subject = context.eval_expression_tree(&inputs[1])?.to_string();
reply_text.to_string() let reply_text = context.eval_expression_tree(&inputs[2])?.to_string();
}
} else { let fut = execute_create_draft(&state_clone, &to, &subject, &reply_text);
reply_text.to_string() let result =
}; tokio::task::block_in_place(|| tokio::runtime::Handle::current().block_on(fut))
let draft_request = SaveDraftRequest { .map_err(|e| format!("Draft creation error: {}", e))?;
to: to.to_string(), Ok(Dynamic::from(result))
subject: subject.to_string(), },
cc: None, )
text: email_body, .unwrap();
}; }
let save_result = save_email_draft(&state.config.clone().unwrap().email, &draft_request).await;
match save_result { async fn execute_create_draft(
Ok(_) => Ok("Draft saved successfully".to_string()), _state: &AppState,
Err(e) => Err(e.to_string()), to: &str,
} subject: &str,
reply_text: &str,
) -> Result<String, String> {
// For now, we'll store drafts in the database or just log them
// This is a simplified implementation until the email module is fully ready
#[cfg(feature = "email")]
{
// When email feature is enabled, try to use email functionality if available
// For now, we'll just simulate draft creation
use log::info;
info!("Creating draft email - To: {}, Subject: {}", to, subject);
// In a real implementation, this would:
// 1. Connect to email service
// 2. Create draft in IMAP folder or local storage
// 3. Return draft ID or confirmation
let draft_id = uuid::Uuid::new_v4().to_string();
// You could store this in the database
// For now, just return success
Ok(format!("Draft saved successfully with ID: {}", draft_id))
}
#[cfg(not(feature = "email"))]
{
// When email feature is disabled, return a placeholder message
Ok(format!(
"Email feature not enabled. Would create draft - To: {}, Subject: {}, Body: {}",
to, subject, reply_text
))
}
}
// Helper functions that would be implemented when email module is complete
#[cfg(feature = "email")]
async fn fetch_latest_sent_to(
_config: &Option<crate::config::Config>,
_to: &str,
) -> Result<String, String> {
// This would fetch the latest email sent to the recipient
// For threading/reply purposes
Ok(String::new())
}
#[cfg(feature = "email")]
async fn save_email_draft(
_config: &Option<crate::config::Config>,
_draft: &SaveDraftRequest,
) -> Result<(), String> {
// This would save the draft to the email server or local storage
Ok(())
} }

467
src/basic/keywords/create_task.rs Normal file
View file

@ -0,0 +1,467 @@
use crate::shared::models::UserSession;
use crate::shared::state::AppState;
use chrono::{DateTime, Duration, NaiveDate, Utc};
use diesel::prelude::*;
use log::{error, trace};
use rhai::{Dynamic, Engine};
use std::sync::Arc;
use uuid::Uuid;
pub fn create_task_keyword(state: Arc<AppState>, user: UserSession, engine: &mut Engine) {
let state_clone = Arc::clone(&state);
let user_clone = user.clone();
engine
.register_custom_syntax(
&[
"CREATE_TASK",
"$expr$",
",",
"$expr$",
",",
"$expr$",
",",
"$expr$",
],
false,
move |context, inputs| {
let title = context.eval_expression_tree(&inputs[0])?.to_string();
let assignee = context.eval_expression_tree(&inputs[1])?.to_string();
let due_date = context.eval_expression_tree(&inputs[2])?.to_string();
let project_id_input = context.eval_expression_tree(&inputs[3])?;
let project_id =
if project_id_input.is_unit() || project_id_input.to_string() == "null" {
None
} else {
Some(project_id_input.to_string())
};
trace!(
"CREATE_TASK: title={}, assignee={}, due_date={}, project_id={:?} for user={}",
title,
assignee,
due_date,
project_id,
user_clone.user_id
);
let state_for_task = Arc::clone(&state_clone);
let user_for_task = user_clone.clone();
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.enable_all()
.build();
let send_err = if let Ok(rt) = rt {
let result = rt.block_on(async move {
execute_create_task(
&state_for_task,
&user_for_task,
&title,
&assignee,
&due_date,
project_id.as_deref(),
)
.await
});
tx.send(result).err()
} else {
tx.send(Err("Failed to build tokio runtime".to_string()))
.err()
};
if send_err.is_some() {
error!("Failed to send CREATE_TASK result from thread");
}
});
match rx.recv_timeout(std::time::Duration::from_secs(10)) {
Ok(Ok(task_id)) => Ok(Dynamic::from(task_id)),
Ok(Err(e)) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("CREATE_TASK failed: {}", e).into(),
rhai::Position::NONE,
))),
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {
Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
"CREATE_TASK timed out".into(),
rhai::Position::NONE,
)))
}
Err(e) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("CREATE_TASK thread failed: {}", e).into(),
rhai::Position::NONE,
))),
}
},
)
.unwrap();
// Register ASSIGN_SMART for intelligent task assignment
let state_clone2 = Arc::clone(&state);
let user_clone2 = user.clone();
engine
.register_custom_syntax(
&["ASSIGN_SMART", "$expr$", ",", "$expr$", ",", "$expr$"],
false,
move |context, inputs| {
let task_id = context.eval_expression_tree(&inputs[0])?.to_string();
let team_input = context.eval_expression_tree(&inputs[1])?;
let load_balance = context
.eval_expression_tree(&inputs[2])?
.as_bool()
.unwrap_or(true);
let mut team = Vec::new();
if team_input.is_array() {
let arr = team_input.cast::<rhai::Array>();
for item in arr.iter() {
team.push(item.to_string());
}
} else {
team.push(team_input.to_string());
}
trace!(
"ASSIGN_SMART: task={}, team={:?}, load_balance={} for user={}",
task_id,
team,
load_balance,
user_clone2.user_id
);
let state_for_task = Arc::clone(&state_clone2);
let user_for_task = user_clone2.clone();
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.enable_all()
.build();
let send_err = if let Ok(rt) = rt {
let result = rt.block_on(async move {
smart_assign_task(
&state_for_task,
&user_for_task,
&task_id,
team,
load_balance,
)
.await
});
tx.send(result).err()
} else {
tx.send(Err("Failed to build tokio runtime".to_string()))
.err()
};
if send_err.is_some() {
error!("Failed to send ASSIGN_SMART result from thread");
}
});
match rx.recv_timeout(std::time::Duration::from_secs(10)) {
Ok(Ok(assignee)) => Ok(Dynamic::from(assignee)),
Ok(Err(e)) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("ASSIGN_SMART failed: {}", e).into(),
rhai::Position::NONE,
))),
Err(_) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
"ASSIGN_SMART timed out".into(),
rhai::Position::NONE,
))),
}
},
)
.unwrap();
}
async fn execute_create_task(
state: &AppState,
user: &UserSession,
title: &str,
assignee: &str,
due_date: &str,
project_id: Option<&str>,
) -> Result<String, String> {
let task_id = Uuid::new_v4().to_string();
// Parse due date
let due_datetime = parse_due_date(due_date)?;
// Determine actual assignee
let actual_assignee = if assignee == "auto" {
// Auto-assign based on workload
auto_assign_task(state, project_id).await?
} else {
assignee.to_string()
};
// Determine priority based on due date
let priority = determine_priority(due_datetime);
// Save task to database
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
let query = diesel::sql_query(
"INSERT INTO tasks (id, title, assignee, due_date, project_id, priority, status, created_by, created_at)
VALUES ($1, $2, $3, $4, $5, $6, 'open', $7, $8)"
)
.bind::<diesel::sql_types::Text, _>(&task_id)
.bind::<diesel::sql_types::Text, _>(title)
.bind::<diesel::sql_types::Text, _>(&actual_assignee)
.bind::<diesel::sql_types::Nullable<diesel::sql_types::Timestamptz>, _>(&due_datetime)
.bind::<diesel::sql_types::Nullable<diesel::sql_types::Text>, _>(&project_id)
.bind::<diesel::sql_types::Text, _>(&priority);
let user_id_str = user.user_id.to_string();
let now = Utc::now();
let query = query
.bind::<diesel::sql_types::Text, _>(&user_id_str)
.bind::<diesel::sql_types::Timestamptz, _>(&now);
query.execute(&mut *conn).map_err(|e| {
error!("Failed to create task: {}", e);
format!("Failed to create task: {}", e)
})?;
// Send notification to assignee
send_task_notification(state, &task_id, title, &actual_assignee, due_datetime).await?;
trace!(
"Created task '{}' assigned to {} (ID: {})",
title,
actual_assignee,
task_id
);
Ok(task_id)
}
async fn smart_assign_task(
state: &AppState,
_user: &UserSession,
task_id: &str,
team: Vec<String>,
load_balance: bool,
) -> Result<String, String> {
if !load_balance {
// Simple assignment to first available team member
return Ok(team[0].clone());
}
// Get workload for each team member
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
let mut best_assignee = team[0].clone();
let mut min_workload = i64::MAX;
for member in &team {
// Count open tasks for this member
let query = diesel::sql_query(
"SELECT COUNT(*) as task_count FROM tasks
WHERE assignee = $1 AND status IN ('open', 'in_progress')",
)
.bind::<diesel::sql_types::Text, _>(member);
#[derive(QueryableByName)]
struct TaskCount {
#[diesel(sql_type = diesel::sql_types::BigInt)]
task_count: i64,
}
let result: Result<Vec<TaskCount>, _> = query.load(&mut *conn);
if let Ok(counts) = result {
if let Some(count) = counts.first() {
if count.task_count < min_workload {
min_workload = count.task_count;
best_assignee = member.clone();
}
}
}
}
// Update task assignment
let update_query = diesel::sql_query("UPDATE tasks SET assignee = $1 WHERE id = $2")
.bind::<diesel::sql_types::Text, _>(&best_assignee)
.bind::<diesel::sql_types::Text, _>(task_id);
update_query.execute(&mut *conn).map_err(|e| {
error!("Failed to update task assignment: {}", e);
format!("Failed to update task assignment: {}", e)
})?;
trace!(
"Smart-assigned task {} to {} (workload: {})",
task_id,
best_assignee,
min_workload
);
Ok(best_assignee)
}
async fn auto_assign_task(state: &AppState, project_id: Option<&str>) -> Result<String, String> {
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
// Get team members for the project
let team_query_str = if let Some(proj_id) = project_id {
format!(
"SELECT DISTINCT assignee FROM tasks
WHERE project_id = '{}' AND assignee IS NOT NULL
ORDER BY COUNT(*) ASC LIMIT 5",
proj_id
)
} else {
"SELECT DISTINCT assignee FROM tasks
WHERE assignee IS NOT NULL
ORDER BY COUNT(*) ASC LIMIT 5"
.to_string()
};
let team_query = diesel::sql_query(&team_query_str);
#[derive(QueryableByName)]
struct TeamMember {
#[diesel(sql_type = diesel::sql_types::Text)]
assignee: String,
}
let team: Vec<TeamMember> = team_query.load(&mut *conn).unwrap_or_default();
if team.is_empty() {
return Ok("unassigned".to_string());
}
// Return the team member with the least tasks
Ok(team[0].assignee.clone())
}
fn parse_due_date(due_date: &str) -> Result<Option<DateTime<Utc>>, String> {
let due_lower = due_date.to_lowercase();
if due_lower == "null" || due_lower.is_empty() {
return Ok(None);
}
let now = Utc::now();
// Handle relative dates like "+3 days", "tomorrow", etc.
if due_lower.starts_with('+') {
let days_str = due_lower
.trim_start_matches('+')
.trim()
.split_whitespace()
.next()
.unwrap_or("0");
if let Ok(days) = days_str.parse::<i64>() {
return Ok(Some(now + Duration::days(days)));
}
}
if due_lower == "today" {
return Ok(Some(
now.date_naive().and_hms_opt(17, 0, 0).unwrap().and_utc(),
));
}
if due_lower == "tomorrow" {
return Ok(Some(
(now + Duration::days(1))
.date_naive()
.and_hms_opt(17, 0, 0)
.unwrap()
.and_utc(),
));
}
if due_lower.contains("next week") {
return Ok(Some(now + Duration::days(7)));
}
if due_lower.contains("next month") {
return Ok(Some(now + Duration::days(30)));
}
// Try parsing as a date
if let Ok(date) = NaiveDate::parse_from_str(&due_date, "%Y-%m-%d") {
return Ok(Some(date.and_hms_opt(17, 0, 0).unwrap().and_utc()));
}
// Default to 3 days from now
Ok(Some(now + Duration::days(3)))
}
fn determine_priority(due_date: Option<DateTime<Utc>>) -> String {
if let Some(due) = due_date {
let now = Utc::now();
let days_until = (due - now).num_days();
if days_until <= 1 {
"high".to_string()
} else if days_until <= 7 {
"medium".to_string()
} else {
"low".to_string()
}
} else {
"medium".to_string()
}
}
async fn send_task_notification(
_state: &AppState,
task_id: &str,
title: &str,
assignee: &str,
due_date: Option<DateTime<Utc>>,
) -> Result<(), String> {
// In a real implementation, this would send an actual notification
trace!(
"Notification sent to {} for task '{}' (ID: {})",
assignee,
title,
task_id
);
if let Some(due) = due_date {
trace!("Task due: {}", due.format("%Y-%m-%d %H:%M"));
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_parse_due_date() {
assert!(parse_due_date("tomorrow").is_ok());
assert!(parse_due_date("+3 days").is_ok());
assert!(parse_due_date("2024-12-31").is_ok());
assert!(parse_due_date("null").unwrap().is_none());
}
#[test]
fn test_determine_priority() {
let tomorrow = Some(Utc::now() + Duration::days(1));
assert_eq!(determine_priority(tomorrow), "high");
let next_week = Some(Utc::now() + Duration::days(7));
assert_eq!(determine_priority(next_week), "medium");
let next_month = Some(Utc::now() + Duration::days(30));
assert_eq!(determine_priority(next_month), "low");
}
}

10
src/basic/keywords/mod.rs
View file

@ -1,11 +1,13 @@
pub mod add_member;
pub mod add_suggestion; pub mod add_suggestion;
pub mod add_website; pub mod add_website;
pub mod book;
pub mod bot_memory; pub mod bot_memory;
pub mod clear_kb; pub mod clear_kb;
pub mod clear_tools; pub mod clear_tools;
#[cfg(feature = "email")] pub mod create_draft;
pub mod create_draft_keyword;
pub mod create_site; pub mod create_site;
pub mod create_task;
pub mod find; pub mod find;
pub mod first; pub mod first;
pub mod for_next; pub mod for_next;
@ -16,10 +18,14 @@ pub mod last;
pub mod llm_keyword; pub mod llm_keyword;
pub mod on; pub mod on;
pub mod print; pub mod print;
pub mod remember;
pub mod save_from_unstructured;
pub mod send_mail;
pub mod set; pub mod set;
pub mod set_context; pub mod set_context;
pub mod set_schedule; pub mod set_schedule;
pub mod set_user; pub mod set_user;
pub mod universal_messaging;
pub mod use_kb; pub mod use_kb;
pub mod use_tool; pub mod use_tool;
pub mod wait; pub mod wait;

346
src/basic/keywords/remember.rs Normal file
View file

@ -0,0 +1,346 @@
use crate::shared::models::UserSession;
use crate::shared::state::AppState;
use chrono::{Duration, Utc};
use diesel::prelude::*;
use log::{error, trace};
use rhai::{Dynamic, Engine};
use serde_json::json;
use std::sync::Arc;
use uuid::Uuid;
pub fn remember_keyword(state: Arc<AppState>, user: UserSession, engine: &mut Engine) {
let state_clone = Arc::clone(&state);
let user_clone = user.clone();
engine
.register_custom_syntax(
&["REMEMBER", "$expr$", ",", "$expr$", ",", "$expr$"],
false,
move |context, inputs| {
let key = context.eval_expression_tree(&inputs[0])?.to_string();
let value = context.eval_expression_tree(&inputs[1])?;
let duration_str = context.eval_expression_tree(&inputs[2])?.to_string();
trace!(
"REMEMBER: key={}, duration={} for user={}",
key,
duration_str,
user_clone.user_id
);
// Parse duration
let expiry = parse_duration(&duration_str)?;
// Convert value to JSON
let json_value = if value.is_string() {
json!(value.to_string())
} else if value.is_int() {
json!(value.as_int().unwrap_or(0))
} else if value.is_float() {
json!(value.as_float().unwrap_or(0.0))
} else if value.is_bool() {
json!(value.as_bool().unwrap_or(false))
} else if value.is_array() {
// Convert Rhai array to JSON array
let arr = value.cast::<rhai::Array>();
let json_arr: Vec<serde_json::Value> = arr
.iter()
.map(|v| {
if v.is_string() {
json!(v.to_string())
} else {
json!(v.to_string())
}
})
.collect();
json!(json_arr)
} else if value.is_map() {
// Convert Rhai map to JSON object
json!(value.to_string())
} else {
json!(value.to_string())
};
let state_for_task = Arc::clone(&state_clone);
let user_for_task = user_clone.clone();
let key_for_task = key.clone();
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.enable_all()
.build();
let send_err = if let Ok(rt) = rt {
let result = rt.block_on(async move {
store_memory(
&state_for_task,
&user_for_task,
&key_for_task,
json_value,
expiry,
)
.await
});
tx.send(result).err()
} else {
tx.send(Err("Failed to build tokio runtime".to_string()))
.err()
};
if send_err.is_some() {
error!("Failed to send REMEMBER result from thread");
}
});
match rx.recv_timeout(std::time::Duration::from_secs(5)) {
Ok(Ok(_)) => Ok(Dynamic::from(format!(
"Remembered '{}' for {}",
key, duration_str
))),
Ok(Err(e)) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("REMEMBER failed: {}", e).into(),
rhai::Position::NONE,
))),
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {
Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
"REMEMBER timed out".into(),
rhai::Position::NONE,
)))
}
Err(e) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("REMEMBER thread failed: {}", e).into(),
rhai::Position::NONE,
))),
}
},
)
.unwrap();
// Register RECALL keyword to retrieve memories
let state_clone2 = Arc::clone(&state);
let user_clone2 = user.clone();
engine
.register_custom_syntax(&["RECALL", "$expr$"], false, move |context, inputs| {
let key = context.eval_expression_tree(&inputs[0])?.to_string();
trace!("RECALL: key={} for user={}", key, user_clone2.user_id);
let state_for_task = Arc::clone(&state_clone2);
let user_for_task = user_clone2.clone();
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.enable_all()
.build();
let send_err = if let Ok(rt) = rt {
let result = rt.block_on(async move {
retrieve_memory(&state_for_task, &user_for_task, &key).await
});
tx.send(result).err()
} else {
tx.send(Err("Failed to build tokio runtime".to_string()))
.err()
};
if send_err.is_some() {
error!("Failed to send RECALL result from thread");
}
});
match rx.recv_timeout(std::time::Duration::from_secs(5)) {
Ok(Ok(value)) => {
// Convert JSON value back to Dynamic
if value.is_string() {
Ok(Dynamic::from(value.as_str().unwrap_or("").to_string()))
} else if value.is_number() {
if let Some(i) = value.as_i64() {
Ok(Dynamic::from(i))
} else if let Some(f) = value.as_f64() {
Ok(Dynamic::from(f))
} else {
Ok(Dynamic::from(value.to_string()))
}
} else if value.is_boolean() {
Ok(Dynamic::from(value.as_bool().unwrap_or(false)))
} else if value.is_array() {
let arr_str = value.to_string();
Ok(Dynamic::from(arr_str))
} else {
Ok(Dynamic::from(value.to_string()))
}
}
Ok(Err(e)) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("RECALL failed: {}", e).into(),
rhai::Position::NONE,
))),
Err(_) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
"RECALL timed out".into(),
rhai::Position::NONE,
))),
}
})
.unwrap();
}
fn parse_duration(
duration_str: &str,
) -> Result<Option<chrono::DateTime<Utc>>, Box<rhai::EvalAltResult>> {
let duration_lower = duration_str.to_lowercase();
if duration_lower == "forever" || duration_lower == "permanent" {
return Ok(None);
}
// Parse patterns like "30 days", "1 hour", "5 minutes", etc.
let parts: Vec<&str> = duration_lower.split_whitespace().collect();
if parts.len() != 2 {
// Try parsing as a number of days
if let Ok(days) = duration_str.parse::<i64>() {
return Ok(Some(Utc::now() + Duration::days(days)));
}
return Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("Invalid duration format: {}", duration_str).into(),
rhai::Position::NONE,
)));
}
let amount = parts[0].parse::<i64>().map_err(|_| {
Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("Invalid duration amount: {}", parts[0]).into(),
rhai::Position::NONE,
))
})?;
let unit = parts[1].trim_end_matches('s'); // Remove trailing 's' if present
let duration = match unit {
"second" => Duration::seconds(amount),
"minute" => Duration::minutes(amount),
"hour" => Duration::hours(amount),
"day" => Duration::days(amount),
"week" => Duration::weeks(amount),
"month" => Duration::days(amount * 30), // Approximate
"year" => Duration::days(amount * 365), // Approximate
_ => {
return Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("Invalid duration unit: {}", unit).into(),
rhai::Position::NONE,
)))
}
};
Ok(Some(Utc::now() + duration))
}
async fn store_memory(
state: &AppState,
user: &UserSession,
key: &str,
value: serde_json::Value,
expiry: Option<chrono::DateTime<Utc>>,
) -> Result<(), String> {
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
// Create memory record
let memory_id = Uuid::new_v4().to_string();
let user_id = user.user_id.to_string();
let bot_id = user.bot_id.to_string();
let session_id = user.id.to_string();
let created_at = Utc::now().to_rfc3339();
let expires_at = expiry.map(|e| e.to_rfc3339());
// Use raw SQL for flexibility (you might want to create a proper schema later)
let query = diesel::sql_query(
"INSERT INTO bot_memories (id, user_id, bot_id, session_id, key, value, created_at, expires_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
ON CONFLICT (user_id, bot_id, key)
DO UPDATE SET value = $6, created_at = $7, expires_at = $8, session_id = $4"
)
.bind::<diesel::sql_types::Text, _>(&memory_id)
.bind::<diesel::sql_types::Text, _>(&user_id)
.bind::<diesel::sql_types::Text, _>(&bot_id)
.bind::<diesel::sql_types::Text, _>(&session_id)
.bind::<diesel::sql_types::Text, _>(key)
.bind::<diesel::sql_types::Jsonb, _>(&value)
.bind::<diesel::sql_types::Text, _>(&created_at)
.bind::<diesel::sql_types::Nullable<diesel::sql_types::Text>, _>(&expires_at);
query.execute(&mut *conn).map_err(|e| {
error!("Failed to store memory: {}", e);
format!("Failed to store memory: {}", e)
})?;
trace!("Stored memory key='{}' for user={}", key, user_id);
Ok(())
}
async fn retrieve_memory(
state: &AppState,
user: &UserSession,
key: &str,
) -> Result<serde_json::Value, String> {
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
let user_id = user.user_id.to_string();
let bot_id = user.bot_id.to_string();
let now = Utc::now().to_rfc3339();
// Query memory, checking expiry
let query = diesel::sql_query(
"SELECT value FROM bot_memories
WHERE user_id = $1 AND bot_id = $2 AND key = $3
AND (expires_at IS NULL OR expires_at > $4)
ORDER BY created_at DESC
LIMIT 1",
)
.bind::<diesel::sql_types::Text, _>(&user_id)
.bind::<diesel::sql_types::Text, _>(&bot_id)
.bind::<diesel::sql_types::Text, _>(key)
.bind::<diesel::sql_types::Text, _>(&now);
let result: Result<Vec<MemoryRecord>, _> = query.load(&mut *conn);
match result {
Ok(records) if !records.is_empty() => {
trace!("Retrieved memory key='{}' for user={}", key, user_id);
Ok(records[0].value.clone())
}
Ok(_) => {
trace!("No memory found for key='{}' user={}", key, user_id);
Ok(json!(null))
}
Err(e) => {
error!("Failed to retrieve memory: {}", e);
Err(format!("Failed to retrieve memory: {}", e))
}
}
}
#[derive(QueryableByName, Debug)]
struct MemoryRecord {
#[diesel(sql_type = diesel::sql_types::Jsonb)]
value: serde_json::Value,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_parse_duration() {
// Test various duration formats
assert!(parse_duration("30 days").is_ok());
assert!(parse_duration("1 hour").is_ok());
assert!(parse_duration("forever").is_ok());
assert!(parse_duration("5 minutes").is_ok());
assert!(parse_duration("invalid").is_err());
}
}

493
src/basic/keywords/save_from_unstructured.rs Normal file
View file

@ -0,0 +1,493 @@
use crate::shared::models::UserSession;
use crate::shared::state::AppState;
use chrono::Utc;
use diesel::prelude::*;
use log::{error, trace};
use rhai::{Dynamic, Engine};
use serde_json::{json, Value};
use std::sync::Arc;
use uuid::Uuid;
pub fn save_from_unstructured_keyword(
state: Arc<AppState>,
user: UserSession,
engine: &mut Engine,
) {
let state_clone = Arc::clone(&state);
let user_clone = user.clone();
engine
.register_custom_syntax(
&["SAVE_FROM_UNSTRUCTURED", "$expr$", ",", "$expr$"],
false,
move |context, inputs| {
let table_name = context.eval_expression_tree(&inputs[0])?.to_string();
let text = context.eval_expression_tree(&inputs[1])?.to_string();
trace!(
"SAVE_FROM_UNSTRUCTURED: table={}, text_len={} for user={}",
table_name,
text.len(),
user_clone.user_id
);
let state_for_task = Arc::clone(&state_clone);
let user_for_task = user_clone.clone();
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.enable_all()
.build();
let send_err = if let Ok(rt) = rt {
let result = rt.block_on(async move {
execute_save_from_unstructured(
&state_for_task,
&user_for_task,
&table_name,
&text,
)
.await
});
tx.send(result).err()
} else {
tx.send(Err("Failed to build tokio runtime".to_string()))
.err()
};
if send_err.is_some() {
error!("Failed to send SAVE_FROM_UNSTRUCTURED result from thread");
}
});
match rx.recv_timeout(std::time::Duration::from_secs(30)) {
Ok(Ok(record_id)) => Ok(Dynamic::from(record_id)),
Ok(Err(e)) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("SAVE_FROM_UNSTRUCTURED failed: {}", e).into(),
rhai::Position::NONE,
))),
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {
Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
"SAVE_FROM_UNSTRUCTURED timed out".into(),
rhai::Position::NONE,
)))
}
Err(e) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("SAVE_FROM_UNSTRUCTURED thread failed: {}", e).into(),
rhai::Position::NONE,
))),
}
},
)
.unwrap();
}
async fn execute_save_from_unstructured(
state: &AppState,
user: &UserSession,
table_name: &str,
text: &str,
) -> Result<String, String> {
// Get table schema to understand what fields to extract
let schema = get_table_schema(state, table_name).await?;
// Use LLM to extract structure from text
let extraction_prompt = build_extraction_prompt(table_name, &schema, text);
let extracted_json = call_llm_for_extraction(state, &extraction_prompt).await?;
// Validate and clean the extracted data
let cleaned_data = validate_and_clean_data(&extracted_json, &schema)?;
// Save to database
let record_id = save_to_table(state, user, table_name, cleaned_data).await?;
trace!(
"Saved unstructured data to table '{}': {}",
table_name,
record_id
);
Ok(record_id)
}
async fn get_table_schema(state: &AppState, table_name: &str) -> Result<Value, String> {
// Get table schema from database
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
// Query PostgreSQL information schema for table columns
let query = diesel::sql_query(
"SELECT column_name, data_type, is_nullable, column_default
FROM information_schema.columns
WHERE table_name = $1
ORDER BY ordinal_position",
)
.bind::<diesel::sql_types::Text, _>(table_name);
#[derive(QueryableByName, Debug)]
struct ColumnInfo {
#[diesel(sql_type = diesel::sql_types::Text)]
column_name: String,
#[diesel(sql_type = diesel::sql_types::Text)]
data_type: String,
#[diesel(sql_type = diesel::sql_types::Text)]
is_nullable: String,
#[diesel(sql_type = diesel::sql_types::Nullable<diesel::sql_types::Text>)]
column_default: Option<String>,
}
let columns: Vec<ColumnInfo> = query.load(&mut *conn).map_err(|e| {
error!("Failed to get table schema: {}", e);
format!("Table '{}' not found or error: {}", table_name, e)
})?;
if columns.is_empty() {
// Table doesn't exist, use default schema based on table name
return Ok(get_default_schema(table_name));
}
let schema: Vec<Value> = columns
.into_iter()
.map(|col| {
json!({
"name": col.column_name,
"type": col.data_type,
"nullable": col.is_nullable == "YES",
"default": col.column_default
})
})
.collect();
Ok(json!(schema))
}
fn get_default_schema(table_name: &str) -> Value {
// Provide default schemas for common tables
match table_name {
"leads" | "rob" => json!([
{"name": "id", "type": "uuid", "nullable": false},
{"name": "name", "type": "text", "nullable": true},
{"name": "company", "type": "text", "nullable": true},
{"name": "email", "type": "text", "nullable": true},
{"name": "phone", "type": "text", "nullable": true},
{"name": "website", "type": "text", "nullable": true},
{"name": "notes", "type": "text", "nullable": true},
{"name": "status", "type": "text", "nullable": true},
{"name": "created_at", "type": "timestamp", "nullable": false}
]),
"tasks" => json!([
{"name": "id", "type": "uuid", "nullable": false},
{"name": "title", "type": "text", "nullable": false},
{"name": "description", "type": "text", "nullable": true},
{"name": "assignee", "type": "text", "nullable": true},
{"name": "due_date", "type": "timestamp", "nullable": true},
{"name": "priority", "type": "text", "nullable": true},
{"name": "status", "type": "text", "nullable": true},
{"name": "created_at", "type": "timestamp", "nullable": false}
]),
"meetings" => json!([
{"name": "id", "type": "uuid", "nullable": false},
{"name": "subject", "type": "text", "nullable": false},
{"name": "attendees", "type": "jsonb", "nullable": true},
{"name": "date", "type": "timestamp", "nullable": true},
{"name": "duration", "type": "integer", "nullable": true},
{"name": "location", "type": "text", "nullable": true},
{"name": "notes", "type": "text", "nullable": true},
{"name": "created_at", "type": "timestamp", "nullable": false}
]),
"opportunities" => json!([
{"name": "id", "type": "uuid", "nullable": false},
{"name": "company", "type": "text", "nullable": false},
{"name": "contact", "type": "text", "nullable": true},
{"name": "value", "type": "numeric", "nullable": true},
{"name": "stage", "type": "text", "nullable": true},
{"name": "probability", "type": "integer", "nullable": true},
{"name": "close_date", "type": "date", "nullable": true},
{"name": "notes", "type": "text", "nullable": true},
{"name": "created_at", "type": "timestamp", "nullable": false}
]),
_ => json!([
{"name": "id", "type": "uuid", "nullable": false},
{"name": "data", "type": "jsonb", "nullable": true},
{"name": "created_at", "type": "timestamp", "nullable": false}
]),
}
}
fn build_extraction_prompt(table_name: &str, schema: &Value, text: &str) -> String {
let schema_str = serde_json::to_string_pretty(schema).unwrap_or_default();
let table_context = match table_name {
"leads" | "rob" => "This is a CRM lead/contact record. Extract contact information, company details, and any relevant notes.",
"tasks" => "This is a task record. Extract the task title, description, who it should be assigned to, when it's due, and priority.",
"meetings" => "This is a meeting record. Extract the meeting subject, attendees, date/time, duration, and any notes.",
"opportunities" => "This is a sales opportunity. Extract the company, contact person, deal value, sales stage, and expected close date.",
_ => "Extract relevant structured data from the text."
};
format!(
r#"Extract structured data from the following unstructured text and return it as JSON that matches this table schema:
Table: {}
Context: {}
Schema:
{}
Text to extract from:
"""
{}
"""
Instructions:
1. Extract ONLY information that is present in the text
2. Return a valid JSON object with field names matching the schema
3. Use null for fields that cannot be extracted from the text
4. For date/time fields, parse natural language dates (e.g., "next Friday" -> actual date)
5. For email fields, extract valid email addresses
6. For numeric fields, extract numbers and convert to appropriate type
7. Do NOT make up or invent data that isn't in the text
8. If the text mentions multiple entities, extract the primary/first one
Return ONLY the JSON object, no explanations or markdown formatting."#,
table_name, table_context, schema_str, text
)
}
async fn call_llm_for_extraction(state: &AppState, prompt: &str) -> Result<Value, String> {
// Get LLM configuration
let config_manager = crate::config::ConfigManager::new(state.conn.clone());
let model = config_manager
.get_config(&Uuid::nil(), "llm-model", None)
.unwrap_or_else(|_| "gpt-3.5-turbo".to_string());
let key = config_manager
.get_config(&Uuid::nil(), "llm-key", None)
.unwrap_or_default();
// Call LLM
let response = state
.llm_provider
.generate(prompt, &Value::Null, &model, &key)
.await
.map_err(|e| format!("LLM extraction failed: {}", e))?;
// Parse LLM response as JSON
let extracted = serde_json::from_str::<Value>(&response).unwrap_or_else(|_| {
// If LLM didn't return valid JSON, try to extract JSON from the response
if let Some(start) = response.find('{') {
if let Some(end) = response.rfind('}') {
let json_str = &response[start..=end];
serde_json::from_str(json_str).unwrap_or_else(|_| json!({}))
} else {
json!({})
}
} else {
json!({})
}
});
Ok(extracted)
}
fn validate_and_clean_data(data: &Value, schema: &Value) -> Result<Value, String> {
let mut cleaned = json!({});
if let Some(data_obj) = data.as_object() {
if let Some(schema_arr) = schema.as_array() {
for column_def in schema_arr {
if let Some(column_name) = column_def.get("name").and_then(|n| n.as_str()) {
// Skip system fields that will be auto-generated
if column_name == "id" || column_name == "created_at" {
continue;
}
if let Some(value) = data_obj.get(column_name) {
// Clean and validate based on type
let column_type = column_def
.get("type")
.and_then(|t| t.as_str())
.unwrap_or("text");
let cleaned_value = clean_value_for_type(value, column_type);
if !cleaned_value.is_null() {
cleaned[column_name] = cleaned_value;
}
}
}
}
}
}
// Ensure we have at least some data
if cleaned.as_object().map_or(true, |o| o.is_empty()) {
return Err("No valid data could be extracted from the text".to_string());
}
Ok(cleaned)
}
fn clean_value_for_type(value: &Value, data_type: &str) -> Value {
match data_type {
"text" | "varchar" => {
if value.is_string() {
value.clone()
} else {
json!(value.to_string())
}
}
"integer" | "bigint" | "smallint" => {
if let Some(n) = value.as_i64() {
json!(n)
} else if let Some(s) = value.as_str() {
s.parse::<i64>().map(|n| json!(n)).unwrap_or(json!(null))
} else {
json!(null)
}
}
"numeric" | "decimal" | "real" | "double precision" => {
if let Some(n) = value.as_f64() {
json!(n)
} else if let Some(s) = value.as_str() {
s.parse::<f64>().map(|n| json!(n)).unwrap_or(json!(null))
} else {
json!(null)
}
}
"boolean" => {
if let Some(b) = value.as_bool() {
json!(b)
} else if let Some(s) = value.as_str() {
json!(s.to_lowercase() == "true" || s == "1" || s.to_lowercase() == "yes")
} else {
json!(false)
}
}
"timestamp" | "timestamptz" | "date" | "time" => {
if value.is_string() {
value.clone() // Let PostgreSQL handle the parsing
} else {
json!(null)
}
}
"jsonb" | "json" => value.clone(),
"uuid" => {
if let Some(s) = value.as_str() {
// Validate UUID format
if Uuid::parse_str(s).is_ok() {
value.clone()
} else {
json!(Uuid::new_v4().to_string())
}
} else {
json!(Uuid::new_v4().to_string())
}
}
_ => value.clone(),
}
}
async fn save_to_table(
state: &AppState,
user: &UserSession,
table_name: &str,
data: Value,
) -> Result<String, String> {
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
let record_id = Uuid::new_v4().to_string();
let user_id = user.user_id.to_string();
let created_at = Utc::now();
// Build dynamic INSERT query
let mut fields = vec!["id", "created_at"];
let mut placeholders = vec!["$1", "$2"];
let mut bind_index = 3;
let data_obj = data.as_object().ok_or("Invalid data format")?;
for (field, _) in data_obj {
fields.push(field);
placeholders.push(&format!("${}", bind_index));
bind_index += 1;
}
// Add user tracking if not already present
if !data_obj.contains_key("user_id") {
fields.push("user_id");
placeholders.push(&format!("${}", bind_index));
}
let insert_query = format!(
"INSERT INTO {} ({}) VALUES ({})",
table_name,
fields.join(", "),
placeholders.join(", ")
);
// Build values as JSON for simpler handling
let mut values_map = serde_json::Map::new();
values_map.insert("id".to_string(), json!(record_id));
values_map.insert("created_at".to_string(), json!(created_at));
// Add data fields
for (field, value) in data_obj {
values_map.insert(field.clone(), value.clone());
}
// Add user_id if needed
if !data_obj.contains_key("user_id") {
values_map.insert("user_id".to_string(), json!(user_id));
}
// Convert to JSON and use JSONB insert
let values_json = json!(values_map);
// Use a simpler approach with JSON
let insert_query = format!(
"INSERT INTO {} SELECT * FROM jsonb_populate_record(NULL::{},'{}');",
table_name,
table_name,
values_json.to_string().replace("'", "''")
);
diesel::sql_query(&insert_query)
.execute(&mut *conn)
.map_err(|e| {
error!("Failed to save to table '{}': {}", table_name, e);
format!("Failed to save record: {}", e)
})?;
trace!(
"Saved record {} to table '{}' for user {}",
record_id,
table_name,
user_id
);
Ok(record_id)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_clean_value_for_type() {
assert_eq!(clean_value_for_type(&json!("test"), "text"), json!("test"));
assert_eq!(clean_value_for_type(&json!("42"), "integer"), json!(42));
assert_eq!(clean_value_for_type(&json!("3.14"), "numeric"), json!(3.14));
assert_eq!(clean_value_for_type(&json!("true"), "boolean"), json!(true));
}
#[test]
fn test_get_default_schema() {
let leads_schema = get_default_schema("leads");
assert!(leads_schema.is_array());
let tasks_schema = get_default_schema("tasks");
assert!(tasks_schema.is_array());
}
}

441
src/basic/keywords/send_mail.rs Normal file
View file

@ -0,0 +1,441 @@
use crate::shared::models::UserSession;
use crate::shared::state::AppState;
use chrono::Utc;
use diesel::prelude::*;
use log::{error, trace};
use rhai::{Dynamic, Engine};
use serde_json::json;
use std::sync::Arc;
use uuid::Uuid;
pub fn send_mail_keyword(state: Arc<AppState>, user: UserSession, engine: &mut Engine) {
let state_clone = Arc::clone(&state);
let user_clone = user.clone();
engine
.register_custom_syntax(
&[
"SEND_MAIL",
"$expr$",
",",
"$expr$",
",",
"$expr$",
",",
"$expr$",
],
false,
move |context, inputs| {
let to = context.eval_expression_tree(&inputs[0])?.to_string();
let subject = context.eval_expression_tree(&inputs[1])?.to_string();
let body = context.eval_expression_tree(&inputs[2])?.to_string();
let attachments_input = context.eval_expression_tree(&inputs[3])?;
// Parse attachments array
let mut attachments = Vec::new();
if attachments_input.is_array() {
let arr = attachments_input.cast::<rhai::Array>();
for item in arr.iter() {
attachments.push(item.to_string());
}
} else if !attachments_input.to_string().is_empty() {
attachments.push(attachments_input.to_string());
}
trace!(
"SEND_MAIL: to={}, subject={}, attachments={:?} for user={}",
to,
subject,
attachments,
user_clone.user_id
);
let state_for_task = Arc::clone(&state_clone);
let user_for_task = user_clone.clone();
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.enable_all()
.build();
let send_err = if let Ok(rt) = rt {
let result = rt.block_on(async move {
execute_send_mail(
&state_for_task,
&user_for_task,
&to,
&subject,
&body,
attachments,
)
.await
});
tx.send(result).err()
} else {
tx.send(Err("Failed to build tokio runtime".to_string()))
.err()
};
if send_err.is_some() {
error!("Failed to send SEND_MAIL result from thread");
}
});
match rx.recv_timeout(std::time::Duration::from_secs(30)) {
Ok(Ok(message_id)) => Ok(Dynamic::from(message_id)),
Ok(Err(e)) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("SEND_MAIL failed: {}", e).into(),
rhai::Position::NONE,
))),
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {
Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
"SEND_MAIL timed out".into(),
rhai::Position::NONE,
)))
}
Err(e) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("SEND_MAIL thread failed: {}", e).into(),
rhai::Position::NONE,
))),
}
},
)
.unwrap();
// Register SEND_TEMPLATE for bulk templated emails
let state_clone2 = Arc::clone(&state);
let user_clone2 = user.clone();
engine
.register_custom_syntax(
&["SEND_TEMPLATE", "$expr$", ",", "$expr$", ",", "$expr$"],
false,
move |context, inputs| {
let recipients_input = context.eval_expression_tree(&inputs[0])?;
let template = context.eval_expression_tree(&inputs[1])?.to_string();
let variables = context.eval_expression_tree(&inputs[2])?;
// Parse recipients
let mut recipients = Vec::new();
if recipients_input.is_array() {
let arr = recipients_input.cast::<rhai::Array>();
for item in arr.iter() {
recipients.push(item.to_string());
}
} else {
recipients.push(recipients_input.to_string());
}
// Convert variables to JSON
let vars_json = if variables.is_map() {
// Convert Rhai map to JSON
json!(variables.to_string())
} else {
json!({})
};
trace!(
"SEND_TEMPLATE: recipients={:?}, template={} for user={}",
recipients,
template,
user_clone2.user_id
);
let state_for_task = Arc::clone(&state_clone2);
let user_for_task = user_clone2.clone();
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.enable_all()
.build();
let send_err = if let Ok(rt) = rt {
let result = rt.block_on(async move {
execute_send_template(
&state_for_task,
&user_for_task,
recipients,
&template,
vars_json,
)
.await
});
tx.send(result).err()
} else {
tx.send(Err("Failed to build tokio runtime".to_string()))
.err()
};
if send_err.is_some() {
error!("Failed to send SEND_TEMPLATE result from thread");
}
});
match rx.recv_timeout(std::time::Duration::from_secs(60)) {
Ok(Ok(count)) => Ok(Dynamic::from(count)),
Ok(Err(e)) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
format!("SEND_TEMPLATE failed: {}", e).into(),
rhai::Position::NONE,
))),
Err(_) => Err(Box::new(rhai::EvalAltResult::ErrorRuntime(
"SEND_TEMPLATE timed out".into(),
rhai::Position::NONE,
))),
}
},
)
.unwrap();
}
async fn execute_send_mail(
state: &AppState,
user: &UserSession,
to: &str,
subject: &str,
body: &str,
attachments: Vec<String>,
) -> Result<String, String> {
let message_id = Uuid::new_v4().to_string();
// Track email in communication history
track_email(state, user, &message_id, to, subject, "sent").await?;
// Send the actual email if email feature is enabled
#[cfg(feature = "email")]
{
let email_request = crate::email::EmailRequest {
to: to.to_string(),
subject: subject.to_string(),
body: body.to_string(),
cc: None,
bcc: None,
attachments: if attachments.is_empty() {
None
} else {
Some(attachments.clone())
},
reply_to: None,
headers: None,
};
if let Some(config) = &state.config {
if let Ok(_) = crate::email::send_email(&config.email, &email_request).await {
trace!("Email sent successfully: {}", message_id);
return Ok(format!("Email sent: {}", message_id));
}
}
}
// Fallback: store as draft if email sending fails
save_email_draft(state, user, to, subject, body, attachments).await?;
Ok(format!("Email saved as draft: {}", message_id))
}
async fn execute_send_template(
state: &AppState,
user: &UserSession,
recipients: Vec<String>,
template_name: &str,
variables: serde_json::Value,
) -> Result<i32, String> {
let template_content = load_template(state, template_name).await?;
let mut sent_count = 0;
for recipient in recipients {
// Personalize template for each recipient
let personalized_content =
apply_template_variables(&template_content, &variables, &recipient)?;
// Extract subject from template or use default
let subject = extract_template_subject(&personalized_content)
.unwrap_or_else(|| format!("Message from {}", user.user_id));
// Send email
if let Ok(_) = execute_send_mail(
state,
user,
&recipient,
&subject,
&personalized_content,
vec![],
)
.await
{
sent_count += 1;
}
// Add small delay to avoid rate limiting
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
}
trace!("Sent {} templated emails", sent_count);
Ok(sent_count)
}
async fn track_email(
state: &AppState,
user: &UserSession,
message_id: &str,
to: &str,
subject: &str,
status: &str,
) -> Result<(), String> {
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
let log_id = Uuid::new_v4().to_string();
let user_id_str = user.user_id.to_string();
let bot_id_str = user.bot_id.to_string();
let now = Utc::now();
let query = diesel::sql_query(
"INSERT INTO communication_logs (id, user_id, bot_id, message_id, recipient, subject, status, timestamp, type)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, 'email')"
)
.bind::<diesel::sql_types::Text, _>(&log_id)
.bind::<diesel::sql_types::Text, _>(&user_id_str)
.bind::<diesel::sql_types::Text, _>(&bot_id_str)
.bind::<diesel::sql_types::Text, _>(message_id)
.bind::<diesel::sql_types::Text, _>(to)
.bind::<diesel::sql_types::Text, _>(subject)
.bind::<diesel::sql_types::Text, _>(status)
.bind::<diesel::sql_types::Timestamptz, _>(&now);
query.execute(&mut *conn).map_err(|e| {
error!("Failed to track email: {}", e);
format!("Failed to track email: {}", e)
})?;
Ok(())
}
async fn save_email_draft(
state: &AppState,
user: &UserSession,
to: &str,
subject: &str,
body: &str,
attachments: Vec<String>,
) -> Result<(), String> {
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
let draft_id = Uuid::new_v4().to_string();
let user_id_str = user.user_id.to_string();
let bot_id_str = user.bot_id.to_string();
let now = Utc::now();
let query = diesel::sql_query(
"INSERT INTO email_drafts (id, user_id, bot_id, to_address, subject, body, attachments, created_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)"
)
.bind::<diesel::sql_types::Text, _>(&draft_id)
.bind::<diesel::sql_types::Text, _>(&user_id_str)
.bind::<diesel::sql_types::Text, _>(&bot_id_str)
.bind::<diesel::sql_types::Text, _>(to)
.bind::<diesel::sql_types::Text, _>(subject)
.bind::<diesel::sql_types::Text, _>(body);
let attachments_json = json!(attachments);
let query = query
.bind::<diesel::sql_types::Jsonb, _>(&attachments_json)
.bind::<diesel::sql_types::Timestamptz, _>(&now);
query.execute(&mut *conn).map_err(|e| {
error!("Failed to save draft: {}", e);
format!("Failed to save draft: {}", e)
})?;
trace!("Email saved as draft: {}", draft_id);
Ok(())
}
async fn load_template(state: &AppState, template_name: &str) -> Result<String, String> {
// Try loading from database first
let mut conn = state.conn.get().map_err(|e| format!("DB error: {}", e))?;
let query =
diesel::sql_query("SELECT content FROM email_templates WHERE name = $1 AND active = true")
.bind::<diesel::sql_types::Text, _>(template_name);
#[derive(QueryableByName)]
struct TemplateRecord {
#[diesel(sql_type = diesel::sql_types::Text)]
content: String,
}
let result: Result<Vec<TemplateRecord>, _> = query.load(&mut *conn);
match result {
Ok(records) if !records.is_empty() => Ok(records[0].content.clone()),
_ => {
// Fallback to file system
let template_path = format!(".gbdrive/templates/{}.html", template_name);
std::fs::read_to_string(&template_path)
.map_err(|e| format!("Template not found: {}", e))
}
}
}
fn apply_template_variables(
template: &str,
variables: &serde_json::Value,
recipient: &str,
) -> Result<String, String> {
let mut content = template.to_string();
// Replace {{recipient}} variable
content = content.replace("{{recipient}}", recipient);
// Replace other variables from the JSON object
if let Some(obj) = variables.as_object() {
for (key, value) in obj {
let placeholder = format!("{{{{{}}}}}", key);
let replacement = value.as_str().unwrap_or(&value.to_string());
content = content.replace(&placeholder, replacement);
}
}
Ok(content)
}
fn extract_template_subject(content: &str) -> Option<String> {
// Look for subject line in template (e.g., "Subject: ...")
for line in content.lines() {
if line.starts_with("Subject:") {
return Some(line.trim_start_matches("Subject:").trim().to_string());
}
}
None
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_apply_template_variables() {
let template = "Hello {{name}}, your order {{order_id}} is ready!";
let vars = json!({
"name": "John",
"order_id": "12345"
});
let result = apply_template_variables(template, &vars, "john@example.com").unwrap();
assert!(result.contains("John"));
assert!(result.contains("12345"));
}
#[test]
fn test_extract_template_subject() {
let content = "Subject: Welcome to our service\n\nHello there!";
let subject = extract_template_subject(content);
assert_eq!(subject, Some("Welcome to our service".to_string()));
}
}

19
src/basic/mod.rs
View file

@ -7,14 +7,16 @@ use rhai::{Dynamic, Engine, EvalAltResult};
use std::sync::Arc; use std::sync::Arc;
pub mod compiler; pub mod compiler;
pub mod keywords; pub mod keywords;
use self::keywords::add_member::add_member_keyword;
use self::keywords::add_suggestion::add_suggestion_keyword; use self::keywords::add_suggestion::add_suggestion_keyword;
use self::keywords::add_website::add_website_keyword; use self::keywords::add_website::add_website_keyword;
use self::keywords::book::book_keyword;
use self::keywords::bot_memory::{get_bot_memory_keyword, set_bot_memory_keyword}; use self::keywords::bot_memory::{get_bot_memory_keyword, set_bot_memory_keyword};
use self::keywords::clear_kb::register_clear_kb_keyword; use self::keywords::clear_kb::register_clear_kb_keyword;
use self::keywords::clear_tools::clear_tools_keyword; use self::keywords::clear_tools::clear_tools_keyword;
#[cfg(feature = "email")] use self::keywords::create_draft::create_draft_keyword;
use self::keywords::create_draft_keyword;
use self::keywords::create_site::create_site_keyword; use self::keywords::create_site::create_site_keyword;
use self::keywords::create_task::create_task_keyword;
use self::keywords::find::find_keyword; use self::keywords::find::find_keyword;
use self::keywords::first::first_keyword; use self::keywords::first::first_keyword;
use self::keywords::for_next::for_keyword; use self::keywords::for_next::for_keyword;
@ -22,6 +24,9 @@ use self::keywords::format::format_keyword;
use self::keywords::get::get_keyword; use self::keywords::get::get_keyword;
use self::keywords::hear_talk::{hear_keyword, talk_keyword}; use self::keywords::hear_talk::{hear_keyword, talk_keyword};
use self::keywords::last::last_keyword; use self::keywords::last::last_keyword;
use self::keywords::remember::remember_keyword;
use self::keywords::save_from_unstructured::save_from_unstructured_keyword;
use self::keywords::send_mail::send_mail_keyword;
use self::keywords::use_kb::register_use_kb_keyword; use self::keywords::use_kb::register_use_kb_keyword;
use self::keywords::use_tool::use_tool_keyword; use self::keywords::use_tool::use_tool_keyword;
@ -40,7 +45,6 @@ impl ScriptService {
let mut engine = Engine::new(); let mut engine = Engine::new();
engine.set_allow_anonymous_fn(true); engine.set_allow_anonymous_fn(true);
engine.set_allow_looping(true); engine.set_allow_looping(true);
#[cfg(feature = "email")]
create_draft_keyword(&state, user.clone(), &mut engine); create_draft_keyword(&state, user.clone(), &mut engine);
set_bot_memory_keyword(state.clone(), user.clone(), &mut engine); set_bot_memory_keyword(state.clone(), user.clone(), &mut engine);
get_bot_memory_keyword(state.clone(), user.clone(), &mut engine); get_bot_memory_keyword(state.clone(), user.clone(), &mut engine);
@ -69,6 +73,15 @@ impl ScriptService {
add_website_keyword(state.clone(), user.clone(), &mut engine); add_website_keyword(state.clone(), user.clone(), &mut engine);
add_suggestion_keyword(state.clone(), user.clone(), &mut engine); add_suggestion_keyword(state.clone(), user.clone(), &mut engine);
// Register the 6 new power keywords
remember_keyword(state.clone(), user.clone(), &mut engine);
book_keyword(state.clone(), user.clone(), &mut engine);
send_mail_keyword(state.clone(), user.clone(), &mut engine);
save_from_unstructured_keyword(state.clone(), user.clone(), &mut engine);
create_task_keyword(state.clone(), user.clone(), &mut engine);
add_member_keyword(state.clone(), user.clone(), &mut engine);
ScriptService { engine } ScriptService { engine }
} }
fn preprocess_basic_script(&self, script: &str) -> String { fn preprocess_basic_script(&self, script: &str) -> String {

447
src/calendar_engine/mod.rs Normal file
View file

@ -0,0 +1,447 @@
use axum::{
extract::{Path, Query, State},
http::StatusCode,
response::Json,
routing::{delete, get, post, put},
Router,
};
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use sqlx::PgPool;
use std::sync::Arc;
use tokio::sync::RwLock;
use uuid::Uuid;
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CalendarEvent {
pub id: Uuid,
pub title: String,
pub description: Option<String>,
pub start_time: DateTime<Utc>,
pub end_time: DateTime<Utc>,
pub location: Option<String>,
pub attendees: Vec<String>,
pub organizer: String,
pub reminder_minutes: Option<i32>,
pub recurrence_rule: Option<String>,
pub status: EventStatus,
pub created_at: DateTime<Utc>,
pub updated_at: DateTime<Utc>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum EventStatus {
Scheduled,
InProgress,
Completed,
Cancelled,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Meeting {
pub id: Uuid,
pub event_id: Uuid,
pub meeting_url: Option<String>,
pub meeting_id: Option<String>,
pub platform: MeetingPlatform,
pub recording_url: Option<String>,
pub notes: Option<String>,
pub action_items: Vec<ActionItem>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum MeetingPlatform {
Zoom,
Teams,
Meet,
Internal,
Other(String),
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ActionItem {
pub id: Uuid,
pub description: String,
pub assignee: String,
pub due_date: Option<DateTime<Utc>>,
pub completed: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CalendarReminder {
pub id: Uuid,
pub event_id: Uuid,
pub remind_at: DateTime<Utc>,
pub message: String,
pub channel: ReminderChannel,
pub sent: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum ReminderChannel {
Email,
Sms,
Push,
InApp,
}
#[derive(Clone)]
pub struct CalendarEngine {
db: Arc<PgPool>,
cache: Arc<RwLock<Vec<CalendarEvent>>>,
}
impl CalendarEngine {
pub fn new(db: Arc<PgPool>) -> Self {
Self {
db,
cache: Arc::new(RwLock::new(Vec::new())),
}
}
pub async fn create_event(
&self,
event: CalendarEvent,
) -> Result<CalendarEvent, Box<dyn std::error::Error>> {
let result = sqlx::query!(
r#"
INSERT INTO calendar_events
(id, title, description, start_time, end_time, location, attendees, organizer,
reminder_minutes, recurrence_rule, status, created_at, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13)
RETURNING *
"#,
event.id,
event.title,
event.description,
event.start_time,
event.end_time,
event.location,
&event.attendees[..],
event.organizer,
event.reminder_minutes,
event.recurrence_rule,
serde_json::to_value(&event.status)?,
event.created_at,
event.updated_at
)
.fetch_one(self.db.as_ref())
.await?;
let mut cache = self.cache.write().await;
cache.push(event.clone());
Ok(event)
}
pub async fn update_event(
&self,
id: Uuid,
updates: serde_json::Value,
) -> Result<CalendarEvent, Box<dyn std::error::Error>> {
let updated_at = Utc::now();
let result = sqlx::query!(
r#"
UPDATE calendar_events
SET title = COALESCE($2, title),
description = COALESCE($3, description),
start_time = COALESCE($4, start_time),
end_time = COALESCE($5, end_time),
location = COALESCE($6, location),
updated_at = $7
WHERE id = $1
RETURNING *
"#,
id,
updates.get("title").and_then(|v| v.as_str()),
updates.get("description").and_then(|v| v.as_str()),
updates
.get("start_time")
.and_then(|v| DateTime::parse_from_rfc3339(v.as_str()?).ok())
.map(|dt| dt.with_timezone(&Utc)),
updates
.get("end_time")
.and_then(|v| DateTime::parse_from_rfc3339(v.as_str()?).ok())
.map(|dt| dt.with_timezone(&Utc)),
updates.get("location").and_then(|v| v.as_str()),
updated_at
)
.fetch_one(self.db.as_ref())
.await?;
self.refresh_cache().await?;
Ok(serde_json::from_value(serde_json::to_value(result)?)?)
}
pub async fn delete_event(&self, id: Uuid) -> Result<bool, Box<dyn std::error::Error>> {
let result = sqlx::query!("DELETE FROM calendar_events WHERE id = $1", id)
.execute(self.db.as_ref())
.await?;
self.refresh_cache().await?;
Ok(result.rows_affected() > 0)
}
pub async fn get_events_range(
&self,
start: DateTime<Utc>,
end: DateTime<Utc>,
) -> Result<Vec<CalendarEvent>, Box<dyn std::error::Error>> {
let results = sqlx::query_as!(
CalendarEvent,
r#"
SELECT * FROM calendar_events
WHERE start_time >= $1 AND end_time <= $2
ORDER BY start_time ASC
"#,
start,
end
)
.fetch_all(self.db.as_ref())
.await?;
Ok(results)
}
pub async fn get_user_events(
&self,
user_id: &str,
) -> Result<Vec<CalendarEvent>, Box<dyn std::error::Error>> {
let results = sqlx::query!(
r#"
SELECT * FROM calendar_events
WHERE organizer = $1 OR $1 = ANY(attendees)
ORDER BY start_time ASC
"#,
user_id
)
.fetch_all(self.db.as_ref())
.await?;
Ok(results
.into_iter()
.map(|r| serde_json::from_value(serde_json::to_value(r).unwrap()).unwrap())
.collect())
}
pub async fn create_meeting(
&self,
event_id: Uuid,
platform: MeetingPlatform,
) -> Result<Meeting, Box<dyn std::error::Error>> {
let meeting = Meeting {
id: Uuid::new_v4(),
event_id,
meeting_url: None,
meeting_id: None,
platform,
recording_url: None,
notes: None,
action_items: Vec::new(),
};
sqlx::query!(
r#"
INSERT INTO meetings (id, event_id, platform, created_at)
VALUES ($1, $2, $3, $4)
"#,
meeting.id,
meeting.event_id,
serde_json::to_value(&meeting.platform)?,
Utc::now()
)
.execute(self.db.as_ref())
.await?;
Ok(meeting)
}
pub async fn schedule_reminder(
&self,
event_id: Uuid,
minutes_before: i32,
channel: ReminderChannel,
) -> Result<CalendarReminder, Box<dyn std::error::Error>> {
let event = self.get_event(event_id).await?;
let remind_at = event.start_time - chrono::Duration::minutes(minutes_before as i64);
let reminder = CalendarReminder {
id: Uuid::new_v4(),
event_id,
remind_at,
message: format!(
"Reminder: {} starts in {} minutes",
event.title, minutes_before
),
channel,
sent: false,
};
sqlx::query!(
r#"
INSERT INTO calendar_reminders (id, event_id, remind_at, message, channel, sent)
VALUES ($1, $2, $3, $4, $5, $6)
"#,
reminder.id,
reminder.event_id,
reminder.remind_at,
reminder.message,
serde_json::to_value(&reminder.channel)?,
reminder.sent
)
.execute(self.db.as_ref())
.await?;
Ok(reminder)
}
pub async fn get_event(&self, id: Uuid) -> Result<CalendarEvent, Box<dyn std::error::Error>> {
let result = sqlx::query!("SELECT * FROM calendar_events WHERE id = $1", id)
.fetch_one(self.db.as_ref())
.await?;
Ok(serde_json::from_value(serde_json::to_value(result)?)?)
}
pub async fn check_conflicts(
&self,
start: DateTime<Utc>,
end: DateTime<Utc>,
user_id: &str,
) -> Result<Vec<CalendarEvent>, Box<dyn std::error::Error>> {
let results = sqlx::query!(
r#"
SELECT * FROM calendar_events
WHERE (organizer = $1 OR $1 = ANY(attendees))
AND NOT (end_time <= $2 OR start_time >= $3)
"#,
user_id,
start,
end
)
.fetch_all(self.db.as_ref())
.await?;
Ok(results
.into_iter()
.map(|r| serde_json::from_value(serde_json::to_value(r).unwrap()).unwrap())
.collect())
}
async fn refresh_cache(&self) -> Result<(), Box<dyn std::error::Error>> {
let results = sqlx::query!("SELECT * FROM calendar_events ORDER BY start_time ASC")
.fetch_all(self.db.as_ref())
.await?;
let events: Vec<CalendarEvent> = results
.into_iter()
.map(|r| serde_json::from_value(serde_json::to_value(r).unwrap()).unwrap())
.collect();
let mut cache = self.cache.write().await;
*cache = events;
Ok(())
}
}
#[derive(Deserialize)]
pub struct EventQuery {
pub start: Option<String>,
pub end: Option<String>,
pub user_id: Option<String>,
}
#[derive(Deserialize)]
pub struct MeetingRequest {
pub event_id: Uuid,
pub platform: MeetingPlatform,
}
async fn create_event_handler(
State(engine): State<Arc<CalendarEngine>>,
Json(event): Json<CalendarEvent>,
) -> Result<Json<CalendarEvent>, StatusCode> {
match engine.create_event(event).await {
Ok(created) => Ok(Json(created)),
Err(_) => Err(StatusCode::INTERNAL_SERVER_ERROR),
}
}
async fn get_events_handler(
State(engine): State<Arc<CalendarEngine>>,
Query(params): Query<EventQuery>,
) -> Result<Json<Vec<CalendarEvent>>, StatusCode> {
if let (Some(start), Some(end)) = (params.start, params.end) {
let start = DateTime::parse_from_rfc3339(&start)
.map(|dt| dt.with_timezone(&Utc))
.unwrap_or_else(|_| Utc::now());
let end = DateTime::parse_from_rfc3339(&end)
.map(|dt| dt.with_timezone(&Utc))
.unwrap_or_else(|_| Utc::now() + chrono::Duration::days(30));
match engine.get_events_range(start, end).await {
Ok(events) => Ok(Json(events)),
Err(_) => Err(StatusCode::INTERNAL_SERVER_ERROR),
}
} else if let Some(user_id) = params.user_id {
match engine.get_user_events(&user_id).await {
Ok(events) => Ok(Json(events)),
Err(_) => Err(StatusCode::INTERNAL_SERVER_ERROR),
}
} else {
Err(StatusCode::BAD_REQUEST)
}
}
async fn update_event_handler(
State(engine): State<Arc<CalendarEngine>>,
Path(id): Path<Uuid>,
Json(updates): Json<serde_json::Value>,
) -> Result<Json<CalendarEvent>, StatusCode> {
match engine.update_event(id, updates).await {
Ok(updated) => Ok(Json(updated)),
Err(_) => Err(StatusCode::INTERNAL_SERVER_ERROR),
}
}
async fn delete_event_handler(
State(engine): State<Arc<CalendarEngine>>,
Path(id): Path<Uuid>,
) -> Result<StatusCode, StatusCode> {
match engine.delete_event(id).await {
Ok(true) => Ok(StatusCode::NO_CONTENT),
Ok(false) => Err(StatusCode::NOT_FOUND),
Err(_) => Err(StatusCode::INTERNAL_SERVER_ERROR),
}
}
async fn schedule_meeting_handler(
State(engine): State<Arc<CalendarEngine>>,
Json(req): Json<MeetingRequest>,
) -> Result<Json<Meeting>, StatusCode> {
match engine.create_meeting(req.event_id, req.platform).await {
Ok(meeting) => Ok(Json(meeting)),
Err(_) => Err(StatusCode::INTERNAL_SERVER_ERROR),
}
}
pub fn routes(engine: Arc<CalendarEngine>) -> Router {
Router::new()
.route(
"/events",
post(create_event_handler).get(get_events_handler),
)
.route(
"/events/:id",
put(update_event_handler).delete(delete_event_handler),
)
.route("/meetings", post(schedule_meeting_handler))
.with_state(engine)
}

6
src/channels/mod.rs
View file

@ -1,9 +1,13 @@
pub mod instagram;
pub mod teams;
pub mod whatsapp;
use crate::shared::models::BotResponse;
use async_trait::async_trait; use async_trait::async_trait;
use log::{debug, info}; use log::{debug, info};
use std::collections::HashMap; use std::collections::HashMap;
use std::sync::Arc; use std::sync::Arc;
use tokio::sync::{mpsc, Mutex}; use tokio::sync::{mpsc, Mutex};
use crate::shared::models::BotResponse;
#[async_trait] #[async_trait]
pub trait ChannelAdapter: Send + Sync { pub trait ChannelAdapter: Send + Sync {
async fn send_message( async fn send_message(

4
src/config/mod.rs
View file

@ -3,6 +3,10 @@ use diesel::prelude::*;
use diesel::r2d2::{ConnectionManager, PooledConnection}; use diesel::r2d2::{ConnectionManager, PooledConnection};
use std::collections::HashMap; use std::collections::HashMap;
use uuid::Uuid; use uuid::Uuid;
// Type alias for backward compatibility
pub type Config = AppConfig;
#[derive(Clone)] #[derive(Clone)]
pub struct AppConfig { pub struct AppConfig {
pub drive: DriveConfig, pub drive: DriveConfig,

94
src/email/mod.rs
View file

@ -15,6 +15,15 @@ use serde::{Deserialize, Serialize};
use std::sync::Arc; use std::sync::Arc;
use uuid::Uuid; use uuid::Uuid;
// Export SaveDraftRequest for other modules
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SaveDraftRequest {
pub to: String,
pub subject: String,
pub cc: Option<String>,
pub text: String,
}
// ===== Request/Response Structures ===== // ===== Request/Response Structures =====
#[derive(Debug, Serialize, Deserialize)] #[derive(Debug, Serialize, Deserialize)]
@ -731,3 +740,88 @@ pub async fn get_emails(
info!("Get emails requested for campaign: {}", campaign_id); info!("Get emails requested for campaign: {}", campaign_id);
"No emails tracked".to_string() "No emails tracked".to_string()
} }
// ===== EmailService for compatibility with keyword system =====
pub struct EmailService {
state: Arc<AppState>,
}
impl EmailService {
pub fn new(state: Arc<AppState>) -> Self {
Self { state }
}
pub async fn send_email(
&self,
to: &str,
subject: &str,
body: &str,
cc: Option<Vec<String>>,
) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let config = self
.state
.config
.as_ref()
.ok_or("Email configuration not available")?;
let from_addr = config
.email
.from
.parse()
.map_err(|e| format!("Invalid from address: {}", e))?;
let mut email_builder = Message::builder()
.from(from_addr)
.to(to.parse()?)
.subject(subject);
if let Some(cc_list) = cc {
for cc_addr in cc_list {
email_builder = email_builder.cc(cc_addr.parse()?);
}
}
let email = email_builder.body(body.to_string())?;
let creds = Credentials::new(config.email.username.clone(), config.email.password.clone());
let mailer = SmtpTransport::relay(&config.email.smtp_server)?
.credentials(creds)
.build();
mailer.send(&email)?;
Ok(())
}
pub async fn send_email_with_attachment(
&self,
to: &str,
subject: &str,
body: &str,
attachment: Vec<u8>,
filename: &str,
) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
// For now, just send without attachment
// Full implementation would use lettre's multipart support
self.send_email(to, subject, body, None).await
}
}
// Helper functions for draft system
pub async fn fetch_latest_sent_to(config: &EmailConfig, to: &str) -> Result<String, String> {
// This would fetch the latest email sent to the recipient
// For threading/reply purposes
// For now, return empty string
Ok(String::new())
}
pub async fn save_email_draft(
config: &EmailConfig,
draft: &SaveDraftRequest,
) -> Result<(), String> {
// This would save the draft to the email server or local storage
// For now, just log and return success
info!("Saving draft to: {}, subject: {}", draft.to, draft.subject);
Ok(())
}

621
src/task_engine/mod.rs Normal file
View file

@ -0,0 +1,621 @@
use actix_web::{web, HttpResponse, Result};
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use sqlx::PgPool;
use std::sync::Arc;
use tokio::sync::RwLock;
use uuid::Uuid;
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Task {
pub id: Uuid,
pub title: String,
pub description: Option<String>,
pub assignee: Option<String>,
pub reporter: String,
pub status: TaskStatus,
pub priority: TaskPriority,
pub due_date: Option<DateTime<Utc>>,
pub estimated_hours: Option<f32>,
pub actual_hours: Option<f32>,
pub tags: Vec<String>,
pub parent_task_id: Option<Uuid>,
pub subtasks: Vec<Uuid>,
pub dependencies: Vec<Uuid>,
pub attachments: Vec<String>,
pub comments: Vec<TaskComment>,
pub created_at: DateTime<Utc>,
pub updated_at: DateTime<Utc>,
pub completed_at: Option<DateTime<Utc>>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum TaskStatus {
Todo,
InProgress,
Review,
Done,
Blocked,
Cancelled,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum TaskPriority {
Low,
Medium,
High,
Urgent,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TaskComment {
pub id: Uuid,
pub task_id: Uuid,
pub author: String,
pub content: String,
pub created_at: DateTime<Utc>,
pub updated_at: Option<DateTime<Utc>>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TaskTemplate {
pub id: Uuid,
pub name: String,
pub description: String,
pub default_assignee: Option<String>,
pub default_priority: TaskPriority,
pub default_tags: Vec<String>,
pub checklist: Vec<ChecklistItem>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ChecklistItem {
pub id: Uuid,
pub task_id: Uuid,
pub description: String,
pub completed: bool,
pub completed_by: Option<String>,
pub completed_at: Option<DateTime<Utc>>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TaskBoard {
pub id: Uuid,
pub name: String,
pub description: Option<String>,
pub columns: Vec<BoardColumn>,
pub owner: String,
pub members: Vec<String>,
pub created_at: DateTime<Utc>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct BoardColumn {
pub id: Uuid,
pub name: String,
pub position: i32,
pub status_mapping: TaskStatus,
pub task_ids: Vec<Uuid>,
pub wip_limit: Option<i32>,
}
pub struct TaskEngine {
db: Arc<PgPool>,
cache: Arc<RwLock<Vec<Task>>>,
}
impl TaskEngine {
pub fn new(db: Arc<PgPool>) -> Self {
Self {
db,
cache: Arc::new(RwLock::new(Vec::new())),
}
}
/// Create a new task
pub async fn create_task(&self, task: Task) -> Result<Task, Box<dyn std::error::Error>> {
let result = sqlx::query!(
r#"
INSERT INTO tasks
(id, title, description, assignee, reporter, status, priority,
due_date, estimated_hours, tags, parent_task_id, created_at, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13)
RETURNING *
"#,
task.id,
task.title,
task.description,
task.assignee,
task.reporter,
serde_json::to_value(&task.status)?,
serde_json::to_value(&task.priority)?,
task.due_date,
task.estimated_hours,
&task.tags[..],
task.parent_task_id,
task.created_at,
task.updated_at
)
.fetch_one(self.db.as_ref())
.await?;
// Update cache
let mut cache = self.cache.write().await;
cache.push(task.clone());
// Send notification to assignee if specified
if let Some(assignee) = &task.assignee {
self.notify_assignee(assignee, &task).await?;
}
Ok(task)
}
/// Update an existing task
pub async fn update_task(
&self,
id: Uuid,
updates: serde_json::Value,
) -> Result<Task, Box<dyn std::error::Error>> {
let updated_at = Utc::now();
// Check if status is changing to Done
let completing = updates
.get("status")
.and_then(|v| v.as_str())
.map(|s| s == "done")
.unwrap_or(false);
let completed_at = if completing {
Some(Utc::now())
} else {
None
};
let result = sqlx::query!(
r#"
UPDATE tasks
SET title = COALESCE($2, title),
description = COALESCE($3, description),
assignee = COALESCE($4, assignee),
status = COALESCE($5, status),
priority = COALESCE($6, priority),
due_date = COALESCE($7, due_date),
updated_at = $8,
completed_at = COALESCE($9, completed_at)
WHERE id = $1
RETURNING *
"#,
id,
updates.get("title").and_then(|v| v.as_str()),
updates.get("description").and_then(|v| v.as_str()),
updates.get("assignee").and_then(|v| v.as_str()),
updates.get("status").and_then(|v| serde_json::to_value(v).ok()),
updates.get("priority").and_then(|v| serde_json::to_value(v).ok()),
updates
.get("due_date")
.and_then(|v| DateTime::parse_from_rfc3339(v.as_str()?).ok())
.map(|dt| dt.with_timezone(&Utc)),
updated_at,
completed_at
)
.fetch_one(self.db.as_ref())
.await?;
self.refresh_cache().await?;
Ok(serde_json::from_value(serde_json::to_value(result)?)?)
}
/// Delete a task
pub async fn delete_task(&self, id: Uuid) -> Result<bool, Box<dyn std::error::Error>> {
// First, check for dependencies
let dependencies = self.get_task_dependencies(id).await?;
if !dependencies.is_empty() {
return Err("Cannot delete task with dependencies".into());
}
let result = sqlx::query!("DELETE FROM tasks WHERE id = $1", id)
.execute(self.db.as_ref())
.await?;
self.refresh_cache().await?;
Ok(result.rows_affected() > 0)
}
/// Get tasks for a specific user
pub async fn get_user_tasks(
&self,
user_id: &str,
) -> Result<Vec<Task>, Box<dyn std::error::Error>> {
let results = sqlx::query!(
r#"
SELECT * FROM tasks
WHERE assignee = $1 OR reporter = $1
ORDER BY priority DESC, due_date ASC
"#,
user_id
)
.fetch_all(self.db.as_ref())
.await?;
Ok(results
.into_iter()
.map(|r| serde_json::from_value(serde_json::to_value(r).unwrap()).unwrap())
.collect())
}
/// Get tasks by status
pub async fn get_tasks_by_status(
&self,
status: TaskStatus,
) -> Result<Vec<Task>, Box<dyn std::error::Error>> {
let results = sqlx::query!(
r#"
SELECT * FROM tasks
WHERE status = $1
ORDER BY priority DESC, created_at ASC
"#,
serde_json::to_value(&status)?
)
.fetch_all(self.db.as_ref())
.await?;
Ok(results
.into_iter()
.map(|r| serde_json::from_value(serde_json::to_value(r).unwrap()).unwrap())
.collect())
}
/// Get overdue tasks
pub async fn get_overdue_tasks(&self) -> Result<Vec<Task>, Box<dyn std::error::Error>> {
let now = Utc::now();
let results = sqlx::query!(
r#"
SELECT * FROM tasks
WHERE due_date < $1 AND status != 'done' AND status != 'cancelled'
ORDER BY due_date ASC
"#,
now
)
.fetch_all(self.db.as_ref())
.await?;
Ok(results
.into_iter()
.map(|r| serde_json::from_value(serde_json::to_value(r).unwrap()).unwrap())
.collect())
}
/// Add a comment to a task
pub async fn add_comment(
&self,
task_id: Uuid,
author: &str,
content: &str,
) -> Result<TaskComment, Box<dyn std::error::Error>> {
let comment = TaskComment {
id: Uuid::new_v4(),
task_id,
author: author.to_string(),
content: content.to_string(),
created_at: Utc::now(),
updated_at: None,
};
sqlx::query!(
r#"
INSERT INTO task_comments (id, task_id, author, content, created_at)
VALUES ($1, $2, $3, $4, $5)
"#,
comment.id,
comment.task_id,
comment.author,
comment.content,
comment.created_at
)
.execute(self.db.as_ref())
.await?;
Ok(comment)
}
/// Create a subtask
pub async fn create_subtask(
&self,
parent_id: Uuid,
subtask: Task,
) -> Result<Task, Box<dyn std::error::Error>> {
let mut subtask = subtask;
subtask.parent_task_id = Some(parent_id);
let created = self.create_task(subtask).await?;
// Update parent's subtasks list
sqlx::query!(
r#"
UPDATE tasks
SET subtasks = array_append(subtasks, $1)
WHERE id = $2
"#,
created.id,
parent_id
)
.execute(self.db.as_ref())
.await?;
Ok(created)
}
/// Get task dependencies
pub async fn get_task_dependencies(
&self,
task_id: Uuid,
) -> Result<Vec<Task>, Box<dyn std::error::Error>> {
let task = self.get_task(task_id).await?;
let mut dependencies = Vec::new();
for dep_id in task.dependencies {
if let Ok(dep_task) = self.get_task(dep_id).await {
dependencies.push(dep_task);
}
}
Ok(dependencies)
}
/// Get a single task by ID
pub async fn get_task(&self, id: Uuid) -> Result<Task, Box<dyn std::error::Error>> {
let result = sqlx::query!("SELECT * FROM tasks WHERE id = $1", id)
.fetch_one(self.db.as_ref())
.await?;
Ok(serde_json::from_value(serde_json::to_value(result)?)?)
}
/// Calculate task progress (percentage)
pub async fn calculate_progress(&self, task_id: Uuid) -> Result<f32, Box<dyn std::error::Error>> {
let task = self.get_task(task_id).await?;
if task.subtasks.is_empty() {
// No subtasks, progress based on status
return Ok(match task.status {
TaskStatus::Todo => 0.0,
TaskStatus::InProgress => 50.0,
TaskStatus::Review => 75.0,
TaskStatus::Done => 100.0,
TaskStatus::Blocked => task.actual_hours.unwrap_or(0.0) / task.estimated_hours.unwrap_or(1.0) * 100.0,
TaskStatus::Cancelled => 0.0,
});
}
// Has subtasks, calculate based on subtask completion
let total = task.subtasks.len() as f32;
let mut completed = 0.0;
for subtask_id in task.subtasks {
if let Ok(subtask) = self.get_task(subtask_id).await {
if matches!(subtask.status, TaskStatus::Done) {
completed += 1.0;
}
}
}
Ok((completed / total) * 100.0)
}
/// Create a task from template
pub async fn create_from_template(
&self,
template_id: Uuid,
assignee: Option<String>,
) -> Result<Task, Box<dyn std::error::Error>> {
let template = sqlx::query!(
"SELECT * FROM task_templates WHERE id = $1",
template_id
)
.fetch_one(self.db.as_ref())
.await?;
let template: TaskTemplate = serde_json::from_value(serde_json::to_value(template)?)?;
let task = Task {
id: Uuid::new_v4(),
title: template.name,
description: Some(template.description),
assignee: assignee.or(template.default_assignee),
reporter: "system".to_string(),
status: TaskStatus::Todo,
priority: template.default_priority,
due_date: None,
estimated_hours: None,
actual_hours: None,
tags: template.default_tags,
parent_task_id: None,
subtasks: Vec::new(),
dependencies: Vec::new(),
attachments: Vec::new(),
comments: Vec::new(),
created_at: Utc::now(),
updated_at: Utc::now(),
completed_at: None,
};
let created = self.create_task(task).await?;
// Create checklist items
for item in template.checklist {
let checklist_item = ChecklistItem {
id: Uuid::new_v4(),
task_id: created.id,
description: item.description,
completed: false,
completed_by: None,
completed_at: None,
};
sqlx::query!(
r#"
INSERT INTO task_checklists (id, task_id, description, completed)
VALUES ($1, $2, $3, $4)
"#,
checklist_item.id,
checklist_item.task_id,
checklist_item.description,
checklist_item.completed
)
.execute(self.db.as_ref())
.await?;
}
Ok(created)
}
/// Send notification to assignee
async fn notify_assignee(
&self,
assignee: &str,
task: &Task,
) -> Result<(), Box<dyn std::error::Error>> {
// This would integrate with your notification system
// For now, just log it
log::info!(
"Notifying {} about new task assignment: {}",
assignee,
task.title
);
Ok(())
}
/// Refresh the cache from database
async fn refresh_cache(&self) -> Result<(), Box<dyn std::error::Error>> {
let results = sqlx::query!("SELECT * FROM tasks ORDER BY created_at DESC")
.fetch_all(self.db.as_ref())
.await?;
let tasks: Vec<Task> = results
.into_iter()
.map(|r| serde_json::from_value(serde_json::to_value(r).unwrap()).unwrap())
.collect();
let mut cache = self.cache.write().await;
*cache = tasks;
Ok(())
}
/// Get task statistics for reporting
pub async fn get_statistics(
&self,
user_id: Option<&str>,
) -> Result<serde_json::Value, Box<dyn std::error::Error>> {
let base_query = if let Some(uid) = user_id {
format!("WHERE assignee = '{}' OR reporter = '{}'", uid, uid)
} else {
String::new()
};
let stats = sqlx::query(&format!(
r#"
SELECT
COUNT(*) FILTER (WHERE status = 'todo') as todo_count,
COUNT(*) FILTER (WHERE status = 'inprogress') as in_progress_count,
COUNT(*) FILTER (WHERE status = 'done') as done_count,
COUNT(*) FILTER (WHERE status = 'blocked') as blocked_count,
COUNT(*) FILTER (WHERE due_date < NOW() AND status != 'done') as overdue_count,
AVG(EXTRACT(EPOCH FROM (completed_at - created_at))/3600) FILTER (WHERE completed_at IS NOT NULL) as avg_completion_hours
FROM tasks
{}
"#,
base_query
))
.fetch_one(self.db.as_ref())
.await?;
Ok(serde_json::to_value(stats)?)
}
}
/// HTTP API handlers
pub mod handlers {
use super::*;
pub async fn create_task_handler(
engine: web::Data<TaskEngine>,
task: web::Json<Task>,
) -> Result<HttpResponse> {
match engine.create_task(task.into_inner()).await {
Ok(created) => Ok(HttpResponse::Ok().json(created)),
Err(e) => Ok(HttpResponse::InternalServerError().json(serde_json::json!({
"error": e.to_string()
}))),
}
}
pub async fn get_tasks_handler(
engine: web::Data<TaskEngine>,
query: web::Query<serde_json::Value>,
) -> Result<HttpResponse> {
if let Some(user_id) = query.get("user_id").and_then(|v| v.as_str()) {
match engine.get_user_tasks(user_id).await {
Ok(tasks) => Ok(HttpResponse::Ok().json(tasks)),
Err(e) => Ok(HttpResponse::InternalServerError().json(serde_json::json!({
"error": e.to_string()
}))),
}
} else if let Some(status) = query.get("status").and_then(|v| v.as_str()) {
let status = serde_json::from_value(serde_json::json!(status)).unwrap_or(TaskStatus::Todo);
match engine.get_tasks_by_status(status).await {
Ok(tasks) => Ok(HttpResponse::Ok().json(tasks)),
Err(e) => Ok(HttpResponse::InternalServerError().json(serde_json::json!({
"error": e.to_string()
}))),
}
} else {
Ok(HttpResponse::BadRequest().json(serde_json::json!({
"error": "Missing user_id or status parameter"
})))
}
}
pub async fn update_task_handler(
engine: web::Data<TaskEngine>,
path: web::Path<Uuid>,
updates: web::Json<serde_json::Value>,
) -> Result<HttpResponse> {
match engine.update_task(path.into_inner(), updates.into_inner()).await {
Ok(updated) => Ok(HttpResponse::Ok().json(updated)),
Err(e) => Ok(HttpResponse::InternalServerError().json(serde_json::json!({
"error": e.to_string()
}))),
}
}
pub async fn get_statistics_handler(
engine: web::Data<TaskEngine>,
query: web::Query<serde_json::Value>,
) -> Result<HttpResponse> {
let user_id = query.get("user_id").and_then(|v| v.as_str());
match engine.get_statistics(user_id).await {
Ok(stats) => Ok(HttpResponse::Ok().json(stats)),
Err(e) => Ok(HttpResponse::InternalServerError().json(serde_json::json!({
"error": e.to_string()
}))),
}
}
}
/// Configure task engine routes
pub fn configure(cfg: &mut web::ServiceConfig) {
cfg.service(
web::scope("/tasks")
.route("", web::post().to(handlers::create_task_handler))
.route("", web::get().to(handlers::get_tasks_handler))
.route("/{id}", web::put().to(handlers::update_task_handler))
.route("/statistics", web::get().to(handlers::get_statistics_handler)),
);
}

338
templates/crm.gbai/crm.gbdialog/case-management.bas Normal file
View file

@ -0,0 +1,338 @@
PARAM action AS STRING
PARAM case_data AS OBJECT
user_id = GET "session.user_id"
case_id = GET "session.case_id"
contact_id = GET "session.contact_id"
current_time = FORMAT NOW() AS "YYYY-MM-DD HH:mm:ss"
IF action = "create" THEN
subject = GET "case_data.subject"
description = GET "case_data.description"
priority = GET "case_data.priority"
IF subject = "" THEN
TALK "What is the issue you're experiencing?"
subject = HEAR
END IF
IF description = "" THEN
TALK "Please describe the issue in detail:"
description = HEAR
END IF
IF priority = "" THEN
TALK "How urgent is this? (low/medium/high/critical)"
priority = HEAR
END IF
case_number = "CS-" + FORMAT NOW() AS "YYYYMMDD" + "-" + FORMAT RANDOM(1000, 9999)
new_case = CREATE OBJECT
SET new_case.id = FORMAT GUID()
SET new_case.case_number = case_number
SET new_case.subject = subject
SET new_case.description = description
SET new_case.status = "new"
SET new_case.priority = priority
SET new_case.contact_id = contact_id
SET new_case.created_at = current_time
SET new_case.assigned_to = user_id
SAVE_FROM_UNSTRUCTURED "cases", FORMAT new_case AS JSON
SET "session.case_id" = new_case.id
REMEMBER "case_" + new_case.id = new_case
TALK "Case " + case_number + " created successfully."
IF priority = "critical" OR priority = "high" THEN
notification = "URGENT: New " + priority + " priority case: " + case_number + " - " + subject
SEND MAIL "support-manager@company.com", "Urgent Case", notification
CREATE_TASK "Resolve case " + case_number + " immediately", "critical", user_id
ELSE
CREATE_TASK "Review case " + case_number, priority, user_id
END IF
activity = CREATE OBJECT
SET activity.type = "case_created"
SET activity.case_id = new_case.id
SET activity.description = "Case created: " + subject
SET activity.created_at = current_time
SAVE_FROM_UNSTRUCTURED "activities", FORMAT activity AS JSON
END IF
IF action = "update_status" THEN
IF case_id = "" THEN
TALK "Enter case number:"
case_number = HEAR
case = FIND "cases", "case_number = '" + case_number + "'"
IF case != NULL THEN
case_id = case.id
ELSE
TALK "Case not found."
EXIT
END IF
END IF
case = FIND "cases", "id = '" + case_id + "'"
IF case = NULL THEN
TALK "Case not found."
EXIT
END IF
TALK "Current status: " + case.status
TALK "Select new status:"
TALK "1. New"
TALK "2. In Progress"
TALK "3. Waiting on Customer"
TALK "4. Waiting on Vendor"
TALK "5. Escalated"
TALK "6. Resolved"
TALK "7. Closed"
status_choice = HEAR
new_status = ""
IF status_choice = "1" THEN
new_status = "new"
ELSE IF status_choice = "2" THEN
new_status = "in_progress"
ELSE IF status_choice = "3" THEN
new_status = "waiting_customer"
ELSE IF status_choice = "4" THEN
new_status = "waiting_vendor"
ELSE IF status_choice = "5" THEN
new_status = "escalated"
ELSE IF status_choice = "6" THEN
new_status = "resolved"
ELSE IF status_choice = "7" THEN
new_status = "closed"
END IF
old_status = case.status
case.status = new_status
case.updated_at = current_time
IF new_status = "resolved" OR new_status = "closed" THEN
case.resolved_at = current_time
TALK "Please provide resolution details:"
resolution = HEAR
case.resolution = resolution
END IF
IF new_status = "escalated" THEN
TALK "Reason for escalation:"
escalation_reason = HEAR
case.escalation_reason = escalation_reason
notification = "Case Escalated: " + case.case_number + " - " + case.subject + "\nReason: " + escalation_reason
SEND MAIL "support-manager@company.com", "Case Escalation", notification
END IF
SAVE_FROM_UNSTRUCTURED "cases", FORMAT case AS JSON
activity = CREATE OBJECT
SET activity.type = "status_change"
SET activity.case_id = case_id
SET activity.description = "Status changed from " + old_status + " to " + new_status
SET activity.created_at = current_time
SAVE_FROM_UNSTRUCTURED "activities", FORMAT activity AS JSON
TALK "Case status updated to " + new_status
IF new_status = "resolved" THEN
contact = FIND "contacts", "id = '" + case.contact_id + "'"
IF contact != NULL AND contact.email != "" THEN
subject = "Case " + case.case_number + " Resolved"
message = "Your case has been resolved.\n\nResolution: " + resolution + "\n\nThank you for your patience."
SEND MAIL contact.email, subject, message
END IF
END IF
END IF
IF action = "add_note" THEN
IF case_id = "" THEN
TALK "Enter case number:"
case_number = HEAR
case = FIND "cases", "case_number = '" + case_number + "'"
IF case != NULL THEN
case_id = case.id
ELSE
TALK "Case not found."
EXIT
END IF
END IF
TALK "Enter your note:"
note_text = HEAR
note = CREATE OBJECT
SET note.id = FORMAT GUID()
SET note.entity_type = "case"
SET note.entity_id = case_id
SET note.body = note_text
SET note.created_by = user_id
SET note.created_at = current_time
SAVE_FROM_UNSTRUCTURED "notes", FORMAT note AS JSON
TALK "Note added to case."
END IF
IF action = "search" THEN
TALK "Search by:"
TALK "1. Case Number"
TALK "2. Subject"
TALK "3. Contact Email"
TALK "4. Status"
search_type = HEAR
IF search_type = "1" THEN
TALK "Enter case number:"
search_term = HEAR
cases = FIND "cases", "case_number = '" + search_term + "'"
ELSE IF search_type = "2" THEN
TALK "Enter subject keywords:"
search_term = HEAR
cases = FIND "cases", "subject LIKE '%" + search_term + "%'"
ELSE IF search_type = "3" THEN
TALK "Enter contact email:"
search_term = HEAR
contact = FIND "contacts", "email = '" + search_term + "'"
IF contact != NULL THEN
cases = FIND "cases", "contact_id = '" + contact.id + "'"
END IF
ELSE IF search_type = "4" THEN
TALK "Enter status (new/in_progress/resolved/closed):"
search_term = HEAR
cases = FIND "cases", "status = '" + search_term + "'"
END IF
IF cases = NULL THEN
TALK "No cases found."
ELSE
TALK "Found cases:"
FOR EACH case IN cases DO
TALK case.case_number + " - " + case.subject + " (" + case.status + ")"
END FOR
END IF
END IF
IF action = "sla_check" THEN
cases = FIND "cases", "status != 'closed' AND status != 'resolved'"
breached_count = 0
warning_count = 0
FOR EACH case IN cases DO
hours_open = HOURS_BETWEEN(case.created_at, current_time)
sla_hours = 24
IF case.priority = "critical" THEN
sla_hours = 2
ELSE IF case.priority = "high" THEN
sla_hours = 4
ELSE IF case.priority = "medium" THEN
sla_hours = 8
END IF
IF hours_open > sla_hours THEN
breached_count = breached_count + 1
notification = "SLA BREACH: Case " + case.case_number + " - Open for " + hours_open + " hours"
SEND MAIL "support-manager@company.com", "SLA Breach Alert", notification
case.sla_breached = true
SAVE_FROM_UNSTRUCTURED "cases", FORMAT case AS JSON
ELSE IF hours_open > sla_hours * 0.8 THEN
warning_count = warning_count + 1
END IF
END FOR
TALK "SLA Status:"
TALK "Breached: " + breached_count + " cases"
TALK "Warning: " + warning_count + " cases"
IF breached_count > 0 THEN
CREATE_TASK "Review SLA breached cases immediately", "critical", user_id
END IF
END IF
IF action = "daily_report" THEN
new_cases = FIND "cases", "DATE(created_at) = DATE('" + current_time + "')"
resolved_cases = FIND "cases", "DATE(resolved_at) = DATE('" + current_time + "')"
open_cases = FIND "cases", "status != 'closed' AND status != 'resolved'"
new_count = 0
resolved_count = 0
open_count = 0
FOR EACH case IN new_cases DO
new_count = new_count + 1
END FOR
FOR EACH case IN resolved_cases DO
resolved_count = resolved_count + 1
END FOR
FOR EACH case IN open_cases DO
open_count = open_count + 1
END FOR
report = "DAILY CASE REPORT - " + current_time + "\n"
report = report + "================================\n"
report = report + "New Cases Today: " + new_count + "\n"
report = report + "Resolved Today: " + resolved_count + "\n"
report = report + "Currently Open: " + open_count + "\n\n"
report = report + "Open Cases by Priority:\n"
critical_cases = FIND "cases", "status != 'closed' AND status != 'resolved' AND priority = 'critical'"
high_cases = FIND "cases", "status != 'closed' AND status != 'resolved' AND priority = 'high'"
medium_cases = FIND "cases", "status != 'closed' AND status != 'resolved' AND priority = 'medium'"
low_cases = FIND "cases", "status != 'closed' AND status != 'resolved' AND priority = 'low'"
critical_count = 0
high_count = 0
medium_count = 0
low_count = 0
FOR EACH case IN critical_cases DO
critical_count = critical_count + 1
END FOR
FOR EACH case IN high_cases DO
high_count = high_count + 1
END FOR
FOR EACH case IN medium_cases DO
medium_count = medium_count + 1
END FOR
FOR EACH case IN low_cases DO
low_count = low_count + 1
END FOR
report = report + "Critical: " + critical_count + "\n"
report = report + "High: " + high_count + "\n"
report = report + "Medium: " + medium_count + "\n"
report = report + "Low: " + low_count + "\n"
SEND MAIL "support-manager@company.com", "Daily Case Report", report
TALK "Daily report sent to management."
END IF

231
templates/crm.gbai/crm.gbdialog/crm-jobs.bas Normal file
View file

@ -0,0 +1,231 @@
PARAM job_name AS STRING
user_id = GET "session.user_id"
current_time = FORMAT NOW() AS "YYYY-MM-DD HH:mm:ss"
IF job_name = "lead_scoring" THEN
leads = FIND "leads", "status != 'converted' AND status != 'unqualified'"
FOR EACH lead IN leads DO
score = 0
days_old = DAYS_BETWEEN(lead.created_at, current_time)
IF days_old < 7 THEN
score = score + 10
ELSE IF days_old < 30 THEN
score = score + 5
END IF
activities = FIND "activities", "lead_id = '" + lead.id + "'"
activity_count = 0
FOR EACH activity IN activities DO
activity_count = activity_count + 1
END FOR
IF activity_count > 10 THEN
score = score + 20
ELSE IF activity_count > 5 THEN
score = score + 10
ELSE IF activity_count > 0 THEN
score = score + 5
END IF
IF lead.email != "" THEN
score = score + 5
END IF
IF lead.phone != "" THEN
score = score + 5
END IF
IF lead.company_name != "" THEN
score = score + 10
END IF
lead.score = score
IF score > 50 THEN
lead.status = "hot"
ELSE IF score > 30 THEN
lead.status = "warm"
ELSE IF score > 10 THEN
lead.status = "cold"
END IF
SAVE_FROM_UNSTRUCTURED "leads", FORMAT lead AS JSON
END FOR
TALK "Lead scoring completed for " + activity_count + " leads"
END IF
IF job_name = "opportunity_reminder" THEN
opportunities = FIND "opportunities", "stage != 'closed_won' AND stage != 'closed_lost'"
FOR EACH opp IN opportunities DO
days_until_close = DAYS_BETWEEN(current_time, opp.close_date)
IF days_until_close = 7 THEN
notification = "Opportunity " + opp.name + " closes in 7 days"
SEND MAIL opp.owner_id, "Opportunity Reminder", notification
CREATE_TASK "Follow up on " + opp.name, "high", opp.owner_id
ELSE IF days_until_close = 1 THEN
notification = "URGENT: Opportunity " + opp.name + " closes tomorrow!"
SEND MAIL opp.owner_id, "Urgent Opportunity Alert", notification
CREATE_TASK "Close deal: " + opp.name, "critical", opp.owner_id
ELSE IF days_until_close < 0 THEN
opp.stage = "closed_lost"
opp.closed_at = current_time
opp.loss_reason = "Expired - no action taken"
SAVE_FROM_UNSTRUCTURED "opportunities", FORMAT opp AS JSON
END IF
END FOR
END IF
IF job_name = "case_escalation" THEN
cases = FIND "cases", "status = 'new' OR status = 'in_progress'"
FOR EACH case IN cases DO
hours_open = HOURS_BETWEEN(case.created_at, current_time)
escalate = false
IF case.priority = "critical" AND hours_open > 2 THEN
escalate = true
ELSE IF case.priority = "high" AND hours_open > 4 THEN
escalate = true
ELSE IF case.priority = "medium" AND hours_open > 8 THEN
escalate = true
ELSE IF case.priority = "low" AND hours_open > 24 THEN
escalate = true
END IF
IF escalate = true AND case.status != "escalated" THEN
case.status = "escalated"
case.escalated_at = current_time
SAVE_FROM_UNSTRUCTURED "cases", FORMAT case AS JSON
notification = "ESCALATION: Case " + case.case_number + " - " + case.subject
SEND MAIL "support-manager@company.com", "Case Escalation", notification
CREATE_TASK "Handle escalated case " + case.case_number, "critical", "support-manager"
END IF
END FOR
END IF
IF job_name = "email_campaign" THEN
leads = FIND "leads", "status = 'warm'"
FOR EACH lead IN leads DO
last_contact = GET "lead_last_contact_" + lead.id
IF last_contact = "" THEN
last_contact = lead.created_at
END IF
days_since_contact = DAYS_BETWEEN(last_contact, current_time)
IF days_since_contact = 3 THEN
subject = "Following up on your interest"
message = "Hi " + lead.contact_name + ",\n\nI wanted to follow up on your recent inquiry..."
SEND MAIL lead.email, subject, message
REMEMBER "lead_last_contact_" + lead.id = current_time
ELSE IF days_since_contact = 7 THEN
subject = "Special offer for you"
message = "Hi " + lead.contact_name + ",\n\nWe have a special offer..."
SEND MAIL lead.email, subject, message
REMEMBER "lead_last_contact_" + lead.id = current_time
ELSE IF days_since_contact = 14 THEN
subject = "Last chance - Limited time offer"
message = "Hi " + lead.contact_name + ",\n\nThis is your last chance..."
SEND MAIL lead.email, subject, message
REMEMBER "lead_last_contact_" + lead.id = current_time
ELSE IF days_since_contact > 30 THEN
lead.status = "cold"
SAVE_FROM_UNSTRUCTURED "leads", FORMAT lead AS JSON
END IF
END FOR
END IF
IF job_name = "activity_cleanup" THEN
old_date = FORMAT ADD_DAYS(NOW(), -90) AS "YYYY-MM-DD"
activities = FIND "activities", "created_at < '" + old_date + "' AND status = 'completed'"
archive_count = 0
FOR EACH activity IN activities DO
archive = CREATE OBJECT
SET archive.original_id = activity.id
SET archive.data = FORMAT activity AS JSON
SET archive.archived_at = current_time
SAVE_FROM_UNSTRUCTURED "activities_archive", FORMAT archive AS JSON
archive_count = archive_count + 1
END FOR
TALK "Archived " + archive_count + " old activities"
END IF
IF job_name = "daily_digest" THEN
new_leads = FIND "leads", "DATE(created_at) = DATE('" + current_time + "')"
new_opportunities = FIND "opportunities", "DATE(created_at) = DATE('" + current_time + "')"
closed_won = FIND "opportunities", "DATE(closed_at) = DATE('" + current_time + "') AND won = true"
new_cases = FIND "cases", "DATE(created_at) = DATE('" + current_time + "')"
lead_count = 0
opp_count = 0
won_count = 0
won_amount = 0
case_count = 0
FOR EACH lead IN new_leads DO
lead_count = lead_count + 1
END FOR
FOR EACH opp IN new_opportunities DO
opp_count = opp_count + 1
END FOR
FOR EACH deal IN closed_won DO
won_count = won_count + 1
won_amount = won_amount + deal.amount
END FOR
FOR EACH case IN new_cases DO
case_count = case_count + 1
END FOR
digest = "DAILY CRM DIGEST - " + current_time + "\n"
digest = digest + "=====================================\n\n"
digest = digest + "NEW ACTIVITY TODAY:\n"
digest = digest + "- New Leads: " + lead_count + "\n"
digest = digest + "- New Opportunities: " + opp_count + "\n"
digest = digest + "- Deals Won: " + won_count + " ($" + won_amount + ")\n"
digest = digest + "- Support Cases: " + case_count + "\n\n"
digest = digest + "PIPELINE STATUS:\n"
open_opps = FIND "opportunities", "stage != 'closed_won' AND stage != 'closed_lost'"
total_pipeline = 0
FOR EACH opp IN open_opps DO
total_pipeline = total_pipeline + opp.amount
END FOR
digest = digest + "- Total Pipeline Value: $" + total_pipeline + "\n"
SEND MAIL "management@company.com", "Daily CRM Digest", digest
TALK "Daily digest sent to management"
END IF
IF job_name = "setup_schedules" THEN
SET SCHEDULE "0 9 * * *" "crm-jobs.bas" "lead_scoring"
SET SCHEDULE "0 10 * * *" "crm-jobs.bas" "opportunity_reminder"
SET SCHEDULE "*/30 * * * *" "crm-jobs.bas" "case_escalation"
SET SCHEDULE "0 14 * * *" "crm-jobs.bas" "email_campaign"
SET SCHEDULE "0 2 * * 0" "crm-jobs.bas" "activity_cleanup"
SET SCHEDULE "0 18 * * *" "crm-jobs.bas" "daily_digest"
TALK "All CRM schedules have been configured"
END IF

293
templates/crm.gbai/crm.gbdialog/lead-management.bas Normal file
View file

@ -0,0 +1,293 @@
PARAM action AS STRING
PARAM lead_data AS OBJECT
lead_id = GET "session.lead_id"
user_id = GET "session.user_id"
current_time = FORMAT NOW() AS "YYYY-MM-DD HH:mm:ss"
IF action = "capture" THEN
lead_name = GET "lead_data.name"
lead_email = GET "lead_data.email"
lead_phone = GET "lead_data.phone"
lead_company = GET "lead_data.company"
lead_source = GET "lead_data.source"
IF lead_email = "" THEN
TALK "I need your email to continue."
lead_email = HEAR
END IF
IF lead_name = "" THEN
TALK "May I have your name?"
lead_name = HEAR
END IF
new_lead = CREATE OBJECT
SET new_lead.id = FORMAT GUID()
SET new_lead.name = lead_name
SET new_lead.email = lead_email
SET new_lead.phone = lead_phone
SET new_lead.company = lead_company
SET new_lead.source = lead_source
SET new_lead.status = "new"
SET new_lead.score = 0
SET new_lead.created_at = current_time
SET new_lead.assigned_to = user_id
SAVE_FROM_UNSTRUCTURED "leads", FORMAT new_lead AS JSON
SET "session.lead_id" = new_lead.id
SET "session.lead_status" = "captured"
REMEMBER "lead_" + new_lead.id = new_lead
TALK "Thank you " + lead_name + "! I've captured your information."
END IF
IF action = "qualify" THEN
lead = FIND "leads", "id = '" + lead_id + "'"
IF lead = NULL THEN
TALK "No lead found to qualify."
EXIT
END IF
score = 0
TALK "I need to ask you a few questions to better assist you."
TALK "What is your company's annual revenue range?"
TALK "1. Under $1M"
TALK "2. $1M - $10M"
TALK "3. $10M - $50M"
TALK "4. Over $50M"
revenue_answer = HEAR
IF revenue_answer = "4" THEN
score = score + 30
ELSE IF revenue_answer = "3" THEN
score = score + 20
ELSE IF revenue_answer = "2" THEN
score = score + 10
ELSE
score = score + 5
END IF
TALK "How many employees does your company have?"
employees = HEAR
IF employees > 500 THEN
score = score + 25
ELSE IF employees > 100 THEN
score = score + 15
ELSE IF employees > 20 THEN
score = score + 10
ELSE
score = score + 5
END IF
TALK "What is your timeline for making a decision?"
TALK "1. This month"
TALK "2. This quarter"
TALK "3. This year"
TALK "4. Just researching"
timeline = HEAR
IF timeline = "1" THEN
score = score + 30
ELSE IF timeline = "2" THEN
score = score + 20
ELSE IF timeline = "3" THEN
score = score + 10
ELSE
score = score + 0
END IF
TALK "Do you have budget allocated for this?"
has_budget = HEAR
IF has_budget = "yes" OR has_budget = "YES" OR has_budget = "Yes" THEN
score = score + 25
ELSE
score = score + 5
END IF
lead_status = "unqualified"
IF score >= 70 THEN
lead_status = "hot"
ELSE IF score >= 50 THEN
lead_status = "warm"
ELSE IF score >= 30 THEN
lead_status = "cold"
END IF
update_lead = CREATE OBJECT
SET update_lead.score = score
SET update_lead.status = lead_status
SET update_lead.qualified_at = current_time
SET update_lead.revenue_range = revenue_answer
SET update_lead.employees = employees
SET update_lead.timeline = timeline
SET update_lead.has_budget = has_budget
SAVE_FROM_UNSTRUCTURED "leads", FORMAT update_lead AS JSON
REMEMBER "lead_score_" + lead_id = score
REMEMBER "lead_status_" + lead_id = lead_status
IF lead_status = "hot" THEN
TALK "Great! You're a perfect fit for our solution. Let me connect you with a specialist."
notification = "Hot lead alert: " + lead.name + " from " + lead.company + " - Score: " + score
SEND MAIL "sales@company.com", "Hot Lead Alert", notification
CREATE_TASK "Follow up with hot lead " + lead.name, "high", user_id
ELSE IF lead_status = "warm" THEN
TALK "Thank you! Based on your needs, I'll have someone reach out within 24 hours."
CREATE_TASK "Contact warm lead " + lead.name, "medium", user_id
ELSE
TALK "Thank you for your time. I'll send you some helpful resources via email."
END IF
END IF
IF action = "convert" THEN
lead = FIND "leads", "id = '" + lead_id + "'"
IF lead = NULL THEN
TALK "No lead found to convert."
EXIT
END IF
IF lead.status = "unqualified" OR lead.status = "cold" THEN
TALK "This lead needs to be qualified first."
EXIT
END IF
account = CREATE OBJECT
SET account.id = FORMAT GUID()
SET account.name = lead.company
SET account.type = "customer"
SET account.owner_id = user_id
SET account.created_from_lead = lead_id
SET account.created_at = current_time
SAVE_FROM_UNSTRUCTURED "accounts", FORMAT account AS JSON
contact = CREATE OBJECT
SET contact.id = FORMAT GUID()
SET contact.account_id = account.id
SET contact.name = lead.name
SET contact.email = lead.email
SET contact.phone = lead.phone
SET contact.primary_contact = true
SET contact.created_from_lead = lead_id
SET contact.created_at = current_time
SAVE_FROM_UNSTRUCTURED "contacts", FORMAT contact AS JSON
opportunity = CREATE OBJECT
SET opportunity.id = FORMAT GUID()
SET opportunity.name = "Opportunity for " + account.name
SET opportunity.account_id = account.id
SET opportunity.contact_id = contact.id
SET opportunity.stage = "qualification"
SET opportunity.probability = 20
SET opportunity.owner_id = user_id
SET opportunity.lead_source = lead.source
SET opportunity.created_at = current_time
SAVE_FROM_UNSTRUCTURED "opportunities", FORMAT opportunity AS JSON
update_lead = CREATE OBJECT
SET update_lead.status = "converted"
SET update_lead.converted_at = current_time
SET update_lead.converted_to_account_id = account.id
SAVE_FROM_UNSTRUCTURED "leads", FORMAT update_lead AS JSON
REMEMBER "account_" + account.id = account
REMEMBER "contact_" + contact.id = contact
REMEMBER "opportunity_" + opportunity.id = opportunity
SET "session.account_id" = account.id
SET "session.contact_id" = contact.id
SET "session.opportunity_id" = opportunity.id
TALK "Successfully converted lead to account: " + account.name
notification = "Lead converted: " + lead.name + " to account " + account.name
SEND MAIL user_id, "Lead Conversion", notification
CREATE_TASK "Initial meeting with " + contact.name, "high", user_id
END IF
IF action = "follow_up" THEN
lead = FIND "leads", "id = '" + lead_id + "'"
IF lead = NULL THEN
TALK "No lead found."
EXIT
END IF
last_contact = GET "lead_last_contact_" + lead_id
days_since = 0
IF last_contact != "" THEN
days_since = DAYS_BETWEEN(last_contact, current_time)
END IF
IF days_since > 7 OR last_contact = "" THEN
subject = "Following up on your inquiry"
message = "Hi " + lead.name + ",\n\nI wanted to follow up on your recent inquiry about our services."
SEND MAIL lead.email, subject, message
activity = CREATE OBJECT
SET activity.id = FORMAT GUID()
SET activity.type = "email"
SET activity.subject = subject
SET activity.lead_id = lead_id
SET activity.created_at = current_time
SAVE_FROM_UNSTRUCTURED "activities", FORMAT activity AS JSON
REMEMBER "lead_last_contact_" + lead_id = current_time
TALK "Follow-up email sent to " + lead.name
ELSE
TALK "Lead was contacted " + days_since + " days ago. Too soon for follow-up."
END IF
END IF
IF action = "nurture" THEN
leads = FIND "leads", "status = 'warm' OR status = 'cold'"
FOR EACH lead IN leads DO
days_old = DAYS_BETWEEN(lead.created_at, current_time)
IF days_old = 3 THEN
content = "5 Tips to Improve Your Business"
ELSE IF days_old = 7 THEN
content = "Case Study: How We Helped Similar Companies"
ELSE IF days_old = 14 THEN
content = "Free Consultation Offer"
ELSE IF days_old = 30 THEN
content = "Special Limited Time Offer"
ELSE
CONTINUE
END IF
SEND MAIL lead.email, content, "Nurture content for day " + days_old
REMEMBER "lead_nurture_" + lead.id + "_day_" + days_old = "sent"
END FOR
TALK "Nurture campaign processed"
END IF

321
templates/crm.gbai/crm.gbdialog/new_email.bas Normal file
View file

@ -0,0 +1,321 @@
' New Email Event Handler
' This script is triggered when a new email is received by the CRM system
' It handles email parsing, sender identification, automatic routing, and case creation
PARAM email_id AS STRING
PARAM from_address AS STRING
PARAM to_addresses AS ARRAY
PARAM cc_addresses AS ARRAY
PARAM subject AS STRING
PARAM body_text AS STRING
PARAM body_html AS STRING
PARAM attachments AS ARRAY
PARAM headers AS OBJECT
PARAM received_at AS DATETIME
' Initialize email context
email_context = {}
email_context.email_id = email_id
email_context.from = from_address
email_context.to = to_addresses
email_context.subject = subject
email_context.received_at = received_at
' Clean email address for lookup
clean_email = LOWERCASE(TRIM(from_address))
IF clean_email CONTAINS "<" THEN
clean_email = EXTRACT_BETWEEN(clean_email, "<", ">")
END IF
' Look up sender in CRM
contact = FIND "contacts", "email", clean_email
lead = NULL
account = NULL
IF contact IS NULL THEN
' Check if sender is a lead
lead = FIND "leads", "email", clean_email
IF lead IS NULL THEN
' Create new lead from email
lead = {}
lead.email = clean_email
lead.lead_source = "email"
lead.lead_status = "new"
lead.notes = "Auto-created from email: " + subject
' Try to extract name from email
IF from_address CONTAINS "<" THEN
display_name = TRIM(EXTRACT_BEFORE(from_address, "<"))
IF display_name != "" THEN
lead.contact_name = display_name
END IF
END IF
' Extract company domain
domain = EXTRACT_AFTER(clean_email, "@")
IF domain != "" AND NOT IS_PERSONAL_EMAIL(domain) THEN
lead.company_name = CAPITALIZE(EXTRACT_BEFORE(domain, "."))
lead.website = "https://" + domain
END IF
lead_id = SAVE "leads", lead
email_context.lead_id = lead_id
email_context.is_new_lead = TRUE
ELSE
email_context.lead_id = lead.id
END IF
ELSE
' Existing contact found
email_context.contact_id = contact.id
email_context.account_id = contact.account_id
IF contact.account_id IS NOT NULL THEN
account = FIND "accounts", "id", contact.account_id
email_context.account = account
END IF
END IF
' Check for email thread/conversation
thread_id = NULL
IF headers.references IS NOT NULL THEN
' Email is part of a thread
thread_references = SPLIT(headers.references, " ")
FOR ref IN thread_references DO
existing_email = FIND "email_tracking", "message_id", ref
IF existing_email IS NOT NULL THEN
thread_id = existing_email.thread_id OR existing_email.id
BREAK
END IF
END FOR
END IF
' Analyze email content
sentiment = ANALYZE_SENTIMENT(body_text)
urgency = DETECT_URGENCY(subject + " " + body_text)
intent = CLASSIFY_INTENT(body_text)
' Determine email category
category = "general"
IF subject CONTAINS "support" OR subject CONTAINS "help" OR subject CONTAINS "issue" OR subject CONTAINS "problem" THEN
category = "support"
ELSE IF subject CONTAINS "quote" OR subject CONTAINS "pricing" OR subject CONTAINS "cost" THEN
category = "sales"
ELSE IF subject CONTAINS "invoice" OR subject CONTAINS "payment" OR subject CONTAINS "billing" THEN
category = "billing"
ELSE IF subject CONTAINS "complaint" OR sentiment = "negative" THEN
category = "complaint"
END IF
' Check for existing open case with this email
existing_case = NULL
IF contact IS NOT NULL THEN
existing_case = FIND "cases" WHERE contact_id = contact.id AND status != "closed" ORDER BY created_at DESC LIMIT 1
ELSE IF lead IS NOT NULL THEN
' Check for case linked to lead's email in case description
existing_case = FIND "cases" WHERE description CONTAINS clean_email AND status != "closed" ORDER BY created_at DESC LIMIT 1
END IF
' Determine priority
priority = "medium"
IF urgency = "high" OR subject CONTAINS "urgent" OR subject CONTAINS "asap" THEN
priority = "high"
ELSE IF account IS NOT NULL AND (account.type = "vip" OR account.type = "enterprise") THEN
priority = "high"
ELSE IF sentiment = "negative" AND category = "complaint" THEN
priority = "high"
ELSE IF category = "billing" THEN
priority = "medium"
ELSE
priority = "low"
END IF
' Create or update case
IF existing_case IS NOT NULL THEN
' Add to existing case
case_update = {}
case_update.status = "updated"
case_update.updated_at = NOW()
IF priority = "high" AND existing_case.priority != "high" THEN
case_update.priority = "high"
END IF
UPDATE "cases", existing_case.id, case_update
' Add note to case
note = {}
note.entity_type = "case"
note.entity_id = existing_case.id
note.title = "Email received: " + subject
note.body = "From: " + from_address + "\n\n" + body_text
note.created_by = "email_system"
SAVE "notes", note
email_context.case_id = existing_case.id
email_context.case_action = "updated"
ELSE IF category = "support" OR category = "complaint" THEN
' Create new case
new_case = {}
new_case.subject = subject
new_case.description = body_text
new_case.status = "new"
new_case.priority = priority
new_case.origin = "email"
new_case.type = category
IF contact IS NOT NULL THEN
new_case.contact_id = contact.id
new_case.account_id = contact.account_id
END IF
' Auto-assign based on rules
assigned_to = NULL
IF category = "complaint" THEN
assigned_to = GET "config", "complaint_handler"
ELSE IF account IS NOT NULL AND account.owner_id IS NOT NULL THEN
assigned_to = account.owner_id
ELSE
' Round-robin assignment
assigned_to = GET_NEXT_AVAILABLE_AGENT()
END IF
new_case.assigned_to = assigned_to
new_case.case_number = GENERATE_CASE_NUMBER()
case_id = SAVE "cases", new_case
email_context.case_id = case_id
email_context.case_action = "created"
' Send notification to assigned agent
IF assigned_to IS NOT NULL THEN
NOTIFY AGENT assigned_to WITH "New case #" + new_case.case_number + " assigned: " + subject
END IF
END IF
' Save email tracking record
email_record = {}
email_record.message_id = email_id
email_record.from_address = from_address
email_record.to_addresses = to_addresses
email_record.cc_addresses = cc_addresses
email_record.subject = subject
email_record.body = body_text
email_record.html_body = body_html
IF email_context.contact_id IS NOT NULL THEN
email_record.contact_id = email_context.contact_id
END IF
IF email_context.lead_id IS NOT NULL THEN
email_record.lead_id = email_context.lead_id
END IF
IF email_context.account_id IS NOT NULL THEN
email_record.account_id = email_context.account_id
END IF
IF email_context.case_id IS NOT NULL THEN
email_record.case_id = email_context.case_id
END IF
email_record.sent_at = received_at
email_record.thread_id = thread_id
SAVE "email_tracking", email_record
' Create activity record
activity = {}
activity.type = "email_received"
activity.subject = "Email: " + subject
activity.description = body_text
activity.status = "completed"
activity.email_message_id = email_id
IF email_context.contact_id IS NOT NULL THEN
activity.contact_id = email_context.contact_id
END IF
IF email_context.lead_id IS NOT NULL THEN
activity.lead_id = email_context.lead_id
END IF
IF email_context.account_id IS NOT NULL THEN
activity.account_id = email_context.account_id
END IF
IF email_context.case_id IS NOT NULL THEN
activity.case_id = email_context.case_id
END IF
activity.assigned_to = assigned_to OR GET "config", "default_email_handler"
SAVE "activities", activity
' Handle attachments
IF attachments IS NOT NULL AND LENGTH(attachments) > 0 THEN
FOR attachment IN attachments DO
doc = {}
doc.name = attachment.filename
doc.file_path = attachment.path
doc.file_size = attachment.size
doc.mime_type = attachment.mime_type
doc.entity_type = "email"
doc.entity_id = email_record.id
doc.uploaded_by = "email_system"
SAVE "documents", doc
END FOR
END IF
' Auto-reply based on category and time
business_hours = GET "config", "business_hours"
current_hour = HOUR(NOW())
is_business_hours = current_hour >= business_hours.start AND current_hour <= business_hours.end
auto_reply = NULL
IF category = "support" AND email_context.case_action = "created" THEN
IF is_business_hours THEN
auto_reply = "Thank you for contacting support. Your case #" + new_case.case_number + " has been created and assigned to our team. We'll respond within 2 business hours."
ELSE
auto_reply = "Thank you for contacting support. Your case #" + new_case.case_number + " has been created. Our business hours are " + business_hours.start + " to " + business_hours.end + ". We'll respond as soon as possible."
END IF
ELSE IF category = "sales" THEN
auto_reply = "Thank you for your interest! A sales representative will contact you within 1 business day."
ELSE IF category = "complaint" THEN
auto_reply = "We've received your message and take your concerns seriously. A manager will contact you within 4 hours."
END IF
IF auto_reply IS NOT NULL AND NOT IS_AUTOREPLY(headers) THEN
SEND EMAIL TO from_address SUBJECT "RE: " + subject BODY auto_reply
END IF
' Update lead score if applicable
IF lead IS NOT NULL THEN
score_increase = 0
IF category = "sales" THEN
score_increase = 10
ELSE IF intent = "purchase_intent" THEN
score_increase = 15
ELSE
score_increase = 5
END IF
UPDATE "leads", lead.id, "score", lead.score + score_increase
' Check if lead should be converted
IF lead.score > 50 AND category = "sales" THEN
TRIGGER "lead_qualification", lead.id
END IF
END IF
' Log email processing
LOG "email_processed", {
"email_id": email_id,
"from": from_address,
"category": category,
"priority": priority,
"sentiment": sentiment,
"case_action": email_context.case_action,
"case_id": email_context.case_id,
"is_new_lead": email_context.is_new_lead,
"auto_replied": auto_reply IS NOT NULL,
"timestamp": NOW()
}
' Return processing result
RETURN email_context

126
templates/crm.gbai/crm.gbdialog/new_session.bas Normal file
View file

@ -0,0 +1,126 @@
' New Session Event Handler
' This script is triggered when a new session starts with the bot
' It handles initial setup, user identification, and welcome messages
PARAM session_id AS STRING
PARAM user_id AS STRING
PARAM channel AS STRING
PARAM metadata AS OBJECT
' Initialize session context
SET session_context = {}
SET session_context.id = session_id
SET session_context.user_id = user_id
SET session_context.channel = channel
SET session_context.start_time = NOW()
SET session_context.metadata = metadata
' Check if user exists in CRM
user = FIND "contacts", "email", user_id
IF user IS NULL THEN
user = FIND "contacts", "phone", user_id
END IF
' Create activity record for new session
activity = {}
activity.type = "session_start"
activity.subject = "New " + channel + " session initiated"
activity.description = "User connected via " + channel + " at " + NOW()
activity.status = "open"
activity.assigned_to = GET "config", "default_agent"
IF user IS NOT NULL THEN
' Existing user found
activity.contact_id = user.id
activity.account_id = user.account_id
' Get user's recent interactions
recent_activities = FIND ALL "activities" WHERE contact_id = user.id ORDER BY created_at DESC LIMIT 5
' Check for open cases
open_cases = FIND ALL "cases" WHERE contact_id = user.id AND status != "closed"
' Set personalized greeting
IF open_cases.count > 0 THEN
greeting = "Welcome back, " + user.first_name + "! I see you have an open support case. Would you like to continue with that?"
SET session_context.has_open_case = TRUE
SET session_context.case_id = open_cases[0].id
ELSE IF recent_activities.count > 0 AND DAYS_BETWEEN(recent_activities[0].created_at, NOW()) < 7 THEN
greeting = "Hi " + user.first_name + "! Good to see you again. How can I help you today?"
ELSE
greeting = "Welcome back, " + user.first_name + "! It's been a while. How can I assist you today?"
END IF
' Update contact's last interaction
UPDATE "contacts", user.id, "last_interaction", NOW()
ELSE
' New user - create lead
lead = {}
lead.lead_source = channel
lead.lead_status = "new"
lead.notes = "Auto-created from " + channel + " session"
' Try to extract contact info from metadata
IF metadata.email IS NOT NULL THEN
lead.email = metadata.email
END IF
IF metadata.phone IS NOT NULL THEN
lead.phone = metadata.phone
END IF
IF metadata.name IS NOT NULL THEN
lead.contact_name = metadata.name
END IF
' Save lead
lead_id = SAVE "leads", lead
activity.lead_id = lead_id
SET session_context.is_new_lead = TRUE
SET session_context.lead_id = lead_id
greeting = "Hello! Welcome to our service. I'm here to help you. May I have your name to better assist you?"
END IF
' Save activity
SAVE "activities", activity
' Store session context
CACHE SET "session:" + session_id, session_context, 3600
' Send greeting
SEND MESSAGE greeting
' Check business hours
business_hours = GET "config", "business_hours"
current_hour = HOUR(NOW())
IF current_hour < business_hours.start OR current_hour > business_hours.end THEN
SEND MESSAGE "Please note that our business hours are " + business_hours.start + " to " + business_hours.end + ". You can still leave a message and we'll get back to you as soon as possible."
END IF
' Set up session monitoring
SCHEDULE IN 300 SECONDS DO
' Check if session is still active after 5 minutes
IF IS_ACTIVE(session_id) THEN
' Session still active, check if user needs help
last_message_time = GET_LAST_MESSAGE_TIME(session_id)
IF SECONDS_BETWEEN(last_message_time, NOW()) > 180 THEN
SEND MESSAGE "I'm still here if you need any assistance. Just let me know how I can help!"
END IF
END IF
END SCHEDULE
' Log session start for analytics
LOG "session_start", {
"session_id": session_id,
"user_id": user_id,
"channel": channel,
"user_type": user IS NOT NULL ? "existing" : "new",
"timestamp": NOW()
}
' Return session context
RETURN session_context

199
templates/crm.gbai/crm.gbdialog/on_transfer.bas Normal file
View file

@ -0,0 +1,199 @@
' On Transfer Event Handler
' This script is triggered when a conversation is transferred between agents or bots
' It handles context preservation, handoff notifications, and transfer logging
PARAM session_id AS STRING
PARAM from_agent AS STRING
PARAM to_agent AS STRING
PARAM transfer_reason AS STRING
PARAM transfer_type AS STRING ' bot_to_human, human_to_bot, human_to_human
PARAM context AS OBJECT
' Get session context from cache
session_context = CACHE GET "session:" + session_id
IF session_context IS NULL THEN
session_context = {}
session_context.session_id = session_id
END IF
' Update session context with transfer info
session_context.last_transfer = NOW()
session_context.transfer_count = (session_context.transfer_count OR 0) + 1
session_context.current_agent = to_agent
session_context.transfer_history = session_context.transfer_history OR []
' Add to transfer history
transfer_record = {}
transfer_record.from = from_agent
transfer_record.to = to_agent
transfer_record.reason = transfer_reason
transfer_record.type = transfer_type
transfer_record.timestamp = NOW()
transfer_record.context_preserved = context
APPEND session_context.transfer_history, transfer_record
' Get user information
user = NULL
IF session_context.contact_id IS NOT NULL THEN
user = FIND "contacts", "id", session_context.contact_id
ELSE IF session_context.lead_id IS NOT NULL THEN
lead = FIND "leads", "id", session_context.lead_id
user = {"first_name": lead.contact_name, "email": lead.email}
END IF
' Create activity for transfer
activity = {}
activity.type = "transfer"
activity.subject = "Conversation transferred from " + from_agent + " to " + to_agent
activity.description = "Transfer reason: " + transfer_reason + "\nTransfer type: " + transfer_type
activity.status = "completed"
activity.assigned_to = to_agent
activity.created_by = from_agent
IF session_context.contact_id IS NOT NULL THEN
activity.contact_id = session_context.contact_id
END IF
IF session_context.lead_id IS NOT NULL THEN
activity.lead_id = session_context.lead_id
END IF
IF session_context.case_id IS NOT NULL THEN
activity.case_id = session_context.case_id
END IF
IF session_context.opportunity_id IS NOT NULL THEN
activity.opportunity_id = session_context.opportunity_id
END IF
SAVE "activities", activity
' Handle different transfer types
IF transfer_type = "bot_to_human" THEN
' Bot to Human handoff
SEND MESSAGE "I'm transferring you to " + to_agent + " who will be better able to assist you."
' Prepare summary for human agent
summary = "=== Transfer Summary ===\n"
summary = summary + "Customer: " + (user.first_name OR "Unknown") + "\n"
summary = summary + "Email: " + (user.email OR "Not provided") + "\n"
summary = summary + "Transfer Reason: " + transfer_reason + "\n"
' Add conversation history
IF context.conversation_history IS NOT NULL THEN
summary = summary + "\n=== Recent Conversation ===\n"
FOR message IN context.conversation_history LAST 10 DO
summary = summary + message.sender + ": " + message.text + "\n"
END FOR
END IF
' Add open issues
IF session_context.case_id IS NOT NULL THEN
case = FIND "cases", "id", session_context.case_id
summary = summary + "\n=== Open Case ===\n"
summary = summary + "Case #: " + case.case_number + "\n"
summary = summary + "Subject: " + case.subject + "\n"
summary = summary + "Priority: " + case.priority + "\n"
END IF
' Send summary to human agent
NOTIFY AGENT to_agent WITH summary
' Update case if exists
IF session_context.case_id IS NOT NULL THEN
UPDATE "cases", session_context.case_id, {
"assigned_to": to_agent,
"status": "in_progress",
"escalated_to": to_agent
}
END IF
ELSE IF transfer_type = "human_to_bot" THEN
' Human to Bot handoff
SEND MESSAGE "You've been transferred back to the automated assistant. How can I help you?"
' Reset bot context
session_context.bot_context = {}
session_context.bot_context.resumed_at = NOW()
session_context.bot_context.previous_human_agent = from_agent
ELSE IF transfer_type = "human_to_human" THEN
' Human to Human handoff
SEND MESSAGE to_agent + " will now assist you with your inquiry."
' Notify new agent
notification = "You've received a transfer from " + from_agent + "\n"
notification = notification + "Customer: " + (user.first_name OR "Unknown") + "\n"
notification = notification + "Reason: " + transfer_reason + "\n"
notification = notification + "Please review the conversation history."
NOTIFY AGENT to_agent WITH notification
' Update assignment in all related entities
IF session_context.case_id IS NOT NULL THEN
UPDATE "cases", session_context.case_id, "assigned_to", to_agent
END IF
IF session_context.opportunity_id IS NOT NULL THEN
UPDATE "opportunities", session_context.opportunity_id, "owner_id", to_agent
END IF
END IF
' Check if this is a VIP customer
IF user IS NOT NULL AND user.account_id IS NOT NULL THEN
account = FIND "accounts", "id", user.account_id
IF account.type = "vip" OR account.type = "enterprise" THEN
NOTIFY AGENT to_agent WITH "⚠️ VIP Customer Alert: " + account.name
' Add VIP handling
session_context.is_vip = TRUE
session_context.account_tier = account.type
END IF
END IF
' Update session cache
CACHE SET "session:" + session_id, session_context, 3600
' Set up quality check
IF transfer_type = "bot_to_human" THEN
SCHEDULE IN 600 SECONDS DO
' After 10 minutes, check satisfaction
IF IS_ACTIVE(session_id) THEN
satisfaction_check = {}
satisfaction_check.session_id = session_id
satisfaction_check.transfer_id = transfer_record.id
satisfaction_check.checked_at = NOW()
SEND MESSAGE "Quick question: Has " + to_agent + " been able to help you with your issue? (Yes/No)"
WAIT FOR RESPONSE AS response TIMEOUT 60
IF response IS NOT NULL THEN
satisfaction_check.response = response
SAVE "transfer_satisfaction", satisfaction_check
IF response CONTAINS "no" OR response CONTAINS "not" THEN
ESCALATE TO SUPERVISOR
END IF
END IF
END IF
END SCHEDULE
END IF
' Log transfer metrics
LOG "conversation_transfer", {
"session_id": session_id,
"from_agent": from_agent,
"to_agent": to_agent,
"transfer_type": transfer_type,
"transfer_reason": transfer_reason,
"customer_type": user IS NOT NULL ? "existing" : "new",
"transfer_number": session_context.transfer_count,
"timestamp": NOW()
}
' Send transfer confirmation
confirmation = {}
confirmation.success = TRUE
confirmation.message = "Transfer completed successfully"
confirmation.new_agent = to_agent
confirmation.session_context = session_context
RETURN confirmation

345
templates/crm.gbai/crm.gbdialog/opportunity-management.bas Normal file
View file

@ -0,0 +1,345 @@
PARAM action AS STRING
PARAM opp_data AS OBJECT
user_id = GET "session.user_id"
opportunity_id = GET "session.opportunity_id"
account_id = GET "session.account_id"
current_time = FORMAT NOW() AS "YYYY-MM-DD HH:mm:ss"
IF action = "create" THEN
opp_name = GET "opp_data.name"
opp_value = GET "opp_data.value"
close_date = GET "opp_data.close_date"
IF account_id = "" THEN
TALK "Which account is this opportunity for?"
account_name = HEAR
account = FIND "accounts", "name LIKE '%" + account_name + "%'"
IF account != NULL THEN
account_id = account.id
ELSE
TALK "Account not found. Please create the account first."
EXIT
END IF
END IF
IF opp_name = "" THEN
TALK "What should we call this opportunity?"
opp_name = HEAR
END IF
IF opp_value = "" THEN
TALK "What is the estimated value of this deal?"
opp_value = HEAR
END IF
IF close_date = "" THEN
TALK "When do you expect to close this deal? (YYYY-MM-DD)"
close_date = HEAR
END IF
opportunity = CREATE OBJECT
SET opportunity.id = FORMAT GUID()
SET opportunity.name = opp_name
SET opportunity.account_id = account_id
SET opportunity.amount = opp_value
SET opportunity.close_date = close_date
SET opportunity.stage = "qualification"
SET opportunity.probability = 10
SET opportunity.owner_id = user_id
SET opportunity.created_at = current_time
SAVE_FROM_UNSTRUCTURED "opportunities", FORMAT opportunity AS JSON
SET "session.opportunity_id" = opportunity.id
REMEMBER "opportunity_" + opportunity.id = opportunity
TALK "Opportunity created: " + opp_name + " valued at $" + opp_value
CREATE_TASK "Qualify opportunity: " + opp_name, "high", user_id
activity = CREATE OBJECT
SET activity.type = "opportunity_created"
SET activity.opportunity_id = opportunity.id
SET activity.description = "Created opportunity: " + opp_name
SET activity.created_at = current_time
SAVE_FROM_UNSTRUCTURED "activities", FORMAT activity AS JSON
END IF
IF action = "update_stage" THEN
IF opportunity_id = "" THEN
TALK "Which opportunity do you want to update?"
opp_name = HEAR
opportunity = FIND "opportunities", "name LIKE '%" + opp_name + "%'"
IF opportunity != NULL THEN
opportunity_id = opportunity.id
ELSE
TALK "Opportunity not found."
EXIT
END IF
END IF
opportunity = FIND "opportunities", "id = '" + opportunity_id + "'"
IF opportunity = NULL THEN
TALK "Opportunity not found."
EXIT
END IF
TALK "Current stage: " + opportunity.stage
TALK "Select new stage:"
TALK "1. Qualification (10%)"
TALK "2. Needs Analysis (20%)"
TALK "3. Value Proposition (50%)"
TALK "4. Decision Makers (60%)"
TALK "5. Proposal (75%)"
TALK "6. Negotiation (90%)"
TALK "7. Closed Won (100%)"
TALK "8. Closed Lost (0%)"
stage_choice = HEAR
new_stage = ""
new_probability = 0
IF stage_choice = "1" THEN
new_stage = "qualification"
new_probability = 10
ELSE IF stage_choice = "2" THEN
new_stage = "needs_analysis"
new_probability = 20
ELSE IF stage_choice = "3" THEN
new_stage = "value_proposition"
new_probability = 50
ELSE IF stage_choice = "4" THEN
new_stage = "decision_makers"
new_probability = 60
ELSE IF stage_choice = "5" THEN
new_stage = "proposal"
new_probability = 75
ELSE IF stage_choice = "6" THEN
new_stage = "negotiation"
new_probability = 90
ELSE IF stage_choice = "7" THEN
new_stage = "closed_won"
new_probability = 100
opportunity.won = true
opportunity.closed_at = current_time
ELSE IF stage_choice = "8" THEN
new_stage = "closed_lost"
new_probability = 0
opportunity.won = false
opportunity.closed_at = current_time
END IF
old_stage = opportunity.stage
opportunity.stage = new_stage
opportunity.probability = new_probability
opportunity.updated_at = current_time
SAVE_FROM_UNSTRUCTURED "opportunities", FORMAT opportunity AS JSON
REMEMBER "opportunity_stage_" + opportunity_id = new_stage
activity = CREATE OBJECT
SET activity.type = "stage_change"
SET activity.opportunity_id = opportunity_id
SET activity.description = "Stage changed from " + old_stage + " to " + new_stage
SET activity.created_at = current_time
SAVE_FROM_UNSTRUCTURED "activities", FORMAT activity AS JSON
TALK "Stage updated to " + new_stage + " (" + new_probability + "%)"
IF new_stage = "closed_won" THEN
TALK "Congratulations! Deal closed for $" + opportunity.amount
notification = "Deal Won: " + opportunity.name + " - $" + opportunity.amount
SEND MAIL "management@company.com", "Deal Won", notification
CREATE_TASK "Onboard new customer: " + opportunity.name, "high", user_id
ELSE IF new_stage = "closed_lost" THEN
TALK "What was the reason for losing this deal?"
loss_reason = HEAR
opportunity.loss_reason = loss_reason
SAVE_FROM_UNSTRUCTURED "opportunities", FORMAT opportunity AS JSON
CREATE_TASK "Analyze lost deal: " + opportunity.name, "low", user_id
END IF
END IF
IF action = "add_product" THEN
IF opportunity_id = "" THEN
TALK "No opportunity selected."
EXIT
END IF
TALK "Enter product name or code:"
product_search = HEAR
product = FIND "products", "name LIKE '%" + product_search + "%' OR code = '" + product_search + "'"
IF product = NULL THEN
TALK "Product not found."
EXIT
END IF
TALK "How many units?"
quantity = HEAR
TALK "Any discount percentage? (0 for none)"
discount = HEAR
line_item = CREATE OBJECT
SET line_item.id = FORMAT GUID()
SET line_item.opportunity_id = opportunity_id
SET line_item.product_id = product.id
SET line_item.product_name = product.name
SET line_item.quantity = quantity
SET line_item.unit_price = product.unit_price
SET line_item.discount = discount
SET line_item.total = quantity * product.unit_price * (1 - discount / 100)
SET line_item.created_at = current_time
SAVE_FROM_UNSTRUCTURED "opportunity_products", FORMAT line_item AS JSON
opportunity = FIND "opportunities", "id = '" + opportunity_id + "'"
opportunity.amount = opportunity.amount + line_item.total
SAVE_FROM_UNSTRUCTURED "opportunities", FORMAT opportunity AS JSON
TALK "Added " + quantity + " x " + product.name + " = $" + line_item.total
END IF
IF action = "generate_quote" THEN
IF opportunity_id = "" THEN
TALK "No opportunity selected."
EXIT
END IF
opportunity = FIND "opportunities", "id = '" + opportunity_id + "'"
products = FIND "opportunity_products", "opportunity_id = '" + opportunity_id + "'"
IF products = NULL THEN
TALK "No products added to this opportunity."
EXIT
END IF
account = FIND "accounts", "id = '" + opportunity.account_id + "'"
contact = FIND "contacts", "account_id = '" + opportunity.account_id + "' AND primary_contact = true"
quote = CREATE OBJECT
SET quote.id = FORMAT GUID()
SET quote.quote_number = "Q-" + FORMAT NOW() AS "YYYYMMDD" + "-" + FORMAT RANDOM(1000, 9999)
SET quote.opportunity_id = opportunity_id
SET quote.account_id = account.id
SET quote.contact_id = contact.id
SET quote.status = "draft"
SET quote.valid_until = FORMAT ADD_DAYS(NOW(), 30) AS "YYYY-MM-DD"
SET quote.subtotal = opportunity.amount
SET quote.tax_rate = 10
SET quote.tax_amount = opportunity.amount * 0.1
SET quote.total = opportunity.amount * 1.1
SET quote.created_at = current_time
SAVE_FROM_UNSTRUCTURED "quotes", FORMAT quote AS JSON
REMEMBER "quote_" + quote.id = quote
quote_content = "QUOTATION\n"
quote_content = quote_content + "Quote #: " + quote.quote_number + "\n"
quote_content = quote_content + "Date: " + current_time + "\n"
quote_content = quote_content + "Valid Until: " + quote.valid_until + "\n\n"
quote_content = quote_content + "To: " + account.name + "\n"
quote_content = quote_content + "Contact: " + contact.name + "\n\n"
quote_content = quote_content + "ITEMS:\n"
FOR EACH item IN products DO
quote_content = quote_content + item.product_name + " x " + item.quantity + " @ $" + item.unit_price + " = $" + item.total + "\n"
END FOR
quote_content = quote_content + "\nSubtotal: $" + quote.subtotal + "\n"
quote_content = quote_content + "Tax (10%): $" + quote.tax_amount + "\n"
quote_content = quote_content + "TOTAL: $" + quote.total + "\n"
CREATE_DRAFT quote_content, "Quote " + quote.quote_number + " for " + account.name
TALK "Quote " + quote.quote_number + " generated for $" + quote.total
IF contact.email != "" THEN
TALK "Send quote to " + contact.name + " at " + contact.email + "? (yes/no)"
send_quote = HEAR
IF send_quote = "yes" OR send_quote = "YES" OR send_quote = "Yes" THEN
subject = "Quote " + quote.quote_number + " from Our Company"
SEND MAIL contact.email, subject, quote_content
quote.status = "sent"
quote.sent_at = current_time
SAVE_FROM_UNSTRUCTURED "quotes", FORMAT quote AS JSON
TALK "Quote sent to " + contact.email
CREATE_TASK "Follow up on quote " + quote.quote_number, "medium", user_id
END IF
END IF
END IF
IF action = "forecast" THEN
opportunities = FIND "opportunities", "stage != 'closed_won' AND stage != 'closed_lost'"
total_pipeline = 0
weighted_pipeline = 0
q1_forecast = 0
q2_forecast = 0
q3_forecast = 0
q4_forecast = 0
FOR EACH opp IN opportunities DO
total_pipeline = total_pipeline + opp.amount
weighted_value = opp.amount * opp.probability / 100
weighted_pipeline = weighted_pipeline + weighted_value
close_month = FORMAT opp.close_date AS "MM"
IF close_month <= "03" THEN
q1_forecast = q1_forecast + weighted_value
ELSE IF close_month <= "06" THEN
q2_forecast = q2_forecast + weighted_value
ELSE IF close_month <= "09" THEN
q3_forecast = q3_forecast + weighted_value
ELSE
q4_forecast = q4_forecast + weighted_value
END IF
END FOR
TALK "SALES FORECAST"
TALK "=============="
TALK "Total Pipeline: $" + total_pipeline
TALK "Weighted Pipeline: $" + weighted_pipeline
TALK ""
TALK "Quarterly Forecast:"
TALK "Q1: $" + q1_forecast
TALK "Q2: $" + q2_forecast
TALK "Q3: $" + q3_forecast
TALK "Q4: $" + q4_forecast
forecast_report = CREATE OBJECT
SET forecast_report.total_pipeline = total_pipeline
SET forecast_report.weighted_pipeline = weighted_pipeline
SET forecast_report.q1 = q1_forecast
SET forecast_report.q2 = q2_forecast
SET forecast_report.q3 = q3_forecast
SET forecast_report.q4 = q4_forecast
SET forecast_report.generated_at = current_time
REMEMBER "forecast_" + FORMAT NOW() AS "YYYYMMDD" = forecast_report
END IF

391
templates/crm.gbai/crm.gbdialog/tables.bas Normal file
View file

@ -0,0 +1,391 @@
' CRM Database Tables Definition
' This file defines all CRM tables using the TABLE keyword
' Tables are automatically created and managed by the system
' Leads table - stores potential customers
TABLE leads
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
company_name VARCHAR(255) NOT NULL
contact_name VARCHAR(255)
email VARCHAR(255) UNIQUE
phone VARCHAR(50)
website VARCHAR(255)
industry VARCHAR(100)
company_size VARCHAR(50)
lead_source VARCHAR(100)
lead_status VARCHAR(50) DEFAULT 'new'
score INTEGER DEFAULT 0
assigned_to VARCHAR(100)
notes TEXT
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
converted_at TIMESTAMP
converted_to_account_id UUID
END TABLE
' Accounts table - stores customer organizations
TABLE accounts
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
name VARCHAR(255) NOT NULL
type VARCHAR(50) DEFAULT 'customer'
industry VARCHAR(100)
annual_revenue DECIMAL(15,2)
employees INTEGER
website VARCHAR(255)
phone VARCHAR(50)
billing_address TEXT
shipping_address TEXT
owner_id VARCHAR(100)
parent_account_id UUID
status VARCHAR(50) DEFAULT 'active'
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Contacts table - stores individual people
TABLE contacts
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
account_id UUID REFERENCES accounts(id)
first_name VARCHAR(100)
last_name VARCHAR(100)
full_name VARCHAR(255) GENERATED ALWAYS AS (first_name || ' ' || last_name) STORED
email VARCHAR(255) UNIQUE
phone VARCHAR(50)
mobile VARCHAR(50)
title VARCHAR(100)
department VARCHAR(100)
lead_id UUID REFERENCES leads(id)
primary_contact BOOLEAN DEFAULT FALSE
do_not_call BOOLEAN DEFAULT FALSE
do_not_email BOOLEAN DEFAULT FALSE
preferred_contact_method VARCHAR(50)
linkedin_url VARCHAR(255)
twitter_handle VARCHAR(100)
notes TEXT
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Opportunities table - stores sales opportunities
TABLE opportunities
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
name VARCHAR(255) NOT NULL
account_id UUID REFERENCES accounts(id)
contact_id UUID REFERENCES contacts(id)
amount DECIMAL(15,2)
probability INTEGER CHECK (probability >= 0 AND probability <= 100)
expected_revenue DECIMAL(15,2) GENERATED ALWAYS AS (amount * probability / 100) STORED
stage VARCHAR(100) DEFAULT 'qualification'
close_date DATE
type VARCHAR(50)
lead_source VARCHAR(100)
next_step TEXT
description TEXT
owner_id VARCHAR(100)
campaign_id UUID
competitor_names TEXT[]
won BOOLEAN
closed_at TIMESTAMP
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Activities table - stores all customer interactions
TABLE activities
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
type VARCHAR(50) NOT NULL
subject VARCHAR(255) NOT NULL
description TEXT
status VARCHAR(50) DEFAULT 'open'
priority VARCHAR(20) DEFAULT 'normal'
due_date TIMESTAMP
completed_date TIMESTAMP
duration_minutes INTEGER
location VARCHAR(255)
' Related entities
account_id UUID REFERENCES accounts(id)
contact_id UUID REFERENCES contacts(id)
opportunity_id UUID REFERENCES opportunities(id)
lead_id UUID REFERENCES leads(id)
parent_activity_id UUID REFERENCES activities(id)
' Assignment and tracking
assigned_to VARCHAR(100)
created_by VARCHAR(100)
modified_by VARCHAR(100)
' Activity-specific fields
call_result VARCHAR(100)
call_duration INTEGER
email_message_id VARCHAR(255)
meeting_notes TEXT
meeting_attendees TEXT[]
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Products table - stores product catalog
TABLE products
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
name VARCHAR(255) NOT NULL
code VARCHAR(100) UNIQUE
description TEXT
category VARCHAR(100)
unit_price DECIMAL(10,2)
cost DECIMAL(10,2)
margin DECIMAL(5,2) GENERATED ALWAYS AS ((unit_price - cost) / unit_price * 100) STORED
quantity_in_stock INTEGER DEFAULT 0
active BOOLEAN DEFAULT TRUE
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Quotes table - stores sales quotes
TABLE quotes
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
quote_number VARCHAR(50) UNIQUE
opportunity_id UUID REFERENCES opportunities(id)
account_id UUID REFERENCES accounts(id)
contact_id UUID REFERENCES contacts(id)
status VARCHAR(50) DEFAULT 'draft'
valid_until DATE
subtotal DECIMAL(15,2)
discount_percent DECIMAL(5,2) DEFAULT 0
discount_amount DECIMAL(15,2) DEFAULT 0
tax_rate DECIMAL(5,2) DEFAULT 0
tax_amount DECIMAL(15,2)
total DECIMAL(15,2)
terms_conditions TEXT
notes TEXT
approved_by VARCHAR(100)
approved_at TIMESTAMP
sent_at TIMESTAMP
created_by VARCHAR(100)
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Quote line items
TABLE quote_items
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
quote_id UUID REFERENCES quotes(id) ON DELETE CASCADE
product_id UUID REFERENCES products(id)
description TEXT
quantity INTEGER NOT NULL
unit_price DECIMAL(10,2) NOT NULL
discount_percent DECIMAL(5,2) DEFAULT 0
total DECIMAL(10,2) GENERATED ALWAYS AS (quantity * unit_price * (1 - discount_percent/100)) STORED
position INTEGER
created_at TIMESTAMP DEFAULT NOW()
END TABLE
' Campaigns table - stores marketing campaigns
TABLE campaigns
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
name VARCHAR(255) NOT NULL
type VARCHAR(50)
status VARCHAR(50) DEFAULT 'planning'
start_date DATE
end_date DATE
budget DECIMAL(15,2)
actual_cost DECIMAL(15,2)
expected_revenue DECIMAL(15,2)
expected_response DECIMAL(5,2)
description TEXT
objective TEXT
num_sent INTEGER DEFAULT 0
num_responses INTEGER DEFAULT 0
num_leads INTEGER DEFAULT 0
num_opportunities INTEGER DEFAULT 0
num_won_opportunities INTEGER DEFAULT 0
revenue_generated DECIMAL(15,2)
roi DECIMAL(10,2) GENERATED ALWAYS AS
(CASE WHEN actual_cost > 0 THEN (revenue_generated - actual_cost) / actual_cost * 100 ELSE 0 END) STORED
owner_id VARCHAR(100)
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Campaign members
TABLE campaign_members
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
campaign_id UUID REFERENCES campaigns(id) ON DELETE CASCADE
lead_id UUID REFERENCES leads(id)
contact_id UUID REFERENCES contacts(id)
status VARCHAR(50) DEFAULT 'sent'
responded BOOLEAN DEFAULT FALSE
response_date TIMESTAMP
created_at TIMESTAMP DEFAULT NOW()
END TABLE
' Cases/Tickets table - stores customer support cases
TABLE cases
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
case_number VARCHAR(50) UNIQUE
subject VARCHAR(255) NOT NULL
description TEXT
status VARCHAR(50) DEFAULT 'new'
priority VARCHAR(20) DEFAULT 'medium'
type VARCHAR(50)
origin VARCHAR(50)
reason VARCHAR(100)
account_id UUID REFERENCES accounts(id)
contact_id UUID REFERENCES contacts(id)
parent_case_id UUID REFERENCES cases(id)
assigned_to VARCHAR(100)
escalated_to VARCHAR(100)
resolution TEXT
resolved_at TIMESTAMP
satisfaction_score INTEGER CHECK (satisfaction_score >= 1 AND satisfaction_score <= 5)
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
closed_at TIMESTAMP
END TABLE
' Email tracking table
TABLE email_tracking
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
message_id VARCHAR(255) UNIQUE
from_address VARCHAR(255)
to_addresses TEXT[]
cc_addresses TEXT[]
subject VARCHAR(255)
body TEXT
html_body TEXT
' Related entities
account_id UUID REFERENCES accounts(id)
contact_id UUID REFERENCES contacts(id)
opportunity_id UUID REFERENCES opportunities(id)
lead_id UUID REFERENCES leads(id)
case_id UUID REFERENCES cases(id)
activity_id UUID REFERENCES activities(id)
' Tracking
sent_at TIMESTAMP
delivered_at TIMESTAMP
opened_at TIMESTAMP
clicked_at TIMESTAMP
bounced BOOLEAN DEFAULT FALSE
bounce_reason TEXT
created_at TIMESTAMP DEFAULT NOW()
END TABLE
' Documents/Attachments table
TABLE documents
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
name VARCHAR(255) NOT NULL
file_path VARCHAR(500)
file_size INTEGER
mime_type VARCHAR(100)
description TEXT
' Polymorphic associations
entity_type VARCHAR(50)
entity_id UUID
uploaded_by VARCHAR(100)
created_at TIMESTAMP DEFAULT NOW()
END TABLE
' Notes table - stores notes for any entity
TABLE notes
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
title VARCHAR(255)
body TEXT NOT NULL
' Polymorphic associations
entity_type VARCHAR(50)
entity_id UUID
is_private BOOLEAN DEFAULT FALSE
created_by VARCHAR(100)
modified_by VARCHAR(100)
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Tags table for categorization
TABLE tags
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
name VARCHAR(100) UNIQUE NOT NULL
color VARCHAR(7)
description TEXT
created_at TIMESTAMP DEFAULT NOW()
END TABLE
' Entity tags junction table
TABLE entity_tags
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
tag_id UUID REFERENCES tags(id) ON DELETE CASCADE
entity_type VARCHAR(50)
entity_id UUID
created_at TIMESTAMP DEFAULT NOW()
UNIQUE(tag_id, entity_type, entity_id)
END TABLE
' Pipeline stages configuration
TABLE pipeline_stages
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
pipeline_type VARCHAR(50) NOT NULL
stage_name VARCHAR(100) NOT NULL
stage_order INTEGER NOT NULL
probability INTEGER DEFAULT 0
is_won BOOLEAN DEFAULT FALSE
is_closed BOOLEAN DEFAULT FALSE
color VARCHAR(7)
created_at TIMESTAMP DEFAULT NOW()
UNIQUE(pipeline_type, stage_order)
END TABLE
' User preferences and settings
TABLE crm_user_settings
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
user_id VARCHAR(100) UNIQUE NOT NULL
default_pipeline VARCHAR(50)
email_signature TEXT
notification_preferences JSONB
dashboard_layout JSONB
list_view_preferences JSONB
timezone VARCHAR(50) DEFAULT 'UTC'
date_format VARCHAR(20) DEFAULT 'YYYY-MM-DD'
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Audit log for tracking changes
TABLE audit_log
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
entity_type VARCHAR(50) NOT NULL
entity_id UUID NOT NULL
action VARCHAR(50) NOT NULL
field_name VARCHAR(100)
old_value TEXT
new_value TEXT
user_id VARCHAR(100)
ip_address VARCHAR(45)
user_agent TEXT
created_at TIMESTAMP DEFAULT NOW()
END TABLE
' Indexes for performance
CREATE INDEX idx_leads_status ON leads(lead_status)
CREATE INDEX idx_leads_assigned ON leads(assigned_to)
CREATE INDEX idx_accounts_owner ON accounts(owner_id)
CREATE INDEX idx_contacts_account ON contacts(account_id)
CREATE INDEX idx_opportunities_account ON opportunities(account_id)
CREATE INDEX idx_opportunities_stage ON opportunities(stage)
CREATE INDEX idx_opportunities_close_date ON opportunities(close_date)
CREATE INDEX idx_activities_due_date ON activities(due_date)
CREATE INDEX idx_activities_assigned ON activities(assigned_to)
CREATE INDEX idx_cases_status ON cases(status)
CREATE INDEX idx_cases_assigned ON cases(assigned_to)
CREATE INDEX idx_audit_entity ON audit_log(entity_type, entity_id)
CREATE INDEX idx_entity_tags ON entity_tags(entity_type, entity_id)

293
templates/erp.gbai/erp.gbdialog/erp-jobs.bas Normal file
View file

@ -0,0 +1,293 @@
PARAM job_name AS STRING
user_id = GET "session.user_id"
current_time = FORMAT NOW() AS "YYYY-MM-DD HH:mm:ss"
IF job_name = "inventory_reorder" THEN
items = FIND "items", "is_purchasable = true AND reorder_point > 0"
reorders_created = 0
FOR EACH item IN items DO
stocks = FIND "inventory_stock", "item_id = '" + item.id + "'"
total_available = 0
FOR EACH stock IN stocks DO
total_available = total_available + stock.quantity_available
END FOR
IF total_available <= item.reorder_point THEN
po = CREATE OBJECT
SET po.id = FORMAT GUID()
SET po.po_number = "PO-AUTO-" + FORMAT NOW() AS "YYYYMMDD" + "-" + FORMAT RANDOM(100, 999)
SET po.status = "draft"
SET po.order_date = current_time
SET po.buyer_id = "system"
SET po.created_by = "system"
SET po.created_at = current_time
vendor_item = FIND "vendor_items", "item_id = '" + item.id + "' AND is_preferred = true"
IF vendor_item != NULL THEN
po.vendor_id = vendor_item.vendor_id
SAVE_FROM_UNSTRUCTURED "purchase_orders", FORMAT po AS JSON
line = CREATE OBJECT
SET line.id = FORMAT GUID()
SET line.po_id = po.id
SET line.line_number = 1
SET line.item_id = item.id
SET line.quantity_ordered = item.reorder_quantity
SET line.unit_price = vendor_item.unit_price
SET line.created_at = current_time
SAVE_FROM_UNSTRUCTURED "purchase_order_lines", FORMAT line AS JSON
reorders_created = reorders_created + 1
CREATE_TASK "Approve reorder PO " + po.po_number + " for " + item.name, "high", "purchasing"
END IF
END IF
END FOR
IF reorders_created > 0 THEN
notification = "Created " + reorders_created + " automatic reorder POs"
SEND MAIL "purchasing@company.com", "Automatic Reorders", notification
END IF
END IF
IF job_name = "low_stock_alert" THEN
items = FIND "items", "minimum_stock_level > 0"
low_stock_items = []
critical_items = []
FOR EACH item IN items DO
stocks = FIND "inventory_stock", "item_id = '" + item.id + "'"
total_on_hand = 0
FOR EACH stock IN stocks DO
total_on_hand = total_on_hand + stock.quantity_on_hand
END FOR
IF total_on_hand < item.minimum_stock_level THEN
stock_ratio = total_on_hand / item.minimum_stock_level
IF stock_ratio < 0.25 THEN
APPEND critical_items, item.name + " (" + total_on_hand + "/" + item.minimum_stock_level + ")"
ELSE
APPEND low_stock_items, item.name + " (" + total_on_hand + "/" + item.minimum_stock_level + ")"
END IF
END IF
END FOR
IF LENGTH(critical_items) > 0 OR LENGTH(low_stock_items) > 0 THEN
alert = "INVENTORY ALERT\n"
alert = alert + "===============\n\n"
IF LENGTH(critical_items) > 0 THEN
alert = alert + "CRITICAL (Below 25%):\n"
FOR EACH item_info IN critical_items DO
alert = alert + "- " + item_info + "\n"
END FOR
alert = alert + "\n"
END IF
IF LENGTH(low_stock_items) > 0 THEN
alert = alert + "LOW STOCK:\n"
FOR EACH item_info IN low_stock_items DO
alert = alert + "- " + item_info + "\n"
END FOR
END IF
SEND MAIL "inventory-manager@company.com", "Low Stock Alert", alert
END IF
END IF
IF job_name = "po_follow_up" THEN
pos = FIND "purchase_orders", "status = 'approved'"
FOR EACH po IN pos DO
days_old = DAYS_BETWEEN(po.order_date, current_time)
IF days_old > 7 THEN
vendor = FIND "vendors", "id = '" + po.vendor_id + "'"
notification = "PO " + po.po_number + " has been approved for " + days_old + " days without receipt"
SEND MAIL po.buyer_id, "PO Follow-up Required", notification
CREATE_TASK "Follow up on PO " + po.po_number + " with " + vendor.name, "medium", po.buyer_id
END IF
END FOR
END IF
IF job_name = "cost_analysis" THEN
start_of_month = FORMAT NOW() AS "YYYY-MM" + "-01"
transactions = FIND "inventory_transactions", "created_at >= '" + start_of_month + "'"
total_receipts_value = 0
total_shipments_value = 0
total_adjustments_value = 0
FOR EACH trans IN transactions DO
IF trans.transaction_type = "receipt" THEN
total_receipts_value = total_receipts_value + trans.total_cost
ELSE IF trans.transaction_type = "shipment" THEN
total_shipments_value = total_shipments_value + ABS(trans.total_cost)
ELSE IF trans.transaction_type = "adjustment" THEN
total_adjustments_value = total_adjustments_value + ABS(trans.total_cost)
END IF
END FOR
report = "MONTHLY INVENTORY COST ANALYSIS\n"
report = report + "================================\n"
report = report + "Period: " + FORMAT NOW() AS "MMMM YYYY" + "\n\n"
report = report + "Receipts Value: $" + total_receipts_value + "\n"
report = report + "Shipments Value: $" + total_shipments_value + "\n"
report = report + "Adjustments Value: $" + total_adjustments_value + "\n"
report = report + "\n"
report = report + "Gross Margin: $" + (total_shipments_value - total_receipts_value) + "\n"
SEND MAIL "cfo@company.com", "Monthly Inventory Cost Report", report
END IF
IF job_name = "vendor_scorecard" THEN
vendors = FIND "vendors", "status = 'active'"
scorecard = "VENDOR SCORECARD - " + current_time + "\n"
scorecard = scorecard + "====================================\n\n"
FOR EACH vendor IN vendors DO
pos = FIND "purchase_orders", "vendor_id = '" + vendor.id + "' AND created_at >= DATE_SUB(NOW(), INTERVAL 90 DAY)"
total_pos = 0
on_time = 0
total_spend = 0
FOR EACH po IN pos DO
total_pos = total_pos + 1
total_spend = total_spend + po.total_amount
IF po.status = "received" THEN
IF po.received_date <= po.expected_date THEN
on_time = on_time + 1
END IF
END IF
END FOR
IF total_pos > 0 THEN
on_time_rate = (on_time / total_pos) * 100
scorecard = scorecard + vendor.name + "\n"
scorecard = scorecard + " Orders: " + total_pos + "\n"
scorecard = scorecard + " Spend: $" + total_spend + "\n"
scorecard = scorecard + " On-Time: " + on_time_rate + "%\n"
IF on_time_rate < 80 THEN
scorecard = scorecard + " WARNING: Low performance\n"
END IF
scorecard = scorecard + "\n"
END IF
END FOR
SEND MAIL "purchasing@company.com", "Vendor Scorecard", scorecard
END IF
IF job_name = "warehouse_capacity" THEN
warehouses = FIND "warehouses", "is_active = true"
capacity_report = "WAREHOUSE CAPACITY REPORT\n"
capacity_report = capacity_report + "========================\n\n"
FOR EACH warehouse IN warehouses DO
stocks = FIND "inventory_stock", "warehouse_id = '" + warehouse.id + "'"
total_units = 0
FOR EACH stock IN stocks DO
total_units = total_units + stock.quantity_on_hand
END FOR
utilization = 0
IF warehouse.capacity_units > 0 THEN
utilization = (total_units / warehouse.capacity_units) * 100
END IF
capacity_report = capacity_report + warehouse.name + "\n"
capacity_report = capacity_report + " Units: " + total_units + " / " + warehouse.capacity_units + "\n"
capacity_report = capacity_report + " Utilization: " + utilization + "%\n"
IF utilization > 90 THEN
capacity_report = capacity_report + " WARNING: Near capacity\n"
CREATE_TASK "Review capacity at " + warehouse.name, "high", "warehouse-manager"
ELSE IF utilization < 20 THEN
capacity_report = capacity_report + " NOTE: Low utilization\n"
END IF
capacity_report = capacity_report + "\n"
END FOR
SEND MAIL "operations@company.com", "Warehouse Capacity Report", capacity_report
END IF
IF job_name = "invoice_aging" THEN
invoices = FIND "invoices", "balance_due > 0"
aging_30 = 0
aging_60 = 0
aging_90 = 0
aging_over_90 = 0
total_30 = 0
total_60 = 0
total_90 = 0
total_over_90 = 0
FOR EACH invoice IN invoices DO
days_old = DAYS_BETWEEN(invoice.invoice_date, current_time)
IF days_old <= 30 THEN
aging_30 = aging_30 + 1
total_30 = total_30 + invoice.balance_due
ELSE IF days_old <= 60 THEN
aging_60 = aging_60 + 1
total_60 = total_60 + invoice.balance_due
ELSE IF days_old <= 90 THEN
aging_90 = aging_90 + 1
total_90 = total_90 + invoice.balance_due
ELSE
aging_over_90 = aging_over_90 + 1
total_over_90 = total_over_90 + invoice.balance_due
customer = FIND "customers", "id = '" + invoice.customer_id + "'"
IF customer != NULL THEN
notification = "Invoice " + invoice.invoice_number + " is over 90 days past due. Amount: $" + invoice.balance_due
CREATE_TASK "Collection: " + customer.name + " - " + invoice.invoice_number, "critical", "collections"
END IF
END IF
END FOR
aging_report = "ACCOUNTS RECEIVABLE AGING\n"
aging_report = aging_report + "=========================\n\n"
aging_report = aging_report + "0-30 days: " + aging_30 + " invoices, $" + total_30 + "\n"
aging_report = aging_report + "31-60 days: " + aging_60 + " invoices, $" + total_60 + "\n"
aging_report = aging_report + "61-90 days: " + aging_90 + " invoices, $" + total_90 + "\n"
aging_report = aging_report + "Over 90 days: " + aging_over_90 + " invoices, $" + total_over_90 + "\n"
aging_report = aging_report + "\n"
aging_report = aging_report + "TOTAL OUTSTANDING: $" + (total_30 + total_60 + total_90 + total_over_90) + "\n"
SEND MAIL "finance@company.com", "AR Aging Report", aging_report
END IF
IF job_name = "setup_schedules" THEN
SET SCHEDULE "0 6 * * *" "erp-jobs.bas" "inventory_reorder"
SET SCHEDULE "0 8,16 * * *" "erp-jobs.bas" "low_stock_alert"
SET SCHEDULE "0 10 * * *" "erp-jobs.bas" "po_follow_up"
SET SCHEDULE "0 0 1 * *" "erp-jobs.bas" "cost_analysis"
SET SCHEDULE "0 9 * * MON" "erp-jobs.bas" "vendor_scorecard"
SET SCHEDULE "0 7 * * *" "erp-jobs.bas" "warehouse_capacity"
SET SCHEDULE "0 11 * * *" "erp-jobs.bas" "invoice_aging"
TALK "All ERP schedules have been configured"
END IF

378
templates/erp.gbai/erp.gbdialog/inventory-management.bas Normal file
View file

@ -0,0 +1,378 @@
PARAM action AS STRING
PARAM item_data AS OBJECT
user_id = GET "session.user_id"
warehouse_id = GET "session.warehouse_id"
current_time = FORMAT NOW() AS "YYYY-MM-DD HH:mm:ss"
IF action = "receive_inventory" THEN
po_number = GET "item_data.po_number"
IF po_number = "" THEN
TALK "Enter Purchase Order number:"
po_number = HEAR
END IF
po = FIND "purchase_orders", "po_number = '" + po_number + "'"
IF po = NULL THEN
TALK "Purchase order not found."
EXIT
END IF
IF po.status = "received" THEN
TALK "This PO has already been received."
EXIT
END IF
po_lines = FIND "purchase_order_lines", "po_id = '" + po.id + "'"
FOR EACH line IN po_lines DO
item = FIND "items", "id = '" + line.item_id + "'"
TALK "Receiving " + item.name + " - Ordered: " + line.quantity_ordered
TALK "Enter quantity received:"
qty_received = HEAR
stock = FIND "inventory_stock", "item_id = '" + item.id + "' AND warehouse_id = '" + warehouse_id + "'"
IF stock = NULL THEN
stock = CREATE OBJECT
SET stock.id = FORMAT GUID()
SET stock.item_id = item.id
SET stock.warehouse_id = warehouse_id
SET stock.quantity_on_hand = qty_received
SET stock.last_movement_date = current_time
SAVE_FROM_UNSTRUCTURED "inventory_stock", FORMAT stock AS JSON
ELSE
stock.quantity_on_hand = stock.quantity_on_hand + qty_received
stock.last_movement_date = current_time
SAVE_FROM_UNSTRUCTURED "inventory_stock", FORMAT stock AS JSON
END IF
transaction = CREATE OBJECT
SET transaction.id = FORMAT GUID()
SET transaction.transaction_type = "receipt"
SET transaction.transaction_number = "REC-" + FORMAT NOW() AS "YYYYMMDD" + "-" + FORMAT RANDOM(1000, 9999)
SET transaction.item_id = item.id
SET transaction.warehouse_id = warehouse_id
SET transaction.quantity = qty_received
SET transaction.unit_cost = line.unit_price
SET transaction.total_cost = qty_received * line.unit_price
SET transaction.reference_type = "purchase_order"
SET transaction.reference_id = po.id
SET transaction.created_by = user_id
SET transaction.created_at = current_time
SAVE_FROM_UNSTRUCTURED "inventory_transactions", FORMAT transaction AS JSON
line.quantity_received = line.quantity_received + qty_received
SAVE_FROM_UNSTRUCTURED "purchase_order_lines", FORMAT line AS JSON
item.last_cost = line.unit_price
item.average_cost = ((item.average_cost * stock.quantity_on_hand) + (qty_received * line.unit_price)) / (stock.quantity_on_hand + qty_received)
SAVE_FROM_UNSTRUCTURED "items", FORMAT item AS JSON
END FOR
po.status = "received"
SAVE_FROM_UNSTRUCTURED "purchase_orders", FORMAT po AS JSON
TALK "Purchase order " + po_number + " received successfully."
notification = "PO " + po_number + " received at warehouse " + warehouse_id
SEND MAIL po.buyer_id, "PO Received", notification
END IF
IF action = "ship_inventory" THEN
so_number = GET "item_data.so_number"
IF so_number = "" THEN
TALK "Enter Sales Order number:"
so_number = HEAR
END IF
so = FIND "sales_orders", "order_number = '" + so_number + "'"
IF so = NULL THEN
TALK "Sales order not found."
EXIT
END IF
so_lines = FIND "sales_order_lines", "order_id = '" + so.id + "'"
can_ship = true
FOR EACH line IN so_lines DO
item = FIND "items", "id = '" + line.item_id + "'"
stock = FIND "inventory_stock", "item_id = '" + item.id + "' AND warehouse_id = '" + warehouse_id + "'"
IF stock = NULL OR stock.quantity_available < line.quantity_ordered THEN
TALK "Insufficient stock for " + item.name + ". Available: " + stock.quantity_available + ", Needed: " + line.quantity_ordered
can_ship = false
END IF
END FOR
IF can_ship = false THEN
TALK "Cannot ship order due to insufficient inventory."
EXIT
END IF
shipment_number = "SHIP-" + FORMAT NOW() AS "YYYYMMDD" + "-" + FORMAT RANDOM(1000, 9999)
FOR EACH line IN so_lines DO
item = FIND "items", "id = '" + line.item_id + "'"
stock = FIND "inventory_stock", "item_id = '" + item.id + "' AND warehouse_id = '" + warehouse_id + "'"
stock.quantity_on_hand = stock.quantity_on_hand - line.quantity_ordered
stock.last_movement_date = current_time
SAVE_FROM_UNSTRUCTURED "inventory_stock", FORMAT stock AS JSON
transaction = CREATE OBJECT
SET transaction.id = FORMAT GUID()
SET transaction.transaction_type = "shipment"
SET transaction.transaction_number = shipment_number
SET transaction.item_id = item.id
SET transaction.warehouse_id = warehouse_id
SET transaction.quantity = 0 - line.quantity_ordered
SET transaction.unit_cost = item.average_cost
SET transaction.total_cost = line.quantity_ordered * item.average_cost
SET transaction.reference_type = "sales_order"
SET transaction.reference_id = so.id
SET transaction.created_by = user_id
SET transaction.created_at = current_time
SAVE_FROM_UNSTRUCTURED "inventory_transactions", FORMAT transaction AS JSON
line.quantity_shipped = line.quantity_ordered
line.cost_of_goods_sold = line.quantity_ordered * item.average_cost
SAVE_FROM_UNSTRUCTURED "sales_order_lines", FORMAT line AS JSON
END FOR
so.status = "shipped"
SAVE_FROM_UNSTRUCTURED "sales_orders", FORMAT so AS JSON
TALK "Order " + so_number + " shipped. Tracking: " + shipment_number
customer = FIND "customers", "id = '" + so.customer_id + "'"
IF customer != NULL AND customer.email != "" THEN
message = "Your order " + so_number + " has been shipped. Tracking: " + shipment_number
SEND MAIL customer.email, "Order Shipped", message
END IF
END IF
IF action = "check_stock" THEN
item_search = GET "item_data.item_search"
IF item_search = "" THEN
TALK "Enter item name or code:"
item_search = HEAR
END IF
items = FIND "items", "name LIKE '%" + item_search + "%' OR item_code = '" + item_search + "'"
IF items = NULL THEN
TALK "No items found."
EXIT
END IF
FOR EACH item IN items DO
TALK "Item: " + item.name + " (" + item.item_code + ")"
stocks = FIND "inventory_stock", "item_id = '" + item.id + "'"
total_on_hand = 0
total_available = 0
total_reserved = 0
FOR EACH stock IN stocks DO
warehouse = FIND "warehouses", "id = '" + stock.warehouse_id + "'"
TALK " " + warehouse.name + ": " + stock.quantity_on_hand + " on hand, " + stock.quantity_available + " available"
total_on_hand = total_on_hand + stock.quantity_on_hand
total_available = total_available + stock.quantity_available
total_reserved = total_reserved + stock.quantity_reserved
END FOR
TALK " TOTAL: " + total_on_hand + " on hand, " + total_available + " available, " + total_reserved + " reserved"
IF total_available < item.minimum_stock_level THEN
TALK " WARNING: Below minimum stock level (" + item.minimum_stock_level + ")"
IF item.reorder_point > 0 AND total_available <= item.reorder_point THEN
TALK " REORDER NEEDED! Reorder quantity: " + item.reorder_quantity
CREATE_TASK "Reorder " + item.name, "high", user_id
END IF
END IF
END FOR
END IF
IF action = "transfer_stock" THEN
TALK "Enter item code:"
item_code = HEAR
item = FIND "items", "item_code = '" + item_code + "'"
IF item = NULL THEN
TALK "Item not found."
EXIT
END IF
TALK "From warehouse code:"
from_warehouse_code = HEAR
from_warehouse = FIND "warehouses", "code = '" + from_warehouse_code + "'"
IF from_warehouse = NULL THEN
TALK "Source warehouse not found."
EXIT
END IF
from_stock = FIND "inventory_stock", "item_id = '" + item.id + "' AND warehouse_id = '" + from_warehouse.id + "'"
IF from_stock = NULL THEN
TALK "No stock in source warehouse."
EXIT
END IF
TALK "Available: " + from_stock.quantity_available
TALK "Transfer quantity:"
transfer_qty = HEAR
IF transfer_qty > from_stock.quantity_available THEN
TALK "Insufficient available quantity."
EXIT
END IF
TALK "To warehouse code:"
to_warehouse_code = HEAR
to_warehouse = FIND "warehouses", "code = '" + to_warehouse_code + "'"
IF to_warehouse = NULL THEN
TALK "Destination warehouse not found."
EXIT
END IF
transfer_number = "TRAN-" + FORMAT NOW() AS "YYYYMMDD" + "-" + FORMAT RANDOM(1000, 9999)
from_stock.quantity_on_hand = from_stock.quantity_on_hand - transfer_qty
from_stock.last_movement_date = current_time
SAVE_FROM_UNSTRUCTURED "inventory_stock", FORMAT from_stock AS JSON
from_transaction = CREATE OBJECT
SET from_transaction.id = FORMAT GUID()
SET from_transaction.transaction_type = "transfer_out"
SET from_transaction.transaction_number = transfer_number
SET from_transaction.item_id = item.id
SET from_transaction.warehouse_id = from_warehouse.id
SET from_transaction.quantity = 0 - transfer_qty
SET from_transaction.unit_cost = item.average_cost
SET from_transaction.created_by = user_id
SET from_transaction.created_at = current_time
SAVE_FROM_UNSTRUCTURED "inventory_transactions", FORMAT from_transaction AS JSON
to_stock = FIND "inventory_stock", "item_id = '" + item.id + "' AND warehouse_id = '" + to_warehouse.id + "'"
IF to_stock = NULL THEN
to_stock = CREATE OBJECT
SET to_stock.id = FORMAT GUID()
SET to_stock.item_id = item.id
SET to_stock.warehouse_id = to_warehouse.id
SET to_stock.quantity_on_hand = transfer_qty
SET to_stock.last_movement_date = current_time
SAVE_FROM_UNSTRUCTURED "inventory_stock", FORMAT to_stock AS JSON
ELSE
to_stock.quantity_on_hand = to_stock.quantity_on_hand + transfer_qty
to_stock.last_movement_date = current_time
SAVE_FROM_UNSTRUCTURED "inventory_stock", FORMAT to_stock AS JSON
END IF
to_transaction = CREATE OBJECT
SET to_transaction.id = FORMAT GUID()
SET to_transaction.transaction_type = "transfer_in"
SET to_transaction.transaction_number = transfer_number
SET to_transaction.item_id = item.id
SET to_transaction.warehouse_id = to_warehouse.id
SET to_transaction.quantity = transfer_qty
SET to_transaction.unit_cost = item.average_cost
SET to_transaction.created_by = user_id
SET to_transaction.created_at = current_time
SAVE_FROM_UNSTRUCTURED "inventory_transactions", FORMAT to_transaction AS JSON
TALK "Transfer " + transfer_number + " completed: " + transfer_qty + " units from " + from_warehouse.name + " to " + to_warehouse.name
END IF
IF action = "cycle_count" THEN
TALK "Enter warehouse code:"
warehouse_code = HEAR
warehouse = FIND "warehouses", "code = '" + warehouse_code + "'"
IF warehouse = NULL THEN
TALK "Warehouse not found."
EXIT
END IF
stocks = FIND "inventory_stock", "warehouse_id = '" + warehouse.id + "'"
count_number = "COUNT-" + FORMAT NOW() AS "YYYYMMDD" + "-" + FORMAT RANDOM(1000, 9999)
adjustments = 0
FOR EACH stock IN stocks DO
item = FIND "items", "id = '" + stock.item_id + "'"
TALK "Item: " + item.name + " (" + item.item_code + ")"
TALK "System quantity: " + stock.quantity_on_hand
TALK "Enter physical count:"
physical_count = HEAR
IF physical_count != stock.quantity_on_hand THEN
variance = physical_count - stock.quantity_on_hand
adjustment = CREATE OBJECT
SET adjustment.id = FORMAT GUID()
SET adjustment.transaction_type = "adjustment"
SET adjustment.transaction_number = count_number
SET adjustment.item_id = item.id
SET adjustment.warehouse_id = warehouse.id
SET adjustment.quantity = variance
SET adjustment.notes = "Cycle count adjustment"
SET adjustment.created_by = user_id
SET adjustment.created_at = current_time
SAVE_FROM_UNSTRUCTURED "inventory_transactions", FORMAT adjustment AS JSON
stock.quantity_on_hand = physical_count
stock.last_counted_date = current_time
stock.last_movement_date = current_time
SAVE_FROM_UNSTRUCTURED "inventory_stock", FORMAT stock AS JSON
adjustments = adjustments + 1
TALK " Adjusted by " + variance + " units"
ELSE
stock.last_counted_date = current_time
SAVE_FROM_UNSTRUCTURED "inventory_stock", FORMAT stock AS JSON
TALK " Count confirmed"
END IF
END FOR
TALK "Cycle count " + count_number + " completed with " + adjustments + " adjustments"
IF adjustments > 0 THEN
notification = "Cycle count " + count_number + " completed at " + warehouse.name + " with " + adjustments + " adjustments"
SEND MAIL "inventory-manager@company.com", "Cycle Count Results", notification
END IF
END IF

347
templates/erp.gbai/erp.gbdialog/purchasing.bas Normal file
View file

@ -0,0 +1,347 @@
PARAM action AS STRING
PARAM purchase_data AS OBJECT
user_id = GET "session.user_id"
current_time = FORMAT NOW() AS "YYYY-MM-DD HH:mm:ss"
IF action = "create_po" THEN
vendor_code = GET "purchase_data.vendor_code"
IF vendor_code = "" THEN
TALK "Enter vendor code:"
vendor_code = HEAR
END IF
vendor = FIND "vendors", "vendor_code = '" + vendor_code + "'"
IF vendor = NULL THEN
TALK "Vendor not found."
EXIT
END IF
po_number = "PO-" + FORMAT NOW() AS "YYYYMMDD" + "-" + FORMAT RANDOM(1000, 9999)
po = CREATE OBJECT
SET po.id = FORMAT GUID()
SET po.po_number = po_number
SET po.vendor_id = vendor.id
SET po.order_date = current_time
SET po.status = "draft"
SET po.buyer_id = user_id
SET po.created_by = user_id
SET po.created_at = current_time
SET po.subtotal = 0
SAVE_FROM_UNSTRUCTURED "purchase_orders", FORMAT po AS JSON
SET "session.po_id" = po.id
REMEMBER "po_" + po.id = po
TALK "Purchase Order " + po_number + " created for " + vendor.name
adding_items = true
line_number = 1
total = 0
WHILE adding_items = true DO
TALK "Enter item code (or 'done' to finish):"
item_code = HEAR
IF item_code = "done" THEN
adding_items = false
ELSE
item = FIND "items", "item_code = '" + item_code + "'"
IF item = NULL THEN
TALK "Item not found. Try again."
ELSE
TALK "Quantity to order:"
quantity = HEAR
TALK "Unit price (or press Enter for last cost: " + item.last_cost + "):"
price_input = HEAR
IF price_input = "" THEN
unit_price = item.last_cost
ELSE
unit_price = price_input
END IF
line = CREATE OBJECT
SET line.id = FORMAT GUID()
SET line.po_id = po.id
SET line.line_number = line_number
SET line.item_id = item.id
SET line.description = item.name
SET line.quantity_ordered = quantity
SET line.unit_price = unit_price
SET line.line_total = quantity * unit_price
SET line.created_at = current_time
SAVE_FROM_UNSTRUCTURED "purchase_order_lines", FORMAT line AS JSON
total = total + line.line_total
line_number = line_number + 1
TALK "Added: " + item.name + " x " + quantity + " @ $" + unit_price
END IF
END IF
END WHILE
po.subtotal = total
po.total_amount = total
SAVE_FROM_UNSTRUCTURED "purchase_orders", FORMAT po AS JSON
TALK "Purchase Order " + po_number + " total: $" + total
END IF
IF action = "approve_po" THEN
po_number = GET "purchase_data.po_number"
IF po_number = "" THEN
TALK "Enter PO number to approve:"
po_number = HEAR
END IF
po = FIND "purchase_orders", "po_number = '" + po_number + "'"
IF po = NULL THEN
TALK "Purchase order not found."
EXIT
END IF
IF po.status != "draft" THEN
TALK "PO status is " + po.status + ". Can only approve draft POs."
EXIT
END IF
po_lines = FIND "purchase_order_lines", "po_id = '" + po.id + "'"
TALK "PO Summary:"
TALK "Vendor: " + po.vendor_id
TALK "Total: $" + po.total_amount
TALK "Items:"
FOR EACH line IN po_lines DO
TALK " - " + line.description + " x " + line.quantity_ordered + " @ $" + line.unit_price
END FOR
TALK "Approve this PO? (yes/no)"
approval = HEAR
IF approval = "yes" OR approval = "YES" OR approval = "Yes" THEN
po.status = "approved"
po.approved_by = user_id
po.approved_date = current_time
SAVE_FROM_UNSTRUCTURED "purchase_orders", FORMAT po AS JSON
vendor = FIND "vendors", "id = '" + po.vendor_id + "'"
IF vendor.email != "" THEN
message = "Purchase Order " + po_number + " has been approved. Total: $" + po.total_amount
SEND MAIL vendor.email, "PO Approved: " + po_number, message
END IF
TALK "PO " + po_number + " approved successfully."
CREATE_TASK "Process PO " + po_number, "high", user_id
ELSE
TALK "PO not approved."
END IF
END IF
IF action = "vendor_performance" THEN
vendor_code = GET "purchase_data.vendor_code"
IF vendor_code = "" THEN
TALK "Enter vendor code:"
vendor_code = HEAR
END IF
vendor = FIND "vendors", "vendor_code = '" + vendor_code + "'"
IF vendor = NULL THEN
TALK "Vendor not found."
EXIT
END IF
pos = FIND "purchase_orders", "vendor_id = '" + vendor.id + "'"
total_pos = 0
on_time = 0
late = 0
total_spend = 0
FOR EACH po IN pos DO
total_pos = total_pos + 1
total_spend = total_spend + po.total_amount
IF po.status = "received" THEN
IF po.received_date <= po.expected_date THEN
on_time = on_time + 1
ELSE
late = late + 1
END IF
END IF
END FOR
on_time_percentage = 0
IF total_pos > 0 THEN
on_time_percentage = (on_time / total_pos) * 100
END IF
TALK "VENDOR PERFORMANCE: " + vendor.name
TALK "================================"
TALK "Total Purchase Orders: " + total_pos
TALK "Total Spend: $" + total_spend
TALK "On-Time Delivery: " + on_time_percentage + "%"
TALK "Late Deliveries: " + late
IF on_time_percentage < 80 THEN
TALK "WARNING: Low on-time delivery rate"
CREATE_TASK "Review vendor " + vendor.name + " performance", "medium", user_id
END IF
END IF
IF action = "reorder_check" THEN
items = FIND "items", "is_purchasable = true"
reorder_needed = 0
FOR EACH item IN items DO
IF item.reorder_point > 0 THEN
stocks = FIND "inventory_stock", "item_id = '" + item.id + "'"
total_available = 0
FOR EACH stock IN stocks DO
total_available = total_available + stock.quantity_available
END FOR
IF total_available <= item.reorder_point THEN
reorder_needed = reorder_needed + 1
TALK "REORDER: " + item.name
TALK " Current stock: " + total_available
TALK " Reorder point: " + item.reorder_point
TALK " Suggested qty: " + item.reorder_quantity
preferred_vendor = FIND "vendor_items", "item_id = '" + item.id + "' AND is_preferred = true"
IF preferred_vendor != NULL THEN
vendor = FIND "vendors", "id = '" + preferred_vendor.vendor_id + "'"
TALK " Preferred vendor: " + vendor.name
END IF
CREATE_TASK "Reorder " + item.name + " (qty: " + item.reorder_quantity + ")", "high", user_id
END IF
END IF
END FOR
IF reorder_needed = 0 THEN
TALK "No items need reordering."
ELSE
TALK "Total items needing reorder: " + reorder_needed
END IF
END IF
IF action = "requisition" THEN
req_number = "REQ-" + FORMAT NOW() AS "YYYYMMDD" + "-" + FORMAT RANDOM(1000, 9999)
TALK "Creating requisition " + req_number
req = CREATE OBJECT
SET req.id = FORMAT GUID()
SET req.req_number = req_number
SET req.requester = user_id
SET req.status = "pending"
SET req.created_at = current_time
SET req.items = []
adding = true
WHILE adding = true DO
TALK "Enter item description (or 'done'):"
item_desc = HEAR
IF item_desc = "done" THEN
adding = false
ELSE
TALK "Quantity needed:"
quantity = HEAR
TALK "Reason/Project:"
reason = HEAR
req_item = CREATE OBJECT
SET req_item.description = item_desc
SET req_item.quantity = quantity
SET req_item.reason = reason
APPEND req.items, req_item
TALK "Added to requisition."
END IF
END WHILE
SAVE_FROM_UNSTRUCTURED "requisitions", FORMAT req AS JSON
TALK "Requisition " + req_number + " created with " + LENGTH(req.items) + " items."
notification = "New requisition " + req_number + " from " + user_id + " needs approval"
SEND MAIL "purchasing@company.com", "New Requisition", notification
CREATE_TASK "Review requisition " + req_number, "medium", "purchasing"
END IF
IF action = "price_comparison" THEN
item_code = GET "purchase_data.item_code"
IF item_code = "" THEN
TALK "Enter item code:"
item_code = HEAR
END IF
item = FIND "items", "item_code = '" + item_code + "'"
IF item = NULL THEN
TALK "Item not found."
EXIT
END IF
vendor_items = FIND "vendor_items", "item_id = '" + item.id + "'"
IF vendor_items = NULL THEN
TALK "No vendor pricing found for this item."
EXIT
END IF
TALK "PRICE COMPARISON: " + item.name
TALK "================================"
best_price = 999999
best_vendor = ""
FOR EACH vi IN vendor_items DO
vendor = FIND "vendors", "id = '" + vi.vendor_id + "'"
TALK vendor.name + ":"
TALK " Unit price: $" + vi.unit_price
TALK " Min order: " + vi.min_order_qty
TALK " Lead time: " + vi.lead_time_days + " days"
IF vi.unit_price < best_price THEN
best_price = vi.unit_price
best_vendor = vendor.name
END IF
END FOR
TALK ""
TALK "Best price: $" + best_price + " from " + best_vendor
END IF

509
templates/erp.gbai/erp.gbdialog/tables.bas Normal file
View file

@ -0,0 +1,509 @@
' ERP Database Tables Definition
' This file defines all ERP tables using the TABLE keyword
' Tables cover inventory, purchasing, manufacturing, finance, and HR modules
' === INVENTORY MANAGEMENT ===
' Items/Products master table
TABLE items
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
item_code VARCHAR(50) UNIQUE NOT NULL
barcode VARCHAR(50) UNIQUE
name VARCHAR(255) NOT NULL
description TEXT
category VARCHAR(100)
subcategory VARCHAR(100)
unit_of_measure VARCHAR(20) DEFAULT 'EACH'
weight DECIMAL(10,3)
dimensions_length DECIMAL(10,2)
dimensions_width DECIMAL(10,2)
dimensions_height DECIMAL(10,2)
minimum_stock_level INTEGER DEFAULT 0
reorder_point INTEGER
reorder_quantity INTEGER
lead_time_days INTEGER DEFAULT 0
is_active BOOLEAN DEFAULT TRUE
is_purchasable BOOLEAN DEFAULT TRUE
is_saleable BOOLEAN DEFAULT TRUE
is_manufactured BOOLEAN DEFAULT FALSE
standard_cost DECIMAL(15,4)
last_cost DECIMAL(15,4)
average_cost DECIMAL(15,4)
selling_price DECIMAL(15,4)
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Warehouses table
TABLE warehouses
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
code VARCHAR(20) UNIQUE NOT NULL
name VARCHAR(100) NOT NULL
type VARCHAR(50) DEFAULT 'standard'
address TEXT
city VARCHAR(100)
state VARCHAR(50)
country VARCHAR(50)
postal_code VARCHAR(20)
contact_person VARCHAR(100)
contact_phone VARCHAR(50)
contact_email VARCHAR(100)
capacity_units INTEGER
current_occupancy INTEGER DEFAULT 0
is_active BOOLEAN DEFAULT TRUE
created_at TIMESTAMP DEFAULT NOW()
END TABLE
' Inventory stock levels
TABLE inventory_stock
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
item_id UUID REFERENCES items(id)
warehouse_id UUID REFERENCES warehouses(id)
location_code VARCHAR(50)
quantity_on_hand DECIMAL(15,3) DEFAULT 0
quantity_reserved DECIMAL(15,3) DEFAULT 0
quantity_available DECIMAL(15,3) GENERATED ALWAYS AS (quantity_on_hand - quantity_reserved) STORED
quantity_on_order DECIMAL(15,3) DEFAULT 0
last_counted_date DATE
last_movement_date TIMESTAMP
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
UNIQUE(item_id, warehouse_id, location_code)
END TABLE
' Inventory transactions
TABLE inventory_transactions
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
transaction_type VARCHAR(50) NOT NULL
transaction_number VARCHAR(50) UNIQUE
item_id UUID REFERENCES items(id)
warehouse_id UUID REFERENCES warehouses(id)
location_code VARCHAR(50)
quantity DECIMAL(15,3) NOT NULL
unit_cost DECIMAL(15,4)
total_cost DECIMAL(15,2)
reference_type VARCHAR(50)
reference_id UUID
notes TEXT
created_by VARCHAR(100)
created_at TIMESTAMP DEFAULT NOW()
END TABLE
' === PURCHASING MODULE ===
' Vendors/Suppliers table
TABLE vendors
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
vendor_code VARCHAR(50) UNIQUE NOT NULL
name VARCHAR(255) NOT NULL
legal_name VARCHAR(255)
tax_id VARCHAR(50)
vendor_type VARCHAR(50)
status VARCHAR(20) DEFAULT 'active'
rating INTEGER CHECK (rating >= 1 AND rating <= 5)
payment_terms VARCHAR(50)
credit_limit DECIMAL(15,2)
currency_code VARCHAR(3) DEFAULT 'USD'
address TEXT
city VARCHAR(100)
state VARCHAR(50)
country VARCHAR(50)
postal_code VARCHAR(20)
phone VARCHAR(50)
email VARCHAR(100)
website VARCHAR(255)
contact_person VARCHAR(100)
bank_account_number VARCHAR(50)
bank_name VARCHAR(100)
notes TEXT
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Purchase orders
TABLE purchase_orders
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
po_number VARCHAR(50) UNIQUE NOT NULL
vendor_id UUID REFERENCES vendors(id)
order_date DATE NOT NULL
expected_date DATE
status VARCHAR(50) DEFAULT 'draft'
buyer_id VARCHAR(100)
ship_to_warehouse_id UUID REFERENCES warehouses(id)
shipping_method VARCHAR(50)
payment_terms VARCHAR(50)
currency_code VARCHAR(3) DEFAULT 'USD'
exchange_rate DECIMAL(10,6) DEFAULT 1.0
subtotal DECIMAL(15,2)
tax_amount DECIMAL(15,2)
shipping_cost DECIMAL(15,2)
total_amount DECIMAL(15,2)
notes TEXT
approved_by VARCHAR(100)
approved_date TIMESTAMP
created_by VARCHAR(100)
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Purchase order lines
TABLE purchase_order_lines
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
po_id UUID REFERENCES purchase_orders(id) ON DELETE CASCADE
line_number INTEGER NOT NULL
item_id UUID REFERENCES items(id)
description TEXT
quantity_ordered DECIMAL(15,3) NOT NULL
quantity_received DECIMAL(15,3) DEFAULT 0
quantity_remaining DECIMAL(15,3) GENERATED ALWAYS AS (quantity_ordered - quantity_received) STORED
unit_price DECIMAL(15,4) NOT NULL
discount_percent DECIMAL(5,2) DEFAULT 0
tax_rate DECIMAL(5,2) DEFAULT 0
line_total DECIMAL(15,2) GENERATED ALWAYS AS (quantity_ordered * unit_price * (1 - discount_percent/100)) STORED
expected_date DATE
created_at TIMESTAMP DEFAULT NOW()
UNIQUE(po_id, line_number)
END TABLE
' === SALES MODULE ===
' Customers table
TABLE customers
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
customer_code VARCHAR(50) UNIQUE NOT NULL
name VARCHAR(255) NOT NULL
legal_name VARCHAR(255)
tax_id VARCHAR(50)
customer_type VARCHAR(50)
status VARCHAR(20) DEFAULT 'active'
credit_rating VARCHAR(10)
credit_limit DECIMAL(15,2)
payment_terms VARCHAR(50)
discount_percent DECIMAL(5,2) DEFAULT 0
currency_code VARCHAR(3) DEFAULT 'USD'
billing_address TEXT
shipping_address TEXT
city VARCHAR(100)
state VARCHAR(50)
country VARCHAR(50)
postal_code VARCHAR(20)
phone VARCHAR(50)
email VARCHAR(100)
website VARCHAR(255)
sales_person_id VARCHAR(100)
notes TEXT
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Sales orders
TABLE sales_orders
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
order_number VARCHAR(50) UNIQUE NOT NULL
customer_id UUID REFERENCES customers(id)
order_date DATE NOT NULL
required_date DATE
promised_date DATE
status VARCHAR(50) DEFAULT 'draft'
sales_person_id VARCHAR(100)
ship_from_warehouse_id UUID REFERENCES warehouses(id)
shipping_method VARCHAR(50)
payment_terms VARCHAR(50)
payment_method VARCHAR(50)
currency_code VARCHAR(3) DEFAULT 'USD'
exchange_rate DECIMAL(10,6) DEFAULT 1.0
subtotal DECIMAL(15,2)
discount_amount DECIMAL(15,2) DEFAULT 0
tax_amount DECIMAL(15,2)
shipping_cost DECIMAL(15,2)
total_amount DECIMAL(15,2)
notes TEXT
approved_by VARCHAR(100)
approved_date TIMESTAMP
created_by VARCHAR(100)
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Sales order lines
TABLE sales_order_lines
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
order_id UUID REFERENCES sales_orders(id) ON DELETE CASCADE
line_number INTEGER NOT NULL
item_id UUID REFERENCES items(id)
description TEXT
quantity_ordered DECIMAL(15,3) NOT NULL
quantity_shipped DECIMAL(15,3) DEFAULT 0
quantity_invoiced DECIMAL(15,3) DEFAULT 0
unit_price DECIMAL(15,4) NOT NULL
discount_percent DECIMAL(5,2) DEFAULT 0
tax_rate DECIMAL(5,2) DEFAULT 0
line_total DECIMAL(15,2) GENERATED ALWAYS AS (quantity_ordered * unit_price * (1 - discount_percent/100)) STORED
cost_of_goods_sold DECIMAL(15,2)
margin DECIMAL(15,2) GENERATED ALWAYS AS (line_total - cost_of_goods_sold) STORED
warehouse_id UUID REFERENCES warehouses(id)
promised_date DATE
created_at TIMESTAMP DEFAULT NOW()
UNIQUE(order_id, line_number)
END TABLE
' === MANUFACTURING MODULE ===
' Bill of Materials (BOM) header
TABLE bill_of_materials
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
bom_number VARCHAR(50) UNIQUE NOT NULL
item_id UUID REFERENCES items(id)
revision VARCHAR(20) DEFAULT 'A'
description TEXT
quantity_per_assembly DECIMAL(15,3) DEFAULT 1
unit_of_measure VARCHAR(20)
status VARCHAR(20) DEFAULT 'active'
effective_date DATE
expiration_date DATE
total_cost DECIMAL(15,4)
labor_cost DECIMAL(15,4)
overhead_cost DECIMAL(15,4)
created_by VARCHAR(100)
approved_by VARCHAR(100)
approved_date DATE
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' BOM components
TABLE bom_components
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
bom_id UUID REFERENCES bill_of_materials(id) ON DELETE CASCADE
component_item_id UUID REFERENCES items(id)
line_number INTEGER NOT NULL
quantity_required DECIMAL(15,6) NOT NULL
unit_of_measure VARCHAR(20)
scrap_percent DECIMAL(5,2) DEFAULT 0
total_quantity DECIMAL(15,6) GENERATED ALWAYS AS (quantity_required * (1 + scrap_percent/100)) STORED
cost_per_unit DECIMAL(15,4)
total_cost DECIMAL(15,4) GENERATED ALWAYS AS (total_quantity * cost_per_unit) STORED
notes TEXT
created_at TIMESTAMP DEFAULT NOW()
UNIQUE(bom_id, line_number)
END TABLE
' Work orders
TABLE work_orders
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
wo_number VARCHAR(50) UNIQUE NOT NULL
item_id UUID REFERENCES items(id)
bom_id UUID REFERENCES bill_of_materials(id)
quantity_to_produce DECIMAL(15,3) NOT NULL
quantity_completed DECIMAL(15,3) DEFAULT 0
quantity_scrapped DECIMAL(15,3) DEFAULT 0
status VARCHAR(50) DEFAULT 'planned'
priority VARCHAR(20) DEFAULT 'normal'
planned_start_date TIMESTAMP
planned_end_date TIMESTAMP
actual_start_date TIMESTAMP
actual_end_date TIMESTAMP
warehouse_id UUID REFERENCES warehouses(id)
work_center VARCHAR(50)
labor_hours_estimated DECIMAL(10,2)
labor_hours_actual DECIMAL(10,2)
notes TEXT
created_by VARCHAR(100)
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' === FINANCIAL MODULE ===
' General ledger accounts
TABLE gl_accounts
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
account_number VARCHAR(20) UNIQUE NOT NULL
account_name VARCHAR(100) NOT NULL
account_type VARCHAR(50) NOT NULL
parent_account_id UUID REFERENCES gl_accounts(id)
currency_code VARCHAR(3) DEFAULT 'USD'
normal_balance VARCHAR(10) CHECK (normal_balance IN ('debit', 'credit'))
is_active BOOLEAN DEFAULT TRUE
is_control_account BOOLEAN DEFAULT FALSE
allow_manual_entry BOOLEAN DEFAULT TRUE
description TEXT
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Journal entries header
TABLE journal_entries
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
journal_number VARCHAR(50) UNIQUE NOT NULL
journal_date DATE NOT NULL
posting_date DATE NOT NULL
period VARCHAR(20)
journal_type VARCHAR(50)
description TEXT
reference_type VARCHAR(50)
reference_number VARCHAR(50)
status VARCHAR(20) DEFAULT 'draft'
total_debit DECIMAL(15,2)
total_credit DECIMAL(15,2)
is_balanced BOOLEAN GENERATED ALWAYS AS (total_debit = total_credit) STORED
posted_by VARCHAR(100)
posted_at TIMESTAMP
reversed_by_id UUID REFERENCES journal_entries(id)
created_by VARCHAR(100)
created_at TIMESTAMP DEFAULT NOW()
END TABLE
' Journal entry lines
TABLE journal_entry_lines
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
journal_entry_id UUID REFERENCES journal_entries(id) ON DELETE CASCADE
line_number INTEGER NOT NULL
account_id UUID REFERENCES gl_accounts(id)
debit_amount DECIMAL(15,2) DEFAULT 0
credit_amount DECIMAL(15,2) DEFAULT 0
description TEXT
dimension1 VARCHAR(50)
dimension2 VARCHAR(50)
dimension3 VARCHAR(50)
created_at TIMESTAMP DEFAULT NOW()
UNIQUE(journal_entry_id, line_number)
CHECK (debit_amount = 0 OR credit_amount = 0)
END TABLE
' Invoices
TABLE invoices
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
invoice_number VARCHAR(50) UNIQUE NOT NULL
invoice_type VARCHAR(20) DEFAULT 'standard'
customer_id UUID REFERENCES customers(id)
vendor_id UUID REFERENCES vendors(id)
order_id UUID
invoice_date DATE NOT NULL
due_date DATE NOT NULL
status VARCHAR(20) DEFAULT 'draft'
currency_code VARCHAR(3) DEFAULT 'USD'
exchange_rate DECIMAL(10,6) DEFAULT 1.0
subtotal DECIMAL(15,2)
discount_amount DECIMAL(15,2) DEFAULT 0
tax_amount DECIMAL(15,2)
total_amount DECIMAL(15,2)
amount_paid DECIMAL(15,2) DEFAULT 0
balance_due DECIMAL(15,2) GENERATED ALWAYS AS (total_amount - amount_paid) STORED
payment_terms VARCHAR(50)
notes TEXT
created_by VARCHAR(100)
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' === HUMAN RESOURCES MODULE ===
' Employees table
TABLE employees
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
employee_number VARCHAR(50) UNIQUE NOT NULL
first_name VARCHAR(100) NOT NULL
last_name VARCHAR(100) NOT NULL
middle_name VARCHAR(100)
full_name VARCHAR(255) GENERATED ALWAYS AS (first_name || ' ' || COALESCE(middle_name || ' ', '') || last_name) STORED
email VARCHAR(100) UNIQUE
phone VARCHAR(50)
mobile VARCHAR(50)
address TEXT
city VARCHAR(100)
state VARCHAR(50)
country VARCHAR(50)
postal_code VARCHAR(20)
date_of_birth DATE
gender VARCHAR(20)
marital_status VARCHAR(20)
national_id VARCHAR(50)
passport_number VARCHAR(50)
department_id UUID
position_title VARCHAR(100)
manager_id UUID REFERENCES employees(id)
hire_date DATE NOT NULL
employment_status VARCHAR(50) DEFAULT 'active'
employment_type VARCHAR(50) DEFAULT 'full-time'
salary DECIMAL(15,2)
hourly_rate DECIMAL(10,2)
commission_percent DECIMAL(5,2)
bank_account_number VARCHAR(50)
bank_name VARCHAR(100)
emergency_contact_name VARCHAR(100)
emergency_contact_phone VARCHAR(50)
notes TEXT
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
END TABLE
' Payroll records
TABLE payroll
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
payroll_number VARCHAR(50) UNIQUE NOT NULL
employee_id UUID REFERENCES employees(id)
pay_period_start DATE NOT NULL
pay_period_end DATE NOT NULL
payment_date DATE NOT NULL
hours_worked DECIMAL(10,2)
overtime_hours DECIMAL(10,2)
regular_pay DECIMAL(15,2)
overtime_pay DECIMAL(15,2)
commission DECIMAL(15,2)
bonus DECIMAL(15,2)
gross_pay DECIMAL(15,2)
tax_deductions DECIMAL(15,2)
other_deductions DECIMAL(15,2)
net_pay DECIMAL(15,2)
payment_method VARCHAR(50)
payment_reference VARCHAR(100)
status VARCHAR(20) DEFAULT 'pending'
approved_by VARCHAR(100)
approved_date TIMESTAMP
created_at TIMESTAMP DEFAULT NOW()
END TABLE
' === SYSTEM TABLES ===
' Audit trail
TABLE erp_audit_log
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
table_name VARCHAR(50) NOT NULL
record_id UUID NOT NULL
action VARCHAR(20) NOT NULL
changed_fields JSONB
old_values JSONB
new_values JSONB
user_id VARCHAR(100)
user_ip VARCHAR(45)
user_agent TEXT
created_at TIMESTAMP DEFAULT NOW()
END TABLE
' System settings
TABLE erp_settings
id UUID PRIMARY KEY DEFAULT uuid_generate_v4()
module VARCHAR(50) NOT NULL
setting_key VARCHAR(100) NOT NULL
setting_value TEXT
data_type VARCHAR(20)
description TEXT
is_encrypted BOOLEAN DEFAULT FALSE
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
UNIQUE(module, setting_key)
END TABLE
' Create indexes for performance
CREATE INDEX idx_inventory_item_warehouse ON inventory_stock(item_id, warehouse_id)
CREATE INDEX idx_po_vendor ON purchase_orders(vendor_id)
CREATE INDEX idx_po_status ON purchase_orders(status)
CREATE INDEX idx_so_customer ON sales_orders(customer_id)
CREATE INDEX idx_so_status ON sales_orders(status)
CREATE INDEX idx_wo_status ON work_orders(status)
CREATE INDEX idx_invoice_customer ON invoices(customer_id)
CREATE INDEX idx_invoice_status ON invoices(status)
CREATE INDEX idx_employee_manager ON employees(manager_id)
CREATE INDEX idx_journal_date ON journal_entries(journal_date)
CREATE INDEX idx_audit_table_record ON erp_audit_log(table_name, record_id)