diff --git a/PROMPT.md b/PROMPT.md index 225d745e..bb99f35a 100644 --- a/PROMPT.md +++ b/PROMPT.md @@ -5,6 +5,44 @@ --- +## Weekly Maintenance - EVERY MONDAY + +### Package Review Checklist + +**Every Monday, review the following:** + +1. **Dependency Updates** + ```bash + cargo outdated + cargo audit + ``` + +2. **Package Consolidation Opportunities** + - Check if new crates can replace custom code + - Look for crates that combine multiple dependencies + - Review `Cargo.toml` for redundant dependencies + +3. **Documentation Sync** + - Verify docs match current implementation + - Check for deprecated keywords/features + - Update examples with latest syntax + +4. **Asset Updates** + - Sync icons with `botui/ui/suite/assets/icons/` + - Verify all screenshots are current + - Check external links are valid + +### Packages to Watch + +| Area | Potential Packages | Purpose | +|------|-------------------|---------| +| Markdown | `pulldown-cmark` | Better MD processing | +| Search | `tantivy` | Full-text search in docs | +| Diagrams | `mermaid` | Replace ASCII diagrams | +| i18n | `fluent` | Translation management | + +--- + ## CRITICAL: Keyword Naming Rules **Keywords NEVER use underscores. Always use spaces.** @@ -554,10 +592,10 @@ FILTER data, "condition" AGGREGATE data, "operation" ``` -### Knowledge Base +### Knowledge Base & Accounts ```basic -USE KB USE KB "collection" +USE ACCOUNT "email@example.com" ' LLM search + file ops CLEAR KB KB STATISTICS KB COLLECTION STATS "name" @@ -577,10 +615,18 @@ USE MODEL "model-name" ```basic TALK "message" HEAR variable -SEND MAIL to, subject, body, attachments +SEND MAIL to, subject, body +SEND MAIL to, subject, body USING "account@example.com" SEND TEMPLATE recipients, template, variables ``` +### File Operations with Accounts +```basic +' account:// path notation (requires USE ACCOUNT first) +COPY "account://user@gmail.com/file.pdf" TO "local.pdf" +COPY "local.xlsx" TO "account://user@outlook.com/data.xlsx" +``` + --- ## Conversation Examples (WhatsApp Style) diff --git a/src/06-gbdialog/keyword-on-change.md b/src/06-gbdialog/keyword-on-change.md new file mode 100644 index 00000000..71fbd849 --- /dev/null +++ b/src/06-gbdialog/keyword-on-change.md @@ -0,0 +1,300 @@ +# ON CHANGE + +Monitors folders for file changes and triggers a script when files are created, modified, or deleted. + +## Syntax + +```basic +' Using account:// syntax (recommended) +ON CHANGE "account://email@domain.com/path/to/folder" + +' Direct provider syntax +ON CHANGE "gdrive:///path/to/folder" +ON CHANGE "onedrive:///path/to/folder" +ON CHANGE "dropbox:///path/to/folder" + +' Local filesystem +ON CHANGE "/local/path/to/folder" + +' With event type filter +ON CHANGE "account://email@domain.com/folder" EVENTS "create, modify" +``` + +## Description + +The `ON CHANGE` keyword registers a folder monitor that triggers a script whenever files change in the specified folder. This works with cloud storage providers (Google Drive, OneDrive, Dropbox) and local filesystem. + +Uses the same `account://` syntax as `COPY`, `MOVE`, and other file operations for consistency. + +## Parameters + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| path | String | Yes | Folder path using account:// or provider:// syntax | +| EVENTS | String | No | Comma-separated list of event types to monitor | + +## Path Syntax + +| Format | Description | Example | +|--------|-------------|---------| +| `account://email/path` | Uses connected account (auto-detects provider) | `account://user@gmail.com/Documents` | +| `gdrive:///path` | Google Drive direct | `gdrive:///shared/reports` | +| `onedrive:///path` | OneDrive direct | `onedrive:///business/docs` | +| `dropbox:///path` | Dropbox direct | `dropbox:///team/assets` | +| `/path` | Local filesystem | `/var/uploads/incoming` | + +## Provider Auto-Detection + +When using `account://` syntax, the provider is auto-detected from the email: + +| Email Domain | Provider | +|--------------|----------| +| `@gmail.com`, `@google.com` | Google Drive | +| `@outlook.com`, `@hotmail.com`, `@live.com` | OneDrive | +| Other | Defaults to Google Drive | + +## Event Types + +| Event | Aliases | Description | +|-------|---------|-------------| +| `create` | `created`, `new` | New file created | +| `modify` | `modified`, `change`, `changed` | File content changed | +| `delete` | `deleted`, `remove`, `removed` | File deleted | +| `rename` | `renamed` | File renamed | +| `move` | `moved` | File moved to different folder | + +Default events (if not specified): `create`, `modify`, `delete` + +## Examples + +### Basic Folder Monitoring + +```basic +' Monitor Google Drive folder via connected account +ON CHANGE "account://user@gmail.com/Documents/invoices" + event = GET LAST "folder_change_events" + TALK "File changed: " + event.file_name + " (" + event.event_type + ")" +END ON +``` + +### Process New Uploads + +```basic +ON CHANGE "account://user@gmail.com/Uploads" EVENTS "create" + event = GET LAST "folder_change_events" + + ' Only process PDF files + IF event.mime_type = "application/pdf" THEN + ' Extract data from invoice + data = SAVE FROM UNSTRUCTURED event.file_path, "invoices" + TALK "Processed invoice: " + data.invoice_number + + ' Move to processed folder + MOVE event.file_path TO "account://user@gmail.com/Processed/" + END IF +END ON +``` + +### Sync Between Providers + +```basic +' When file is added to Google Drive, copy to OneDrive +ON CHANGE "account://user@gmail.com/Shared/Reports" EVENTS "create" + event = GET LAST "folder_change_events" + + ' Copy to OneDrive backup + COPY event.file_path TO "account://user@outlook.com/Backup/Reports/" + + TALK "Synced " + event.file_name + " to OneDrive" +END ON +``` + +### Monitor for Deletions + +```basic +ON CHANGE "gdrive:///archive" EVENTS "delete" + event = GET LAST "folder_change_events" + + ' Log deletion to audit table + INSERT INTO "audit_log", { + "action": "file_deleted", + "file_path": event.file_path, + "file_name": event.file_name, + "deleted_at": NOW() + } + + ' Send notification + SEND MAIL "admin@company.com", "File Deleted", " + File was deleted from archive: + Name: " + event.file_name + " + Path: " + event.file_path + " + " +END ON +``` + +### Watch for Modifications + +```basic +ON CHANGE "account://user@gmail.com/Config" EVENTS "modify" + event = GET LAST "folder_change_events" + + ' Reload configuration when config files change + IF event.file_name = "settings.json" THEN + config = READ event.file_path + SET BOT MEMORY "config", config + TALK "Configuration reloaded" + END IF +END ON +``` + +### Process Images + +```basic +ON CHANGE "account://user@gmail.com/Photos/Raw" EVENTS "create" + event = GET LAST "folder_change_events" + + ' Check if it's an image + IF INSTR(event.mime_type, "image/") > 0 THEN + ' Generate thumbnail using LLM vision + description = IMAGE event.file_path + + ' Save metadata + INSERT INTO "photos", { + "file_path": event.file_path, + "description": description, + "size": event.file_size, + "uploaded_at": NOW() + } + + TALK "Processed image: " + event.file_name + END IF +END ON +``` + +### Multi-Folder Monitoring + +```basic +' Monitor multiple folders with different handlers + +ON CHANGE "account://user@gmail.com/Invoices" + event = GET LAST "folder_change_events" + ' Process invoices + CREATE TASK "Process invoice: " + event.file_name, event.file_path, "normal" +END ON + +ON CHANGE "account://user@gmail.com/Contracts" + event = GET LAST "folder_change_events" + ' Process contracts + SEND MAIL "legal@company.com", "New Contract", "Please review: " + event.file_name +END ON + +ON CHANGE "account://user@outlook.com/Reports" EVENTS "create, modify" + event = GET LAST "folder_change_events" + ' Sync reports + COPY event.file_path TO "gdrive:///Shared/Reports/" +END ON +``` + +### Local Filesystem Monitoring + +```basic +' Monitor local upload directory +ON CHANGE "/var/www/uploads/incoming" + event = GET LAST "folder_change_events" + + ' Scan for viruses + result = RUN BASH "clamscan " + event.file_path + + IF INSTR(result, "FOUND") > 0 THEN + ' Quarantine infected file + MOVE event.file_path TO "/var/quarantine/" + SEND MAIL "security@company.com", "Virus Detected", event.file_name + ELSE + ' Move to processed + MOVE event.file_path TO "/var/www/uploads/processed/" + END IF +END ON +``` + +## Change Event Properties + +When a file change is detected, the event object contains: + +| Property | Type | Description | +|----------|------|-------------| +| `id` | UUID | Unique event identifier | +| `monitor_id` | UUID | ID of the monitor that triggered | +| `event_type` | String | Type of change (create, modify, delete, rename, move) | +| `file_path` | String | Full path to the file | +| `file_id` | String | Provider-specific file ID | +| `file_name` | String | File name without path | +| `file_size` | Integer | Size in bytes | +| `mime_type` | String | MIME type of the file | +| `old_path` | String | Previous path (for rename/move events) | + +## Database Tables + +### folder_monitors + +Stores the monitor configuration: + +| Column | Type | Description | +|--------|------|-------------| +| `id` | UUID | Monitor ID | +| `bot_id` | UUID | Bot that owns this monitor | +| `provider` | VARCHAR | Provider (gdrive, onedrive, dropbox, local) | +| `account_email` | VARCHAR | Email from account:// path | +| `folder_path` | VARCHAR | Path being monitored | +| `folder_id` | VARCHAR | Provider-specific folder ID | +| `script_path` | VARCHAR | Script to execute | +| `is_active` | BOOLEAN | Whether monitor is active | +| `watch_subfolders` | BOOLEAN | Include subfolders | +| `event_types_json` | TEXT | JSON array of event types | +| `last_change_token` | VARCHAR | Provider change token | + +### folder_change_events + +Logs detected changes: + +| Column | Type | Description | +|--------|------|-------------| +| `id` | UUID | Event ID | +| `monitor_id` | UUID | Monitor that triggered | +| `event_type` | VARCHAR | Type of change | +| `file_path` | VARCHAR | File path | +| `file_name` | VARCHAR | File name | +| `file_size` | BIGINT | Size in bytes | +| `mime_type` | VARCHAR | MIME type | +| `processed` | BOOLEAN | Whether event was processed | + +## Best Practices + +1. **Use account:// syntax** - Consistent with other file operations +2. **Filter events** - Only monitor events you need to reduce load +3. **Handle all event types** - Check `event_type` before processing +4. **Avoid circular triggers** - Moving files can trigger other monitors +5. **Process idempotently** - Events may be delivered more than once +6. **Clean up processed events** - Archive old events periodically + +## Comparison with Other Event Keywords + +| Keyword | Trigger Source | Use Case | +|---------|---------------|----------| +| `ON CHANGE` | File system changes | Sync files, process uploads | +| `ON EMAIL` | Incoming emails | Process messages, auto-reply | +| `ON INSERT` | Database inserts | React to new data | +| `SET SCHEDULE` | Time-based | Periodic tasks | +| `WEBHOOK` | HTTP requests | External integrations | + +## Related Keywords + +- [ON EMAIL](./keyword-on-email.md) - Email monitoring +- [ON](./keyword-on.md) - Database trigger events +- [SET SCHEDULE](./keyword-set-schedule.md) - Time-based scheduling +- [COPY](./keyword-copy.md) - Copy files with account:// syntax +- [MOVE](./keyword-move.md) - Move files with account:// syntax +- [USE ACCOUNT](./keyword-use-account.md) - Connect cloud accounts + +--- + +**TriggerKind:** `FolderChange = 6` diff --git a/src/06-gbdialog/keyword-on-email.md b/src/06-gbdialog/keyword-on-email.md new file mode 100644 index 00000000..4ae6fc12 --- /dev/null +++ b/src/06-gbdialog/keyword-on-email.md @@ -0,0 +1,236 @@ +# ON EMAIL + +Monitors an email address and triggers a script when new emails arrive. + +## Syntax + +```basic +' Basic - trigger on any email +ON EMAIL "address@example.com" + +' With FROM filter - only from specific sender +ON EMAIL "address@example.com" FROM "sender@example.com" + +' With SUBJECT filter - only matching subject +ON EMAIL "address@example.com" SUBJECT "pattern" +``` + +## Description + +The `ON EMAIL` keyword registers an email monitor that triggers a script whenever new emails arrive at the specified address. This is part of the event-driven programming model, similar to `SET SCHEDULE` and `ON` (for database triggers). + +When an email arrives, the system creates an entry in `email_received_events` which can be processed by your script. + +## Parameters + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| address | String | Yes | Email address to monitor | +| FROM | String | No | Filter to only trigger for emails from this sender | +| SUBJECT | String | No | Filter to only trigger for emails matching this subject pattern | + +## Examples + +### Basic Email Monitoring + +```basic +' Monitor support inbox for any incoming email +ON EMAIL "support@company.com" + email = GET LAST "email_received_events" + TALK "New support request from " + email.from_address + TALK "Subject: " + email.subject +END ON +``` + +### Filter by Sender + +```basic +' Only trigger for emails from a specific supplier +ON EMAIL "orders@company.com" FROM "supplier@vendor.com" + email = GET LAST "email_received_events" + + ' Process supplier invoice + IF email.has_attachments THEN + attachments = email.attachments + FOR EACH att IN attachments + IF att.mime_type = "application/pdf" THEN + COPY att.filename TO "account://finance@company.com/Invoices/" + END IF + NEXT + END IF +END ON +``` + +### Filter by Subject + +```basic +' Only trigger for urgent alerts +ON EMAIL "alerts@company.com" SUBJECT "URGENT" + email = GET LAST "email_received_events" + + ' Send SMS notification to on-call team + SMS "+1-555-0100", "URGENT ALERT: " + email.subject + + ' Create task for follow-up + CREATE TASK "Handle urgent alert", email.subject, "high" +END ON +``` + +### Process Attachments + +```basic +ON EMAIL "invoices@company.com" + email = GET LAST "email_received_events" + + IF email.has_attachments THEN + FOR EACH attachment IN email.attachments + ' Save to drive + filename = attachment.filename + COPY attachment TO "account://user@gmail.com/Invoices/" + filename + + ' Extract data if PDF + IF attachment.mime_type = "application/pdf" THEN + data = SAVE FROM UNSTRUCTURED attachment, "invoices" + TALK "Processed invoice: " + data.invoice_number + END IF + NEXT + END IF +END ON +``` + +### Auto-Reply System + +```basic +ON EMAIL "info@company.com" + email = GET LAST "email_received_events" + + ' Check if it's a common question + response = LLM "Classify this email and suggest a response: " + email.subject + + IF response.category = "pricing" THEN + SEND MAIL email.from_address, "RE: " + email.subject, " + Thank you for your inquiry about pricing. + Please visit our pricing page: https://company.com/pricing + + Best regards, + Company Team + " + ELSE IF response.category = "support" THEN + ' Create support ticket + CREATE TASK "Support: " + email.subject, email.from_address, "normal" + + SEND MAIL email.from_address, "RE: " + email.subject, " + Your support request has been received. + Ticket ID: " + task.id + " + We'll respond within 24 hours. + " + END IF +END ON +``` + +### Multi-Account Monitoring + +```basic +' Monitor multiple email addresses +ON EMAIL "sales@company.com" + email = GET LAST "email_received_events" + SET BOT MEMORY "lead_source", "sales_inbox" + ' Process sales inquiry +END ON + +ON EMAIL "support@company.com" + email = GET LAST "email_received_events" + SET BOT MEMORY "lead_source", "support_inbox" + ' Process support request +END ON + +ON EMAIL "billing@company.com" FROM "stripe.com" + email = GET LAST "email_received_events" + ' Process Stripe payment notifications +END ON +``` + +## Email Event Properties + +When an email is received, the event object contains: + +| Property | Type | Description | +|----------|------|-------------| +| `id` | UUID | Unique event identifier | +| `monitor_id` | UUID | ID of the monitor that triggered | +| `message_uid` | Integer | Email server message UID | +| `message_id` | String | Email Message-ID header | +| `from_address` | String | Sender email address | +| `to_addresses` | Array | List of recipient addresses | +| `subject` | String | Email subject line | +| `has_attachments` | Boolean | Whether email has attachments | +| `attachments` | Array | List of attachment objects | + +### Attachment Properties + +| Property | Type | Description | +|----------|------|-------------| +| `filename` | String | Original filename | +| `mime_type` | String | MIME type (e.g., "application/pdf") | +| `size` | Integer | Size in bytes | + +## Database Tables + +### email_monitors + +Stores the monitor configuration: + +| Column | Type | Description | +|--------|------|-------------| +| `id` | UUID | Monitor ID | +| `bot_id` | UUID | Bot that owns this monitor | +| `email_address` | VARCHAR | Address being monitored | +| `script_path` | VARCHAR | Script to execute | +| `is_active` | BOOLEAN | Whether monitor is active | +| `filter_from` | VARCHAR | Optional FROM filter | +| `filter_subject` | VARCHAR | Optional SUBJECT filter | +| `last_uid` | BIGINT | Last processed message UID | + +### email_received_events + +Logs received emails: + +| Column | Type | Description | +|--------|------|-------------| +| `id` | UUID | Event ID | +| `monitor_id` | UUID | Monitor that triggered | +| `message_uid` | BIGINT | Email server UID | +| `from_address` | VARCHAR | Sender address | +| `subject` | VARCHAR | Email subject | +| `has_attachments` | BOOLEAN | Has attachments flag | +| `processed` | BOOLEAN | Whether event was processed | + +## Best Practices + +1. **Use filters when possible** - Reduce unnecessary script executions +2. **Handle errors gracefully** - Emails may have unexpected formats +3. **Process attachments safely** - Validate MIME types before processing +4. **Avoid infinite loops** - Don't send emails that trigger your own monitors +5. **Log important actions** - Track what was processed for debugging + +## Comparison with Other Event Keywords + +| Keyword | Trigger Source | Use Case | +|---------|---------------|----------| +| `ON EMAIL` | Incoming emails | Process messages, auto-reply | +| `ON CHANGE` | File system changes | Sync files, process uploads | +| `ON INSERT` | Database inserts | React to new data | +| `SET SCHEDULE` | Time-based | Periodic tasks | +| `WEBHOOK` | HTTP requests | External integrations | + +## Related Keywords + +- [ON CHANGE](./keyword-on-change.md) - File/folder monitoring +- [ON](./keyword-on.md) - Database trigger events +- [SET SCHEDULE](./keyword-set-schedule.md) - Time-based scheduling +- [SEND MAIL](./keyword-send-mail.md) - Send emails +- [WEBHOOK](./keyword-webhook.md) - HTTP webhooks + +--- + +**TriggerKind:** `EmailReceived = 5` diff --git a/src/06-gbdialog/keyword-send-mail.md b/src/06-gbdialog/keyword-send-mail.md index 091b2582..3b3c3765 100644 --- a/src/06-gbdialog/keyword-send-mail.md +++ b/src/06-gbdialog/keyword-send-mail.md @@ -1,11 +1,12 @@ # SEND MAIL -Send email messages with optional attachments and HTML formatting. +Send email messages. ## Syntax ```basic SEND MAIL to, subject, body +SEND MAIL to, subject, body USING "account@example.com" ``` ## Parameters @@ -15,21 +16,18 @@ SEND MAIL to, subject, body | `to` | String | Recipient email address(es), comma-separated for multiple | | `subject` | String | Email subject line | | `body` | String | Email body (plain text or HTML) | +| `account` | String | (Optional) Connected account to send through | ## Description -The `SEND MAIL` keyword sends emails using the SMTP configuration defined in `config.csv`. It supports: +The `SEND MAIL` keyword sends emails using either: -- Plain text and HTML emails -- Multiple recipients -- CC and BCC (via extended syntax) -- File attachments -- Email templates -- Delivery tracking +1. **Default SMTP** - Configuration from `config.csv` +2. **Connected Account** - Send through Gmail, Outlook, etc. configured in Sources app ## Configuration -Email settings in `config.csv`: +Default SMTP in `config.csv`: ```csv name,value @@ -42,233 +40,64 @@ email-pass,smtp-password ## Examples -### Simple Text Email ```basic SEND MAIL "user@example.com", "Welcome!", "Thank you for signing up." ``` -### Multiple Recipients ```basic -recipients = "john@example.com, jane@example.com, bob@example.com" +recipients = "john@example.com, jane@example.com" SEND MAIL recipients, "Team Update", "Meeting tomorrow at 3 PM" ``` -### HTML Email ```basic body = "

Welcome!

Thank you for joining us.

" -body = body + "" SEND MAIL "user@example.com", "Getting Started", body ``` -### Dynamic Content +## USING Clause + +Send through a connected account configured in Suite → Sources → Accounts: + ```basic -order_id = GET "order_id" -subject = "Order #" + order_id + " Confirmation" -body = "Hello " + user_name + ", your order has been confirmed." -SEND MAIL user_email, subject, body +SEND MAIL "customer@example.com", "Subject", body USING "support@company.com" ``` -### With Error Handling -```basic -email = HEAR "Enter your email address:" -IF email CONTAINS "@" AND email CONTAINS "." THEN - SEND MAIL email, "Verification Code", "Your code is: 123456" - TALK "Email sent successfully!" -ELSE - TALK "Invalid email address" -END IF -``` - -### Notification System -```basic -' Send notification to admin when issue detected -admin_email = GET BOT MEMORY "admin_email" -IF error_detected THEN - subject = "Bot Alert" - body = "Issue detected at " + NOW() - SEND MAIL admin_email, subject, body -END IF -``` - -### Bulk Email with Personalization -```basic -subscribers = GET "subscribers.list" -FOR EACH email IN subscribers - body = "Dear " + username + ", here's your weekly update..." - SEND MAIL email, "Weekly Newsletter", body - WAIT 1 ' Rate limiting -NEXT -``` - -## Extended Syntax - -### With CC and BCC -```basic -' Using structured format -email_data = { - "to": "primary@example.com", - "cc": "copy@example.com", - "bcc": "hidden@example.com", - "subject": "Report", - "body": "Please review attached report." -} -SEND MAIL email_data -``` - -### With Attachments -```basic -' Attach file from drive -email_data = { - "to": "user@example.com", - "subject": "Invoice", - "body": "Please find invoice attached.", - "attachments": ["invoice.pdf"] -} -SEND MAIL email_data -``` - -### Using Templates -```basic -' Load and fill template -template = LOAD_TEMPLATE "welcome_email" -template = REPLACE(template, "{{name}}", user_name) -template = REPLACE(template, "{{date}}", TODAY()) -SEND MAIL user_email, "Welcome!", template -``` - -## Email Validation - -Always validate email addresses before sending: +The email appears from that account's address with proper authentication. ```basic -email = HEAR "Your email:" -IF email CONTAINS "@" AND email CONTAINS "." THEN - parts = SPLIT(email, "@") - IF LENGTH(parts) = 2 THEN - domain = parts[1] - IF domain CONTAINS "." THEN - SEND MAIL email, "Test", "This is a test" - ELSE - TALK "Please enter a valid email" - END IF - ELSE - TALK "Please enter a valid email" - END IF -ELSE - TALK "Please enter a valid email" -END IF +SEND MAIL "customer@example.com", "Ticket Update", "Your ticket has been resolved." USING "support@company.com" ``` ## Delivery Status -Check email delivery status: - ```basic status = SEND MAIL "user@example.com", "Test", "Message" IF status = "sent" THEN TALK "Email delivered successfully" -ELSE IF status = "queued" THEN - TALK "Email queued for delivery" -ELSE - TALK "Email delivery failed: " + status -END IF -``` - -## Rate Limiting - -Implement rate limiting to avoid spam: - -```basic -last_sent = GET BOT MEMORY "last_email_time" -IF TIME_DIFF(NOW(), last_sent) < 60 THEN - TALK "Please wait before sending another email" -ELSE - SEND MAIL email, subject, body - SET BOT MEMORY "last_email_time", NOW() -END IF -``` - -## Error Handling - -Common error scenarios: - -```basic -' Check email format before sending -IF recipient CONTAINS "@" AND recipient CONTAINS "." THEN - status = SEND MAIL recipient, subject, body - IF status = "sent" THEN - TALK "Email sent successfully" - ELSE IF status = "smtp_error" THEN - TALK "Email server is unavailable" - ELSE IF status = "auth_error" THEN - TALK "Email authentication failed" - LOG "Check SMTP credentials in config.csv" - ELSE - TALK "Failed to send email: " + status - END IF -ELSE - TALK "The email address is invalid" END IF ``` ## Best Practices -1. **Validate Recipients**: Always validate email addresses -2. **Rate Limit**: Implement delays for bulk emails -3. **Handle Failures**: Use try-catch for error handling -4. **Log Attempts**: Keep records of sent emails -5. **Test Configuration**: Verify SMTP settings before production -6. **Use Templates**: Maintain consistent formatting -7. **Respect Privacy**: Use BCC for multiple recipients -8. **Include Unsubscribe**: Add opt-out links for marketing emails - -## Security Considerations - -- Never log email passwords -- Use environment variables for sensitive data -- Implement SPF, DKIM, and DMARC for deliverability -- Sanitize user input in email bodies -- Use TLS/SSL for SMTP connections +1. Use connected accounts for better deliverability +2. Validate email addresses before sending +3. Implement delays for bulk emails +4. Handle failures gracefully ## Troubleshooting -### Email Not Sending +| Issue | Cause | Solution | +|-------|-------|----------| +| Auth failed | Invalid credentials | Check config.csv or re-authenticate account | +| Not sending | Firewall blocking | Verify port 587/465 is open | +| Going to spam | No domain auth | Configure SPF/DKIM | +| Account not found | Not configured | Add account in Suite → Sources | -1. Check SMTP configuration in `config.csv` -2. Verify firewall allows port 587/465 -3. Test credentials manually -4. Check email server logs +## See Also -### Authentication Failed - -Check SMTP configuration: -1. Verify credentials in `config.csv` -2. Ensure SMTP server allows your connection -3. Check if port 587/465 is open -4. Verify TLS/SSL settings match server requirements - -### Emails Going to Spam - -- Set proper FROM address -- Include text version with HTML -- Avoid spam trigger words -- Configure domain authentication (SPF/DKIM) - -## Related Keywords - -- [GET](./keyword-get.md) - Retrieve user data for emails -- [FORMAT](./keyword-format.md) - Format email content -- [WAIT](./keyword-wait.md) - Rate limiting between emails -- [SET SCHEDULE](./keyword-set-schedule.md) - Schedule email sending +- [USE ACCOUNT](./keyword-use-account.md) +- [WAIT](./keyword-wait.md) ## Implementation Located in `src/basic/keywords/send_mail.rs` - -The implementation uses: -- `lettre` crate for SMTP -- Async email sending -- Connection pooling for performance -- Retry logic for failed attempts -- HTML sanitization for security diff --git a/src/06-gbdialog/keyword-use-account.md b/src/06-gbdialog/keyword-use-account.md new file mode 100644 index 00000000..5f7b23e3 --- /dev/null +++ b/src/06-gbdialog/keyword-use-account.md @@ -0,0 +1,102 @@ +# USE ACCOUNT + +Activate a connected account for LLM search and file operations. + +## Syntax + +```basic +USE ACCOUNT "email@example.com" +``` + +## Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| `email` | String | Email address of connected account | + +## Description + +Enables a connected account in **mixed mode**: + +1. **LLM Search** - Account content (emails, calendar, files) becomes searchable for RAG +2. **File Operations** - Enables `account://` path notation for COPY, MOVE, etc. + +Accounts must be configured in **Suite → Sources → Accounts** before use. + +## Examples + +```basic +USE ACCOUNT "support@company.com" +``` + +```basic +USE ACCOUNT "support@company.com" +USE ACCOUNT "sales@company.com" +``` + +```basic +USE KB "product-docs" +USE ACCOUNT "support@company.com" +``` + +## Mixed Mode Behavior + +### LLM Search + +Once enabled, the LLM automatically searches the account when answering questions. + +### File Operations (account:// paths) + +```basic +USE ACCOUNT "user@gmail.com" + +COPY "account://user@gmail.com/Documents/report.pdf" TO "reports/report.pdf" +``` + +### Path Notation + +| Path | Description | +|------|-------------| +| `account://email/path` | File in connected account's drive | +| `local/path` | Local bot storage (.gbdrive) | + +## Supported Providers + +| Provider | LLM Search | File Operations | +|----------|------------|-----------------| +| Gmail | Emails, Calendar | Google Drive | +| Outlook | Emails, Calendar | OneDrive | +| Google Workspace | Full | Google Drive | +| Microsoft 365 | Full | OneDrive, SharePoint | + +## Default Accounts + +In `config.csv`: + +```csv +name,value +default-accounts,support@company.com;sales@company.com +``` + +## Prerequisites + +Configure accounts in Suite before use: + +1. Open Suite → **Sources** +2. Click **Accounts** tab +3. Click **Add Account** +4. Select provider (Gmail, Outlook, etc.) +5. Complete OAuth authentication + +## Security + +- Only accounts configured in Sources are accessible +- OAuth tokens stored securely in Vault +- Each session can only access authorized accounts +- All file access is logged for audit + +## See Also + +- [USE KB](./keyword-use-kb.md) +- [SEND MAIL](./keyword-send-mail.md) +- [Sources Sync Strategy](../08-config/sources-sync-strategy.md) \ No newline at end of file diff --git a/src/08-config/README.md b/src/08-config/README.md index d9db3ca8..80323fa3 100644 --- a/src/08-config/README.md +++ b/src/08-config/README.md @@ -109,6 +109,9 @@ theme-title,Company Assistant - [config.csv Format](./config-csv.md) - Complete reference - [LLM Configuration](./llm-config.md) - Language model settings - [Parameters](./parameters.md) - All available parameters +- [System Limits](./system-limits.md) - Rate limits, package sizes, and configurable limits +- [Sources Sync Strategy](./sources-sync-strategy.md) - Gmail, Outlook, Drive sync to vector DB +- [Secrets Management](./secrets-management.md) - Secure credential storage ---
diff --git a/src/08-config/sources-sync-strategy.md b/src/08-config/sources-sync-strategy.md new file mode 100644 index 00000000..852b8000 --- /dev/null +++ b/src/08-config/sources-sync-strategy.md @@ -0,0 +1,92 @@ +# Sources Sync Strategy + +Connect external data sources (Gmail, Outlook, Google Drive, OneDrive) to General Bots for LLM search and file operations. + +## Overview + +Configure accounts in **Suite → Sources → Accounts**. Once connected: + +1. **LLM Search** - Emails, calendar, files indexed and searchable via RAG +2. **File Operations** - Access files using `account://` path notation +3. **Email Sending** - Send through connected accounts with `SEND MAIL ... USING` + +## Supported Sources + +| Source | LLM Search | File Operations | Email Send | +|--------|------------|-----------------|------------| +| Gmail | Emails, Calendar | Google Drive | Yes | +| Outlook | Emails, Calendar | OneDrive | Yes | +| Google Workspace | Full | Google Drive | Yes | +| Microsoft 365 | Full | OneDrive, SharePoint | Yes | +| Custom IMAP | Emails | No | Yes | + +## Configuration + +All account setup is done through Suite → Sources → Accounts tab: + +1. Click **Add Account** +2. Select provider (Google, Microsoft, IMAP) +3. Complete OAuth authentication +4. Configure sync settings + +## USE ACCOUNT Keyword + +Enables an account for LLM search and file operations: + +```basic +USE ACCOUNT "support@company.com" +``` + +| Capability | Description | +|------------|-------------| +| LLM Search | Account content included in RAG queries | +| File Access | `account://` paths work for COPY, MOVE, etc. | + +## File Operations with account:// Paths + +```basic +USE ACCOUNT "user@gmail.com" + +COPY "account://user@gmail.com/Documents/report.pdf" TO "local/report.pdf" +COPY "data.xlsx" TO "account://user@gmail.com/Shared/data.xlsx" +``` + +| Keyword | Works with account:// | +|---------|----------------------| +| COPY | Yes | +| MOVE | Yes | +| DELETE | Yes | +| DIR | Yes | +| EXISTS | Yes | +| LOAD | Yes | +| SAVE | Yes | + +## Sending Email Through Accounts + +```basic +SEND MAIL "customer@example.com", "Subject", body USING "support@company.com" +``` + +## Default Accounts + +In `config.csv`: + +```csv +name,value +default-accounts,support@company.com;sales@company.com +``` + +Bot starts with these accounts enabled. + +## Security + +- OAuth tokens stored in Vault +- File permissions respected +- All access logged for audit +- Tokens auto-refresh + +## Related + +- [USE ACCOUNT Keyword](../06-gbdialog/keyword-use-account.md) +- [SEND MAIL Keyword](../06-gbdialog/keyword-send-mail.md) +- [System Limits](system-limits.md) \ No newline at end of file diff --git a/src/08-config/system-limits.md b/src/08-config/system-limits.md new file mode 100644 index 00000000..2eca9761 --- /dev/null +++ b/src/08-config/system-limits.md @@ -0,0 +1,468 @@ +# System Limits Reference + +This document provides a comprehensive reference for all system limits, rate limits, package sizes, and configurable parameters in General Bots. Each limit includes the config.csv key, default value, and the source code location where it's enforced. + +--- + +## Quick Reference + +| Category | Limit | Default | Config Key | +|----------|-------|---------|------------| +| Package Size | Total package | 100 MB | `package-max-size` | +| Package Size | Single file | 10 MB | `user-file-limit` | +| Package Size | File count | 1,000 | `user-file-count` | +| Package Size | Script size | 1 MB | `script-max-size` | +| Session | Message history | 50 | `session-message-history` | +| Session | Variable storage | 1 MB | `session-variable-limit` | +| Session | Concurrent sessions | 1,000 | `session-max-concurrent` | +| Session | Rate limit | 60/min | `session-rate-limit` | +| Session | Timeout | 30 min | `session-timeout` | +| Knowledge Base | Collections | 50 | `kb-max-collections` | +| Knowledge Base | Document size | 50 MB | `kb-doc-max-size` | +| File Upload | Per file | 10 MB | `upload-max-size` | +| File Upload | Attachment | 25 MB | `attachment-max-size` | +| API | Rate limit | 100/min | `api-rate-limit` | +| Loop Safety | Max iterations | 100,000 | `loop-max-iterations` | +| GOTO Safety | Max iterations | 1,000,000 | `goto-max-iterations` | + +--- + +## Package Size Limits + +Controls the size and composition of `.gbai` packages. + +### Total Package Size + +| Property | Value | +|----------|-------| +| **Config Key** | `package-max-size` | +| **Default** | 104,857,600 (100 MB) | +| **Unit** | Bytes | +| **Source** | `botserver/src/core/package_manager/mod.rs` | + +```csv +name,value +package-max-size,209715200 +``` + +### Single Document Size + +| Property | Value | +|----------|-------| +| **Config Key** | `user-file-limit` | +| **Default** | 10,485,760 (10 MB) | +| **Unit** | Bytes | +| **Source** | `botserver/src/core/package_manager/mod.rs` | + +### File Count Per Package + +| Property | Value | +|----------|-------| +| **Config Key** | `user-file-count` | +| **Default** | 1,000 | +| **Unit** | Files | +| **Source** | `botserver/src/core/package_manager/mod.rs` | + +### Script File Size + +| Property | Value | +|----------|-------| +| **Config Key** | `script-max-size` | +| **Default** | 1,048,576 (1 MB) | +| **Unit** | Bytes | +| **Source** | `botserver/src/basic/compiler/mod.rs` | + +--- + +## Session Limits + +Controls resource usage per user session. + +### Message History + +| Property | Value | +|----------|-------| +| **Config Key** | `session-message-history` | +| **Default** | 50 | +| **Unit** | Messages | +| **Source** | `botserver/src/core/session/mod.rs` | +| **Notes** | Messages kept in LLM context window | + +### Variable Storage + +| Property | Value | +|----------|-------| +| **Config Key** | `session-variable-limit` | +| **Default** | 1,048,576 (1 MB) | +| **Unit** | Bytes | +| **Source** | `botserver/src/core/session/mod.rs` | +| **Notes** | Total size of all session variables | + +### Concurrent Sessions + +| Property | Value | +|----------|-------| +| **Config Key** | `session-max-concurrent` | +| **Default** | 1,000 | +| **Unit** | Sessions | +| **Source** | `botserver/src/core/session/mod.rs` | +| **Notes** | Per server instance | + +### Session Rate Limit + +| Property | Value | +|----------|-------| +| **Config Key** | `session-rate-limit` | +| **Default** | 60 | +| **Unit** | Messages per minute | +| **Source** | `botserver/src/core/session/mod.rs` | + +### Session Timeout + +| Property | Value | +|----------|-------| +| **Config Key** | `session-timeout` | +| **Default** | 1,800 (30 minutes) | +| **Unit** | Seconds | +| **Source** | `botserver/src/core/session/mod.rs` | + +--- + +## Knowledge Base Limits + +Controls document ingestion and vector storage. + +### Maximum Collections + +| Property | Value | +|----------|-------| +| **Config Key** | `kb-max-collections` | +| **Default** | 50 | +| **Unit** | Collections | +| **Source** | `botserver/src/basic/keywords/kb.rs` | + +### Document Size by Type + +| File Type | Max Size | Config Key | +|-----------|----------|------------| +| PDF | 50 MB | `kb-pdf-max-size` | +| Word (.docx) | 25 MB | `kb-word-max-size` | +| Excel (.xlsx) | 25 MB | `kb-excel-max-size` | +| Text/Markdown | 10 MB | `kb-text-max-size` | +| Images | 10 MB | `kb-image-max-size` | + +### RAG Parameters + +| Config Key | Default | Description | +|------------|---------|-------------| +| `rag-top-k` | 10 | Number of chunks to retrieve | +| `rag-chunk-size` | 512 | Tokens per chunk | +| `rag-chunk-overlap` | 50 | Overlap between chunks | +| `rag-hybrid-enabled` | true | Enable hybrid search | +| `rag-rerank-enabled` | false | Enable reranking | + +--- + +## File Upload Limits + +Controls file upload operations. + +### Standard Upload + +| Property | Value | +|----------|-------| +| **Config Key** | `upload-max-size` | +| **Default** | 10,485,760 (10 MB) | +| **Unit** | Bytes | +| **Source** | `botserver/src/api/upload.rs` | + +### Email Attachment + +| Property | Value | +|----------|-------| +| **Config Key** | `attachment-max-size` | +| **Default** | 26,214,400 (25 MB) | +| **Unit** | Bytes | +| **Source** | `botserver/src/basic/keywords/send_mail.rs` | + +### Archive Extraction + +| Limit | Default | Config Key | +|-------|---------|------------| +| Archive size | 100 MB | `extract-archive-max-size` | +| Extracted size | 500 MB | `extract-output-max-size` | +| Files in archive | 10,000 | `extract-max-files` | +| Path depth | 10 | `extract-max-depth` | + +--- + +## API Rate Limits + +Controls API request rates. + +### General API + +| Property | Value | +|----------|-------| +| **Config Key** | `api-rate-limit` | +| **Default** | 100 | +| **Unit** | Requests per minute | +| **Source** | `botserver/src/api/middleware/rate_limit.rs` | + +### Endpoint-Specific Limits + +| Endpoint Category | Limit | Config Key | +|-------------------|-------|------------| +| Standard endpoints | 100/min | `api-rate-limit` | +| Compliance scans | 5/hour | `api-scan-rate-limit` | +| Report generation | 10/hour | `api-report-rate-limit` | +| LLM inference | 20/min | `llm-rate-limit` | +| Embedding | 100/min | `embedding-rate-limit` | + +### Rate Limit Headers + +All API responses include rate limit headers: + +| Header | Description | +|--------|-------------| +| `X-RateLimit-Limit` | Maximum requests allowed | +| `X-RateLimit-Remaining` | Requests remaining in window | +| `X-RateLimit-Reset` | Unix timestamp when limit resets | + +--- + +## Loop Safety Limits + +Prevents infinite loops in BASIC scripts. + +### WHILE/DO Loops + +| Property | Value | +|----------|-------| +| **Config Key** | `loop-max-iterations` | +| **Default** | 100,000 | +| **Unit** | Iterations | +| **Source** | `botserver/src/basic/keywords/procedures.rs` | + +### GOTO State Machine + +| Property | Value | +|----------|-------| +| **Config Key** | `goto-max-iterations` | +| **Default** | 1,000,000 | +| **Unit** | Iterations | +| **Source** | `botserver/src/basic/compiler/goto_transform.rs` | + +--- + +## Sandbox Limits + +Controls code execution sandbox resources. + +### Memory Limit + +| Property | Value | +|----------|-------| +| **Config Key** | `sandbox-memory-mb` | +| **Default** | 256 | +| **Unit** | Megabytes | +| **Source** | `botserver/src/basic/keywords/code_sandbox.rs` | + +### CPU Limit + +| Property | Value | +|----------|-------| +| **Config Key** | `sandbox-cpu-percent` | +| **Default** | 50 | +| **Unit** | Percent | +| **Source** | `botserver/src/basic/keywords/code_sandbox.rs` | + +### Execution Timeout + +| Property | Value | +|----------|-------| +| **Config Key** | `sandbox-timeout` | +| **Default** | 30 | +| **Unit** | Seconds | +| **Source** | `botserver/src/basic/keywords/code_sandbox.rs` | + +--- + +## Communication Limits + +### WhatsApp + +| Limit | Default | Config Key | +|-------|---------|------------| +| Messages per second | 10 | `whatsapp-rate-limit` | +| Broadcast recipients | 1,000 | `whatsapp-broadcast-max` | +| Template message size | 1,024 | `whatsapp-template-max-size` | + +### Email + +| Limit | Default | Config Key | +|-------|---------|------------| +| Recipients per email | 50 | `email-max-recipients` | +| Emails per hour | 100 | `email-rate-limit` | +| Attachment size | 25 MB | `email-attachment-max-size` | + +### Delegate to Bot + +| Property | Value | +|----------|-------| +| **Config Key** | `delegate-message-max-size` | +| **Default** | 1,048,576 (1 MB) | +| **Unit** | Bytes | +| **Source** | `botserver/src/basic/keywords/delegate_to_bot.rs` | + +| Property | Value | +|----------|-------| +| **Config Key** | `delegate-timeout` | +| **Default** | 300 | +| **Unit** | Seconds | +| **Source** | `botserver/src/basic/keywords/delegate_to_bot.rs` | + +--- + +## Storage Limits + +### User Storage Quota + +| Property | Value | +|----------|-------| +| **Config Key** | `user-storage-quota` | +| **Default** | 104,857,600 (100 MB) | +| **Unit** | Bytes | +| **Source** | `botserver/src/basic/keywords/drive.rs` | + +### Download Link Expiry + +| Property | Value | +|----------|-------| +| **Config Key** | `download-link-expiry` | +| **Default** | 86,400 (24 hours) | +| **Unit** | Seconds | +| **Source** | `botserver/src/basic/keywords/download.rs` | + +--- + +## LLM Limits + +### Token Limits + +| Config Key | Default | Description | +|------------|---------|-------------| +| `llm-max-tokens` | 4,096 | Max output tokens | +| `llm-context-window` | 8,192 | Context window size | +| `llm-temperature` | 0.7 | Default temperature | + +### Tokens Per Minute (TPM) + +| Property | Value | +|----------|-------| +| **Config Key** | `llm-tpm-limit` | +| **Default** | 20,000 | +| **Unit** | Tokens per minute | +| **Source** | `botcoder/src/main.rs` | +| **Env Var** | `LLM_TPM` | + +--- + +## A2A Protocol Limits + +### Maximum Hops + +| Property | Value | +|----------|-------| +| **Config Key** | `a2a-max-hops` | +| **Default** | 5 | +| **Unit** | Hops | +| **Source** | `botserver/src/basic/keywords/a2a_protocol.rs` | +| **Notes** | Prevents infinite delegation chains | + +--- + +## Video/Audio Limits + +### Player Limits + +| Config Key | Default | Description | +|------------|---------|-------------| +| `player-max-file-size-mb` | 100 | Max video file size | +| `player-default-volume` | 80 | Default volume (0-100) | +| `player-preload` | metadata | Preload strategy | + +--- + +## Configuring Limits + +### Via config.csv + +Add entries to your bot's `config.csv` file: + +```csv +name,value +package-max-size,209715200 +session-rate-limit,120 +api-rate-limit,200 +llm-max-tokens,8192 +``` + +### Via Environment Variables + +Some limits can be set via environment variables (overrides config.csv): + +| Environment Variable | Config Key | +|---------------------|------------| +| `LLM_TPM` | `llm-tpm-limit` | +| `SESSION_TIMEOUT` | `session-timeout` | +| `API_RATE_LIMIT` | `api-rate-limit` | + +### Via API + +Update limits programmatically: + +```basic +SET CONFIG "session-rate-limit" TO "120" +SET CONFIG "api-rate-limit" TO "200" +``` + +--- + +## Monitoring Limits + +### Viewing Current Limits + +```basic +config = GET CONFIG "api-rate-limit" +TALK "Current API rate limit: " + config +``` + +### Rate Limit Errors + +When limits are exceeded, the system returns: + +| HTTP Status | Error Code | Description | +|-------------|------------|-------------| +| 429 | `RATE_LIMITED` | Too many requests | +| 413 | `PAYLOAD_TOO_LARGE` | File/request too large | +| 507 | `INSUFFICIENT_STORAGE` | Storage quota exceeded | + +--- + +## Best Practices + +1. **Start Conservative**: Begin with default limits and increase as needed +2. **Monitor Usage**: Track rate limit headers to understand usage patterns +3. **Plan for Scale**: Increase limits gradually as traffic grows +4. **Document Changes**: Track limit changes in your bot's changelog +5. **Test Limits**: Verify your application handles limit errors gracefully + +--- + +## Related Documentation + +- [Session Management](../01-introduction/sessions.md) +- [Package Structure](../02-templates/gbai.md) +- [Knowledge Base](../03-knowledge-base/README.md) +- [API Reference](../10-rest/README.md) \ No newline at end of file diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 78b14057..76121752 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -143,6 +143,7 @@ - [BOT REFLECTION](./06-gbdialog/keyword-bot-reflection.md) - [RUN PYTHON / JAVASCRIPT / BASH](./06-gbdialog/keyword-run-code.md) - [USE KB](./06-gbdialog/keyword-use-kb.md) + - [USE ACCOUNT](./06-gbdialog/keyword-use-account.md) - [CLEAR KB](./06-gbdialog/keyword-clear-kb.md) - [USE WEBSITE](./06-gbdialog/keyword-use-website.md) - [USE TOOL](./06-gbdialog/keyword-use-tool.md) @@ -150,6 +151,8 @@ - [GET](./06-gbdialog/keyword-get.md) - [SET](./06-gbdialog/keyword-set.md) - [ON](./06-gbdialog/keyword-on.md) + - [ON EMAIL](./06-gbdialog/keyword-on-email.md) + - [ON CHANGE](./06-gbdialog/keyword-on-change.md) - [SET SCHEDULE](./06-gbdialog/keyword-set-schedule.md) - [CREATE SITE](./06-gbdialog/keyword-create-site.md) - [CREATE DRAFT](./06-gbdialog/keyword-create-draft.md) @@ -248,6 +251,8 @@ - [Drive Integration](./08-config/drive.md) - [Multimodal Configuration](./08-config/multimodal.md) - [Secrets Management](./08-config/secrets-management.md) + - [System Limits](./08-config/system-limits.md) + - [Sources Sync Strategy](./08-config/sources-sync-strategy.md) # Part IX - Tools and Integration