Commit graph

47 commits

Author SHA1 Message Date
0a24cd4b50 Fix build errors and unused imports in core, security and package_manager modules 2026-01-24 22:04:47 -03:00
66abce913f Feature gating refactor: modular compilation with minimal feature set 2026-01-22 19:45:18 -03:00
fc9c07d204 Refactor schema.rs into modular files 2026-01-22 13:57:40 -03:00
ee9341163f Add PRODUCTS, PRODUCT, SEARCH PRODUCTS keywords for ERP integration 2026-01-14 10:20:07 -03:00
1c4cc2f986 Add SEARCH keyword and store tools: classify-product, search-products, calculate-shipping 2026-01-14 10:13:33 -03:00
a2783f9b32 Fix 5 errors and 32 warnings: calendar, compliance, billing_alert_broadcast, unused vars 2026-01-13 22:21:25 -03:00
c67aaa677a feat(security): Complete security infrastructure implementation
SECURITY MODULES ADDED:
- security/auth.rs: Full RBAC with roles (Anonymous, User, Moderator, Admin, SuperAdmin, Service, Bot, BotOwner, BotOperator, BotViewer) and permissions
- security/cors.rs: Hardened CORS (no wildcard in production, env-based config)
- security/panic_handler.rs: Panic catching middleware with safe 500 responses
- security/path_guard.rs: Path traversal protection, null byte prevention
- security/request_id.rs: UUID request tracking with correlation IDs
- security/error_sanitizer.rs: Sensitive data redaction from responses
- security/zitadel_auth.rs: Zitadel token introspection and role mapping
- security/sql_guard.rs: SQL injection prevention with table whitelist
- security/command_guard.rs: Command injection prevention
- security/secrets.rs: Zeroizing secret management
- security/validation.rs: Input validation utilities
- security/rate_limiter.rs: Rate limiting with governor crate
- security/headers.rs: Security headers (CSP, HSTS, X-Frame-Options)

MAIN.RS UPDATES:
- Replaced tower_http::cors::Any with hardened create_cors_layer()
- Added panic handler middleware
- Added request ID tracking middleware
- Set global panic hook

SECURITY STATUS:
- 0 unwrap() in production code
- 0 panic! in production code
- 0 unsafe blocks
- cargo audit: PASS (no vulnerabilities)
- Estimated completion: ~98%

Remaining: Wire auth middleware to handlers, audit logs for sensitive data
2025-12-28 19:29:18 -03:00
14b7cf70af feat(autotask): Implement AutoTask system with intent classification and app generation
- Add IntentClassifier with 7 intent types (APP_CREATE, TODO, MONITOR, ACTION, SCHEDULE, GOAL, TOOL)
- Add AppGenerator with LLM-powered app structure analysis
- Add DesignerAI for modifying apps through conversation
- Add app_server for serving generated apps with clean URLs
- Add db_api for CRUD operations on bot database tables
- Add ask_later keyword for pending info collection
- Add migration 6.1.1 with tables: pending_info, auto_tasks, execution_plans, task_approvals, task_decisions, safety_audit_log, generated_apps, intent_classifications, designer_changes
- Write apps to S3 drive and sync to SITE_ROOT for serving
- Clean URL structure: /apps/{app_name}/
- Integrate with DriveMonitor for file sync

Based on Chapter 17 - Autonomous Tasks specification
2025-12-27 21:10:09 -03:00
5da86bbef2 Fix clippy warnings: match arms, async/await, Debug impls, formatting
- Fix match arms with identical bodies by consolidating patterns
- Fix case-insensitive file extension comparisons using eq_ignore_ascii_case
- Fix unnecessary Debug formatting in log/format macros
- Fix clone_from usage instead of clone assignment
- Fix let...else patterns where appropriate
- Fix format! append to String using write! macro
- Fix unwrap_or with function calls to use unwrap_or_else
- Add missing fields to manual Debug implementations
- Fix duplicate code in if blocks
- Add type aliases for complex types
- Rename struct fields to avoid common prefixes
- Various other clippy warning fixes

Note: Some 'unused async' warnings remain for functions that are
called with .await but don't contain await internally - these are
kept async for API compatibility.
2025-12-26 08:59:25 -03:00
883c6d07e1 Remove all code comments and fix ratatui version 2025-12-23 18:40:58 -03:00
a491cc13a6 Update botserver 2025-12-21 23:40:43 -03:00
57bf5b8754 feat: add ON EMAIL and ON CHANGE keywords for event-driven monitoring
- Add ON EMAIL keyword with FROM/SUBJECT filters
- Add ON CHANGE keyword with account:// syntax (gdrive, onedrive, dropbox, local)
- Add TriggerKind::EmailReceived (5) and FolderChange (6)
- Add migration 6.1.3_bot_hierarchy_monitors with:
  - email_monitors, folder_monitors tables
  - email_received_events, folder_change_events tables
  - user_organizations table
  - Bot hierarchy: parent_bot_id, enabled_tabs_json, inherit_parent_config
- Add 26 unit tests (12 on_email, 12 on_change, 2 trigger_kind)
- Update PROMPT.md with weekly maintenance checklist
- Zero warnings, zero errors
2025-12-18 16:17:58 -03:00
fda88c0d94 Bootstrap started! 6.1.0
- Add rss and scraper dependencies for web data keywords
- Add SMS keyword with priority support (low, normal, high, urgent)
- Add web_data keywords: RSS, SCRAPE, SCRAPE_ALL, SCRAPE_TABLE, SCRAPE_LINKS, SCRAPE_IMAGES
- Add ai_tools keywords: TRANSLATE, OCR, SENTIMENT, CLASSIFY
- Improve LLM server health check with better diagnostics and increased timeout
- Fix compilation errors and warnings
- Register SMS keywords in BASIC engine
2025-12-10 18:22:02 -03:00
5d93505012 feat(attendance): Add LLM-assisted attendant features
- Real-time tips when customer messages arrive
- Message polishing with one click
- Smart reply generation (3 contextual suggestions)
- Auto-summary when attendant takes conversation
- LLM-powered sentiment analysis with escalation warnings

WhatsApp Attendant Commands:
- /queue, /take, /status, /transfer, /resolve
- /tips, /polish, /replies, /summary, /help
- Portuguese versions: /fila, /pegar, /dicas, /polir, /respostas, /resumo

Config options (config.csv):
- attendant-llm-tips
- attendant-polish-message
- attendant-smart-replies
- attendant-auto-summary
- attendant-sentiment-analysis

API Endpoints:
- POST /api/attendance/llm/tips
- POST /api/attendance/llm/polish
- POST /api/attendance/llm/smart-replies
- GET /api/attendance/llm/summary/{session_id}
- POST /api/attendance/llm/sentiment
- GET /api/attendance/llm/config/{bot_id}

Uses bot's system prompt for consistency between bot and human-assisted responses.
2025-12-05 13:47:15 -03:00
6da93f6c36 feat: unified keywords with spaces, ON ERROR RESUME NEXT, unified DELETE
Keywords now use spaces instead of underscores:
- SEND MAIL (was SEND_MAIL)
- GENERATE PDF (was GENERATE_PDF)
- MERGE PDF (was MERGE_PDF)
- SET HEADER (was SET_HEADER)
- CLEAR HEADERS (was CLEAR_HEADERS)

New ON ERROR RESUME NEXT implementation:
- ON ERROR RESUME NEXT - enable error trapping
- ON ERROR GOTO 0 - disable error trapping
- CLEAR ERROR - clear error state
- ERROR MESSAGE - get last error message
- ERR - get error number

Unified DELETE keyword:
- DELETE url - HTTP DELETE (auto-detected)
- DELETE table, filter - Database DELETE
- DELETE path - File DELETE

Changes:
- errors/on_error.rs: New VB-style error handling
- errors/mod.rs: Include on_error module
- send_mail.rs: SEND MAIL with spaces
- file_operations.rs: GENERATE PDF, MERGE PDF with spaces
- data_operations.rs: Unified DELETE with auto-detection
- http_operations.rs: Cleaned up DELETE HTTP
- compiler/mod.rs: Removed underscore normalization
- mod.rs: Updated command list
2025-12-05 09:55:13 -03:00
340ac8840a - New templates. 2025-12-03 07:15:54 -03:00
c4c9521dd9 - Split into botui. 2025-12-02 21:09:43 -03:00
c6bbd50946 , dt.month, dt.hour, dt.is_weekend, etc.)
- Add startup wizard module for first-run configuration
- Add white-label branding system with .product file support
- Add bot manager for lifecycle, MinIO buckets, and templates
- Add version tracking registry for component updates
- Create comparison doc: BASIC vs n8n/Zapier/Make/Copilot
- Add WhatsApp-style sample dialogs to template documentation
- Add data traceability SVG diagram ```
2025-11-30 15:07:29 -03:00
06c75cb690 Looking at this diff, I can see it's a comprehensive documentation
update and code refactoring focused on:

1. Adding new documentation pages to the table of contents
2. Restructuring the bot templates documentation
3. Changing keyword syntax from underscore format to space format (e.g.,
   `SET_BOT_MEMORY` → `SET BOT MEMORY`)
4. Updating compiler and keyword registration to support the new
   space-based syntax
5. Adding new keyword modules (social media, lead scoring, templates,
   etc.)

Refactor BASIC keywords to use spaces instead of underscores

Change keyword syntax from underscore format (SET_BOT_MEMORY) to more
natural space-separated format (SET BOT MEMORY) throughout the codebase.

Key changes:
- Update Rhai custom syntax registration to use space tokens
- Simplify compiler preprocessing (fewer replacements needed)
- Update all template .bas files to use new syntax
- Expand documentation with consolidated examples and new sections
- Add new keyword modules: social_media, lead_scoring, send_template,
  core_functions, qrcode, sms, procedures, import_export, llm_macros,
  on_form_submit
2025-11-30 10:53:59 -03:00
d1e3c25eb8 Add tar/flate2 deps and document new BASIC keywords
Add flate2 and tar dependencies for archive extraction support in file
operations. Update documentation with:

- New BASIC keywords: SWITCH/CASE, WEBHOOK, INSTR, IS_NUMERIC
- HTTP operations: POST, PUT, PATCH, DELETE_HTTP, GRAPHQL, SOAP
- Data operations: SAVE, INSERT, UPDATE, DELETE, MERGE, FILTER, etc.
- File operations: READ, WRITE, COMPRESS, EXTRACT, GENERATE_PDF, etc.

Simplify README and add appendices for external services and environment
variables. Add monitoring dashboard and player UI docs.
2025-11-30 07:53:58 -03:00
39bc972855 Add multimodal module for botmodels integration
Introduces IMAGE, VIDEO, AUDIO, and SEE keywords for BASIC scripts that
connect to the botmodels service for AI-powered media generation and
vision/captioning capabilities.

- Add BotModelsClient for HTTP communication with botmodels service
- Implement BASIC keywords: IMAGE, VIDEO, AUDIO (generation), SEE
  (captioning)
- Support configuration via config.csv for models
2025-11-29 20:40:08 -03:00
4b5d18db71 Add metadata and refactor BASIC compiler
- Add package metadata (keywords, categories) to Cargo.toml
- Add #[must_use] attributes to constructor methods
- Simplify conditional logic by inverting if-else blocks
- Replace string formatting with interpolation syntax
2025-11-27 15:19:17 -03:00
f40cb6c7b4 Fix typos in bot file extensions and keyword names
Changed incorrect references to .vbs files to .bas and corrected
USE_WEBSITE keyword naming. Also added missing fields to API response
structure and clarified that start.bas is optional for bots.
2025-11-26 22:54:22 -03:00
e013f3f800 - Even more keywords. 2025-11-22 12:26:16 -03:00
20696dd64f - New templates. 2025-11-22 01:27:29 -03:00
a39bfce495 - New stuff, 6.1. 2025-11-21 23:23:53 -03:00
7356b20e01 @media (prefers-color-scheme: dark)
-  Enhanced accessibility features (focus states, reduced motion)
-  Added connection status component styles
-  Improved responsive design
-  Added utility classes for common patterns

-  Added semantic HTML5 elements (`<header>`, `<main>`, `<nav>`)
-  Comprehensive ARIA labels and roles for accessibility
-  Keyboard navigation support (Alt+1-4 for sections, Esc for menus)
-  Better event handling and state management
-  Theme change subscriber with meta theme-color sync
-  Online/offline connection monitoring
-  Enhanced console logging with app info

-  `THEMES.md` (400+ lines) - Complete theme system guide
-  `README.md` (433+ lines) - Main application documentation
-  `COMPONENTS.md` (773+ lines) - UI component library reference
-  `QUICKSTART.md` (359+ lines) - Quick start guide for developers
-  `REBUILD_NOTES.md` - This summary document

**Theme files define base colors:** ```css :root { --primary: 217 91%
60%; /* HSL: blue */ --background: 0 0% 100%; /* HSL: white */ } ```

**App.css bridges to working variables:** ```css :root { --accent-color:
hsl(var(--primary)); --primary-bg: hsl(var(--background));
--accent-light: hsla(var(--primary) / 0.1); } ```

**Components use working variables:** ```css .button { background:
var(--accent-color); color: hsl(var(--primary-foreground)); } ```

-  Keyboard shortcuts (Alt+1-4, Esc)
-  System dark mode detection
-  Theme change event subscription
-  Automatic document title updates
-  Meta theme-color synchronization
-  Enhanced console logging
-  Better error handling
-  Improved accessibility

-  Theme switching via dropdown
-  Theme persistence to localStorage
-  Apps menu with section switching
-  Dynamic section loading (Chat, Drive, Tasks, Mail)
-  WebSocket chat functionality
-  Alpine.js integration for other modules
-  Responsive design
-  Loading states

- [x] Theme switching works across all 19 themes
- [x] All sections load correctly
- [x] Keyboard shortcuts functional
- [x] Responsive on mobile/tablet/desktop
- [x] Accessibility features working
- [x] No console errors
- [x] Theme persistence works
- [x] Dark mode detection works

``` documentation/ ├── README.md # Main docs - start here ├──
QUICKSTART.md # 5-minute guide ├── THEMES.md # Theme system details ├──
COMPONENTS.md # UI component library └── REBUILD_NOTES.md # This summary
```

1. **HSL Bridge System**: Allows theme files to use shadcn-style HSL
   variables while the app automatically derives working CSS properties
2. **No Breaking Changes**: All existing functionality preserved and
   enhanced
3. **Developer-Friendly**: Comprehensive documentation for customization
4. **Accessibility First**: ARIA labels, keyboard navigation, focus
   management
5. **Performance Optimized**: Instant theme switching, minimal reflows

- **Rebuild**:  Complete
- **Testing**:  Passed
- **Documentation**:  Complete
- **Production Ready**:  Yes

The rebuild successfully integrates the theme system throughout the UI
while maintaining all functionality and adding comprehensive
documentation for future development.
2025-11-21 09:28:02 -03:00
baea0b942c feat: refactor prompt compaction and clean up test files
- Renamed `execute_compact_prompt` to `compact_prompt_for_bots` and simplified logic
- Removed redundant comments and empty lines in test files
- Consolidated prompt compaction threshold handling
- Cleaned up UI logging implementation by removing unnecessary whitespace
- Improved code organization in ui_tree module

The changes focus on code quality improvements, removing clutter, and making the prompt compaction logic more straightforward. Test files were cleaned up to be more concise.
2025-11-11 18:32:52 -03:00
2d6f214193 feat: remove unused dependencies and clean up Cargo.lock
Removed several unused dependencies from Cargo.lock including:
- auto_generate_cdp
- headless_chrome
- scraper
- cssparser and related crates
- dtoa and dtoa-short
- string_cache and related crates
- tendril
- tungstenite 0.27.0

Also updated ureq dependency to single version (removed duplicate entry). This cleanup reduces the dependency tree and removes unused code.
2025-11-05 13:46:03 -03:00
5b08bd887e feat(compiler): clean up stale schedules on script recompilation
Added tracking of previously scheduled scripts using a `HashSet` and initialized it in `BasicCompiler::new`. Updated `compile_file` and `preprocess_basic` to require mutable access, allowing schedule cleanup before processing. Implemented logic to delete existing scheduled automations for a script using Diesel queries, ensuring old schedules are removed when a script is recompiled without a `SET_SCHEDULE`. Added necessary Diesel imports and `TriggerKind` reference. This prevents duplicate or orphaned scheduled tasks.
2025-11-05 10:15:36 -03:00
7f1e6dc91c feat: refactor auth and models, update LLM fallback strategy
- Simplified auth module by removing unused imports and code
- Cleaned up shared models by removing unused structs (Organization, User, Bot, etc.)
- Updated add-req.sh to comment out unused directories
- Modified LLM fallback strategy in README with additional notes about model behaviors

The changes focus on removing unused code and improving documentation while maintaining existing functionality. The auth module was significantly reduced by removing redundant code, and similar cleanup was applied to shared models. The build script was adjusted to reflect currently used directories.
2025-11-04 23:11:33 -03:00
49d6debc83 feat(automation): add unique constraint and refactor action execution
- Added UNIQUE constraint on system_automations (bot_id, kind, param) to prevent duplicate automations
- Refactored execute_action to accept full Automation struct instead of just param
- Simplified bot name resolution by using automation.bot_id directly
- Improved error handling in action execution with proper error propagation
- Removed redundant bot name lookup logic by leveraging existing bot_id
2025-11-02 20:57:53 -03:00
3972863911 refactor(state): rename resource clients and improve keyword syntax
Updated references from `redis_client`, `s3_client`, and `custom_conn` to unified names `cache`, `drive`, and `conn` for consistency across modules. Adjusted `add_suggestion_keyword` to use clearer parameter naming and enhanced custom syntax registration for better readability and maintainability.
2025-11-01 20:53:45 -03:00
01351f60fd feat(auth): add suggestion support and clean up code
- Added new ADD_SUGGESTION keyword handler to support sending suggestions in responses
- Removed unused env import in hear_talk module
- Simplified bot_id assignment to use static string
- Added suggestions field to BotResponse struct
- Improved SET_CONTEXT keyword to take both name and value parameters
- Fixed whitespace in auth handler
- Enhanced error handling for suggestion sending

The changes improve the suggestion system functionality while cleaning up unused code and standardizing response handling.
2025-10-31 20:55:13 -03:00
866fe9049f feat(keywords): add suggestion keyword to ScriptService initialization 2025-10-30 09:09:15 -03:00
098c6464e3 - Added logic to Knowledge management. 2025-10-18 18:19:08 -03:00
09a9c8f3cd Enhance bot memory and Redis guards
- Derive bot_id from BOT_GUID env var
- Guard concurrent runs with Redis
- Read CACHE_URL for Redis connection
- Extend bot memory keyword to accept comma as separator
- Increase LLM timeouts to 180s (local and legacy)
- Update templates to use bot memory (GET_BOT_MEMORY/SET_BOT_MEMORY)
- Fix start script path to announcements.gbai
2025-10-16 14:22:28 -03:00
d584f8dfc3 Refactor async GET and LLM, add connection UI
- Execute GET requests in a dedicated thread with its own Tokio runtime,
  add timeout handling and clearer error messages.
- Tighten `is_safe_path` checks and simplify HTTP/S3 logic.
- Change `llm_keyword` to accept `Arc<AppState>`, add prompt builder,
  run LLM generation in an isolated thread with timeout.
- Update keyword registration call in `basic/mod.rs`.
- Convert template script to use `let` declarations and return a
  boolean.
- Introduce connection‑status indicator in the web UI with styles,
  automatic reconnection attempts, and proper WS/WSS handling for voice.
2025-10-15 21:18:01 -03:00
bb9c733fd5 - GET ketyowrd for buckets. 2025-10-15 12:45:15 -03:00
16516b23bb Refactor server code and add auth API fixes 2025-10-14 13:51:27 -03:00
cd33995435 New features for start.bas 2025-10-13 18:11:03 -03:00
3d0e8df36e - New features for start.bas 2025-10-13 17:43:03 -03:00
9efeffd80a Refactor to Arc<AppState> for shared state
- Migrate core services to store Arc<AppState> and use locks
- Centralize state in AppState with Arc-wrapped managers
- Update handlers to pass Arc<AppState> via web::Data
- Add Default for AppState and initialize components in main
- Update debug.json program path from gbserver to botserver
2025-10-12 20:12:49 -03:00
a16d9affe7 - main.rs is compiling again. 2025-10-11 20:02:14 -03:00
a1dd7b5826 - Remove all compilation errors. 2025-10-11 12:29:03 -03:00
d1a8185baa - Remove of all warnings. 2025-10-07 10:53:09 -03:00
6d58491281 Migration to Rust and free from Azure. 2025-10-06 10:30:17 -03:00