Commit graph

3358 commits

Author SHA1 Message Date
b033be3e64 refactor(automation, llm): simplify service init and prompt handling
- Remove `scripts_dir` from `AutomationService` and its constructor, as it was unused.
- Drop LLM and embedding server readiness checks; the service now only schedules periodic tasks.
- Increase the health‑check interval from 5 seconds to 15 seconds for reduced load.
- Streamline the LLM keyword prompt to a concise `"User: {}"` format, removing verbose boilerplate.
- Remove unnecessary logging and LLM cache handling code from the bot orchestrator, cleaning up unused environment variable checks and cache queries.
2025-11-03 08:23:03 -03:00
a5bfda4d09 feat(automation): add LLM server readiness check and improve user session handling
- Added LLM server readiness check in AutomationService before starting tasks
- Renamed `user` parameter to `user_session` in execute_talk for clarity
- Updated BotResponse fields to use user_session data instead of hardcoded values
- Improved Redis key generation in execute_talk to use user_session fields
- Removed commented Redis code in set_current_context_keyword

The changes ensure proper initialization of automation tasks by checking LLM server availability first, and improve code clarity by using more descriptive variable names for user session data.
2025-11-02 23:54:14 -03:00
1d7d0e10c0 feat(automation): add unique constraint and refactor action execution
- Added UNIQUE constraint on system_automations (bot_id, kind, param) to prevent duplicate automations
- Refactored execute_action to accept full Automation struct instead of just param
- Simplified bot name resolution by using automation.bot_id directly
- Improved error handling in action execution with proper error propagation
- Removed redundant bot name lookup logic by leveraging existing bot_id
2025-11-02 20:57:53 -03:00
d2ee695d8b feat: add bot_id to system_automations and enhance schedule handling
- Introduced `bot_id` column in `system_automations` table (migration 6.0.0.sql) and updated the Diesel schema/model to include it.
- Adjusted migrations to remove the hard‑coded “Update Summary” automation and only create an index on the `name` column.
- Extended the `SET_SCHEDULE` keyword:
  - Added a second string argument for the script name.
  - Passed the invoking user's `bot_id` to the database layer.
  - Updated function signature to accept a full `UserSession` instead of discarding it.
- Modified `execute_set_schedule` to store `bot_id`, script name, and activation flag; added conflict handling on `(bot_id, param)` to update schedule and reset trigger state.
- Updated imports and logging to reflect new parameters.

These changes enable per‑bot automation management, allow specifying the script to run, and improve idempotent schedule updates.
2025-11-02 19:32:25 -03:00
0f842adf69 feat(automation): update logging and script handling
- Update RUST_LOG environment variable in launch.json to exclude more verbose crates
- Change automation kind from 3 to 0 in migrations/6.0.5.sql
- Replace debug with trace logging for schedule evaluation in automation service
- Modify script path resolution to use .ast extension instead of .bas
- Improve error message when script reading fails
- Remove MinIO download fallback for scripts (simplifying script handling)

These changes improve logging verbosity control and simplify script handling by removing the MinIO fallback mechanism. The trace-level
2025-11-02 19:03:52 -03:00
fea1574518 feat: enforce config load errors and add dynamic LLM model handling
- Updated `BootstrapManager` to use `AppConfig::from_env().expect(...)` and `AppConfig::from_database(...).expect(...)` ensuring failures are explicit rather than silently ignored.
- Refactored error propagation in bootstrap flow to use `?` where appropriate, improving reliability of configuration loading.
- Added import of `llm_models` in `bot` module and introduced `ConfigManager` usage to fetch the LLM model identifier at runtime.
- Integrated dynamic LLM model handler selection via `llm_models::get_handler(&model)`.
- Replaced static environment variable retrieval for embedding configuration with runtime
2025-11-02 18:36:21 -03:00
e7fa2f7bdb feat(llm): add cancel_job support and integrate session cleanup
Introduce a new `cancel_job` method in the `LLMProvider` trait to allow cancellation of ongoing LLM tasks. Implement no-op versions for OpenAI, Anthropic, and Mock providers. Update the WebSocket handler to invoke job cancellation when a session closes, ensuring better resource management and preventing orphaned tasks. Also, fix unused variable warning in `add_suggestion.rs`.
2025-11-02 15:13:47 -03:00
87fc13b08a feat(keywords): add CLEAR_SUGGESTIONS command and implement in dialog
Added new CLEAR_SUGGESTIONS keyword command that clears user suggestions from Redis cache. Implemented the command in the announcements dialog start script to prevent duplicate suggestions. The command handles Redis connection errors gracefully and logs appropriate debug information.
2025-11-02 14:21:38 -03:00
f2beef5d86 feat(logging): replace debug with trace for verbose operations
Changed debug logging to trace level for Redis suggestion operations and drive monitor object listings. This better reflects the verbosity level of these operations since they are more detailed debugging information. Also updated "Redis client" to "Cache client" in a debug message for consistency with the broader cache abstraction.
2025-11-02 14:16:35 -03:00
4bb3664dfd feat(automation): improve logging and bot path resolution
- Updated RUST_LOG environment variable in launch.json to include more detailed debug logging configuration
- Changed trace! logs to debug! in AutomationService for better visibility
- Replaced environment variable usage with get_default_bot helper function
- Improved bot path resolution by using bot name from database
- Added error handling for bot name query
- Simplified S3 bucket name generation using get_default_bot
- Removed unused imports and environment variable dependencies
2025-11-02 14:08:49 -03:00
6244c99854 feat: include context length info in BotResponse messages
Add `context_length` and `context_max_length` fields to the `BotResponse` struct across the codebase.
Initialize these fields with default values (0) for existing response constructions and populate them using `ConfigManager` to retrieve the configured maximum context size for each bot.
Import `ConfigManager` in `bot/mod.rs` to access configuration.
These changes enable tracking of the current and maximum context sizes in bot responses, supporting future features that rely on context management.
2025-11-02 12:46:37 -03:00
6e36384f45 feat: simplify message content in hear_talk function
Removed the redundant "I heard: " prefix from the message content in the execute_talk function. The message is now stored as-is without additional formatting, making it more straightforward and consistent with other message handling.
2025-11-02 12:28:07 -03:00
a5d3343c09 feat(ui): update design system with new theme and visual enhancements
- Replace Orbitron/Inter fonts with Space Grotesk/JetBrains Mono
- Implement new color scheme with primary, secondary, and accent variables
- Add radial gradient background with animated glow effects
- Introduce grain overlay for texture
- Redesign sidebar with improved transitions and blur effects
- Update empty state styling and remove neon effects
- Optimize layout spacing and shadows
- Streamline API call in chat initialization

The changes modernize the UI with a more cohesive design language, better visual hierarchy, and improved readability while maintaining functionality. The new theme features a dark color palette with vibrant accents and subtle textures for depth.
2025-11-02 11:13:40 -03:00
9673f41195 feat: add styled suggestion buttons and move them from messages to footer
Added new CSS classes for suggestion buttons and container, moving them from message content to footer. Removed inline styles in favor of CSS classes for better maintainability. The suggestions now appear at the top of the footer with consistent styling and hover effects.
2025-11-02 11:01:36 -03:00
9010396a87 feat(ui): improve suggestion handling and input clearing
Enhanced suggestion handling to include empty state in message selection and improved UX by:
- Clearing input field after selecting a suggestion
- Adding selected suggestion as user message to chat
- Maintaining all existing functionality while simplifying the flow

Changes ensure more intuitive user interaction with suggestions while keeping the chat interface clean.
2025-11-02 10:51:42 -03:00
888bfc859d feat: refactor Redis operations to synchronous in add_suggestion
Changed async Redis operations to synchronous in add_suggestion_keyword function. Removed unnecessary async/await and tokio::spawn since the operations are now blocking. This simplifies the code while maintaining the same functionality of storing suggestions and context state in Redis. Error handling remains robust with proper early returns.
2025-11-02 10:45:57 -03:00
acca68493f feat: simplify Redis key format and clean up code
- Remove comments from launch.json configuration
- Simplify Redis key format in set_current_context_keyword by removing context_name
- Remove unused /api/start endpoint and related session start logic
- Change log level from info to trace for config sync messages
- Add trace logging to drive_monitor module

The changes focus on code cleanup and simplification, particularly around session handling and logging. The Redis key format was simplified as the context_name was deemed unnecessary for uniqueness.
2025-11-02 07:50:57 -03:00
0342e1cac9 refactor(state): rename resource clients and improve keyword syntax
Updated references from `redis_client`, `s3_client`, and `custom_conn` to unified names `cache`, `drive`, and `conn` for consistency across modules. Adjusted `add_suggestion_keyword` to use clearer parameter naming and enhanced custom syntax registration for better readability and maintainability.
2025-11-01 20:53:45 -03:00
fda0b0c9e8 feat(auth): add bot name lookup and suggestion support in bot responses
Implemented a new `bot_from_name` method in `AuthService` to retrieve a bot's UUID by its name, enabling explicit bot selection via a `bot_name` query parameter in the authentication endpoint. Updated `auth_handler` to prioritize this parameter, fall back to the first active bot when necessary, and handle cases where no active bots exist.

Extended `BotOrchestrator` to import the `Suggestion` model and fetch suggestion data from Redis for each user session. Integrated these suggestions into the `BotResponse` payload, ensuring clients receive relevant suggestions alongside bot messages.

These changes improve bot selection flexibility and enrich the response data with contextual suggestions.
2025-11-01 17:56:53 -03:00
d6fcc346fc refactor(bot): reorganize imports, improve logging and formatting
- Reordered and deduplicated `use` statements, adding missing imports for `langcache`, `DriveMonitor`, `generate_embeddings`, `Qdrant` client, and `chrono::Utc`.
- Moved Diesel prelude import into the function scope where it is used to limit its visibility.
- Refactored async task spawning to handle errors more clearly and consistently.
- Enhanced string formatting for bucket names and log messages, introducing multiline `warn!` and `error!` calls for better readability.
- Applied consistent code style (spacing, line breaks) across the module to improve maintainability.
2025-11-01 15:16:43 -03:00
4e781b1815 feat(drive_monitor): restart LLaMA servers on llm- config changes
Add logic to detect changes in llm-related configuration properties when syncing gbot configs. If any llm- keys differ from stored values, automatically restart LLaMA servers to apply updates. This ensures model servers stay in sync with configuration changes without unnecessary restarts.
2025-11-01 14:23:40 -03:00
6d68585c71 refactor(config): replace raw SQL with Diesel query for bot config
Updated `ConfigManager::get_config` to use Diesel query builder instead of raw SQL for improved safety and maintainability. Adjusted parameter naming and integrated schema references. Also refactored `ensure_llama_servers_running` to fetch configuration from the database using `AppState` and `ConfigManager`. Removed unused imports in bootstrap module.
2025-11-01 11:48:46 -03:00
7170663b6f feat: remove unused bot config update logic from bootstrap
The commit removes the unused bot configuration update functionality from the bootstrap manager. This includes:
1. Removing the fallback logic in component installation that would update bot configs
2. Removing the entire update_bot_config method which was used to insert/update configuration values

The code was likely removed because this functionality is no longer needed or has been moved elsewhere in the codebase. The removal simplifies the bootstrap process by eliminating unused database operations.
2025-11-01 09:50:28 -03:00
122c839818 feat: add bot ID to DriveMonitor and simplify config sync
- Added bot_id parameter to DriveMonitor::new() and pass it through BotOrchestrator
- Modified DriveMonitor to store bot_id as a field
- Simplified ConfigManager::sync_gbot_config() to accept content directly instead of file path
- Removed file reading logic from sync_gbot_config
- Cleaned up logging messages in config sync
- Improved error handling and logging in DriveMonitor's gbot check

These changes better associate monitoring with specific bots and make the config sync more flexible by accepting content directly rather than requiring a file path.
2025-11-01 09:38:15 -03:00
96c4283d29 feat(config): refactor config loading and remove unused code
- Removed database-based config loading from server_configuration table
- Simplified AppConfig::from_database to use empty HashMap since config is now loaded from bot_configuration table via drive_monitor
- Removed unused load_config_from_db function
- Removed config sync hash checking logic from ConfigManager
- Cleaned up config update logic in ConfigManager

The changes reflect the shift to loading configuration from the bot_configuration table instead of server_configuration, removing now-unused code paths and simplifying the configuration loading process.
2025-11-01 09:18:02 -03:00
eb647530bf feat(config): refactor AI config into separate LLM and embedding configs
- Split AIConfig into separate LLMConfig and embedding config structs
- Update create_site.rs to use config.llm instead of config.ai
- Improve config loading comments in bootstrap manager
- Add new LLM-related environment variables with defaults
- Maintain backward compatibility with existing config loading
- Clean up unused AIConfig struct and related code

The change better organizes the AI-related configuration by separating LLM and embedding configurations, making the code more maintainable and flexible for future AI service integrations.
2025-11-01 08:43:14 -03:00
a19ba2bef7 feat: cleanup unused imports in bootstrap module
- Remove unused diesel::Connection import
- Rename unused database_url variable to _database_url
- Mark unused config parameter in upload_templates_to_drive with underscore
- Maintain same functionality while cleaning up compiler warnings
2025-11-01 07:25:36 -03:00
8bf347a9a2 feat: refactor database connection handling and add LLM component
- Replace direct database connection establishment with shared `establish_pg_connection` utility
- Add "llm" to required components list in bootstrap manager
- Lower default RUST_LOG level from debug to info in VSCode config
- Clean up imports and connection error messages
- Remove hardcoded database URL strings in favor of centralized connection handling

The changes improve code maintainability by centralizing database connection logic and adding support for the new LLM component in the bootstrap process.
2025-11-01 07:20:04 -03:00
6f59cdaab6 feat(auth): add suggestion support and clean up code
- Added new ADD_SUGGESTION keyword handler to support sending suggestions in responses
- Removed unused env import in hear_talk module
- Simplified bot_id assignment to use static string
- Added suggestions field to BotResponse struct
- Improved SET_CONTEXT keyword to take both name and value parameters
- Fixed whitespace in auth handler
- Enhanced error handling for suggestion sending

The changes improve the suggestion system functionality while cleaning up unused code and standardizing response handling.
2025-10-31 20:55:13 -03:00
683955a4a4
Merge pull request #445 from ChristopherCastilho/main
Add_suggestion fix
2025-10-31 16:05:24 -03:00
ea765811a2
Merge branch 'main' into main 2025-10-31 16:05:14 -03:00
8af46bbb06 feat: capture Redis RPUSH length and log HSET result for suggestions
Changed the Redis RPUSH command to return the new list length (`Result<i64, RedisError>`) and added a debug log that includes this length. Updated the HSET command to capture its result (`Result<i64, RedisError>`), logging the number of fields added on success and handling errors explicitly. Removed unnecessary `unwrap_or_else` and added comments to clarify behavior, improving observability and error handling for suggestion storage.
2025-10-31 15:58:54 -03:00
a3555c1a84 feat(auth): refactor authentication handler and update logging
- Changed RUST_LOG level from 'trace' to 'debug' in VSCode launch config
- Simplified auth handler by removing direct bot ID lookup logic
- Added HttpRequest parameter to auth handler
- Implemented bot_from_url helper for bot identification
- Updated auth script path to use bot-specific directory structure
- Removed unused warn import
- Improved error handling for auth script reading and execution

The changes streamline the authentication process by:
1. Moving bot identification to a dedicated helper function
2. Making the auth script path dynamic based on bot name
3. Reducing log verbosity in development
4. Cleaning up unused imports and code
2025-10-31 15:40:52 -03:00
8c53161129 feat: simplify bot name handling and improve logging
- Added RUST_LOG=trace to VSCode debug configurations for better debugging
- Removed unused imports (uuid, Selectable, SelectableHelper) from bootstrap module
- Simplified bot name handling by removing name formatting logic, using raw bot folder names
- Renamed check_default_gbot to check_gbot for consistency
- Improved logging format in drive monitor initialization
- Fixed S3 bucket reference in bootstrap manager (removed 'templates/' prefix)
2025-10-31 07:30:37 -03:00
4f12608668 Refactor BotOrchestrator for async bot mounting
* Introduced async mounting
2025-10-30 21:02:44 -03:00
f85b19efda feat(bot, bootstrap): add bot mounting and improve DB handling
Introduce bot mounting logic in `BotOrchestrator` to load and manage active bots from the database. Enhance `BootstrapManager` by refining Diesel query usage and connection handling for better reliability and maintainability.
2025-10-30 18:48:16 -03:00
d803ffd24e feat: remove LLM configuration defaults migration
The migration file 6.0.6.sql was deleted as it contained default LLM server configurations that are no longer needed. The file was originally used to set up local LLM server settings including model paths and URLs, but these configurations are now handled differently in the system.
2025-10-30 17:36:15 -03:00
a01c7505cb feat(config): remove default server config and update database credentials
Removed the default server configuration section from migrations as these values should now be managed through environment variables. Updated default database username from 'postgres' to 'gbuser' in bootstrap and changed default database name from 'gbuser' to 'botserver' in config to align with the removed server configuration defaults.
2025-10-30 17:32:21 -03:00
4337bd4229 feat(suggestions): enhance suggestion handling and context management in WebSocket 2025-10-30 14:53:06 -03:00
17d6cd33be feat(bootstrap): add CSV config loading and improve legacy mode handling
Added functionality to load configuration from CSV files in S3 when available, with fallback to existing methods. The changes include:

1. Added new `load_config_from_csv` method to handle CSV config loading
2. Improved legacy mode handling by attempting CSV load before falling back to database
3. Added new dependencies (uuid, Arc, Mutex) for config management
4. Enhanced error handling and logging for config loading scenarios

The changes maintain backward compatibility while adding the ability to load configurations from CSV files stored in S3, which provides more flexibility in configuration management.
2025-10-30 13:29:46 -03:00
5ec0d3d2ad feat: remove default config.csv upload from bootstrap
Removed the automatic upload of default config.csv to S3 during bootstrap initialization. This functionality was previously uploading a local config file to the S3 bucket 'default.gbai/default.gbot' if the file existed. The change simplifies the bootstrap process by removing this implicit behavior.
2025-10-30 13:18:23 -03:00
b25b658cfd feat(config): add drive config persistence to .env file
Added functionality to write drive configuration (MinIO settings) to a .env file when the AppConfig is created. The new write_drive_config_to_env function handles the file operations and logs warnings if the write fails. This change ensures drive configuration is persisted for future use while maintaining backward compatibility.
2025-10-30 13:14:06 -03:00
e93935d5a3 refactor: clean up S3 client creation and imports in bootstrap module
- Renamed unclear function `c` to more descriptive `create_s3_operator`
- Removed duplicate `create_s3_operator` function implementation
- Consolidated imports from aws_sdk_s3 and diesel crates
- Removed unused import `aws_s3_bucket_create`
- Improved code formatting and organization

The changes make the code more maintainable by removing duplication and improving clarity while maintaining the same functionality.
2025-10-30 13:05:35 -03:00
792186cf76 feat(s3): Finally improve directory upload with proper path handling
- Added bucket parameter to upload_directory_recursive for explicit bucket specification
- Improved key construction logic to handle path prefixes more robustly
- Removed redundant trace logs and added more informative upload messages
- Fixed potential issue with bucket name extraction from prefix
- Ensured consistent path separator handling in S3 keys

The changes make the S3 upload functionality more reliable and maintainable by:
1. Explicitly passing bucket names instead of extracting from prefixes
2. Properly handling path separators in S3 keys
3. Providing clearer logging for upload operations
2025-10-30 12:52:21 -03:00
5a6e36e6c2 feat: replace opendal with AWS SDK for S3 operations
- Added AWS SDK S3 dependencies including aws-config, aws-sdk-s3, and related crates
- Removed opendal dependency and replaced with AWS SDK S3 client
- Implemented new get_file_content helper function using AWS SDK
- Updated MinIOHandler to use AWS SDK client instead of opendal Operator
- Modified file change detection to work with AWS SDK's S3 client

The change was made to standardize on AWS's official SDK for S3 operations, which provides better maintenance and feature support compared to the opendal crate. This also aligns with AWS best practices for interacting with S3 services.
2025-10-30 12:35:25 -03:00
2b906b7a9d Merge branch 'main' of https://github.com/ChristopherCastilho/botserver 2025-10-30 09:09:42 -03:00
254de04d28 feat(keywords): add suggestion keyword to ScriptService initialization 2025-10-30 09:09:15 -03:00
11e5d48f24
Merge branch 'GeneralBots:main' into main 2025-10-29 18:55:20 -03:00
d278b95fac feat(keywords, ui): add suggestion module and UI for context switching
Added new `add_suggestion` module to support suggestion handling logic.
Updated `index.html` to include a dynamic suggestions container that fetches and displays context suggestions.
This improves user experience by enabling quick context changes through interactive suggestion buttons.
2025-10-29 18:54:33 -03:00
0330b8fdb8 feat(bootstrap): improve bucket creation and error handling logic
Enhance the bucket creation process in `BootstrapManager` to handle existing buckets gracefully. Adds logic to reuse the default template when a bucket already exists and ensures proper path formatting before creation. This improves reliability and prevents redundant bucket creation errors.
2025-10-29 14:49:25 -03:00