Expanded README with detailed feature matrix and enterprise capabilities for the self-host AI automation platform. Simplified setup instructions by removing redundant configuration and build steps to improve clarity and onboarding experience.
- Update RUST_LOG configuration in launch.json to include trace level and additional module filters
- Uncomment and enable multiple directories in add-req.sh script
- Add execute_compact_prompt function to automation module
- Extend BasicCompiler comment detection to handle single quotes
- Modify BotOrchestrator system message prefix from "SYSTEM" to "SYS"
- Add placeholder for compact prompt automation in BotOrchestrator initialization
Changes improve debugging capabilities and enable previously commented-out automation features while maintaining existing functionality.
Add the `cron` crate (v0.15.0) to Cargo.toml and Cargo.lock to enable scheduling capabilities.
Introduce a new `broadcast_theme_change` helper in `src/automation/mod.rs` that parses CSV theme data and pushes JSON theme update events to all active response channels.
Clean up unused imports in the automation module and add `ConfigManager` import for future configuration handling.
Update `add-req.sh` to adjust the list of processed directories (comment out `auth`, enable `basic`, `config`, `context`, and `drive_monitor`).
These changes lay groundwork for scheduled tasks and dynamic theme updates across the application.
- Updated botserver version from 6.0.5 to 6.0.7 in Cargo.toml and Cargo.lock
- Removed old Rodrigo Rodriguez entry from authors list
- Added new Rodrigo Rodriguez entry with updated email
- Maintained all other existing authors in the list
Added support for configurable conversation history limits through bot configuration. The bot now reads 'prompt-history' from config (defaulting to -1 for unlimited) and trims the conversation history accordingly before generating prompts. Updated announcements bot template to use history limit of 2 messages instead of the previous compact setting.
Removed `IF NOT EXISTS` from the unique constraint in `system_automations` to ensure proper enforcement, and deleted the unused `floatLogo` click event listener to clean up UI behavior.
Added new configuration options for theme colors (green, yellow) and a custom logo URL to enhance branding and visual customization in announcement templates.
Updated the regex pattern in DeepseekR3Handler to use (?s) flag for dot-matches-newline behavior when removing <think> tags. Added comprehensive test case that verifies the handler correctly processes content with multiline think tags. Also made styling changes to the web interface, though the full diff was truncated.
- Simplified build_llm_prompt by removing redundant formatting
- Added info logging for LLM model and processed content
- Updated README with development philosophy note
- Adjusted announcement schedule timing from 55 to 59 minutes past the hour
Mark current_tokens and max_tokens parameters as unused in get_system_metrics function by prefixing them with underscores. This change clarifies that these parameters are intentionally unused in the function implementation while maintaining the function signature for potential future use.
Removed several unused dependencies from Cargo.lock including:
- auto_generate_cdp
- headless_chrome
- scraper
- cssparser and related crates
- dtoa and dtoa-short
- string_cache and related crates
- tendril
- tungstenite 0.27.0
Also updated ureq dependency to single version (removed duplicate entry). This cleanup reduces the dependency tree and removes unused code.
- Update prompt formatting in BotOrchestrator to use clearer labels (SYSTEM/CONTEXT) with emphasis markers
- Remove unused token_ratio field from SystemMetrics struct
- Increase default context size (2048->4096) and prediction length (512->1024) in config
- Clean up metrics calculation by removing redundant token ratio computation
The changes improve readability of system prompts and simplify metrics collection while increasing default model capacity.
Added the sysinfo crate (v0.37.2) to gather system metrics. This includes:
- New dependencies: sysinfo, ntapi, objc2-core-foundation, objc2-io-kit
- Updated windows-core to specific version 0.62.2
- Initial system metrics integration in bot module
The change enables monitoring system resources which will be used for performance optimization and health monitoring.
- Add 'keyword' to LLM processing log message for better context
- Replace simple string replace with regex for removing <think> tags in DeepseekR3 model
- The changes provide more precise logging and more robust content processing
- Added migration 6.0.6 to enforce a unique constraint on `(bot_id, kind, param)` in `system_automations`, preventing “no unique or exclusion constraint matching the ON CONFLICT specification” errors, and created a supporting index.
- Added migration 6.0.7 to replace the `clicks` table with a correctly defined primary key and a unique `(campaign_id, email)` constraint, satisfying Diesel
Added tracking of previously scheduled scripts using a `HashSet` and initialized it in `BasicCompiler::new`. Updated `compile_file` and `preprocess_basic` to require mutable access, allowing schedule cleanup before processing. Implemented logic to delete existing scheduled automations for a script using Diesel queries, ensuring old schedules are removed when a script is recompiled without a `SET_SCHEDULE`. Added necessary Diesel imports and `TriggerKind` reference. This prevents duplicate or orphaned scheduled tasks.
Changed the set_schedule function to first attempt updating existing records before inserting new ones. This improves efficiency by avoiding unnecessary insert conflicts and subsequent updates. The logic now:
1. Tries to update matching existing schedule first
2. Only performs insert if no matching record was found
3. Maintains same functionality but with better performance
- Extend `system_automations` with a non‑null `bot_id` UUID column, create an index on it, and add a unique constraint on `(bot_id, kind, param)` to support upserts.
- Add a unique constraint on `bot_configuration.config_key` to prevent duplicate configuration keys.
- Include migration guards to ensure the new constraint is only created once.
- Remove automatic writing of drive configuration to a `.env` file, cleaning up side‑effects during config loading.
- Change database connection handling to require `DATABASE_URL` to be set (no fallback), making the environment initialization explicit.
- Introduced `#[cfg(test)] pub mod auth_test;` in `src/auth/mod.rs` to expose authentication tests.
- Added `pub mod tests;` declarations in `src/lib.rs` and `src/main.rs` for centralized test utilities.
- Removed unnecessary blank lines and a placeholder comment in `src/shared/models.rs` and `src/tests/test_util.rs`.
- Minor formatting adjustments to improve code readability and maintainability.
- Simplified auth module by removing unused imports and code
- Cleaned up shared models by removing unused structs (Organization, User, Bot, etc.)
- Updated add-req.sh to comment out unused directories
- Modified LLM fallback strategy in README with additional notes about model behaviors
The changes focus on removing unused code and improving documentation while maintaining existing functionality. The auth module was significantly reduced by removing redundant code, and similar cleanup was applied to shared models. The build script was adjusted to reflect currently used directories.
Removed the test bucket creation code from the main function as it was only used for testing purposes. The production code should not include test functionality in the main execution path. This cleanup improves code quality and separation of concerns.
- Added CLI and IDE prompt files to the add-req.sh script
- Removed unused bucket creation code from BotOrchestrator
- Deleted unused create_bucket module from main.rs
The changes streamline the codebase by removing unused bucket creation functionality while expanding the prompt file inclusion for better LLM context handling.
Updated the directory list in add-req.sh to include previously commented-out directories and added new ones. The changes reflect a more complete set of project modules, including newly added ones like 'bootstrap', 'llm_models', and 'ui'. This ensures the script will process requirements for all current project components.
Added check_cmd field to ComponentConfig struct and implemented health check commands for all services including minio, postgres, valkey, stalwart, caddy, zitadel, forgejo, and forgejo-runner. This allows for proper health monitoring of each service. Also updated IDE guidelines to prohibit magic values and defaults.
- Refine IDE prompt to clarify identifier/character requirements in English
- Add info and debug logging for component startup process in bootstrap
- Fix PostgreSQL command formatting and improve output redirection
- Standardize Valkey command formatting and fix execution command
- Remove LLM server execution command (now handled separately)
- Clean up unused PostgreSQL status check code for tables component
Only upload templates to Drive buckets if the bucket was just created. Skip upload if bucket already exists to avoid overwriting existing content.
- Change from always uploading to conditionally uploading based on bucket existence
- Add info log when skipping upload for existing buckets
- Maintain same upload behavior for newly created buckets
Enable automation, basic, config, context, session, kb, and tools directories while disabling bootstrap, file, and drive_monitor to refine which modules are processed by add-req.sh.
Enhanced Chapter 1 documentation with detailed session architecture, storage layers, and API endpoints. Added new Part XI on authentication and security to SUMMARY.md, introducing chapters on user and bot authentication, password security, and API endpoints. Improves clarity and coverage of system interaction and security concepts.
Added interaction count tracking for sessions with Redis or in-memory fallback. Implemented conversation history replacement functionality to compact and update message history. The changes include:
- New AtomicUsize counter in SessionManager for interaction tracking
- increment_and_get_interaction_count method with Redis support
- replace_conversation_history to update and compact message history
- Maintains existing functionality while adding new features
The large `process_message` function was deleted from `src/bot/mod.rs`. Its responsibilities have been migrated to newer, more modular handlers, eliminating dead code and simplifying the BotOrchestrator. This refactor reduces complexity, improves maintainability, and aligns the codebase with the updated message processing architecture.
- Reformatted `update_session_context` call for better readability.
- Moved context‑change (type 4) handling to a later stage in the processing pipeline and removed early return, ensuring proper flow.
- Adjusted deduplication logic formatting and clarified condition.
- Restored saving of user messages after context handling, preserving message history.
- Added detailed logging of the LLM prompt for debugging.
- Simplified JSON extraction for `message_type` and applied minor whitespace clean‑ups.
- Overall code refactor improves maintainability and corrects context‑change handling behavior.
- Deduplicate consecutive messages with same role in conversation history
- Add n_predict configuration option for LLM server
- Prevent duplicate message storage in session manager
- Update announcement schedule timing from 37 to 55 minutes
- Add default n_predict value in default bot config
- Refactor cron matching to use individual variables for each time component with additional debug logging
- Replace SETEX with atomic SET NX EX for job locking in Redis
- Add better error handling and logging for job execution tracking
- Skip execution if Redis is unavailable or job is already held
- Add verbose flag to LLM server startup command for better logging
- Added duplicate filtering for suggestions in both backend (Rust) and frontend (JavaScript)
- Changed announcement summary schedule from every 15 minutes to hourly at :37
- Simplified LLM prompt for document summarization
- Updated LLM server configuration with reduced GPU layers and increased context size
- Removed memory mapping and lock settings for LLM server
- Improved HTML formatting and added missing newline at EOF
Added SET_SCHEDULE directive to run the update-summary script every 15 minutes. This ensures the announcements summary stays current by regularly fetching and processing the latest news document.
Added support for selecting an LLM model based on configuration and processing the raw response. The execute_llm_generation function now:
1. Fetches the configured model using ConfigManager
2. Gets the appropriate handler for the model
3. Processes the raw response through the handler before returning
This provides more flexibility in model selection and allows for model-specific response handling.
Enhances BotOrchestrator by integrating optional semantic caching via LangCache for faster LLM responses. Also refactors message saving to occur before and after direct mode handling, and simplifies context change logic for better clarity and flow.
- Added `trace!` logging in `bot_memory.rs` to record retrieved memory values for easier debugging.
- Refactored `BotOrchestrator` in `bot/mod.rs`:
- Removed duplicate session save block and consolidated message persistence.
- Replaced low‑level LLM streaming with a structured `UserMessage` and `stream_response` workflow, improving error handling and readability.
- Updated configuration loading in `config/mod.rs`:
- Imported `get_default_bot` and enhanced `get_config` to fall back to the default bot configuration when the primary query fails.
- Established a fresh DB connection for the fallback path to avoid borrowing issues.
- Remove redundant background and filter properties from logo-icon and assistant-avatar
- Remove redundant filter property from light theme logo-icon
- Invert assistant-avatar image in light theme for better visibility
- Move theme-toggle button further right (60px to 160px) to prevent overlap
- Remove `scripts_dir` from `AutomationService` and its constructor, as it was unused.
- Drop LLM and embedding server readiness checks; the service now only schedules periodic tasks.
- Increase the health‑check interval from 5 seconds to 15 seconds for reduced load.
- Streamline the LLM keyword prompt to a concise `"User: {}"` format, removing verbose boilerplate.
- Remove unnecessary logging and LLM cache handling code from the bot orchestrator, cleaning up unused environment variable checks and cache queries.
- Added LLM server readiness check in AutomationService before starting tasks
- Renamed `user` parameter to `user_session` in execute_talk for clarity
- Updated BotResponse fields to use user_session data instead of hardcoded values
- Improved Redis key generation in execute_talk to use user_session fields
- Removed commented Redis code in set_current_context_keyword
The changes ensure proper initialization of automation tasks by checking LLM server availability first, and improve code clarity by using more descriptive variable names for user session data.
- Added UNIQUE constraint on system_automations (bot_id, kind, param) to prevent duplicate automations
- Refactored execute_action to accept full Automation struct instead of just param
- Simplified bot name resolution by using automation.bot_id directly
- Improved error handling in action execution with proper error propagation
- Removed redundant bot name lookup logic by leveraging existing bot_id
- Introduced `bot_id` column in `system_automations` table (migration 6.0.0.sql) and updated the Diesel schema/model to include it.
- Adjusted migrations to remove the hard‑coded “Update Summary” automation and only create an index on the `name` column.
- Extended the `SET_SCHEDULE` keyword:
- Added a second string argument for the script name.
- Passed the invoking user's `bot_id` to the database layer.
- Updated function signature to accept a full `UserSession` instead of discarding it.
- Modified `execute_set_schedule` to store `bot_id`, script name, and activation flag; added conflict handling on `(bot_id, param)` to update schedule and reset trigger state.
- Updated imports and logging to reflect new parameters.
These changes enable per‑bot automation management, allow specifying the script to run, and improve idempotent schedule updates.
- Update RUST_LOG environment variable in launch.json to exclude more verbose crates
- Change automation kind from 3 to 0 in migrations/6.0.5.sql
- Replace debug with trace logging for schedule evaluation in automation service
- Modify script path resolution to use .ast extension instead of .bas
- Improve error message when script reading fails
- Remove MinIO download fallback for scripts (simplifying script handling)
These changes improve logging verbosity control and simplify script handling by removing the MinIO fallback mechanism. The trace-level
- Updated `BootstrapManager` to use `AppConfig::from_env().expect(...)` and `AppConfig::from_database(...).expect(...)` ensuring failures are explicit rather than silently ignored.
- Refactored error propagation in bootstrap flow to use `?` where appropriate, improving reliability of configuration loading.
- Added import of `llm_models` in `bot` module and introduced `ConfigManager` usage to fetch the LLM model identifier at runtime.
- Integrated dynamic LLM model handler selection via `llm_models::get_handler(&model)`.
- Replaced static environment variable retrieval for embedding configuration with runtime
Introduce a new `cancel_job` method in the `LLMProvider` trait to allow cancellation of ongoing LLM tasks. Implement no-op versions for OpenAI, Anthropic, and Mock providers. Update the WebSocket handler to invoke job cancellation when a session closes, ensuring better resource management and preventing orphaned tasks. Also, fix unused variable warning in `add_suggestion.rs`.
Added new CLEAR_SUGGESTIONS keyword command that clears user suggestions from Redis cache. Implemented the command in the announcements dialog start script to prevent duplicate suggestions. The command handles Redis connection errors gracefully and logs appropriate debug information.
Changed debug logging to trace level for Redis suggestion operations and drive monitor object listings. This better reflects the verbosity level of these operations since they are more detailed debugging information. Also updated "Redis client" to "Cache client" in a debug message for consistency with the broader cache abstraction.
- Updated RUST_LOG environment variable in launch.json to include more detailed debug logging configuration
- Changed trace! logs to debug! in AutomationService for better visibility
- Replaced environment variable usage with get_default_bot helper function
- Improved bot path resolution by using bot name from database
- Added error handling for bot name query
- Simplified S3 bucket name generation using get_default_bot
- Removed unused imports and environment variable dependencies
Add `context_length` and `context_max_length` fields to the `BotResponse` struct across the codebase.
Initialize these fields with default values (0) for existing response constructions and populate them using `ConfigManager` to retrieve the configured maximum context size for each bot.
Import `ConfigManager` in `bot/mod.rs` to access configuration.
These changes enable tracking of the current and maximum context sizes in bot responses, supporting future features that rely on context management.
