Commit graph

3902 commits

Author SHA1 Message Date
58adf7c4ae fix: Set default llm_server_path and correct model file paths
Some checks failed
BotServer CI / build (push) Has been cancelled
When no default.gbai/config.csv exists, the system now:
- Sets default llm_server_path to ./botserver-stack/bin/llm/build/bin
- Uses correct relative paths to model files: ../../../../data/llm/
- Uses actual model filenames from 3rdparty.toml

This fixes the issue where LLM/embedding servers couldn't find model files
because the paths were constructed incorrectly.

Model filenames:
- LLM: DeepSeek-R1-Distill-Qwen-1.5B-Q3_K_M.gguf
- Embedding: bge-small-en-v1.5-f32.gguf

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 13:11:14 +00:00
0e6e2bfc6d fix: Correct default LLM model to deepseek-small
All checks were successful
BotServer CI / build (push) Successful in 8m57s
Changed the default LLM model from glm-4 to deepseek-small to match
the model defined in 3rdparty.toml ([models.deepseek_small]).

This ensures that when no default.gbai/config.csv exists, the system
uses the correct default local model.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 12:56:15 +00:00
337bef3bad fix: Use default local LLM models when config is empty
Some checks failed
BotServer CI / build (push) Has been cancelled
When no default.gbai/config.csv exists or when llm-model/embedding-model
config is empty, the system now uses default local models instead of
skipping server startup.

Changes:
- Default LLM model: glm-4
- Default Embedding model: bge-small-en-v1.5
- Logs when using defaults

This fixes the issue where the "default" bot would fail to load LLM
and Embedding services when no config.csv was present, causing the
error: "not loading embedding neither llm local for default bot"

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 12:54:40 +00:00
cf7bd7ffa2 fix: Use spawn_blocking for Redis cache connection to prevent freezing
All checks were successful
BotServer CI / build (push) Successful in 8m38s
The init_redis() function was using synchronous blocking calls
(redis::Client::get_connection()) inside an async function, which
blocked the entire tokio runtime and caused botserver to freeze.

Changes:
- Wrap Redis connection calls in tokio::task::spawn_blocking()
- Runs blocking operations in separate thread pool
- Prevents tokio runtime from freezing during cache connection

This fixes the issue where botserver would hang indefinitely
when connecting to Valkey/Redis cache.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 12:21:33 +00:00
fc34461b2f fix: Improve cache_health_check TCP test to eliminate false positives
All checks were successful
BotServer CI / build (push) Successful in 8m28s
The previous /dev/tcp test was giving false positives, reporting that
Valkey was running when it was actually down. This caused bootstrap to
skip starting Valkey, leading to botserver hanging on cache connection.

Changes:
- Use nc (netcat) with -z flag for reliable port checking
- Final fallback: /dev/tcp with actual PING/PONG verification
- Only returns true if port is open AND responds correctly

This ensures cache_health_check() accurately reports Valkey status.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 12:02:23 +00:00
d243666cc2 fix: Support valkey-cli in cache_health_check for Valkey-only setups
All checks were successful
BotServer CI / build (push) Successful in 8m19s
- Try valkey-cli first (preferred for Valkey installations)
- Fall back to redis-cli (for Redis installations)
- Fall back to TCP connection test (works for both)

This fixes environments that only have Valkey installed without
Redis symlinks or redis-cli.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 11:48:02 +00:00
ba70bd97bc fix: Bootstrap should check Valkey is running and use correct component name
All checks were successful
BotServer CI / build (push) Successful in 8m56s
- Fix component name mismatch: "redis" -> "cache" in bootstrap_manager
- Add cache_health_check() function to verify Valkey is responding
- Add health check loop after starting cache (12s wait with PING test)
- Ensures cache is ready before proceeding with bootstrap

This fixes the issue where botserver would hang waiting for cache
connection because the cache component was never started.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 11:26:27 +00:00
dec3a086ab fix: Extract Valkey to cache/bin/ directory
All checks were successful
BotServer CI / build (push) Successful in 8m1s
- Binaries at cache/bin/valkey-server (correct production path)
- Use --strip-components=1 for extraction
- Matches /opt/gbo/bin/botserver-stack/bin/cache/bin/

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:22:59 +00:00
bfbdf8396f fix: Extract Valkey binaries directly to cache/ directory
Some checks failed
BotServer CI / build (push) Has been cancelled
- Use --strip-components=2 to flatten tarball structure
- Binaries go to cache/valkey-server (not cache/bin/valkey-server)
- Matches production path: /opt/gbo/bin/botserver-stack/bin/cache/

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:21:20 +00:00
0af972eda6 fix: Use correct valkey-server binary path
Some checks failed
BotServer CI / build (push) Has been cancelled
- Binary is at cache/bin/bin/valkey-server after extraction
- Update exec_cmd and check_cmd to use bin/ subdirectory
- Create symlinks at parent level for convenience

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:20:56 +00:00
14b3a0e6c0 fix: Remove redis symlinks from Valkey installation
Some checks failed
BotServer CI / build (push) Has been cancelled
- Use valkey-server and valkey-cli directly
- No redis compatibility symlinks needed
- Simplifies installation

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:20:08 +00:00
92d21fe3f7 fix: Use Valkey 8.1.5-jammy for GLIBC 2.36 compatibility
All checks were successful
BotServer CI / build (push) Successful in 8m54s
- noble version requires GLIBC 2.38 (Ubuntu 24.04)
- jammy version works with GLIBC 2.36 (Ubuntu 22.04)
- System has GLIBC 2.36, needs compatible binary

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:01:19 +00:00
0e1013e4ca fix: Downgrade Valkey from 9.0.2 to 8.1.5
Some checks failed
BotServer CI / build (push) Has been cancelled
- Use valkey-8.1.5-noble-x86_64.tar.gz instead of 9.0.2-jammy
- More stable version for production use

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 00:54:25 +00:00
278b92d5fa feat: Increase cache retry delay to 5 seconds
All checks were successful
BotServer CI / build (push) Successful in 8m25s
- Change retry interval from 1s to 5s between attempts
- Reduce attempts from 30 to 12 (still 60s total wait time)
- Gives Valkey more time to stabilize between connection attempts
- Helps with slow-to-start services during bootstrap

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 00:37:59 +00:00
8d0fd9da3d feat: Add retry logic for cache/Valkey connection
All checks were successful
BotServer CI / build (push) Successful in 8m19s
- Wait up to 30 seconds for cache to be ready
- Retry every 1 second with progress logging
- Prevents race condition during service startup
- Ensures suggestions feature works when Valkey starts after botserver

Fixes issue where cache connection failed during bootstrap if
Valkey wasn't immediately ready.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 00:24:46 +00:00
4e4aca0986 fix: Use precompiled Valkey binary from download.valkey.io
All checks were successful
BotServer CI / build (push) Successful in 7m53s
- Update 3rdparty.toml to download valkey-9.0.2-jammy-x86_64.tar.gz
- Remove source compilation approach (no gcc/make required)
- Update installer to handle precompiled binaries directly
- Fix symlinks for redis compatibility

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 23:51:33 +00:00
9453ff4c65 fix: Improve cache/valkey and LLM configuration
All checks were successful
BotServer CI / build (push) Successful in 8m15s
- Add connection verification for Redis/Valkey cache with PING test
- Support CACHE_URL, REDIS_URL, and VALKEY_URL environment variables
- Add better error messages when cache is unavailable
- Add LLM_URL and LLM_MODEL environment variable support
- LLM configuration now checks env vars first, then database, then defaults
- This ensures local LLM (http://localhost:8081) is used as proper fallback

Fixes suggestions button not working when valkey is unavailable
and improves LLM configuration when no bot config.csv exists.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 23:30:07 +00:00
8a8008a72c fix: Resolve compilation errors
All checks were successful
BotServer CI / build (push) Successful in 13m3s
- Escape format placeholders in designer_ai.rs ({{botname}})
- Remove undefined 'prefix' filter in drive_monitor
- Fix type mismatch in use_tool.rs (str vs &String)
- Remove unused TextExpressionMethods import

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 22:15:35 +00:00
e34848507d fix: Update multiple modules for i18n and drive monitoring
Some checks failed
BotServer CI / build (push) Failing after 6m22s
- Update auto_task modules (app_generator, designer_ai, intent_classifier)
- Refactor use_tool.rs for better structure
- Update bot core and website crawler
- Improve drive_monitor and local_file_monitor
- Update bootstrap module

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 22:06:57 +00:00
d1d591ddcb chore: Force CI rebuild with i18n debug-embed support
All checks were successful
BotServer CI / build (push) Successful in 8m18s
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 19:42:42 +00:00
35fafad088 chore: Force CI rebuild with updated i18n
All checks were successful
BotServer CI / build (push) Successful in 8m33s
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 19:04:47 +00:00
23c93cbb3d feat: Restore .env file creation for local Vault bootstrap
All checks were successful
BotServer CI / build (push) Successful in 9m6s
- Add TLS certificate generation to Vault post-install commands (CA, server, client, PostgreSQL)
- Add initialize_vault_local() function to handle Vault initialization for local installs
- Add ensure_env_file_exists() function to create .env when Vault already initialized
- Modify start() method to call Vault initialization after successful start (local mode)
- Fix Vault CLI flags: use -tls-skip-verify (not -skip-verify or -skip-tls-verify)

This restores the behavior where .env is automatically created with VAULT_ADDR,
VAULT_TOKEN, and VAULT_CACERT during local bootstrap, matching the LXC container
deployment behavior in facade.rs.

Fixes issue where .env file was only created for LXC deployments but not for
local installations.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 16:35:24 +00:00
bd92ff7bfd fix(bootstrap): bootstrap_completed only checks file existence, not content
All checks were successful
BotServer CI / build (push) Successful in 8m50s
2026-02-14 13:49:15 +00:00
0c44f7784d Debug: Add logging for missing translation keys
All checks were successful
BotServer CI / build (push) Successful in 10m8s
2026-02-14 12:25:46 +00:00
4b44602d39 Update bootstrap and core modules
All checks were successful
BotServer CI / build (push) Successful in 10m27s
2026-02-14 09:54:14 +00:00
17cb4ef147 Fix: Update source files and fix notify dependency reference
All checks were successful
BotServer CI / build (push) Successful in 11m35s
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-13 22:30:57 +00:00
5ea171d126 Refactor: Split large files into modular subdirectories
Some checks failed
BotServer CI / build (push) Failing after 1m34s
Split 20+ files over 1000 lines into focused subdirectories for better
maintainability and code organization. All changes maintain backward
compatibility through re-export wrappers.

Major splits:
- attendance/llm_assist.rs (2074→7 modules)
- basic/keywords/face_api.rs → face_api/ (7 modules)
- basic/keywords/file_operations.rs → file_ops/ (8 modules)
- basic/keywords/hear_talk.rs → hearing/ (6 modules)
- channels/wechat.rs → wechat/ (10 modules)
- channels/youtube.rs → youtube/ (5 modules)
- contacts/mod.rs → contacts_api/ (6 modules)
- core/bootstrap/mod.rs → bootstrap/ (5 modules)
- core/shared/admin.rs → admin_*.rs (5 modules)
- designer/canvas.rs → canvas_api/ (6 modules)
- designer/mod.rs → designer_api/ (6 modules)
- docs/handlers.rs → handlers_api/ (11 modules)
- drive/mod.rs → drive_handlers.rs, drive_types.rs
- learn/mod.rs → types.rs
- main.rs → main_module/ (7 modules)
- meet/webinar.rs → webinar_api/ (8 modules)
- paper/mod.rs → (10 modules)
- security/auth.rs → auth_api/ (7 modules)
- security/passkey.rs → (4 modules)
- sources/mod.rs → sources_api/ (5 modules)
- tasks/mod.rs → task_api/ (5 modules)

Stats: 38,040 deletions, 1,315 additions across 318 files

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 21:09:30 +00:00
3566a8c87f Fix batismo tool natural language processing
- Enhanced conversational input handling for batismo tool
- Improved keyword extraction and format recognition
- Fixed field extraction from informal user messages
- Better natural language understanding

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 20:17:49 +00:00
76abcea5e9 Fix start.bas repeated execution and add tool calling system
- Add Redis-based tracking to prevent start.bas from running repeatedly
  when clicking suggestion buttons. start.bas now executes only once per
  session with a 24-hour expiration on the tracking key.

- Add generic tool executor (ToolExecutor) for parsing and executing
  tool calls from any LLM provider. Works with Claude, OpenAI, and
  other providers that use standard tool calling formats.

- Update both start.bas execution paths (WebSocket handler and LLM
  message handler) to check Redis before executing.

- Fix suggestion duplication by clearing suggestions from Redis after
  fetching them.

- Add rate limiter for LLM API calls.

- Improve error handling and logging throughout.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 13:49:54 +00:00
6215908536 Add ConfigWatcher and fix model routing
New features:
- Add ConfigWatcher for hot-reloading config.csv from ~/data
- Add LocalFileMonitor for watching ~/data/*.gbai directories
- Add GLM LLM provider implementation
- Add tool context for LLM tool calling

Bug fixes:
- Fix model routing to respect session → bot → default hierarchy
- Fix ConfigWatcher to handle local embedded (llm-server=true)
- Skip DriveMonitor for default bot (managed via ConfigWatcher)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 15:10:27 +00:00
84458b2a69 feat: Add BOTSERVER_PORT environment variable override
All checks were successful
BotServer CI / build (push) Successful in 12m37s
- Check BOTSERVER_PORT env var before database config
- Override server_port from database if BOTSERVER_PORT is set
- Apply to both from_database() and from_env() config paths
- Allows easy port configuration via environment variable
2026-02-06 12:56:52 -03:00
125b9d4389 Remove embed-ui from default features, fix embedded UI path
All checks were successful
BotServer CI / build (push) Successful in 12m5s
- Removed embed-ui from default features (botserver is backend API only)
- UI embedding is botui's responsibility, not botserver's
- Fixed rust-embed folder path in embedded_ui.rs
- Resolves CI compilation errors
2026-02-06 09:40:37 -03:00
e8a400d86d Remove Kubernetes deployment, add embed-ui deployment documentation
Some checks failed
BotServer CI / build (push) Failing after 12m5s
- Removed Kubernetes deployment files (focus on embed-ui feature instead)
- Added deploy-ui.sh script for manual UI file deployment
- Added deploy/README.md with comprehensive deployment guide
- Updated README.md with embed-ui feature explanation
- Simplified deployment: embed-ui feature creates self-contained binary
2026-02-06 09:26:44 -03:00
373957cb25 Fix: Add embed-ui to default features
Production binaries should embed UI assets by default to avoid
requiring external ui/suite folder in deployment.

This ensures botserver binary contains all HTMX, HTML, CSS, and JS
assets needed for the web interface.
2026-02-05 22:52:12 -03:00
5cedd9c379 Fix: Remove mold linker from cargo config
Some checks failed
BotServer CI / build (push) Failing after 17m29s
Mold linker is not available on the system and was causing build failures.
Removed rustflags setting to use default lld linker instead.
2026-02-05 22:43:26 -03:00
7da1442c91 Fix PostgreSQL readiness check timeout in bootstrap
All checks were successful
BotServer CI / build (push) Successful in 13m13s
- Use local pg_isready path when available (./botserver-stack/bin/tables/bin/pg_isready)
- Fall back to system pg_isready if local binary not found
- Prevents 30-second timeout during bootstrap when PostgreSQL is actually running
- Applied to both readiness checks in start_all() method
2026-02-05 22:23:04 -03:00
07ff7a6297 Fix PostgreSQL health check - remove database-specific check
All checks were successful
BotServer CI / build (push) Successful in 12m18s
- Removed -d 'postgres' parameter from pg_isready health checks
- Health check now only verifies server connection on port 5432
- Fixes false positive failures when PostgreSQL is running but specific database has issues
- PostgreSQL logs showed 'database system is ready' but health check was failing
2026-02-05 21:55:47 -03:00
de3e5c4fec Fix PostgreSQL ready check for already-running instances
All checks were successful
BotServer CI / build (push) Successful in 14m23s
Add pg_isready health check to the 'already running' branch to ensure
PostgreSQL is properly detected as ready, even when running as a
non-interactive user (sudo -u gbuser).

This complements the previous fix for fresh PostgreSQL starts.
2026-02-05 21:09:23 -03:00
3258f3ef85 Fix PostgreSQL startup failure for non-interactive users
All checks were successful
BotServer CI / build (push) Successful in 11m58s
Changed pg_isready checks from '-U gbuser' to '-d postgres' to properly
detect PostgreSQL readiness during bootstrap. The gbuser database doesn't
exist yet during startup, causing pg_isready to fail and bootstrap to timeout.

This fixes the issue when running botserver as a non-interactive user
(e.g., sudo -u gbuser).
2026-02-05 20:40:52 -03:00
87931e930c ci: adjust CARGO_BUILD_JOBS to 2
All checks were successful
BotServer CI / build (push) Successful in 22m11s
2026-02-05 20:01:15 -03:00
cdb1197920 ci: increase CARGO_BUILD_JOBS to 6 for faster builds
Some checks are pending
BotServer CI / build (push) Waiting to run
2026-02-05 19:54:31 -03:00
717ca8c35b Fix: Remove undefined db_password variable in database creation
Some checks failed
BotServer CI / build (push) Failing after 11m39s
2026-02-05 19:06:59 -03:00
309c5bc55a Fix: Ensure botserver database exists before running migrations
Some checks are pending
BotServer CI / build (push) Waiting to run
2026-02-05 19:02:01 -03:00
6cfbf013e2 Fix missing Router import in non-embed-ui block
All checks were successful
BotServer CI / build (push) Successful in 9m54s
2026-02-05 18:41:34 -03:00
35af28a041 Fix compilation errors: duplicate imports, feature-gated functions, type mismatch
Some checks failed
BotServer CI / build (push) Failing after 7m28s
2026-02-05 18:31:14 -03:00
52bccf0551 Adjust CARGO_BUILD_JOBS to 3
Some checks failed
BotServer CI / build (push) Failing after 11m25s
2026-02-05 18:12:41 -03:00
1a2d65c531 Fix: Remove path filters and cache for Forgejo compatibility
Some checks failed
BotServer CI / build (push) Has been cancelled
2026-02-05 18:08:30 -03:00
d4a7d1246b Fix build.rs: only watch botui/ui/suite if exists (CI compatibility) 2026-02-05 15:28:27 -03:00
9b9015bfa1 Keep 3rdparty.toml embedded in binary (already using include_str!) 2026-02-05 14:59:57 -03:00
24b516406a Add automatic log dump when component fails to start 2026-02-05 14:53:37 -03:00