Commit graph

3938 commits

Author SHA1 Message Date
1bbb94d500 fix(directory): improve error message for invalid credentials
Some checks failed
BotServer CI / build (push) Failing after 2m34s
- Updated error message to explain when credentials are found but invalid
- Clarified that credentials might be from a previous installation
- Added step-by-step solution for resetting credentials
- Removed misleading 'No admin credentials found' message

The error now accurately reflects the actual problem: authentication
failure rather than missing credentials.
2026-03-01 10:17:30 -03:00
8016caa790 fix(bootstrap): create OAuth client when Zitadel is already running
Some checks failed
BotServer CI / build (push) Failing after 3m20s
- Added OAuth client creation to the 'already running' branch
- Previously, OAuth client was only created when Zitadel was started
- This fixes the issue where OAuth client wasn't created on subsequent runs

Fixes the OAuth client creation issue discovered during testing.
2026-03-01 10:11:42 -03:00
2a6c599c75 feat(directory): read admin credentials from ~/.gb-setup-credentials
Some checks failed
BotServer CI / build (push) Failing after 5m3s
- Updated setup_directory() to read credentials from saved file
- Added read_saved_credentials() to parse ~/.gb-setup-credentials
- Added get_admin_credentials() to try multiple sources
- Removed default credentials approach (doesn't work)
- Improved error messages with solution steps

This matches the working approach from commit 86cfccc2 where
credentials were saved during first bootstrap and reused for
OAuth client creation on subsequent runs.
2026-03-01 10:06:30 -03:00
eb5c12c466 fix(directory): add .await to ensure_admin_token() calls
Some checks failed
BotServer CI / build (push) Failing after 5m28s
Fixed compilation errors by adding .await to all ensure_admin_token() calls:
- create_organization()
- create_user()
- save_config()

The method was made async but the calls weren't updated.
2026-03-01 09:52:31 -03:00
2c92a81302 merge: Unify master into main - all commits unified
Some checks failed
BotServer CI / build (push) Failing after 6m9s
2026-03-01 07:43:07 -03:00
8f495c75ec WIP: Local changes before merging master into main 2026-03-01 07:40:11 -03:00
bbdf243c86 fix: OAuth client creation during bootstrap
Some checks failed
BotServer CI / build (push) Failing after 6m2s
- Add password grant authentication support in DirectorySetup
- Extract initial admin credentials from Zitadel log file
- Fix race condition in Zitadel startup (wait for health check before starting)
- Create parent directories before saving config
- Add retry logic for OAuth client creation
- Improve error handling with detailed messages

Fixes authentication service not configured error after bootstrap.
2026-03-01 07:31:17 -03:00
0b1b17406d chore(core): various email and security updates 2026-02-24 19:02:48 -03:00
764f058653 fix: update work directory paths to use botserver-stack/data/system/work
All checks were successful
BotServer CI / build (push) Successful in 7m4s
Updated all hardcoded work/ directory references to use the correct
relative path from the current working directory:
- botserver-stack/data/system/work

This ensures consistent file location resolution regardless of where
botserver is run from (/home/rodriguez/src/gb/ or /opt/gbo/bin/).

Changes:
- local_file_monitor.rs: Use std::env::current_dir() for work_root
- drive_monitor/mod.rs: Use work_root PathBuf for tool compilation
- website_crawler_service.rs: Use std::env::current_dir() for work_path

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-22 16:20:07 -03:00
1856215d05 chore: update dependencies and formatting
All checks were successful
BotServer CI / build (push) Successful in 7m30s
2026-02-22 15:55:39 -03:00
affb702e90 fix(tools): auto-create work directory if missing in production
All checks were successful
BotServer CI / build (push) Successful in 8m12s
Added logic to create botserver-stack/data/system/work directory
if it doesn't exist. This ensures production deployments work
without manual directory setup.

Changes:
- Added fs::create_dir_all() in use_tool.rs
- Added fs::create_dir_all() in tool_context.rs
- Logs when directory is created
- Fixes production deployment where /system/work may not exist

This ensures the tool loading works in fresh production environments
where the work folder hasn't been populated yet.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-22 12:58:58 -03:00
febfa2e708 fix(tools): use relative path for tool .mcp.json files
All checks were successful
BotServer CI / build (push) Successful in 8m37s
Changed tool loading to use relative path from current directory instead
of hardcoded HOME/gb path. This makes the code portable across different
deployment environments.

- Updated use_tool.rs to use std::env::current_dir()
- Updated tool_context.rs to use std::env::current_dir()
- Added PathBuf import to both files
- Tools now load from botserver-stack/data/system/work/

Fixes issue where tools weren't being loaded because .mcp.json files
were in a different location than expected.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-22 11:21:07 -03:00
de017241f2 fix: Complete security remediation - RCE and SSRF fixes
All checks were successful
BotServer CI / build (push) Successful in 7m34s
- Fixed RCE vulnerability in trusted_shell_script_arg execution
- Fixed SSRF vulnerability in GET command with internal IP blocking
- Updated SafeCommand to use explicit positional arguments

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 01:14:14 +00:00
e143968179 feat: Add JWT secret rotation and health verification
SEC-02: Implement credential rotation security improvements

- Add JWT secret rotation to rotate-secret command
- Generate 64-character HS512-compatible secrets
- Automatic .env backup with timestamp
- Atomic file updates via temp+rename pattern
- Add health verification for rotated credentials
- Route rotate-secret, rotate-secrets, vault commands in CLI
- Add verification attempts for database and JWT endpoints

Security improvements:
- JWT_SECRET now rotatable (previously impossible)
- Automatic rollback via backup files
- Health checks catch configuration errors
- Clear warnings about token invalidation

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-19 19:42:41 +00:00
df9b228a35 feat(drive): Implement S3 handlers (list/read/write/delete) 2026-02-19 12:38:43 +00:00
98813fbdc8 chore: Fix warnings and clean TODO refs 2026-02-19 12:18:39 +00:00
ac5b814536 fix(security): Fix unsafe code, CORS logic, and expect usage 2026-02-19 12:06:05 +00:00
d7211a6c19 fix: Resolve unused import and variable warnings 2026-02-19 11:48:17 +00:00
3b21ab5ef9 fix: prevent duplicate message in chat when tool is executed
- Track tool_was_executed flag in stream_response
- Send empty content in final is_complete message when tool already sent results
- Prevents the LLM's pre-tool text from appearing twice in the chat UI
- DB message saving is unaffected (uses full_response_clone before the check)
2026-02-18 20:31:34 +00:00
b1118f977d fix: Correct parameter names in tool .bas files to match database schema
- Tool 06: Change tipoExibicao to tipoDescricao (matches pedidos_uso_imagem table)
- Tool 07: Change tipoExibicao to categoriaDescricao (matches licenciamentos table)
- Both tools now compile and execute successfully with database inserts

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-18 17:51:47 +00:00
f7c60362e3 fix: Add SAVE statement conversion for tool compilation
- Add convert_save_for_tools() to convert SAVE to INSERT syntax
- Generate map-based INSERT: let __data__ = #{field: value, ...}; INSERT "table", __data__
- Fix parameter names to match database schema (tipoExibicao -> tipoDescricao)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-18 17:50:04 +00:00
9b86b204f2 fix: Add # comment support and remove hardcoded column lists
- Support # as comment marker like ' in BASIC preprocessor
- Remove hardcoded column lists from get_table_field_names()
- Let runtime use database schema dynamically via get_table_columns()
- Fix SELECT/CASE conversion to add semicolons to body statements

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-18 17:19:30 +00:00
848b875698 fix: Use PORT env var instead of BOTSERVER_PORT, default to port 9000
- Change BOTSERVER_PORT to PORT for consistency with .env.embedded
- Update default port from 8080 to 9000 in config
- Fix service port references in security integration
- Update directory setup ExternalPort to 9000

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 14:43:16 +00:00
c264ad1294 fix: Improve tool call handling and BASIC SELECT/CASE conversion
All checks were successful
BotServer CI / build (push) Successful in 9m34s
- Add tool_call_buffer to accumulate JSON chunks across multiple LLM responses
- Handle incomplete tool call JSON that spans multiple chunks
- Convert SELECT...CASE/END SELECT to Rhai match expressions
- Fix NOT IN operator conversion to !in for IF conditions
2026-02-16 09:30:19 +00:00
4ca7e5da40 fix: Handle tool calls in JSON array format
All checks were successful
BotServer CI / build (push) Successful in 9m25s
Fixed issue where LLM tool calls returned as JSON arrays were not being
detected and were displayed as raw JSON in the chat instead of being executed.

The parse_tool_call method now handles:
- Single tool call objects
- Arrays of tool calls (OpenAI standard format)

This prevents tool call JSON from appearing in the chat window and ensures
tools are executed properly.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-16 00:19:03 +00:00
b92ef7c034 feat: Add detailed logging for session tools loading in WebSocket
All checks were successful
BotServer CI / build (push) Successful in 10m35s
Add error and warning logs to help diagnose why session tools are not
working in production. Logs now show:
- Number of tools loaded successfully
- Detailed error messages when tool loading fails
- Bot name lookup failures

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 23:57:17 +00:00
0883fe9cce fix: Avoid workspace conflict in Forgejo CI
All checks were successful
BotServer CI / build (push) Successful in 7m54s
Remove actions/checkout with custom path that causes multiple workspace
roots error. Clone botserver repository directly in Setup Workspace step
instead of using checkout action.

Fixes error: "multiple workspace roots found in the same workspace"

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 23:26:55 +00:00
8257d49967 feat: Load session tools in WebSocket connection
Some checks failed
BotServer CI / build (push) Has been cancelled
This loads and sends the available tools to the client when establishing
a WebSocket connection. Tools are loaded based on the bot configuration
and sent in the initial welcome message.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 23:19:38 +00:00
0a1bd25869 fix: Increase default n_predict to 512 for DeepSeek R1 reasoning
All checks were successful
BotServer CI / build (push) Successful in 9m26s
DeepSeek R1 model outputs reasoning_content first, then content.
With n_predict=50, responses were truncated during reasoning phase.
Increased to 512 to allow full reasoning + response.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 20:27:35 +00:00
a9cbbbffa0 fix: Use correct default LLM model name for local DeepSeek server
Changed default model from 'gpt-3.5-turbo' to 'DeepSeek-R1-Distill-Qwen-1.5B-Q3_K_M.gguf'
in bot message handler. This ensures the local llama-server receives the correct model
name and can process requests properly.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 20:23:04 +00:00
1cee912b72 fix: Correct LLM model paths and remove unnecessary cd command
- Change model paths to use ./data/llm/ instead of relative paths from build dir
- Remove cd command when starting llama-server to keep botserver root as cwd
- This fixes model loading when servers are started from different directories
- Both LLM and embedding servers now start successfully

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 20:15:17 +00:00
e9a428ab1c fix: Auto-create bot database when not configured
Modified get_bot_pool() to automatically create the database for a bot
if it doesn't exist, instead of failing with "No database configured" error.

This fixes the issue where bots created after the initial sync don't have
a database_name set in the bots table, causing table creation to fail.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 14:57:22 +00:00
0c9665dd8b fix: Enable vector_db by default with health check and fallback to local LLM
- Add vector_db_health_check() function to verify Qdrant availability
- Add wait loop for vector_db startup in bootstrap (15 seconds)
- Fallback to local LLM when external URL configured but no API key provided
- Prevent external LLM (api.z.ai) usage without authentication key

This fixes the production issues:
- Qdrant vector database not available at https://localhost:6333
- External LLM being used instead of local when no key is configured
- Ensures vector_db is properly started and ready before use

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 14:54:17 +00:00
307809bbdd fix: Handle empty config values for LLM server arguments
All checks were successful
BotServer CI / build (push) Successful in 8m3s
The config_manager.get_config() can return Ok("") for empty config values,
which would pass through unwrap_or_else() without using the default.

Added checks after config retrieval to use defaults when config values
are empty strings:
- gpu_layers: "20" (default for GPU layers)
- n_moe: "4" (default for MoE)
- parallel: "1" (default for parallel)
- n_predict: "50" (default for predict)
- n_ctx_size: "32000" (default for context size)

This fixes the error: "error while handling argument --n-gpu-layers: stoi"

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 13:17:26 +00:00
58adf7c4ae fix: Set default llm_server_path and correct model file paths
Some checks failed
BotServer CI / build (push) Has been cancelled
When no default.gbai/config.csv exists, the system now:
- Sets default llm_server_path to ./botserver-stack/bin/llm/build/bin
- Uses correct relative paths to model files: ../../../../data/llm/
- Uses actual model filenames from 3rdparty.toml

This fixes the issue where LLM/embedding servers couldn't find model files
because the paths were constructed incorrectly.

Model filenames:
- LLM: DeepSeek-R1-Distill-Qwen-1.5B-Q3_K_M.gguf
- Embedding: bge-small-en-v1.5-f32.gguf

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 13:11:14 +00:00
0e6e2bfc6d fix: Correct default LLM model to deepseek-small
All checks were successful
BotServer CI / build (push) Successful in 8m57s
Changed the default LLM model from glm-4 to deepseek-small to match
the model defined in 3rdparty.toml ([models.deepseek_small]).

This ensures that when no default.gbai/config.csv exists, the system
uses the correct default local model.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 12:56:15 +00:00
337bef3bad fix: Use default local LLM models when config is empty
Some checks failed
BotServer CI / build (push) Has been cancelled
When no default.gbai/config.csv exists or when llm-model/embedding-model
config is empty, the system now uses default local models instead of
skipping server startup.

Changes:
- Default LLM model: glm-4
- Default Embedding model: bge-small-en-v1.5
- Logs when using defaults

This fixes the issue where the "default" bot would fail to load LLM
and Embedding services when no config.csv was present, causing the
error: "not loading embedding neither llm local for default bot"

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 12:54:40 +00:00
cf7bd7ffa2 fix: Use spawn_blocking for Redis cache connection to prevent freezing
All checks were successful
BotServer CI / build (push) Successful in 8m38s
The init_redis() function was using synchronous blocking calls
(redis::Client::get_connection()) inside an async function, which
blocked the entire tokio runtime and caused botserver to freeze.

Changes:
- Wrap Redis connection calls in tokio::task::spawn_blocking()
- Runs blocking operations in separate thread pool
- Prevents tokio runtime from freezing during cache connection

This fixes the issue where botserver would hang indefinitely
when connecting to Valkey/Redis cache.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 12:21:33 +00:00
fc34461b2f fix: Improve cache_health_check TCP test to eliminate false positives
All checks were successful
BotServer CI / build (push) Successful in 8m28s
The previous /dev/tcp test was giving false positives, reporting that
Valkey was running when it was actually down. This caused bootstrap to
skip starting Valkey, leading to botserver hanging on cache connection.

Changes:
- Use nc (netcat) with -z flag for reliable port checking
- Final fallback: /dev/tcp with actual PING/PONG verification
- Only returns true if port is open AND responds correctly

This ensures cache_health_check() accurately reports Valkey status.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 12:02:23 +00:00
d243666cc2 fix: Support valkey-cli in cache_health_check for Valkey-only setups
All checks were successful
BotServer CI / build (push) Successful in 8m19s
- Try valkey-cli first (preferred for Valkey installations)
- Fall back to redis-cli (for Redis installations)
- Fall back to TCP connection test (works for both)

This fixes environments that only have Valkey installed without
Redis symlinks or redis-cli.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 11:48:02 +00:00
ba70bd97bc fix: Bootstrap should check Valkey is running and use correct component name
All checks were successful
BotServer CI / build (push) Successful in 8m56s
- Fix component name mismatch: "redis" -> "cache" in bootstrap_manager
- Add cache_health_check() function to verify Valkey is responding
- Add health check loop after starting cache (12s wait with PING test)
- Ensures cache is ready before proceeding with bootstrap

This fixes the issue where botserver would hang waiting for cache
connection because the cache component was never started.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 11:26:27 +00:00
dec3a086ab fix: Extract Valkey to cache/bin/ directory
All checks were successful
BotServer CI / build (push) Successful in 8m1s
- Binaries at cache/bin/valkey-server (correct production path)
- Use --strip-components=1 for extraction
- Matches /opt/gbo/bin/botserver-stack/bin/cache/bin/

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:22:59 +00:00
bfbdf8396f fix: Extract Valkey binaries directly to cache/ directory
Some checks failed
BotServer CI / build (push) Has been cancelled
- Use --strip-components=2 to flatten tarball structure
- Binaries go to cache/valkey-server (not cache/bin/valkey-server)
- Matches production path: /opt/gbo/bin/botserver-stack/bin/cache/

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:21:20 +00:00
0af972eda6 fix: Use correct valkey-server binary path
Some checks failed
BotServer CI / build (push) Has been cancelled
- Binary is at cache/bin/bin/valkey-server after extraction
- Update exec_cmd and check_cmd to use bin/ subdirectory
- Create symlinks at parent level for convenience

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:20:56 +00:00
14b3a0e6c0 fix: Remove redis symlinks from Valkey installation
Some checks failed
BotServer CI / build (push) Has been cancelled
- Use valkey-server and valkey-cli directly
- No redis compatibility symlinks needed
- Simplifies installation

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:20:08 +00:00
92d21fe3f7 fix: Use Valkey 8.1.5-jammy for GLIBC 2.36 compatibility
All checks were successful
BotServer CI / build (push) Successful in 8m54s
- noble version requires GLIBC 2.38 (Ubuntu 24.04)
- jammy version works with GLIBC 2.36 (Ubuntu 22.04)
- System has GLIBC 2.36, needs compatible binary

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:01:19 +00:00
0e1013e4ca fix: Downgrade Valkey from 9.0.2 to 8.1.5
Some checks failed
BotServer CI / build (push) Has been cancelled
- Use valkey-8.1.5-noble-x86_64.tar.gz instead of 9.0.2-jammy
- More stable version for production use

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 00:54:25 +00:00
278b92d5fa feat: Increase cache retry delay to 5 seconds
All checks were successful
BotServer CI / build (push) Successful in 8m25s
- Change retry interval from 1s to 5s between attempts
- Reduce attempts from 30 to 12 (still 60s total wait time)
- Gives Valkey more time to stabilize between connection attempts
- Helps with slow-to-start services during bootstrap

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 00:37:59 +00:00
8d0fd9da3d feat: Add retry logic for cache/Valkey connection
All checks were successful
BotServer CI / build (push) Successful in 8m19s
- Wait up to 30 seconds for cache to be ready
- Retry every 1 second with progress logging
- Prevents race condition during service startup
- Ensures suggestions feature works when Valkey starts after botserver

Fixes issue where cache connection failed during bootstrap if
Valkey wasn't immediately ready.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 00:24:46 +00:00
4e4aca0986 fix: Use precompiled Valkey binary from download.valkey.io
All checks were successful
BotServer CI / build (push) Successful in 7m53s
- Update 3rdparty.toml to download valkey-9.0.2-jammy-x86_64.tar.gz
- Remove source compilation approach (no gcc/make required)
- Update installer to handle precompiled binaries directly
- Fix symlinks for redis compatibility

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 23:51:33 +00:00