Commit graph

3971 commits

Author SHA1 Message Date
ad4aca21ff feat: implement WhatsApp bursting rate limit and consume activation tokens
All checks were successful
BotServer CI / build (push) Successful in 11m4s
2026-03-10 21:18:14 -03:00
e98de24fe6 chore: update submodules
All checks were successful
BotServer CI / build (push) Successful in 9m56s
2026-03-10 19:39:31 -03:00
1053c86a73 fix: whatsapp dynamic routing and openai tool call accumulation
All checks were successful
BotServer CI / build (push) Successful in 13m40s
2026-03-10 17:19:17 -03:00
786d404938 feat: handle 429 rate limit in OpenAI non-stream generate
All checks were successful
BotServer CI / build (push) Successful in 11m7s
2026-03-10 15:26:10 -03:00
f34d401c2e feat: handle 429 rate limit in OpenAI client
Some checks failed
BotServer CI / build (push) Has been cancelled
2026-03-10 15:21:40 -03:00
260a13e77d refactor: apply various fixes across botserver
Some checks failed
BotServer CI / build (push) Has been cancelled
2026-03-10 15:15:21 -03:00
82bfd0a443 Fix Bedrock config for OpenAI GPT-OSS models
All checks were successful
BotServer CI / build (push) Successful in 12m35s
2026-03-10 12:36:24 -03:00
c523cee177 Use Redis to track last sent time per WhatsApp recipient
All checks were successful
BotServer CI / build (push) Successful in 13m37s
- Store last_sent timestamp in Redis (whatsapp:last_sent:<phone>)
- Always wait 6 seconds between messages to same recipient
- Persists across restarts
2026-03-09 21:00:45 -03:00
4bda4ba897 Fix WhatsApp streaming: add response content to buffer
All checks were successful
BotServer CI / build (push) Successful in 8m50s
2026-03-09 20:23:38 -03:00
9f35863bff Simplify hallucination detector: only stop if 50+ repetitions
All checks were successful
BotServer CI / build (push) Successful in 9m8s
- Simple: count pattern repetitions, stop at 50
- Async API with Redis-backed counting
- 60-second window for cleanup
2026-03-09 20:02:29 -03:00
032de108fd Relax hallucination detector: ignore Markdown separators, increase thresholds
- Ignore ---, ***, ___ (legitimate Markdown)
- Increase consecutive threshold: 5 → 10
- Increase occurrence threshold: 8 → 15
- Increase token threshold: 10 → 15
2026-03-09 19:50:34 -03:00
47e1013700 Fix Redis BLPOP return type: handle (key, value) tuple
All checks were successful
BotServer CI / build (push) Successful in 13m52s
2026-03-09 19:43:47 -03:00
5d3174b24e Fix compilation: add Debug derive and type annotations
Some checks failed
BotServer CI / build (push) Has been cancelled
2026-03-09 19:32:36 -03:00
d22ce019b6 Implement Meta WhatsApp official rate limits: 1msg/6s, 45 burst, 4^X retry
Some checks failed
BotServer CI / build (push) Failing after 8m45s
- Add Redis-backed message queue with per-recipient tracking
- Enforce 1 message per 6 seconds per recipient (0.17 msg/s)
- Support burst mode: up to 45 messages in 6-second window
- Implement proportional cooldown after burst
- Add exponential backoff retry on error 131056 (4^X seconds)
- Update botbook with official Meta rate limits
- Add unit tests for burst mode and rate limiting
- Fix config inheritance bug: delete all keys before sync
2026-03-09 19:16:41 -03:00
77c35ccde5 feat: add WhatsApp rate limiting and LLM hallucination detection
All checks were successful
BotServer CI / build (push) Successful in 11m51s
2026-03-09 17:22:47 -03:00
24709f7811 feat(whatsapp): add phone_number_id based bot routing
All checks were successful
BotServer CI / build (push) Successful in 12m13s
- Add resolve_bot_by_phone_number_id function for automatic routing
- Webhooks now route to correct bot based on whatsapp-phone-number-id
- Enables multiple WhatsApp numbers to use single webhook URL
- Falls back to default bot if no match found

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-09 12:30:26 -03:00
c072fb936e fix(llm): load system-prompt from config.csv correctly
All checks were successful
BotServer CI / build (push) Successful in 17m27s
- Move system_prompt retrieval inside spawn_blocking closure
- Include system_prompt in the return tuple to fix scope issue
- Add trace logging for debugging system-prompt loading
- GLM-5 and other LLM providers now correctly receive custom system prompts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-09 11:55:05 -03:00
97661d75e2 feat(whatsapp): isolate lists as single messages and remove code blocks
- Split list detection into numbered and bullet list items
- Add looks_like_list_start() to detect when list is beginning
- Add looks_like_list_end() to detect when list has ended
- Add split_text_before_list() to separate text before list
- Add split_list_from_text() to separate list from text after
- Update streaming logic to send lists as isolated messages
- Add code block removal (triple backticks and inline backticks)
- Add comprehensive unit tests for list detection functions

Resolves: Lists being mixed with other text in WhatsApp messages
Resolves: JavaScript/C# code leaking into WhatsApp messages
2026-03-08 14:52:59 -03:00
c5d69f9752 feat(whatsapp): Add /clear command for session history
- Add /clear command handler to allow users to clear their conversation history
- Implement clear_session_history() function using diesel delete
- Remove dead code (unused list processing functions)
- Add message deduplication using Redis cache

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-07 18:58:00 -03:00
85b4653899 Fix WhatsApp streaming to send complete messages
- Accumulate all content before sending (no chunking)
- Only send when is_final = true
- Fixes list (li/ul) handling - lists sent as one complete message

- Improves WhatsApp user experience by sending complete formatted responses
- Removes complex chunked logic in favor of simplicity

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-06 18:52:53 -03:00
859db6b8a0 fix: Lower KB search thresholds and add Cloudflare AI embedding support
All checks were successful
BotServer CI / build (push) Successful in 10m35s
- Lower score_threshold in kb_indexer.rs from 0.5 to 0.3
- Lower website search threshold in kb_context.rs from 0.6 to 0.4
- Lower KB search threshold in kb_context.rs from 0.7 to 0.5
- Add Cloudflare AI (/ai/run/) URL detection in cache.rs
- Add Cloudflare AI request format ({"text": ...}) in cache.rs
- Add Cloudflare AI response parsing (result.data) in cache.rs

This fixes the issue where KB search returned 0 results even with
114 chunks indexed. The high thresholds were filtering out all results.
2026-03-05 00:11:08 -03:00
8500949fcd fix: Lower KB search thresholds and add Cloudflare AI embedding support
- Lower score_threshold in kb_indexer.rs from 0.5 to 0.3
- Lower website search threshold in kb_context.rs from 0.6 to 0.4
- Lower KB search threshold in kb_context.rs from 0.7 to 0.5
- Add Cloudflare AI (/ai/run/) URL detection in cache.rs
- Add Cloudflare AI request format ({"text": ...}) in cache.rs
- Add Cloudflare AI response parsing (result.data) in cache.rs

This fixes the issue where KB search returned 0 results even with
114 chunks indexed. The high thresholds were filtering out all results.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-05 00:06:17 -03:00
e389178a36 feat: Add HuggingFace embedding API support with authentication
- Added api_key field to LocalEmbeddingService for authentication
- Added embedding-key config parameter support in bootstrap
- Smart URL handling: doesn't append /embedding for full URLs (HuggingFace, OpenAI, etc.)
- Detects HuggingFace URLs and uses correct request format (inputs instead of input/model)
- Handles multiple response formats:
  - HuggingFace: direct array [0.1, 0.2, ...]
  - Standard/OpenAI: {"data": [{"embedding": [...]}]}
- Added Authorization header with Bearer token when api_key is provided
- Improved error messages with full response details

Fixes embedding errors when using HuggingFace Inference API
2026-03-04 16:54:25 -03:00
d5b877f8e8 fix: Improve embedding error handling and add semantic cache toggle
- Enhanced error messages in LocalEmbeddingService to show actual HTTP status and response
- Added semantic-cache-enabled config parameter to disable semantic matching when embedding service unavailable
- Improved error logging with full response details for debugging production issues
- Prevents 'Invalid embedding response' errors by allowing graceful fallback
2026-03-04 16:49:22 -03:00
5404e3e7ba feat: Enhance KB context, embedding generator, and website crawler
- Improved kb_context with better context management
- Enhanced embedding_generator with extended functionality (+231 lines)
- Updated kb_indexer with improved indexing logic
- Expanded website_crawler_service capabilities (+230 lines)
- Updated use_website keyword implementation
- Refined bootstrap_manager and utils
- Improved drive monitoring and local file monitor
- Added server enhancements
2026-03-04 15:43:37 -03:00
7a22798c23 Fix deployment module: add types, router, handlers and fix forgejo integration 2026-03-03 14:30:44 -03:00
6195062482 fix: add missing PathBuf import 2026-03-03 10:03:49 -03:00
b2d6a51ebd fix: correct tool file path resolution to match architecture
- Updated get_tool_bas_path to look in /opt/gbo/data (source) first
- Then check botserver-stack/data/system/work (compiled) second
- Removed incorrect $HOME/data and $HOME/gb/work paths
- Fixes 'Tool file not found' error when executing inscricao tool

According to AGENTS.md architecture:
- Bots are in /opt/gbo/data primary
- They are compiled into work directory by local_file_monitor
- tool_executor was looking in wrong directories
2026-03-03 10:01:33 -03:00
c03398fe56 feat: Add deployment API endpoints and routes (Phase 0 completion)
Some checks failed
BotServer CI / build (push) Failing after 1m45s
- Add API endpoints to deployment/mod.rs:
  - GET /api/deployment/targets - List available deployment targets
  - POST /api/deployment/deploy - Deploy application to selected target
- Register deployment routes in main application router
- Support for internal GB Platform and external Forgejo deployments
- Proper error handling with ErrorSanitizer
- SQL injection protection with sql_guard

Phase 0: Deployment Infrastructure - COMPLETE 
2026-03-02 07:42:28 -03:00
b42a7e5cb2 feat: Add Phase 1 Code Editor Integration (P0 - Critical)
Some checks failed
BotServer CI / build (push) Failing after 1m53s
Phase 1: Code Editor Integration
- Add Monaco Editor to vendor directory
- Create editor.html component with full Monaco integration
- Create API endpoints for file operations (editor.rs)
  - GET /api/editor/file/:file_path - Read file
  - POST /api/editor/file/:file_path - Save file
  - GET /api/editor/files - List files
- Features:
  - File tree sidebar
  - Multi-file tabs
  - Syntax highlighting for 10+ languages
  - Auto-save with WebSocket sync preparation
  - Keyboard shortcuts (Ctrl+S, Ctrl+P)
  - Status bar with language, encoding, cursor position
  - Custom GB dark theme
  - Publish integration with deployment modal
2026-03-02 07:26:35 -03:00
33d6f90ba8 feat: Add Phase 0 deployment infrastructure (CRITICAL)
Some checks failed
BotServer CI / build (push) Failing after 2m6s
Phase 0.1: Deployment Router
- Create deployment module with DeploymentRouter
- Support internal (GB Platform) and external (Forgejo) deployment
- Add proper error handling and result types

Phase 0.2: Forgejo Integration
- Create ForgejoClient for repository management
- Implement git push functionality with git2
- Add CI/CD workflow generation for HTMX, React, Vue apps
- Support custom domains and automated deployments

Phase 0.3: Backend preparation
- Add deployment types and configuration structures
- Prepare integration with orchestrator
2026-03-02 07:12:30 -03:00
1e71c9be09 feat: Add comprehensive stage progress logging
Some checks failed
BotServer CI / build (push) Failing after 2m39s
- Add detailed logging for all 5 pipeline stages (PLAN, BUILD, REVIEW, DEPLOY, MONITOR)
- Log stage start/complete events with agent IDs and progress details
- Add resource creation/deletion logging in drive_handlers
- Improve pipeline summary logging with task ID, nodes, resources, and URL

This addresses the requirement for textual progress in console logs.
2026-03-01 22:36:14 -03:00
c326581a9e fix(zitadel): resolve OAuth client initialization timing issue
- Fix PAT extraction timing with retry loop (waits up to 60s for PAT in logs)
- Add sync command to flush filesystem buffers before extraction
- Improve logging with progress messages and PAT verification
- Refactor setup code into consolidated setup.rs module
- Fix YAML indentation for PatPath and MachineKeyPath
- Change Zitadel init parameter from --config to --steps

The timing issue occurred because:
1. Zitadel writes PAT to logs at startup (~18:08:59)
2. Post-install extraction ran too early (~18:09:35)
3. PAT file wasn't created until ~18:10:38 (63s after installation)
4. OAuth client creation failed because PAT file didn't exist yet

With the retry loop:
- Waits for PAT to appear in logs with sync+grep check
- Extracts PAT immediately when found
- OAuth client creation succeeds
- directory_config.json saved with valid credentials
- Login flow works end-to-end

Tested: Full reset.sh and login verification successful
2026-03-01 19:06:09 -03:00
1bbb94d500 fix(directory): improve error message for invalid credentials
Some checks failed
BotServer CI / build (push) Failing after 2m34s
- Updated error message to explain when credentials are found but invalid
- Clarified that credentials might be from a previous installation
- Added step-by-step solution for resetting credentials
- Removed misleading 'No admin credentials found' message

The error now accurately reflects the actual problem: authentication
failure rather than missing credentials.
2026-03-01 10:17:30 -03:00
8016caa790 fix(bootstrap): create OAuth client when Zitadel is already running
Some checks failed
BotServer CI / build (push) Failing after 3m20s
- Added OAuth client creation to the 'already running' branch
- Previously, OAuth client was only created when Zitadel was started
- This fixes the issue where OAuth client wasn't created on subsequent runs

Fixes the OAuth client creation issue discovered during testing.
2026-03-01 10:11:42 -03:00
2a6c599c75 feat(directory): read admin credentials from ~/.gb-setup-credentials
Some checks failed
BotServer CI / build (push) Failing after 5m3s
- Updated setup_directory() to read credentials from saved file
- Added read_saved_credentials() to parse ~/.gb-setup-credentials
- Added get_admin_credentials() to try multiple sources
- Removed default credentials approach (doesn't work)
- Improved error messages with solution steps

This matches the working approach from commit 86cfccc2 where
credentials were saved during first bootstrap and reused for
OAuth client creation on subsequent runs.
2026-03-01 10:06:30 -03:00
eb5c12c466 fix(directory): add .await to ensure_admin_token() calls
Some checks failed
BotServer CI / build (push) Failing after 5m28s
Fixed compilation errors by adding .await to all ensure_admin_token() calls:
- create_organization()
- create_user()
- save_config()

The method was made async but the calls weren't updated.
2026-03-01 09:52:31 -03:00
2c92a81302 merge: Unify master into main - all commits unified
Some checks failed
BotServer CI / build (push) Failing after 6m9s
2026-03-01 07:43:07 -03:00
8f495c75ec WIP: Local changes before merging master into main 2026-03-01 07:40:11 -03:00
bbdf243c86 fix: OAuth client creation during bootstrap
Some checks failed
BotServer CI / build (push) Failing after 6m2s
- Add password grant authentication support in DirectorySetup
- Extract initial admin credentials from Zitadel log file
- Fix race condition in Zitadel startup (wait for health check before starting)
- Create parent directories before saving config
- Add retry logic for OAuth client creation
- Improve error handling with detailed messages

Fixes authentication service not configured error after bootstrap.
2026-03-01 07:31:17 -03:00
0b1b17406d chore(core): various email and security updates 2026-02-24 19:02:48 -03:00
764f058653 fix: update work directory paths to use botserver-stack/data/system/work
All checks were successful
BotServer CI / build (push) Successful in 7m4s
Updated all hardcoded work/ directory references to use the correct
relative path from the current working directory:
- botserver-stack/data/system/work

This ensures consistent file location resolution regardless of where
botserver is run from (/home/rodriguez/src/gb/ or /opt/gbo/bin/).

Changes:
- local_file_monitor.rs: Use std::env::current_dir() for work_root
- drive_monitor/mod.rs: Use work_root PathBuf for tool compilation
- website_crawler_service.rs: Use std::env::current_dir() for work_path

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-22 16:20:07 -03:00
1856215d05 chore: update dependencies and formatting
All checks were successful
BotServer CI / build (push) Successful in 7m30s
2026-02-22 15:55:39 -03:00
affb702e90 fix(tools): auto-create work directory if missing in production
All checks were successful
BotServer CI / build (push) Successful in 8m12s
Added logic to create botserver-stack/data/system/work directory
if it doesn't exist. This ensures production deployments work
without manual directory setup.

Changes:
- Added fs::create_dir_all() in use_tool.rs
- Added fs::create_dir_all() in tool_context.rs
- Logs when directory is created
- Fixes production deployment where /system/work may not exist

This ensures the tool loading works in fresh production environments
where the work folder hasn't been populated yet.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-22 12:58:58 -03:00
febfa2e708 fix(tools): use relative path for tool .mcp.json files
All checks were successful
BotServer CI / build (push) Successful in 8m37s
Changed tool loading to use relative path from current directory instead
of hardcoded HOME/gb path. This makes the code portable across different
deployment environments.

- Updated use_tool.rs to use std::env::current_dir()
- Updated tool_context.rs to use std::env::current_dir()
- Added PathBuf import to both files
- Tools now load from botserver-stack/data/system/work/

Fixes issue where tools weren't being loaded because .mcp.json files
were in a different location than expected.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-22 11:21:07 -03:00
de017241f2 fix: Complete security remediation - RCE and SSRF fixes
All checks were successful
BotServer CI / build (push) Successful in 7m34s
- Fixed RCE vulnerability in trusted_shell_script_arg execution
- Fixed SSRF vulnerability in GET command with internal IP blocking
- Updated SafeCommand to use explicit positional arguments

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 01:14:14 +00:00
e143968179 feat: Add JWT secret rotation and health verification
SEC-02: Implement credential rotation security improvements

- Add JWT secret rotation to rotate-secret command
- Generate 64-character HS512-compatible secrets
- Automatic .env backup with timestamp
- Atomic file updates via temp+rename pattern
- Add health verification for rotated credentials
- Route rotate-secret, rotate-secrets, vault commands in CLI
- Add verification attempts for database and JWT endpoints

Security improvements:
- JWT_SECRET now rotatable (previously impossible)
- Automatic rollback via backup files
- Health checks catch configuration errors
- Clear warnings about token invalidation

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-19 19:42:41 +00:00
df9b228a35 feat(drive): Implement S3 handlers (list/read/write/delete) 2026-02-19 12:38:43 +00:00
98813fbdc8 chore: Fix warnings and clean TODO refs 2026-02-19 12:18:39 +00:00
ac5b814536 fix(security): Fix unsafe code, CORS logic, and expect usage 2026-02-19 12:06:05 +00:00