Commit graph

65 commits

Author SHA1 Message Date
3762cae53b Fix: XLSX and XLS files must be correctly indexed
- Changed extract_xlsx_text() to use open_workbook() instead of open_workbook_from_rs()
- Changed extract_xls_text() to use open_workbook() instead of open_workbook_from_rs()
- The calamine 0.26 API uses open_workbook(path) for direct file access
- This matches the pattern used in import_export.rs and vectordb.rs
- Fixes issue #498 where XLSX/XLS files were not being properly indexed into KB

Root cause: open_workbook_from_rs() is not the correct API for calamine 0.26
Impact: XLSX and XLS files in .gbkb folders were failing to index into vector DB
Fix: Use standard open_workbook() API that accepts file path directly
2026-04-29 19:23:52 -03:00
47cdb2bf78 refactor: remove deprecated documentation, configuration, and utility scripts 2026-04-29 10:35:16 -03:00
c445aa6de7 test(ci): trivial change to measure incremental build time with sccache
All checks were successful
BotServer CI / build (push) Successful in 25m59s
2026-04-29 10:13:14 -03:00
7177e64079 chore: tweak startup message 2026-04-29 08:43:47 -03:00
2c8ee74428 fix: correct info! macro format string 2026-04-29 08:43:47 -03:00
9814b7c2c9 feat: add GENERAL BOTS ASCII logo on startup 2026-04-29 08:43:46 -03:00
945dd9a116 chore: bump auth log version 2026-04-29 08:43:46 -03:00
5d9b1ebf80 chore: refine auth handler log format for traceability 2026-04-29 08:43:46 -03:00
c5b98c4e88 ci: timing test - incremental build
Some checks failed
BotServer CI / build (push) Has been cancelled
2026-04-28 19:46:27 -03:00
96fe7e5827 fix: Clean Cargo.toml junk, fix YAML indentation, sccache incremental=0 only in CI
Some checks failed
BotServer CI / build (push) Failing after 1m2s
2026-04-28 18:44:13 -03:00
203e874180 fix: Verify sccache cache preservation with CARGO_INCREMENTAL=0 2026-04-28 18:16:44 -03:00
bb027b22c8 test: Trigger CI to verify runner polling
All checks were successful
BotServer CI / build (push) Successful in 30m21s
2026-04-28 17:35:30 -03:00
848eda7b68 trigger: Force new CI job with updated workflow
Some checks are pending
BotServer CI / build (push) Waiting to run
2026-04-28 17:22:39 -03:00
54882bcddd test: Trigger rebuild to test mold+sccache
Some checks failed
BotServer CI / build (push) Failing after 8s
2026-04-28 16:14:45 -03:00
1a36f4aed2 fix: Embedding server auto-config and KB indexing fixes
- botserver/src/llm/local.rs:
  * Auto-configure embedding-url when empty (http://localhost:8082/v1/embeddings)
  * Auto-configure llm-url when empty (http://localhost:8081/v1/chat/completions)
  * Persist defaults to bot_configuration table via ConfigManager
  * Fix llama-server command path and add LD_LIBRARY_PATH
  * Fix working_dir for Linux (not just Windows)
  * Fix embedding server args: --embeddings --pooling mean --ctx-size 512

- botserver/src/core/bootstrap/bootstrap_manager.rs:
  * Temp disable alm-ci startup (causes 20s hang in bootstrap)
  * Remove unused alm_ci_health_check import

- Database changes:
  * bot_configuration.embedding-model: 'bge-small-en-v1.5-f32.gguf' (full filename)
  * bot_configuration.embedding-url: 'http://localhost:8082/v1/embeddings' (auto-generated)

- Testing:
  * Embedding server responds on port 8082
  * Generates 384-dimension embeddings
  * KB file exists: minio://default.gbai/default.gbkb/manual/test-kb-manual.txt
  * Next: Verify KB indexing and search functionality

Refs: #498
2026-04-28 14:27:00 -03:00
30056c29da Fix: ADD SWITCHER keyword normalization
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
- Root cause: Compiler wasn't converting 'ADD SWITCHER' to 'ADD_SWITCHER'
- Impact: Users writing 'ADD SWITCHER' (with space) had keyword fail to parse
- Files: botserver/src/basic/compiler/mod.rs (added normalization)
- Files: botserver/src/basic/mod.rs (fixed case-sensitive 'AS' check, removed redundant pattern)
- Testing: Verified syntax patterns match after normalization
2026-04-28 09:54:03 -03:00
6083beb248 Fix debug log format
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
2026-04-27 21:11:15 -03:00
6636f376f2 Fix: as_deref -> as_ref for String
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
2026-04-27 21:03:23 -03:00
12e4d1bd21 Use debug! macro for switcher logging (no file logging)
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
2026-04-27 20:59:29 -03:00
1ad2f7e528 Fix: use OpenOptions for file logging instead of non-existent append_to_file
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
2026-04-27 20:53:16 -03:00
8d82efa4e6 Add file-based debug logging to trace switcher storage/retrieval
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
Write to /tmp/switcher_debug.log when add_switcher() and
get_switchers() are called to debug why switchers aren't
rendering on stage.
2026-04-27 20:47:16 -03:00
47988eff92 Add debug logging to trace switcher storage/retrieval
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
Added info! logs to add_switcher() and get_switchers() to debug
why switchers are not rendering on stage.
2026-04-27 20:30:38 -03:00
2cc4fa65df Remove Redis flag - start.bas now runs on every WebSocket connection/message
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
Previously start.bas was prevented from re-running via a Redis flag
(set for 24h). This prevented switchers from being re-sent when
users reloaded the page. Now start.bas executes unconditionally
on every WebSocket connection and every first message.
2026-04-27 20:15:52 -03:00
ce0bb3791a Fix switcher Redis key mismatch: use DB session ID instead of WebSocket session ID
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
The switchers were stored in Redis with the DB session ID (session.id)
but retrieved using the WebSocket session ID, causing switchers to
never be found. Fix by saving session.id before the move into
spawn_blocking closure.
2026-04-27 19:52:37 -03:00
d723974410 fix(drive-monitor): sync bas files when not yet indexed + INFRA.md MinIO docs
Some checks failed
Botlib CI / build (push) Failing after 1s
BotServer CI / build (push) Failing after 1s
Bottest CI / build (push) Failing after 0s
BotUI CI / build (push) Failing after 1s
- Fix DriveMonitor skipping .bas files that exist but have indexed=false
  and same ETag (needs_reindex was computed but not used for bas sync)
- Mark bas files as indexed after successful sync to work dir
- Update INFRA.md: MinIO connection setup (port 9100, creds from drive.service)
- Remove erroneously created bottemplates start.bas (bots live in MinIO)
2026-04-27 17:27:00 -03:00
50977b5fa7 fix: KB indexing fails when Qdrant collection exists but has no vectors
- Check Qdrant collection exists and has points before skipping re-index
- Previously only checked database document_count, causing issues when:
  * Qdrant collection was deleted but DB still showed docs
  * Vectors were not properly indexed (indexed_vectors_count = 0)
  * New PDFs added to existing KB folder
- Now validates both DB and Qdrant state before deciding to skip indexing
- Logs warning when DB and Qdrant are out of sync

Fixes issue where KB from PDF files in drive were not being indexed to Qdrant
2026-04-27 16:49:38 +00:00
602a7da1dd fix: ADD SWITCHER keyword - compiler early normalization broke Rhai custom syntax
- Remove 'ADD SWITCHER' → 'ADD_SWITCHER' early replacement in preprocess_basic
  which prevented convert_multiword_keywords from properly handling the AS keyword
- Add ADD_SWITCHER (underscore) pattern alongside ADD SWITCHER (space) pattern
  so both forms are correctly converted to Rhai custom syntax
- Fix skip condition: only skip ADD_SWITCHER lines when 'as' is lowercase
  (already properly converted), not when AS is uppercase (needs conversion)
- Fix ADD\s+MEMBER double-escaped regex → ADD_MEMBER + ADD\s+MEMBER patterns
- Add CLEAR SWITCHERS pattern to convert_multiword_keywords
- Add CLEAR_SWITCHERS() skip condition

Closes #495
2026-04-27 11:19:59 +00:00
173418a141 ci: trigger botserver workflow for #495 stage deploy 2026-04-27 11:19:59 +00:00
75ca8cf513 Update botserver: Add bot access control with org membership check (#499) 2026-04-26 19:53:03 -03:00
7d2d8a5674 feat: Add bot access control with org membership check (#499)
All checks were successful
Botlib CI / build (push) Successful in 53s
BotServer CI / build (push) Successful in 19m23s
Bottest CI / build (push) Successful in 45s
BotUI CI / build (push) Successful in 58s
- Add is_public column to bots table (migration 6.3.2-01)
- Add user_organizations to Diesel schema for org membership lookup
- Implement check_bot_access(): public bots allow all users, private bots
  require user membership in the bot's organization via user_organizations
- Wire access check into websocket_handler before WS upgrade
- Read is_public from bots table instead of bot_configuration
- Add database_name field to Bot model
2026-04-26 19:45:30 -03:00
0911448548 feat: Add switcher toggle functionality
- Added SWITCHER_TOGGLE message type (8) for reprocessing last user message with active switchers
- Backend: Handler fetches last user question from DB, mutates message in-place, injects switcher prompts into system_prompt
- Backend: Switcher replays skip message_history save to avoid duplication
- Frontend: toggleSwitcher() sends SWITCHER_TOGGLE when input empty, sendMessage() when text present
- Frontend: Added TOOL_EXEC and SWITCHER_TOGGLE to MessageType constants
- Fixed session_id shadowing bug in DB query (used session_id_for_query)
- Preserves conversation history for LLM context when reprocessing with switchers
2026-04-26 16:36:03 -03:00
bd113323c7 fix: ADD SWITCHER parse_parameters handles AS keyword split (3-param case)
Some checks failed
Botlib CI / build (push) Successful in 24s
BotServer CI / build (push) Failing after 3m42s
Bottest CI / build (push) Successful in 41s
BotUI CI / build (push) Failing after 27s
2026-04-25 21:24:14 +00:00
8a7792a984 ci: trigger CI for #495 ADD SWITCHER fix 2026-04-25 14:30:51 +00:00
867c5c5be4 fix: ADD SWITCHER keyword - use register_custom_syntax instead of register_fn
The compiler outputs ADD_SWITCHER "x" as "y" format but the keyword
was registered as a plain Rhai function expecting add_switcher(x, y).
Changed to register_custom_syntax matching the output format, consistent
with ADD_SUGGESTION and CLEAR SWITCHERS patterns.

Fixes #495
2026-04-25 13:15:24 +00:00
c34b719515 Fix: remove .expect() from register_fn call in add_switcher_keyword
All checks were successful
Botlib CI / build (push) Successful in 2s
BotServer CI / build (push) Successful in 5m56s
Bottest CI / build (push) Successful in 29s
BotUI CI / build (push) Successful in 13s
2026-04-24 16:48:35 -03:00
21bcde15a6 Merge remote changes
Some checks failed
Botlib CI / build (push) Successful in 10s
BotServer CI / build (push) Failing after 1m25s
Bottest CI / build (push) Successful in 27s
BotUI CI / build (push) Successful in 16s
2026-04-24 16:45:26 -03:00
ab97e2da5d Fix: ADD SWITCHER keyword registration - Change from custom_syntax to register_fn to match multiword conversion pattern - Allows ADD SWITCHER "x" as "y" to convert to add_switcher("x", "y") 2026-04-24 16:42:08 -03:00
9f7da0a291 fix: ADD_SWITCHER keyword syntax - special handling for 'as' keyword format
All checks were successful
Botlib CI / build (push) Successful in 7s
Bottest CI / build (push) Successful in 31s
BotUI CI / build (push) Successful in 18s
The ADD_SWITCHER keyword was generating comma-separated params (ADD_SWITCHER(id, label))
instead of the expected Rhai custom syntax (ADD_SWITCHER id as label).

This fix adds special handling in the multiword processor to output the correct
format when processing ADD_SWITCHER with 2 parameters.

Fixes #495
2026-04-24 18:53:20 +00:00
3048832a2d feat: Add simple form of ADD SWITCHER keyword without AS clause
All checks were successful
Botlib CI / build (push) Successful in 13s
Bottest CI / build (push) Successful in 31s
BotUI CI / build (push) Successful in 15s
- ADD_SWITCHER "tables" now works (label defaults to switcher ID)
- ADD_SWITCHER "tables" AS "Tabelas" still works (custom label)
- Follows same pattern as ADD_SUGGESTION simple/AS forms
- Fixes #495
2026-04-24 18:22:17 +00:00
6ad3ebff00 debug: Add logging for switcher prompt injection
All checks were successful
Botlib CI / build (push) Successful in 9s
Bottest CI / build (push) Successful in 34s
BotUI CI / build (push) Successful in 17s
2026-04-24 15:46:26 +00:00
02aaf528a9 Merge branch 'main' of https://alm.pragmatismo.com.br/GeneralBots/generalbots
All checks were successful
Botlib CI / build (push) Successful in 18s
Bottest CI / build (push) Successful in 40s
BotUI CI / build (push) Successful in 20s
2026-04-24 15:37:47 +00:00
6f9fdf5edb fix: Change DriveMonitor logging to trace level - Only log to info when actual changes occur in gbdialog, gbkb, or gbot - Reduces log noise from每秒 scans 2026-04-24 15:37:11 +00:00
6211e51d28 Fix: Remove invalid DriveConfig::from_vault() call
Some checks are pending
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
Botlib CI / build (push) Successful in 5s
BotServer CI / build (push) Successful in 6m4s
- DriveConfig::default() already loads from Vault
- No need for separate from_vault() method
- CI should compile now
2026-04-24 10:51:25 -03:00
3c5c0949c3 Fix: Remove hardcoded defaults, require Vault for production config
Some checks failed
Botlib CI / build (push) Successful in 12s
BotServer CI / build (push) Failing after 1m49s
Bottest CI / build (push) Successful in 35s
BotUI CI / build (push) Successful in 16s
- AppConfig::default() and from_env() now read PORT from environment
- DriveConfig loads from Vault first, falls back to env vars for development
- DATABASE_URL must be set (from Vault via .env in production)
- No more hardcoded 8080 - reads from PORT env var
- Removed hardcoded paths like /opt/gbo/data, now from env or reasonable defaults
- Development still works with defaults, production requires Vault via .env
2026-04-24 10:43:34 -03:00
53867a3ef6 Fix: Read PORT from environment variable instead of hardcoded 8080
- Changed AppConfig::from_env() to read PORT from std::env
- Falls back to 8080 if PORT not set
- Also reads HOST, SITE_PATH, DATA_DIR from environment
- Fixes issue where botserver ignored .env PORT=5858 setting
2026-04-24 10:36:33 -03:00
b1f537a433 fix: Use management v1 API for Zitadel user search (PAT token compat)
Some checks failed
Botlib CI / build (push) Successful in 3s
BotServer CI / build (push) Failing after 5m17s
Bottest CI / build (push) Successful in 36s
BotUI CI / build (push) Successful in 16s
The v2 user search API rejects PAT tokens (returns Token.Invalid).
The management v1 API accepts PAT tokens correctly.
Also adds fallback for user ID field (id vs userId) in login response.
2026-04-23 23:14:03 +00:00
f694e4a818 fix: Use management v1 API for Zitadel user search (PAT token compat)
The v2 user search API rejects PAT tokens (returns Token.Invalid).
The management v1 API accepts PAT tokens correctly.
Also adds fallback for user ID field (id vs userId) in login response.
2026-04-23 22:44:17 +00:00
cf00a51245 feat: Separate switchers from suggestions - server-side prompt resolution
Some checks failed
Botlib CI / build (push) Successful in 22s
BotServer CI / build (push) Failing after 0s
Bottest CI / build (push) Successful in 31s
BotUI CI / build (push) Successful in 52s
- Add Switcher struct to botlib (id, label, prompt, color, icon)
- Separate Redis key switchers:{bot_id}:{session_id} from suggestions
- Server-side SWITCHER_PROMPT_MAP resolves prompts (not frontend)
- Frontend sends active_switchers array in USER WS message
- Remove SYSTEM message dispatch (type 7 deprecated)
- Remove hardcoded promptMap from chat.html
- Render switchers from BotResponse.switchers (backend-driven)
- Support custom switchers via ADD_SWITCHER keyword
- Filter switcher-type items from get_suggestions()
- Add switchers field to all BotResponse construction sites
2026-04-23 21:36:09 +00:00
b20544d1a0 Merge branch 'main' of github.com:generalbots/generalbots into main
Some checks failed
Botlib CI / build (push) Failing after 2s
Bottest CI / build (push) Failing after 1s
BotServer CI / build (push) Failing after 3m40s
BotUI CI / build (push) Failing after 22s
2026-04-22 21:31:52 -03:00
61c8f709c4 fix(stage): fix message type collision and switcher logic in chat ui 2026-04-22 21:30:26 -03:00