Commit graph

76 commits

Author SHA1 Message Date
21e0cb09e4 fix: resolve syntax errors and stabilize start.bas guard logic
All checks were successful
BotServer CI / build (push) Successful in 56s
2026-04-30 17:14:09 -03:00
d2314ce44b fix: unified atomic redis guard for start.bas in both websocket and stream flows
All checks were successful
BotServer CI / build (push) Successful in 47s
2026-04-30 17:08:38 -03:00
e9b1a3078c fix: claude client should not duplicate /v1/messages in URL
All checks were successful
BotServer CI / build (push) Successful in 48s
2026-04-30 16:56:45 -03:00
d15b712180 fix: sanitize malformed HTML tags (remove spaces after <)
All checks were successful
BotServer CI / build (push) Successful in 2m21s
2026-04-30 16:47:11 -03:00
debe29a6dc fix: start.bas redundant execution with atomic SET NX guard
All checks were successful
BotServer CI / build (push) Successful in 42s
2026-04-30 16:45:37 -03:00
5c54f7bc72 fix: robust HTML stripping for XLSX to remove CSS garbage and decode entities
All checks were successful
BotServer CI / build (push) Successful in 46s
2026-04-30 16:10:36 -03:00
2cf30ac388 trigger CI: actual file change
All checks were successful
BotServer CI / build (push) Successful in 19m24s
2026-04-30 15:40:54 -03:00
8c8e548377 fix: XLSX HTML stripping + start.bas session execution bug
Some checks failed
BotServer CI / build (push) Failing after 8m23s
2026-04-30 15:27:01 -03:00
2012f29324 fix: Strip HTML tags from XLSX content and frontend display
Some checks failed
BotServer CI / build (push) Failing after 7m9s
- Backend: Add strip_html_tags() function to remove HTML from XLSX cells
- Frontend: Strip HTML tags before displaying bot messages
- Prompt: Update PROMPT.md to instruct GPT not to show raw HTML

Fixes issue where XLSX cell content with HTML formatting
was being displayed as raw HTML tags in chat responses.
2026-04-30 14:15:12 -03:00
aeb567fb7f feat: optimize Salesianos bot with rich HTML, thinking strip, and updated deployment with backup
All checks were successful
BotServer CI / build (push) Successful in 51m13s
2026-04-30 09:50:03 -03:00
749b340cec Fix #498: XLSX and XLS files must be correctly indexed
- Changed extract_xlsx_text() to use open_workbook() instead of open_workbook_from_rs()
- Changed extract_xls_text() to use open_workbook() instead of open_workbook_from_rs()
- The calamine 0.26 API uses open_workbook(path) for direct file access
- This matches the pattern used in import_export.rs and vectordb.rs
- Updated AGENTS.md to clarify bots are stored as MinIO buckets
- Added test start.bas with USE KB "sheetlib" for testing KB injection

Root cause: open_workbook_from_rs() is not the correct API for calamine 0.26
Impact: XLSX and XLS files in .gbkb folders were failing to index into vector DB
Fix: Use standard open_workbook() API that accepts file path directly
2026-04-29 20:38:52 -03:00
3762cae53b Fix: XLSX and XLS files must be correctly indexed
- Changed extract_xlsx_text() to use open_workbook() instead of open_workbook_from_rs()
- Changed extract_xls_text() to use open_workbook() instead of open_workbook_from_rs()
- The calamine 0.26 API uses open_workbook(path) for direct file access
- This matches the pattern used in import_export.rs and vectordb.rs
- Fixes issue #498 where XLSX/XLS files were not being properly indexed into KB

Root cause: open_workbook_from_rs() is not the correct API for calamine 0.26
Impact: XLSX and XLS files in .gbkb folders were failing to index into vector DB
Fix: Use standard open_workbook() API that accepts file path directly
2026-04-29 19:23:52 -03:00
47cdb2bf78 refactor: remove deprecated documentation, configuration, and utility scripts 2026-04-29 10:35:16 -03:00
c445aa6de7 test(ci): trivial change to measure incremental build time with sccache
All checks were successful
BotServer CI / build (push) Successful in 25m59s
2026-04-29 10:13:14 -03:00
7177e64079 chore: tweak startup message 2026-04-29 08:43:47 -03:00
2c8ee74428 fix: correct info! macro format string 2026-04-29 08:43:47 -03:00
9814b7c2c9 feat: add GENERAL BOTS ASCII logo on startup 2026-04-29 08:43:46 -03:00
945dd9a116 chore: bump auth log version 2026-04-29 08:43:46 -03:00
5d9b1ebf80 chore: refine auth handler log format for traceability 2026-04-29 08:43:46 -03:00
c5b98c4e88 ci: timing test - incremental build
Some checks failed
BotServer CI / build (push) Has been cancelled
2026-04-28 19:46:27 -03:00
96fe7e5827 fix: Clean Cargo.toml junk, fix YAML indentation, sccache incremental=0 only in CI
Some checks failed
BotServer CI / build (push) Failing after 1m2s
2026-04-28 18:44:13 -03:00
203e874180 fix: Verify sccache cache preservation with CARGO_INCREMENTAL=0 2026-04-28 18:16:44 -03:00
bb027b22c8 test: Trigger CI to verify runner polling
All checks were successful
BotServer CI / build (push) Successful in 30m21s
2026-04-28 17:35:30 -03:00
848eda7b68 trigger: Force new CI job with updated workflow
Some checks are pending
BotServer CI / build (push) Waiting to run
2026-04-28 17:22:39 -03:00
54882bcddd test: Trigger rebuild to test mold+sccache
Some checks failed
BotServer CI / build (push) Failing after 8s
2026-04-28 16:14:45 -03:00
1a36f4aed2 fix: Embedding server auto-config and KB indexing fixes
- botserver/src/llm/local.rs:
  * Auto-configure embedding-url when empty (http://localhost:8082/v1/embeddings)
  * Auto-configure llm-url when empty (http://localhost:8081/v1/chat/completions)
  * Persist defaults to bot_configuration table via ConfigManager
  * Fix llama-server command path and add LD_LIBRARY_PATH
  * Fix working_dir for Linux (not just Windows)
  * Fix embedding server args: --embeddings --pooling mean --ctx-size 512

- botserver/src/core/bootstrap/bootstrap_manager.rs:
  * Temp disable alm-ci startup (causes 20s hang in bootstrap)
  * Remove unused alm_ci_health_check import

- Database changes:
  * bot_configuration.embedding-model: 'bge-small-en-v1.5-f32.gguf' (full filename)
  * bot_configuration.embedding-url: 'http://localhost:8082/v1/embeddings' (auto-generated)

- Testing:
  * Embedding server responds on port 8082
  * Generates 384-dimension embeddings
  * KB file exists: minio://default.gbai/default.gbkb/manual/test-kb-manual.txt
  * Next: Verify KB indexing and search functionality

Refs: #498
2026-04-28 14:27:00 -03:00
30056c29da Fix: ADD SWITCHER keyword normalization
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
- Root cause: Compiler wasn't converting 'ADD SWITCHER' to 'ADD_SWITCHER'
- Impact: Users writing 'ADD SWITCHER' (with space) had keyword fail to parse
- Files: botserver/src/basic/compiler/mod.rs (added normalization)
- Files: botserver/src/basic/mod.rs (fixed case-sensitive 'AS' check, removed redundant pattern)
- Testing: Verified syntax patterns match after normalization
2026-04-28 09:54:03 -03:00
6083beb248 Fix debug log format
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
2026-04-27 21:11:15 -03:00
6636f376f2 Fix: as_deref -> as_ref for String
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
2026-04-27 21:03:23 -03:00
12e4d1bd21 Use debug! macro for switcher logging (no file logging)
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
2026-04-27 20:59:29 -03:00
1ad2f7e528 Fix: use OpenOptions for file logging instead of non-existent append_to_file
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
2026-04-27 20:53:16 -03:00
8d82efa4e6 Add file-based debug logging to trace switcher storage/retrieval
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
Write to /tmp/switcher_debug.log when add_switcher() and
get_switchers() are called to debug why switchers aren't
rendering on stage.
2026-04-27 20:47:16 -03:00
47988eff92 Add debug logging to trace switcher storage/retrieval
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
Added info! logs to add_switcher() and get_switchers() to debug
why switchers are not rendering on stage.
2026-04-27 20:30:38 -03:00
2cc4fa65df Remove Redis flag - start.bas now runs on every WebSocket connection/message
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
Previously start.bas was prevented from re-running via a Redis flag
(set for 24h). This prevented switchers from being re-sent when
users reloaded the page. Now start.bas executes unconditionally
on every WebSocket connection and every first message.
2026-04-27 20:15:52 -03:00
ce0bb3791a Fix switcher Redis key mismatch: use DB session ID instead of WebSocket session ID
Some checks are pending
Botlib CI / build (push) Waiting to run
BotServer CI / build (push) Waiting to run
Bottest CI / build (push) Waiting to run
BotUI CI / build (push) Waiting to run
The switchers were stored in Redis with the DB session ID (session.id)
but retrieved using the WebSocket session ID, causing switchers to
never be found. Fix by saving session.id before the move into
spawn_blocking closure.
2026-04-27 19:52:37 -03:00
d723974410 fix(drive-monitor): sync bas files when not yet indexed + INFRA.md MinIO docs
Some checks failed
Botlib CI / build (push) Failing after 1s
BotServer CI / build (push) Failing after 1s
Bottest CI / build (push) Failing after 0s
BotUI CI / build (push) Failing after 1s
- Fix DriveMonitor skipping .bas files that exist but have indexed=false
  and same ETag (needs_reindex was computed but not used for bas sync)
- Mark bas files as indexed after successful sync to work dir
- Update INFRA.md: MinIO connection setup (port 9100, creds from drive.service)
- Remove erroneously created bottemplates start.bas (bots live in MinIO)
2026-04-27 17:27:00 -03:00
50977b5fa7 fix: KB indexing fails when Qdrant collection exists but has no vectors
- Check Qdrant collection exists and has points before skipping re-index
- Previously only checked database document_count, causing issues when:
  * Qdrant collection was deleted but DB still showed docs
  * Vectors were not properly indexed (indexed_vectors_count = 0)
  * New PDFs added to existing KB folder
- Now validates both DB and Qdrant state before deciding to skip indexing
- Logs warning when DB and Qdrant are out of sync

Fixes issue where KB from PDF files in drive were not being indexed to Qdrant
2026-04-27 16:49:38 +00:00
602a7da1dd fix: ADD SWITCHER keyword - compiler early normalization broke Rhai custom syntax
- Remove 'ADD SWITCHER' → 'ADD_SWITCHER' early replacement in preprocess_basic
  which prevented convert_multiword_keywords from properly handling the AS keyword
- Add ADD_SWITCHER (underscore) pattern alongside ADD SWITCHER (space) pattern
  so both forms are correctly converted to Rhai custom syntax
- Fix skip condition: only skip ADD_SWITCHER lines when 'as' is lowercase
  (already properly converted), not when AS is uppercase (needs conversion)
- Fix ADD\s+MEMBER double-escaped regex → ADD_MEMBER + ADD\s+MEMBER patterns
- Add CLEAR SWITCHERS pattern to convert_multiword_keywords
- Add CLEAR_SWITCHERS() skip condition

Closes #495
2026-04-27 11:19:59 +00:00
173418a141 ci: trigger botserver workflow for #495 stage deploy 2026-04-27 11:19:59 +00:00
75ca8cf513 Update botserver: Add bot access control with org membership check (#499) 2026-04-26 19:53:03 -03:00
7d2d8a5674 feat: Add bot access control with org membership check (#499)
All checks were successful
Botlib CI / build (push) Successful in 53s
BotServer CI / build (push) Successful in 19m23s
Bottest CI / build (push) Successful in 45s
BotUI CI / build (push) Successful in 58s
- Add is_public column to bots table (migration 6.3.2-01)
- Add user_organizations to Diesel schema for org membership lookup
- Implement check_bot_access(): public bots allow all users, private bots
  require user membership in the bot's organization via user_organizations
- Wire access check into websocket_handler before WS upgrade
- Read is_public from bots table instead of bot_configuration
- Add database_name field to Bot model
2026-04-26 19:45:30 -03:00
0911448548 feat: Add switcher toggle functionality
- Added SWITCHER_TOGGLE message type (8) for reprocessing last user message with active switchers
- Backend: Handler fetches last user question from DB, mutates message in-place, injects switcher prompts into system_prompt
- Backend: Switcher replays skip message_history save to avoid duplication
- Frontend: toggleSwitcher() sends SWITCHER_TOGGLE when input empty, sendMessage() when text present
- Frontend: Added TOOL_EXEC and SWITCHER_TOGGLE to MessageType constants
- Fixed session_id shadowing bug in DB query (used session_id_for_query)
- Preserves conversation history for LLM context when reprocessing with switchers
2026-04-26 16:36:03 -03:00
bd113323c7 fix: ADD SWITCHER parse_parameters handles AS keyword split (3-param case)
Some checks failed
Botlib CI / build (push) Successful in 24s
BotServer CI / build (push) Failing after 3m42s
Bottest CI / build (push) Successful in 41s
BotUI CI / build (push) Failing after 27s
2026-04-25 21:24:14 +00:00
8a7792a984 ci: trigger CI for #495 ADD SWITCHER fix 2026-04-25 14:30:51 +00:00
867c5c5be4 fix: ADD SWITCHER keyword - use register_custom_syntax instead of register_fn
The compiler outputs ADD_SWITCHER "x" as "y" format but the keyword
was registered as a plain Rhai function expecting add_switcher(x, y).
Changed to register_custom_syntax matching the output format, consistent
with ADD_SUGGESTION and CLEAR SWITCHERS patterns.

Fixes #495
2026-04-25 13:15:24 +00:00
c34b719515 Fix: remove .expect() from register_fn call in add_switcher_keyword
All checks were successful
Botlib CI / build (push) Successful in 2s
BotServer CI / build (push) Successful in 5m56s
Bottest CI / build (push) Successful in 29s
BotUI CI / build (push) Successful in 13s
2026-04-24 16:48:35 -03:00
21bcde15a6 Merge remote changes
Some checks failed
Botlib CI / build (push) Successful in 10s
BotServer CI / build (push) Failing after 1m25s
Bottest CI / build (push) Successful in 27s
BotUI CI / build (push) Successful in 16s
2026-04-24 16:45:26 -03:00
ab97e2da5d Fix: ADD SWITCHER keyword registration - Change from custom_syntax to register_fn to match multiword conversion pattern - Allows ADD SWITCHER "x" as "y" to convert to add_switcher("x", "y") 2026-04-24 16:42:08 -03:00
9f7da0a291 fix: ADD_SWITCHER keyword syntax - special handling for 'as' keyword format
All checks were successful
Botlib CI / build (push) Successful in 7s
Bottest CI / build (push) Successful in 31s
BotUI CI / build (push) Successful in 18s
The ADD_SWITCHER keyword was generating comma-separated params (ADD_SWITCHER(id, label))
instead of the expected Rhai custom syntax (ADD_SWITCHER id as label).

This fix adds special handling in the multiword processor to output the correct
format when processing ADD_SWITCHER with 2 parameters.

Fixes #495
2026-04-24 18:53:20 +00:00
3048832a2d feat: Add simple form of ADD SWITCHER keyword without AS clause
All checks were successful
Botlib CI / build (push) Successful in 13s
Bottest CI / build (push) Successful in 31s
BotUI CI / build (push) Successful in 15s
- ADD_SWITCHER "tables" now works (label defaults to switcher ID)
- ADD_SWITCHER "tables" AS "Tabelas" still works (custom label)
- Follows same pattern as ADD_SUGGESTION simple/AS forms
- Fixes #495
2026-04-24 18:22:17 +00:00