Commit graph

99 commits

Author SHA1 Message Date
7cbfe43319 Update attendance, keywords, calendar, compliance, console, core, drive, email, llm, msteams, security, and tasks modules 2025-12-24 09:29:27 -03:00
883c6d07e1 Remove all code comments and fix ratatui version 2025-12-23 18:40:58 -03:00
e2a5bf091a Update server components and keywords 2025-12-23 15:52:35 -03:00
5619cf2e16 fix: add DNS configuration to containers before apt-get
- Add Google DNS (8.8.8.8, 8.8.4.4) to /etc/resolv.conf
- Fixes 'Temporary failure resolving' errors in containers
2025-12-19 22:32:57 -03:00
2cff6bf430 fix: make exec_in_container return errors instead of silently warning
- Changed from warn() to error() and return Err()
- Added info logging for each command executed
- Now we can see why downloads/installs fail
2025-12-19 19:59:15 -03:00
27b5184f82 fix: pass VAULT_ADDR inside container via bash -c
- env() on Command only sets host env, not container env
- Use bash -c with VAULT_ADDR=... prefix for init and unseal commands
2025-12-19 13:19:50 -03:00
110ca2f3de fix: store vault-unseal-keys in working directory, not /opt/gbo/secrets 2025-12-19 12:28:22 -03:00
c750e26c3b feat: auto-generate .env and vault-unseal-keys on vault install
- Vault is automatically initialized with 5 keys, threshold 3
- Creates /opt/gbo/secrets/vault-unseal-keys with chmod 600
- Creates or appends to .env with VAULT_ADDR, VAULT_TOKEN, VAULT_UNSEAL_KEYS_FILE
- Vault is automatically unsealed after init
- No manual steps required for initial setup
2025-12-19 12:20:55 -03:00
4062c79cce feat: use secrets file for Vault auto-unseal (more secure)
- VAULT_UNSEAL_KEYS_FILE points to /opt/gbo/secrets/vault-unseal-keys
- Unseal keys stored in separate file with chmod 600
- Not in .env (which might be in git, logs, etc.)
- Output shows step-by-step setup instructions
2025-12-19 11:36:00 -03:00
2a8f884864 fix: improve container IP detection and add VAULT_UNSEAL_KEYS to .env
- Wait for network before getting IP
- Try hostname -I as fallback for IP detection
- Add VAULT_UNSEAL_KEY_1/2/3 to .env output for auto-unseal
- botserver can auto-unseal Vault on startup using these keys
2025-12-19 11:02:27 -03:00
3eb86772fa feat: show connection info after container install
- Only Vault returns .env variables (VAULT_ADDR, VAULT_TOKEN)
- All other components show 'botserver vault put' commands to store credentials
- Added proper vault init/unseal instructions
- CLI now prints InstallResult with IP, ports, and connection info
2025-12-19 10:49:09 -03:00
79e83dd3b8 fix: install base packages (wget, unzip, curl) in containers before download
- Vault download was failing because unzip was not installed
- Now all containers get wget, unzip, curl, ca-certificates as base packages
- Also added vault config.hcl generation in pre-install commands
2025-12-19 09:48:15 -03:00
2c61c816b5 fix: allow container install without Vault configured
- Make create_container_service work without DATABASE_URL from Vault
- Components like vault, vector_db that don't need DB can now install first
- DB_PASSWORD defaults to empty string when Vault is not available
2025-12-19 09:33:56 -03:00
8efeeaed4b fix: make container installation fully non-interactive
- Add automatic lxd init --auto before container creation
- Use DEBIAN_FRONTEND=noninteractive for apt-get
- Add apt-get update -qq before installing packages
2025-12-18 21:20:47 -03:00
3987546ebc feat: add fallback images for LXC container creation and install-dependencies.sh
- Try ubuntu:24.04, ubuntu:22.04, images:debian/12, images:debian/11 in order
- Clean up failed container attempts before trying next image
- Add install-dependencies.sh script for runtime library installation
- Fixes 404 error when LXD image server has outdated cached images
2025-12-18 21:09:26 -03:00
49b2886614 Add rotate-secret and rotate-secrets commands for credential rotation 2025-12-18 11:20:31 -03:00
f4d2172366 Add version --all command 2025-12-18 11:06:19 -03:00
952a822e1d Add vault CLI commands for secret migration 2025-12-18 09:40:16 -03:00
37c1cee805 Fix: Skip starting components that are already running (minio, valkey) 2025-12-16 10:48:42 -03:00
f1f0436fdd Fix vault health check - quote URL to prevent & being interpreted as shell background 2025-12-15 18:04:53 -03:00
562d3cf0e4 Add debug logs to package manager start function 2025-12-15 17:50:03 -03:00
4486e31320 Fix llm-server default to true, convert debug eprintln to trace 2025-12-15 16:33:13 -03:00
02117e2eec Fix Zitadel masterkey to come from Vault
- Add masterkey field to gbo/directory secret in Vault during bootstrap
- Generate random 32-char masterkey if not exists
- Update Zitadel exec_cmd to read masterkey from Vault
- Remove hardcoded masterkey values
2025-12-15 13:56:50 -03:00
37c2db21ef feat: configurable stack paths and local installer support
- Add BOTSERVER_INSTALLERS_PATH env var to use local installers without downloading
- Replace hardcoded ./botserver-stack paths with configurable stack_path
- Add stack_dir() and vault_bin() helper methods in BootstrapManager
- Add Port: 8300 to Zitadel config to fix port binding issue
- Start Directory service before setup_directory() call
- Add SKIP_LLM_SERVER env var to skip local LLM in tests
- Update template loading to check ../bottemplates and botserver-templates paths
2025-12-14 15:58:54 -03:00
7647774339 Move Directory (Zitadel) to port 8300
- Directory/Zitadel: 8080 -> 8300
- BotServer API remains on 8080
- Updated all references in bootstrap, installer, oauth, config
2025-12-11 08:43:28 -03:00
696a044d75 Update llama.cpp to b7345 with platform-specific builds and checksums
- Update 3rdparty.toml: llama.cpp b4547 -> b7345 with SHA256 checksums
- Add config/llm_releases.json with complete checksums for all 24 release assets
- Fix Windows binary naming in installer.rs (win-cpu-x64, win-cpu-arm64)
- Add Vulkan detection for Windows
- Add platform-specific variants: CUDA 12/13, Vulkan, HIP, SYCL, OpenCL
2025-12-10 12:54:52 -03:00
137a5c8012 fix(llm): Restore llama.cpp b6148 which was working 2025-12-10 10:46:54 -03:00
9fefd6c1fa fix(llm): Use pre-built llama.cpp binaries, remove compilation
- Revert to pre-built binaries (b4547) instead of compiling from source
- Remove cmake/build-essential dependencies
- Remove post_install compilation commands
- Fix exec_cmd path back to {{BIN_PATH}}/llama-server
2025-12-10 09:00:05 -03:00
051c8f720c fix(llm): Compile llama.cpp from source for CPU compatibility
Instead of downloading pre-built binaries (which may require AVX2),
compile llama.cpp from source during installation. This ensures:
- Works on older CPUs (Sandy Bridge, Haswell, etc.)
- Uses GGML_NATIVE=ON to optimize for the current CPU
- Binary path updated to build/bin/llama-server

Reverts the AVX2 detection that was incorrectly disabling LLM.
2025-12-10 08:43:27 -03:00
af6b797572 fix(llm): Detect AVX2 support and gracefully disable LLM on incompatible CPUs
- Add cpu_supports_avx2() function to check /proc/cpuinfo for AVX2 flag
- Skip LLM binary download on CPUs without AVX2 (pre-built llama.cpp requires it)
- Add runtime check for llama-server binary compatibility (catches 'Illegal instruction')
- Provide helpful error messages about alternatives (compile from source or use external API)
- Sandy Bridge (2nd gen i7) and older CPUs now gracefully skip local LLM
2025-12-10 08:35:35 -03:00
8711406865 fix(bootstrap): Improve Vault startup diagnostics and error handling
- Fix create_conn/establish_pg_connection to return Result instead of panicking
- Fix AppConfig::from_env to not require database access (circular dependency)
- Add #[cfg(test)] to AppState Default impl to prevent accidental panic
- Add extensive debug logging for Vault startup troubleshooting
- Remove Stdio::null() from start() to allow shell redirections to work
- Add direct vault start test in bootstrap for debugging
- Make Vault setup failure fatal (was silently continuing)
2025-12-10 08:30:49 -03:00
224ddd4a03 chore: Remove emoji icons from log messages and UI
Replace emoji icons with plain text/ASCII equivalents to prevent
encoding issues and improve console compatibility:
- Replace checkmarks with *
- Replace x marks with x
- Replace status indicators with text [OK], [ERR], etc.
- Remove decorative emojis from info/debug log messages
- Keep functional emojis in user-facing chat/sentiment features
2025-12-09 07:55:11 -03:00
185aba3631 fix(bootstrap): Suppress all subprocess output to prevent console UI corruption
- Redirect Vault exec_cmd output to logs/vault.log
- Add stdout/stderr null pipes to component spawn in installer.rs
- Suppress output in run_commands() in facade.rs
- All component output now goes to respective log files in logs/
2025-12-09 01:09:04 -03:00
a755d4d13e feat(console): Show UI immediately with live system logs
- Add state_channel field to XtreeUI for receiving AppState updates
- Add set_state_channel() method to enable async state communication
- Poll for AppState in event loop to initialize panels when ready
- UI now shows loading state instantly, logs stream in real-time
- Transitions to full interactive mode when AppState is received
2025-12-08 23:35:33 -03:00
05fa0e95c5 fix: remove duplicate logger init causing 'cannot set logger' error
Logger is already initialized in main.rs, cli.rs was trying to
initialize it again when CLI commands were run.
2025-12-08 17:59:43 -03:00
559154b923 fix: preserve cached files in botserver-installers after extraction
- Don't delete archive files from cache after tar/zip extraction
- Copy binaries from cache instead of moving them
- Cached files are now preserved for offline installation reuse
2025-12-08 14:19:55 -03:00
eed537ac42 feat: add offline installer cache and health endpoints
- Add /health and /api/health endpoints for botui connectivity
- Create 3rdparty.toml with all download URLs for offline bundles
- Add botserver-installers/ cache directory for downloaded files
- Implement DownloadCache module with:
  - Automatic cache lookup before downloading
  - Support for pre-populated offline bundles
  - SHA256 checksum verification (optional)
  - Cache management utilities (list, clear, size)
- Update download_and_install to use cache system
- Data files (models) also cached for reuse

Cache behavior:
- First run: downloads to botserver-installers/
- Subsequent runs: uses cached files
- Delete botserver-stack/ without losing downloads
- Pre-populate cache for fully offline installation
2025-12-08 14:08:49 -03:00
17d5a263f1 Fix config.csv loading on startup
- Disable TLS on Vault for local development (HTTP instead of HTTPS)
- Fix bot_configuration id column type mismatch (TEXT -> UUID)
- Add migration 6.1.1 to convert config table id columns to UUID
- Fix sync_config_csv_to_db to use UUID binding for id column
- Make start_all async with proper Vault startup sequence
- Sync default.gbai config.csv to existing 'Default Bot' from migrations
- Add diagnostic logging for config loading
- Change default LLM/embedding URLs from https to http for local dev
2025-12-08 00:19:29 -03:00
6ff2b32f2c feat(bootstrap): implement mTLS for Vault access
- Add create_vault_config() function to generate config.hcl with mTLS settings
- Configure Vault to require client certificate verification
- Generate client certificate for botserver in bootstrap
- Update .env to include mTLS paths (VAULT_CACERT, VAULT_CLIENT_CERT, VAULT_CLIENT_KEY)
- Remove unused import in tls.rs
2025-12-07 02:13:28 -03:00
b611f8c535 Add --stack-path CLI option for custom stack path
Allows overriding the default botserver-stack location via:
- --stack-path /path/to/stack CLI argument
- BOTSERVER_STACK_PATH environment variable

Useful for testing botserver bootstrap in isolated temp directories.
2025-12-06 14:55:42 -03:00
c76de427e3 Implement real database functions, remove TODOs and placeholders
- CRM Lead Scoring: Implement get_lead_score_from_db and update_lead_score_in_db
  using bot_memories table with diesel queries
- Bot Manager: Implement real org lookup from database and template loading from filesystem
- KB Manager: Implement get_collection_info to query Qdrant for real statistics
- Analytics: Replace placeholder metrics with actual database queries for users,
  sessions, and storage stats
- Email Setup: Implement Stalwart admin account creation via management API
- Add CollectionInfo struct for Qdrant collection metadata

All implementations use diesel for database operations, no sqlx.
2025-12-03 22:23:30 -03:00
c4c9521dd9 - Split into botui. 2025-12-02 21:09:43 -03:00
48288e8523 ``` Add KB Statistics keywords and infrastructure documentation
- Add KB Statistics keywords for Qdrant vector database monitoring: KB
  STATISTICS, KB COLLECTION STATS, KB DOCUMENTS COUNT, KB DOCUMENTS
  ADDED SINCE, KB LIST COLLECTIONS, KB STORAGE SIZE

- Add comprehensive infrastructure documentation:
  - Scaling and load balancing with LXC containers
  - Infrastructure design with encryption, sharding strategies
  - Observ
2025-11-30 16:25:51 -03:00
c6bbd50946 , dt.month, dt.hour, dt.is_weekend, etc.)
- Add startup wizard module for first-run configuration
- Add white-label branding system with .product file support
- Add bot manager for lifecycle, MinIO buckets, and templates
- Add version tracking registry for component updates
- Create comparison doc: BASIC vs n8n/Zapier/Make/Copilot
- Add WhatsApp-style sample dialogs to template documentation
- Add data traceability SVG diagram ```
2025-11-30 15:07:29 -03:00
ad1cd3ee36 HTMX enters. 2025-11-29 16:29:28 -03:00
c67ff407b1 Add .env.example with comprehensive configuration template
The commit adds a complete example environment configuration file
documenting all available settings for BotServer, including logging,
database, server, drive, LLM, Redis, email, and feature flags.

Also removes hardcoded environment variable usage throughout the
codebase, replacing them with configuration via config.csv or
appropriate defaults. This includes:

- WhatsApp, Teams, Instagram adapter configurations
- Weather API key handling
- Email and directory service configurations
- Console feature conditionally compiles monitoring code
- Improved logging configuration with library suppression
2025-11-28 13:19:03 -03:00
7f2967b04b Add SQLx dependencies for calendar feature 2025-11-27 23:10:43 -03:00
4b5d18db71 Add metadata and refactor BASIC compiler
- Add package metadata (keywords, categories) to Cargo.toml
- Add #[must_use] attributes to constructor methods
- Simplify conditional logic by inverting if-else blocks
- Replace string formatting with interpolation syntax
2025-11-27 15:19:17 -03:00
53bac1f5ec - Refactor folder as features. 2025-11-22 22:55:35 -03:00