Commit graph

28 commits

Author SHA1 Message Date
e200f47864 Fix Zitadel masterkey to come from Vault
- Add masterkey field to gbo/directory secret in Vault during bootstrap
- Generate random 32-char masterkey if not exists
- Update Zitadel exec_cmd to read masterkey from Vault
- Remove hardcoded masterkey values
2025-12-15 13:56:50 -03:00
17618f692c feat: configurable stack paths and local installer support
- Add BOTSERVER_INSTALLERS_PATH env var to use local installers without downloading
- Replace hardcoded ./botserver-stack paths with configurable stack_path
- Add stack_dir() and vault_bin() helper methods in BootstrapManager
- Add Port: 8300 to Zitadel config to fix port binding issue
- Start Directory service before setup_directory() call
- Add SKIP_LLM_SERVER env var to skip local LLM in tests
- Update template loading to check ../bottemplates and botserver-templates paths
2025-12-14 15:58:54 -03:00
48f3cfb6f3 feat: Add Auto Task system - Intent Compiler, MCP Client, Safety Layer
- intent_compiler.rs: LLM-to-BASIC translation engine
- auto_task.rs: Auto-executing task data structures
- mcp_client.rs: Model Context Protocol server integration
- safety_layer.rs: Constraints, simulation, audit trail
- autotask_api.rs: HTTP API handlers for Auto Task UI
- Updated mod.rs with new modules and keyword list
2025-12-12 12:33:17 -03:00
777a3eae63 Move Directory (Zitadel) to port 8300
- Directory/Zitadel: 8080 -> 8300
- BotServer API remains on 8080
- Updated all references in bootstrap, installer, oauth, config
2025-12-11 08:43:28 -03:00
49304a70b5 Fix Vault re-init to preserve other services + simplify shutdown message
- When Vault unseal fails, only restart Vault - NOT full bootstrap
- Preserve PostgreSQL, Redis, MinIO, etc. when Vault needs re-init
- Simplify shutdown message to 3 lines with pragmatismo.com.br
- Never kill all stack processes just for Vault issues
2025-12-10 18:41:45 -03:00
0047ad3937 fix(bootstrap): Kill existing stack processes at bootstrap start
- Call kill_stack_processes() at the beginning of bootstrap()
- Add port-based killing using fuser for ports 8200, 5432, 9000, etc.
- Use -f (pattern match) instead of -x (exact) for process names
- Fix process names: 'vault server', 'llama-server' instead of 'vault', 'ollama'
- Increase wait time to 1 second for processes to terminate

This fixes issues on dev machines where old processes from a deleted
stack are still running and blocking ports.
2025-12-10 08:54:51 -03:00
2da5f0ccdf fix(bootstrap): Improve Vault startup diagnostics and error handling
- Fix create_conn/establish_pg_connection to return Result instead of panicking
- Fix AppConfig::from_env to not require database access (circular dependency)
- Add #[cfg(test)] to AppState Default impl to prevent accidental panic
- Add extensive debug logging for Vault startup troubleshooting
- Remove Stdio::null() from start() to allow shell redirections to work
- Add direct vault start test in bootstrap for debugging
- Make Vault setup failure fatal (was silently continuing)
2025-12-10 08:30:49 -03:00
0b9ad6c80d fix(bootstrap): NEVER delete user data, suppress migration output, protect secrets
Critical safety improvements:

1. REMOVED clean_stack_directory() - NEVER delete botserver-stack
   - Contains user data, configs, databases
   - Only reset_vault_only() for Vault re-initialization

2. Single instance check
   - check_single_instance() uses .lock file with PID
   - Prevents multiple botserver processes on same stack

3. Protect existing Vault secrets
   - Check if secret exists before writing
   - Never overwrite customer credentials in distributed environments
   - Especially critical for encryption key

4. Suppress migration output
   - Use MigrationHarness directly instead of HarnessWithOutput
   - Prevents console UI corruption from migration messages
2025-12-09 09:04:56 -03:00
a2b091914f fix(bootstrap): start_all() must re-bootstrap when Vault unseal fails
When start_all() detects Vault init.json is missing (unseal fails),
it now:
1. Kills all stack processes
2. Cleans the stack directory
3. Runs full bootstrap()
4. Returns (bootstrap starts all services)

Also made SecretsManager init failure in start_all() a hard error
instead of a warning - if Vault is supposed to be working but
SecretsManager can't connect, that's a fatal error.
2025-12-09 08:56:58 -03:00
1e98083d13 fix(bootstrap): Write .env IMMEDIATELY after Vault unseal, re-init SecretsManager
The core issue was that .env was written at the END of setup_vault(),
but other components needed Vault credentials BEFORE that.

Now:
1. Unseal Vault
2. Write .env with VAULT_ADDR and VAULT_TOKEN immediately
3. Re-initialize SecretsManager so it connects to Vault
4. THEN store secrets in Vault

This ensures SecretsManager is properly configured before any
code tries to use create_conn() or other Vault-dependent functions.
2025-12-09 08:52:19 -03:00
4ea3b76adc fix(bootstrap): Skip early SecretsManager init if bootstrap incomplete, add logging
- Only initialize SecretsManager early if .env and init.json exist
- Fix error handling for bootstrap() - no longer silently ignores failures
- Add detailed logging to trace bootstrap flow
- Log component installation decisions (installed, needs_install flags)
2025-12-09 08:10:47 -03:00
824b12365b fix(bootstrap): Initialize SecretsManager after Vault starts in start_all/ensure_services_running
When services are already configured (bootstrap completed), the code
calls start_all() or ensure_services_running() which starts Vault
but didn't initialize SecretsManager. This caused create_conn() to
fail with 'Vault not configured' even though Vault was running.

Now both functions initialize SecretsManager after Vault is unsealed,
ensuring database connections can retrieve credentials from Vault.
2025-12-09 07:58:39 -03:00
583e764bb9 chore: Remove emoji icons from log messages and UI
Replace emoji icons with plain text/ASCII equivalents to prevent
encoding issues and improve console compatibility:
- Replace checkmarks with *
- Replace x marks with x
- Replace status indicators with text [OK], [ERR], etc.
- Remove decorative emojis from info/debug log messages
- Keep functional emojis in user-facing chat/sentiment features
2025-12-09 07:55:11 -03:00
b204aebd50 fix(bootstrap): Run migrations with direct password before Vault is set up
The bootstrap order is vault -> tables -> directory. When tables
component was processed, it tried to call establish_pg_connection()
which requires Vault, but Vault setup only happens when processing
the vault component (which is earlier in the loop but the if-block
for tables runs for the tables component, not vault).

Fix: Run migrations directly with the generated password during the
tables component setup, before Vault is configured. This avoids the
dependency on Vault being set up.
2025-12-09 07:49:01 -03:00
f3e38d8d8b feat(console): Show UI immediately with live system logs
- Add state_channel field to XtreeUI for receiving AppState updates
- Add set_state_channel() method to enable async state communication
- Poll for AppState in event loop to initialize panels when ready
- UI now shows loading state instantly, logs stream in real-time
- Transitions to full interactive mode when AppState is received
2025-12-08 23:35:33 -03:00
89ff716bef Fix Vault CLI mTLS issue - unset client cert env vars before CLI commands 2025-12-08 09:14:31 -03:00
f7ccc95e60 Fix config.csv loading on startup
- Disable TLS on Vault for local development (HTTP instead of HTTPS)
- Fix bot_configuration id column type mismatch (TEXT -> UUID)
- Add migration 6.1.1 to convert config table id columns to UUID
- Fix sync_config_csv_to_db to use UUID binding for id column
- Make start_all async with proper Vault startup sequence
- Sync default.gbai config.csv to existing 'Default Bot' from migrations
- Add diagnostic logging for config loading
- Change default LLM/embedding URLs from https to http for local dev
2025-12-08 00:19:29 -03:00
59f3bb8c7e refactor: standardize component names for certificates and DNS
- Rename service names to component names:
  - postgres -> tables
  - minio -> drive
  - redis -> cache
  - qdrant -> vectordb
  - mail -> email

- Update bootstrap certificate generation with new hostnames
- Update CoreDNS zone file with component-based DNS entries
- Update CA service certificates to use component names
- Update CA directory creation to use component names

All certificates include 127.0.0.1 as SAN for IP-based mTLS access
2025-12-07 10:42:02 -03:00
cfa1a01bc9 feat(bootstrap): implement mTLS for Vault access
- Add create_vault_config() function to generate config.hcl with mTLS settings
- Configure Vault to require client certificate verification
- Generate client certificate for botserver in bootstrap
- Update .env to include mTLS paths (VAULT_CACERT, VAULT_CLIENT_CERT, VAULT_CLIENT_KEY)
- Remove unused import in tls.rs
2025-12-07 02:13:28 -03:00
ad311944b8 Update dependencies and remove problematic crates
Drop image (with ravif/paste), sqlx, zitadel, and related dependencies
that were causing compilation issues. Replace image processing with
direct png crate usage. Update rcgen to 0.14 with new API changes.
Refactor CA certificate generation to use Issuer pattern.
2025-12-03 16:05:30 -03:00
b7975497aa - Split into botui. 2025-12-02 21:09:43 -03:00
ffd492a75f Add toml dependency for Askama custom filters config 2025-11-30 23:48:08 -03:00
5aa175845e - More htmx. 2025-11-29 17:27:13 -03:00
9ecbd927f0 HTMX enters. 2025-11-29 16:29:28 -03:00
b4250785c8 Update default features and add quick start guide
Replaces ui-server with console in default features, adds comprehensive
quick start documentation, implements automatic database migrations at
startup, and ensures critical services (PostgreSQL and MinIO) are
started automatically.

Key changes:
- Console UI now enable
2025-11-28 13:50:28 -03:00
f8e2e0360b Add metadata and refactor BASIC compiler
- Add package metadata (keywords, categories) to Cargo.toml
- Add #[must_use] attributes to constructor methods
- Simplify conditional logic by inverting if-else blocks
- Replace string formatting with interpolation syntax
2025-11-27 15:19:17 -03:00
3add3ccbfa Fix typos in bot file extensions and keyword names
Changed incorrect references to .vbs files to .bas and corrected
USE_WEBSITE keyword naming. Also added missing fields to API response
structure and clarified that start.bas is optional for bots.
2025-11-26 22:54:22 -03:00
bf0ed6223f - Refactor folder as features. 2025-11-22 22:55:35 -03:00