docs: Enhance Playwright testing guide with backend validation

Add comprehensive bot testing and backend validation documentation:
- Bot-specific URL pattern (http://localhost:3000/<botname>)
- Backend validation commands for PostgreSQL, Qdrant, Redis
- End-to-end process validation workflow
- Enhanced test flows with database/service verification
- Expanded testing checklist with backend checks

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Rodrigo Rodriguez 2026-02-17 15:15:33 +00:00
parent a31d7b355a
commit faeb77fc07

119
README.md
View file

@ -92,15 +92,36 @@ Place local bot packages in `/opt/gbo/data/` for automatic loading and monitorin
BotServer automatically installs, configures, and manages all infrastructure components on first run. **DO NOT manually start these services** - BotServer handles everything. BotServer automatically installs, configures, and manages all infrastructure components on first run. **DO NOT manually start these services** - BotServer handles everything.
| Component | Purpose | Port | Binary Location | Managed By | **Automatic Service Lifecycle:**
|-----------|---------|------|-----------------|------------| 1. **Start**: When botserver starts, it automatically launches all infrastructure components (PostgreSQL, Vault, MinIO, Valkey, Qdrant, etc.)
| **Vault** | Secrets management | 8200 | `botserver-stack/bin/vault/vault` | botserver | 2. **Credentials**: BotServer retrieves all service credentials (passwords, tokens, API keys) from Vault
| **PostgreSQL** | Primary database | 5432 | `botserver-stack/bin/tables/bin/postgres` | botserver | 3. **Connection**: BotServer uses these credentials to establish secure connections to each service
| **MinIO** | Object storage (S3-compatible) | 9000/9001 | `botserver-stack/bin/drive/minio` | botserver | 4. **Query**: All database queries, cache operations, and storage requests are authenticated using Vault-managed credentials
| **Zitadel** | Identity/Authentication | 8300 | `botserver-stack/bin/directory/zitadel` | botserver |
| **Qdrant** | Vector database (embeddings) | 6333 | `botserver-stack/bin/vector_db/qdrant` | botserver | **Credential Flow:**
| **Valkey** | Cache/Queue (Redis-compatible) | 6379 | `botserver-stack/bin/cache/valkey-server` | botserver | ```
| **Llama.cpp** | Local LLM server | 8081 | `botserver-stack/bin/llm/build/bin/llama-server` | botserver | botserver starts
Launch PostgreSQL, MinIO, Valkey, Qdrant
Connect to Vault
Retrieve service credentials (from database)
Authenticate with each service using retrieved credentials
Ready to handle requests
```
| Component | Purpose | Port | Binary Location | Credentials From |
|-----------|---------|------|-----------------|------------------|
| **Vault** | Secrets management | 8200 | `botserver-stack/bin/vault/vault` | Auto-unsealed |
| **PostgreSQL** | Primary database | 5432 | `botserver-stack/bin/tables/bin/postgres` | Vault → database |
| **MinIO** | Object storage (S3-compatible) | 9000/9001 | `botserver-stack/bin/drive/minio` | Vault → database |
| **Zitadel** | Identity/Authentication | 8300 | `botserver-stack/bin/directory/zitadel` | Vault → database |
| **Qdrant** | Vector database (embeddings) | 6333 | `botserver-stack/bin/vector_db/qdrant` | Vault → database |
| **Valkey** | Cache/Queue (Redis-compatible) | 6379 | `botserver-stack/bin/cache/valkey-server` | Vault → database |
| **Llama.cpp** | Local LLM server | 8081 | `botserver-stack/bin/llm/build/bin/llama-server` | Vault → database |
### 📦 Component Installation System ### 📦 Component Installation System
@ -319,9 +340,12 @@ cd botserver-stack/bin/cache && ./valkey-cli ping
The script handles BOTH servers properly: The script handles BOTH servers properly:
1. Stop existing processes cleanly 1. Stop existing processes cleanly
2. Build botserver and botui sequentially (no race conditions) 2. Build botserver and botui sequentially (no race conditions)
3. Start botserver in background → auto-bootstrap infrastructure 3. Start botserver in background → **automatically starts all infrastructure services (PostgreSQL, Vault, MinIO, Valkey, Qdrant)**
4. Start botui in background → proxy to botserver 4. BotServer retrieves credentials from Vault and authenticates with all services
5. Show process IDs and monitoring commands 5. Start botui in background → proxy to botserver
6. Show process IDs and monitoring commands
**Infrastructure services are fully automated - no manual configuration required!**
**Monitor startup:** **Monitor startup:**
```bash ```bash
@ -383,8 +407,16 @@ All infrastructure services (PostgreSQL, Vault, Redis, Qdrant, MinIO, etc.) are
- **Configurations:** `botserver-stack/conf/` - **Configurations:** `botserver-stack/conf/`
- **Data storage:** `botserver-stack/data/` - **Data storage:** `botserver-stack/data/`
- **Service logs:** `botserver-stack/logs/` (check here for troubleshooting) - **Service logs:** `botserver-stack/logs/` (check here for troubleshooting)
- **Credentials:** Stored in Vault, retrieved by botserver at startup
**Do NOT install or reference global PostgreSQL, Redis, or other services.** When botserver starts, it automatically launches all required stack services. If you encounter service errors, check the individual service logs in `./botserver-stack/logs/[service]/` directories. **Do NOT install or reference global PostgreSQL, Redis, or other services.** When botserver starts, it automatically:
1. Launches all required stack services
2. Connects to Vault
3. Retrieves credentials from the `bot_configuration` database table
4. Authenticates with each service using retrieved credentials
5. Begins handling requests with authenticated connections
If you encounter service errors, check the individual service logs in `./botserver-stack/logs/[service]/` directories.
### UI File Deployment - Production Options ### UI File Deployment - Production Options
@ -497,7 +529,8 @@ cargo test -p bottest
2. **Take snapshot** - Use `mcp__playwright__browser_snapshot` to see current page state 2. **Take snapshot** - Use `mcp__playwright__browser_snapshot` to see current page state
3. **Test user flows** - Use click, type, fill_form, etc. to interact with UI 3. **Test user flows** - Use click, type, fill_form, etc. to interact with UI
4. **Verify results** - Check for expected content, errors in console, network requests 4. **Verify results** - Check for expected content, errors in console, network requests
5. **Report findings** - Always include screenshot evidence with `browser_take_screenshot` 5. **Validate backend** - Check database and services to confirm process completion
6. **Report findings** - Always include screenshot evidence with `browser_take_screenshot`
**Available Playwright MCP Tools:** **Available Playwright MCP Tools:**
- `browser_navigate` - Navigate to URL - `browser_navigate` - Navigate to URL
@ -510,14 +543,42 @@ cargo test -p bottest
- `browser_network_requests` - Inspect API calls - `browser_network_requests` - Inspect API calls
- `browser_close` - Close browser when done - `browser_close` - Close browser when done
**Bot-Specific Testing URL Pattern:**
```
http://localhost:3000/<botname>
```
Navigate directly to a specific bot's interface to test that bot's functionality.
**YOLO Testing Workflow:** **YOLO Testing Workflow:**
``` ```
1. Navigate → http://localhost:3000 1. Navigate → http://localhost:3000/<botname> or http://localhost:3000
2. Snapshot → Analyze page structure 2. Snapshot → Analyze page structure
3. Click → Target element using ref from snapshot 3. Click → Target element using ref from snapshot
4. Wait → For navigation/updates (browser_wait_for) 4. Wait → For navigation/updates (browser_wait_for)
5. Verify → Console messages, network status 5. Verify → Console messages, network status
6. Screenshot → Document test results 6. Validate Backend → Check database tables and service logs
7. Screenshot → Document test results
```
**Backend Validation Checks:**
After UI interactions, validate backend state:
```bash
# Check database for created/updated records
psql -h localhost -p 5432 -U postgres -d botdb -c "SELECT * FROM bots WHERE name = '<botname>';"
psql -h localhost -p 5432 -U postgres -d botdb -c "SELECT * FROM sessions WHERE bot_id = '<bot_id>' ORDER BY created_at DESC LIMIT 5;"
# Check service logs for errors
tail -50 botserver-stack/logs/tables/postgres.log
tail -50 botserver-stack/logs/vault/vault.log
tail -50 botserver-stack/logs/cache/valkey.log
# Verify vector embeddings (for knowledge bases)
curl -s http://localhost:6333/collections/website_<hash>/points/scroll | jq '.result.points | length'
# Check message queue/worker status
redis-cli -p 6379 PING
redis-cli -p 6379 KEYS "session:*"
``` ```
**Testing Checklist:** **Testing Checklist:**
@ -527,19 +588,35 @@ cargo test -p bottest
- ✅ WebSocket connections establish - ✅ WebSocket connections establish
- ✅ Console shows no JavaScript errors - ✅ Console shows no JavaScript errors
- ✅ Network requests return 200/201/204 - ✅ Network requests return 200/201/204
- ✅ Database records created/updated correctly
- ✅ Service logs show no errors
- ✅ Vector embeddings populated (if applicable)
- ✅ Cache/queue operations succeed
**Critical Test Flows:** **Critical Test Flows:**
- **Login/Authentication** → Navigate, enter credentials, verify session - **Login/Authentication** → Navigate, enter credentials, verify session in database
- **Bot Creation** → Click "New Bot", fill form, verify creation - **Bot Creation** → Navigate to `/botname`, click "New Bot", fill form, verify creation in `bots` table
- **Chat Interface** → Send message, verify WebSocket response - **Chat Interface** → Send message to `/botname`, verify WebSocket response, check `messages` table
- **File Upload** → Upload .bas file, verify compilation - **File Upload** → Upload .bas file to `/botname`, verify compilation, check `botserver.log` for compile results
- **Drive Sync** → Trigger sync, verify files appear - **Drive Sync** → Trigger sync from `/botname`, verify files appear, check `drive_monitor` logs
- **Knowledge Base** → Add document via `/botname`, verify embeddings in Qdrant, check `website_embeddings` table
- **Tool Execution** → Trigger tool from chat, verify execution in logs, check side effects in database
**End-to-End Process Validation:**
1. **Frontend:** User action via Playwright (click, type, submit)
2. **Network:** Capture API request via `browser_network_requests`
3. **Backend:** Check botserver.log for request processing
4. **Database:** Query relevant tables for state changes
5. **Services:** Verify component logs (PostgreSQL, Vault, Qdrant, etc.)
6. **Response:** Verify UI update via Playwright snapshot
**Error Handling in YOLO Mode:** **Error Handling in YOLO Mode:**
- If navigation fails: Check if servers running (`ps aux | grep botserver`) - If navigation fails: Check if servers running (`ps aux | grep botserver`)
- If element not found: Take snapshot to debug current page state - If element not found: Take snapshot to debug current page state
- If console errors: Extract and report to user for fixing - If console errors: Extract and report to user for fixing
- If network failures: Check API endpoints and CORS configuration - If network failures: Check API endpoints and CORS configuration
- If database mismatch: Check database logs, verify transaction committed
- If service errors: Check component logs in `botserver-stack/logs/`
### Integration Testing ### Integration Testing