Add testing chapter (17) to botbook
- Created src/17-testing/ chapter with complete testing documentation - Added README.md - testing overview and structure - Added e2e-testing.md - comprehensive E2E testing guide (447 lines) - Added architecture.md - testing architecture and patterns (410 lines) - Updated SUMMARY.md to include Part XIV - Testing - All E2E testing documentation now centralized in botbook - Developers refer to botbook and test PROMPT.md, not scattered files
This commit is contained in:
parent
c26fb1e3a2
commit
de8282f0a2
4 changed files with 942 additions and 0 deletions
76
src/17-testing/README.md
Normal file
76
src/17-testing/README.md
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
# Testing
|
||||
|
||||
General Bots uses a comprehensive testing framework including unit tests, integration tests, and end-to-end (E2E) tests to ensure platform reliability and quality.
|
||||
|
||||
## Overview
|
||||
|
||||
The testing strategy covers:
|
||||
|
||||
- **Unit Tests** - Individual component testing
|
||||
- **Integration Tests** - Service interaction testing
|
||||
- **E2E Tests** - Complete user journey validation
|
||||
|
||||
## Test Structure
|
||||
|
||||
All tests are organized in the `bottest` package:
|
||||
|
||||
```
|
||||
bottest/
|
||||
├── src/ # Test utilities and harness
|
||||
├── tests/
|
||||
│ ├── unit/ # Unit tests
|
||||
│ ├── integration/ # Integration tests
|
||||
│ └── e2e/ # End-to-end tests
|
||||
├── benches/ # Performance benchmarks
|
||||
└── Cargo.toml
|
||||
```
|
||||
|
||||
## Running Tests
|
||||
|
||||
### All Tests
|
||||
```bash
|
||||
cd gb/bottest
|
||||
cargo test
|
||||
```
|
||||
|
||||
### Specific Test Types
|
||||
```bash
|
||||
# Unit tests
|
||||
cargo test --lib
|
||||
|
||||
# Integration tests
|
||||
cargo test --test integration
|
||||
|
||||
# E2E tests
|
||||
cargo test --test e2e -- --nocapture
|
||||
```
|
||||
|
||||
## Test Harness
|
||||
|
||||
The test harness provides utilities for setting up test environments:
|
||||
|
||||
```rust
|
||||
use bottest::prelude::*;
|
||||
|
||||
#[tokio::test]
|
||||
async fn my_test() {
|
||||
let ctx = TestHarness::full().await.unwrap();
|
||||
// Test code here
|
||||
ctx.cleanup().await.unwrap();
|
||||
}
|
||||
```
|
||||
|
||||
## Continuous Integration
|
||||
|
||||
Tests run automatically on:
|
||||
- Pull requests
|
||||
- Commits to main branch
|
||||
- Pre-release checks
|
||||
|
||||
See the repository's CI/CD configuration for details.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [End-to-End Testing](./e2e-testing.md) - Browser automation and user flow testing
|
||||
- [Performance Testing](./performance.md) - Benchmarking and profiling
|
||||
- [Test Architecture](./architecture.md) - Design patterns and best practices
|
||||
410
src/17-testing/architecture.md
Normal file
410
src/17-testing/architecture.md
Normal file
|
|
@ -0,0 +1,410 @@
|
|||
# Testing Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
The General Bots testing framework is designed with a multi-layered, isolated approach to ensure comprehensive coverage from individual components to complete user workflows.
|
||||
|
||||
## Architecture Diagram
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Test Execution Layer │
|
||||
│ (GitHub Actions, CI/CD, Local Development) │
|
||||
└────────────────────┬────────────────────────────────────────┘
|
||||
│
|
||||
┌────────────┼────────────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌─────────┐ ┌─────────┐ ┌──────────┐
|
||||
│ Unit │ │ Integr. │ │ E2E │
|
||||
│ Tests │ │ Tests │ │ Tests │
|
||||
└────┬────┘ └────┬────┘ └─────┬────┘
|
||||
│ │ │
|
||||
└────────────┼────────────┘
|
||||
│
|
||||
┌────────────▼────────────┐
|
||||
│ Test Harness Layer │
|
||||
│ (Context, Utils, Mocks) │
|
||||
└────────────┬────────────┘
|
||||
│
|
||||
┌────────────┼────────────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌─────────┐ ┌─────────┐ ┌──────────┐
|
||||
│BotServer│ │ Browser│ │ Services │
|
||||
│(Testing)│ │ (WebDrv)│ │(Mock/Iso)│
|
||||
└─────────┘ └─────────┘ └──────────┘
|
||||
│ │ │
|
||||
└────────────┼────────────┘
|
||||
│
|
||||
┌────────────▼────────────┐
|
||||
│ Temporary Stack Layer │
|
||||
│ (Isolated Environments) │
|
||||
└────────────┬────────────┘
|
||||
│
|
||||
┌────────────┴────────────┐
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────────┐ ┌──────────────┐
|
||||
│ PostgreSQL │ │ Redis, MinIO │
|
||||
│ (Isolated) │ │ (Isolated) │
|
||||
└─────────────┘ └──────────────┘
|
||||
```
|
||||
|
||||
## Test Layers
|
||||
|
||||
### 1. Unit Tests
|
||||
|
||||
**Purpose**: Test individual components in isolation
|
||||
|
||||
**Scope**:
|
||||
- Single functions or methods
|
||||
- Mocked external dependencies
|
||||
- No database or external services
|
||||
|
||||
**Example**:
|
||||
```rust
|
||||
#[test]
|
||||
fn test_message_formatting() {
|
||||
let msg = format_message("Hello");
|
||||
assert_eq!(msg, "Hello!");
|
||||
}
|
||||
```
|
||||
|
||||
**Location**: `bottest/tests/unit/`
|
||||
|
||||
### 2. Integration Tests
|
||||
|
||||
**Purpose**: Test multiple components working together
|
||||
|
||||
**Scope**:
|
||||
- Multi-component interactions
|
||||
- Real database connections
|
||||
- Service integration
|
||||
- Error handling across components
|
||||
|
||||
**Example**:
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_message_storage_and_retrieval() {
|
||||
let db = setup_test_db().await;
|
||||
let msg = Message::new("Hello");
|
||||
db.save(&msg).await.unwrap();
|
||||
let retrieved = db.get(msg.id).await.unwrap();
|
||||
assert_eq!(retrieved.text, "Hello");
|
||||
}
|
||||
```
|
||||
|
||||
**Location**: `bottest/tests/integration/`
|
||||
|
||||
### 3. End-to-End Tests
|
||||
|
||||
**Purpose**: Test complete user workflows
|
||||
|
||||
**Scope**:
|
||||
- Complete user journeys
|
||||
- Browser interactions
|
||||
- Multi-phase workflows
|
||||
- Real-world scenarios
|
||||
|
||||
**Phases**:
|
||||
1. Platform Loading
|
||||
2. BotServer Initialization
|
||||
3. User Authentication
|
||||
4. Chat Interaction
|
||||
5. Logout & Session Management
|
||||
|
||||
**Example**:
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_complete_platform_flow_login_chat_logout() {
|
||||
let ctx = E2ETestContext::setup_with_browser().await?;
|
||||
|
||||
verify_platform_loading(&ctx).await?;
|
||||
verify_botserver_running(&ctx).await?;
|
||||
test_user_login(browser, &ctx).await?;
|
||||
test_chat_interaction(browser, &ctx).await?;
|
||||
test_user_logout(browser, &ctx).await?;
|
||||
|
||||
ctx.close().await;
|
||||
}
|
||||
```
|
||||
|
||||
**Location**: `bottest/tests/e2e/`
|
||||
|
||||
## Test Harness
|
||||
|
||||
The test harness provides utilities for test setup and context management:
|
||||
|
||||
```
|
||||
TestHarness
|
||||
├── Setup utilities
|
||||
│ ├── Create test database
|
||||
│ ├── Start mock services
|
||||
│ ├── Initialize configurations
|
||||
│ └── Provision test data
|
||||
├── Context management
|
||||
│ ├── Resource tracking
|
||||
│ ├── Cleanup coordination
|
||||
│ └── Error handling
|
||||
└── Helper functions
|
||||
├── HTTP requests
|
||||
├── Browser interactions
|
||||
└── Service mocking
|
||||
```
|
||||
|
||||
### E2ETestContext
|
||||
|
||||
Provides complete environment for E2E testing:
|
||||
|
||||
```rust
|
||||
pub struct E2ETestContext {
|
||||
pub ctx: TestContext,
|
||||
pub server: BotServerInstance,
|
||||
pub browser: Option<Browser>,
|
||||
}
|
||||
|
||||
impl E2ETestContext {
|
||||
pub async fn setup() -> Result<Self>
|
||||
pub async fn setup_with_browser() -> Result<Self>
|
||||
pub fn base_url(&self) -> &str
|
||||
pub fn has_browser(&self) -> bool
|
||||
pub async fn close(self)
|
||||
}
|
||||
```
|
||||
|
||||
## Temporary Stack Architecture
|
||||
|
||||
Isolated test environments for complete system integration:
|
||||
|
||||
```
|
||||
/tmp/botserver-test-{timestamp}-{id}/
|
||||
├── postgres/
|
||||
│ ├── data/ ← PostgreSQL data files
|
||||
│ ├── postgresql.log ← Database logs
|
||||
│ └── postgresql.conf ← Configuration
|
||||
├── redis/
|
||||
│ ├── data/ ← Redis persistence
|
||||
│ └── redis.log
|
||||
├── minio/
|
||||
│ ├── data/ ← S3-compatible storage
|
||||
│ └── minio.log
|
||||
├── botserver/
|
||||
│ ├── config/
|
||||
│ │ ├── config.toml ← Application config
|
||||
│ │ └── .env ← Environment variables
|
||||
│ ├── logs/
|
||||
│ │ ├── botserver.log ← Main application logs
|
||||
│ │ ├── api.log ← API logs
|
||||
│ │ └── debug.log ← Debug logs
|
||||
│ ├── cache/ ← Local cache
|
||||
│ └── state.json ← Stack metadata
|
||||
└── env.stack ← Connection strings for tests
|
||||
```
|
||||
|
||||
## Isolation Strategy
|
||||
|
||||
### Service Isolation
|
||||
|
||||
Each test gets dedicated service instances:
|
||||
|
||||
- **Database**: Separate PostgreSQL cluster on port 5433
|
||||
- **Cache**: Separate Redis instance on port 6380
|
||||
- **Storage**: Separate MinIO instance on port 9001
|
||||
- **API**: Separate BotServer on port 8000
|
||||
|
||||
### Network Isolation
|
||||
|
||||
- All services on localhost (127.0.0.1)
|
||||
- Non-standard ports to avoid conflicts
|
||||
- Docker containers for complete OS-level isolation
|
||||
|
||||
### Data Isolation
|
||||
|
||||
- Separate database schemas per test
|
||||
- Temporary file systems for storage
|
||||
- No shared configuration between tests
|
||||
- Automatic cleanup on completion
|
||||
|
||||
## Test Execution Flow
|
||||
|
||||
```
|
||||
1. Test Initialization
|
||||
├─ Parse environment variables
|
||||
├─ Check prerequisites (WebDriver, services)
|
||||
└─ Create test context
|
||||
|
||||
2. Stack Setup
|
||||
├─ Create temporary directory
|
||||
├─ Initialize databases
|
||||
├─ Start services
|
||||
└─ Wait for readiness
|
||||
|
||||
3. Test Execution
|
||||
├─ Setup phase
|
||||
├─ Action phase
|
||||
├─ Verification phase
|
||||
└─ Assertion phase
|
||||
|
||||
4. Cleanup
|
||||
├─ Close browser connections
|
||||
├─ Shutdown services gracefully
|
||||
├─ Remove temporary directories
|
||||
└─ Report results
|
||||
```
|
||||
|
||||
## Browser Automation
|
||||
|
||||
Uses WebDriver (Selenium) protocol for browser testing:
|
||||
|
||||
```
|
||||
Test Code
|
||||
↓
|
||||
Reqwest HTTP Client
|
||||
↓
|
||||
WebDriver Protocol (JSON-RPC)
|
||||
↓
|
||||
chromedriver / Selenium Server
|
||||
↓
|
||||
Chrome/Chromium Browser
|
||||
↓
|
||||
Test Verification
|
||||
```
|
||||
|
||||
### WebDriver Commands
|
||||
|
||||
- Navigate to URL
|
||||
- Find elements by selector
|
||||
- Click buttons and links
|
||||
- Fill form inputs
|
||||
- Wait for elements
|
||||
- Execute JavaScript
|
||||
- Take screenshots
|
||||
- Get element text
|
||||
|
||||
## Error Handling
|
||||
|
||||
Comprehensive error handling at all levels:
|
||||
|
||||
```
|
||||
Test Execution
|
||||
│
|
||||
├─ Setup Error
|
||||
│ └─ Fail fast, preserve environment
|
||||
│
|
||||
├─ Execution Error
|
||||
│ ├─ Log detailed context
|
||||
│ ├─ Capture screenshots
|
||||
│ └─ Optionally preserve stack
|
||||
│
|
||||
└─ Cleanup Error
|
||||
└─ Log warning, continue cleanup
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Test Execution Times
|
||||
|
||||
- **Unit Tests**: ~0.1-1 second
|
||||
- **Integration Tests**: ~1-10 seconds
|
||||
- **E2E Tests**: ~30-60 seconds
|
||||
- **Full Suite**: ~2-3 minutes
|
||||
|
||||
### Optimization Strategies
|
||||
|
||||
1. **Parallel Execution**: Run independent tests simultaneously
|
||||
2. **Caching**: Reuse expensive resources
|
||||
3. **Lazy Loading**: Initialize only needed components
|
||||
4. **Release Mode**: Use `--release` for faster compilation
|
||||
5. **Selective Testing**: Run only relevant tests during development
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
### GitHub Actions Workflow
|
||||
|
||||
```
|
||||
Trigger (push/PR)
|
||||
↓
|
||||
Setup Environment
|
||||
├─ Install Rust
|
||||
├─ Start WebDriver
|
||||
└─ Setup test infrastructure
|
||||
↓
|
||||
Run Tests
|
||||
├─ Unit tests
|
||||
├─ Integration tests
|
||||
└─ E2E tests
|
||||
↓
|
||||
Collect Artifacts
|
||||
├─ Test results
|
||||
├─ Coverage reports
|
||||
├─ Screenshots/logs
|
||||
└─ Performance metrics
|
||||
↓
|
||||
Report Results
|
||||
└─ Pass/fail status
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Test Organization
|
||||
|
||||
- Keep tests focused and single-purpose
|
||||
- Use descriptive names
|
||||
- Group related tests
|
||||
- Organize by layer (unit/integration/e2e)
|
||||
|
||||
### 2. Test Design
|
||||
|
||||
- Make tests independent
|
||||
- Use realistic data
|
||||
- Test both happy and error paths
|
||||
- Avoid test interdependencies
|
||||
|
||||
### 3. Test Maintenance
|
||||
|
||||
- Keep tests up to date with code
|
||||
- Remove obsolete tests
|
||||
- Refactor test helpers
|
||||
- Monitor test execution time
|
||||
|
||||
### 4. Test Documentation
|
||||
|
||||
- Document complex test logic
|
||||
- Explain test prerequisites
|
||||
- Document setup/teardown
|
||||
- Include troubleshooting tips
|
||||
|
||||
## Debugging
|
||||
|
||||
### Debug Helpers
|
||||
|
||||
- `RUST_LOG=debug` - Verbose logging
|
||||
- `HEADED=1` - Show browser UI
|
||||
- `--nocapture` - Print test output
|
||||
- `--test-threads=1` - Run sequentially
|
||||
|
||||
### Debug Techniques
|
||||
|
||||
- Check server logs
|
||||
- Review screenshots
|
||||
- Inspect HTTP requests
|
||||
- Step through code
|
||||
- Use REPL for experimentation
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
1. **Load Testing** - Concurrent user scenarios
|
||||
2. **Visual Regression** - Screenshot comparison
|
||||
3. **Accessibility Testing** - WCAG compliance
|
||||
4. **Security Testing** - Vulnerability scanning
|
||||
5. **Performance Profiling** - Memory and CPU analysis
|
||||
6. **Multi-region** - Test across deployments
|
||||
7. **Snapshot Testing** - Compare outputs over time
|
||||
|
||||
## References
|
||||
|
||||
- [End-to-End Testing Guide](./e2e-testing.md)
|
||||
- [Test Harness API](./test-harness.md)
|
||||
- [CI/CD Integration](./ci-cd.md)
|
||||
- [Performance Benchmarking](./performance.md)
|
||||
447
src/17-testing/e2e-testing.md
Normal file
447
src/17-testing/e2e-testing.md
Normal file
|
|
@ -0,0 +1,447 @@
|
|||
# End-to-End Testing
|
||||
|
||||
End-to-end (E2E) testing validates complete user workflows from platform loading through authentication, interaction, and logout.
|
||||
|
||||
## Overview
|
||||
|
||||
E2E tests simulate real user interactions:
|
||||
|
||||
1. **Platform Loading** - UI and API infrastructure operational
|
||||
2. **BotServer Initialization** - Backend service running and ready
|
||||
3. **User Authentication** - Login workflow functional
|
||||
4. **Chat Interaction** - Message sending and receiving
|
||||
5. **Logout** - Session management and access control
|
||||
|
||||
## Complete Platform Flow Test
|
||||
|
||||
The main E2E test validates the entire user journey:
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_complete_platform_flow_login_chat_logout() {
|
||||
// Setup
|
||||
let ctx = E2ETestContext::setup_with_browser().await?;
|
||||
let browser = ctx.browser.as_ref().unwrap();
|
||||
|
||||
// Phase 1: Platform Loading
|
||||
verify_platform_loading(&ctx).await?;
|
||||
|
||||
// Phase 2: BotServer Running
|
||||
verify_botserver_running(&ctx).await?;
|
||||
|
||||
// Phase 3: User Login
|
||||
test_user_login(browser, &ctx).await?;
|
||||
|
||||
// Phase 4: Chat Interaction
|
||||
test_chat_interaction(browser, &ctx).await?;
|
||||
|
||||
// Phase 5: Logout
|
||||
test_user_logout(browser, &ctx).await?;
|
||||
|
||||
ctx.close().await;
|
||||
}
|
||||
```
|
||||
|
||||
## Test Phases
|
||||
|
||||
### Phase 1: Platform Loading
|
||||
|
||||
Verifies UI and API infrastructure:
|
||||
|
||||
```rust
|
||||
verify_platform_loading(&ctx).await?
|
||||
```
|
||||
|
||||
Checks:
|
||||
- Health endpoint responds with 2xx status
|
||||
- API endpoints are accessible
|
||||
- Database migrations completed
|
||||
- Services are initialized
|
||||
|
||||
### Phase 2: BotServer Initialization
|
||||
|
||||
Verifies the backend service is operational:
|
||||
|
||||
```rust
|
||||
verify_botserver_running(&ctx).await?
|
||||
```
|
||||
|
||||
Checks:
|
||||
- Process is alive and responding
|
||||
- Configuration properly loaded
|
||||
- Dependencies connected (DB, cache, storage)
|
||||
- Health checks pass
|
||||
|
||||
### Phase 3: User Authentication
|
||||
|
||||
Tests the login workflow:
|
||||
|
||||
```rust
|
||||
test_user_login(browser, &ctx).await?
|
||||
```
|
||||
|
||||
Tests:
|
||||
- Navigate to login page
|
||||
- Form elements present and functional
|
||||
- Accept valid test credentials (test@example.com / TestPassword123!)
|
||||
- Create session and authentication token
|
||||
- Redirect to dashboard/chat interface
|
||||
|
||||
### Phase 4: Chat Interaction
|
||||
|
||||
Tests messaging functionality:
|
||||
|
||||
```rust
|
||||
test_chat_interaction(browser, &ctx).await?
|
||||
```
|
||||
|
||||
Tests:
|
||||
- Chat interface loads correctly
|
||||
- User can type and send messages
|
||||
- Bot responds with valid output
|
||||
- Message history persists
|
||||
- Multiple exchanges work correctly
|
||||
|
||||
### Phase 5: Logout & Session Management
|
||||
|
||||
Tests secure session handling:
|
||||
|
||||
```rust
|
||||
test_user_logout(browser, &ctx).await?
|
||||
```
|
||||
|
||||
Tests:
|
||||
- Logout button/action works
|
||||
- Session is invalidated
|
||||
- User redirected to login page
|
||||
- Protected routes block unauthenticated access
|
||||
- Cannot access chat after logout
|
||||
|
||||
## Running E2E Tests
|
||||
|
||||
### HTTP-Only Tests (No Browser Required)
|
||||
|
||||
These tests verify API and infrastructure without browser automation:
|
||||
|
||||
```bash
|
||||
cd gb/bottest
|
||||
|
||||
# Platform loading verification
|
||||
cargo test --test e2e test_platform_loading_http_only -- --nocapture
|
||||
|
||||
# BotServer startup verification
|
||||
cargo test --test e2e test_botserver_startup -- --nocapture
|
||||
```
|
||||
|
||||
Execution time: ~2-5 seconds
|
||||
|
||||
### Complete Flow Tests (Requires WebDriver)
|
||||
|
||||
Full browser-based tests with user interactions:
|
||||
|
||||
```bash
|
||||
# Start WebDriver first
|
||||
chromedriver --port=4444 &
|
||||
|
||||
# Run complete platform flow
|
||||
cargo test --test e2e test_complete_platform_flow_login_chat_logout -- --nocapture
|
||||
|
||||
# Run simplified flow
|
||||
cargo test --test e2e test_login_and_chat_flow -- --nocapture
|
||||
```
|
||||
|
||||
Execution time: ~30-60 seconds
|
||||
|
||||
## WebDriver Setup
|
||||
|
||||
### Option 1: Local Installation
|
||||
|
||||
```bash
|
||||
# Download chromedriver from https://chromedriver.chromium.org/
|
||||
# Place in PATH, then start:
|
||||
chromedriver --port=4444
|
||||
```
|
||||
|
||||
### Option 2: Docker
|
||||
|
||||
```bash
|
||||
docker run -d -p 4444:4444 selenium/standalone-chrome
|
||||
```
|
||||
|
||||
### Option 3: Docker Compose
|
||||
|
||||
```bash
|
||||
docker-compose up -d webdriver
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Control test behavior:
|
||||
|
||||
| Variable | Default | Purpose |
|
||||
|----------|---------|---------|
|
||||
| `HEADED` | unset | Show browser window instead of headless |
|
||||
| `WEBDRIVER_URL` | `http://localhost:4444` | WebDriver server endpoint |
|
||||
| `SKIP_E2E_TESTS` | unset | Skip E2E tests if set |
|
||||
| `RUST_LOG` | info | Logging level: debug, info, warn, error |
|
||||
| `KEEP_TEMP_STACK_ON_ERROR` | unset | Preserve temp directory on failure |
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# Show browser UI for debugging
|
||||
HEADED=1 cargo test --test e2e -- --nocapture
|
||||
|
||||
# Use custom WebDriver
|
||||
WEBDRIVER_URL=http://localhost:4445 cargo test --test e2e -- --nocapture
|
||||
|
||||
# Verbose logging
|
||||
RUST_LOG=debug cargo test --test e2e -- --nocapture
|
||||
|
||||
# Run single-threaded with output
|
||||
cargo test --test e2e -- --nocapture --test-threads=1
|
||||
```
|
||||
|
||||
## Test Helpers
|
||||
|
||||
Reusable helper functions for custom tests:
|
||||
|
||||
```rust
|
||||
// Verify platform is operational
|
||||
verify_platform_loading(&ctx) -> Result<()>
|
||||
|
||||
// Verify BotServer is running
|
||||
verify_botserver_running(&ctx) -> Result<()>
|
||||
|
||||
// Perform login with credentials
|
||||
test_user_login(browser, &ctx) -> Result<()>
|
||||
|
||||
// Send message and wait for response
|
||||
test_chat_interaction(browser, &ctx) -> Result<()>
|
||||
|
||||
// Logout and verify session invalidation
|
||||
test_user_logout(browser, &ctx) -> Result<()>
|
||||
```
|
||||
|
||||
## Test Context
|
||||
|
||||
Setup a test context for E2E testing:
|
||||
|
||||
```rust
|
||||
use bottest::prelude::*;
|
||||
use bottest::web::{Browser, BrowserConfig};
|
||||
|
||||
// HTTP-only context
|
||||
let ctx = E2ETestContext::setup().await?;
|
||||
|
||||
// With browser automation
|
||||
let ctx = E2ETestContext::setup_with_browser().await?;
|
||||
let browser = ctx.browser.as_ref().unwrap();
|
||||
|
||||
// Access base URL
|
||||
let url = ctx.base_url();
|
||||
|
||||
// Access running server
|
||||
let is_running = ctx.server.is_running();
|
||||
|
||||
// Cleanup
|
||||
ctx.close().await;
|
||||
```
|
||||
|
||||
## Common Issues
|
||||
|
||||
### WebDriver Not Available
|
||||
|
||||
**Problem**: Test fails with "WebDriver not available"
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Start WebDriver
|
||||
chromedriver --port=4444
|
||||
# or
|
||||
docker run -d -p 4444:4444 selenium/standalone-chrome
|
||||
```
|
||||
|
||||
### Port Already in Use
|
||||
|
||||
**Problem**: Services fail to start due to port conflicts
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Kill existing services
|
||||
pkill -f chromedriver
|
||||
pkill -f botserver
|
||||
pkill -f postgres
|
||||
pkill -f redis-server
|
||||
```
|
||||
|
||||
### Test Hangs or Timeout
|
||||
|
||||
**Problem**: Test appears to hang or timeout
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Run with timeout and verbose output
|
||||
timeout 120s RUST_LOG=debug cargo test --test e2e test_name -- --nocapture --test-threads=1
|
||||
```
|
||||
|
||||
### Browser Connection Issues
|
||||
|
||||
**Problem**: Browser fails to connect to WebDriver
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Use different WebDriver port
|
||||
WEBDRIVER_URL=http://localhost:4445 cargo test --test e2e -- --nocapture
|
||||
```
|
||||
|
||||
## Debugging
|
||||
|
||||
### View Test Output
|
||||
```bash
|
||||
# Show all output
|
||||
cargo test --test e2e test_name -- --nocapture
|
||||
|
||||
# With timestamps
|
||||
RUST_LOG=debug cargo test --test e2e test_name -- --nocapture
|
||||
|
||||
# Save to file
|
||||
cargo test --test e2e test_name -- --nocapture 2>&1 | tee test.log
|
||||
```
|
||||
|
||||
### Watch Browser in Action
|
||||
```bash
|
||||
# Run with visible browser
|
||||
HEADED=1 cargo test --test e2e test_name -- --nocapture --test-threads=1
|
||||
```
|
||||
|
||||
### Check Server Logs
|
||||
```bash
|
||||
# Monitor logs while tests run
|
||||
tail -f /tmp/bottest-*/botserver.log
|
||||
|
||||
# In another terminal:
|
||||
cargo test --test e2e test_name -- --nocapture
|
||||
```
|
||||
|
||||
## Performance
|
||||
|
||||
Typical execution times:
|
||||
|
||||
| Test | Time | Resources |
|
||||
|------|------|-----------|
|
||||
| Platform loading (HTTP-only) | ~2s | Minimal |
|
||||
| BotServer startup (HTTP-only) | ~5s | Minimal |
|
||||
| Login and chat flow | ~20s | Browser + Memory |
|
||||
| Complete flow with all phases | ~45s | Browser + Memory |
|
||||
| Full E2E test suite | ~2-3 min | High |
|
||||
|
||||
Use release mode for faster execution:
|
||||
```bash
|
||||
cargo test --test e2e --release -- --nocapture
|
||||
```
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
### GitHub Actions Example
|
||||
|
||||
```yaml
|
||||
name: E2E Tests
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
e2e:
|
||||
runs-on: ubuntu-latest
|
||||
services:
|
||||
chromedriver:
|
||||
image: selenium/standalone-chrome
|
||||
options: --shm-size=2gb
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions-rs/toolchain@v1
|
||||
with:
|
||||
toolchain: stable
|
||||
- run: cd gb/bottest && cargo test --test e2e -- --nocapture
|
||||
```
|
||||
|
||||
## Temporary Stack Architecture (Future)
|
||||
|
||||
When BotServer implements `--temp-stack`, E2E tests will run in isolated environments:
|
||||
|
||||
```bash
|
||||
botserver --temp-stack
|
||||
# Creates: /tmp/botserver-test-{timestamp}-{random}/
|
||||
# With isolated: PostgreSQL, Redis, MinIO, Mock LLM
|
||||
# Auto-cleanup after test completion
|
||||
```
|
||||
|
||||
Benefits:
|
||||
- ✓ Isolation - Each test in separate environment
|
||||
- ✓ Reproducibility - Consistent setup every time
|
||||
- ✓ Automation - No manual configuration
|
||||
- ✓ Safety - Won't interfere with development
|
||||
- ✓ Cleanup - Automatic resource management
|
||||
- ✓ Parallel - Multiple tests simultaneously
|
||||
- ✓ CI/CD Ready - Perfect for automated pipelines
|
||||
|
||||
## Writing Custom E2E Tests
|
||||
|
||||
Create new test files in `gb/bottest/tests/e2e/`:
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_my_feature() {
|
||||
// Setup context
|
||||
let ctx = E2ETestContext::setup_with_browser().await?;
|
||||
let browser = ctx.browser.as_ref().unwrap();
|
||||
|
||||
// Navigate to feature
|
||||
browser.navigate(&format!("{}/my-feature", ctx.base_url())).await?;
|
||||
|
||||
// Interact with UI
|
||||
browser.click("button.action").await?;
|
||||
browser.wait_for_element(".result", Duration::from_secs(10)).await?;
|
||||
|
||||
// Verify results
|
||||
let text = browser.get_text(".result").await?;
|
||||
assert_eq!(text, "Expected result");
|
||||
|
||||
// Cleanup
|
||||
ctx.close().await;
|
||||
}
|
||||
```
|
||||
|
||||
Register in `tests/e2e/mod.rs`:
|
||||
|
||||
```rust
|
||||
mod my_feature;
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Keep tests focused** - Test one user workflow per test
|
||||
2. **Use meaningful names** - `test_complete_platform_flow` not `test_1`
|
||||
3. **Explicit waits** - Use `wait_for_element` instead of `sleep`
|
||||
4. **Test realistic flows** - Use actual test credentials
|
||||
5. **Verify results explicitly** - Check status codes, UI elements, and state
|
||||
6. **Clean up properly** - Always call `ctx.close().await`
|
||||
7. **Handle errors gracefully** - Use `?` operator for error propagation
|
||||
8. **Make tests independent** - Don't rely on test execution order
|
||||
|
||||
## Test Success Criteria
|
||||
|
||||
✓ Platform fully loads without errors
|
||||
✓ BotServer starts and becomes ready
|
||||
✓ User can login with credentials
|
||||
✓ Chat messages are sent and responses received
|
||||
✓ User can logout and session is invalidated
|
||||
✓ Protected routes block unauthenticated access
|
||||
✓ Tests run consistently multiple times
|
||||
✓ Tests complete within acceptable time (~60 seconds)
|
||||
|
||||
## See Also
|
||||
|
||||
- [Testing Overview](./README.md) - Testing strategy and structure
|
||||
- [Performance Testing](./performance.md) - Benchmarks and load tests
|
||||
- [Test Architecture](./architecture.md) - Design patterns and best practices
|
||||
- [Integration Testing](./integration.md) - Multi-component testing
|
||||
|
|
@ -347,6 +347,15 @@
|
|||
- [Automation Migration](./14-migration/automation.md)
|
||||
- [Validation and Testing](./14-migration/validation.md)
|
||||
|
||||
# Part XIV - Testing
|
||||
|
||||
- [Chapter 17: Testing](./17-testing/README.md)
|
||||
- [End-to-End Testing](./17-testing/e2e-testing.md)
|
||||
- [Testing Architecture](./17-testing/architecture.md)
|
||||
- [Performance Testing](./17-testing/performance.md)
|
||||
- [Best Practices](./17-testing/best-practices.md)
|
||||
- [CI/CD Integration](./17-testing/ci-cd.md)
|
||||
|
||||
# Appendices
|
||||
|
||||
- [Appendix A: Database Model](./15-appendix/README.md)
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue