Add docs, UI pages, code scanner, and Docker deployment guide
- Add CRM contacts template documentation - Add Docker deployment documentation with compose examples - Add BASIC code scanner for security compliance checking - Add visual dialog designer UI (designer.html) - Add drive file manager UI (drive/index.html) - Add sources browser UI (sources/index.html) - Add compliance report tool UI (tools/compliance.html)
This commit is contained in:
parent
3ca3a2c3e3
commit
5edb45133f
9 changed files with 7316 additions and 0 deletions
|
|
@ -22,6 +22,7 @@
|
||||||
- [.gbtheme UI Theming](./chapter-02/gbtheme.md)
|
- [.gbtheme UI Theming](./chapter-02/gbtheme.md)
|
||||||
- [.gbdrive File Storage](./chapter-02/gbdrive.md)
|
- [.gbdrive File Storage](./chapter-02/gbdrive.md)
|
||||||
- [Bot Templates](./chapter-02/templates.md)
|
- [Bot Templates](./chapter-02/templates.md)
|
||||||
|
- [Template: CRM Contacts](./chapter-02/template-crm-contacts.md)
|
||||||
|
|
||||||
# Part III - Knowledge Base
|
# Part III - Knowledge Base
|
||||||
|
|
||||||
|
|
@ -151,6 +152,7 @@
|
||||||
- [Architecture Overview](./chapter-07-gbapp/architecture.md)
|
- [Architecture Overview](./chapter-07-gbapp/architecture.md)
|
||||||
- [Building from Source](./chapter-07-gbapp/building.md)
|
- [Building from Source](./chapter-07-gbapp/building.md)
|
||||||
- [Container Deployment (LXC)](./chapter-07-gbapp/containers.md)
|
- [Container Deployment (LXC)](./chapter-07-gbapp/containers.md)
|
||||||
|
- [Docker Deployment](./chapter-07-gbapp/docker-deployment.md)
|
||||||
- [Scaling and Load Balancing](./chapter-07-gbapp/scaling.md)
|
- [Scaling and Load Balancing](./chapter-07-gbapp/scaling.md)
|
||||||
- [Infrastructure Design](./chapter-07-gbapp/infrastructure.md)
|
- [Infrastructure Design](./chapter-07-gbapp/infrastructure.md)
|
||||||
- [Observability](./chapter-07-gbapp/observability.md)
|
- [Observability](./chapter-07-gbapp/observability.md)
|
||||||
|
|
|
||||||
441
docs/src/chapter-02/template-crm-contacts.md
Normal file
441
docs/src/chapter-02/template-crm-contacts.md
Normal file
|
|
@ -0,0 +1,441 @@
|
||||||
|
# CRM Contacts Template
|
||||||
|
|
||||||
|
The CRM Contacts template provides a complete contact management solution with natural language interface. Users can add, search, update, and manage contacts through conversational interactions.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
|----------|-------|
|
||||||
|
| Template ID | `crm/contacts.gbai` |
|
||||||
|
| Category | CRM |
|
||||||
|
| Complexity | Intermediate |
|
||||||
|
| Dependencies | Database, Email (optional) |
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- Add new contacts with validation
|
||||||
|
- Search contacts by name, email, or phone
|
||||||
|
- Update contact information
|
||||||
|
- Tag and categorize contacts
|
||||||
|
- Export contacts to CSV
|
||||||
|
- Integration with email campaigns
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
```basic
|
||||||
|
DEPLOY TEMPLATE "crm/contacts.gbai"
|
||||||
|
```
|
||||||
|
|
||||||
|
Or copy the template folder to your work directory:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp -r templates/crm/contacts.gbai work/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
Add these settings to your `config.csv`:
|
||||||
|
|
||||||
|
| Key | Value | Description |
|
||||||
|
|-----|-------|-------------|
|
||||||
|
| `contacts-table` | `contacts` | Database table name |
|
||||||
|
| `require-email` | `true` | Require email for new contacts |
|
||||||
|
| `allow-duplicates` | `false` | Allow duplicate entries |
|
||||||
|
| `default-tags` | `lead` | Default tags for new contacts |
|
||||||
|
|
||||||
|
## Sample Conversation
|
||||||
|
|
||||||
|
<div class="chat-container">
|
||||||
|
|
||||||
|
<div class="chat-message user">
|
||||||
|
<div class="chat-bubble">
|
||||||
|
<div class="chat-sender">You</div>
|
||||||
|
<div class="chat-text">Hi, I need to add a new contact</div>
|
||||||
|
<div class="chat-time">10:32 AM</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="chat-message bot">
|
||||||
|
<div class="chat-bubble">
|
||||||
|
<div class="chat-sender">CRM Bot</div>
|
||||||
|
<div class="chat-text">I'd be happy to help you add a new contact! 📇
|
||||||
|
|
||||||
|
What's the contact's full name?</div>
|
||||||
|
<div class="chat-time">10:32 AM</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="chat-message user">
|
||||||
|
<div class="chat-bubble">
|
||||||
|
<div class="chat-sender">You</div>
|
||||||
|
<div class="chat-text">Maria Santos</div>
|
||||||
|
<div class="chat-time">10:32 AM</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="chat-message bot">
|
||||||
|
<div class="chat-bubble">
|
||||||
|
<div class="chat-sender">CRM Bot</div>
|
||||||
|
<div class="chat-text">Got it! Maria Santos.
|
||||||
|
|
||||||
|
What's Maria's email address?</div>
|
||||||
|
<div class="chat-time">10:32 AM</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="chat-message user">
|
||||||
|
<div class="chat-bubble">
|
||||||
|
<div class="chat-sender">You</div>
|
||||||
|
<div class="chat-text">maria.santos@company.com</div>
|
||||||
|
<div class="chat-time">10:33 AM</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="chat-message bot">
|
||||||
|
<div class="chat-bubble">
|
||||||
|
<div class="chat-sender">CRM Bot</div>
|
||||||
|
<div class="chat-text">Perfect! And what's the phone number? (You can skip this by typing "skip")</div>
|
||||||
|
<div class="chat-time">10:33 AM</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="chat-message user">
|
||||||
|
<div class="chat-bubble">
|
||||||
|
<div class="chat-sender">You</div>
|
||||||
|
<div class="chat-text">+55 11 98765-4321</div>
|
||||||
|
<div class="chat-time">10:33 AM</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="chat-message bot">
|
||||||
|
<div class="chat-bubble">
|
||||||
|
<div class="chat-sender">CRM Bot</div>
|
||||||
|
<div class="chat-text">Which company does Maria work for?</div>
|
||||||
|
<div class="chat-time">10:33 AM</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="chat-message user">
|
||||||
|
<div class="chat-bubble">
|
||||||
|
<div class="chat-sender">You</div>
|
||||||
|
<div class="chat-text">Tech Solutions Ltd</div>
|
||||||
|
<div class="chat-time">10:34 AM</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="chat-message bot">
|
||||||
|
<div class="chat-bubble">
|
||||||
|
<div class="chat-sender">CRM Bot</div>
|
||||||
|
<div class="chat-text">Great! Here's the contact I'm about to save:
|
||||||
|
|
||||||
|
📋 **Contact Details**
|
||||||
|
━━━━━━━━━━━━━━━━━
|
||||||
|
👤 **Name:** Maria Santos
|
||||||
|
📧 **Email:** maria.santos@company.com
|
||||||
|
📱 **Phone:** +55 11 98765-4321
|
||||||
|
🏢 **Company:** Tech Solutions Ltd
|
||||||
|
🏷️ **Tags:** lead
|
||||||
|
|
||||||
|
Should I save this contact?</div>
|
||||||
|
<div class="chat-time">10:34 AM</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="chat-message user">
|
||||||
|
<div class="chat-bubble">
|
||||||
|
<div class="chat-sender">You</div>
|
||||||
|
<div class="chat-text">Yes, save it</div>
|
||||||
|
<div class="chat-time">10:34 AM</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="chat-message bot">
|
||||||
|
<div class="chat-bubble">
|
||||||
|
<div class="chat-sender">CRM Bot</div>
|
||||||
|
<div class="chat-text">✅ Contact saved successfully!
|
||||||
|
|
||||||
|
Maria Santos has been added to your contacts with ID #1247.
|
||||||
|
|
||||||
|
What would you like to do next?
|
||||||
|
• Add another contact
|
||||||
|
• Search contacts
|
||||||
|
• View recent contacts</div>
|
||||||
|
<div class="chat-time">10:34 AM</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="chat-message user">
|
||||||
|
<div class="chat-bubble">
|
||||||
|
<div class="chat-sender">You</div>
|
||||||
|
<div class="chat-text">Search for contacts at Tech Solutions</div>
|
||||||
|
<div class="chat-time">10:35 AM</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="chat-message bot">
|
||||||
|
<div class="chat-bubble">
|
||||||
|
<div class="chat-sender">CRM Bot</div>
|
||||||
|
<div class="chat-text">🔍 Found 3 contacts at "Tech Solutions":
|
||||||
|
|
||||||
|
1. **Maria Santos** - maria.santos@company.com
|
||||||
|
📱 +55 11 98765-4321 | 🏷️ lead
|
||||||
|
|
||||||
|
2. **João Silva** - joao.silva@techsolutions.com
|
||||||
|
📱 +55 11 91234-5678 | 🏷️ customer
|
||||||
|
|
||||||
|
3. **Ana Costa** - ana.costa@techsolutions.com
|
||||||
|
📱 +55 11 99876-5432 | 🏷️ prospect
|
||||||
|
|
||||||
|
Type a number to see full details, or tell me what else you need.</div>
|
||||||
|
<div class="chat-time">10:35 AM</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
|
## Dialog Files
|
||||||
|
|
||||||
|
### start.bas
|
||||||
|
|
||||||
|
The main entry point for the CRM contacts bot:
|
||||||
|
|
||||||
|
```basic
|
||||||
|
' CRM Contacts - Main Dialog
|
||||||
|
' Handles contact management through conversation
|
||||||
|
|
||||||
|
SET CONTEXT "You are a helpful CRM assistant that manages contacts.
|
||||||
|
Be professional and efficient. Always confirm before saving data."
|
||||||
|
|
||||||
|
ADD SUGGESTION "add" AS "Add new contact"
|
||||||
|
ADD SUGGESTION "search" AS "Search contacts"
|
||||||
|
ADD SUGGESTION "recent" AS "View recent"
|
||||||
|
ADD SUGGESTION "export" AS "Export to CSV"
|
||||||
|
|
||||||
|
BEGIN TALK
|
||||||
|
Welcome to Contact Manager! 📇
|
||||||
|
|
||||||
|
I can help you:
|
||||||
|
• Add new contacts
|
||||||
|
• Search existing contacts
|
||||||
|
• Update contact information
|
||||||
|
• Export your contact list
|
||||||
|
|
||||||
|
What would you like to do?
|
||||||
|
END TALK
|
||||||
|
```
|
||||||
|
|
||||||
|
### add-contact.bas
|
||||||
|
|
||||||
|
Handles adding new contacts:
|
||||||
|
|
||||||
|
```basic
|
||||||
|
' Add Contact Dialog
|
||||||
|
|
||||||
|
TALK "I'd be happy to help you add a new contact! 📇"
|
||||||
|
TALK "What's the contact's full name?"
|
||||||
|
|
||||||
|
HEAR contact_name AS STRING
|
||||||
|
|
||||||
|
TALK "Got it! " + contact_name + "."
|
||||||
|
TALK "What's " + contact_name + "'s email address?"
|
||||||
|
|
||||||
|
HEAR contact_email AS EMAIL
|
||||||
|
|
||||||
|
TALK "Perfect! And what's the phone number? (You can skip this by typing \"skip\")"
|
||||||
|
|
||||||
|
HEAR contact_phone AS STRING
|
||||||
|
|
||||||
|
IF contact_phone = "skip" THEN
|
||||||
|
contact_phone = ""
|
||||||
|
END IF
|
||||||
|
|
||||||
|
TALK "Which company does " + contact_name + " work for?"
|
||||||
|
|
||||||
|
HEAR contact_company AS STRING
|
||||||
|
|
||||||
|
' Build confirmation message
|
||||||
|
WITH contact_data
|
||||||
|
name = contact_name
|
||||||
|
email = contact_email
|
||||||
|
phone = contact_phone
|
||||||
|
company = contact_company
|
||||||
|
tags = "lead"
|
||||||
|
created_at = NOW()
|
||||||
|
END WITH
|
||||||
|
|
||||||
|
TALK "Great! Here's the contact I'm about to save:"
|
||||||
|
TALK ""
|
||||||
|
TALK "📋 **Contact Details**"
|
||||||
|
TALK "━━━━━━━━━━━━━━━━━"
|
||||||
|
TALK "👤 **Name:** " + contact_name
|
||||||
|
TALK "📧 **Email:** " + contact_email
|
||||||
|
TALK "📱 **Phone:** " + contact_phone
|
||||||
|
TALK "🏢 **Company:** " + contact_company
|
||||||
|
TALK "🏷️ **Tags:** lead"
|
||||||
|
TALK ""
|
||||||
|
TALK "Should I save this contact?"
|
||||||
|
|
||||||
|
HEAR confirmation AS STRING
|
||||||
|
|
||||||
|
IF INSTR(LOWER(confirmation), "yes") > 0 OR INSTR(LOWER(confirmation), "save") > 0 THEN
|
||||||
|
SAVE "contacts", contact_data
|
||||||
|
|
||||||
|
contact_id = LAST("contacts", "id")
|
||||||
|
|
||||||
|
TALK "✅ Contact saved successfully!"
|
||||||
|
TALK ""
|
||||||
|
TALK contact_name + " has been added to your contacts with ID #" + contact_id + "."
|
||||||
|
ELSE
|
||||||
|
TALK "No problem! The contact was not saved."
|
||||||
|
END IF
|
||||||
|
|
||||||
|
TALK ""
|
||||||
|
TALK "What would you like to do next?"
|
||||||
|
TALK "• Add another contact"
|
||||||
|
TALK "• Search contacts"
|
||||||
|
TALK "• View recent contacts"
|
||||||
|
```
|
||||||
|
|
||||||
|
### search-contact.bas
|
||||||
|
|
||||||
|
Handles contact search:
|
||||||
|
|
||||||
|
```basic
|
||||||
|
' Search Contact Dialog
|
||||||
|
|
||||||
|
TALK "🔍 What would you like to search for?"
|
||||||
|
TALK "You can search by name, email, company, or phone number."
|
||||||
|
|
||||||
|
HEAR search_term AS STRING
|
||||||
|
|
||||||
|
' Search across multiple fields
|
||||||
|
results = FIND "contacts" WHERE
|
||||||
|
name LIKE "%" + search_term + "%" OR
|
||||||
|
email LIKE "%" + search_term + "%" OR
|
||||||
|
company LIKE "%" + search_term + "%" OR
|
||||||
|
phone LIKE "%" + search_term + "%"
|
||||||
|
|
||||||
|
result_count = COUNT(results)
|
||||||
|
|
||||||
|
IF result_count = 0 THEN
|
||||||
|
TALK "No contacts found matching \"" + search_term + "\"."
|
||||||
|
TALK ""
|
||||||
|
TALK "Would you like to:"
|
||||||
|
TALK "• Try a different search"
|
||||||
|
TALK "• Add a new contact"
|
||||||
|
ELSE
|
||||||
|
TALK "🔍 Found " + result_count + " contact(s) matching \"" + search_term + "\":"
|
||||||
|
TALK ""
|
||||||
|
|
||||||
|
counter = 1
|
||||||
|
FOR EACH contact IN results
|
||||||
|
TALK counter + ". **" + contact.name + "** - " + contact.email
|
||||||
|
TALK " 📱 " + contact.phone + " | 🏷️ " + contact.tags
|
||||||
|
TALK ""
|
||||||
|
counter = counter + 1
|
||||||
|
NEXT
|
||||||
|
|
||||||
|
TALK "Type a number to see full details, or tell me what else you need."
|
||||||
|
END IF
|
||||||
|
```
|
||||||
|
|
||||||
|
## Database Schema
|
||||||
|
|
||||||
|
The template creates this table structure:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE TABLE contacts (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
name VARCHAR(255) NOT NULL,
|
||||||
|
email VARCHAR(255) UNIQUE,
|
||||||
|
phone VARCHAR(50),
|
||||||
|
company VARCHAR(255),
|
||||||
|
tags VARCHAR(255) DEFAULT 'lead',
|
||||||
|
notes TEXT,
|
||||||
|
created_at TIMESTAMP DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP DEFAULT NOW(),
|
||||||
|
created_by UUID REFERENCES users(id)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX idx_contacts_email ON contacts(email);
|
||||||
|
CREATE INDEX idx_contacts_company ON contacts(company);
|
||||||
|
CREATE INDEX idx_contacts_tags ON contacts(tags);
|
||||||
|
```
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
|
||||||
|
The template exposes these REST endpoints:
|
||||||
|
|
||||||
|
| Method | Endpoint | Description |
|
||||||
|
|--------|----------|-------------|
|
||||||
|
| GET | `/api/v1/contacts` | List all contacts |
|
||||||
|
| GET | `/api/v1/contacts/:id` | Get single contact |
|
||||||
|
| POST | `/api/v1/contacts` | Create contact |
|
||||||
|
| PUT | `/api/v1/contacts/:id` | Update contact |
|
||||||
|
| DELETE | `/api/v1/contacts/:id` | Delete contact |
|
||||||
|
| GET | `/api/v1/contacts/search?q=` | Search contacts |
|
||||||
|
| GET | `/api/v1/contacts/export` | Export to CSV |
|
||||||
|
|
||||||
|
## Customization
|
||||||
|
|
||||||
|
### Adding Custom Fields
|
||||||
|
|
||||||
|
Edit `tables.bas` to add custom fields:
|
||||||
|
|
||||||
|
```basic
|
||||||
|
TABLE contacts
|
||||||
|
FIELD id AS INTEGER PRIMARY KEY
|
||||||
|
FIELD name AS STRING(255) REQUIRED
|
||||||
|
FIELD email AS EMAIL UNIQUE
|
||||||
|
FIELD phone AS PHONE
|
||||||
|
FIELD company AS STRING(255)
|
||||||
|
FIELD tags AS STRING(255)
|
||||||
|
FIELD notes AS TEXT
|
||||||
|
' Add your custom fields below
|
||||||
|
FIELD linkedin AS STRING(255)
|
||||||
|
FIELD job_title AS STRING(255)
|
||||||
|
FIELD lead_source AS STRING(100)
|
||||||
|
FIELD lead_score AS INTEGER DEFAULT 0
|
||||||
|
END TABLE
|
||||||
|
```
|
||||||
|
|
||||||
|
### Changing Default Tags
|
||||||
|
|
||||||
|
Update `config.csv`:
|
||||||
|
|
||||||
|
```csv
|
||||||
|
key,value
|
||||||
|
default-tags,"prospect,website"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Adding Validation
|
||||||
|
|
||||||
|
Edit `add-contact.bas` to add custom validation:
|
||||||
|
|
||||||
|
```basic
|
||||||
|
' Validate email domain
|
||||||
|
IF NOT INSTR(contact_email, "@company.com") THEN
|
||||||
|
TALK "⚠️ Warning: This email is not from your company domain."
|
||||||
|
END IF
|
||||||
|
|
||||||
|
' Check for duplicates
|
||||||
|
existing = FIND "contacts" WHERE email = contact_email
|
||||||
|
IF COUNT(existing) > 0 THEN
|
||||||
|
TALK "⚠️ A contact with this email already exists!"
|
||||||
|
TALK "Would you like to update the existing contact instead?"
|
||||||
|
' Handle duplicate logic
|
||||||
|
END IF
|
||||||
|
```
|
||||||
|
|
||||||
|
## Related Templates
|
||||||
|
|
||||||
|
- [Sales Pipeline](./template-sales-pipeline.md) - Track deals and opportunities
|
||||||
|
- [Marketing Campaigns](./template-marketing.md) - Email campaigns and automation
|
||||||
|
- [Customer Support](./template-helpdesk.md) - Support ticket management
|
||||||
|
|
||||||
|
## Support
|
||||||
|
|
||||||
|
For issues with this template:
|
||||||
|
- Check the [troubleshooting guide](../chapter-13-community/README.md)
|
||||||
|
- Open an issue on [GitHub](https://github.com/GeneralBots/BotServer/issues)
|
||||||
|
- Join the [community chat](https://discord.gg/generalbots)
|
||||||
505
docs/src/chapter-07-gbapp/docker-deployment.md
Normal file
505
docs/src/chapter-07-gbapp/docker-deployment.md
Normal file
|
|
@ -0,0 +1,505 @@
|
||||||
|
# Docker Deployment
|
||||||
|
|
||||||
|
General Bots supports multiple Docker deployment strategies to fit your infrastructure needs. This guide covers all available options from single-container deployments to full orchestrated environments.
|
||||||
|
|
||||||
|
> **Note**: Docker support is currently **experimental**. While functional, some features may change in future releases.
|
||||||
|
|
||||||
|
## Deployment Options Overview
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────────────────┐
|
||||||
|
│ DEPLOYMENT OPTIONS │
|
||||||
|
├─────────────────────────────────────────────────────────────────────────┤
|
||||||
|
│ │
|
||||||
|
│ Option 1: All-in-One Container │
|
||||||
|
│ ┌─────────────────────────────────────────────────────────────────┐ │
|
||||||
|
│ │ botserver container │ │
|
||||||
|
│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │
|
||||||
|
│ │ │PostgreSQL│ │ MinIO │ │ Qdrant │ │ Vault │ │BotServer│ │ │
|
||||||
|
│ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │ │
|
||||||
|
│ └─────────────────────────────────────────────────────────────────┘ │
|
||||||
|
│ │
|
||||||
|
│ Option 2: Microservices (Separate Containers) │
|
||||||
|
│ ┌───────────┐ ┌───────────┐ ┌───────────┐ ┌───────────┐ │
|
||||||
|
│ │ PostgreSQL│ │ MinIO │ │ Qdrant │ │ Vault │ │
|
||||||
|
│ └─────┬─────┘ └─────┬─────┘ └─────┬─────┘ └─────┬─────┘ │
|
||||||
|
│ │ │ │ │ │
|
||||||
|
│ └─────────────┴──────┬──────┴─────────────┘ │
|
||||||
|
│ │ │
|
||||||
|
│ ┌──────┴──────┐ │
|
||||||
|
│ │ BotServer │ │
|
||||||
|
│ └─────────────┘ │
|
||||||
|
│ │
|
||||||
|
└─────────────────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
## Option 1: All-in-One Container
|
||||||
|
|
||||||
|
The simplest deployment option runs everything inside a single container. This is ideal for:
|
||||||
|
- Development environments
|
||||||
|
- Small deployments
|
||||||
|
- Quick testing
|
||||||
|
- Resource-constrained environments
|
||||||
|
|
||||||
|
### Quick Start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run -d \
|
||||||
|
--name botserver \
|
||||||
|
-p 8000:8000 \
|
||||||
|
-p 9000:9000 \
|
||||||
|
-v botserver-data:/opt/gbo/data \
|
||||||
|
-e ADMIN_PASS=your-secure-password \
|
||||||
|
pragmatismo/botserver:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
### Docker Compose (All-in-One)
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
version: '3.8'
|
||||||
|
|
||||||
|
services:
|
||||||
|
botserver:
|
||||||
|
image: pragmatismo/botserver:latest
|
||||||
|
container_name: botserver
|
||||||
|
restart: unless-stopped
|
||||||
|
ports:
|
||||||
|
- "8000:8000" # Main API
|
||||||
|
- "9000:9000" # MinIO/Drive
|
||||||
|
- "9001:9001" # MinIO Console
|
||||||
|
volumes:
|
||||||
|
- botserver-data:/opt/gbo/data
|
||||||
|
- botserver-conf:/opt/gbo/conf
|
||||||
|
- botserver-logs:/opt/gbo/logs
|
||||||
|
- ./work:/opt/gbo/work # Your bot packages
|
||||||
|
environment:
|
||||||
|
- ADMIN_PASS=${ADMIN_PASS:-changeme}
|
||||||
|
- DOMAIN=${DOMAIN:-localhost}
|
||||||
|
- TZ=UTC
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 10s
|
||||||
|
retries: 3
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
botserver-data:
|
||||||
|
botserver-conf:
|
||||||
|
botserver-logs:
|
||||||
|
```
|
||||||
|
|
||||||
|
### Resource Requirements (All-in-One)
|
||||||
|
|
||||||
|
| Resource | Minimum | Recommended |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| CPU | 2 cores | 4+ cores |
|
||||||
|
| RAM | 4GB | 8GB+ |
|
||||||
|
| Storage | 20GB | 50GB+ |
|
||||||
|
|
||||||
|
## Option 2: Microservices Deployment
|
||||||
|
|
||||||
|
For production environments, we recommend running each component as a separate container. This provides:
|
||||||
|
- Independent scaling
|
||||||
|
- Better resource allocation
|
||||||
|
- Easier maintenance and updates
|
||||||
|
- High availability options
|
||||||
|
|
||||||
|
### Docker Compose (Microservices)
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
version: '3.8'
|
||||||
|
|
||||||
|
services:
|
||||||
|
# PostgreSQL - Primary Database
|
||||||
|
postgres:
|
||||||
|
image: postgres:16-alpine
|
||||||
|
container_name: gb-postgres
|
||||||
|
restart: unless-stopped
|
||||||
|
volumes:
|
||||||
|
- postgres-data:/var/lib/postgresql/data
|
||||||
|
environment:
|
||||||
|
POSTGRES_USER: botserver
|
||||||
|
POSTGRES_PASSWORD: ${DB_PASSWORD}
|
||||||
|
POSTGRES_DB: botserver
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U botserver"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 5
|
||||||
|
networks:
|
||||||
|
- gb-network
|
||||||
|
|
||||||
|
# MinIO - Object Storage / Drive
|
||||||
|
minio:
|
||||||
|
image: minio/minio:latest
|
||||||
|
container_name: gb-minio
|
||||||
|
restart: unless-stopped
|
||||||
|
command: server /data --console-address ":9001"
|
||||||
|
ports:
|
||||||
|
- "9000:9000"
|
||||||
|
- "9001:9001"
|
||||||
|
volumes:
|
||||||
|
- minio-data:/data
|
||||||
|
environment:
|
||||||
|
MINIO_ROOT_USER: ${DRIVE_ACCESSKEY}
|
||||||
|
MINIO_ROOT_PASSWORD: ${DRIVE_SECRET}
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 20s
|
||||||
|
retries: 3
|
||||||
|
networks:
|
||||||
|
- gb-network
|
||||||
|
|
||||||
|
# Qdrant - Vector Database
|
||||||
|
qdrant:
|
||||||
|
image: qdrant/qdrant:latest
|
||||||
|
container_name: gb-qdrant
|
||||||
|
restart: unless-stopped
|
||||||
|
ports:
|
||||||
|
- "6333:6333"
|
||||||
|
- "6334:6334"
|
||||||
|
volumes:
|
||||||
|
- qdrant-data:/qdrant/storage
|
||||||
|
environment:
|
||||||
|
QDRANT__SERVICE__GRPC_PORT: 6334
|
||||||
|
networks:
|
||||||
|
- gb-network
|
||||||
|
|
||||||
|
# Vault - Secrets Management
|
||||||
|
vault:
|
||||||
|
image: hashicorp/vault:latest
|
||||||
|
container_name: gb-vault
|
||||||
|
restart: unless-stopped
|
||||||
|
cap_add:
|
||||||
|
- IPC_LOCK
|
||||||
|
ports:
|
||||||
|
- "8200:8200"
|
||||||
|
volumes:
|
||||||
|
- vault-data:/vault/data
|
||||||
|
- ./vault-config:/vault/config
|
||||||
|
environment:
|
||||||
|
VAULT_ADDR: http://127.0.0.1:8200
|
||||||
|
command: server -config=/vault/config/config.hcl
|
||||||
|
networks:
|
||||||
|
- gb-network
|
||||||
|
|
||||||
|
# Redis - Caching (Optional but recommended)
|
||||||
|
redis:
|
||||||
|
image: redis:7-alpine
|
||||||
|
container_name: gb-redis
|
||||||
|
restart: unless-stopped
|
||||||
|
volumes:
|
||||||
|
- redis-data:/data
|
||||||
|
command: redis-server --appendonly yes
|
||||||
|
networks:
|
||||||
|
- gb-network
|
||||||
|
|
||||||
|
# InfluxDB - Time Series (Optional - for analytics)
|
||||||
|
influxdb:
|
||||||
|
image: influxdb:2.7-alpine
|
||||||
|
container_name: gb-influxdb
|
||||||
|
restart: unless-stopped
|
||||||
|
ports:
|
||||||
|
- "8086:8086"
|
||||||
|
volumes:
|
||||||
|
- influxdb-data:/var/lib/influxdb2
|
||||||
|
environment:
|
||||||
|
DOCKER_INFLUXDB_INIT_MODE: setup
|
||||||
|
DOCKER_INFLUXDB_INIT_USERNAME: admin
|
||||||
|
DOCKER_INFLUXDB_INIT_PASSWORD: ${INFLUX_PASSWORD}
|
||||||
|
DOCKER_INFLUXDB_INIT_ORG: pragmatismo
|
||||||
|
DOCKER_INFLUXDB_INIT_BUCKET: metrics
|
||||||
|
networks:
|
||||||
|
- gb-network
|
||||||
|
|
||||||
|
# BotServer - Main Application
|
||||||
|
botserver:
|
||||||
|
image: pragmatismo/botserver:latest
|
||||||
|
container_name: gb-botserver
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
|
minio:
|
||||||
|
condition: service_healthy
|
||||||
|
qdrant:
|
||||||
|
condition: service_started
|
||||||
|
ports:
|
||||||
|
- "8000:8000"
|
||||||
|
volumes:
|
||||||
|
- ./work:/opt/gbo/work
|
||||||
|
- botserver-logs:/opt/gbo/logs
|
||||||
|
environment:
|
||||||
|
# Database
|
||||||
|
DATABASE_URL: postgres://botserver:${DB_PASSWORD}@postgres:5432/botserver
|
||||||
|
|
||||||
|
# Drive/Storage
|
||||||
|
DRIVE_URL: http://minio:9000
|
||||||
|
DRIVE_ACCESSKEY: ${DRIVE_ACCESSKEY}
|
||||||
|
DRIVE_SECRET: ${DRIVE_SECRET}
|
||||||
|
|
||||||
|
# Vector DB
|
||||||
|
QDRANT_URL: http://qdrant:6333
|
||||||
|
|
||||||
|
# Vault
|
||||||
|
VAULT_ADDR: http://vault:8200
|
||||||
|
VAULT_TOKEN: ${VAULT_TOKEN}
|
||||||
|
|
||||||
|
# Redis
|
||||||
|
REDIS_URL: redis://redis:6379
|
||||||
|
|
||||||
|
# InfluxDB
|
||||||
|
INFLUX_URL: http://influxdb:8086
|
||||||
|
INFLUX_TOKEN: ${INFLUX_TOKEN}
|
||||||
|
INFLUX_ORG: pragmatismo
|
||||||
|
INFLUX_BUCKET: metrics
|
||||||
|
|
||||||
|
# General
|
||||||
|
ADMIN_PASS: ${ADMIN_PASS}
|
||||||
|
DOMAIN: ${DOMAIN}
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 10s
|
||||||
|
retries: 3
|
||||||
|
networks:
|
||||||
|
- gb-network
|
||||||
|
|
||||||
|
networks:
|
||||||
|
gb-network:
|
||||||
|
driver: bridge
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
postgres-data:
|
||||||
|
minio-data:
|
||||||
|
qdrant-data:
|
||||||
|
vault-data:
|
||||||
|
redis-data:
|
||||||
|
influxdb-data:
|
||||||
|
botserver-logs:
|
||||||
|
```
|
||||||
|
|
||||||
|
### Environment File (.env)
|
||||||
|
|
||||||
|
Create a `.env` file with your configuration:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Database
|
||||||
|
DB_PASSWORD=your-secure-db-password
|
||||||
|
|
||||||
|
# Drive/MinIO
|
||||||
|
DRIVE_ACCESSKEY=minioadmin
|
||||||
|
DRIVE_SECRET=your-minio-secret
|
||||||
|
|
||||||
|
# Vault
|
||||||
|
VAULT_TOKEN=your-vault-token
|
||||||
|
|
||||||
|
# InfluxDB
|
||||||
|
INFLUX_PASSWORD=your-influx-password
|
||||||
|
INFLUX_TOKEN=your-influx-token
|
||||||
|
|
||||||
|
# General
|
||||||
|
ADMIN_PASS=your-admin-password
|
||||||
|
DOMAIN=your-domain.com
|
||||||
|
```
|
||||||
|
|
||||||
|
## Building Custom Images
|
||||||
|
|
||||||
|
### Dockerfile for BotServer
|
||||||
|
|
||||||
|
```dockerfile
|
||||||
|
FROM rust:1.75-slim-bookworm AS builder
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
COPY . .
|
||||||
|
RUN cargo build --release
|
||||||
|
|
||||||
|
FROM debian:bookworm-slim
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y \
|
||||||
|
ca-certificates \
|
||||||
|
libssl3 \
|
||||||
|
curl \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
WORKDIR /opt/gbo
|
||||||
|
|
||||||
|
COPY --from=builder /app/target/release/botserver /opt/gbo/bin/
|
||||||
|
COPY --from=builder /app/templates /opt/gbo/templates/
|
||||||
|
COPY --from=builder /app/ui /opt/gbo/ui/
|
||||||
|
|
||||||
|
ENV PATH="/opt/gbo/bin:${PATH}"
|
||||||
|
|
||||||
|
EXPOSE 8000
|
||||||
|
|
||||||
|
CMD ["botserver"]
|
||||||
|
```
|
||||||
|
|
||||||
|
### Multi-Architecture Build
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build for multiple architectures
|
||||||
|
docker buildx build \
|
||||||
|
--platform linux/amd64,linux/arm64 \
|
||||||
|
-t pragmatismo/botserver:latest \
|
||||||
|
--push .
|
||||||
|
```
|
||||||
|
|
||||||
|
## Kubernetes Deployment
|
||||||
|
|
||||||
|
For large-scale deployments, use Kubernetes:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
name: botserver
|
||||||
|
labels:
|
||||||
|
app: botserver
|
||||||
|
spec:
|
||||||
|
replicas: 3
|
||||||
|
selector:
|
||||||
|
matchLabels:
|
||||||
|
app: botserver
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
app: botserver
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: botserver
|
||||||
|
image: pragmatismo/botserver:latest
|
||||||
|
ports:
|
||||||
|
- containerPort: 8000
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
memory: "512Mi"
|
||||||
|
cpu: "250m"
|
||||||
|
limits:
|
||||||
|
memory: "2Gi"
|
||||||
|
cpu: "1000m"
|
||||||
|
env:
|
||||||
|
- name: DATABASE_URL
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: botserver-secrets
|
||||||
|
key: database-url
|
||||||
|
livenessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health
|
||||||
|
port: 8000
|
||||||
|
initialDelaySeconds: 30
|
||||||
|
periodSeconds: 10
|
||||||
|
readinessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health
|
||||||
|
port: 8000
|
||||||
|
initialDelaySeconds: 5
|
||||||
|
periodSeconds: 5
|
||||||
|
---
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Service
|
||||||
|
metadata:
|
||||||
|
name: botserver
|
||||||
|
spec:
|
||||||
|
selector:
|
||||||
|
app: botserver
|
||||||
|
ports:
|
||||||
|
- port: 80
|
||||||
|
targetPort: 8000
|
||||||
|
type: LoadBalancer
|
||||||
|
```
|
||||||
|
|
||||||
|
## Health Checks and Monitoring
|
||||||
|
|
||||||
|
All containers expose health endpoints:
|
||||||
|
|
||||||
|
| Service | Health Endpoint |
|
||||||
|
|---------|-----------------|
|
||||||
|
| BotServer | `GET /health` |
|
||||||
|
| PostgreSQL | `pg_isready` command |
|
||||||
|
| MinIO | `GET /minio/health/live` |
|
||||||
|
| Qdrant | `GET /health` |
|
||||||
|
| Vault | `GET /v1/sys/health` |
|
||||||
|
| Redis | `redis-cli ping` |
|
||||||
|
| InfluxDB | `GET /health` |
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Container Won't Start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check logs
|
||||||
|
docker logs gb-botserver
|
||||||
|
|
||||||
|
# Check if dependencies are running
|
||||||
|
docker ps
|
||||||
|
|
||||||
|
# Verify network connectivity
|
||||||
|
docker network inspect gb-network
|
||||||
|
```
|
||||||
|
|
||||||
|
### Database Connection Issues
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Test database connection from botserver container
|
||||||
|
docker exec -it gb-botserver psql $DATABASE_URL -c "SELECT 1"
|
||||||
|
|
||||||
|
# Check PostgreSQL logs
|
||||||
|
docker logs gb-postgres
|
||||||
|
```
|
||||||
|
|
||||||
|
### Storage Issues
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check MinIO status
|
||||||
|
docker exec -it gb-minio mc admin info local
|
||||||
|
|
||||||
|
# Check volume mounts
|
||||||
|
docker inspect gb-botserver | jq '.[0].Mounts'
|
||||||
|
```
|
||||||
|
|
||||||
|
### Memory Issues
|
||||||
|
|
||||||
|
If containers are being killed due to OOM:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# Increase memory limits in docker-compose.yml
|
||||||
|
services:
|
||||||
|
botserver:
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
memory: 4G
|
||||||
|
reservations:
|
||||||
|
memory: 2G
|
||||||
|
```
|
||||||
|
|
||||||
|
## Migration from Non-Docker
|
||||||
|
|
||||||
|
To migrate an existing installation to Docker:
|
||||||
|
|
||||||
|
1. **Backup your data**:
|
||||||
|
```bash
|
||||||
|
pg_dump botserver > backup.sql
|
||||||
|
mc cp --recursive /path/to/drive minio/backup/
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Start Docker containers**
|
||||||
|
|
||||||
|
3. **Restore data**:
|
||||||
|
```bash
|
||||||
|
docker exec -i gb-postgres psql -U botserver < backup.sql
|
||||||
|
docker exec -it gb-minio mc cp --recursive /backup minio/drive/
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Copy bot packages** to the `work` volume
|
||||||
|
|
||||||
|
5. **Verify** everything works via the health endpoints
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
- [Scaling and Load Balancing](./scaling.md)
|
||||||
|
- [Infrastructure Design](./infrastructure.md)
|
||||||
|
- [Observability](./observability.md)
|
||||||
739
src/compliance/code_scanner.rs
Normal file
739
src/compliance/code_scanner.rs
Normal file
|
|
@ -0,0 +1,739 @@
|
||||||
|
//! Code Scanner for BASIC Files
|
||||||
|
//!
|
||||||
|
//! Scans .bas files for security issues, fragile code patterns, and misconfigurations.
|
||||||
|
//! Used by the /apicompliance endpoint to generate compliance reports.
|
||||||
|
|
||||||
|
use chrono::{DateTime, Utc};
|
||||||
|
use regex::Regex;
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use std::collections::HashMap;
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use walkdir::WalkDir;
|
||||||
|
|
||||||
|
/// Issue severity levels
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, PartialOrd, Ord)]
|
||||||
|
#[serde(rename_all = "lowercase")]
|
||||||
|
pub enum IssueSeverity {
|
||||||
|
Info,
|
||||||
|
Low,
|
||||||
|
Medium,
|
||||||
|
High,
|
||||||
|
Critical,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl std::fmt::Display for IssueSeverity {
|
||||||
|
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||||
|
match self {
|
||||||
|
IssueSeverity::Info => write!(f, "info"),
|
||||||
|
IssueSeverity::Low => write!(f, "low"),
|
||||||
|
IssueSeverity::Medium => write!(f, "medium"),
|
||||||
|
IssueSeverity::High => write!(f, "high"),
|
||||||
|
IssueSeverity::Critical => write!(f, "critical"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Issue types for categorization
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||||
|
#[serde(rename_all = "snake_case")]
|
||||||
|
pub enum IssueType {
|
||||||
|
PasswordInConfig,
|
||||||
|
HardcodedSecret,
|
||||||
|
DeprecatedKeyword,
|
||||||
|
FragileCode,
|
||||||
|
ConfigurationIssue,
|
||||||
|
UnderscoreInKeyword,
|
||||||
|
MissingVault,
|
||||||
|
InsecurePattern,
|
||||||
|
DeprecatedIfInput,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl std::fmt::Display for IssueType {
|
||||||
|
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||||
|
match self {
|
||||||
|
IssueType::PasswordInConfig => write!(f, "Password in Config"),
|
||||||
|
IssueType::HardcodedSecret => write!(f, "Hardcoded Secret"),
|
||||||
|
IssueType::DeprecatedKeyword => write!(f, "Deprecated Keyword"),
|
||||||
|
IssueType::FragileCode => write!(f, "Fragile Code"),
|
||||||
|
IssueType::ConfigurationIssue => write!(f, "Configuration Issue"),
|
||||||
|
IssueType::UnderscoreInKeyword => write!(f, "Underscore in Keyword"),
|
||||||
|
IssueType::MissingVault => write!(f, "Missing Vault Config"),
|
||||||
|
IssueType::InsecurePattern => write!(f, "Insecure Pattern"),
|
||||||
|
IssueType::DeprecatedIfInput => write!(f, "Deprecated IF...input Pattern"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A single compliance issue found in the code
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
pub struct CodeIssue {
|
||||||
|
pub id: String,
|
||||||
|
pub severity: IssueSeverity,
|
||||||
|
pub issue_type: IssueType,
|
||||||
|
pub title: String,
|
||||||
|
pub description: String,
|
||||||
|
pub file_path: String,
|
||||||
|
pub line_number: Option<usize>,
|
||||||
|
pub code_snippet: Option<String>,
|
||||||
|
pub remediation: String,
|
||||||
|
pub category: String,
|
||||||
|
pub detected_at: DateTime<Utc>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Scan result for a single bot
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
pub struct BotScanResult {
|
||||||
|
pub bot_id: String,
|
||||||
|
pub bot_name: String,
|
||||||
|
pub scanned_at: DateTime<Utc>,
|
||||||
|
pub files_scanned: usize,
|
||||||
|
pub issues: Vec<CodeIssue>,
|
||||||
|
pub stats: ScanStats,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Statistics for a scan
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
|
||||||
|
pub struct ScanStats {
|
||||||
|
pub critical: usize,
|
||||||
|
pub high: usize,
|
||||||
|
pub medium: usize,
|
||||||
|
pub low: usize,
|
||||||
|
pub info: usize,
|
||||||
|
pub total: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ScanStats {
|
||||||
|
pub fn add_issue(&mut self, severity: &IssueSeverity) {
|
||||||
|
match severity {
|
||||||
|
IssueSeverity::Critical => self.critical += 1,
|
||||||
|
IssueSeverity::High => self.high += 1,
|
||||||
|
IssueSeverity::Medium => self.medium += 1,
|
||||||
|
IssueSeverity::Low => self.low += 1,
|
||||||
|
IssueSeverity::Info => self.info += 1,
|
||||||
|
}
|
||||||
|
self.total += 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn merge(&mut self, other: &ScanStats) {
|
||||||
|
self.critical += other.critical;
|
||||||
|
self.high += other.high;
|
||||||
|
self.medium += other.medium;
|
||||||
|
self.low += other.low;
|
||||||
|
self.info += other.info;
|
||||||
|
self.total += other.total;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Full compliance scan result
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
pub struct ComplianceScanResult {
|
||||||
|
pub scanned_at: DateTime<Utc>,
|
||||||
|
pub duration_ms: u64,
|
||||||
|
pub bots_scanned: usize,
|
||||||
|
pub total_files: usize,
|
||||||
|
pub stats: ScanStats,
|
||||||
|
pub bot_results: Vec<BotScanResult>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Pattern definition for scanning
|
||||||
|
struct ScanPattern {
|
||||||
|
regex: Regex,
|
||||||
|
issue_type: IssueType,
|
||||||
|
severity: IssueSeverity,
|
||||||
|
title: String,
|
||||||
|
description: String,
|
||||||
|
remediation: String,
|
||||||
|
category: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Code scanner for BASIC files
|
||||||
|
pub struct CodeScanner {
|
||||||
|
patterns: Vec<ScanPattern>,
|
||||||
|
base_path: PathBuf,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl CodeScanner {
|
||||||
|
/// Create a new code scanner
|
||||||
|
pub fn new(base_path: impl AsRef<Path>) -> Self {
|
||||||
|
let patterns = Self::build_patterns();
|
||||||
|
Self {
|
||||||
|
patterns,
|
||||||
|
base_path: base_path.as_ref().to_path_buf(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Build the list of patterns to scan for
|
||||||
|
fn build_patterns() -> Vec<ScanPattern> {
|
||||||
|
let mut patterns = Vec::new();
|
||||||
|
|
||||||
|
// Critical: Password/secret patterns in code
|
||||||
|
patterns.push(ScanPattern {
|
||||||
|
regex: Regex::new(r#"(?i)password\s*=\s*["'][^"']+["']"#).unwrap(),
|
||||||
|
issue_type: IssueType::PasswordInConfig,
|
||||||
|
severity: IssueSeverity::Critical,
|
||||||
|
title: "Hardcoded Password".to_string(),
|
||||||
|
description: "A password is hardcoded in the source code. This is a critical security risk.".to_string(),
|
||||||
|
remediation: "Move the password to Vault using: vault_password = GET VAULT SECRET \"password_key\"".to_string(),
|
||||||
|
category: "Security".to_string(),
|
||||||
|
});
|
||||||
|
|
||||||
|
patterns.push(ScanPattern {
|
||||||
|
regex: Regex::new(r#"(?i)(api[_-]?key|apikey|secret[_-]?key|client[_-]?secret)\s*=\s*["'][^"']{8,}["']"#).unwrap(),
|
||||||
|
issue_type: IssueType::HardcodedSecret,
|
||||||
|
severity: IssueSeverity::Critical,
|
||||||
|
title: "Hardcoded API Key/Secret".to_string(),
|
||||||
|
description: "An API key or secret is hardcoded in the source code.".to_string(),
|
||||||
|
remediation: "Store secrets in Vault and retrieve with GET VAULT SECRET".to_string(),
|
||||||
|
category: "Security".to_string(),
|
||||||
|
});
|
||||||
|
|
||||||
|
patterns.push(ScanPattern {
|
||||||
|
regex: Regex::new(r#"(?i)token\s*=\s*["'][a-zA-Z0-9_\-]{20,}["']"#).unwrap(),
|
||||||
|
issue_type: IssueType::HardcodedSecret,
|
||||||
|
severity: IssueSeverity::High,
|
||||||
|
title: "Hardcoded Token".to_string(),
|
||||||
|
description: "A token appears to be hardcoded in the source code.".to_string(),
|
||||||
|
remediation: "Store tokens securely in Vault".to_string(),
|
||||||
|
category: "Security".to_string(),
|
||||||
|
});
|
||||||
|
|
||||||
|
// High: Deprecated IF...input pattern
|
||||||
|
patterns.push(ScanPattern {
|
||||||
|
regex: Regex::new(r#"(?i)IF\s+.*\binput\b"#).unwrap(),
|
||||||
|
issue_type: IssueType::DeprecatedIfInput,
|
||||||
|
severity: IssueSeverity::Medium,
|
||||||
|
title: "Deprecated IF...input Pattern".to_string(),
|
||||||
|
description:
|
||||||
|
"Using IF with raw input variable. Prefer HEAR AS for type-safe input handling."
|
||||||
|
.to_string(),
|
||||||
|
remediation: "Replace with: HEAR response AS STRING\nIF response = \"value\" THEN"
|
||||||
|
.to_string(),
|
||||||
|
category: "Code Quality".to_string(),
|
||||||
|
});
|
||||||
|
|
||||||
|
// Medium: Underscore in keywords
|
||||||
|
patterns.push(ScanPattern {
|
||||||
|
regex: Regex::new(r#"(?i)\b(GET_BOT_MEMORY|SET_BOT_MEMORY|GET_USER_MEMORY|SET_USER_MEMORY|USE_KB|USE_TOOL|SEND_MAIL|CREATE_TASK)\b"#).unwrap(),
|
||||||
|
issue_type: IssueType::UnderscoreInKeyword,
|
||||||
|
severity: IssueSeverity::Low,
|
||||||
|
title: "Underscore in Keyword".to_string(),
|
||||||
|
description: "Keywords should use spaces instead of underscores for consistency.".to_string(),
|
||||||
|
remediation: "Use spaces: GET BOT MEMORY, SET BOT MEMORY, etc.".to_string(),
|
||||||
|
category: "Naming Convention".to_string(),
|
||||||
|
});
|
||||||
|
|
||||||
|
// Medium: POST TO INSTAGRAM with inline credentials
|
||||||
|
patterns.push(ScanPattern {
|
||||||
|
regex: Regex::new(r#"(?i)POST\s+TO\s+INSTAGRAM\s+\w+\s*,\s*\w+"#).unwrap(),
|
||||||
|
issue_type: IssueType::InsecurePattern,
|
||||||
|
severity: IssueSeverity::High,
|
||||||
|
title: "Instagram Credentials in Code".to_string(),
|
||||||
|
description:
|
||||||
|
"Instagram username/password passed directly. Use secure credential storage."
|
||||||
|
.to_string(),
|
||||||
|
remediation: "Store Instagram credentials in Vault and retrieve securely.".to_string(),
|
||||||
|
category: "Security".to_string(),
|
||||||
|
});
|
||||||
|
|
||||||
|
// Low: Direct SQL in BASIC
|
||||||
|
patterns.push(ScanPattern {
|
||||||
|
regex: Regex::new(r#"(?i)(SELECT|INSERT|UPDATE|DELETE)\s+.*(FROM|INTO|SET)\s+"#)
|
||||||
|
.unwrap(),
|
||||||
|
issue_type: IssueType::FragileCode,
|
||||||
|
severity: IssueSeverity::Medium,
|
||||||
|
title: "Raw SQL Query".to_string(),
|
||||||
|
description: "Raw SQL queries in BASIC code may be vulnerable to injection."
|
||||||
|
.to_string(),
|
||||||
|
remediation:
|
||||||
|
"Use parameterized queries or the built-in data operations (SAVE, GET, etc.)"
|
||||||
|
.to_string(),
|
||||||
|
category: "Security".to_string(),
|
||||||
|
});
|
||||||
|
|
||||||
|
// Info: Eval or dynamic execution
|
||||||
|
patterns.push(ScanPattern {
|
||||||
|
regex: Regex::new(r#"(?i)\bEVAL\s*\("#).unwrap(),
|
||||||
|
issue_type: IssueType::FragileCode,
|
||||||
|
severity: IssueSeverity::High,
|
||||||
|
title: "Dynamic Code Execution".to_string(),
|
||||||
|
description: "EVAL can execute arbitrary code and is a security risk.".to_string(),
|
||||||
|
remediation: "Avoid EVAL. Use structured control flow instead.".to_string(),
|
||||||
|
category: "Security".to_string(),
|
||||||
|
});
|
||||||
|
|
||||||
|
// Check for base64 encoded secrets (potential obfuscated credentials)
|
||||||
|
patterns.push(ScanPattern {
|
||||||
|
regex: Regex::new(
|
||||||
|
r#"(?i)(password|secret|key|token)\s*=\s*["'][A-Za-z0-9+/=]{40,}["']"#,
|
||||||
|
)
|
||||||
|
.unwrap(),
|
||||||
|
issue_type: IssueType::HardcodedSecret,
|
||||||
|
severity: IssueSeverity::High,
|
||||||
|
title: "Potential Encoded Secret".to_string(),
|
||||||
|
description: "A base64-like string is assigned to a sensitive variable.".to_string(),
|
||||||
|
remediation: "Remove encoded secrets from code. Use Vault for secret management."
|
||||||
|
.to_string(),
|
||||||
|
category: "Security".to_string(),
|
||||||
|
});
|
||||||
|
|
||||||
|
// AWS credentials pattern
|
||||||
|
patterns.push(ScanPattern {
|
||||||
|
regex: Regex::new(r#"(?i)(AKIA[0-9A-Z]{16})"#).unwrap(),
|
||||||
|
issue_type: IssueType::HardcodedSecret,
|
||||||
|
severity: IssueSeverity::Critical,
|
||||||
|
title: "AWS Access Key".to_string(),
|
||||||
|
description: "An AWS access key ID is hardcoded in the source code.".to_string(),
|
||||||
|
remediation: "Remove immediately and rotate the key. Use IAM roles or Vault."
|
||||||
|
.to_string(),
|
||||||
|
category: "Security".to_string(),
|
||||||
|
});
|
||||||
|
|
||||||
|
// Private key patterns
|
||||||
|
patterns.push(ScanPattern {
|
||||||
|
regex: Regex::new(r#"-----BEGIN\s+(RSA\s+)?PRIVATE\s+KEY-----"#).unwrap(),
|
||||||
|
issue_type: IssueType::HardcodedSecret,
|
||||||
|
severity: IssueSeverity::Critical,
|
||||||
|
title: "Private Key in Code".to_string(),
|
||||||
|
description: "A private key is embedded in the source code.".to_string(),
|
||||||
|
remediation: "Remove private key immediately. Store in secure key management system."
|
||||||
|
.to_string(),
|
||||||
|
category: "Security".to_string(),
|
||||||
|
});
|
||||||
|
|
||||||
|
// Connection strings with credentials
|
||||||
|
patterns.push(ScanPattern {
|
||||||
|
regex: Regex::new(r#"(?i)(postgres|mysql|mongodb|redis)://[^:]+:[^@]+@"#).unwrap(),
|
||||||
|
issue_type: IssueType::HardcodedSecret,
|
||||||
|
severity: IssueSeverity::Critical,
|
||||||
|
title: "Database Credentials in Connection String".to_string(),
|
||||||
|
description: "Database connection string contains embedded credentials.".to_string(),
|
||||||
|
remediation: "Use environment variables or Vault for database credentials.".to_string(),
|
||||||
|
category: "Security".to_string(),
|
||||||
|
});
|
||||||
|
|
||||||
|
patterns
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Scan all bots in the base path
|
||||||
|
pub async fn scan_all(
|
||||||
|
&self,
|
||||||
|
) -> Result<ComplianceScanResult, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
|
let start_time = std::time::Instant::now();
|
||||||
|
let mut bot_results = Vec::new();
|
||||||
|
let mut total_stats = ScanStats::default();
|
||||||
|
let mut total_files = 0;
|
||||||
|
|
||||||
|
// Find all .gbai directories (bot packages)
|
||||||
|
let templates_path = self.base_path.join("templates");
|
||||||
|
let work_path = self.base_path.join("work");
|
||||||
|
|
||||||
|
let mut bot_paths = Vec::new();
|
||||||
|
|
||||||
|
// Scan templates directory
|
||||||
|
if templates_path.exists() {
|
||||||
|
for entry in WalkDir::new(&templates_path).max_depth(3) {
|
||||||
|
if let Ok(entry) = entry {
|
||||||
|
let path = entry.path();
|
||||||
|
if path.is_dir() {
|
||||||
|
let name = path.file_name().unwrap_or_default().to_string_lossy();
|
||||||
|
if name.ends_with(".gbai") || name.ends_with(".gbdialog") {
|
||||||
|
bot_paths.push(path.to_path_buf());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Scan work directory (deployed bots)
|
||||||
|
if work_path.exists() {
|
||||||
|
for entry in WalkDir::new(&work_path).max_depth(3) {
|
||||||
|
if let Ok(entry) = entry {
|
||||||
|
let path = entry.path();
|
||||||
|
if path.is_dir() {
|
||||||
|
let name = path.file_name().unwrap_or_default().to_string_lossy();
|
||||||
|
if name.ends_with(".gbai") || name.ends_with(".gbdialog") {
|
||||||
|
bot_paths.push(path.to_path_buf());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Scan each bot
|
||||||
|
for bot_path in &bot_paths {
|
||||||
|
let result = self.scan_bot(bot_path).await?;
|
||||||
|
total_files += result.files_scanned;
|
||||||
|
total_stats.merge(&result.stats);
|
||||||
|
bot_results.push(result);
|
||||||
|
}
|
||||||
|
|
||||||
|
let duration_ms = start_time.elapsed().as_millis() as u64;
|
||||||
|
|
||||||
|
Ok(ComplianceScanResult {
|
||||||
|
scanned_at: Utc::now(),
|
||||||
|
duration_ms,
|
||||||
|
bots_scanned: bot_results.len(),
|
||||||
|
total_files,
|
||||||
|
stats: total_stats,
|
||||||
|
bot_results,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Scan a specific bot directory
|
||||||
|
pub async fn scan_bot(
|
||||||
|
&self,
|
||||||
|
bot_path: &Path,
|
||||||
|
) -> Result<BotScanResult, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
|
let bot_name = bot_path
|
||||||
|
.file_name()
|
||||||
|
.unwrap_or_default()
|
||||||
|
.to_string_lossy()
|
||||||
|
.to_string();
|
||||||
|
|
||||||
|
let bot_id = format!("{:x}", md5::compute(&bot_name));
|
||||||
|
|
||||||
|
let mut issues = Vec::new();
|
||||||
|
let mut stats = ScanStats::default();
|
||||||
|
let mut files_scanned = 0;
|
||||||
|
|
||||||
|
// Find all .bas files in the bot directory
|
||||||
|
for entry in WalkDir::new(bot_path) {
|
||||||
|
if let Ok(entry) = entry {
|
||||||
|
let path = entry.path();
|
||||||
|
if path.is_file() {
|
||||||
|
let extension = path.extension().unwrap_or_default().to_string_lossy();
|
||||||
|
if extension == "bas" || extension == "csv" {
|
||||||
|
files_scanned += 1;
|
||||||
|
let file_issues = self.scan_file(path).await?;
|
||||||
|
for issue in file_issues {
|
||||||
|
stats.add_issue(&issue.severity);
|
||||||
|
issues.push(issue);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check for missing Vault configuration
|
||||||
|
let config_path = bot_path.join("config.csv");
|
||||||
|
if config_path.exists() {
|
||||||
|
let vault_configured = self.check_vault_config(&config_path).await?;
|
||||||
|
if !vault_configured {
|
||||||
|
let issue = CodeIssue {
|
||||||
|
id: uuid::Uuid::new_v4().to_string(),
|
||||||
|
severity: IssueSeverity::Info,
|
||||||
|
issue_type: IssueType::MissingVault,
|
||||||
|
title: "Vault Not Configured".to_string(),
|
||||||
|
description: "This bot is not configured to use Vault for secrets management.".to_string(),
|
||||||
|
file_path: config_path.to_string_lossy().to_string(),
|
||||||
|
line_number: None,
|
||||||
|
code_snippet: None,
|
||||||
|
remediation: "Add VAULT_ADDR and VAULT_TOKEN to configuration for secure secret management.".to_string(),
|
||||||
|
category: "Configuration".to_string(),
|
||||||
|
detected_at: Utc::now(),
|
||||||
|
};
|
||||||
|
stats.add_issue(&issue.severity);
|
||||||
|
issues.push(issue);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sort issues by severity (critical first)
|
||||||
|
issues.sort_by(|a, b| b.severity.cmp(&a.severity));
|
||||||
|
|
||||||
|
Ok(BotScanResult {
|
||||||
|
bot_id,
|
||||||
|
bot_name,
|
||||||
|
scanned_at: Utc::now(),
|
||||||
|
files_scanned,
|
||||||
|
issues,
|
||||||
|
stats,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Scan a single file for issues
|
||||||
|
async fn scan_file(
|
||||||
|
&self,
|
||||||
|
file_path: &Path,
|
||||||
|
) -> Result<Vec<CodeIssue>, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
|
let content = tokio::fs::read_to_string(file_path).await?;
|
||||||
|
let mut issues = Vec::new();
|
||||||
|
|
||||||
|
let relative_path = file_path
|
||||||
|
.strip_prefix(&self.base_path)
|
||||||
|
.unwrap_or(file_path)
|
||||||
|
.to_string_lossy()
|
||||||
|
.to_string();
|
||||||
|
|
||||||
|
for (line_number, line) in content.lines().enumerate() {
|
||||||
|
let line_num = line_number + 1;
|
||||||
|
|
||||||
|
// Skip comments
|
||||||
|
let trimmed = line.trim();
|
||||||
|
if trimmed.starts_with("REM") || trimmed.starts_with("'") || trimmed.starts_with("//") {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
for pattern in &self.patterns {
|
||||||
|
if pattern.regex.is_match(line) {
|
||||||
|
// Redact sensitive information in the snippet
|
||||||
|
let snippet = self.redact_sensitive(line);
|
||||||
|
|
||||||
|
let issue = CodeIssue {
|
||||||
|
id: uuid::Uuid::new_v4().to_string(),
|
||||||
|
severity: pattern.severity.clone(),
|
||||||
|
issue_type: pattern.issue_type.clone(),
|
||||||
|
title: pattern.title.clone(),
|
||||||
|
description: pattern.description.clone(),
|
||||||
|
file_path: relative_path.clone(),
|
||||||
|
line_number: Some(line_num),
|
||||||
|
code_snippet: Some(snippet),
|
||||||
|
remediation: pattern.remediation.clone(),
|
||||||
|
category: pattern.category.clone(),
|
||||||
|
detected_at: Utc::now(),
|
||||||
|
};
|
||||||
|
issues.push(issue);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Ok(issues)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Redact sensitive information in code snippets
|
||||||
|
fn redact_sensitive(&self, line: &str) -> String {
|
||||||
|
let mut result = line.to_string();
|
||||||
|
|
||||||
|
// Redact quoted strings that look like secrets
|
||||||
|
let secret_pattern = Regex::new(r#"(["'])[^"']{8,}(["'])"#).unwrap();
|
||||||
|
result = secret_pattern
|
||||||
|
.replace_all(&result, "$1***REDACTED***$2")
|
||||||
|
.to_string();
|
||||||
|
|
||||||
|
// Redact AWS keys
|
||||||
|
let aws_pattern = Regex::new(r#"AKIA[0-9A-Z]{16}"#).unwrap();
|
||||||
|
result = aws_pattern
|
||||||
|
.replace_all(&result, "AKIA***REDACTED***")
|
||||||
|
.to_string();
|
||||||
|
|
||||||
|
result
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Check if Vault is configured for a bot
|
||||||
|
async fn check_vault_config(
|
||||||
|
&self,
|
||||||
|
config_path: &Path,
|
||||||
|
) -> Result<bool, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
|
let content = tokio::fs::read_to_string(config_path).await?;
|
||||||
|
|
||||||
|
// Check for Vault-related configuration
|
||||||
|
let has_vault = content.to_lowercase().contains("vault_addr")
|
||||||
|
|| content.to_lowercase().contains("vault_token")
|
||||||
|
|| content.to_lowercase().contains("vault-");
|
||||||
|
|
||||||
|
Ok(has_vault)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Scan specific bots by ID
|
||||||
|
pub async fn scan_bots(
|
||||||
|
&self,
|
||||||
|
bot_ids: &[String],
|
||||||
|
) -> Result<ComplianceScanResult, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
|
if bot_ids.is_empty() || bot_ids.contains(&"all".to_string()) {
|
||||||
|
return self.scan_all().await;
|
||||||
|
}
|
||||||
|
|
||||||
|
// For specific bots, we'd need to look them up by ID
|
||||||
|
// For now, scan all and filter
|
||||||
|
let mut full_result = self.scan_all().await?;
|
||||||
|
full_result
|
||||||
|
.bot_results
|
||||||
|
.retain(|r| bot_ids.contains(&r.bot_id) || bot_ids.contains(&r.bot_name));
|
||||||
|
|
||||||
|
// Recalculate stats
|
||||||
|
let mut new_stats = ScanStats::default();
|
||||||
|
for bot in &full_result.bot_results {
|
||||||
|
new_stats.merge(&bot.stats);
|
||||||
|
}
|
||||||
|
full_result.stats = new_stats;
|
||||||
|
full_result.bots_scanned = full_result.bot_results.len();
|
||||||
|
|
||||||
|
Ok(full_result)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Generate a compliance report in various formats
|
||||||
|
pub struct ComplianceReporter;
|
||||||
|
|
||||||
|
impl ComplianceReporter {
|
||||||
|
/// Generate HTML report
|
||||||
|
pub fn to_html(result: &ComplianceScanResult) -> String {
|
||||||
|
let mut html = String::new();
|
||||||
|
|
||||||
|
html.push_str("<!DOCTYPE html><html><head><title>Compliance Report</title>");
|
||||||
|
html.push_str("<style>body{font-family:system-ui;margin:20px;}table{border-collapse:collapse;width:100%;}th,td{border:1px solid #ddd;padding:8px;text-align:left;}.critical{color:#dc2626;}.high{color:#ea580c;}.medium{color:#d97706;}.low{color:#65a30d;}.info{color:#0891b2;}</style>");
|
||||||
|
html.push_str("</head><body>");
|
||||||
|
|
||||||
|
html.push_str(&format!("<h1>Compliance Scan Report</h1>"));
|
||||||
|
html.push_str(&format!("<p>Scanned at: {}</p>", result.scanned_at));
|
||||||
|
html.push_str(&format!("<p>Duration: {}ms</p>", result.duration_ms));
|
||||||
|
html.push_str(&format!("<p>Bots scanned: {}</p>", result.bots_scanned));
|
||||||
|
html.push_str(&format!("<p>Files scanned: {}</p>", result.total_files));
|
||||||
|
|
||||||
|
html.push_str("<h2>Summary</h2>");
|
||||||
|
html.push_str(&format!(
|
||||||
|
"<p class='critical'>Critical: {}</p>",
|
||||||
|
result.stats.critical
|
||||||
|
));
|
||||||
|
html.push_str(&format!("<p class='high'>High: {}</p>", result.stats.high));
|
||||||
|
html.push_str(&format!(
|
||||||
|
"<p class='medium'>Medium: {}</p>",
|
||||||
|
result.stats.medium
|
||||||
|
));
|
||||||
|
html.push_str(&format!("<p class='low'>Low: {}</p>", result.stats.low));
|
||||||
|
html.push_str(&format!("<p class='info'>Info: {}</p>", result.stats.info));
|
||||||
|
|
||||||
|
html.push_str("<h2>Issues</h2>");
|
||||||
|
html.push_str("<table><tr><th>Severity</th><th>Type</th><th>File</th><th>Line</th><th>Description</th></tr>");
|
||||||
|
|
||||||
|
for bot in &result.bot_results {
|
||||||
|
for issue in &bot.issues {
|
||||||
|
html.push_str(&format!(
|
||||||
|
"<tr><td class='{}'>{}</td><td>{}</td><td>{}</td><td>{}</td><td>{}</td></tr>",
|
||||||
|
issue.severity.to_string(),
|
||||||
|
issue.severity,
|
||||||
|
issue.issue_type,
|
||||||
|
issue.file_path,
|
||||||
|
issue
|
||||||
|
.line_number
|
||||||
|
.map(|n| n.to_string())
|
||||||
|
.unwrap_or("-".to_string()),
|
||||||
|
issue.description
|
||||||
|
));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
html.push_str("</table></body></html>");
|
||||||
|
html
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Generate JSON report
|
||||||
|
pub fn to_json(result: &ComplianceScanResult) -> Result<String, serde_json::Error> {
|
||||||
|
serde_json::to_string_pretty(result)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Generate CSV report
|
||||||
|
pub fn to_csv(result: &ComplianceScanResult) -> String {
|
||||||
|
let mut csv = String::new();
|
||||||
|
csv.push_str("Severity,Type,Category,File,Line,Title,Description,Remediation\n");
|
||||||
|
|
||||||
|
for bot in &result.bot_results {
|
||||||
|
for issue in &bot.issues {
|
||||||
|
csv.push_str(&format!(
|
||||||
|
"{},{},{},{},{},{},{},{}\n",
|
||||||
|
issue.severity,
|
||||||
|
issue.issue_type,
|
||||||
|
issue.category,
|
||||||
|
issue.file_path,
|
||||||
|
issue
|
||||||
|
.line_number
|
||||||
|
.map(|n| n.to_string())
|
||||||
|
.unwrap_or("-".to_string()),
|
||||||
|
escape_csv(&issue.title),
|
||||||
|
escape_csv(&issue.description),
|
||||||
|
escape_csv(&issue.remediation)
|
||||||
|
));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
csv
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Escape a string for CSV output
|
||||||
|
fn escape_csv(s: &str) -> String {
|
||||||
|
if s.contains(',') || s.contains('"') || s.contains('\n') {
|
||||||
|
format!("\"{}\"", s.replace('"', "\"\""))
|
||||||
|
} else {
|
||||||
|
s.to_string()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_pattern_matching() {
|
||||||
|
let scanner = CodeScanner::new("/tmp/test");
|
||||||
|
|
||||||
|
// Test password detection
|
||||||
|
let password_pattern = scanner
|
||||||
|
.patterns
|
||||||
|
.iter()
|
||||||
|
.find(|p| matches!(p.issue_type, IssueType::PasswordInConfig))
|
||||||
|
.unwrap();
|
||||||
|
assert!(password_pattern.regex.is_match(r#"password = "secret123""#));
|
||||||
|
assert!(password_pattern.regex.is_match(r#"PASSWORD = 'mypass'"#));
|
||||||
|
|
||||||
|
// Test underscore keyword detection
|
||||||
|
let underscore_pattern = scanner
|
||||||
|
.patterns
|
||||||
|
.iter()
|
||||||
|
.find(|p| matches!(p.issue_type, IssueType::UnderscoreInKeyword))
|
||||||
|
.unwrap();
|
||||||
|
assert!(underscore_pattern.regex.is_match("GET_BOT_MEMORY"));
|
||||||
|
assert!(underscore_pattern.regex.is_match("SET_USER_MEMORY"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_severity_ordering() {
|
||||||
|
assert!(IssueSeverity::Critical > IssueSeverity::High);
|
||||||
|
assert!(IssueSeverity::High > IssueSeverity::Medium);
|
||||||
|
assert!(IssueSeverity::Medium > IssueSeverity::Low);
|
||||||
|
assert!(IssueSeverity::Low > IssueSeverity::Info);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_stats_merge() {
|
||||||
|
let mut stats1 = ScanStats {
|
||||||
|
critical: 1,
|
||||||
|
high: 2,
|
||||||
|
medium: 3,
|
||||||
|
low: 4,
|
||||||
|
info: 5,
|
||||||
|
total: 15,
|
||||||
|
};
|
||||||
|
|
||||||
|
let stats2 = ScanStats {
|
||||||
|
critical: 1,
|
||||||
|
high: 1,
|
||||||
|
medium: 1,
|
||||||
|
low: 1,
|
||||||
|
info: 1,
|
||||||
|
total: 5,
|
||||||
|
};
|
||||||
|
|
||||||
|
stats1.merge(&stats2);
|
||||||
|
|
||||||
|
assert_eq!(stats1.critical, 2);
|
||||||
|
assert_eq!(stats1.high, 3);
|
||||||
|
assert_eq!(stats1.total, 20);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_csv_escape() {
|
||||||
|
assert_eq!(escape_csv("simple"), "simple");
|
||||||
|
assert_eq!(escape_csv("with,comma"), "\"with,comma\"");
|
||||||
|
assert_eq!(escape_csv("with\"quote"), "\"with\"\"quote\"");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_redact_sensitive() {
|
||||||
|
let scanner = CodeScanner::new("/tmp/test");
|
||||||
|
|
||||||
|
let line = r#"password = "supersecretpassword123""#;
|
||||||
|
let redacted = scanner.redact_sensitive(line);
|
||||||
|
assert!(redacted.contains("***REDACTED***"));
|
||||||
|
assert!(!redacted.contains("supersecretpassword123"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -2,6 +2,12 @@
|
||||||
//!
|
//!
|
||||||
//! This module provides automated compliance monitoring, audit logging,
|
//! This module provides automated compliance monitoring, audit logging,
|
||||||
//! risk assessment, and security policy enforcement capabilities.
|
//! risk assessment, and security policy enforcement capabilities.
|
||||||
|
//!
|
||||||
|
//! Includes code scanning for BASIC files to detect:
|
||||||
|
//! - Hardcoded passwords and secrets
|
||||||
|
//! - Deprecated keywords and patterns
|
||||||
|
//! - Configuration issues
|
||||||
|
//! - Security vulnerabilities
|
||||||
|
|
||||||
use chrono::{DateTime, Utc};
|
use chrono::{DateTime, Utc};
|
||||||
use serde::{Deserialize, Serialize};
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
@ -9,10 +15,17 @@ use std::collections::HashMap;
|
||||||
|
|
||||||
pub mod access_review;
|
pub mod access_review;
|
||||||
pub mod audit;
|
pub mod audit;
|
||||||
|
pub mod code_scanner;
|
||||||
pub mod policy_checker;
|
pub mod policy_checker;
|
||||||
pub mod risk_assessment;
|
pub mod risk_assessment;
|
||||||
pub mod training_tracker;
|
pub mod training_tracker;
|
||||||
|
|
||||||
|
// Re-export commonly used types from code_scanner
|
||||||
|
pub use code_scanner::{
|
||||||
|
CodeIssue, CodeScanner, ComplianceReporter, ComplianceScanResult, IssueSeverity, IssueType,
|
||||||
|
ScanStats,
|
||||||
|
};
|
||||||
|
|
||||||
/// Compliance framework types
|
/// Compliance framework types
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||||
pub enum ComplianceFramework {
|
pub enum ComplianceFramework {
|
||||||
|
|
|
||||||
1465
ui/suite/designer.html
Normal file
1465
ui/suite/designer.html
Normal file
File diff suppressed because it is too large
Load diff
1460
ui/suite/drive/index.html
Normal file
1460
ui/suite/drive/index.html
Normal file
File diff suppressed because it is too large
Load diff
1653
ui/suite/sources/index.html
Normal file
1653
ui/suite/sources/index.html
Normal file
File diff suppressed because it is too large
Load diff
1038
ui/suite/tools/compliance.html
Normal file
1038
ui/suite/tools/compliance.html
Normal file
File diff suppressed because it is too large
Load diff
Loading…
Add table
Reference in a new issue