botserver/docs/STALWART_API_MAPPING.md

Stalwart API Mapping for General Bots

Version: 6.1.0
Purpose: Map Stalwart native features vs General Bots custom tables

---

Overview

Stalwart Mail Server provides a comprehensive REST Management API. Many email features that we might implement in our database are already available natively in Stalwart. This document maps what to use from Stalwart vs what we manage ourselves.

---

API Base URL

https://{stalwart_host}:{port}/api

Default ports:

• HTTP: 8080
• HTTPS: 443

---

Feature Mapping

✅ USE STALWART API (Do NOT create tables)

Feature	Stalwart Endpoint	Notes
User/Account Management	GET/POST/PATCH/DELETE /principal/{id}	Create, update, delete email accounts
Email Queue	GET /queue/messages	List queued messages for delivery
Queue Status	GET /queue/status	Check if queue is running
Queue Control	PATCH /queue/status/start PATCH /queue/status/stop	Start/stop queue processing
Reschedule Delivery	PATCH /queue/messages/{id}	Retry failed deliveries
Cancel Delivery	DELETE /queue/messages/{id}	Cancel queued message
Distribution Lists	POST /principal with type: "list"	Mailing lists are "principals"
DKIM Signatures	POST /dkim	Create DKIM keys per domain
DNS Records	GET /dns/records/{domain}	Get required DNS records
Spam Training	POST /spam-filter/train/spam POST /spam-filter/train/ham	Train spam filter
Spam Classification	POST /spam-filter/classify	Test spam score
Telemetry/Metrics	GET /telemetry/metrics	Server metrics for monitoring
Live Metrics	GET /telemetry/metrics/live	Real-time metrics (WebSocket)
Logs	GET /logs	Query server logs
Traces	GET /telemetry/traces	Delivery traces
Live Tracing	GET /telemetry/traces/live	Real-time tracing
DMARC Reports	GET /reports/dmarc	Incoming DMARC reports
TLS Reports	GET /reports/tls	TLS-RPT reports
ARF Reports	GET /reports/arf	Abuse feedback reports
Troubleshooting	GET /troubleshoot/delivery/{recipient}	Debug delivery issues
DMARC Check	POST /troubleshoot/dmarc	Test DMARC/SPF/DKIM
Settings	GET/POST /settings	Server configuration
Undelete	GET/POST /store/undelete/{account_id}	Recover deleted messages
Account Purge	GET /store/purge/account/{id}	Purge account data
Encryption Settings	GET/POST /account/crypto	Encryption-at-rest
2FA/App Passwords	GET/POST /account/auth	Authentication settings

⚠️ USE BOTH (Stalwart + Our Tables)

Feature	Stalwart	Our Table	Why Both?
Auto-Responders	Sieve scripts via settings	email_auto_responders	We store UI config, sync to Stalwart Sieve
Email Rules/Filters	Sieve scripts	email_rules	We store UI-friendly rules, compile to Sieve
Shared Mailboxes	Principal with shared access	shared_mailboxes	We track permissions, Stalwart handles access

✅ USE OUR TABLES (Stalwart doesn't provide)

Feature	Our Table	Why?
Global Email Signature	global_email_signatures	Bot-level branding, not in Stalwart
User Email Signature	email_signatures	User preferences, append before send
Scheduled Send	scheduled_emails	We queue and release at scheduled time
Email Templates	email_templates	Business templates with variables
Email Labels	email_labels, email_label_assignments	UI organization, not IMAP folders
Email Tracking	sent_email_tracking (existing)	Open/click tracking pixels

---

Stalwart API Integration Code

Client Setup

// src/email/stalwart_client.rs
pub struct StalwartClient {
    base_url: String,
    auth_token: String,
    http_client: reqwest::Client,
}

impl StalwartClient {
    pub fn new(base_url: &str, token: &str) -> Self {
        Self {
            base_url: base_url.to_string(),
            auth_token: token.to_string(),
            http_client: reqwest::Client::new(),
        }
    }

    async fn request<T: DeserializeOwned>(&self, method: Method, path: &str, body: Option<Value>) -> Result<T> {
        let url = format!("{}{}", self.base_url, path);
        let mut req = self.http_client.request(method, &url)
            .header("Authorization", format!("Bearer {}", self.auth_token));
        
        if let Some(b) = body {
            req = req.json(&b);
        }
        
        let resp = req.send().await?;
        let data: ApiResponse<T> = resp.json().await?;
        Ok(data.data)
    }
}

Queue Monitoring (for Analytics Dashboard)

impl StalwartClient {
    /// Get email queue status for monitoring dashboard
    pub async fn get_queue_status(&self) -> Result<QueueStatus> {
        let status: bool = self.request(Method::GET, "/api/queue/status", None).await?;
        let messages: QueueList = self.request(Method::GET, "/api/queue/messages?limit=100", None).await?;
        
        Ok(QueueStatus {
            is_running: status,
            total_queued: messages.total,
            messages: messages.items,
        })
    }

    /// Get queued message details
    pub async fn get_queued_message(&self, message_id: &str) -> Result<QueuedMessage> {
        self.request(Method::GET, &format!("/api/queue/messages/{}", message_id), None).await
    }

    /// Retry failed delivery
    pub async fn retry_delivery(&self, message_id: &str) -> Result<bool> {
        self.request(Method::PATCH, &format!("/api/queue/messages/{}", message_id), None).await
    }

    /// Cancel queued message
    pub async fn cancel_delivery(&self, message_id: &str) -> Result<bool> {
        self.request(Method::DELETE, &format!("/api/queue/messages/{}", message_id), None).await
    }

    /// Stop all queue processing
    pub async fn stop_queue(&self) -> Result<bool> {
        self.request(Method::PATCH, "/api/queue/status/stop", None).await
    }

    /// Resume queue processing
    pub async fn start_queue(&self) -> Result<bool> {
        self.request(Method::PATCH, "/api/queue/status/start", None).await
    }
}

Account/Principal Management

impl StalwartClient {
    /// Create email account
    pub async fn create_account(&self, email: &str, password: &str, display_name: &str) -> Result<u64> {
        let body = json!({
            "type": "individual",
            "name": email.split('@').next().unwrap_or(email),
            "emails": [email],
            "secrets": [password],
            "description": display_name,
            "quota": 0,
            "roles": ["user"]
        });
        
        self.request(Method::POST, "/api/principal", Some(body)).await
    }

    /// Create distribution list
    pub async fn create_distribution_list(&self, name: &str, email: &str, members: Vec<String>) -> Result<u64> {
        let body = json!({
            "type": "list",
            "name": name,
            "emails": [email],
            "members": members,
            "description": format!("Distribution list: {}", name)
        });
        
        self.request(Method::POST, "/api/principal", Some(body)).await
    }

    /// Get account details
    pub async fn get_account(&self, account_id: &str) -> Result<Principal> {
        self.request(Method::GET, &format!("/api/principal/{}", account_id), None).await
    }

    /// Update account
    pub async fn update_account(&self, account_id: &str, updates: Vec<AccountUpdate>) -> Result<()> {
        let body: Vec<Value> = updates.iter().map(|u| json!({
            "action": u.action,
            "field": u.field,
            "value": u.value
        })).collect();
        
        self.request(Method::PATCH, &format!("/api/principal/{}", account_id), Some(json!(body))).await
    }

    /// Delete account
    pub async fn delete_account(&self, account_id: &str) -> Result<()> {
        self.request(Method::DELETE, &format!("/api/principal/{}", account_id), None).await
    }
}

Sieve Rules (Auto-Responders & Filters)

impl StalwartClient {
    /// Set vacation/out-of-office auto-responder via Sieve
    pub async fn set_auto_responder(&self, account_id: &str, config: &AutoResponderConfig) -> Result<()> {
        let sieve_script = self.generate_vacation_sieve(config);
        
        let updates = vec![json!({
            "type": "set",
            "prefix": format!("sieve.scripts.{}.vacation", account_id),
            "value": sieve_script
        })];
        
        self.request(Method::POST, "/api/settings", Some(json!(updates))).await
    }

    fn generate_vacation_sieve(&self, config: &AutoResponderConfig) -> String {
        let mut script = String::from("require [\"vacation\", \"variables\"];\n\n");
        
        if let Some(start) = &config.start_date {
            script.push_str(&format!("# Active from: {}\n", start));
        }
        if let Some(end) = &config.end_date {
            script.push_str(&format!("# Active until: {}\n", end));
        }
        
        script.push_str(&format!(
            r#"vacation :days 1 :subject "{}" "{}";"#,
            config.subject.replace('"', "\\\""),
            config.body_plain.replace('"', "\\\"")
        ));
        
        script
    }

    /// Set email filter rule via Sieve
    pub async fn set_filter_rule(&self, account_id: &str, rule: &EmailRule) -> Result<()> {
        let sieve_script = self.generate_filter_sieve(rule);
        
        let updates = vec![json!({
            "type": "set",
            "prefix": format!("sieve.scripts.{}.filter_{}", account_id, rule.id),
            "value": sieve_script
        })];
        
        self.request(Method::POST, "/api/settings", Some(json!(updates))).await
    }

    fn generate_filter_sieve(&self, rule: &EmailRule) -> String {
        let mut script = String::from("require [\"fileinto\", \"reject\", \"vacation\"];\n\n");
        
        // Generate conditions
        for condition in &rule.conditions {
            match condition.field.as_str() {
                "from" => script.push_str(&format!(
                    "if header :contains \"From\" \"{}\" {{\n",
                    condition.value
                )),
                "subject" => script.push_str(&format!(
                    "if header :contains \"Subject\" \"{}\" {{\n",
                    condition.value
                )),
                _ => {}
            }
        }
        
        // Generate actions
        for action in &rule.actions {
            match action.action_type.as_str() {
                "move" => script.push_str(&format!("  fileinto \"{}\";\n", action.value)),
                "delete" => script.push_str("  discard;\n"),
                "mark_read" => script.push_str("  setflag \"\\\\Seen\";\n"),
                _ => {}
            }
        }
        
        if rule.stop_processing {
            script.push_str("  stop;\n");
        }
        
        script.push_str("}\n");
        script
    }
}

Telemetry & Monitoring

impl StalwartClient {
    /// Get server metrics for dashboard
    pub async fn get_metrics(&self) -> Result<Metrics> {
        self.request(Method::GET, "/api/telemetry/metrics", None).await
    }

    /// Get server logs
    pub async fn get_logs(&self, page: u32, limit: u32) -> Result<LogList> {
        self.request(
            Method::GET, 
            &format!("/api/logs?page={}&limit={}", page, limit), 
            None
        ).await
    }

    /// Get delivery traces
    pub async fn get_traces(&self, trace_type: &str, page: u32) -> Result<TraceList> {
        self.request(
            Method::GET,
            &format!("/api/telemetry/traces?type={}&page={}&limit=50", trace_type, page),
            None
        ).await
    }

    /// Get specific trace details
    pub async fn get_trace(&self, trace_id: &str) -> Result<Vec<TraceEvent>> {
        self.request(Method::GET, &format!("/api/telemetry/trace/{}", trace_id), None).await
    }

    /// Get DMARC reports
    pub async fn get_dmarc_reports(&self, page: u32) -> Result<ReportList> {
        self.request(Method::GET, &format!("/api/reports/dmarc?page={}&limit=50", page), None).await
    }

    /// Get TLS reports
    pub async fn get_tls_reports(&self, page: u32) -> Result<ReportList> {
        self.request(Method::GET, &format!("/api/reports/tls?page={}&limit=50", page), None).await
    }
}

Spam Filter

impl StalwartClient {
    /// Train message as spam
    pub async fn train_spam(&self, raw_message: &str) -> Result<()> {
        self.http_client
            .post(&format!("{}/api/spam-filter/train/spam", self.base_url))
            .header("Authorization", format!("Bearer {}", self.auth_token))
            .header("Content-Type", "message/rfc822")
            .body(raw_message.to_string())
            .send()
            .await?;
        Ok(())
    }

    /// Train message as ham (not spam)
    pub async fn train_ham(&self, raw_message: &str) -> Result<()> {
        self.http_client
            .post(&format!("{}/api/spam-filter/train/ham", self.base_url))
            .header("Authorization", format!("Bearer {}", self.auth_token))
            .header("Content-Type", "message/rfc822")
            .body(raw_message.to_string())
            .send()
            .await?;
        Ok(())
    }

    /// Classify message (get spam score)
    pub async fn classify_message(&self, message: &SpamClassifyRequest) -> Result<SpamClassifyResult> {
        self.request(Method::POST, "/api/spam-filter/classify", Some(json!(message))).await
    }
}

---

Monitoring Dashboard Integration

Endpoints to Poll

Metric	Endpoint	Poll Interval
Queue Size	GET /queue/messages	30s
Queue Status	GET /queue/status	30s
Server Metrics	GET /telemetry/metrics	60s
Recent Logs	GET /logs?limit=100	60s
Delivery Traces	GET /telemetry/traces?type=delivery.attempt-start	60s
Failed Deliveries	GET /queue/messages?filter=status:failed	60s

WebSocket Endpoints (Real-time)

Feature	Endpoint	Token Endpoint
Live Metrics	ws://.../telemetry/metrics/live	GET /telemetry/live/metrics-token
Live Traces	ws://.../telemetry/traces/live	GET /telemetry/live/tracing-token

---

Tables to REMOVE from Migration

Based on this mapping, these tables are REDUNDANT and should be removed:

-- REMOVE: Stalwart handles distribution lists via principals
-- DROP TABLE IF EXISTS distribution_lists;

-- KEEP: We need this for UI config, but sync to Stalwart Sieve
-- email_auto_responders (KEEP but add stalwart_sieve_id column)

-- KEEP: We need this for UI config, but sync to Stalwart Sieve  
-- email_rules (KEEP but add stalwart_sieve_id column)

---

Migration Updates Needed

The current 6.1.0_enterprise_suite migration already correctly:

1. ✅ Keeps global_email_signatures - Stalwart doesn't have this
2. ✅ Keeps email_signatures - User preference, not in Stalwart
3. ✅ Keeps scheduled_emails - We manage scheduling
4. ✅ Keeps email_templates - Business feature
5. ✅ Keeps email_labels - UI organization
6. ✅ Has stalwart_sieve_id in email_auto_responders - For sync
7. ✅ Has stalwart_sieve_id in email_rules - For sync
8. ✅ Has stalwart_account_id in shared_mailboxes - For sync

The distribution_lists table could potentially be removed since Stalwart handles lists as principals, BUT we may want to keep it for:

• Caching/faster lookups
• UI metadata not stored in Stalwart
• Offline resilience

Recommendation: Keep distribution_lists but sync with Stalwart principals.

---

Sync Strategy

On Create (Our DB → Stalwart)

async fn create_distribution_list(db: &Pool, stalwart: &StalwartClient, list: NewDistributionList) -> Result<Uuid> {
    // 1. Create in Stalwart first
    let stalwart_id = stalwart.create_distribution_list(
        &list.name,
        &list.email_alias,
        list.members.clone()
    ).await?;
    
    // 2. Store in our DB with stalwart reference
    let id = db.insert_distribution_list(DistributionList {
        name: list.name,
        email_alias: list.email_alias,
        members_json: serde_json::to_string(&list.members)?,
        stalwart_principal_id: Some(stalwart_id.to_string()),
        ..Default::default()
    }).await?;
    
    Ok(id)
}

On Update (Sync both)

async fn update_distribution_list(db: &Pool, stalwart: &StalwartClient, id: Uuid, updates: ListUpdates) -> Result<()> {
    // 1. Get current record
    let list = db.get_distribution_list(&id).await?;
    
    // 2. Update Stalwart if we have a reference
    if let Some(stalwart_id) = &list.stalwart_principal_id {
        stalwart.update_principal(stalwart_id, updates.to_stalwart_updates()).await?;
    }
    
    // 3. Update our DB
    db.update_distribution_list(&id, updates).await?;
    
    Ok(())
}

On Delete (Both)

async fn delete_distribution_list(db: &Pool, stalwart: &StalwartClient, id: Uuid) -> Result<()> {
    let list = db.get_distribution_list(&id).await?;
    
    // 1. Delete from Stalwart
    if let Some(stalwart_id) = &list.stalwart_principal_id {
        stalwart.delete_principal(stalwart_id).await?;
    }
    
    // 2. Delete from our DB
    db.delete_distribution_list(&id).await?;
    
    Ok(())
}

---

Summary

Category	Use Stalwart	Use Our Tables	Use Both
Account Management	✅
Email Queue	✅
Queue Monitoring	✅
Distribution Lists			✅
Auto-Responders			✅
Email Rules/Filters			✅
Shared Mailboxes			✅
Email Signatures		✅
Scheduled Send		✅
Email Templates		✅
Email Labels		✅
Email Tracking		✅
Spam Training	✅
Telemetry/Logs	✅
DMARC/TLS Reports	✅