GeneralBots/botserver
2
1
Fork 0
Code Issues Pull requests Projects Releases 323 Packages Wiki Activity Actions

feat(email): implement email read tracking with pixel support

Browse source 
- Add email-read-pixel config parameter to enable/disable tracking
- Implement tracking pixel injection in HTML emails
- Add sent_email_tracking table with migration
- Create 4 new API endpoints:
  - GET /api/email/tracking/pixel/{id} - serve pixel & record read
  - GET /api/email/tracking/status/{id} - get email read status
  - GET /api/email/tracking/list - list all tracked emails
  - GET /api/email/tracking/stats - get aggregate statistics
- Store tracking data: read_at, read_count, IP, user_agent
- Integrate with send_email() to auto-inject pixel when enabled
This commit is contained in:
Rodrigo Rodriguez (Pragmatismo) 2025-12-04 18:15:09 -03:00
parent 896156609b
commit 0c11cf8d5c
14 changed files with 689 additions and 76 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

42
PROMPT.md
View file

@ -5,6 +5,48 @@
---
## Official Icons - MANDATORY
**NEVER generate icons with LLM. ALWAYS use official SVG icons from assets.**
Icons are stored in two locations (kept in sync):
- `botui/ui/suite/assets/icons/` - Runtime icons for UI
- `botbook/src/assets/icons/` - Documentation icons
### Available Icons
| Icon | File | Usage |
|------|------|-------|
| Logo | `gb-logo.svg` | Main GB branding |
| Bot | `gb-bot.svg` | Bot/assistant representation |
| Analytics | `gb-analytics.svg` | Charts, metrics, dashboards |
| Calendar | `gb-calendar.svg` | Scheduling, events |
| Chat | `gb-chat.svg` | Conversations, messaging |
| Compliance | `gb-compliance.svg` | Security, auditing |
| Designer | `gb-designer.svg` | Workflow automation |
| Drive | `gb-drive.svg` | File storage, documents |
| Mail | `gb-mail.svg` | Email functionality |
| Meet | `gb-meet.svg` | Video conferencing |
| Paper | `gb-paper.svg` | Document editing |
| Research | `gb-research.svg` | Search, investigation |
| Sources | `gb-sources.svg` | Knowledge bases |
| Tasks | `gb-tasks.svg` | Task management |
### Icon Guidelines
- All icons use `stroke="currentColor"` for CSS theming
- ViewBox: `0 0 24 24`
- Stroke width: `1.5`
- Rounded line caps and joins
**DO NOT:**
- Generate new icons with AI/LLM
- Use emoji or unicode symbols as icons
- Use external icon libraries
- Create inline SVG content
---
## Project Overview
BotServer is the core backend for General Bots - an open-source conversational AI platform built in Rust. It provides:

8
docs/guides/templates.md
View file

@ -79,7 +79,7 @@ name,value
theme-title,My CRM Bot
theme-color1,#2196F3
theme-color2,#E3F2FD
prompt-history,2
episodic-memory-history,2
```
### 4. Customize Knowledge Base
@ -144,8 +144,8 @@ theme-title,My Template
theme-color1,#1565C0
theme-color2,#E3F2FD
theme-logo,https://example.com/logo.svg
prompt-history,2
prompt-compact,4
episodic-memory-history,2
episodic-memory-threshold,4
```
### Step 3: Create Start Dialog
@ -239,7 +239,7 @@ Email support@example.com.
- Use clear, descriptive `theme-title`
- Choose accessible color combinations
- Set appropriate `prompt-history` (2-4 recommended)
- Set appropriate `episodic-memory-history` (2-4 recommended)
### Knowledge Base

32
docs/reference/configuration.md
View file

@ -50,28 +50,31 @@ theme-color2,#E3F2FD
| Red | `#C62828` | `#FFEBEE` |
| Dark | `#212121` | `#424242` |
## Prompt Settings
## Episodic Memory Settings
| Setting | Description | Default | Range |
|---------|-------------|---------|-------|
| `prompt-history` | Messages in context | `2` | 1-10 |
| `prompt-compact` | Compact mode threshold | `4` | 2-20 |
| `prompt-max-tokens` | Max response tokens | `2048` | 256-8192 |
| `prompt-temperature` | Response creativity | `0.7` | 0.0-2.0 |
| `episodic-memory-history` | Messages in context | `2` | 1-10 |
| `episodic-memory-threshold` | Compaction threshold | `4` | 2-20 |
| `episodic-memory-enabled` | Enable episodic memory | `true` | Boolean |
| `episodic-memory-model` | Model for summarization | `fast` | String |
| `episodic-memory-max-episodes` | Max episodes per user | `100` | 1-1000 |
| `episodic-memory-retention-days` | Days to retain episodes | `365` | 1-3650 |
| `episodic-memory-auto-summarize` | Auto-summarize conversations | `true` | Boolean |
```csv
name,value
prompt-history,2
prompt-compact,4
prompt-max-tokens,2048
prompt-temperature,0.7
episodic-memory-history,2
episodic-memory-threshold,4
episodic-memory-enabled,true
episodic-memory-auto-summarize,true
```
### History Settings
- `prompt-history=1`: Minimal context, faster responses
- `prompt-history=2`: Balanced (recommended)
- `prompt-history=5`: More context, slower responses
- `episodic-memory-history=1`: Minimal context, faster responses
- `episodic-memory-history=2`: Balanced (recommended)
- `episodic-memory-history=5`: More context, slower responses
## LLM Settings
@ -286,9 +289,8 @@ theme-title,Acme Support Bot
theme-color1,#1565C0
theme-color2,#E3F2FD
theme-logo,https://acme.com/logo.svg
prompt-history,2
prompt-compact,4
prompt-temperature,0.7
episodic-memory-history,2
episodic-memory-threshold,4
llm-provider,openai
llm-model,gpt-4-turbo
feature-voice,false

19
migrations/6.2.1_email_tracking/down.sql Normal file
View file

@ -0,0 +1,19 @@
-- Down migration: Remove email tracking table and related objects
-- Drop trigger first
DROP TRIGGER IF EXISTS trigger_update_sent_email_tracking_updated_at ON sent_email_tracking;
-- Drop function
DROP FUNCTION IF EXISTS update_sent_email_tracking_updated_at();
-- Drop indexes
DROP INDEX IF EXISTS idx_sent_email_tracking_tracking_id;
DROP INDEX IF EXISTS idx_sent_email_tracking_bot_id;
DROP INDEX IF EXISTS idx_sent_email_tracking_account_id;
DROP INDEX IF EXISTS idx_sent_email_tracking_to_email;
DROP INDEX IF EXISTS idx_sent_email_tracking_sent_at;
DROP INDEX IF EXISTS idx_sent_email_tracking_is_read;
DROP INDEX IF EXISTS idx_sent_email_tracking_read_status;
-- Drop table
DROP TABLE IF EXISTS sent_email_tracking;

56
migrations/6.2.1_email_tracking/up.sql Normal file
View file

@ -0,0 +1,56 @@
-- Email Read Tracking Table
-- Stores sent email tracking data for read receipt functionality
-- Enabled via config.csv: email-read-pixel,true
CREATE TABLE IF NOT EXISTS sent_email_tracking (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tracking_id UUID NOT NULL UNIQUE,
bot_id UUID NOT NULL,
account_id UUID NOT NULL,
from_email VARCHAR(255) NOT NULL,
to_email VARCHAR(255) NOT NULL,
cc TEXT,
bcc TEXT,
subject TEXT NOT NULL,
sent_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
is_read BOOLEAN NOT NULL DEFAULT FALSE,
read_at TIMESTAMPTZ,
read_count INTEGER NOT NULL DEFAULT 0,
first_read_ip VARCHAR(45),
last_read_ip VARCHAR(45),
user_agent TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
-- Indexes for efficient queries
CREATE INDEX IF NOT EXISTS idx_sent_email_tracking_tracking_id ON sent_email_tracking(tracking_id);
CREATE INDEX IF NOT EXISTS idx_sent_email_tracking_bot_id ON sent_email_tracking(bot_id);
CREATE INDEX IF NOT EXISTS idx_sent_email_tracking_account_id ON sent_email_tracking(account_id);
CREATE INDEX IF NOT EXISTS idx_sent_email_tracking_to_email ON sent_email_tracking(to_email);
CREATE INDEX IF NOT EXISTS idx_sent_email_tracking_sent_at ON sent_email_tracking(sent_at DESC);
CREATE INDEX IF NOT EXISTS idx_sent_email_tracking_is_read ON sent_email_tracking(is_read);
CREATE INDEX IF NOT EXISTS idx_sent_email_tracking_read_status ON sent_email_tracking(bot_id, is_read, sent_at DESC);
-- Trigger to auto-update updated_at
CREATE OR REPLACE FUNCTION update_sent_email_tracking_updated_at()
RETURNS TRIGGER AS $$
BEGIN
NEW.updated_at = NOW();
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
DROP TRIGGER IF EXISTS trigger_update_sent_email_tracking_updated_at ON sent_email_tracking;
CREATE TRIGGER trigger_update_sent_email_tracking_updated_at
BEFORE UPDATE ON sent_email_tracking
FOR EACH ROW
EXECUTE FUNCTION update_sent_email_tracking_updated_at();
-- Add comment for documentation
COMMENT ON TABLE sent_email_tracking IS 'Tracks sent emails for read receipt functionality via tracking pixel';
COMMENT ON COLUMN sent_email_tracking.tracking_id IS 'Unique ID embedded in tracking pixel URL';
COMMENT ON COLUMN sent_email_tracking.is_read IS 'Whether the email has been opened (pixel loaded)';
COMMENT ON COLUMN sent_email_tracking.read_count IS 'Number of times the email was opened';
COMMENT ON COLUMN sent_email_tracking.first_read_ip IS 'IP address of first email open';
COMMENT ON COLUMN sent_email_tracking.last_read_ip IS 'IP address of most recent email open';

66
src/basic/keywords/episodic_memory.rs
View file

@ -29,11 +29,12 @@
//! ```csv
//! name,value
//! episodic-memory-enabled,true
//! episodic-summary-threshold,20
//! episodic-summary-model,fast
//! episodic-max-episodes,100
//! episodic-retention-days,365
//! episodic-auto-summarize,true
//! episodic-memory-threshold,4
//! episodic-memory-history,2
//! episodic-memory-model,fast
//! episodic-memory-max-episodes,100
//! episodic-memory-retention-days,365
//! episodic-memory-auto-summarize,true
//! ```
use chrono::{DateTime, Duration, Utc};
@ -170,9 +171,11 @@ pub struct EpisodicMemoryConfig {
/// Whether episodic memory is enabled
pub enabled: bool,
/// Message count threshold before auto-summarization
pub summary_threshold: usize,
pub threshold: usize,
/// Number of recent exchanges to keep in full
pub history: usize,
/// Model to use for summarization
pub summary_model: String,
pub model: String,
/// Maximum episodes to keep per user
pub max_episodes: usize,
/// Days to retain episodes
@ -185,8 +188,9 @@ impl Default for EpisodicMemoryConfig {
fn default() -> Self {
EpisodicMemoryConfig {
enabled: true,
summary_threshold: 20,
summary_model: "fast".to_string(),
threshold: 4,
history: 2,
model: "fast".to_string(),
max_episodes: 100,
retention_days: 365,
auto_summarize: true,
@ -222,24 +226,28 @@ impl EpisodicMemoryManager {
.get("episodic-memory-enabled")
.map(|v| v == "true")
.unwrap_or(true),
summary_threshold: config_map
.get("episodic-summary-threshold")
threshold: config_map
.get("episodic-memory-threshold")
.and_then(|v| v.parse().ok())
.unwrap_or(20),
summary_model: config_map
.get("episodic-summary-model")
.unwrap_or(4),
history: config_map
.get("episodic-memory-history")
.and_then(|v| v.parse().ok())
.unwrap_or(2),
model: config_map
.get("episodic-memory-model")
.cloned()
.unwrap_or_else(|| "fast".to_string()),
max_episodes: config_map
.get("episodic-max-episodes")
.get("episodic-memory-max-episodes")
.and_then(|v| v.parse().ok())
.unwrap_or(100),
retention_days: config_map
.get("episodic-retention-days")
.get("episodic-memory-retention-days")
.and_then(|v| v.parse().ok())
.unwrap_or(365),
auto_summarize: config_map
.get("episodic-auto-summarize")
.get("episodic-memory-auto-summarize")
.map(|v| v == "true")
.unwrap_or(true),
};
@ -248,9 +256,17 @@ impl EpisodicMemoryManager {
/// Check if auto-summarization should trigger
pub fn should_summarize(&self, message_count: usize) -> bool {
self.config.enabled
&& self.config.auto_summarize
&& message_count >= self.config.summary_threshold
self.config.enabled && self.config.auto_summarize && message_count >= self.config.threshold
}
/// Get number of recent exchanges to keep in full
pub fn get_history_to_keep(&self) -> usize {
self.config.history
}
/// Get the threshold value
pub fn get_threshold(&self) -> usize {
self.config.threshold
}
/// Generate the summarization prompt
@ -668,7 +684,8 @@ mod tests {
fn test_default_config() {
let config = EpisodicMemoryConfig::default();
assert!(config.enabled);
assert_eq!(config.summary_threshold, 20);
assert_eq!(config.threshold, 4);
assert_eq!(config.history, 2);
assert_eq!(config.max_episodes, 100);
}
@ -676,14 +693,15 @@ mod tests {
fn test_should_summarize() {
let manager = EpisodicMemoryManager::new(EpisodicMemoryConfig {
enabled: true,
summary_threshold: 10,
threshold: 4,
history: 2,
auto_summarize: true,
..Default::default()
});
assert!(!manager.should_summarize(5));
assert!(!manager.should_summarize(2));
assert!(manager.should_summarize(4));
assert!(manager.should_summarize(10));
assert!(manager.should_summarize(15));
}
#[test]

498
src/email/mod.rs
View file

@ -1,6 +1,6 @@
use crate::{config::EmailConfig, core::urls::ApiUrls, shared::state::AppState};
use axum::{
extract::{Path, State},
extract::{Path, Query, State},
http::StatusCode,
response::{IntoResponse, Response},
Json,
@ -10,10 +10,11 @@ use axum::{
Router,
};
use base64::{engine::general_purpose, Engine as _};
use chrono::{DateTime, Utc};
use diesel::prelude::*;
use imap::types::Seq;
use lettre::{transport::smtp::authentication::Credentials, Message, SmtpTransport, Transport};
use log::info;
use log::{debug, info, warn};
use mailparse::{parse_mail, MailHeaderMap};
use serde::{Deserialize, Serialize};
use std::sync::Arc;
@ -64,6 +65,11 @@ pub fn configure() -> Router<Arc<AppState>> {
)
.route("/api/email/:id", get(get_email_content_htmx))
.route("/api/email/:id", delete(delete_email_htmx))
// Email read tracking endpoints
.route("/api/email/tracking/pixel/{tracking_id}", get(serve_tracking_pixel))
.route("/api/email/tracking/status/{tracking_id}", get(get_tracking_status))
.route("/api/email/tracking/list", get(list_sent_emails_tracking))
.route("/api/email/tracking/stats", get(get_tracking_stats))
}
// Export SaveDraftRequest for other modules
@ -77,6 +83,65 @@ pub struct SaveDraftRequest {
pub body: String,
}
// ===== Email Tracking Structures =====
/// Sent email tracking record
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SentEmailTracking {
pub id: String,
pub tracking_id: String,
pub bot_id: String,
pub account_id: String,
pub from_email: String,
pub to_email: String,
pub cc: Option<String>,
pub bcc: Option<String>,
pub subject: String,
pub sent_at: DateTime<Utc>,
pub read_at: Option<DateTime<Utc>>,
pub read_count: i32,
pub first_read_ip: Option<String>,
pub last_read_ip: Option<String>,
pub user_agent: Option<String>,
pub is_read: bool,
}
/// Tracking status response for UI
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TrackingStatusResponse {
pub tracking_id: String,
pub to_email: String,
pub subject: String,
pub sent_at: String,
pub is_read: bool,
pub read_at: Option<String>,
pub read_count: i32,
}
/// Query params for tracking pixel
#[derive(Debug, Deserialize)]
pub struct TrackingPixelQuery {
pub t: Option<String>, // Additional tracking token
}
/// Query params for listing tracked emails
#[derive(Debug, Deserialize)]
pub struct ListTrackingQuery {
pub account_id: Option<String>,
pub limit: Option<i64>,
pub offset: Option<i64>,
pub filter: Option<String>, // "all", "read", "unread"
}
/// Tracking statistics response
#[derive(Debug, Serialize)]
pub struct TrackingStatsResponse {
pub total_sent: i64,
pub total_read: i64,
pub read_rate: f64,
pub avg_time_to_read_hours: Option<f64>,
}
// ===== Request/Response Structures =====
#[derive(Debug, Serialize, Deserialize)]
@ -598,6 +663,17 @@ pub async fn send_email(
format!("{} <{}>", display_name, from_email)
};
// Check if email-read-pixel is enabled in bot config
let pixel_enabled = is_tracking_pixel_enabled(&state, None).await;
let tracking_id = Uuid::new_v4();
// Build email body with tracking pixel if enabled
let final_body = if pixel_enabled && request.is_html {
inject_tracking_pixel(&request.body, &tracking_id.to_string(), &state).await
} else {
request.body.clone()
};
// Build email
let mut email_builder = Message::builder()
.from(
@ -609,15 +685,15 @@ pub async fn send_email(
.to
.parse()
.map_err(|e| EmailError(format!("Invalid to address: {}", e)))?)
.subject(request.subject);
.subject(request.subject.clone());
if let Some(cc) = request.cc {
if let Some(ref cc) = request.cc {
email_builder = email_builder.cc(cc
.parse()
.map_err(|e| EmailError(format!("Invalid cc address: {}", e)))?);
}
if let Some(bcc) = request.bcc {
if let Some(ref bcc) = request.bcc {
email_builder = email_builder.bcc(
bcc.parse()
.map_err(|e| EmailError(format!("Invalid bcc address: {}", e)))?,
@ -625,7 +701,7 @@ pub async fn send_email(
}
let email = email_builder
.body(request.body)
.body(final_body)
.map_err(|e| EmailError(format!("Failed to build email: {}", e)))?;
// Send email
@ -640,7 +716,31 @@ pub async fn send_email(
.send(&email)
.map_err(|e| EmailError(format!("Failed to send email: {}", e)))?;
info!("Email sent successfully from account {}", account_uuid);
// Save tracking record if pixel tracking is enabled
if pixel_enabled {
let conn = state.conn.clone();
let to_email = request.to.clone();
let subject = request.subject.clone();
let cc_clone = request.cc.clone();
let bcc_clone = request.bcc.clone();
let _ = tokio::task::spawn_blocking(move || {
save_email_tracking_record(
conn,
tracking_id,
account_uuid,
Uuid::nil(), // bot_id - would come from session in production
&from_email,
&to_email,
cc_clone.as_deref(),
bcc_clone.as_deref(),
&subject,
)
})
.await;
}
info!("Email sent successfully from account {} with tracking_id {}", account_uuid, tracking_id);
Ok(Json(ApiResponse {
success: true,
@ -793,6 +893,390 @@ pub async fn save_click(
(StatusCode::OK, [("content-type", "image/gif")], pixel)
}
// ===== Email Read Tracking Functions =====
/// 1x1 transparent GIF pixel bytes
const TRACKING_PIXEL: [u8; 43] = [
0x47, 0x49, 0x46, 0x38, 0x39, 0x61, 0x01, 0x00, 0x01, 0x00, 0x80, 0x00, 0x00, 0xFF, 0xFF,
0xFF, 0x00, 0x00, 0x00, 0x21, 0xF9, 0x04, 0x01, 0x00, 0x00, 0x00, 0x00, 0x2C, 0x00, 0x00,
0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00, 0x02, 0x02, 0x44, 0x01, 0x00, 0x3B,
];
/// Check if email-read-pixel is enabled in config
async fn is_tracking_pixel_enabled(state: &Arc<AppState>, bot_id: Option<Uuid>) -> bool {
let config_manager = crate::core::config::ConfigManager::new(state.conn.clone());
let bot_id = bot_id.unwrap_or(Uuid::nil());
config_manager
.get_config(&bot_id, "email-read-pixel", Some("false"))
.map(|v| v.to_lowercase() == "true")
.unwrap_or(false)
}
/// Inject tracking pixel into HTML email body
async fn inject_tracking_pixel(html_body: &str, tracking_id: &str, state: &Arc<AppState>) -> String {
// Get base URL from config or use default
let config_manager = crate::core::config::ConfigManager::new(state.conn.clone());
let base_url = config_manager
.get_config(&Uuid::nil(), "server-url", Some("http://localhost:8080"))
.unwrap_or_else(|| "http://localhost:8080".to_string());
let pixel_url = format!("{}/api/email/tracking/pixel/{}", base_url, tracking_id);
let pixel_html = format!(
r#"<img src="{}" width="1" height="1" style="display:none;visibility:hidden;width:1px;height:1px;border:0;" alt="" />"#,
pixel_url
);
// Insert pixel before closing </body> tag, or at the end if no body tag
if html_body.to_lowercase().contains("</body>") {
html_body.replace("</body>", &format!("{}</body>", pixel_html))
.replace("</BODY>", &format!("{}</BODY>", pixel_html))
} else {
format!("{}{}", html_body, pixel_html)
}
}
/// Save email tracking record to database
fn save_email_tracking_record(
conn: crate::shared::utils::DbPool,
tracking_id: Uuid,
account_id: Uuid,
bot_id: Uuid,
from_email: &str,
to_email: &str,
cc: Option<&str>,
bcc: Option<&str>,
subject: &str,
) -> Result<(), String> {
let mut db_conn = conn.get().map_err(|e| format!("DB connection error: {}", e))?;
let id = Uuid::new_v4();
let now = Utc::now();
diesel::sql_query(
r#"INSERT INTO sent_email_tracking
(id, tracking_id, bot_id, account_id, from_email, to_email, cc, bcc, subject, sent_at, read_count, is_read)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, 0, false)"#
)
.bind::<diesel::sql_types::Uuid, _>(id)
.bind::<diesel::sql_types::Uuid, _>(tracking_id)
.bind::<diesel::sql_types::Uuid, _>(bot_id)
.bind::<diesel::sql_types::Uuid, _>(account_id)
.bind::<diesel::sql_types::Text, _>(from_email)
.bind::<diesel::sql_types::Text, _>(to_email)
.bind::<diesel::sql_types::Nullable<diesel::sql_types::Text>, _>(cc)
.bind::<diesel::sql_types::Nullable<diesel::sql_types::Text>, _>(bcc)
.bind::<diesel::sql_types::Text, _>(subject)
.bind::<diesel::sql_types::Timestamptz, _>(now)
.execute(&mut db_conn)
.map_err(|e| format!("Failed to save tracking record: {}", e))?;
debug!("Saved email tracking record: tracking_id={}", tracking_id);
Ok(())
}
/// Serve tracking pixel and record email open
pub async fn serve_tracking_pixel(
Path(tracking_id): Path<String>,
State(state): State<Arc<AppState>>,
Query(_query): Query<TrackingPixelQuery>,
headers: axum::http::HeaderMap,
) -> impl IntoResponse {
// Extract client info from headers
let client_ip = headers
.get("x-forwarded-for")
.and_then(|v| v.to_str().ok())
.map(|s| s.split(',').next().unwrap_or(s).trim().to_string())
.or_else(|| {
headers
.get("x-real-ip")
.and_then(|v| v.to_str().ok())
.map(|s| s.to_string())
});
let user_agent = headers
.get("user-agent")
.and_then(|v| v.to_str().ok())
.map(|s| s.to_string());
// Parse tracking ID
if let Ok(tracking_uuid) = Uuid::parse_str(&tracking_id) {
let conn = state.conn.clone();
let ip_clone = client_ip.clone();
let ua_clone = user_agent.clone();
// Update tracking record asynchronously
let _ = tokio::task::spawn_blocking(move || {
update_email_read_status(conn, tracking_uuid, ip_clone, ua_clone)
})
.await;
info!("Email read tracked: tracking_id={}, ip={:?}", tracking_id, client_ip);
} else {
warn!("Invalid tracking ID received: {}", tracking_id);
}
// Always return the pixel, regardless of tracking success
// This prevents email clients from showing broken images
(
StatusCode::OK,
[
("content-type", "image/gif"),
("cache-control", "no-store, no-cache, must-revalidate, max-age=0"),
("pragma", "no-cache"),
("expires", "0"),
],
TRACKING_PIXEL.to_vec(),
)
}
/// Update email read status in database
fn update_email_read_status(
conn: crate::shared::utils::DbPool,
tracking_id: Uuid,
client_ip: Option<String>,
user_agent: Option<String>,
) -> Result<(), String> {
let mut db_conn = conn.get().map_err(|e| format!("DB connection error: {}", e))?;
let now = Utc::now();
// Update tracking record - increment read count, set first/last read info
diesel::sql_query(
r#"UPDATE sent_email_tracking
SET
is_read = true,
read_count = read_count + 1,
read_at = COALESCE(read_at, $2),
first_read_ip = COALESCE(first_read_ip, $3),
last_read_ip = $3,
user_agent = COALESCE(user_agent, $4),
updated_at = $2
WHERE tracking_id = $1"#
)
.bind::<diesel::sql_types::Uuid, _>(tracking_id)
.bind::<diesel::sql_types::Timestamptz, _>(now)
.bind::<diesel::sql_types::Nullable<diesel::sql_types::Text>, _>(client_ip.as_deref())
.bind::<diesel::sql_types::Nullable<diesel::sql_types::Text>, _>(user_agent.as_deref())
.execute(&mut db_conn)
.map_err(|e| format!("Failed to update tracking record: {}", e))?;
debug!("Updated email read status: tracking_id={}", tracking_id);
Ok(())
}
/// Get tracking status for a specific email
pub async fn get_tracking_status(
Path(tracking_id): Path<String>,
State(state): State<Arc<AppState>>,
) -> Result<Json<ApiResponse<TrackingStatusResponse>>, EmailError> {
let tracking_uuid = Uuid::parse_str(&tracking_id)
.map_err(|_| EmailError("Invalid tracking ID".to_string()))?;
let conn = state.conn.clone();
let result = tokio::task::spawn_blocking(move || {
get_tracking_record(conn, tracking_uuid)
})
.await
.map_err(|e| EmailError(format!("Task join error: {}", e)))?
.map_err(EmailError)?;
Ok(Json(ApiResponse {
success: true,
data: Some(result),
message: None,
}))
}
/// Get tracking record from database
fn get_tracking_record(
conn: crate::shared::utils::DbPool,
tracking_id: Uuid,
) -> Result<TrackingStatusResponse, String> {
let mut db_conn = conn.get().map_err(|e| format!("DB connection error: {}", e))?;
#[derive(QueryableByName)]
struct TrackingRow {
#[diesel(sql_type = diesel::sql_types::Uuid)]
tracking_id: Uuid,
#[diesel(sql_type = diesel::sql_types::Text)]
to_email: String,
#[diesel(sql_type = diesel::sql_types::Text)]
subject: String,
#[diesel(sql_type = diesel::sql_types::Timestamptz)]
sent_at: DateTime<Utc>,
#[diesel(sql_type = diesel::sql_types::Bool)]
is_read: bool,
#[diesel(sql_type = diesel::sql_types::Nullable<diesel::sql_types::Timestamptz>)]
read_at: Option<DateTime<Utc>>,
#[diesel(sql_type = diesel::sql_types::Integer)]
read_count: i32,
}
let row: TrackingRow = diesel::sql_query(
r#"SELECT tracking_id, to_email, subject, sent_at, is_read, read_at, read_count
FROM sent_email_tracking WHERE tracking_id = $1"#
)
.bind::<diesel::sql_types::Uuid, _>(tracking_id)
.get_result(&mut db_conn)
.map_err(|e| format!("Tracking record not found: {}", e))?;
Ok(TrackingStatusResponse {
tracking_id: row.tracking_id.to_string(),
to_email: row.to_email,
subject: row.subject,
sent_at: row.sent_at.to_rfc3339(),
is_read: row.is_read,
read_at: row.read_at.map(|dt| dt.to_rfc3339()),
read_count: row.read_count,
})
}
/// List sent emails with tracking status
pub async fn list_sent_emails_tracking(
State(state): State<Arc<AppState>>,
Query(query): Query<ListTrackingQuery>,
) -> Result<Json<ApiResponse<Vec<TrackingStatusResponse>>>, EmailError> {
let conn = state.conn.clone();
let result = tokio::task::spawn_blocking(move || {
list_tracking_records(conn, query)
})
.await
.map_err(|e| EmailError(format!("Task join error: {}", e)))?
.map_err(EmailError)?;
Ok(Json(ApiResponse {
success: true,
data: Some(result),
message: None,
}))
}
/// List tracking records from database
fn list_tracking_records(
conn: crate::shared::utils::DbPool,
query: ListTrackingQuery,
) -> Result<Vec<TrackingStatusResponse>, String> {
let mut db_conn = conn.get().map_err(|e| format!("DB connection error: {}", e))?;
let limit = query.limit.unwrap_or(50);
let offset = query.offset.unwrap_or(0);
#[derive(QueryableByName)]
struct TrackingRow {
#[diesel(sql_type = diesel::sql_types::Uuid)]
tracking_id: Uuid,
#[diesel(sql_type = diesel::sql_types::Text)]
to_email: String,
#[diesel(sql_type = diesel::sql_types::Text)]
subject: String,
#[diesel(sql_type = diesel::sql_types::Timestamptz)]
sent_at: DateTime<Utc>,
#[diesel(sql_type = diesel::sql_types::Bool)]
is_read: bool,
#[diesel(sql_type = diesel::sql_types::Nullable<diesel::sql_types::Timestamptz>)]
read_at: Option<DateTime<Utc>>,
#[diesel(sql_type = diesel::sql_types::Integer)]
read_count: i32,
}
// Build query based on filter
let base_query = match query.filter.as_deref() {
Some("read") => {
r#"SELECT tracking_id, to_email, subject, sent_at, is_read, read_at, read_count
FROM sent_email_tracking WHERE is_read = true
ORDER BY sent_at DESC LIMIT $1 OFFSET $2"#
}
Some("unread") => {
r#"SELECT tracking_id, to_email, subject, sent_at, is_read, read_at, read_count
FROM sent_email_tracking WHERE is_read = false
ORDER BY sent_at DESC LIMIT $1 OFFSET $2"#
}
_ => {
r#"SELECT tracking_id, to_email, subject, sent_at, is_read, read_at, read_count
FROM sent_email_tracking
ORDER BY sent_at DESC LIMIT $1 OFFSET $2"#
}
};
let rows: Vec<TrackingRow> = diesel::sql_query(base_query)
.bind::<diesel::sql_types::BigInt, _>(limit)
.bind::<diesel::sql_types::BigInt, _>(offset)
.load(&mut db_conn)
.map_err(|e| format!("Query failed: {}", e))?;
Ok(rows
.into_iter()
.map(|row| TrackingStatusResponse {
tracking_id: row.tracking_id.to_string(),
to_email: row.to_email,
subject: row.subject,
sent_at: row.sent_at.to_rfc3339(),
is_read: row.is_read,
read_at: row.read_at.map(|dt| dt.to_rfc3339()),
read_count: row.read_count,
})
.collect())
}
/// Get tracking statistics
pub async fn get_tracking_stats(
State(state): State<Arc<AppState>>,
) -> Result<Json<ApiResponse<TrackingStatsResponse>>, EmailError> {
let conn = state.conn.clone();
let result = tokio::task::spawn_blocking(move || {
calculate_tracking_stats(conn)
})
.await
.map_err(|e| EmailError(format!("Task join error: {}", e)))?
.map_err(EmailError)?;
Ok(Json(ApiResponse {
success: true,
data: Some(result),
message: None,
}))
}
/// Calculate tracking statistics from database
fn calculate_tracking_stats(
conn: crate::shared::utils::DbPool,
) -> Result<TrackingStatsResponse, String> {
let mut db_conn = conn.get().map_err(|e| format!("DB connection error: {}", e))?;
#[derive(QueryableByName)]
struct StatsRow {
#[diesel(sql_type = diesel::sql_types::BigInt)]
total_sent: i64,
#[diesel(sql_type = diesel::sql_types::BigInt)]
total_read: i64,
#[diesel(sql_type = diesel::sql_types::Nullable<diesel::sql_types::Double>)]
avg_time_hours: Option<f64>,
}
let stats: StatsRow = diesel::sql_query(
r#"SELECT
COUNT(*) as total_sent,
COUNT(*) FILTER (WHERE is_read = true) as total_read,
AVG(EXTRACT(EPOCH FROM (read_at - sent_at)) / 3600) FILTER (WHERE is_read = true) as avg_time_hours
FROM sent_email_tracking"#
)
.get_result(&mut db_conn)
.map_err(|e| format!("Stats query failed: {}", e))?;
let read_rate = if stats.total_sent > 0 {
(stats.total_read as f64 / stats.total_sent as f64) * 100.0
} else {
0.0
};
Ok(TrackingStatsResponse {
total_sent: stats.total_sent,
total_read: stats.total_read,
read_rate,
avg_time_to_read_hours: stats.avg_time_hours,
})
}
pub async fn get_emails(
Path(campaign_id): Path<String>,
State(_state): State<Arc<AppState>>,

14
src/llm/episodic_memory.rs
View file

@ -37,20 +37,17 @@ async fn process_episodic_memory(
for session in sessions {
let config_manager = ConfigManager::new(state.conn.clone());
// Support both old and new config key names for backwards compatibility
let threshold = config_manager
.get_config(&session.bot_id, "episodic-memory-threshold", None)
.or_else(|_| config_manager.get_config(&session.bot_id, "prompt-compact", None))
.unwrap_or_default()
.parse::<i32>()
.unwrap_or(4); // Default to 4 if not configured
.unwrap_or(4);
let history_to_keep = config_manager
.get_config(&session.bot_id, "episodic-memory-history", None)
.or_else(|_| config_manager.get_config(&session.bot_id, "prompt-history", None))
.unwrap_or_default()
.parse::<usize>()
.unwrap_or(2); // Default to 2 if not configured
.unwrap_or(2);
if threshold == 0 {
return Ok(());
@ -75,7 +72,6 @@ async fn process_episodic_memory(
.position(|(role, _)| role == "episodic" || role == "compact")
.map(|pos| history.len() - pos - 1);
// Calculate start index: if there's a summary, start after it; otherwise start from 0
let start_index = last_summary_index.map(|idx| idx + 1).unwrap_or(0);
for (_i, (role, _)) in history.iter().enumerate().skip(start_index) {
@ -112,7 +108,6 @@ async fn process_episodic_memory(
history_to_keep
);
// Determine which messages to summarize and which to keep
let total_messages = history.len() - start_index;
let messages_to_summarize = if total_messages > history_to_keep {
total_messages - history_to_keep
@ -132,7 +127,6 @@ async fn process_episodic_memory(
conversation
.push_str("Please summarize this conversation between user and bot: \n\n [[[***** \n");
// Only summarize messages beyond the history_to_keep threshold
for (role, content) in history.iter().skip(start_index).take(messages_to_summarize) {
if role == "episodic" || role == "compact" {
continue;
@ -170,7 +164,6 @@ async fn process_episodic_memory(
session.id,
summary.len()
);
// Use handler to filter <think> content
let handler = llm_models::get_handler(
config_manager
.get_config(&session.bot_id, "llm-model", None)
@ -187,7 +180,7 @@ async fn process_episodic_memory(
session.id, e
);
trace!("Using fallback summary for session {}", session.id);
format!("EPISODIC MEMORY: {}", filtered) // Fallback
format!("EPISODIC MEMORY: {}", filtered)
}
};
info!(
@ -195,7 +188,6 @@ async fn process_episodic_memory(
session.id, messages_to_summarize, history_to_keep
);
// Save the episodic memory (role 9 = episodic/compact)
{
let mut session_manager = state.session_manager.lock().await;
session_manager.save_message(session.id, session.user_id, 9, &summarized, 1)?;

12
templates/announcements.gbai/annoucements.gbot/config.csv
View file

@ -1,7 +1,7 @@
name,value
prompt-history, 2
prompt-compact, 4
theme-color1, #0d2b55
theme-color2, #fff9c2
theme-logo, https://pragmatismo.com.br/icons/general-bots.svg
theme-title, Announcements General Bots
episodic-memory-history,2
episodic-memory-threshold,4
theme-color1,#0d2b55
theme-color2,#fff9c2
theme-logo,https://pragmatismo.com.br/icons/general-bots.svg
theme-title,Announcements General Bots

1 name value
2 prompt-history episodic-memory-history 2
3 prompt-compact episodic-memory-threshold 4
4 theme-color1 #0d2b55
5 theme-color2 #fff9c2
6 theme-logo https://pragmatismo.com.br/icons/general-bots.svg
7 theme-title Announcements General Bots

4
templates/crm/contacts.gbai/contacts.gbot/config.csv
View file

@ -1,6 +1,6 @@
name,value
prompt-history,2
prompt-compact,4
episodic-memory-history,2
episodic-memory-threshold,4
theme-color1,#1565C0
theme-color2,#E3F2FD
theme-logo,https://pragmatismo.com.br/icons/general-bots.svg

1 name value
2 prompt-history episodic-memory-history 2
3 prompt-compact episodic-memory-threshold 4
4 theme-color1 #1565C0
5 theme-color2 #E3F2FD
6 theme-logo https://pragmatismo.com.br/icons/general-bots.svg

4
templates/crm/sales-pipeline.gbai/sales-pipeline.gbot/config.csv
View file

@ -1,6 +1,6 @@
name,value
prompt-history,2
prompt-compact,4
episodic-memory-history,2
episodic-memory-threshold,4
theme-color1,#2E7D32
theme-color2,#E8F5E9
theme-logo,https://pragmatismo.com.br/icons/general-bots.svg

1 name value
2 prompt-history episodic-memory-history 2
3 prompt-compact episodic-memory-threshold 4
4 theme-color1 #2E7D32
5 theme-color2 #E8F5E9
6 theme-logo https://pragmatismo.com.br/icons/general-bots.svg

2
templates/default.gbai/default.gbot/config.csv
View file

@ -13,7 +13,7 @@ llm-cache-ttl,3600
llm-cache-semantic,true
llm-cache-threshold,0.95
,
prompt-compact,4
episodic-memory-threshold,4
,
mcp-server,false
,

1 name value
13 llm-cache-semantic true
14 llm-cache-threshold 0.95
15
16 prompt-compact episodic-memory-threshold 4
17
18 mcp-server false
19

4
templates/hr/employees.gbai/employees.gbot/config.csv
View file

@ -1,6 +1,6 @@
name,value
prompt-history,2
prompt-compact,4
episodic-memory-history,2
episodic-memory-threshold,4
theme-color1,#2E7D32
theme-color2,#E8F5E9
theme-logo,https://pragmatismo.com.br/icons/general-bots.svg

1 name value
2 prompt-history episodic-memory-history 2
3 prompt-compact episodic-memory-threshold 4
4 theme-color1 #2E7D32
5 theme-color2 #E8F5E9
6 theme-logo https://pragmatismo.com.br/icons/general-bots.svg

4
templates/it/helpdesk.gbai/helpdesk.gbot/config.csv
View file

@ -1,6 +1,6 @@
name,value
prompt-history,2
prompt-compact,4
episodic-memory-history,2
episodic-memory-threshold,4
theme-color1,#1565C0
theme-color2,#E3F2FD
theme-logo,https://pragmatismo.com.br/icons/general-bots.svg

1 name value
2 prompt-history episodic-memory-history 2
3 prompt-compact episodic-memory-threshold 4
4 theme-color1 #1565C0
5 theme-color2 #E3F2FD
6 theme-logo https://pragmatismo.com.br/icons/general-bots.svg