GeneralBots/botserver
2
1
Fork 0
Code Issues Pull requests Projects Releases 323 Packages Wiki Activity Actions

Update PROMPT.md and feature gating

Browse source
This commit is contained in:
Rodrigo Rodriguez (Pragmatismo) 2026-01-22 20:24:05 -03:00
parent 66abce913f
commit f8a907bd81
8 changed files with 338 additions and 200 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

77
PROMPT.md
View file

@ -1,6 +1,7 @@
# botserver Development Prompt Guide
# botserver Development Guide
**Version:** 6.1.0
**Version:** 6.2.0
**Purpose:** Main API server for General Bots (Axum + Diesel + Rhai BASIC + HTMX in botui)
---
@ -10,7 +11,7 @@
---
## ABSOLUTE PROHIBITIONS
## ❌ ABSOLUTE PROHIBITIONS
```
❌ NEVER use #![allow()] or #[allow()] in source code
@ -29,7 +30,7 @@
---
## SECURITY REQUIREMENTS
## 🔐 SECURITY REQUIREMENTS
### Error Handling
@ -113,7 +114,7 @@ Command::new("/usr/bin/tool").arg(safe).output()?;
---
## CODE PATTERNS
## ✅ CODE PATTERNS
### Format Strings - Inline Variables
@ -172,23 +173,39 @@ date.with_hour(9).and_then(|d| d.with_minute(0)).unwrap_or(date)
---
## DATABASE STANDARDS
## 📁 KEY DIRECTORIES
- TABLES AND INDEXES ONLY (no views, triggers, functions)
- JSON columns: use TEXT with `_json` suffix
- Use diesel - no sqlx
```
src/
├── core/ # Bootstrap, config, routes
├── basic/ # Rhai BASIC interpreter
│ └── keywords/ # BASIC keyword implementations
├── security/ # Security modules
├── shared/ # Shared types, models
├── tasks/ # AutoTask system
└── auto_task/ # App generator
```
---
## FRONTEND RULES
## 🗄️ DATABASE STANDARDS
- Use HTMX - minimize JavaScript
- NO external CDN - all assets local
- Server-side rendering with Askama templates
- **TABLES AND INDEXES ONLY** (no views, triggers, functions)
- **JSON columns:** use TEXT with `_json` suffix
- **ORM:** Use diesel - no sqlx
- **Migrations:** Located in `botserver/migrations/`
---
## DEPENDENCIES
## 🎨 FRONTEND RULES
- **Use HTMX** - minimize JavaScript
- **NO external CDN** - all assets local
- **Server-side rendering** with Askama templates
---
## 📦 KEY DEPENDENCIES
| Library | Version | Purpose |
|---------|---------|---------|
@ -202,35 +219,7 @@ date.with_hour(9).and_then(|d| d.with_minute(0)).unwrap_or(date)
---
## COMPILATION POLICY - CRITICAL
**NEVER compile during development. NEVER run `cargo build`. Use static analysis only.**
### Workflow
1. Make all code changes
2. Use `diagnostics` tool for static analysis (NOT compilation)
3. Fix any errors found by diagnostics
4. **At the end**, inform user what needs restart
### After All Changes Complete
| Change Type | User Action Required |
|-------------|----------------------|
| Rust code (`.rs` files) | "Recompile and restart **botserver**" |
| HTML templates (`.html` in botui) | "Browser refresh only" |
| CSS/JS files | "Browser refresh only" |
| Askama templates (`.html` in botserver) | "Recompile and restart **botserver**" |
| Database migrations | "Run migration, then restart **botserver**" |
| Cargo.toml changes | "Recompile and restart **botserver**" |
**Format:** At the end of your response, always state:
- ✅ **No restart needed** - browser refresh only
- 🔄 **Restart botserver** - recompile required
---
## KEY REMINDERS
## 🔑 REMEMBER
- **ZERO WARNINGS** - fix every clippy warning
- **ZERO COMMENTS** - no comments, no doc comments
@ -239,6 +228,6 @@ date.with_hour(9).and_then(|d| d.with_minute(0)).unwrap_or(date)
- **NO UNWRAP/EXPECT** - use ? or combinators
- **PARAMETERIZED SQL** - never format! for queries
- **VALIDATE COMMANDS** - never pass raw user input
- **USE DIAGNOSTICS** - never call cargo clippy directly
- **INLINE FORMAT ARGS** - `format!("{name}")` not `format!("{}", name)`
- **USE SELF** - in impl blocks, use Self not type name
- **Version 6.2.0** - do not change without approval

152
TASKS.md
View file

@ -1,152 +0,0 @@
# Deep Feature Gating Plan: Schema & Migrations
## Objective
Reduce binary size and runtime overhead by strictly feature-gating Database Schema (`schema.rs`) and Migrations.
The goal is that a "minimal" build (chat-only) should:
1. NOT compile code for `tasks`, `mail`, `calendar`, etc. tables.
2. NOT run database migrations for those features (preventing unused tables in DB).
3. NOT fail compilation due to missing table definitions in `joinable!` macros.
---
## 🚀 Phase 1: Split Diesel Schema (The Hard Part)
The current `schema.rs` is monolithic (3000+ lines).
Diesel requires `joinable!` and `allow_tables_to_appear_in_same_query!` macros to know about relationship graphs.
### **1.1 Analysis & grouping**
Identify which tables belong to which feature and strictly define their relationships.
- **Core**: `users`, `bots`, `sessions`, `organizations`, `rbac_*`
- **Tasks**: `tasks`, `task_comments` (links to `users`, `projects`)
- **Mail**: `email_*` (links to `users`)
- **Drive**: (Uses S3, but maybe metadata tables?)
- **People/CRM**: `crm_*` (links to `users`, `organizations`)
### **1.2 Create Modular Schema Files**
Instead of one file, we will separate them and use `mod.rs` to aggregate.
- `src/core/shared/schema/mod.rs` (Aggregator)
- `src/core/shared/schema/core.rs` (Base tables)
- `src/core/shared/schema/tasks.rs` (Gated `#[cfg(feature="tasks")]`)
- `src/core/shared/schema/mail.rs` (Gated `#[cfg(feature="mail")]`)
...etc.
### **1.3 Solve the `allow_tables_to_appear_in_same_query!` Problem**
The single massive macro prevents gating.
**Solution**: Break the "Universe" assumption.
1. Define a `core_tables!` macro.
2. Define `task_tables!` macro (gated).
3. We might lose the ability to write arbitrary joins between *any* two tables if they are in different clusters, unless we explicitly define `joinable!` in the aggregator.
4. **Strategy**:
- Keep `joinable!` definitions relative to the feature.
- If `TableA` (Core) joins `TableB` (Tasks), the `joinable!` must be gated by `tasks`.
- The `allow_tables...` output must be constructed dynamically or strict subsets defined.
### **1.4 Comprehensive Feature Mapping**
Based on `Cargo.toml`, we need to map all optional features to their schema requirements.
| Feature | Tables (Prefix/Name) | Migrations |
|---------|-----------------------|------------|
| (core) | `users`, `bots`, `sessions`, `orgs`, `rbac_*`, `system_automations`, `bot_memories`, `kb_*` | `migrations/core/` |
| `tasks` | `tasks`, `task_comments` | `migrations/tasks/` |
| `calendar`| `calendars`, `calendar_events`, `calendar_*` | `migrations/calendar/` |
| `mail` | `email_*`, `distribution_lists`, `global_email_signatures` | `migrations/mail/` |
| `people` | `crm_contacts`, `crm_accounts`, `crm_leads`, `crm_opportunities`, `crm_activities`, `crm_pipeline_*`, `people_*` | `migrations/people/` |
| `compliance`| `compliance_*`, `legal_*`, `cookie_consents`, `data_*` | `migrations/compliance/` |
| `meet` | `meet_*`, `meeting_*`, `whiteboard_*` | `migrations/meet/` |
| `attendant`| `attendant_*` | `migrations/attendant/` |
| `analytics`| `dashboards`, `dashboard_*` | `migrations/analytics/` |
| `drive` | (No direct tables? Check `files` or `drive_*`?) | `migrations/drive/` |
| `billing` | `billing_*`, `products`, `services`, `price_*`, `inventory_*` (See note) | `migrations/billing/` |
| `social` | `social_*` | `migrations/social/` |
| `canvas` | `canvases`, `canvas_*` | `migrations/canvas/` |
| `workspaces`| `workspaces`, `workspace_*` | `migrations/workspaces/` |
| `research` | `research_*` | `migrations/research/` |
| `goals` | `okr_*` | `migrations/goals/` |
| `feedback` | `support_tickets`, `ticket_*` | `migrations/tickets/` |
**Note**: `billing` and `tickets` are currently ungated modules in `main.rs`. We should create features or group them. For now, assume they are part of admin/core or need gating.
---
## 🚀 Phase 2: Feature-Gated Migrations
Currently, `embed_migrations!("migrations")` embeds everything.
### **2.1 Split Migration Directories**
Refactor the flat `migrations/` folder into feature-specific directories (requires custom runner logic or Diesel configuration trickery).
Alternately (easier code-wise):
- Keep flat structure but **edit the migration SQL files** to be conditional? No, SQL doesn't support `#[cfg]`.
- **Better Approach**: Use multiple `embed_migrations!` calls.
- `migrations/core/` -> Always run.
- `migrations/tasks/` -> Run if `tasks` feature on.
- `migrations/mail/` -> Run if `mail` feature on.
**Action Plan**:
1. Organize `migrations/` into subdirectories: `core/`, `tasks/`, `mail/`, `people/`.
2. Update `utils.rs` migration logic:
```rust
pub fn run_migrations(conn: &mut PgConnection) {
// Core
const CORE: EmbeddedMigrations = embed_migrations!("migrations/core");
conn.run_pending_migrations(CORE).unwrap();
#[cfg(feature = "tasks")] {
const TASKS: EmbeddedMigrations = embed_migrations!("migrations/tasks");
conn.run_pending_migrations(TASKS).unwrap();
}
// ...
}
```
### **2.2 Verify Migration Dependencies**
Ensure that `tasks` migrations (which might foreign-key to `users`) only run *after* `core` migrations are applied.
---
## 🚀 Phase 3: Fix Dependent Code (Strict Gating)
Once tables are gated, any code referencing `schema::tasks` will fail to compile if the feature is off.
(We have already done most of this in Models/Logic, but Schema was the safety net).
### **3.1 Verify Models imports**
Ensure `models/tasks.rs` uses `crate::schema::tasks` inside the same cfg gate.
### **3.2 Fix Cross-Feature Joins**
If Core code joins `users` with `tasks` (e.g. "get all user tasks"), that code chunk MUST be gated.
---
## Execution Checklist
### Schema
- [x] Create `src/core/shared/schema/` directory
- [x] Move core tables to `schema/core.rs`
- [x] Move task tables to `schema/tasks.rs` (gated)
- [x] Move mail tables to `schema/mail.rs` (gated)
- [x] Move people tables to `schema/people.rs` (gated)
- [x] Move tickets tables to `schema/tickets.rs` (gated)
- [x] Move billing tables to `schema/billing.rs` (gated)
- [x] Move attendant tables to `schema/attendant.rs` (gated)
- [x] Move calendar tables to `schema/calendar.rs` (gated)
- [x] Move goals tables to `schema/goals.rs` (gated)
- [x] Move canvas tables to `schema/canvas.rs` (gated)
- [x] Move workspaces tables to `schema/workspaces.rs` (gated)
- [x] Move social tables to `schema/social.rs` (gated)
- [x] Move analytics tables to `schema/analytics.rs` (gated)
- [x] Move compliance tables to `schema/compliance.rs` (gated)
- [x] Move meet tables to `schema/meet.rs` (gated)
- [x] Move research tables to `schema/research.rs` (gated)
- [x] Refactor `schema/mod.rs` to export modules conditionally
- [x] Split `joinable!` declarations into relevant files
- [x] Refactor `allow_tables_to_appear_in_same_query!` (Skipped/Implicit)
### Migrations
- [x] Audit `migrations/` folder
- [x] Create folder structure `migrations/{core,tasks,mail,people...}`
- [x] Move SQL files to appropriate folders
- [x] Update `run_migrations` in `utils.rs` to run feature-specific migration sets
### Validation
- [ ] `cargo check --no-default-features` (Run ONLY after all migration splits are verified)
- [ ] `cargo check --features tasks`
- [ ] `cargo check --all-features`

82
src/core/features.rs Normal file
View file

@ -0,0 +1,82 @@
/// List of features compiled into this binary
pub const COMPILED_FEATURES: &[&str] = &[
#[cfg(feature = "chat")]
"chat",
#[cfg(feature = "mail")]
"mail",
#[cfg(feature = "email")]
"email", // Alias for mail
#[cfg(feature = "calendar")]
"calendar",
#[cfg(feature = "drive")]
"drive",
#[cfg(feature = "tasks")]
"tasks",
#[cfg(feature = "docs")]
"docs",
#[cfg(feature = "paper")]
"paper",
#[cfg(feature = "sheet")]
"sheet",
#[cfg(feature = "slides")]
"slides",
#[cfg(feature = "meet")]
"meet",
#[cfg(feature = "research")]
"research",
#[cfg(feature = "people")]
"people",
#[cfg(feature = "social")]
"social",
#[cfg(feature = "analytics")]
"analytics",
#[cfg(feature = "monitoring")]
"monitoring",
#[cfg(feature = "admin")]
"admin",
#[cfg(feature = "automation")]
"automation",
#[cfg(feature = "cache")]
"cache",
#[cfg(feature = "directory")]
"directory",
// Add other app features as they are defined in Cargo.toml
#[cfg(feature = "project")]
"project",
#[cfg(feature = "goals")]
"goals",
#[cfg(feature = "workspace")]
"workspace",
#[cfg(feature = "tickets")]
"tickets",
#[cfg(feature = "billing")]
"billing",
#[cfg(feature = "products")]
"products",
#[cfg(feature = "video")]
"video",
#[cfg(feature = "player")]
"player",
#[cfg(feature = "canvas")]
"canvas",
#[cfg(feature = "learn")]
"learn",
#[cfg(feature = "sources")]
"sources",
#[cfg(feature = "dashboards")]
"dashboards",
#[cfg(feature = "designer")]
"designer",
#[cfg(feature = "editor")]
"editor",
#[cfg(feature = "attendant")]
"attendant",
#[cfg(feature = "tools")]
"tools",
];
/// Check if a feature is compiled into the binary
pub fn is_feature_compiled(name: &str) -> bool {
COMPILED_FEATURES.contains(&name)
}

167
src/core/manifest.rs Normal file
View file

@ -0,0 +1,167 @@
use serde::{Deserialize, Serialize};
use crate::core::features::COMPILED_FEATURES;
use crate::core::product::PRODUCT_CONFIG;
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct WorkspaceManifest {
pub version: String,
pub features: FeatureManifest,
pub apps: Vec<AppInfo>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct FeatureManifest {
pub compiled: Vec<String>,
pub enabled: Vec<String>,
pub disabled: Vec<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AppInfo {
pub name: String,
pub enabled: bool,
pub compiled: bool,
pub category: String,
}
impl WorkspaceManifest {
pub fn new() -> Self {
let compiled: Vec<String> = COMPILED_FEATURES.iter().map(|s| s.to_string()).collect();
let enabled: Vec<String> = PRODUCT_CONFIG
.read()
.ok()
.as_ref()
.map(|c| c.get_enabled_apps())
.unwrap_or_default()
.into_iter()
.filter(|app| compiled.contains(app))
.collect();
let disabled: Vec<String> = compiled
.iter()
.filter(|app| !enabled.contains(app))
.cloned()
.collect();
let apps = Self::build_app_info(&compiled, &enabled);
Self {
version: env!("CARGO_PKG_VERSION").to_string(),
features: FeatureManifest {
compiled,
enabled,
disabled,
},
apps,
}
}
fn build_app_info(compiled: &[String], enabled: &[String]) -> Vec<AppInfo> {
let mut apps = Vec::new();
// Communication apps
for app in ["chat", "mail", "calendar", "meet", "people", "social"] {
if compiled.contains(&app.to_string()) {
apps.push(AppInfo {
name: app.to_string(),
enabled: enabled.contains(&app.to_string()),
compiled: true,
category: "communication".to_string(),
});
}
}
// Productivity apps
for app in ["drive", "tasks", "docs", "paper", "sheet", "slides"] {
if compiled.contains(&app.to_string()) {
apps.push(AppInfo {
name: app.to_string(),
enabled: enabled.contains(&app.to_string()),
compiled: true,
category: "productivity".to_string(),
});
}
}
// Project management
for app in ["project", "goals", "workspace", "tickets"] {
if compiled.contains(&app.to_string()) {
apps.push(AppInfo {
name: app.to_string(),
enabled: enabled.contains(&app.to_string()),
compiled: true,
category: "project_management".to_string(),
});
}
}
// Business apps
for app in ["billing", "products", "analytics"] {
if compiled.contains(&app.to_string()) {
apps.push(AppInfo {
name: app.to_string(),
enabled: enabled.contains(&app.to_string()),
compiled: true,
category: "business".to_string(),
});
}
}
// Media apps
for app in ["video", "player"] {
if compiled.contains(&app.to_string()) {
apps.push(AppInfo {
name: app.to_string(),
enabled: enabled.contains(&app.to_string()),
compiled: true,
category: "media".to_string(),
});
}
}
// Learning apps
for app in ["canvas", "learn", "research"] {
if compiled.contains(&app.to_string()) {
apps.push(AppInfo {
name: app.to_string(),
enabled: enabled.contains(&app.to_string()),
compiled: true,
category: "learning".to_string(),
});
}
}
// Developer apps
for app in ["sources", "dashboards", "designer", "editor", "tools"] {
if compiled.contains(&app.to_string()) {
apps.push(AppInfo {
name: app.to_string(),
enabled: enabled.contains(&app.to_string()),
compiled: true,
category: "developer".to_string(),
});
}
}
// System apps
for app in ["automation", "cache", "directory", "admin", "monitoring", "attendant"] {
if compiled.contains(&app.to_string()) {
apps.push(AppInfo {
name: app.to_string(),
enabled: enabled.contains(&app.to_string()),
compiled: true,
category: "system".to_string(),
});
}
}
apps
}
}
impl Default for WorkspaceManifest {
fn default() -> Self {
Self::new()
}
}

2
src/core/mod.rs
View file

@ -6,9 +6,11 @@ pub mod bot_database;
pub mod config;
pub mod directory;
pub mod dns;
pub mod features;
pub mod i18n;
pub mod kb;
pub mod large_org_optimizer;
pub mod manifest;
pub mod middleware;
pub mod oauth;
pub mod organization;

47
src/core/product.rs
View file

@ -293,12 +293,27 @@ pub fn replace_branding(text: &str) -> String {
/// Helper function to get product config for serialization
pub fn get_product_config_json() -> serde_json::Value {
// Get compiled features from our new module
let compiled = crate::core::features::COMPILED_FEATURES;
// Get current config
let config = PRODUCT_CONFIG.read().ok();
// Determine effective apps (intersection of enabled + compiled)
let effective_apps: Vec<String> = config
.as_ref()
.map(|c| c.get_enabled_apps())
.unwrap_or_default()
.into_iter()
.filter(|app| compiled.contains(&app.as_str()) || app == "settings" || app == "auth") // Always allow settings/auth
.collect();
match config {
Some(c) => serde_json::json!({
"name": c.name,
"apps": c.get_enabled_apps(),
"apps": effective_apps,
"compiled_features": compiled,
"version": env!("CARGO_PKG_VERSION"),
"theme": c.theme,
"logo": c.logo,
"favicon": c.favicon,
@ -308,12 +323,21 @@ pub fn get_product_config_json() -> serde_json::Value {
}),
None => serde_json::json!({
"name": "General Bots",
"apps": [],
"apps": compiled, // If no config, show all compiled
"compiled_features": compiled,
"version": env!("CARGO_PKG_VERSION"),
"theme": "sentient",
})
}
}
/// Get workspace manifest with detailed feature information
pub fn get_workspace_manifest() -> serde_json::Value {
let manifest = crate::core::manifest::WorkspaceManifest::new();
serde_json::to_value(manifest).unwrap_or_else(|_| serde_json::json!({}))
}
/// Middleware to check if an app is enabled before allowing API access
pub async fn app_gate_middleware(
req: axum::http::Request<axum::body::Body>,
@ -361,6 +385,25 @@ pub async fn app_gate_middleware(
// Check if the app is enabled
if let Some(app) = app_name {
// First check: is it even compiled?
// Note: settings, auth, admin are core features usually, but we check anyway if they are in features list
// Some core apps like settings might not be in feature flags explicitly or always enabled.
// For simplicity, if it's not in compiled features but is a known core route, we might allow it,
// but here we enforce strict feature containment.
// Exception: 'settings' and 'auth' are often core.
if app != "settings" && app != "auth" && !crate::core::features::is_feature_compiled(app) {
let error_response = serde_json::json!({
"error": "not_implemented",
"message": format!("The '{}' feature is not compiled in this build", app),
"code": 501
});
return (
StatusCode::NOT_IMPLEMENTED,
axum::Json(error_response)
).into_response();
}
if !is_app_enabled(app) {
let error_response = serde_json::json!({
"error": "app_disabled",

7
src/main.rs
View file

@ -346,6 +346,7 @@ async fn run_axum_server(
.add_anonymous_path("/healthz")
.add_anonymous_path("/api/health")
.add_anonymous_path("/api/product")
.add_anonymous_path("/api/manifest")
.add_anonymous_path("/api/i18n")
.add_anonymous_path("/api/auth/login")
.add_anonymous_path("/api/auth/refresh")
@ -425,10 +426,16 @@ async fn run_axum_server(
Json(get_product_config_json())
}
async fn get_workspace_manifest() -> Json<serde_json::Value> {
use crate::core::product::get_workspace_manifest;
Json(get_workspace_manifest())
}
let mut api_router = Router::new()
.route("/health", get(health_check_simple))
.route(ApiUrls::HEALTH, get(health_check))
.route("/api/product", get(get_product_config))
.route("/api/manifest", get(get_workspace_manifest))
.route(ApiUrls::SESSIONS, post(create_session))
.route(ApiUrls::SESSIONS, get(get_sessions))
.route(ApiUrls::SESSION_HISTORY, get(get_session_history))

2
src/tasks/PROMPT.md
View file

@ -1,6 +1,6 @@
# AutoTask LLM Executor - Prompt Guide
**Version:** 6.1.0
**Version:** 6.2.0
**Purpose:** Guide LLM to generate and execute automated tasks using BASIC scripts
---