Update submodules: app generator, logging, knowledge base, web search, designer magic
This commit is contained in:
parent
84d87f178b
commit
ceb57e820a
8 changed files with 248 additions and 7 deletions
241
PENDING_TASKS.md
Normal file
241
PENDING_TASKS.md
Normal file
|
|
@ -0,0 +1,241 @@
|
|||
# General Bots - Pending Tasks for Next Sessions
|
||||
|
||||
**Created:** Session cleanup
|
||||
**Purpose:** Consolidated list of pending work for LLM continuation
|
||||
|
||||
---
|
||||
|
||||
## ✅ COMPLETED THIS SESSION
|
||||
|
||||
### 1. Sources Module - Knowledge Base Backend ✅
|
||||
**Location:** `botserver/src/sources/knowledge_base.rs`
|
||||
|
||||
**Implemented:**
|
||||
- `POST /api/sources/kb/upload` - Upload documents for ingestion
|
||||
- `GET /api/sources/kb/list` - List ingested sources
|
||||
- `POST /api/sources/kb/query` - Query knowledge base with full-text search
|
||||
- `GET /api/sources/kb/:id` - Get source details
|
||||
- `DELETE /api/sources/kb/:id` - Remove source
|
||||
- `POST /api/sources/kb/reindex` - Re-process sources
|
||||
- `GET /api/sources/kb/stats` - Get knowledge base statistics
|
||||
|
||||
**Features:**
|
||||
- Document chunking with configurable size/overlap
|
||||
- Text extraction for PDF, DOCX, TXT, Markdown, HTML, CSV, XLSX
|
||||
- Full-text search with PostgreSQL ts_rank
|
||||
- Status tracking (pending, processing, indexed, failed, reindexing)
|
||||
|
||||
---
|
||||
|
||||
### 2. Research Module - Web Search Backend ✅
|
||||
**Location:** `botserver/src/research/web_search.rs`
|
||||
|
||||
**Implemented:**
|
||||
- `POST /api/research/web/search` - Web search via DuckDuckGo
|
||||
- `POST /api/research/web/summarize` - Summarize search results
|
||||
- `POST /api/research/web/deep` - Deep research with multiple queries
|
||||
- `GET /api/research/web/history` - Search history
|
||||
- `GET /api/research/web/instant` - Instant answers from DuckDuckGo API
|
||||
|
||||
**Features:**
|
||||
- DuckDuckGo HTML scraping (no API key required)
|
||||
- Result parsing with favicon extraction
|
||||
- Related query generation
|
||||
- Citation tracking
|
||||
|
||||
---
|
||||
|
||||
### 3. App Generator - Full LLM-Based Generation ✅
|
||||
**Location:** `botserver/src/auto_task/app_generator.rs`
|
||||
|
||||
**Completely rewritten to:**
|
||||
- Generate ALL files (HTML, CSS, JS, BAS) via LLM
|
||||
- Removed ALL hardcoded templates
|
||||
- Single LLM call generates complete app structure
|
||||
- Includes tables, pages, tools, schedulers
|
||||
|
||||
---
|
||||
|
||||
### 4. App Logging System ✅
|
||||
**Location:** `botserver/src/auto_task/app_logs.rs`
|
||||
|
||||
**Implemented:**
|
||||
- Server-side log storage per app
|
||||
- Client-side JavaScript logger (`/api/app-logs/logger.js`)
|
||||
- Error context injection into Designer prompts
|
||||
- Auto-cleanup scheduler (D-1 retention)
|
||||
|
||||
**Endpoints:**
|
||||
- `POST /api/app-logs/client` - Receive client logs
|
||||
- `GET /api/app-logs/list` - List logs with filters
|
||||
- `GET /api/app-logs/stats` - Log statistics
|
||||
- `POST /api/app-logs/clear/{app_name}` - Clear app logs
|
||||
- `GET /api/app-logs/logger.js` - Client logger script
|
||||
|
||||
---
|
||||
|
||||
### 5. Database Migration ✅
|
||||
**Location:** `botserver/migrations/6.1.3_knowledge_base_sources/`
|
||||
|
||||
**Created tables:**
|
||||
- `knowledge_sources` - Uploaded documents metadata
|
||||
- `knowledge_chunks` - Text chunks for RAG
|
||||
- `research_search_history` - Search history tracking
|
||||
|
||||
---
|
||||
|
||||
## 🔴 HIGH PRIORITY
|
||||
|
||||
### 1. Calendar UI Completion
|
||||
|
||||
**Location:** `botui/ui/suite/calendar/`
|
||||
**Backend exists:** `botserver/src/calendar/` (fully implemented with CalDAV)
|
||||
|
||||
**What's missing:**
|
||||
- Week view
|
||||
- Day view
|
||||
- Drag-and-drop event moving
|
||||
- Recurring events UI
|
||||
- Calendar sharing UI
|
||||
|
||||
**Backend is complete** - just needs frontend polish.
|
||||
|
||||
---
|
||||
|
||||
### 2. Vector Embeddings Integration
|
||||
|
||||
**Location:** `botserver/src/sources/knowledge_base.rs`
|
||||
|
||||
**What's needed:**
|
||||
- Connect to LLM for embedding generation
|
||||
- Store embeddings in PostgreSQL pgvector
|
||||
- Implement semantic search alongside full-text search
|
||||
- Integrate with existing `drive/vectordb.rs`
|
||||
|
||||
---
|
||||
|
||||
## 🟡 MEDIUM PRIORITY
|
||||
|
||||
### 3. Meet Module - LiveKit Integration
|
||||
|
||||
**Location:** `botserver/src/meet/`
|
||||
**UI exists:** `botui/ui/suite/meet/`
|
||||
|
||||
**What's missing:**
|
||||
- LiveKit server configuration documentation
|
||||
- Room creation and management
|
||||
- Participant tracking
|
||||
- Recording integration
|
||||
|
||||
**Requires external setup:**
|
||||
- LiveKit server (self-hosted or cloud)
|
||||
- TURN/STUN servers for WebRTC
|
||||
|
||||
---
|
||||
|
||||
### 4. Custom Domain - Config.csv Integration
|
||||
|
||||
**Location:** `botserver/src/core/dns/`
|
||||
|
||||
**Current state:** DNS routes exist but config.csv parsing not connected
|
||||
|
||||
**What's needed:**
|
||||
```csv
|
||||
# In bot's config.csv
|
||||
appname-domain,app.customerdomain.com
|
||||
```
|
||||
|
||||
- Parse `appname-domain` from config.csv during bot load
|
||||
- Register with DNS service automatically
|
||||
- Auto-provision SSL via Let's Encrypt
|
||||
|
||||
---
|
||||
|
||||
### 5. Designer Magic Button - LLM Integration
|
||||
|
||||
**Location:**
|
||||
- `botui/ui/suite/designer.html` (dialog designer - DONE)
|
||||
- `botui/ui/suite/editor.html` (code editor - DONE)
|
||||
- `botserver/src/designer/mod.rs` (endpoints - DONE)
|
||||
|
||||
**What's missing:**
|
||||
- Connect `/api/v1/editor/magic` to actual LLM when `feature = "llm"` is enabled
|
||||
- Currently uses fallback suggestions only
|
||||
- Need to test with LLM enabled
|
||||
|
||||
---
|
||||
|
||||
## 🟢 LOW PRIORITY / POLISH
|
||||
|
||||
### 6. SEO Meta Tags Verification
|
||||
|
||||
Verify all HTMX pages have proper SEO:
|
||||
- `botui/ui/suite/**/*.html`
|
||||
- Generated apps from `app_generator.rs`
|
||||
|
||||
Required tags:
|
||||
```html
|
||||
<meta name="description" content="...">
|
||||
<meta name="robots" content="noindex, nofollow">
|
||||
<meta property="og:title" content="...">
|
||||
<meta property="og:description" content="...">
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 7. Login Flow Documentation
|
||||
|
||||
**Credentials shown during setup:**
|
||||
- Displayed in terminal with box formatting
|
||||
- Pauses for user to copy
|
||||
- NOT saved to file (security)
|
||||
|
||||
**Location of display:** `botserver/src/core/package_manager/setup/directory_setup.rs`
|
||||
|
||||
Consider adding:
|
||||
- First-login wizard to change password
|
||||
- Email verification flow
|
||||
- Password recovery
|
||||
|
||||
---
|
||||
|
||||
## 📋 Session Continuation Notes
|
||||
|
||||
### Files Modified This Session:
|
||||
- `botserver/src/auto_task/app_generator.rs` - Complete rewrite for LLM-only generation
|
||||
- `botserver/src/auto_task/app_logs.rs` - NEW: App logging system
|
||||
- `botserver/src/auto_task/mod.rs` - Added app_logs exports and routes
|
||||
- `botserver/src/sources/mod.rs` - Added knowledge_base module
|
||||
- `botserver/src/sources/knowledge_base.rs` - NEW: KB ingestion backend
|
||||
- `botserver/src/research/mod.rs` - Added web_search module
|
||||
- `botserver/src/research/web_search.rs` - NEW: Web search backend
|
||||
- `botserver/src/designer/mod.rs` - Added error context to prompts
|
||||
- `botserver/migrations/6.1.3_knowledge_base_sources/` - NEW: DB migration
|
||||
|
||||
### Build Status:
|
||||
- `cargo check -p botserver` - ✅ 0 errors, 0 warnings
|
||||
|
||||
### How to Continue:
|
||||
1. Pick a HIGH PRIORITY task
|
||||
2. Read the relevant source files
|
||||
3. Implement missing functionality
|
||||
4. Test with `cargo check`
|
||||
5. Update this file when complete
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Quick Start for Next Session
|
||||
|
||||
```
|
||||
Start with:
|
||||
1. "Complete Calendar UI - add week/day views"
|
||||
OR
|
||||
2. "Add vector embeddings to knowledge base"
|
||||
OR
|
||||
3. "Test app generator with LLM enabled"
|
||||
|
||||
Context files to read first:
|
||||
- botui/ui/suite/calendar/
|
||||
- botserver/src/sources/knowledge_base.rs
|
||||
- botserver/src/auto_task/app_generator.rs
|
||||
```
|
||||
2
botapp
2
botapp
|
|
@ -1 +1 @@
|
|||
Subproject commit bc875b511f7a9be11b8cb49d8be4f33e2998f015
|
||||
Subproject commit 5a82552a64982ef391605c37c09de805f2d97ae0
|
||||
2
botbook
2
botbook
|
|
@ -1 +1 @@
|
|||
Subproject commit 2ec504448d13f1cb3fed486be9c6a99387258889
|
||||
Subproject commit d5d147ce0ce54e9d6519440b87bb4dea5ddcfecc
|
||||
|
|
@ -1 +1 @@
|
|||
Subproject commit f2afa7e7bebac98272940715f499c50381b30d4f
|
||||
Subproject commit a3add8ada34a0313bac4285d20441bb643f74962
|
||||
2
botlib
2
botlib
|
|
@ -1 +1 @@
|
|||
Subproject commit 94f333f983aa7f4fcfc541b9d6f52f2261d3095a
|
||||
Subproject commit a50d229346167c539d3fbae2dcf26160df311e41
|
||||
|
|
@ -1 +1 @@
|
|||
Subproject commit 9920538708da467c925953df03960e9c991a3004
|
||||
Subproject commit 643c101d7dad4242c76cf16fac93d6af690d9d5f
|
||||
2
bottest
2
bottest
|
|
@ -1 +1 @@
|
|||
Subproject commit b38574c5885d356b026f5060af51ad02d2fcef21
|
||||
Subproject commit 1232b2fc6540595fe683115bb3499ccddf1d7270
|
||||
2
botui
2
botui
|
|
@ -1 +1 @@
|
|||
Subproject commit d96c546c6afadb572c462406668f8f89729fe247
|
||||
Subproject commit 3f95c4645d69b68186849e5aac68c433e93bd545
|
||||
Loading…
Add table
Reference in a new issue