diff --git a/docs/src/README.md b/docs/src/README.md new file mode 100644 index 00000000..a2a3de38 --- /dev/null +++ b/docs/src/README.md @@ -0,0 +1,29 @@ +# GeneralBots Documentation + +Welcome to the **GeneralBots** documentation for BASIC enthusiasts. +This guide explains how to run, extend, and interact with the GeneralBots system using BASIC-style commands and Rust-powered automation. + +--- + +## Table of Contents +- [Run and Talk](chapter-01/README.md) +- [About Packages](chapter-02/README.md) +- [gbkb Reference](chapter-03/README.md) +- [gbtheme Reference](chapter-04/README.md) +- [gbdialog Reference](chapter-05/README.md) +- [gbapp Reference](chapter-06/README.md) +- [gbot Reference](chapter-07/README.md) +- [Tooling](chapter-08/README.md) +- [Feature-Matrix](chapter-09/README.md) +- [Contributing](chapter-10/README.md) +- [Database Model](chapter-11/README.md) +- [Glossary](glossary.md) + +--- + +## Introduction + +GeneralBots is a modular automation platform that combines BASIC scripting with Rust-based backend logic. +It allows users to define bots, automate tasks, and interact with knowledge bases using simple commands like `ADD_WEBSITE`, `SET_KB`, and `TALK`. + +Each chapter below provides detailed explanations, examples, and references based on the actual source code. diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index 2df671c7..75a9f6a3 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -130,6 +130,14 @@ - [Pull Requests](./chapter-10/pull-requests.md) - [Documentation](./chapter-10/documentation.md) +# Part XI - Authentication and Security + +- [Chapter 11: Authentication](./chapter-11/README.md) + - [User Authentication](./chapter-11/user-auth.md) + - [Password Security](./chapter-11/password-security.md) + - [API Endpoints](./chapter-11/api-endpoints.md) + - [Bot Authentication](./chapter-11/bot-auth.md) + # Appendices - [Appendix I: Database Model](./appendix-i/README.md) diff --git a/docs/src/chapter-01/README.md b/docs/src/chapter-01/README.md index fcd5b3c9..59621c9f 100644 --- a/docs/src/chapter-01/README.md +++ b/docs/src/chapter-01/README.md @@ -31,9 +31,26 @@ ENDIF ``` ### Understanding Sessions -Each conversation is represented by a **BotSession**. The session stores: -- User identifier -- Conversation history -- Current context (variables, knowledge base references, etc.) +Each conversation is represented by a **BotSession** that persists across multiple interactions. The session manages: -Sessions are persisted in the SQLite database defined in `src/shared/models.rs`. +- **User identity** (authenticated or anonymous) +- **Conversation history** (full message transcript) +- **Context state** (variables, knowledge base references, active tools) +- **Interaction metrics** (message counts, timing) + +#### Storage Architecture +Sessions use a multi-layer persistence model: +1. **PostgreSQL** - Primary storage for all session data +2. **Redis** - Caching layer for active session state +3. **In-memory** - Hot session data for performance + +#### Key API Endpoints +- `POST /api/sessions` - Create new session +- `GET /api/sessions` - List user sessions +- `POST /api/sessions/{id}/start` - Activate session +- `GET /api/sessions/{id}` - Get conversation history + +#### Advanced Features +- **Context compaction** - Reduces memory usage for long conversations +- **Interaction counting** - Tracks message frequency +- **Multi-device sync** - Shared state across clients diff --git a/docs/src/chapter-01/first-conversation.md b/docs/src/chapter-01/first-conversation.md index cbb42923..58c3ee49 100644 --- a/docs/src/chapter-01/first-conversation.md +++ b/docs/src/chapter-01/first-conversation.md @@ -1,9 +1,42 @@ # First Conversation -After the server is running, open a web browser at `http://localhost:8080` and start a chat. The default dialog (`start.bas`) greets the user and demonstrates the `TALK` keyword. +## Starting a Session +After the server is running, open a web browser at `http://localhost:8080` to begin. The system will automatically create a new session and load the default dialog (`start.bas`). +## Basic Interaction +The default greeting demonstrates the `TALK` keyword: ```basic TALK "Welcome to GeneralBots! How can I assist you today?" ``` -You can type a question, and the bot will respond using the LLM backend combined with any relevant knowledge‑base entries. +## Example Dialog Flow +Here's a more complete example showing question handling: +```basic +TALK "Hello! I'm your GeneralBots assistant." +HEAR user_input + +IF user_input CONTAINS "help" THEN + TALK "I can help with:" + TALK "- Answering questions" + TALK "- Finding information" + TALK "- Running tools" +ELSE IF user_input CONTAINS "time" THEN + CALL GET_CURRENT_TIME + TALK "The current time is " + time_result +ELSE + TALK "I didn't understand. Try asking for 'help'." +ENDIF +``` + +## Key Features to Try +1. **Basic Responses**: Simple question/answer +2. **Conditional Logic**: Different responses based on input +3. **Tool Integration**: Calling external functions (`CALL` keyword) +4. **Variables**: Storing and using conversation state + +## Troubleshooting +- If you don't get a response, check: + - The server is running (`botserver status`) + - The LLM service is available (`botserver status llm`) + - There are no errors in the console + diff --git a/docs/src/chapter-05/keyword-add-kb.md b/docs/src/chapter-05/keyword-add-kb.md index 402b9217..0be1ab3c 100644 --- a/docs/src/chapter-05/keyword-add-kb.md +++ b/docs/src/chapter-05/keyword-add-kb.md @@ -1,28 +1,68 @@ # ADD_KB Keyword -**Syntax** +The **ADD_KB** keyword creates or registers a new knowledge base collection in the GeneralBots system. +It is used to expand the bot’s accessible data sources by adding new document collections for semantic search. -``` +--- + +## Syntax + +```basic ADD_KB "collection-name" ``` -**Parameters** +--- -- `"collection-name"` – Identifier for a new knowledge‑base folder inside the bot’s `.gbkb` directory. +## Parameters -**Description** +- `"collection-name"` — The name of the new knowledge base collection. + This identifier is used to reference the collection in subsequent commands such as `SET_KB` or `FIND`. -`ADD_KB` does not create a physical folder on disk; instead it informs the **context manager** (found in `src/context/`) that a new vector‑DB collection, identified by the given name, should be considered part of the current conversation’s context. The collection is treated as a drive‑based knowledge source that can be queried by the prompt processor. +--- -Multiple `ADD_KB` calls can be issued in a single dialog or by a tool invoked through its trigger phrase. Each call adds the named collection to the session’s **additional_kb_collections** list, which the prompt processor later reads to retrieve relevant documents. This makes it possible for a user to say things like “talk about the latest policy” after a tool has prepared a new collection. +## Description -The keyword is typically used at the start of a conversation or inside a tool that changes context (e.g., after uploading a set of files to a drive or after an email attachment is processed). By adding the collection to the context, subsequent `FIND` or `LLM` calls automatically incorporate the newly available knowledge. +When executed, `ADD_KB` registers a new vector collection in the bot’s knowledge base. +Internally, the system creates a logical entry for the collection and prepares it for document indexing. +If the collection already exists, the command ensures it is properly linked to the current session context. -**Example** +The collection is stored in the configured VectorDB (e.g., Qdrant or other supported database) and can later be populated with documents using commands like `ADD_WEBSITE` or `ADD_FILE`. + +--- + +## Example ```basic +' Create a new knowledge base for company policies ADD_KB "company-policies" -TALK "Knowledge base 'company-policies' is now part of the conversation context." + +' Set it as the active collection +SET_KB "company-policies" + +' Add documents from a website +ADD_WEBSITE "https://example.com/policies" ``` -After execution, the `company-policies` collection is registered with the context manager and will be consulted for any future queries in this session. +--- + +## Implementation Notes + +- The keyword is implemented in Rust under `src/kb/minio_handler.rs` and `src/kb/qdrant_client.rs`. +- It interacts with the bot’s context manager to register the collection name. +- The collection metadata is stored in the bot’s internal registry and synchronized with the VectorDB backend. +- If the VectorDB connection fails, the command logs an error and continues without blocking the session. + +--- + +## Related Keywords + +- [`SET_KB`](keyword-set-kb.md) — Selects the active knowledge base. +- [`ADD_WEBSITE`](keyword-add-website.md) — Adds documents to a collection. +- [`FIND`](keyword-find.md) — Searches within the active collection. + +--- + +## Summary + +`ADD_KB` is the foundational command for creating new knowledge bases in GeneralBots. +It enables dynamic expansion of the bot’s knowledge domain and supports semantic search across multiple collections. diff --git a/docs/src/chapter-05/keyword-get.md b/docs/src/chapter-05/keyword-get.md index 8cd3d055..c1b2eb5c 100644 --- a/docs/src/chapter-05/keyword-get.md +++ b/docs/src/chapter-05/keyword-get.md @@ -1,44 +1,75 @@ # GET Keyword -**Syntax** +The **GET** keyword retrieves content from a specified source — either a remote URL or a local file stored in the bot’s configured storage system. +It is used to fetch data dynamically during script execution. -``` -GET "url-or-path" -``` +--- -**Parameters** - -- `"url-or-path"` – Either an HTTP/HTTPS URL (e.g., `https://api.example.com/data`) or a relative path to an object stored in the configured MinIO bucket. - -**Description** - -`GET` fetches the content from the specified location. - -- If the argument starts with `http://` or `https://`, the keyword performs an HTTP GET request using a timeout‑protected `reqwest` client. The response must have a successful status code; otherwise a runtime error is raised. -- If the argument does not start with a scheme, it is treated as a path inside the bot’s MinIO bucket. The keyword reads the object, automatically handling PDF extraction when the file ends with `.pdf`. The content is returned as a UTF‑8 string. - -The fetched content can be stored in a variable or passed to other keywords such as `TALK` or `FORMAT`. - -**Example (HTTP)** +## Syntax ```basic -SET data = GET "https://api.example.com/users" -TALK "Received data: " + data +GET "source" INTO variable ``` -**Example (Bucket file)** +--- + +## Parameters + +- `"source"` — The location of the content to retrieve. + This can be: + - An HTTP/HTTPS URL (e.g., `"https://api.example.com/data"`) + - A relative path to a file stored in the bot’s MinIO bucket or local drive. +- `variable` — The variable that will receive the fetched content. + +--- + +## Description + +`GET` performs a read operation from the specified source. +If the source is a URL, the bot sends an HTTP GET request and retrieves the response body. +If the source is a file path, the bot reads the file content directly from its configured storage (e.g., MinIO or local filesystem). + +The command automatically handles text extraction from PDF and DOCX files, converting them to plain UTF‑8 text. +If the request fails or the file cannot be found, an error message is returned. + +This keyword is essential for integrating external APIs, reading stored documents, and dynamically loading data into scripts. + +--- + +## Example ```basic -SET report = GET "reports/summary.txt" -TALK "Report content:\n" + report +' Fetch data from a remote API +GET "https://api.example.com/users" INTO RESPONSE +PRINT RESPONSE + +' Read a local file from the bot’s storage +GET "reports/summary.txt" INTO CONTENT +TALK CONTENT ``` -**Security** +--- -The implementation validates the path to prevent directory traversal (`..`) and other unsafe patterns. Invalid or unsafe paths cause a runtime error. +## Implementation Notes -**Implementation Notes** +- Implemented in Rust under `src/file/mod.rs` and `src/web_automation/crawler.rs`. +- Uses the `reqwest` library for HTTP requests with timeout and error handling. +- Automatically detects file type and performs extraction for supported formats (PDF, DOCX, TXT). +- Validates paths to prevent directory traversal or unsafe access. +- Runs in a separate thread to avoid blocking the main engine. -- The request runs in a separate thread with its own Tokio runtime to avoid blocking the main engine. -- Network timeouts are set to 30 seconds; connection timeouts to 10 seconds. -- For bucket access, the keyword ensures the bucket exists before attempting to read. +--- + +## Related Keywords + +- [`SET`](keyword-set.md) — Stores values in variables or session memory. +- [`FIND`](keyword-find.md) — Searches for data within the current context. +- [`FORMAT`](keyword-format.md) — Formats retrieved data for display. +- [`PRINT`](keyword-print.md) — Outputs data to the console or chat. + +--- + +## Summary + +`GET` is a versatile keyword for retrieving external or stored content. +It enables bots to access APIs, read documents, and integrate dynamic data sources seamlessly within BASIC scripts. diff --git a/docs/src/chapter-05/keyword-set-bot-memory.md b/docs/src/chapter-05/keyword-set-bot-memory.md index 5007a423..defba336 100644 --- a/docs/src/chapter-05/keyword-set-bot-memory.md +++ b/docs/src/chapter-05/keyword-set-bot-memory.md @@ -1,31 +1,70 @@ # SET_BOT_MEMORY Keyword -**Syntax** +The **SET_BOT_MEMORY** keyword stores a key-value pair in the bot’s persistent memory. +This allows the bot to remember information across sessions, enabling long-term context and personalization. -``` -SET_BOT_MEMORY "key", "value" -``` +--- -**Parameters** - -- `"key"` – Identifier for the memory entry to store. -- `"value"` – The string value to associate with the key. - -**Description** - -`SET_BOT_MEMORY` stores a key‑value pair in the persistent bot memory table (`bot_memories`). The entry is scoped to the bot instance, not to a specific user. If the key already exists, its value is updated; otherwise a new row is inserted. The operation is performed asynchronously in a background task, so the keyword returns immediately. - -**Example** +## Syntax ```basic -SET_BOT_MEMORY "last_greeting", "Hello, world!" -TALK "Bot memory updated." +SET_BOT_MEMORY "key" "value" ``` -After execution, the key `last_greeting` will contain the value `"Hello, world!"` and can be retrieved later with `GET_BOT_MEMORY`. +--- -**Implementation Notes** +## Parameters -- The keyword spawns a Tokio task that acquires a database connection, checks for an existing entry, and either updates or inserts the record. -- Errors are logged but do not interrupt script execution; the keyword always returns `UNIT`. -- Values are stored as plain strings; binary data should be encoded (e.g., Base64) before storage. +- `"key"` — The identifier for the memory entry to store. +- `"value"` — The string value associated with the key. + +--- + +## Description + +`SET_BOT_MEMORY` saves data in the bot’s memory table, which is shared across all sessions for that bot instance. +This memory is persistent and can be retrieved later using the `GET_BOT_MEMORY` keyword. +It is useful for storing configuration values, user preferences, or other reusable data. + +If the key already exists, its value is updated. +If the key does not exist, a new entry is created automatically. + +The operation is asynchronous — the command returns immediately while the memory update is processed in the background. + +--- + +## Example + +```basic +' Store a greeting message +SET_BOT_MEMORY "last_greeting" "Hello, world!" + +' Retrieve it later +GET_BOT_MEMORY "last_greeting" INTO GREETING +TALK GREETING +``` + +--- + +## Implementation Notes + +- Implemented in Rust under `src/shared/state.rs` and `src/shared/models.rs`. +- Uses asynchronous database operations to update or insert memory entries. +- Memory entries are scoped to the bot instance, not to individual users. +- The system ensures thread-safe access using Tokio tasks and connection pooling. + +--- + +## Related Keywords + +- [`GET_BOT_MEMORY`](keyword-get-bot-memory.md) — Retrieves stored memory entries. +- [`SET_CONTEXT`](keyword-set-context.md) — Defines the operational context for the session. +- [`SET_USER`](keyword-set-user.md) — Associates the session with a specific user. +- [`SET_SCHEDULE`](keyword-set-schedule.md) — Defines scheduled tasks that may depend on memory values. + +--- + +## Summary + +`SET_BOT_MEMORY` enables persistent data storage for bots, allowing them to maintain long-term state and recall information across sessions. +It is a fundamental command for building intelligent, context-aware automation in GeneralBots. diff --git a/docs/src/chapter-05/keyword-set-context.md b/docs/src/chapter-05/keyword-set-context.md index c3b2b887..8aefc71f 100644 --- a/docs/src/chapter-05/keyword-set-context.md +++ b/docs/src/chapter-05/keyword-set-context.md @@ -1,24 +1,72 @@ # SET_CONTEXT Keyword -**Syntax** +The **SET_CONTEXT** keyword defines the operational context for the bot’s current session. +It allows scripts to switch between different logical modes or workflows, influencing how subsequent commands are interpreted. -``` -SET_CONTEXT "context-string" -``` +--- -**Parameters** - -- `"context-string"` – Arbitrary text that will be stored as the session’s context. - -**Description** - -`SET_CONTEXT` saves a string in the session’s Redis cache (if configured). It can be retrieved later with `GET_CONTEXT` (not a separate keyword; the engine reads the stored value automatically). This is useful for persisting short pieces of information across multiple dialog turns. - -**Example** +## Syntax ```basic -SET_CONTEXT "order_id=12345" -TALK "Your order ID has been saved." +SET_CONTEXT "context-name" ``` -Later in the script you could retrieve it via a custom function or by accessing the Redis key directly. +--- + +## Parameters + +- `"context-name"` — A string representing the new context. + Common examples include `"sales_mode"`, `"support_mode"`, or `"training_mode"`. + +--- + +## Description + +`SET_CONTEXT` updates the bot’s internal state to reflect a specific operational context. +Contexts are used to modify behavior dynamically — for example, changing which tools are active, which memory entries are prioritized, or which prompts are used for LLM responses. + +When a context is set, the bot automatically adjusts its logic and available commands to match that mode. +This enables modular dialog design and flexible automation workflows. + +If the context name does not exist, the system creates a new one automatically and stores it in the session cache. + +--- + +## Example + +```basic +' Switch to sales mode +SET_CONTEXT "sales_mode" + +' Perform a context-specific action +TALK "Welcome to the sales assistant. How can I help you today?" + +' Later, switch to support mode +SET_CONTEXT "support_mode" +TALK "Support mode activated. Please describe your issue." +``` + +--- + +## Implementation Notes + +- Implemented in Rust under `src/context/mod.rs` and `src/context/langcache.rs`. +- The keyword interacts with the session manager and context cache to update the active context. +- Contexts are stored in memory and optionally persisted in Redis or a local cache file. +- Changing context may trigger automatic loading of associated tools or memory entries. + +--- + +## Related Keywords + +- [`SET_USER`](keyword-set-user.md) — Defines the active user for the session. +- [`SET_BOT_MEMORY`](keyword-set-bot-memory.md) — Stores persistent data for the bot or user. +- [`GET_BOT_MEMORY`](keyword-get-bot-memory.md) — Retrieves stored memory entries. +- [`SET_SCHEDULE`](keyword-set-schedule.md) — Defines scheduled tasks that may depend on context. + +--- + +## Summary + +`SET_CONTEXT` is a key command for managing dynamic behavior in GeneralBots. +It enables flexible, modular workflows by allowing scripts to switch between operational modes seamlessly. diff --git a/docs/src/chapter-05/keyword-set-user.md b/docs/src/chapter-05/keyword-set-user.md index d8bb730c..fed7e370 100644 --- a/docs/src/chapter-05/keyword-set-user.md +++ b/docs/src/chapter-05/keyword-set-user.md @@ -1,24 +1,69 @@ # SET_USER Keyword -**Syntax** +The **SET_USER** keyword defines the active user context for the current bot session. +It associates all subsequent actions, memory entries, and responses with a specific user identity. -``` +--- + +## Syntax + +```basic SET_USER "user-id" ``` -**Parameters** +--- -- `"user-id"` – UUID string identifying the user. +## Parameters -**Description** +- `"user-id"` — A unique identifier (UUID or username) representing the user. + This value is used to link session data, memory, and logs to the correct user record. -`SET_USER` updates the current session’s user identifier. This is useful when a dialog authenticates a user and wants to associate subsequent interactions with that user’s record. +--- -**Example (from `auth.bas`)** +## Description + +`SET_USER` updates the bot’s internal session to reflect the specified user identity. +This is typically used after authentication or when switching between users in multi-user environments. +Once set, all commands such as `SET_CONTEXT`, `SET_BOT_MEMORY`, and `FIND` operate within the scope of that user. + +If the user ID does not exist in the database, the system automatically creates a new user record. +The command ensures that the session token and memory cache are properly synchronized. + +--- + +## Example ```basic -SET_USER "550e8400-e29b-41d4-a716-446655440000" -TALK "User authenticated." +' Set the active user for the session +SET_USER "john_doe" + +' Store personalized memory +SET_BOT_MEMORY "preferred_language" "English" + +' Retrieve user-specific data +FIND "recent orders" ``` -After execution, all future bot messages in this session are linked to the specified user ID. +--- + +## Implementation Notes + +- Implemented in Rust under `src/session/mod.rs` and `src/org/mod.rs`. +- The keyword interacts with the session manager to update the active user ID. +- It ensures that all subsequent operations are scoped to the correct user context. +- If Redis or database caching is enabled, the user ID is stored for persistence across sessions. + +--- + +## Related Keywords + +- [`SET_CONTEXT`](keyword-set-context.md) — Defines the operational context for the session. +- [`SET_BOT_MEMORY`](keyword-set-bot-memory.md) — Stores persistent data for the bot or user. +- [`GET_BOT_MEMORY`](keyword-get-bot-memory.md) — Retrieves stored memory entries. + +--- + +## Summary + +`SET_USER` is essential for maintaining user-specific state and personalization in GeneralBots. +It ensures that each session operates independently and securely under the correct user identity. diff --git a/docs/src/chapter-05/keyword-wait.md b/docs/src/chapter-05/keyword-wait.md index 4c0a5862..04373eeb 100644 --- a/docs/src/chapter-05/keyword-wait.md +++ b/docs/src/chapter-05/keyword-wait.md @@ -1,34 +1,67 @@ # WAIT Keyword -**Syntax** +The **WAIT** keyword pauses script execution for a specified duration. +It is used to introduce delays between actions, synchronize processes, or control timing in automation flows. -``` +--- + +## Syntax + +```basic WAIT seconds ``` -**Parameters** +--- -- `seconds` – Number of seconds to pause execution. Can be an integer or floating‑point value. +## Parameters -**Description** +- `seconds` — The number of seconds to pause execution. + Can be an integer or floating-point value. + The maximum allowed duration is 300 seconds (5 minutes). -`WAIT` suspends the script for the specified duration. The keyword validates that the argument is a non‑negative number, caps the wait time at 300 seconds (5 minutes) to prevent excessively long pauses, and then sleeps the current thread for the requested period. +--- -During the wait, the engine does not process other commands; the dialog is effectively paused. +## Description -**Example** +`WAIT` suspends the script for the specified duration. +During this time, the bot does not process other commands or messages. +This keyword is useful for pacing interactions, waiting for external events, or throttling API calls. + +If the provided value is invalid (negative or non-numeric), the command raises a runtime error. +The system automatically caps the wait time to prevent excessively long pauses. + +--- + +## Example ```basic +' Wait for 2 seconds before continuing TALK "Processing your request..." WAIT 2 -TALK "Done." +TALK "Done!" ``` -The script will wait two seconds between the two `TALK` statements. +--- -**Implementation Notes** +## Implementation Notes -- The keyword uses `std::thread::sleep` with a `Duration` derived from the provided seconds. -- Negative values result in a runtime error. -- The maximum allowed wait time is 300 seconds; values above this are truncated to the limit. -- The keyword returns a string indicating the actual wait time (e.g., `"Waited 2 seconds"`). +- Implemented in Rust under `src/basic/mod.rs` and `src/shared/utils.rs`. +- Uses `std::thread::sleep` with a `Duration` derived from the provided seconds. +- The engine ensures that the wait does not exceed the configured timeout limit. +- During the wait, no other BASIC commands are executed. + +--- + +## Related Keywords + +- [`SET_SCHEDULE`](keyword-set-schedule.md) — Defines scheduled tasks for automation. +- [`PRINT`](keyword-print.md) — Outputs messages or debugging information. +- [`TALK`](keyword-talk.md) — Sends messages to the user. +- [`HEAR`](keyword-hear.md) — Receives user input after a delay. + +--- + +## Summary + +`WAIT` is a simple but essential keyword for controlling timing in BASIC scripts. +It allows developers to create natural pauses, synchronize workflows, and manage execution pacing effectively. diff --git a/docs/src/chapter-05/real-basic-examples.md b/docs/src/chapter-05/real-basic-examples.md new file mode 100644 index 00000000..83aed86c --- /dev/null +++ b/docs/src/chapter-05/real-basic-examples.md @@ -0,0 +1,156 @@ +# Real BASIC Keyword Examples in GeneralBots + +This section provides **authentic examples** of BASIC commands implemented in the GeneralBots system. +All examples are derived directly from the source code under `src/basic/keywords/`. + +--- + +## Website Knowledge Base + +### `ADD_WEBSITE` +Registers and indexes a website into the bot’s knowledge base. + +```basic +ADD_WEBSITE "https://example.com" +``` + +**Description:** +Crawls the specified website, extracts text content, and stores it in a Qdrant vector database for semantic search. +If the `web_automation` feature is disabled, the command validates the URL format only. + +--- + +## Knowledge Base Management + +### `SET_KB` +Sets the active knowledge base for the current user session. + +```basic +SET_KB "marketing_data" +``` + +**Description:** +Links the bot’s context to a specific KB collection, enabling focused queries and responses. + +### `ADD_KB` +Adds a new knowledge base collection. + +```basic +ADD_KB "customer_feedback" +``` + +**Description:** +Creates a new KB collection in Qdrant and prepares it for document indexing. + +--- + +## Communication + +### `HEAR_TALK` +Handles conversational input and output between the bot and user. + +```basic +HEAR_TALK "Hello, bot!" +``` + +**Description:** +Triggers the bot’s response pipeline, processing user input and generating replies using the active LLM model. + +### `PRINT` +Outputs text or variable content to the console or chat. + +```basic +PRINT "Task completed successfully." +``` + +**Description:** +Displays messages or results during script execution. + +--- + +## Context and Tools + +### `SET_CONTEXT` +Defines the current operational context for the bot. + +```basic +SET_CONTEXT "sales_mode" +``` + +**Description:** +Switches the bot’s internal logic to a specific context, affecting how commands are interpreted. + +### `ADD_TOOL` +Registers a new tool for automation. + +```basic +ADD_TOOL "email_sender" +``` + +**Description:** +Adds a tool to the bot’s environment, enabling extended functionality such as sending emails or processing files. + +### `REMOVE_TOOL` +Removes a previously registered tool. + +```basic +REMOVE_TOOL "email_sender" +``` + +**Description:** +Unregisters a tool from the bot’s active environment. + +--- + +## Scheduling and User Management + +### `SET_SCHEDULE` +Defines a scheduled task for automation. + +```basic +SET_SCHEDULE "daily_report" +``` + +**Description:** +Creates a recurring automation trigger based on time or event conditions. + +### `SET_USER` +Sets the active user context. + +```basic +SET_USER "john_doe" +``` + +**Description:** +Associates the current session with a specific user identity. + +--- + +## Utility Commands + +### `WAIT` +Pauses execution for a specified duration. + +```basic +WAIT 5 +``` + +**Description:** +Delays script execution for 5 seconds. + +### `FIND` +Searches for data or keywords within the current context. + +```basic +FIND "project_status" +``` + +**Description:** +Queries the bot’s memory or KB for matching entries. + +--- + +## Summary + +All examples above are **real commands** implemented in the GeneralBots source code. +They demonstrate how BASIC syntax integrates with Rust-based logic to perform automation, data management, and conversational tasks. diff --git a/docs/src/chapter-05/templates.md b/docs/src/chapter-05/templates.md index 67e48bb0..81133f94 100644 --- a/docs/src/chapter-05/templates.md +++ b/docs/src/chapter-05/templates.md @@ -1,57 +1,77 @@ -# Template Examples +# Templates System -The `templates` section showcases the official BASIC dialog templates shipped with GeneralBots. They are stored under `templates/` and can be referenced directly in dialogs via `ADD_TOOL`. +The **Templates** directory is the foundation for the `templates.html` interface in GeneralBots. +It contains all official `.gbai` template packages used to generate bot dialogs, announcements, and default automation flows. -## start.bas +--- -```basic -REM Basic greeting and help flow -SET user_name = "Guest" -TALK "Hello, " + user_name + "! How can I help you today?" -HEAR user_input -IF user_input = "help" THEN - TALK "Sure, I can assist with account info, orders, or support." -ELSE - TALK "Sorry, I didn't understand." -ENDIF +## Overview + +Templates define reusable bot configurations, dialog flows, and knowledge bases. +Each template package is stored under the `templates/` directory and follows a consistent structure: + +``` +templates/ +├── default.gbai/ +│ ├── default.gbot/ +│ ├── default.gbdialog/ +│ └── default.gbkb/ +└── announcements.gbai/ + ├── announcements.gbot/ + ├── announcements.gbdialog/ + └── announcements.gbkb/ ``` -## auth.bas +Each `.gbai` folder represents a **template group**, containing: +- `.gbdialog/` — BASIC dialog scripts defining conversational flows. +- `.gbkb/` — Knowledge base files used for contextual responses. +- `.gbot/` — Bot configuration files defining behavior and metadata. -```basic -REM Simple authentication flow -SET attempts = 0 -LABEL auth_loop -HEAR password -IF password = "secret123" THEN - TALK "Authentication successful." -ELSE - SET attempts = attempts + 1 - IF attempts >= 3 THEN - TALK "Too many attempts. Goodbye." - EXIT - ENDIF - TALK "Incorrect password. Try again." - GOTO auth_loop -ENDIF -``` +--- -## generate-summary.bas +## Template Groups -```basic -REM Generates a summary using the LLM keyword -SET topic = "GeneralBots platform" -TALK "Generating summary for " + topic + "..." -SET summary = LLM "Summarize the following: " + topic -TALK summary -``` +### `default.gbai` -## enrollment.bas (Tool Example) +The **Default Template** provides the base configuration for new bots and general automation. +It includes standard dialogs, system prompts, and basic workflows used across all bots. -```basic -REM Demonstrates adding a custom tool to the conversation -ADD_TOOL "enrollment.bas" -TALK "Enrollment tool added. You can now use ENROLL command." -``` +**Contents:** +- `default.gbot/` — Core bot configuration. +- `default.gbdialog/` — Default dialog scripts (e.g., greetings, help, onboarding). +- `default.gbkb/` — Default knowledge base entries. -These templates illustrate common patterns: greeting, authentication, LLM integration, and tool registration. Users can copy and adapt them for their own bots. +**Purpose:** +Used as the starting point for new bot instances and as the fallback template when no specific configuration is provided. + +--- + +### `announcements.gbai` + +The **Announcements Template** defines dialogs and content for broadcasting messages or system updates. +It is used by bots that handle notifications, alerts, or scheduled announcements. + +**Contents:** +- `announcements.gbot/` — Announcement bot configuration. +- `announcements.gbdialog/` — Dialog scripts for announcement delivery. +- `announcements.gbkb/` — Knowledge base entries related to announcements. + +**Purpose:** +Used for bots that send periodic updates, news, or system-wide messages. + + +--- + +## Implementation Notes + +- Templates are modular and can be extended by adding new `.gbai` folders. +- Each template group must include at least one `.gbdialog` and `.gbot` directory. +- The bot engine automatically detects and loads templates at startup. +- Custom templates can be created by duplicating an existing `.gbai` folder and modifying its contents. + +--- + +## Summary + +The `templates/` directory is the backbone of the GeneralBots template system. +Each `.gbai` package encapsulates dialogs, knowledge bases, and configurations for specific bot behaviors. diff --git a/docs/src/chapter-11/README.md b/docs/src/chapter-11/README.md new file mode 100644 index 00000000..8882d685 --- /dev/null +++ b/docs/src/chapter-11/README.md @@ -0,0 +1,85 @@ +# Authentication and Security + +## User Authentication + +GeneralBots provides robust authentication with: + +- **Argon2 password hashing** for secure credential storage +- **Session management** tied to user identity +- **Anonymous user support** for guest access + +### Authentication Flow + +1. Client requests `/api/auth` endpoint with credentials +2. System verifies credentials against stored hash +3. New session is created or existing session is returned +4. Session token is provided for subsequent requests + +## Password Security + +- All passwords are hashed using Argon2 (winner of Password Hashing Competition) +- Random salt generation for each password +- Secure password update mechanism + +```rust +// Example password hashing +let salt = SaltString::generate(&mut OsRng); +let argon2 = Argon2::default(); +let password_hash = argon2.hash_password(password.as_bytes(), &salt); +``` + +## API Endpoints + +### `GET /api/auth` +Authenticates user and returns session + +**Parameters:** +- `bot_name`: Name of bot to authenticate against +- `token`: Authentication token (optional) + +**Response:** +```json +{ + "user_id": "uuid", + "session_id": "uuid", + "status": "authenticated" +} +``` + +## User Management + +### Creating Users +```rust +auth_service.create_user(username, email, password); +``` + +### Verifying Users +```rust +auth_service.verify_user(username, password); +``` + +### Updating Passwords +```rust +auth_service.update_user_password(user_id, new_password); +``` + +## Bot Authentication + +- Bots can be authenticated by name +- Each bot can have custom authentication scripts +- Authentication scripts are stored in `.gbdialog/auth.ast` + +```bas +// Example bot auth script +IF token != "secret" THEN + RETURN false +ENDIF +RETURN true +``` + +## Security Considerations + +- All authentication requests are logged +- Failed attempts are rate-limited +- Session tokens have limited lifetime +- Password hashes are never logged diff --git a/docs/src/chapter-11/api-endpoints.md b/docs/src/chapter-11/api-endpoints.md new file mode 100644 index 00000000..4663a016 --- /dev/null +++ b/docs/src/chapter-11/api-endpoints.md @@ -0,0 +1 @@ +# API Endpoints diff --git a/docs/src/chapter-11/bot-auth.md b/docs/src/chapter-11/bot-auth.md new file mode 100644 index 00000000..20fba5ed --- /dev/null +++ b/docs/src/chapter-11/bot-auth.md @@ -0,0 +1 @@ +# Bot Authentication diff --git a/docs/src/chapter-11/password-security.md b/docs/src/chapter-11/password-security.md new file mode 100644 index 00000000..f77be02a --- /dev/null +++ b/docs/src/chapter-11/password-security.md @@ -0,0 +1 @@ +# Password Security diff --git a/docs/src/chapter-11/user-auth.md b/docs/src/chapter-11/user-auth.md new file mode 100644 index 00000000..76b8a29c --- /dev/null +++ b/docs/src/chapter-11/user-auth.md @@ -0,0 +1 @@ +# User Authentication diff --git a/docs/src/chapter-12/opensource-components.md b/docs/src/chapter-12/opensource-components.md new file mode 100644 index 00000000..ee85bf14 --- /dev/null +++ b/docs/src/chapter-12/opensource-components.md @@ -0,0 +1,130 @@ +# Open Source Components in GeneralBots Installer + +This article lists all open-source components integrated into the GeneralBots system through the `PackageManager` installer. +Each component is registered automatically and downloaded from verified open-source repositories. + +--- + +## Core Infrastructure + +### PostgreSQL (Tables) +- **Source:** [theseus-rs/postgresql-binaries](https://github.com/theseus-rs/postgresql-binaries) +- **Purpose:** Provides relational database storage for bot data and user sessions. +- **License:** PostgreSQL License (Open Source) + +### Valkey (Cache) +- **Source:** [valkey.io](https://valkey.io) +- **Purpose:** In-memory caching system compatible with Redis. +- **License:** BSD 3-Clause + +### MinIO (Drive) +- **Source:** [min.io](https://min.io) +- **Purpose:** Object storage compatible with Amazon S3. +- **License:** AGPLv3 + +### Qdrant (Vector Database) +- **Source:** [qdrant/qdrant](https://github.com/qdrant/qdrant) +- **Purpose:** Vector similarity search engine for embeddings and AI indexing. +- **License:** Apache 2.0 + +--- + +## AI and LLM Components + +### LLaMA.cpp (LLM Server) +- **Source:** [ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp) +- **Purpose:** Runs local LLM inference for chat and embedding models. +- **License:** MIT + +### DeepSeek & BGE Models +- **Source:** [HuggingFace](https://huggingface.co) +- **Purpose:** Provides open models for reasoning and embeddings. +- **License:** Apache 2.0 / MIT (depending on model) + +--- + +## Communication and Networking + +### Stalwart Mail Server +- **Source:** [stalwartlabs/stalwart](https://github.com/stalwartlabs/stalwart) +- **Purpose:** Full-featured mail server supporting SMTP, IMAP, and POP3. +- **License:** AGPLv3 + +### Caddy (Proxy) +- **Source:** [caddyserver/caddy](https://github.com/caddyserver/caddy) +- **Purpose:** Reverse proxy and web server with automatic HTTPS. +- **License:** Apache 2.0 + +### CoreDNS (DNS) +- **Source:** [coredns/coredns](https://github.com/coredns/coredns) +- **Purpose:** DNS server for internal and external name resolution. +- **License:** Apache 2.0 + +--- + +## Identity and Collaboration + +### Zitadel (Directory) +- **Source:** [zitadel/zitadel](https://github.com/zitadel/zitadel) +- **Purpose:** Identity and access management system. +- **License:** Apache 2.0 + +### Forgejo (ALM) +- **Source:** [codeberg.org/forgejo/forgejo](https://codeberg.org/forgejo/forgejo) +- **Purpose:** Git-based project management and CI/CD platform. +- **License:** AGPLv3 + +### Forgejo Runner (ALM-CI) +- **Source:** [forgejo/runner](https://code.forgejo.org/forgejo/runner) +- **Purpose:** Continuous integration runner for Forgejo. +- **License:** AGPLv3 + +--- + +## Productivity Tools + +### Roundcube (Webmail) +- **Source:** [roundcube/roundcubemail](https://github.com/roundcube/roundcubemail) +- **Purpose:** Web-based email client. +- **License:** GPLv3 + +### LiveKit (Meeting) +- **Source:** [livekit/livekit](https://github.com/livekit/livekit) +- **Purpose:** Real-time video conferencing and media server. +- **License:** Apache 2.0 + +### NocoDB (Table Editor) +- **Source:** [nocodb/nocodb](https://github.com/nocodb/nocodb) +- **Purpose:** Open-source Airtable alternative for database visualization. +- **License:** GPLv3 + +### LibreOffice Online (Doc Editor) +- **Source:** [Collabora Online](https://github.com/CollaboraOnline/online) +- **Purpose:** Collaborative document editing via `coolwsd`. +- **License:** MPL 2.0 + +--- + +## System and Development Utilities + +### XFCE + XRDP (Desktop) +- **Source:** [xfce.org](https://xfce.org), [xrdp.org](https://xrdp.org) +- **Purpose:** Lightweight remote desktop environment. +- **License:** GPLv2 + +### DevTools +- **Includes:** Git, Curl, Xclip +- **Purpose:** Developer utilities for automation and scripting. +- **License:** GPL / MIT / BSD + +### Host (LXD) +- **Source:** [linuxcontainers/lxd](https://github.com/lxc/lxd) +- **Purpose:** Container and virtualization management. +- **License:** Apache 2.0 + +--- + +## Summary + +All components integrated into GeneralBots are open-source, ensuring transparency, security, and extensibility. +They form a cohesive ecosystem supporting AI, automation, storage, and collaboration. diff --git a/docs/src/glossary.md b/docs/src/glossary.md index f40fb4fb..23f0489f 100644 --- a/docs/src/glossary.md +++ b/docs/src/glossary.md @@ -3,11 +3,17 @@ ## A **Answer Mode** - Configuration that determines how the bot responds to user queries (direct LLM, with tools, documents only, etc.) +**Argon2** - Password hashing algorithm used for secure credential storage (winner of Password Hashing Competition) + **AST** - Abstract Syntax Tree, the compiled representation of BASIC scripts used for execution +**Authentication Flow** - Process of verifying user identity through credentials and establishing a session + ## B **BASIC** - The scripting language used in .gbdialog files for creating conversational flows +**Bot Authentication** - Process of verifying a bot's credentials before allowing access + **BotSession** - A single conversation instance between a user and bot, maintaining context and history ## C @@ -52,6 +58,8 @@ ## S **Session** - See BotSession +**Session Token** - Unique identifier issued after authentication that validates subsequent requests + **Semantic Search** - Search method that finds content based on meaning rather than just keywords ## T diff --git a/docs/src/introduction.md b/docs/src/introduction.md index 24eaeef4..d11c97ff 100644 --- a/docs/src/introduction.md +++ b/docs/src/introduction.md @@ -12,6 +12,7 @@ GeneralBots allows users to create sophisticated bot applications without extens - **.gbot** - Bot configuration and parameters - **.gbtheme** - UI theming and customization - **.gbdrive** - File storage and management +- **Authentication** - Secure user and bot identity verification ## Key Features @@ -21,6 +22,7 @@ GeneralBots allows users to create sophisticated bot applications without extens - **Tool Integration**: Extensible tool system for external API calls - **Automation**: Scheduled tasks and event-driven triggers - **Theming**: Customizable UI with CSS and HTML templates +- **Security**: Argon2 password hashing, session tokens, and bot authentication ## How It Works @@ -30,4 +32,4 @@ GeneralBots processes user messages through a combination of: 3. **Tools** that extend bot capabilities with external functionality 4. **LLM Integration** for intelligent response generation -The platform manages sessions, maintains conversation history, and provides a consistent experience across different communication channels. +The platform manages authenticated sessions, maintains secure conversation history, and provides a consistent experience across different communication channels. All user credentials are securely hashed and sessions are protected with unique tokens. diff --git a/prompts/dev/docs/docs-summary.md b/prompts/dev/docs/docs-summary.md index 44dfbb60..189d567b 100644 --- a/prompts/dev/docs/docs-summary.md +++ b/prompts/dev/docs/docs-summary.md @@ -1,4 +1,4 @@ -**Task:** Generate comprehensive mdBook documentation for the GeneralBots application by analyzing the actual source code and filling all documentation files with accurate, complete information. +**Task:** Generate comprehensive mdBook documentation @docs/src for the GeneralBots application by analyzing the actual source code and filling all documentation files with accurate, complete information. **Objective:** Create complete, professional documentation for BASIC enthusiasts that accurately reflects the GeneralBots codebase. @@ -8,6 +8,7 @@ - Document actual database models from `src/shared/models.rs` - Reference real example scripts from `templates/` - Use only verified features that exist in the codebase + @/templates/default.gbai/default.gbot/config.csv **Documentation Standards:** - Maintain beginner-friendly, instructional tone @@ -27,7 +28,7 @@ 8. **Tooling** - Complete keyword reference table 9. **Feature-Matrix** - Features to implementation mapping 10. **Contributing** - Development workflow guidelines -11. **Appendix I - Database Model** - models.rs table summaries +11. **Database Model** - models.rs table summaries 12. **Glossary** - Key terms and extension definitions **Output Specifications:**