new(all): Initial import.
This commit is contained in:
Rodrigo Rodriguez
parent
df7c776edb
commit
175b1314bf
3 changed files with 319 additions and 148 deletions
335
README.md
335
README.md
|
|
@ -1,171 +1,216 @@
|
||||||
# General Bots 6 (GB6) Platform Architecture
|
# General Bots 6 (GB6) Platform
|
||||||
|
|
||||||
## Overview
|
## Vision
|
||||||
General Bots 6 (GB6) is a billion-scale real-time communication platform that integrates advanced bot capabilities, WebRTC-based multimedia communication, and enterprise-grade messaging. The platform is designed to support massive concurrent usage while maintaining high performance and reliability.
|
GB6 is a billion-scale real-time communication platform integrating advanced bot capabilities, WebRTC multimedia, and enterprise-grade messaging, built with Rust for maximum performance and reliability and BASIC-WebAssembly VM.
|
||||||
|
|
||||||
## Key Capabilities
|
## 🌟 Key Features
|
||||||
- **Scale**: Supports billions of active users and millions of concurrent rooms
|
|
||||||
- **Performance**: Sub-second message delivery with 4K video streaming capabilities
|
|
||||||
- **Storage**: Petabyte-scale message and media storage
|
|
||||||
- **Distribution**: Global presence with regional optimization
|
|
||||||
- **Reliability**: 99.99% uptime guarantee with zero message loss
|
|
||||||
- **Security**: Enterprise-grade security with multi-tenant isolation
|
|
||||||
|
|
||||||
## System Architecture
|
### Scale & Performance
|
||||||
|
- Billion+ active users support
|
||||||
|
- Sub-second message delivery
|
||||||
|
- 4K video streaming
|
||||||
|
- 99.99% uptime guarantee
|
||||||
|
- Zero message loss
|
||||||
|
- Petabyte-scale storage
|
||||||
|
|
||||||
### 1. Multi-Tenant Core
|
### Core Services
|
||||||
|
- **API Service** (gb-api)
|
||||||
|
- Axum-based REST & WebSocket
|
||||||
|
- Multi-tenant request routing
|
||||||
|
- Authentication & Authorization
|
||||||
|
- File handling & streaming
|
||||||
|
|
||||||
#### Customer Management
|
- **Media Processing** (gb-media)
|
||||||
- **Organization Hierarchy**
|
- WebRTC integration
|
||||||
- Multi-level customer organization structure
|
- GStreamer transcoding
|
||||||
- Independent instance management
|
- Real-time track management
|
||||||
- Regional deployment controls
|
- Professional recording
|
||||||
- Resource quota enforcement
|
|
||||||
|
|
||||||
#### Subscription & Billing
|
|
||||||
- Usage tracking and analytics
|
|
||||||
- Flexible billing models
|
|
||||||
- Resource allocation management
|
|
||||||
- Quota monitoring and enforcement
|
|
||||||
|
|
||||||
### 2. Communication Infrastructure
|
- **Messaging** (gb-messaging)
|
||||||
|
- Kafka event processing
|
||||||
|
- RabbitMQ integration
|
||||||
|
- WebSocket communication
|
||||||
|
- Redis PubSub
|
||||||
|
|
||||||
#### Real-time Rooms
|
- **Storage** (gb-storage)
|
||||||
- **WebRTC Integration**
|
- PostgreSQL with sharding
|
||||||
- High-quality audio/video streaming
|
- Redis caching
|
||||||
- Dynamic track management
|
- TiKV distributed storage
|
||||||
- Intelligent participant handling
|
- Customer data management
|
||||||
- Scalable room management
|
|
||||||
|
|
||||||
- **Media Features**
|
|
||||||
- Professional recording capabilities
|
|
||||||
- Zoom-like video conferencing
|
|
||||||
- TikTok-style live streaming
|
|
||||||
- Advanced media processing
|
|
||||||
|
|
||||||
#### Messaging System
|
## 🏗 Architecture
|
||||||
- **Core Messaging**
|
|
||||||
- Sharded message processing
|
|
||||||
- Guaranteed message persistence
|
|
||||||
- Real-time delivery optimization
|
|
||||||
- Sophisticated message routing
|
|
||||||
|
|
||||||
- **Message Features**
|
|
||||||
- Delivery status tracking
|
|
||||||
- Full-text message search
|
|
||||||
- Thread management
|
|
||||||
- Rich media support
|
|
||||||
|
|
||||||
### 3. Technical Infrastructure
|
### Multi-Tenant Core
|
||||||
|
- Organization hierarchy
|
||||||
|
- Instance management
|
||||||
|
- Resource quotas
|
||||||
|
- Usage analytics
|
||||||
|
|
||||||
#### Storage Architecture
|
### Communication Infrastructure
|
||||||
- **Relational Data (PostgreSQL)**
|
- WebRTC rooms
|
||||||
- Customer-based sharding
|
- Real-time messaging
|
||||||
- Optimized table partitioning
|
- Media processing
|
||||||
- Read replica distribution
|
- Video conferencing
|
||||||
|
|
||||||
- **Real-time Data (TiKV)**
|
|
||||||
- Distributed key-value storage
|
|
||||||
- High-performance lookups
|
|
||||||
- Real-time data access
|
|
||||||
|
|
||||||
- **Caching Layer (Redis)**
|
|
||||||
- Session management
|
|
||||||
- Rate limiting implementation
|
|
||||||
- Temporary data storage
|
|
||||||
|
|
||||||
#### Message Processing
|
|
||||||
- **Event Processing (Kafka)**
|
|
||||||
- Sharded topic management
|
|
||||||
- Efficient message routing
|
|
||||||
- Real-time event streaming
|
|
||||||
|
|
||||||
- **Real-time Updates (Redis Pub/Sub)**
|
|
||||||
- Presence management
|
|
||||||
- Status synchronization
|
|
||||||
- Instant notifications
|
|
||||||
|
|
||||||
#### Media Infrastructure
|
### Storage Architecture
|
||||||
- **WebRTC Services**
|
```sql
|
||||||
- Media server clustering
|
-- Customer Sharding Example
|
||||||
- Track multiplexing
|
CREATE TABLE customers (
|
||||||
- Real-time processing
|
id UUID PRIMARY KEY,
|
||||||
- Efficient media storage
|
name TEXT,
|
||||||
|
subscription_tier TEXT,
|
||||||
|
status TEXT,
|
||||||
|
max_instances INTEGER
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
### 4. Operational Excellence
|
### Message Processing
|
||||||
|
```rust
|
||||||
|
// Kafka Producer Example
|
||||||
|
pub async fn publish<T: Serialize>(
|
||||||
|
&self,
|
||||||
|
topic: &str,
|
||||||
|
key: &str,
|
||||||
|
message: &T,
|
||||||
|
) -> Result<()>
|
||||||
|
```
|
||||||
|
|
||||||
#### Monitoring
|
## 🛠 Installation
|
||||||
- System health metrics
|
|
||||||
- Resource utilization tracking
|
### Prerequisites
|
||||||
- Performance analytics
|
- Rust 1.70+
|
||||||
- Quality assessments
|
- Kubernetes cluster
|
||||||
- Error monitoring
|
- PostgreSQL 13+
|
||||||
|
- Redis 6+
|
||||||
|
- Kafka 3.0+
|
||||||
|
- GStreamer
|
||||||
|
|
||||||
|
### Kubernetes Setup
|
||||||
|
```bash
|
||||||
|
# Initialize cluster
|
||||||
|
./setup-k8s.sh
|
||||||
|
|
||||||
|
# Deploy platform
|
||||||
|
./deploy.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### Build & Run
|
||||||
|
```bash
|
||||||
|
# Build all services
|
||||||
|
cargo build --workspace
|
||||||
|
|
||||||
|
# Run tests
|
||||||
|
cargo test --workspace
|
||||||
|
|
||||||
|
# Start API service
|
||||||
|
cargo run -p gb-api
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📊 Monitoring & Operations
|
||||||
|
|
||||||
|
### Health Metrics
|
||||||
|
- System performance
|
||||||
|
- Resource utilization
|
||||||
|
- Error rates
|
||||||
- Latency tracking
|
- Latency tracking
|
||||||
|
|
||||||
#### Scaling Operations
|
### Scaling Operations
|
||||||
- Automated scaling rules
|
- Auto-scaling rules
|
||||||
- Dynamic shard management
|
- Shard management
|
||||||
- Intelligent load balancing
|
- Load balancing
|
||||||
- Robust failover systems
|
- Failover systems
|
||||||
- Seamless data migration
|
|
||||||
|
|
||||||
#### Security Framework
|
## 🔒 Security
|
||||||
- Multi-factor authentication
|
|
||||||
- Role-based authorization
|
### Authentication & Authorization
|
||||||
- Rate limit enforcement
|
- Multi-factor auth
|
||||||
|
- Role-based access
|
||||||
|
- Rate limiting
|
||||||
- End-to-end encryption
|
- End-to-end encryption
|
||||||
- Comprehensive audit logging
|
|
||||||
|
|
||||||
## Implementation Specifications
|
### Data Protection
|
||||||
|
- Tenant isolation
|
||||||
|
- Encryption at rest
|
||||||
|
- Secure communications
|
||||||
|
- Audit logging
|
||||||
|
|
||||||
### Technology Stack
|
## 🚀 Development
|
||||||
- **Core Services**: Rust for performance-critical components
|
|
||||||
- **Benefits**:
|
|
||||||
- Maximum performance
|
|
||||||
- Memory safety guarantees
|
|
||||||
- Reliable concurrency
|
|
||||||
- System stability
|
|
||||||
|
|
||||||
### Sharding Strategy
|
### Project Structure
|
||||||
- Customer-ID based sharding
|
```
|
||||||
- Instance-level isolation
|
general-bots/
|
||||||
- Geographic distribution
|
├── gb-api/ # API service
|
||||||
- Data locality optimization
|
├── gb-core/ # Core functionality
|
||||||
|
├── gb-media/ # Media processing
|
||||||
|
├── gb-messaging/ # Message brokers
|
||||||
|
├── gb-storage/ # Data storage
|
||||||
|
├── gb-utils/ # Utilities
|
||||||
|
├── k8s/ # Kubernetes configs
|
||||||
|
└── migrations/ # DB migrations
|
||||||
|
```
|
||||||
|
|
||||||
### Performance Targets
|
### Configuration
|
||||||
- Billion+ concurrent connections
|
```env
|
||||||
- Millisecond-level message delivery
|
DATABASE_URL=postgresql://user:password@localhost:5432/gbdb
|
||||||
- 4K video streaming support
|
REDIS_URL=redis://localhost:6379
|
||||||
- Petabyte-scale data management
|
KAFKA_BROKERS=localhost:9092
|
||||||
|
RABBIT_URL=amqp://guest:guest@localhost:5672
|
||||||
|
```
|
||||||
|
|
||||||
### Reliability Standards
|
## 🌍 Deployment
|
||||||
- 99.99% platform availability
|
|
||||||
- Zero message loss guarantee
|
|
||||||
- Automated failover systems
|
|
||||||
- Multiple data redundancy
|
|
||||||
|
|
||||||
## Development Guidelines
|
### Global Infrastructure
|
||||||
|
- Edge presence
|
||||||
|
- Regional optimization
|
||||||
|
- Content delivery
|
||||||
|
- Traffic management
|
||||||
|
|
||||||
### Multi-tenant Considerations
|
### Disaster Recovery
|
||||||
1. Strict tenant isolation
|
- Automated backups
|
||||||
2. Resource quota management
|
- Multi-region failover
|
||||||
3. Security boundary enforcement
|
- Data replication
|
||||||
4. Performance isolation
|
- System redundancy
|
||||||
5. Independent scaling capabilities
|
|
||||||
6. Tenant-specific monitoring
|
|
||||||
|
|
||||||
### Bot Integration
|
## 🤝 Contributing
|
||||||
1. Automated workflow support
|
|
||||||
2. Custom bot development
|
|
||||||
3. AI/ML integration capabilities
|
|
||||||
4. Event-driven automation
|
|
||||||
5. Bot resource management
|
|
||||||
6. Performance monitoring
|
|
||||||
|
|
||||||
## Deployment Architecture
|
1. Fork repository
|
||||||
1. Global edge presence
|
2. Create feature branch
|
||||||
2. Regional data centers
|
3. Implement changes
|
||||||
3. Content delivery optimization
|
4. Add tests
|
||||||
4. Traffic management
|
5. Submit PR
|
||||||
5. Disaster recovery
|
|
||||||
6. Backup systems
|
## 📝 License
|
||||||
|
|
||||||
|
Licensed under terms specified in workspace configuration.
|
||||||
|
|
||||||
|
## 🆘 Support
|
||||||
|
|
||||||
|
### Issues
|
||||||
|
- Check existing issues
|
||||||
|
- Provide reproduction steps
|
||||||
|
- Include relevant logs
|
||||||
|
- Follow up on discussions
|
||||||
|
|
||||||
|
### Documentation
|
||||||
|
- API references
|
||||||
|
- Integration guides
|
||||||
|
- Deployment docs
|
||||||
|
- Best practices
|
||||||
|
|
||||||
|
## 🔮 Roadmap
|
||||||
|
|
||||||
|
### Short Term
|
||||||
|
- Enhanced media processing
|
||||||
|
- Additional messaging protocols
|
||||||
|
- Improved scalability
|
||||||
|
- Extended monitoring
|
||||||
|
|
||||||
|
### Long Term
|
||||||
|
- AI/ML integration
|
||||||
|
- Advanced analytics
|
||||||
|
- Global expansion
|
||||||
|
- Enterprise features
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
Built with ❤️ from Brazil, using Rust for maximum performance and reliability.
|
||||||
|
|
|
||||||
|
|
@ -15,8 +15,6 @@ kubectl apply -f k8s/base/monitoring.yaml
|
||||||
# Deploy application components
|
# Deploy application components
|
||||||
kubectl apply -f k8s/base/api.yaml
|
kubectl apply -f k8s/base/api.yaml
|
||||||
kubectl apply -f k8s/base/webrtc.yaml
|
kubectl apply -f k8s/base/webrtc.yaml
|
||||||
kubectl apply -f k8s/base/vm.yaml
|
|
||||||
kubectl apply -f k8s/base/nlp.yaml
|
|
||||||
kubectl apply -f k8s/base/image.yaml
|
kubectl apply -f k8s/base/image.yaml
|
||||||
kubectl apply -f k8s/base/document.yaml
|
kubectl apply -f k8s/base/document.yaml
|
||||||
|
|
||||||
|
|
@ -25,7 +23,7 @@ kubectl apply -f k8s/base/ingress.yaml
|
||||||
|
|
||||||
# Create DB.
|
# Create DB.
|
||||||
|
|
||||||
cargo run -p gb-migrations --bin migrations
|
#cargo run -p gb-migrations --bin migrations
|
||||||
|
|
||||||
echo "Deployment completed successfully!"
|
echo "Deployment completed successfully!"
|
||||||
echo "Please wait for all pods to be ready..."
|
echo "Please wait for all pods to be ready..."
|
||||||
|
|
|
||||||
128
setupk.sh
Executable file
128
setupk.sh
Executable file
|
|
@ -0,0 +1,128 @@
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# Enable error handling
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Function to check command status
|
||||||
|
check_status() {
|
||||||
|
if [ $? -eq 0 ]; then
|
||||||
|
echo "✅ $1 successful"
|
||||||
|
else
|
||||||
|
echo "❌ $1 failed"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
echo "🚀 Starting Kubernetes installation..."
|
||||||
|
|
||||||
|
# Update system
|
||||||
|
echo "📦 Updating system packages..."
|
||||||
|
sudo apt-get update && sudo apt-get upgrade -y
|
||||||
|
check_status "System update"
|
||||||
|
|
||||||
|
# Install prerequisites
|
||||||
|
echo "📦 Installing prerequisites..."
|
||||||
|
sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
|
||||||
|
check_status "Prerequisites installation"
|
||||||
|
|
||||||
|
# Install containerd
|
||||||
|
echo "🐋 Installing containerd..."
|
||||||
|
sudo apt-get install -y containerd
|
||||||
|
check_status "Containerd installation"
|
||||||
|
|
||||||
|
# Configure containerd
|
||||||
|
echo "⚙️ Configuring containerd..."
|
||||||
|
sudo mkdir -p /etc/containerd
|
||||||
|
sudo containerd config default | sudo tee /etc/containerd/config.toml > /dev/null
|
||||||
|
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
|
||||||
|
sudo systemctl restart containerd
|
||||||
|
sudo systemctl enable containerd
|
||||||
|
check_status "Containerd configuration"
|
||||||
|
|
||||||
|
# Disable swap
|
||||||
|
echo "⚙️ Disabling swap..."
|
||||||
|
sudo swapoff -a
|
||||||
|
sudo sed -i '/swap/d' /etc/fstab
|
||||||
|
check_status "Swap disabled"
|
||||||
|
|
||||||
|
# Load kernel modules
|
||||||
|
echo "⚙️ Loading kernel modules..."
|
||||||
|
sudo modprobe overlay
|
||||||
|
sudo modprobe br_netfilter
|
||||||
|
check_status "Kernel modules loaded"
|
||||||
|
|
||||||
|
# Configure system settings
|
||||||
|
echo "⚙️ Configuring system settings..."
|
||||||
|
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
|
||||||
|
net.bridge.bridge-nf-call-iptables = 1
|
||||||
|
net.bridge.bridge-nf-call-ip6tables = 1
|
||||||
|
net.ipv4.ip_forward = 1
|
||||||
|
EOF
|
||||||
|
sudo sysctl --system
|
||||||
|
check_status "System settings configuration"
|
||||||
|
|
||||||
|
# Add Kubernetes repository
|
||||||
|
echo "📦 Adding Kubernetes repository..."
|
||||||
|
sudo mkdir -p /etc/apt/keyrings
|
||||||
|
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
|
||||||
|
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
|
||||||
|
check_status "Kubernetes repository addition"
|
||||||
|
|
||||||
|
# Install Kubernetes components
|
||||||
|
echo "📦 Installing Kubernetes components..."
|
||||||
|
sudo apt-get update
|
||||||
|
sudo apt-get install -y kubelet kubeadm kubectl
|
||||||
|
sudo apt-mark hold kubelet kubeadm kubectl
|
||||||
|
check_status "Kubernetes components installation"
|
||||||
|
|
||||||
|
# Initialize Kubernetes cluster
|
||||||
|
echo "🚀 Initializing Kubernetes cluster..."
|
||||||
|
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
|
||||||
|
check_status "Kubernetes initialization"
|
||||||
|
|
||||||
|
# Configure kubectl
|
||||||
|
echo "⚙️ Configuring kubectl..."
|
||||||
|
mkdir -p $HOME/.kube
|
||||||
|
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
|
||||||
|
sudo chown $(id -u):$(id -g) $HOME/.kube/config
|
||||||
|
check_status "kubectl configuration"
|
||||||
|
|
||||||
|
# Install Flannel network plugin
|
||||||
|
echo "🔌 Installing Flannel network plugin..."
|
||||||
|
kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
|
||||||
|
check_status "Flannel installation"
|
||||||
|
|
||||||
|
# Allow scheduling on control-plane node
|
||||||
|
echo "⚙️ Enabling workload scheduling on control-plane..."
|
||||||
|
kubectl taint nodes --all node-role.kubernetes.io/control-plane- || true
|
||||||
|
kubectl taint nodes --all node-role.kubernetes.io/master- || true
|
||||||
|
check_status "Node configuration"
|
||||||
|
|
||||||
|
# Verify installation
|
||||||
|
echo "🔍 Verifying installation..."
|
||||||
|
kubectl get nodes
|
||||||
|
check_status "Node verification"
|
||||||
|
|
||||||
|
echo "✨ Kubernetes installation completed successfully!"
|
||||||
|
echo "🔍 Cluster status:"
|
||||||
|
kubectl cluster-info
|
||||||
|
echo "📝 Node status:"
|
||||||
|
kubectl get nodes
|
||||||
|
|
||||||
|
# Save cluster join command if needed
|
||||||
|
echo "💾 Saving cluster join command..."
|
||||||
|
sudo kubeadm token create --print-join-command > $HOME/k8s_join_command.txt
|
||||||
|
chmod 600 $HOME/k8s_join_command.txt
|
||||||
|
echo "Join command saved to $HOME/k8s_join_command.txt"
|
||||||
|
|
||||||
|
echo "
|
||||||
|
✅ Installation complete!
|
||||||
|
To start using your cluster:
|
||||||
|
kubectl get nodes
|
||||||
|
kubectl get pods --all-namespaces
|
||||||
|
|
||||||
|
To reset the cluster if needed:
|
||||||
|
sudo kubeadm reset
|
||||||
|
sudo rm -rf /etc/cni/net.d
|
||||||
|
sudo rm -rf $HOME/.kube/config
|
||||||
|
"
|
||||||
Loading…
Add table
Reference in a new issue