Mikolaj Wojciech Gorski
44b45b7212
🎭 Core Transformation: - Reframe project as AI companion bot with Kasane Teto personality - Focus on natural conversation, multimodal interaction, and character roleplay - Position video recording as one tool in AI toolkit rather than main feature 🏗️ Architecture Improvements: - Refactor messageCreate.js into modular command system (35 lines vs 310+) - Create dedicated videoRecording service with clean API - Implement commandHandler for extensible command routing - Add centralized configuration system (videoConfig.js) - Separate concerns: events, services, config, documentation 📚 Documentation Overhaul: - Consolidate scattered READMEs into organized docs/ directory - Create comprehensive documentation covering: * AI architecture and capabilities * Natural interaction patterns and personality * Setup guides for AI services and Docker deployment * Commands reference focused on conversational AI * Troubleshooting and development guidelines - Transform root README into compelling AI companion overview 🤖 AI-Ready Foundation: - Document integration points for: * Language models (GPT-4/Claude) for conversation * Vision models (GPT-4V/CLIP) for image analysis * Voice synthesis (ElevenLabs) for speaking * Memory systems for conversation continuity * Personality engine for character consistency 🔧 Technical Updates: - Integrate custom discord.js-selfbot-v13 submodule with enhanced functionality - Update package.json dependencies for AI and multimedia capabilities - Maintain Docker containerization with improved architecture - Add development and testing infrastructure 📖 New Documentation Structure: docs/ ├── README.md (documentation hub) ├── setup.md (installation & AI configuration) ├── interactions.md (how to chat with Teto) ├── ai-architecture.md (technical AI systems overview) ├── commands.md (natural language interactions) ├── personality.md (character understanding) ├── development.md (contributing guidelines) ├── troubleshooting.md (problem solving) └── [additional specialized guides] ✨ This update transforms the project from a simple recording bot into a foundation for an engaging AI companion that can naturally interact through text, voice, and visual content while maintaining authentic Kasane Teto personality traits.
246 lines
No EOL
9.2 KiB
Markdown
246 lines
No EOL
9.2 KiB
Markdown
# Video Recording Feature
|
|
|
|
This document explains how to use the enhanced video recording functionality in the Teto Discord bot running in Docker containers.
|
|
|
|
> **🐳 Docker Deployment**: This bot is designed to run in Docker containers. See [DOCKER_VOICE_SETUP.md](DOCKER_VOICE_SETUP.md) for detailed Docker-specific setup instructions.
|
|
|
|
## Features
|
|
|
|
### 1. DM Pickup
|
|
When you send a direct message to the bot containing "teto", the bot will respond with "I'm here! What do you need?"
|
|
|
|
**Usage:**
|
|
- Send a DM to the bot with any message containing "teto"
|
|
- Example: "Hey teto, are you there?"
|
|
|
|
### 2. Video Recording
|
|
The bot can join voice channels and record video/audio from users in real-time.
|
|
|
|
**Usage:**
|
|
1. Join a voice channel in a Discord server
|
|
2. Type `xbox record that` in any text channel
|
|
3. The bot will join your voice channel and start video recording
|
|
4. Recording automatically stops after 30 seconds (demo limit)
|
|
5. Video is saved directly to MKV format (Matroska container)
|
|
|
|
**Stop Recording:**
|
|
- Type `stop recording` or `xbox stop` to manually stop recording
|
|
|
|
### 3. Status Check
|
|
Check the bot's current status and recent recordings.
|
|
|
|
**Usage:**
|
|
- Type `teto status` to see active recordings, file count, and recent files
|
|
|
|
## Commands
|
|
|
|
| Command | Description | Location |
|
|
|---------|-------------|----------|
|
|
| `teto` | Bot pickup response | DM only |
|
|
| `xbox record that` | Start video recording | Server channels |
|
|
| `stop recording` | Stop current recording | Server channels |
|
|
| `xbox stop` | Stop current recording (alternative) | Server channels |
|
|
| `teto status` | Show bot status and recording info | Any channel |
|
|
| `hello teto` | Basic greeting | Any channel |
|
|
|
|
> **📁 File Storage**: In Docker, video recordings are saved to `/tmp/output/` inside the container, which is mounted to `./output/` on your host system.
|
|
|
|
## Technical Details
|
|
|
|
### Recording Process
|
|
1. **Join Voice Channel**: Bot joins the requested voice channel
|
|
2. **Stream Connection**: Creates a stream connection to the target user
|
|
3. **Video Capture**: Records video/audio using Discord's video stream API
|
|
4. **Direct Output**: Saves directly to MKV format using FFmpeg
|
|
5. **File Management**: Creates final video file in Matroska container format
|
|
|
|
### Video Specifications
|
|
- **Container**: MKV (Matroska) - supports both video and audio
|
|
- **Audio**: Opus codec, 48kHz sample rate
|
|
- **Video**: VP8/H.264 (Discord's formats)
|
|
- **Processing**: Real-time FFmpeg encoding
|
|
- **Output Format**: .mkv files with video and audio streams
|
|
|
|
### File Naming
|
|
- **Video files**: `recording-YYYY-MM-DDTHH-MM-SS-sssZ.mkv` (final output)
|
|
- **Container Location**: `/tmp/output/` (inside Docker container)
|
|
- **Host Location**: `./output/` (accessible from your host system via volume mount)
|
|
|
|
## Requirements
|
|
|
|
### Dependencies (Docker Container)
|
|
- `opusscript` - For audio processing ✅ (installed via npm)
|
|
- `discord.js-selfbot-v13` - Main bot library ✅
|
|
- `ffmpeg` - For video/audio processing ✅ (installed in Docker image)
|
|
- `libopus0` & `libopus-dev` - Opus codec support ✅ (installed in Docker image)
|
|
- `libsodium23` & `libsodium-dev` - Encryption for Discord voice ✅ (installed in Docker image)
|
|
- `sodium` - Native encryption library ✅ (installed via npm)
|
|
- `pulseaudio` - Audio system ✅ (installed in Docker image)
|
|
|
|
### System Requirements
|
|
- Docker and Docker Compose installed on host
|
|
- Volume mount: `./output:/tmp/output` configured
|
|
- Sufficient disk space in host `./output/` directory (video files are larger)
|
|
- Container runs with SYS_ADMIN capability for Discord functionality
|
|
- Build tools for compiling native encryption libraries
|
|
|
|
### Discord Permissions
|
|
The bot needs the following permissions:
|
|
- View channels
|
|
- Send messages
|
|
- Connect to voice channels
|
|
- Use voice activity (to receive audio)
|
|
|
|
## Usage Examples
|
|
|
|
### Docker Container Setup
|
|
```bash
|
|
# Start the bot container
|
|
docker-compose up --build
|
|
|
|
# Check container status
|
|
docker-compose ps
|
|
```
|
|
|
|
### Basic Recording Session
|
|
```
|
|
You: xbox record that
|
|
Bot: Trying to start recording... in General Voice
|
|
Bot: Starting recording... 📹
|
|
[30 seconds later]
|
|
Bot: Recording stopped automatically after 30 seconds (demo limit). Duration: 30s. Processing video file...
|
|
Bot: 🎥 Video recording saved as: recording-2024-01-15T14-30-45-123Z.mkv
|
|
Bot: 📁 File location: Container: /tmp/output/ | Host: ./output/
|
|
```
|
|
|
|
### Manual Stop
|
|
```
|
|
You: xbox record that
|
|
Bot: Starting recording... 📹
|
|
You: xbox stop
|
|
Bot: Recording stopped by user. Duration: 15s. Processing video file...
|
|
Bot: 🎥 Video recording saved as: recording-2024-01-15T14-31-00-456Z.mkv
|
|
```
|
|
|
|
### Status Check
|
|
```
|
|
You: teto status
|
|
Bot: 🤖 **Teto Status Report (Docker)**
|
|
📹 Active recordings: 0
|
|
📁 Container output: /tmp/output
|
|
📁 Host output: ./output (volume mounted)
|
|
🎥 Video files: 5
|
|
📋 Recent files: recording-2024-01-15T14-30-45-123Z.mkv, recording-2024-01-15T14-31-00-456Z.mkv
|
|
⏰ Bot uptime: 3600s
|
|
🐳 Running in Docker container
|
|
```
|
|
|
|
## Advanced Features
|
|
|
|
### User-Specific Recording
|
|
- Records video/audio from the user who requested the recording
|
|
- Uses Discord's stream connection API for direct capture
|
|
- Creates single video file with both video and audio streams
|
|
|
|
### Direct Video Output
|
|
- Video is processed directly to MKV format using FFmpeg
|
|
- No temporary file conversion needed
|
|
- Matroska container supports both video and audio streams
|
|
|
|
### Error Handling
|
|
- Graceful handling of voice connection failures
|
|
- Automatic cleanup of resources on errors
|
|
- Informative error messages for troubleshooting
|
|
|
|
## Docker Troubleshooting
|
|
|
|
### Container Issues
|
|
|
|
1. **Bot doesn't join voice channel**
|
|
- Check container logs: `docker-compose logs teto_ai`
|
|
- Ensure bot has "Connect" permission for the voice channel
|
|
- Verify container has proper Discord dependencies installed
|
|
|
|
2. **No video/audio captured**
|
|
- Confirm target user has video/audio activity during recording
|
|
- Check if FFmpeg is installed in container: `docker exec -it teto_ai which ffmpeg`
|
|
- Verify Opus libraries are working: `docker exec -it teto_ai ffmpeg -codecs | grep opus`
|
|
- Check encryption libraries: `docker exec -it teto_ai ldconfig -p | grep sodium`
|
|
|
|
3. **Files not appearing on host**
|
|
- Verify volume mount: `docker inspect teto_ai | grep -A 5 "Mounts"`
|
|
- Check permissions: `ls -la ./output/`
|
|
- Ensure container can write to mounted directory
|
|
|
|
4. **Video processing fails**
|
|
- Check container logs for FFmpeg errors
|
|
- Verify container has sufficient disk space
|
|
- Ensure tmpfs has enough space (512MB allocated)
|
|
- Check libsodium encryption libraries are installed
|
|
|
|
### Common Docker Commands
|
|
```bash
|
|
# View real-time logs
|
|
docker-compose logs -f teto_ai
|
|
|
|
# Access container shell
|
|
docker exec -it teto_ai bash
|
|
|
|
# Restart container
|
|
docker-compose restart teto_ai
|
|
|
|
# Check container resource usage
|
|
docker stats teto_ai
|
|
```
|
|
|
|
### Debug Information
|
|
- All recording events are logged with `[Docker]` prefix
|
|
- Container logs available via `docker-compose logs`
|
|
- File operations tracked in both container and host context
|
|
- Voice connection status reported with container-specific details
|
|
|
|
### File Management
|
|
- **Container**: MKV files created directly in `/tmp/output/`
|
|
- **Host**: Final video files persist in `./output/` directory
|
|
- **Cleanup**: Empty recordings automatically removed
|
|
- **Monitoring**: Use `teto status` and `docker exec` to check disk usage
|
|
|
|
## Security & Privacy Notes
|
|
|
|
- **Video Storage**: Recordings stored in Docker container at `/tmp/output/`, accessible on host at `./output/`
|
|
- **Container Security**: Bot runs as non-root user (`bot`) with limited capabilities
|
|
- **Data Retention**: Video files persist in host volume until manually deleted
|
|
- **Privacy**: Only captures video/audio from the specific user who requested recording
|
|
- **Encryption**: Uses libsodium for Discord voice protocol encryption
|
|
- **Isolation**: Container provides filesystem isolation from host system
|
|
|
|
## Legal Considerations
|
|
|
|
- **Consent**: Ensure all participants consent to being recorded
|
|
- **Local Laws**: Recording may be subject to local privacy laws
|
|
- **Discord TOS**: Using selfbots may violate Discord's Terms of Service
|
|
- **Educational Use**: This implementation is for educational purposes only
|
|
- **Data Location**: Consider data sovereignty if deploying containers in cloud environments
|
|
|
|
## Performance Notes (Docker Environment)
|
|
|
|
- **CPU Usage**: Video processing requires moderate to high CPU resources within container
|
|
- **Memory Usage**: ~100-200MB base container memory + video/audio buffers
|
|
- **Disk Usage**: MKV files ~15-30MB per minute, stored in host volume
|
|
- **Container Resources**: 512MB tmpfs allocated for temporary video processing
|
|
- **Network**: Standard Discord voice/video bandwidth, no additional container overhead
|
|
|
|
## Future Enhancements
|
|
|
|
Potential improvements:
|
|
- Multi-user video recording support
|
|
- Multiple video format support (MP4, WebM, etc.)
|
|
- Configurable recording duration limits
|
|
- Video quality settings and bitrate control
|
|
- Cloud storage integration (S3, etc.) from container
|
|
- Real-time video effects processing
|
|
- Container orchestration with Kubernetes
|
|
- Multi-container deployment for scaling
|
|
- Persistent volume claims for enterprise deployments
|
|
|
|
> **🐳 For detailed Docker setup instructions, troubleshooting, and deployment guides, see [DOCKER_VOICE_SETUP.md](DOCKER_VOICE_SETUP.md)** |