📚 ADD: Introduce Semaphore UI Runner Setup Guide and update configuration settings for parallel task execution

2025-10-21 11:52:07 +02:00
parent aa65b7e30c
commit 440a4c56a3
2 changed files with 364 additions and 58 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,323 @@
+# Semaphore UI Runner Setup Guide
+
+This guide explains how to set up and manage Semaphore UI runners for parallel task execution.
+
+## Overview
+
+Semaphore UI supports remote runners to execute tasks in parallel across multiple machines. This setup allows you to:
+- Run multiple tasks simultaneously
+- Distribute workload across different servers
+- Scale your automation infrastructure
+
+## Current Configuration
+
+- **Semaphore UI URL**: https://ansible.phx-erp.de
+- **Max Parallel Tasks**: 3
+- **Registration Token**: `Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h`
+- **Database**: PostgreSQL (ansible-postgres container)
+
+## Prerequisites
+
+- Docker and Docker Compose installed
+- Access to the Semaphore UI web interface
+- Network connectivity between runner and Semaphore server
+
+## Setting Up a New Runner
+
+### Method 1: Using the Existing Docker Container (Recommended)
+
+If you want to add another runner using the same container setup:
+
+#### Step 1: Access the Semaphore Container
+```bash
+# Navigate to your project directory
+cd /opt/phx
+
+# Access the Semaphore container
+docker exec -it ansible-semaphore /bin/bash
+```
+
+#### Step 2: Register the Runner
+```bash
+# Inside the container, register a new runner
+echo "Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h" | semaphore runner register --config /etc/semaphore/config.json --stdin-registration-token
+```
+
+#### Step 3: Start the Runner
+```bash
+# Start the runner in the background
+semaphore runner start --config /etc/semaphore/config.json &
+```
+
+#### Step 4: Verify Runner Status
+```bash
+# Check if the runner is running
+ps aux | grep "semaphore runner"
+```
+
+### Method 2: Setting Up a Remote Runner on Another Machine
+
+#### Step 1: Install Semaphore Runner
+On the remote machine where you want to run tasks:
+
+```bash
+# Download the latest Semaphore binary
+wget https://github.com/semaphoreui/semaphore/releases/latest/download/semaphore_linux_amd64.tar.gz
+
+# Extract the binary
+tar -xzf semaphore_linux_amd64.tar.gz
+
+# Make it executable
+chmod +x semaphore
+
+# Move to a system path
+sudo mv semaphore /usr/local/bin/
+```
+
+#### Step 2: Create Runner Configuration
+Create a configuration file for the remote runner:
+
+```bash
+# Create runner config directory
+sudo mkdir -p /etc/semaphore
+
+# Create runner configuration file
+sudo tee /etc/semaphore/config.runner.json > /dev/null <<EOF
+{
+  "runner": {
+    "registration_token": "Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h",
+    "api_url": "https://ansible.phx-erp.de"
+  }
+}
+EOF
+```
+
+#### Step 3: Register the Remote Runner
+```bash
+# Register the runner with the Semaphore server
+echo "Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h" | semaphore runner register --config /etc/semaphore/config.runner.json --stdin-registration-token
+```
+
+#### Step 4: Start the Remote Runner
+```bash
+# Start the runner as a service
+semaphore runner start --config /etc/semaphore/config.runner.json
+```
+
+### Method 3: Using Docker on Remote Machine
+
+#### Step 1: Create Docker Compose for Runner
+On the remote machine, create a `docker-compose.runner.yml`:
+
+```yaml
+version: '3.8'
+services:
+  semaphore-runner:
+    image: semaphoreui/semaphore:latest
+    container_name: semaphore-runner
+    restart: unless-stopped
+    environment:
+      - SEMAPHORE_RUNNER_REGISTRATION_TOKEN=Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h
+      - SEMAPHORE_RUNNER_API_URL=https://ansible.phx-erp.de
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock  # If you need Docker-in-Docker
+      - ./runner-config:/etc/semaphore
+    command: >
+      sh -c "
+        echo 'Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h' | 
+        semaphore runner register --stdin-registration-token &&
+        semaphore runner start
+      "
+```
+
+#### Step 2: Start the Runner Container
+```bash
+# Start the runner container
+docker-compose -f docker-compose.runner.yml up -d
+```
+
+## Managing Runners
+
+### Viewing Registered Runners
+```bash
+# Access the Semaphore container
+docker exec -it ansible-semaphore /bin/bash
+
+# List registered runners (if supported by your version)
+semaphore runner list --config /etc/semaphore/config.json
+```
+
+### Stopping a Runner
+```bash
+# Stop runner process
+docker exec ansible-semaphore pkill -f "semaphore runner start"
+
+# Or if running on remote machine
+pkill -f "semaphore runner start"
+```
+
+### Unregistering a Runner
+```bash
+# Unregister a runner
+semaphore runner unregister --config /etc/semaphore/config.json
+```
+
+## Troubleshooting
+
+### Common Issues
+
+#### 1. Runner Registration Fails
+**Error**: `registration token cannot be empty`
+
+**Solution**: Ensure you're using the correct token and method:
+```bash
+echo "Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h" | semaphore runner register --config /etc/semaphore/config.json --stdin-registration-token
+```
+
+#### 2. HTTP 400 Errors When Starting Runner
+**Error**: `error status code 400` when checking for new jobs
+
+**Possible Causes**:
+- Network connectivity issues
+- Incorrect API URL configuration
+- Server not ready to accept runners
+
+**Solutions**:
+1. Verify network connectivity to `https://ansible.phx-erp.de`
+2. Check if Semaphore UI is accessible
+3. Ensure the server is properly configured for remote runners
+
+#### 3. Runner Configuration Issues
+**Error**: Configuration validation fails
+
+**Solution**: Verify your configuration file structure:
+```json
+{
+  "runner": {
+    "registration_token": "Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h",
+    "api_url": "https://ansible.phx-erp.de"
+  }
+}
+```
+
+### Debugging Runner Issues
+
+#### View Runner Logs
+```bash
+# View recent logs
+docker logs ansible-semaphore --tail 50
+
+# Follow logs in real-time
+docker logs ansible-semaphore --follow
+
+# View logs with timestamps
+docker logs ansible-semaphore --timestamps
+
+# View logs from specific time
+docker logs ansible-semaphore --since 10m
+
+# Filter for runner-specific errors
+docker logs ansible-semaphore | grep -E "(runner|error|400|invalid)"
+
+# Filter for specific error types
+docker logs ansible-semaphore | grep "invalid public key"
+docker logs ansible-semaphore | grep "error status code"
+```
+
+#### Enable Debug Logging
+```bash
+# Start runner with debug logging
+semaphore runner start --config /etc/semaphore/config.json --log-level DEBUG
+```
+
+#### Check Runner Status Inside Container
+```bash
+# Access container and check processes
+docker exec -it ansible-semaphore /bin/bash
+ps aux | grep runner
+
+# Check system logs inside container
+journalctl -f  # if systemd is available
+```
+
+## Parallel Task Configuration
+
+### Current Settings
+Your Semaphore configuration supports:
+- **Max Parallel Tasks**: 3 (configurable in `config.json`)
+- **Max Tasks per Template**: 3
+- **Remote Runner Support**: Enabled
+
+### Increasing Parallel Capacity
+To handle more parallel tasks:
+
+1. **Add More Runners**: Set up additional runner machines
+2. **Increase Limits**: Update `max_parallel_tasks` in `config.json`
+3. **Optimize Resources**: Ensure runners have adequate CPU/memory
+
+### Example: Scaling to 5 Parallel Tasks
+```bash
+# Update configuration
+sed -i 's/"max_parallel_tasks": 3/"max_parallel_tasks": 5/' /opt/phx/semaphore-config/config/config.json
+
+# Restart Semaphore
+docker-compose restart semaphore
+```
+
+## Security Considerations
+
+1. **Token Security**: Keep your registration token secure
+2. **Network Security**: Use VPN or secure networks for remote runners
+3. **Access Control**: Limit runner access to necessary resources only
+4. **Regular Updates**: Keep Semaphore and runners updated
+
+## Monitoring and Maintenance
+
+### Health Checks
+```bash
+# Check if runners are responding
+curl -k https://ansible.phx-erp.de/api/health
+
+# Monitor runner processes
+docker exec ansible-semaphore ps aux | grep runner
+```
+
+### Log Rotation
+Configure log rotation to prevent disk space issues:
+```bash
+# Add to crontab for log cleanup
+0 2 * * * docker exec ansible-semaphore find /tmp -name "*.log" -mtime +7 -delete
+```
+
+## Support and Resources
+
+- **Semaphore Documentation**: https://docs.semaphoreui.com/
+- **GitHub Repository**: https://github.com/semaphoreui/semaphore
+- **Configuration Reference**: https://docs.semaphoreui.com/administration-guide/configuration/
+- **Runner Documentation**: https://docs.semaphoreui.com/administration-guide/runners/
+
+## Quick Commands Reference
+
+```bash
+# Register runner
+echo "TOKEN" | semaphore runner register --config /etc/semaphore/config.json --stdin-registration-token
+
+# Start runner
+semaphore runner start --config /etc/semaphore/config.json
+
+# Stop runner
+pkill -f "semaphore runner start"
+
+# Unregister runner
+semaphore runner unregister --config /etc/semaphore/config.json
+
+# Check runner status
+ps aux | grep "semaphore runner"
+
+# View logs
+docker logs ansible-semaphore
+```
+
+---
+
+**Note**: Replace `TOKEN` with your actual registration token: `Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h`