semaphore/README.md

Semaphore UI Runner Setup Guide

This guide explains how to set up and manage Semaphore UI runners for parallel task execution.

Overview

Semaphore UI supports remote runners to execute tasks in parallel across multiple machines. This setup allows you to:

• Run multiple tasks simultaneously
• Distribute workload across different servers
• Scale your automation infrastructure

Current Configuration

• Semaphore UI URL: https://ansible.phx-erp.de
• Max Parallel Tasks: 3
• Registration Token: Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h
• Database: PostgreSQL (ansible-postgres container)

Prerequisites

• Docker and Docker Compose installed
• Access to the Semaphore UI web interface
• Network connectivity between runner and Semaphore server

Setting Up a New Runner

Method 1: Using the Existing Docker Container (Recommended)

If you want to add another runner using the same container setup:

Step 1: Access the Semaphore Container

# Navigate to your project directory
cd /opt/phx

# Access the Semaphore container
docker exec -it ansible-semaphore /bin/bash

Step 2: Register the Runner

# Inside the container, register a new runner
echo "Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h" | semaphore runner register --config /etc/semaphore/config.json --stdin-registration-token

Step 3: Start the Runner

# Start the runner in the background
semaphore runner start --config /etc/semaphore/config.json &

Step 4: Verify Runner Status

# Check if the runner is running
ps aux | grep "semaphore runner"

Method 2: Setting Up a Remote Runner on Another Machine

Step 1: Install Semaphore Runner

On the remote machine where you want to run tasks:

# Download the latest Semaphore binary
wget https://github.com/semaphoreui/semaphore/releases/latest/download/semaphore_linux_amd64.tar.gz

# Extract the binary
tar -xzf semaphore_linux_amd64.tar.gz

# Make it executable
chmod +x semaphore

# Move to a system path
sudo mv semaphore /usr/local/bin/

Step 2: Create Runner Configuration

Create a configuration file for the remote runner:

# Create runner config directory
sudo mkdir -p /etc/semaphore

# Create runner configuration file
sudo tee /etc/semaphore/config.runner.json > /dev/null <<EOF
{
  "runner": {
    "registration_token": "Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h",
    "api_url": "https://ansible.phx-erp.de"
  }
}
EOF

Step 3: Register the Remote Runner

# Register the runner with the Semaphore server
echo "Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h" | semaphore runner register --config /etc/semaphore/config.runner.json --stdin-registration-token

Step 4: Start the Remote Runner

# Start the runner as a service
semaphore runner start --config /etc/semaphore/config.runner.json

Method 3: Using Docker on Remote Machine

Step 1: Create Docker Compose for Runner

On the remote machine, create a docker-compose.runner.yml:

version: '3.8'
services:
  semaphore-runner:
    image: semaphoreui/semaphore:latest
    container_name: semaphore-runner
    restart: unless-stopped
    environment:
      - SEMAPHORE_RUNNER_REGISTRATION_TOKEN=Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h
      - SEMAPHORE_RUNNER_API_URL=https://ansible.phx-erp.de
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock  # If you need Docker-in-Docker
      - ./runner-config:/etc/semaphore
    command: >
      sh -c "
        echo 'Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h' | 
        semaphore runner register --stdin-registration-token &&
        semaphore runner start
      "

Step 2: Start the Runner Container

# Start the runner container
docker-compose -f docker-compose.runner.yml up -d

Managing Runners

Viewing Registered Runners

# Access the Semaphore container
docker exec -it ansible-semaphore /bin/bash

# List registered runners (if supported by your version)
semaphore runner list --config /etc/semaphore/config.json

Stopping a Runner

# Stop runner process
docker exec ansible-semaphore pkill -f "semaphore runner start"

# Or if running on remote machine
pkill -f "semaphore runner start"

Unregistering a Runner

# Unregister a runner
semaphore runner unregister --config /etc/semaphore/config.json

Troubleshooting

Common Issues

1. Runner Registration Fails

Error: registration token cannot be empty

Solution: Ensure you're using the correct token and method:

echo "Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h" | semaphore runner register --config /etc/semaphore/config.json --stdin-registration-token

2. HTTP 400 Errors When Starting Runner

Error: error status code 400 when checking for new jobs

Possible Causes:

• Network connectivity issues
• Incorrect API URL configuration
• Server not ready to accept runners

Solutions:

1. Verify network connectivity to https://ansible.phx-erp.de
2. Check if Semaphore UI is accessible
3. Ensure the server is properly configured for remote runners

3. Runner Configuration Issues

Error: Configuration validation fails

Solution: Verify your configuration file structure:

{
  "runner": {
    "registration_token": "Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h",
    "api_url": "https://ansible.phx-erp.de"
  }
}

Debugging Runner Issues

View Runner Logs

# View recent logs
docker logs ansible-semaphore --tail 50

# Follow logs in real-time
docker logs ansible-semaphore --follow

# View logs with timestamps
docker logs ansible-semaphore --timestamps

# View logs from specific time
docker logs ansible-semaphore --since 10m

# Filter for runner-specific errors
docker logs ansible-semaphore | grep -E "(runner|error|400|invalid)"

# Filter for specific error types
docker logs ansible-semaphore | grep "invalid public key"
docker logs ansible-semaphore | grep "error status code"

Enable Debug Logging

# Start runner with debug logging
semaphore runner start --config /etc/semaphore/config.json --log-level DEBUG

Check Runner Status Inside Container

# Access container and check processes
docker exec -it ansible-semaphore /bin/bash
ps aux | grep runner

# Check system logs inside container
journalctl -f  # if systemd is available

Parallel Task Configuration

Current Settings

Your Semaphore configuration supports:

• Max Parallel Tasks: 3 (configurable in config.json)
• Max Tasks per Template: 3
• Remote Runner Support: Enabled

Increasing Parallel Capacity

To handle more parallel tasks:

1. Add More Runners: Set up additional runner machines
2. Increase Limits: Update max_parallel_tasks in config.json
3. Optimize Resources: Ensure runners have adequate CPU/memory

Example: Scaling to 5 Parallel Tasks

# Update configuration
sed -i 's/"max_parallel_tasks": 3/"max_parallel_tasks": 5/' /opt/phx/semaphore-config/config/config.json

# Restart Semaphore
docker-compose restart semaphore

Security Considerations

1. Token Security: Keep your registration token secure
2. Network Security: Use VPN or secure networks for remote runners
3. Access Control: Limit runner access to necessary resources only
4. Regular Updates: Keep Semaphore and runners updated

Monitoring and Maintenance

Health Checks

# Check if runners are responding
curl -k https://ansible.phx-erp.de/api/health

# Monitor runner processes
docker exec ansible-semaphore ps aux | grep runner

Log Rotation

Configure log rotation to prevent disk space issues:

# Add to crontab for log cleanup
0 2 * * * docker exec ansible-semaphore find /tmp -name "*.log" -mtime +7 -delete

Support and Resources

• Semaphore Documentation: https://docs.semaphoreui.com/
• GitHub Repository: https://github.com/semaphoreui/semaphore
• Configuration Reference: https://docs.semaphoreui.com/administration-guide/configuration/
• Runner Documentation: https://docs.semaphoreui.com/administration-guide/runners/

Quick Commands Reference

# Register runner
echo "TOKEN" | semaphore runner register --config /etc/semaphore/config.json --stdin-registration-token

# Start runner
semaphore runner start --config /etc/semaphore/config.json

# Stop runner
pkill -f "semaphore runner start"

# Unregister runner
semaphore runner unregister --config /etc/semaphore/config.json

# Check runner status
ps aux | grep "semaphore runner"

# View logs
docker logs ansible-semaphore

---

Note: Replace TOKEN with your actual registration token: Z*!!MPPi68C3Vganq2BbV*PBNTxxtaLCfk9M.XG@z!h