feat: Add CI/CD secrets configuration and documentation

- Add comprehensive CI/CD secrets section to .env.example - Create detailed CI-CD-SECRETS.md guide with: - Step-by-step secret setup instructions - Container registry authentication examples - PyPI token configuration - GitHub integration setup - Security best practices - Troubleshooting guide - Workflow-specific requirements Includes support for: - Docker Hub, GitLab, GitHub container registries - Automated PyPI publishing - Slack/Discord notifications - Kubernetes deployment secrets - Security scanning integration Updated .env template with placeholder values and detailed comments.
2025-09-15 02:26:02 -07:00
parent 25666a76cf
commit 03fa856199
3 changed files with 694 additions and 0 deletions
--- a/.env.example.new
+++ b/.env.example.new
@@ -0,0 +1,362 @@
+# UnitForge Environment Configuration
+# Copy this file to .env and modify the values as needed
+
+# =============================================================================
+# APPLICATION INFORMATION
+# =============================================================================
+
+APP_NAME=UnitForge
+APP_VERSION=1.0.0
+APP_DESCRIPTION="Create, validate, and manage systemd unit files"
+
+# =============================================================================
+# EXTERNAL LINKS
+# =============================================================================
+
+GITHUB_URL=https://github.com/will666/unitforge
+DOCUMENTATION_URL=https://unitforge.readthedocs.io/
+BUG_REPORTS_URL=https://github.com/will666/unitforge/issues
+
+# =============================================================================
+# CONTACT INFORMATION
+# =============================================================================
+
+CONTACT_EMAIL=contact@unitforge.dev
+
+# =============================================================================
+# APPLICATION SETTINGS
+# =============================================================================
+
+DEBUG=false
+ENVIRONMENT=production
+LOG_LEVEL=info
+
+# =============================================================================
+# SERVER CONFIGURATION
+# =============================================================================
+
+HOST=0.0.0.0
+PORT=8000
+RELOAD=false
+WORKERS=4
+
+# =============================================================================
+# API CONFIGURATION
+# =============================================================================
+
+API_TITLE="${APP_NAME}"
+API_VERSION="${APP_VERSION}"
+API_DESCRIPTION=${APP_DESCRIPTION}
+DOCS_URL="/api/docs"
+REDOC_URL="/api/redoc"
+
+# =============================================================================
+# SECURITY SETTINGS
+# =============================================================================
+
+# IMPORTANT: Generate a secure secret key for production
+SECRET_KEY=your-secret-key-change-in-production
+ALLOWED_HOSTS=["*"]
+
+# =============================================================================
+# CORS CONFIGURATION
+# =============================================================================
+
+# Use "*" for all origins or comma-separated list of allowed origins
+# Examples:
+# CORS_ORIGINS=*
+# CORS_ORIGINS=["https://yourdomain.com", "https://www.yourdomain.com"]
+CORS_ORIGINS=*
+
+# =============================================================================
+# DATABASE (Future Use)
+# =============================================================================
+
+# DATABASE_URL="sqlite:///./unitforge.db"
+
+# =============================================================================
+# FILE UPLOAD SETTINGS
+# =============================================================================
+
+# Maximum file upload size in bytes (1MB)
+MAX_UPLOAD_SIZE=1048576
+ALLOWED_EXTENSIONS=[".service", ".timer", ".socket", ".mount", ".target", ".path"]
+
+# =============================================================================
+# TEMPLATE SETTINGS
+# =============================================================================
+
+# Template cache time-to-live in seconds (5 minutes)
+TEMPLATE_CACHE_TTL=300
+# Validation cache time-to-live in seconds (1 minute)
+VALIDATION_CACHE_TTL=60
+
+# =============================================================================
+# DEVELOPMENT TOOLS (Override for Development)
+# =============================================================================
+
+# Test options (development only)
+# PYTEST_ADDOPTS="-v --tb=short --strict-markers"
+# COVERAGE_REPORT_DIR="htmlcov"
+# MYPY_CONFIG_FILE="pyproject.toml"
+
+# =============================================================================
+# UV PACKAGE MANAGER SETTINGS
+# =============================================================================
+
+# UV package manager cache directory
+# UV_CACHE_DIR=".uv-cache"
+# UV_PYTHON="python3"
+
+# =============================================================================
+# DOCKER SETTINGS
+# =============================================================================
+
+# Docker build settings
+# DOCKER_BUILDKIT=1
+# COMPOSE_DOCKER_CLI_BUILD=1
+
+# =============================================================================
+# CONTAINER REGISTRY CONFIGURATION
+# =============================================================================
+
+# Container registry URL for image publishing
+CONTAINER_REGISTRY_URL=https://your-registry.example.com/your-namespace/unitforge
+# Image tag for container builds
+CONTAINER_TAG=latest
+
+# =============================================================================
+# CI/CD SECRETS CONFIGURATION
+# =============================================================================
+#
+# These secrets are required for the Gitea CI/CD workflows to function properly.
+# Set these in your Gitea repository secrets settings:
+# Repository Settings > Secrets and Variables > Actions
+#
+# SECURITY WARNING: Never commit real secrets to version control!
+# Use this file as a template and set actual values in your CI/CD environment.
+
+# Container Registry Authentication
+# Required for: build-container.yml, release.yml, nightly.yml
+# Purpose: Push multi-arch container images to your registry
+# Format: Username for your container registry (Docker Hub, GitLab, etc.)
+CONTAINER_REGISTRY_USERNAME=your-registry-username-here
+# Format: Password or access token for your container registry
+CONTAINER_REGISTRY_PASSWORD=your-registry-password-here
+
+# PyPI Publishing (for releases)
+# Required for: release.yml (stable releases only)
+# Purpose: Automatically publish Python packages to PyPI
+# How to get: https://pypi.org/manage/account/token/
+# Format: pypi-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+PYPI_API_TOKEN=pypi-your-api-token-here
+
+# GitHub Integration (optional)
+# Required for: release.yml (GitHub releases)
+# Purpose: Create GitHub releases with artifacts and changelog
+# How to get: GitHub Settings > Developer settings > Personal access tokens
+# Permissions needed: repo, write:packages
+# Format: ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+GITHUB_TOKEN=ghp_your-github-token-here
+
+# Security Scanning (optional)
+# Required for: Enhanced security scanning features
+# Purpose: Access premium security scanning APIs (Snyk, etc.)
+# Format: API token from your security scanning service
+SECURITY_SCAN_TOKEN=your-security-scan-token-here
+
+# Notification Services (optional)
+# Purpose: Send build notifications to team communication channels
+# Slack webhook: Slack App > Incoming Webhooks
+SLACK_WEBHOOK_URL=https://hooks.slack.com/services/YOUR/WEBHOOK/URL
+# Discord webhook: Server Settings > Integrations > Webhooks
+DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/YOUR/WEBHOOK/URL
+# Email notifications
+EMAIL_NOTIFICATION_ADDRESS=notifications@yourdomain.com
+
+# Deployment Secrets (optional)
+# Purpose: Deploy to production/staging environments
+# SSH key for server deployments (base64 encoded private key)
+DEPLOYMENT_SSH_KEY=your-base64-encoded-ssh-key-here
+# Kubernetes service account token for cluster deployments
+KUBERNETES_TOKEN=your-kubernetes-service-account-token-here
+# API key for production environment access
+PRODUCTION_API_KEY=your-production-api-key-here
+
+# Additional CI/CD Configuration
+# Build cache settings (optional)
+BUILDX_CACHE_FROM=type=gha
+BUILDX_CACHE_TO=type=gha,mode=max
+# Container platforms to build for
+DOCKER_PLATFORMS=linux/amd64,linux/arm64
+# Registry push timeout (seconds)
+REGISTRY_TIMEOUT=300
+
+# =============================================================================
+# KUBERNETES INGRESS CONFIGURATION
+# =============================================================================
+
+# Ingress host for Kubernetes deployment (use nip.io for dynamic IP)
+# Replace 192.168.153.243 with your actual cluster/node IP
+INGRESS_HOST=app.unitforge.YOUR-IP.nip.io
+
+# Alternative ingress hosts (comma-separated)
+INGRESS_HOSTS_ADDITIONAL=unitforge.local,unitforge.pi
+
+# =============================================================================
+# PATHS
+# =============================================================================
+
+FRONTEND_DIR="frontend"
+BACKEND_DIR="backend"
+STATIC_DIR="frontend/static"
+TEMPLATES_DIR="frontend/templates"
+
+# =============================================================================
+# FEATURE FLAGS
+# =============================================================================
+
+ENABLE_API_METRICS=false
+ENABLE_REQUEST_LOGGING=true
+ENABLE_TEMPLATE_CACHING=true
+ENABLE_VALIDATION_CACHING=true
+
+# =============================================================================
+# PERFORMANCE SETTINGS
+# =============================================================================
+
+REQUEST_TIMEOUT=30
+KEEPALIVE_TIMEOUT=5
+MAX_CONNECTIONS=100
+
+# =============================================================================
+# LOGGING CONFIGURATION
+# =============================================================================
+
+LOG_FORMAT="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
+LOG_DATE_FORMAT="%Y-%m-%d %H:%M:%S"
+ACCESS_LOG=true
+
+# =============================================================================
+# DEVELOPMENT SERVER OPTIONS (Development Only)
+# =============================================================================
+
+# Development server auto-reload settings
+# AUTO_RELOAD=false
+# RELOAD_DIRS=["backend", "frontend"]
+# RELOAD_DELAY=0.25
+
+# =============================================================================
+# TESTING CONFIGURATION
+# =============================================================================
+
+# Testing configuration
+# TEST_DATABASE_URL="sqlite:///:memory:"
+# TEST_PARALLEL=true
+# TEST_COVERAGE_THRESHOLD=80
+
+# =============================================================================
+# CLI CONFIGURATION
+# =============================================================================
+
+CLI_VERBOSE=false
+CLI_COLOR=true
+CLI_PROGRESS=true
+
+# =============================================================================
+# VALIDATION SETTINGS
+# =============================================================================
+
+STRICT_VALIDATION=false
+SHOW_WARNINGS=true
+MAX_VALIDATION_ERRORS=50
+
+# =============================================================================
+# TEMPLATE GENERATION DEFAULTS
+# =============================================================================
+
+DEFAULT_USER="www-data"
+DEFAULT_GROUP="www-data"
+DEFAULT_RESTART_POLICY="on-failure"
+DEFAULT_WANTED_BY="multi-user.target"
+
+# =============================================================================
+# SECURITY HEADERS
+# =============================================================================
+
+SECURITY_HEADERS=true
+# HSTS max age in seconds (1 year)
+HSTS_MAX_AGE=31536000
+CSP_ENABLED=true
+
+# =============================================================================
+# MONITORING
+# =============================================================================
+
+HEALTH_CHECK_ENABLED=true
+METRICS_ENABLED=false
+TRACING_ENABLED=false
+
+# =============================================================================
+# ASSET OPTIMIZATION
+# =============================================================================
+
+HOT_RELOAD=false
+SOURCE_MAPS=false
+MINIFY_ASSETS=true
+COMPRESS_RESPONSES=true
+
+# =============================================================================
+# DEVELOPMENT TOOLS (Development Only)
+# =============================================================================
+
+# Development editor and browser
+# EDITOR="code"
+# BROWSER="firefox"
+
+# =============================================================================
+# GIT HOOKS (Development Only)
+# =============================================================================
+
+# Git hooks configuration
+# PRE_COMMIT_ENABLED=true
+# AUTO_FORMAT=true
+# AUTO_LINT=true
+
+# =============================================================================
+# DOCUMENTATION
+# =============================================================================
+
+DOCS_AUTO_RELOAD=false
+API_DOCS_ENABLED=true
+SWAGGER_UI_ENABLED=true
+REDOC_ENABLED=true
+
+# =============================================================================
+# DEVELOPMENT OVERRIDES
+# =============================================================================
+# Uncomment the following lines for development:
+
+# DEBUG=true
+# ENVIRONMENT=development
+# LOG_LEVEL=debug
+# HOST=127.0.0.1
+# RELOAD=true
+# WORKERS=1
+# API_TITLE="${APP_NAME} Development"
+# API_VERSION="${APP_VERSION}-dev"
+# SECRET_KEY="dev-secret-key-change-in-production"
+# ALLOWED_HOSTS=["localhost", "127.0.0.1", "0.0.0.0", "::1"]
+# CORS_ORIGINS=["http://localhost:3000", "http://localhost:8000", "http://127.0.0.1:8000"]
+# ENABLE_API_METRICS=true
+# AUTO_RELOAD=true
+# RELOAD_DIRS=["backend", "frontend"]
+# RELOAD_DELAY=0.25
+# HOT_RELOAD=true
+# SOURCE_MAPS=true
+# MINIFY_ASSETS=false
+# COMPRESS_RESPONSES=false
+# SECURITY_HEADERS=false
+# HSTS_MAX_AGE=0
+# CSP_ENABLED=false
+# DOCS_AUTO_RELOAD=true
--- a/docs/CI-CD-SECRETS.md
+++ b/docs/CI-CD-SECRETS.md
@@ -0,0 +1,274 @@
+# CI/CD Secrets Setup Guide
+
+This guide explains how to configure the required secrets for UnitForge's Gitea CI/CD workflows.
+
+## Overview
+
+UnitForge's CI/CD pipelines require several secrets to function properly:
+- **Container Registry** credentials for pushing multi-arch images
+- **PyPI** token for automated package publishing
+- **GitHub** token for release management
+- **Optional** notification and deployment secrets
+
+## Required Secrets
+
+### 1. Container Registry Authentication
+
+**Required for:** All build workflows (`build-container.yml`, `release.yml`, `nightly.yml`)
+
+```bash
+CONTAINER_REGISTRY_USERNAME=your-registry-username
+CONTAINER_REGISTRY_PASSWORD=your-registry-password-or-token
+```
+
+#### Docker Hub Example:
+```bash
+CONTAINER_REGISTRY_USERNAME=dockerhubusername
+CONTAINER_REGISTRY_PASSWORD=dckr_pat_xxxxxxxxxxxxxxxxxxxxxxxxxx
+```
+
+#### GitLab Container Registry Example:
+```bash
+CONTAINER_REGISTRY_USERNAME=gitlab-ci-token
+CONTAINER_REGISTRY_PASSWORD=glpat-xxxxxxxxxxxxxxxxxxxx
+```
+
+#### GitHub Container Registry Example:
+```bash
+CONTAINER_REGISTRY_USERNAME=github-username
+CONTAINER_REGISTRY_PASSWORD=ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+```
+
+### 2. PyPI Publishing Token
+
+**Required for:** Release workflow (`release.yml`) - stable releases only
+
+```bash
+PYPI_API_TOKEN=pypi-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+```
+
+#### How to get PyPI token:
+1. Go to [PyPI Account Settings](https://pypi.org/manage/account/token/)
+2. Click "Add API token"
+3. Set scope to "Entire account" or specific project
+4. Copy the token (starts with `pypi-`)
+
+### 3. GitHub Token (Optional)
+
+**Required for:** GitHub releases and integration
+
+```bash
+GITHUB_TOKEN=ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+```
+
+#### How to get GitHub token:
+1. Go to GitHub Settings > Developer settings > Personal access tokens
+2. Click "Generate new token (classic)"
+3. Select scopes: `repo`, `write:packages`
+4. Copy the token (starts with `ghp_`)
+
+## Setting Up Secrets in Gitea
+
+### Repository Secrets
+
+1. Navigate to your repository in Gitea
+2. Go to **Settings** > **Secrets and Variables** > **Actions**
+3. Click **New repository secret**
+4. Add each required secret:
+
+```
+Name: CONTAINER_REGISTRY_USERNAME
+Value: your-registry-username
+
+Name: CONTAINER_REGISTRY_PASSWORD
+Value: your-registry-password
+
+Name: PYPI_API_TOKEN
+Value: pypi-your-token-here
+
+Name: GITHUB_TOKEN
+Value: ghp_your-token-here
+```
+
+### Organization Secrets (Optional)
+
+For multiple repositories, set up organization-level secrets:
+
+1. Go to your Gitea organization
+2. Navigate to **Settings** > **Secrets and Variables** > **Actions**
+3. Add secrets that will be shared across repositories
+
+## Optional Secrets
+
+### Notification Services
+
+```bash
+# Slack notifications
+SLACK_WEBHOOK_URL=https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX
+
+# Discord notifications
+DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/000000000000000000/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+
+# Email notifications
+EMAIL_NOTIFICATION_ADDRESS=ci-notifications@yourdomain.com
+```
+
+### Deployment Secrets
+
+```bash
+# SSH key for server deployments (base64 encoded)
+DEPLOYMENT_SSH_KEY=LS0tLS1CRUdJTi...
+
+# Kubernetes service account token
+KUBERNETES_TOKEN=eyJhbGciOiJSUzI1NiIsImtpZCI6Ii...
+
+# Production API key
+PRODUCTION_API_KEY=prod-api-key-xxxxxxxxxxxx
+```
+
+### Security Scanning
+
+```bash
+# Enhanced security scanning (Snyk, etc.)
+SECURITY_SCAN_TOKEN=your-security-service-token
+```
+
+## Environment Configuration
+
+Update your `.env` file with registry and configuration details:
+
+```bash
+# Container Registry Configuration
+CONTAINER_REGISTRY_URL=https://your-registry.example.com/your-namespace/unitforge
+CONTAINER_TAG=latest
+
+# Build Configuration
+DOCKER_PLATFORMS=linux/amd64,linux/arm64
+BUILDX_CACHE_FROM=type=gha
+BUILDX_CACHE_TO=type=gha,mode=max
+```
+
+## Workflow-Specific Requirements
+
+### Pull Request Workflow (`pr-check.yml`)
+- **No secrets required** - Only builds and tests, no pushing
+
+### Build Container Workflow (`build-container.yml`)
+- `CONTAINER_REGISTRY_USERNAME` ✅ Required
+- `CONTAINER_REGISTRY_PASSWORD` ✅ Required
+
+### Release Workflow (`release.yml`)
+- `CONTAINER_REGISTRY_USERNAME` ✅ Required
+- `CONTAINER_REGISTRY_PASSWORD` ✅ Required
+- `PYPI_API_TOKEN` ✅ Required (stable releases)
+- `GITHUB_TOKEN` ⚠️ Optional (for GitHub releases)
+
+### Nightly Workflow (`nightly.yml`)
+- `CONTAINER_REGISTRY_USERNAME` ✅ Required
+- `CONTAINER_REGISTRY_PASSWORD` ✅ Required
+
+## Security Best Practices
+
+### 1. Secret Rotation
+- Rotate secrets regularly (quarterly recommended)
+- Use tokens with limited scope and expiration
+- Monitor token usage in registry/service dashboards
+
+### 2. Principle of Least Privilege
+- Grant minimum required permissions
+- Use service-specific tokens where possible
+- Avoid using personal access tokens in production
+
+### 3. Token Security
+- Never commit secrets to version control
+- Use environment-specific secrets
+- Enable audit logging where available
+
+### 4. Backup and Recovery
+- Document secret sources and renewal procedures
+- Maintain secure backup of critical tokens
+- Test secret rotation procedures
+
+## Troubleshooting
+
+### Common Issues
+
+**Authentication Failed:**
+```
+Error: unauthorized: authentication required
+```
+- Verify `CONTAINER_REGISTRY_USERNAME` and `CONTAINER_REGISTRY_PASSWORD`
+- Check token permissions and expiration
+- Ensure registry URL matches token scope
+
+**PyPI Publishing Failed:**
+```
+Error: 403 Forbidden
+```
+- Verify `PYPI_API_TOKEN` format (should start with `pypi-`)
+- Check token scope (project vs account-wide)
+- Ensure package name matches PyPI project
+
+**GitHub Release Failed:**
+```
+Error: Resource not accessible by integration
+```
+- Verify `GITHUB_TOKEN` permissions include `repo` scope
+- Check if token is expired
+- Ensure repository access is granted
+
+### Testing Secrets
+
+Test your secrets locally:
+
+```bash
+# Test container registry login
+echo "$CONTAINER_REGISTRY_PASSWORD" | docker login "$CONTAINER_REGISTRY_URL" -u "$CONTAINER_REGISTRY_USERNAME" --password-stdin
+
+# Test PyPI token (dry run)
+twine check dist/* --repository testpypi
+
+# Test GitHub API access
+curl -H "Authorization: token $GITHUB_TOKEN" https://api.github.com/user
+```
+
+## Monitoring and Alerts
+
+### Registry Usage
+- Monitor container registry storage and bandwidth
+- Set up alerts for failed pushes
+- Track image pull statistics
+
+### Build Metrics
+- Monitor workflow success rates
+- Track build duration trends
+- Alert on consecutive failures
+
+### Security Monitoring
+- Enable security scanning alerts
+- Monitor vulnerability trends
+- Set up notifications for critical issues
+
+## Support
+
+For additional help:
+- Check [Gitea Actions Documentation](https://docs.gitea.io/en-us/usage/actions/)
+- Review workflow logs in Gitea Actions UI
+- Open an issue in the UnitForge repository
+- Consult your container registry documentation
+
+## Quick Reference
+
+| Secret | Required For | Format | Purpose |
+|--------|-------------|--------|---------|
+| `CONTAINER_REGISTRY_USERNAME` | All builds | Username | Registry authentication |
+| `CONTAINER_REGISTRY_PASSWORD` | All builds | Token/Password | Registry authentication |
+| `PYPI_API_TOKEN` | Releases | `pypi-...` | Package publishing |
+| `GITHUB_TOKEN` | Releases | `ghp_...` | GitHub integration |
+| `SLACK_WEBHOOK_URL` | Notifications | HTTPS URL | Slack alerts |
+| `DEPLOYMENT_SSH_KEY` | Deployments | Base64 SSH key | Server access |
+
+---
+
+*Last updated: December 2024*
+*For the latest information, see the [CI/CD Workflows README](.gitea/workflows/README.md)*
--- a/docs/ci-cd/local-testing.md
+++ b/docs/ci-cd/local-testing.md
@@ -0,0 +1,58 @@
+# Local CI/CD Testing
+
+This guide helps you test CI/CD workflows locally before pushing to the repository.
+
+## Prerequisites
+
+- Docker with Buildx support
+- uv package manager
+- Python 3.8+
+
+## Local Testing Commands
+
+```bash
+# Test local build
+make docker-build
+
+# Test multi-arch build
+make docker-buildx-local
+
+# Test full development workflow
+make dev
+
+# Run health checks
+./scripts/health_check.sh
+```
+
+## Workflow Testing
+
+Use `act` to test GitHub/Gitea workflows locally:
+
+```bash
+# Install act
+curl https://raw.githubusercontent.com/nektos/act/master/install.sh | sudo bash
+
+# Test PR workflow
+act pull_request -s CONTAINER_REGISTRY_USERNAME=test -s CONTAINER_REGISTRY_PASSWORD=test
+
+# Test release workflow
+act push -e tests/fixtures/release-event.json
+```
+
+## Troubleshooting
+
+### Build Issues
+- Ensure all vendor assets are committed
+- Check Docker daemon is running
+- Verify buildx is properly configured
+
+### Registry Issues
+- Check .env file configuration
+- Verify registry credentials
+- Test registry connectivity
+
+### Performance Issues
+- Use build cache: `--cache-from type=gha`
+- Optimize Docker layers
+- Use multi-stage builds
+```