unitforge/.gitea/workflows/README.md

Gitea CI/CD Workflows

This directory contains the CI/CD workflows for UnitForge using Gitea Actions. These workflows automate testing, building, and deploying multi-architecture container images.

Workflows Overview

1. build-container.yml - Main Build Pipeline

Triggers: Push to main/develop, tags starting with v*

Features:

• Runs comprehensive tests (linting, type checking, security)
• Builds multi-arch container images (linux/amd64, linux/arm64)
• Pushes to container registry
• Security scanning with Trivy
• Automatic deployment to staging/production

Jobs:

• test - Run linting, tests, and security checks
• build-and-push - Build and push multi-arch container images
• security-scan - Vulnerability scanning
• deploy-staging - Deploy to staging (develop branch)
• deploy-production - Deploy to production (tags)

2. pr-check.yml - Pull Request Validation

Triggers: Pull requests to main/develop

Features:

• Full test suite including coverage reporting
• Multi-arch build testing (no push)
• Container startup verification
• Configuration validation
• PR summary with build status

Jobs:

• test - Complete test suite with coverage
• build-test - Test multi-arch builds without pushing
• validate-config - Validate project configuration
• pr-summary - Generate build status summary

3. release.yml - Release Pipeline

Triggers: Tags matching v* pattern

Features:

• Version validation and metadata extraction
• Full test suite across all Python versions
• Multi-arch container builds with release tags
• Security scanning with vulnerability blocking
• GitHub release creation with artifacts
• PyPI package publishing (stable releases only)
• Production deployment

Jobs:

• validate-release - Version format and metadata validation
• test-and-build - Comprehensive testing and Python package build
• build-container - Multi-arch container build with release tags
• security-scan - Security scanning with critical vulnerability blocking
• create-release - GitHub release with artifacts and changelog
• publish-package - PyPI publishing (stable releases only)
• deploy-production - Production deployment
• notify-release - Release completion notification

4. nightly.yml - Nightly Builds

Triggers: Daily at 2 AM UTC, manual dispatch

Features:

• Change detection (skips if no commits in 24h)
• Multi-Python version testing matrix
• Performance testing
• Comprehensive security scanning
• Old image cleanup
• Detailed reporting

Jobs:

• check-changes - Detect if build is needed
• nightly-tests - Test across Python versions (3.8-3.12)
• build-nightly - Build nightly images with date/commit tags
• performance-test - Basic performance validation
• security-scan-nightly - Comprehensive security analysis
• cleanup-old-nightlies - Remove old nightly images
• notify-results - Build status notification

Configuration

Required Secrets

Set these secrets in your Gitea repository settings:

# Container Registry
CONTAINER_REGISTRY_USERNAME=your-registry-username
CONTAINER_REGISTRY_PASSWORD=your-registry-password

# PyPI Publishing (for releases)
PYPI_API_TOKEN=your-pypi-token

# GitHub (if using GitHub releases)
GITHUB_TOKEN=your-github-token

Environment Variables

The workflows use these environment variables:

env:
  REGISTRY: gitea-http.taildb3494.ts.net
  IMAGE_NAME: will/unitforge

Update these in each workflow file to match your registry and image name.

Multi-Architecture Support

All workflows build for multiple architectures:

• linux/amd64 - Standard x86_64 architecture
• linux/arm64 - ARM64 architecture (Apple Silicon, ARM servers)

This is configured using Docker Buildx with the platform specification:

platforms: linux/amd64,linux/arm64

Container Registry Integration

Image Tags

Different workflows create different image tags:

Main builds (build-container.yml):

• main - Latest from main branch
• develop - Latest from develop branch
• latest - Latest stable release

Release builds (release.yml):

• v1.2.3 - Specific version
• 1.2 - Major.minor version
• 1 - Major version (stable releases only)
• latest - Latest stable release

Nightly builds (nightly.yml):

• nightly-20240101-abc1234 - Date and commit SHA
• nightly-latest - Latest nightly build

Registry Configuration

The workflows are configured for a self-hosted registry. To use a different registry:

1. Update the REGISTRY environment variable
2. Ensure authentication secrets are set correctly
3. Verify registry supports multi-arch manifests

Development Workflow

Branch Strategy

• main - Production-ready code
• develop - Integration branch for features
• feature/* - Feature branches (create PRs to develop)
• hotfix/* - Critical fixes (create PRs to main)

Release Process

1. Prepare Release:

   git checkout main
   git pull origin main
   git tag v1.2.3
   git push origin v1.2.3
   

2. Automatic Process:

   • Release workflow triggers
   • Tests run across all Python versions
   • Multi-arch container images built
   • Security scanning performed
   • GitHub release created
   • PyPI package published (if stable)
   • Production deployment triggered

3. Manual Verification:

   • Check workflow completion
   • Verify container images in registry
   • Test deployed application
   • Monitor for issues

Local Development

Test builds locally using the Makefile:

# Setup development environment
make setup-dev

# Run tests and linting
make dev

# Build container image locally
make docker-buildx-local

# Test multi-arch build (requires buildx)
make docker-buildx-setup
docker buildx build --platform linux/amd64,linux/arm64 -t unitforge:test .

Debugging Workflows

Common Issues

1. Missing Vendor Assets:

   Error: Missing bootstrap CSS file
   

   Ensure all static assets are committed to the repository.

2. Registry Authentication:

   Error: unauthorized
   

   Verify CONTAINER_REGISTRY_USERNAME and CONTAINER_REGISTRY_PASSWORD secrets.

3. Build Platform Issues:

   Error: multiple platforms feature is currently not supported
   

   Ensure Docker Buildx is properly set up in the runner.

Workflow Debugging

1. Enable Debug Logging: Add to workflow:

   env:
     ACTIONS_STEP_DEBUG: true
     ACTIONS_RUNNER_DEBUG: true
   

2. Test Locally: Use act to test workflows locally:

   act -j test -s CONTAINER_REGISTRY_USERNAME=test -s CONTAINER_REGISTRY_PASSWORD=test
   

3. Check Build Logs:

   • View detailed logs in Gitea Actions UI
   • Check container registry for pushed images
   • Verify security scan results

Security

Image Scanning

All container images are scanned for vulnerabilities using Trivy:

• PR builds: Informational scanning
• Main builds: Upload results to security dashboard
• Release builds: Block on critical vulnerabilities
• Nightly builds: Comprehensive analysis

Secrets Management

• Use Gitea repository secrets for sensitive data
• Never commit credentials to repository
• Rotate secrets regularly
• Use least-privilege access

Build Security

• Multi-stage Dockerfile minimizes attack surface
• Non-root user in containers
• Dependency scanning included
• Static analysis with security tools

Monitoring and Notifications

Build Status

Monitor workflow status:

• Gitea Actions dashboard
• Email notifications (configure in Gitea)
• External monitoring (webhook integrations)

Metrics

Track important metrics:

• Build success rate
• Build duration
• Image size trends
• Security vulnerability counts

Alerts

Set up alerts for:

• Failed builds on main/develop
• Security vulnerabilities in releases
• Performance regression in nightly builds
• Registry storage usage

Customization

Adding New Platforms

To support additional architectures:

1. Update platform list:

   platforms: linux/amd64,linux/arm64,linux/arm/v7
   

2. Ensure base images support the platform

3. Test builds on target architecture

Custom Deployment

Modify deployment jobs for your infrastructure:

deploy-production:
  steps:
    - name: Deploy to Kubernetes
      run: |
        kubectl set image deployment/unitforge \
          unitforge=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.ref_name }}

Integration with External Tools

Add steps for external integrations:

- name: Update monitoring
  run: |
    curl -X POST "$MONITORING_WEBHOOK" \
      -d "version=${{ github.ref_name }}" \
      -d "image=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.ref_name }}"

Troubleshooting

Performance Issues

If builds are slow:

• Enable build caching (already configured)
• Use faster runners if available
• Parallelize independent jobs
• Optimize Docker layer caching

Storage Issues

Manage registry storage:

• Implement cleanup policies
• Use image compression
• Remove unused layers
• Monitor storage usage

Network Issues

For registry connectivity problems:

• Check network policies
• Verify DNS resolution
• Test registry endpoint manually
• Review firewall rules

Contributing

When modifying workflows:

1. Test changes in feature branch
2. Document any new requirements
3. Update this README if needed
4. Ensure backward compatibility
5. Test with actual builds before merging

For questions or issues with the CI/CD workflows, please create an issue in the repository.