neon-vortex/CICD_PIPELINE_SETUP.md

Neon Vortex CI/CD Pipeline Setup

This document describes the complete CI/CD pipeline that automatically builds and deploys Neon Vortex when you push to Gitea.

Pipeline Flow

1. Git Push → Push code to Gitea repository
2. Gitea Webhook → Triggers Tekton EventListener
3. Tekton Pipeline → Builds container image with Kaniko
4. Harbor Registry → Stores the new image
5. Flux Image Automation → Detects new image and updates HelmRelease
6. Flux CD → Deploys updated application to cluster

Prerequisites

Before setting up the pipeline, ensure you have:

• Tekton Pipelines installed in your cluster
• Tekton Triggers installed in your cluster
• Flux CD installed and configured
• Harbor registry credentials configured as a Kubernetes secret

Installation Steps

1. Install Tekton (if not already installed)

# Install Tekton Pipelines
kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml

# Install Tekton Triggers
kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/latest/release.yaml
kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/latest/interceptors.yaml

2. Ensure Harbor Registry Secret Exists

The pipeline expects a secret named harbor-registry in the default namespace:

# Check if secret exists
kubectl get secret harbor-registry -n default

# If not, create it (replace with your Harbor credentials)
kubectl create secret docker-registry harbor-registry \
  --docker-server=images.caffeinetux.com \
  --docker-username=admin \
  --docker-password=YOUR_PASSWORD \
  --docker-email=admin@cluster.local \
  -n default

3. Update Webhook Secret

Edit the tekton-pipeline.yaml file and change the webhook secret:

stringData:
  secretToken: "your-random-secret-here"

Generate a random secret:

openssl rand -hex 32

4. Apply All Configurations

# Apply Tekton pipeline and triggers
kubectl apply -f tekton-pipeline.yaml

# Apply Flux image automation
kubectl apply -f flux-imagerepository.yaml
kubectl apply -f flux-imagepolicy.yaml
kubectl apply -f flux-imageupdateautomation.yaml

# Apply updated HelmRelease
kubectl apply -f flux-helmrelease.yaml

5. Configure Gitea Webhook

1. Go to your Gitea repository: http://192.168.1.49:13001/admin/neon-vortex
2. Click Settings → Webhooks → Add Webhook → Gitea
3. Configure:
   • Target URL: http://el-neon-vortex-listener.default.svc.cluster.local:8080
     • Or if using NodePort: http://<NODE_IP>:30081
   • HTTP Method: POST
   • POST Content Type: application/json
   • Secret: Use the same secret from step 3
   • Trigger On: Push events
   • Branch filter: main
4. Click Add Webhook

6. Enable Flux Write Access to Git Repository

Flux needs write access to update the HelmRelease file. Update the GitRepository secret:

# Create or update git credentials for Flux
kubectl create secret generic flux-git-auth \
  --from-literal=username=admin \
  --from-literal=password=YOUR_GITEA_TOKEN \
  -n flux-system \
  --dry-run=client -o yaml | kubectl apply -f -

# Update the GitRepository to use the secret
kubectl patch gitrepository neon-vortex -n flux-system --type merge -p '{"spec":{"secretRef":{"name":"flux-git-auth"}}}'

How It Works

Image Tagging Strategy

The pipeline uses semantic versioning with git commit SHA:

• Pattern: v1.0.<git-commit-sha>
• Example: v1.0.67a6ae146f1f93aa5a58896419347e543b62fd88

Flux Image Policy

The ImagePolicy watches for new images matching the semver pattern >=1.0.0 and automatically updates the HelmRelease when a new image is pushed.

Automatic Deployment

When Flux detects a new image:

1. Updates flux-helmrelease.yaml with the new tag
2. Commits the change back to the repository
3. Reconciles the HelmRelease
4. Kubernetes pulls the new image and performs a rolling update

Monitoring the Pipeline

Check Tekton Pipeline Runs

# List recent pipeline runs
kubectl get pipelinerun -n default

# Watch a specific run
kubectl logs -f <pipelinerun-name> -n default

Check Flux Image Automation

# Check ImageRepository status
flux get image repository neon-vortex -n flux-system

# Check ImagePolicy status
flux get image policy neon-vortex -n flux-system

# Check ImageUpdateAutomation status
flux get image update neon-vortex -n flux-system

Check HelmRelease Status

# Check deployment status
flux get helmrelease neon-vortex -n default

# Check pods
kubectl get pods -n default -l app.kubernetes.io/name=neon-vortex

Troubleshooting

Webhook Not Triggering

1. Check EventListener is running:

kubectl get pods -n default | grep neon-vortex-listener

2. Check EventListener logs:

kubectl logs -l eventlistener=neon-vortex-listener -n default

3. Test webhook from Gitea settings page

Build Failing

Check the PipelineRun logs:

kubectl get pipelinerun -n default
kubectl logs <pipelinerun-name> -n default -f

Image Not Updating

1. Check ImageRepository can access Harbor:

flux get image repository neon-vortex -n flux-system

2. Check ImagePolicy is finding images:

kubectl describe imagepolicy neon-vortex -n flux-system

3. Verify Flux has write access to Git:

kubectl logs -n flux-system deployment/image-automation-controller

Manual Trigger

To manually trigger a build without pushing to Git:

# Create a PipelineRun manually
kubectl create -f - <<EOF
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  generateName: neon-vortex-build-manual-
  namespace: default
spec:
  serviceAccountName: tekton-build-sa
  pipelineRef:
    name: neon-vortex-build-pipeline
  workspaces:
  - name: shared-workspace
    volumeClaimTemplate:
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
  - name: docker-credentials
    secret:
      secretName: harbor-registry
  params:
  - name: git-url
    value: http://192.168.1.49:13001/admin/neon-vortex.git
  - name: git-revision
    value: main
  - name: image-name
    value: images.caffeinetux.com/apps/neon-vortex
  - name: image-tag
    value: v1.0.0
EOF

Security Considerations

1. Webhook Secret: Use a strong random secret for Gitea webhooks
2. Registry Credentials: Keep Harbor credentials in Kubernetes secrets
3. Git Credentials: Use tokens instead of passwords for Flux git access
4. RBAC: The pipeline uses minimal RBAC permissions

Next Steps

• Set up image scanning with Trivy
• Add automated testing in the pipeline
• Configure notifications (Slack, email) for build status
• Implement multi-environment deployments (dev, staging, prod)