Simplify CI/CD pipeline with CronJob-based build trigger

- Add CronJob that polls git repository every 2 minutes for changes - Automatically triggers Kaniko build jobs when new commits detected - Images tagged with both 'latest' and 'v1.0.<commit-sha>' for versioning - Remove complex Tekton/Flux image automation dependencies - Add comprehensive simple CI/CD setup documentation This provides a reliable, simple CI/CD pipeline: 1. Push code to Gitea 2. CronJob detects changes within 2 minutes 3. Kaniko builds and pushes to Harbor 4. Flux deploys latest image automatically 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-22 21:59:33 -05:00
parent 473de53cff
commit d919cbd263
6 changed files with 550 additions and 2 deletions
--- a/SIMPLE_CICD_SETUP.md
+++ b/SIMPLE_CICD_SETUP.md
@@ -0,0 +1,271 @@
+# Simple CI/CD Pipeline for Neon Vortex
+
+This pipeline automatically builds and deploys your application when you push changes to Gitea.
+
+## How It Works
+
+The pipeline uses a simple, reliable approach with Kubernetes CronJob:
+
+1. **Git Push** → You push code to your Gitea repository
+2. **CronJob Poll** → Every 2 minutes, a CronJob checks for new commits
+3. **Kaniko Build** → When changes are detected, Kaniko builds a new container image
+4. **Harbor Registry** → The new image is pushed to Harbor with tags:
+   - `latest` - Always points to the most recent build
+   - `v1.0.<commit-sha>` - Specific version tag (e.g., `v1.0.473de53`)
+5. **Flux CD** → Automatically deploys the `latest` tag to your cluster
+
+## What Was Installed
+
+The following resources were created in your cluster:
+
+```bash
+# Check the CronJob (runs every 2 minutes)
+kubectl get cronjob neon-vortex-build-trigger -n default
+
+# View recent build trigger jobs
+kubectl get jobs -n default -l app=neon-vortex
+
+# Check build logs
+kubectl logs -n default -l job-name=<job-name> -c kaniko
+```
+
+## Current Status
+
+```bash
+# Verify the CronJob is active
+kubectl get cronjob -n default
+
+# Check when the next build check will run
+kubectl describe cronjob neon-vortex-build-trigger -n default | grep "Last Schedule"
+
+# See if any builds are currently running
+kubectl get jobs -n default
+```
+
+## How to Trigger a Build
+
+### Automatic (Recommended)
+Just push your code to Gitea:
+
+```bash
+git add .
+git commit -m "Your changes"
+git push origin main
+```
+
+Within 2 minutes, the CronJob will detect the change and start a build automatically.
+
+### Manual Trigger
+To manually trigger a build immediately:
+
+```bash
+# Create a job from the CronJob
+kubectl create job --from=cronjob/neon-vortex-build-trigger manual-build-$(date +%s) -n default
+```
+
+## Monitoring Builds
+
+### Watch for New Builds
+
+```bash
+# Watch jobs being created
+kubectl get jobs -n default -w
+
+# Follow build logs (replace JOB_NAME)
+kubectl logs -f -n default job/JOB_NAME -c kaniko
+```
+
+### Check Build History
+
+```bash
+# List all build jobs
+kubectl get jobs -n default -l app=neon-vortex
+
+# Get details of a specific job
+kubectl describe job <job-name> -n default
+```
+
+### Check Deployed Version
+
+```bash
+# See current image in use
+kubectl get deployment neon-vortex -n default -o jsonpath='{.spec.template.spec.containers[0].image}'
+
+# Check HelmRelease status
+flux get helmrelease neon-vortex -n default
+```
+
+## Image Tagging Strategy
+
+Each build creates two tags:
+- **`latest`**: Always updated, used by Flux for deployment
+- **`v1.0.<commit-sha>`**: Permanent version tag for rollbacks
+
+Example:
+```
+images.caffeinetux.com/apps/neon-vortex:latest
+images.caffeinetux.com/apps/neon-vortex:v1.0.473de53
+```
+
+## Rolling Back
+
+To roll back to a previous version:
+
+```bash
+# List available versions in Harbor
+# (You'll need to check your Harbor web UI or use Docker registry API)
+
+# Update the HelmRelease to use a specific version
+flux suspend helmrelease neon-vortex -n default
+
+kubectl patch helmrelease neon-vortex -n default --type merge -p '
+{
+  "spec": {
+    "values": {
+      "image": {
+        "tag": "v1.0.67a6ae1"
+      }
+    }
+  }
+}'
+
+flux resume helmrelease neon-vortex -n default
+flux reconcile helmrelease neon-vortex -n default
+```
+
+## Adjusting Build Frequency
+
+The CronJob checks for changes every 2 minutes by default. To change this:
+
+```bash
+# Edit the CronJob
+kubectl edit cronjob neon-vortex-build-trigger -n default
+
+# Change the schedule line:
+# */2 * * * * = every 2 minutes
+# */5 * * * * = every 5 minutes
+# */10 * * * * = every 10 minutes
+```
+
+## Troubleshooting
+
+### Builds Not Starting
+
+1. Check if CronJob is running:
+```bash
+kubectl get cronjob neon-vortex-build-trigger -n default
+```
+
+2. Check recent CronJob executions:
+```bash
+kubectl get jobs -n default -l app=neon-vortex
+```
+
+3. View trigger logs:
+```bash
+# Find the most recent trigger job
+TRIGGER_JOB=$(kubectl get jobs -n default -l cronjob=neon-vortex-build-trigger --sort-by=.metadata.creationTimestamp -o jsonpath='{.items[-1].metadata.name}')
+
+# View its logs
+kubectl logs job/$TRIGGER_JOB -n default
+```
+
+### Build Failures
+
+1. Check the build job status:
+```bash
+kubectl get jobs -n default
+kubectl describe job <failed-job-name> -n default
+```
+
+2. View build logs:
+```bash
+kubectl logs -n default job/<job-name> -c git-clone
+kubectl logs -n default job/<job-name> -c kaniko
+```
+
+3. Common issues:
+   - **Harbor authentication**: Check `harbor-registry` secret exists
+   - **Git access**: Ensure Gitea is accessible from cluster
+   - **Dockerfile errors**: Check Dockerfile syntax in `htlm/Dockerfile`
+
+### Harbor Authentication Issues
+
+Verify the Harbor registry secret:
+```bash
+kubectl get secret harbor-registry -n default
+kubectl get secret harbor-registry -n default -o jsonpath='{.data.\.dockerconfigjson}' | base64 -d
+```
+
+Recreate if needed:
+```bash
+kubectl delete secret harbor-registry -n default
+
+kubectl create secret docker-registry harbor-registry \
+  --docker-server=images.caffeinetux.com \
+  --docker-username=admin \
+  --docker-password=YOUR_PASSWORD \
+  -n default
+```
+
+### Deployment Not Updating
+
+1. Check if new image exists in Harbor
+2. Verify HelmRelease is using `pullPolicy: Always`:
+```bash
+kubectl get helmrelease neon-vortex -n default -o yaml | grep -A5 image
+```
+
+3. Force Flux to reconcile:
+```bash
+flux reconcile helmrelease neon-vortex -n default
+```
+
+4. Force pods to restart and pull new image:
+```bash
+kubectl rollout restart deployment neon-vortex -n default
+```
+
+## Cleanup Old Build Jobs
+
+Jobs are automatically deleted after 1 hour (ttlSecondsAfterFinished: 3600). To manually clean up:
+
+```bash
+# Delete all completed jobs older than 1 hour
+kubectl delete jobs -n default -l app=neon-vortex --field-selector status.successful=1
+```
+
+## Security Notes
+
+1. **Git Access**: The CronJob pulls from Gitea without authentication (public repo)
+   - If your repo is private, add credentials to the git clone command
+2. **Harbor Credentials**: Stored securely in Kubernetes secret `harbor-registry`
+3. **RBAC**: The build trigger has minimal permissions (only create/manage jobs)
+
+## Next Steps
+
+Consider these enhancements:
+
+1. **Add Tests**: Run tests before building the image
+2. **Notifications**: Send Slack/email alerts on build success/failure
+3. **Multi-stage Builds**: Optimize image size
+4. **Image Scanning**: Add Trivy or Clair for vulnerability scanning
+5. **Blue/Green Deployment**: Implement canary releases
+6. **Prometheus Metrics**: Monitor build success rates
+
+## Quick Reference
+
+```bash
+# Check everything is running
+kubectl get cronjob,jobs,pods -n default -l app=neon-vortex
+
+# Trigger immediate build
+kubectl create job --from=cronjob/neon-vortex-build-trigger manual-$(date +%s) -n default
+
+# Watch deployment status
+watch kubectl get helmrelease,deployment,pods -n default
+
+# View recent changes
+flux get helmrelease neon-vortex -n default
+kubectl get events -n default --sort-by='.lastTimestamp' | grep neon-vortex
+```
--- a/build-trigger-cronjob.yaml
+++ b/build-trigger-cronjob.yaml
@@ -0,0 +1,194 @@
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: build-trigger-script
+  namespace: default
+data:
+  trigger-build.sh: |
+    #!/bin/bash
+    set -e
+
+    # Configuration
+    GIT_URL="http://192.168.1.49:13001/admin/neon-vortex.git"
+    REPO_DIR="/tmp/repo"
+    LAST_COMMIT_FILE="/data/last_commit"
+
+    echo "Checking for new commits..."
+
+    # Clone or update repository
+    if [ ! -d "$REPO_DIR" ]; then
+      git clone "$GIT_URL" "$REPO_DIR"
+    else
+      cd "$REPO_DIR"
+      git fetch origin main
+      git reset --hard origin/main
+    fi
+
+    cd "$REPO_DIR"
+    CURRENT_COMMIT=$(git rev-parse HEAD)
+    SHORT_COMMIT=$(git rev-parse --short HEAD)
+
+    echo "Current commit: $CURRENT_COMMIT"
+
+    # Read last processed commit
+    LAST_COMMIT=""
+    if [ -f "$LAST_COMMIT_FILE" ]; then
+      LAST_COMMIT=$(cat "$LAST_COMMIT_FILE")
+      echo "Last processed commit: $LAST_COMMIT"
+    fi
+
+    # Check if there are new commits
+    if [ "$CURRENT_COMMIT" != "$LAST_COMMIT" ]; then
+      echo "New commit detected! Triggering build..."
+
+      # Create a new build job with unique name
+      TIMESTAMP=$(date +%s)
+      JOB_NAME="neon-vortex-build-$TIMESTAMP"
+
+      # Generate build job YAML
+      cat <<EOF | kubectl apply -f -
+    apiVersion: batch/v1
+    kind: Job
+    metadata:
+      name: $JOB_NAME
+      namespace: default
+      labels:
+        app: neon-vortex
+        build-commit: "$SHORT_COMMIT"
+    spec:
+      ttlSecondsAfterFinished: 3600
+      template:
+        spec:
+          restartPolicy: Never
+          initContainers:
+          - name: git-clone
+            image: alpine/git:latest
+            command:
+            - sh
+            - -c
+            - |
+              git clone $GIT_URL /workspace
+              cd /workspace
+              git checkout $CURRENT_COMMIT
+            volumeMounts:
+            - name: workspace
+              mountPath: /workspace
+          containers:
+          - name: kaniko
+            image: gcr.io/kaniko-project/executor:latest
+            args:
+            - "--dockerfile=/workspace/htlm/Dockerfile"
+            - "--context=/workspace/htlm"
+            - "--destination=images.caffeinetux.com/apps/neon-vortex:v1.0.$SHORT_COMMIT"
+            - "--destination=images.caffeinetux.com/apps/neon-vortex:latest"
+            - "--cache=true"
+            - "--cache-repo=images.caffeinetux.com/apps/neon-vortex/cache"
+            volumeMounts:
+            - name: workspace
+              mountPath: /workspace
+            - name: docker-config
+              mountPath: /kaniko/.docker
+          volumes:
+          - name: workspace
+            emptyDir: {}
+          - name: docker-config
+            secret:
+              secretName: harbor-registry
+              items:
+              - key: .dockerconfigjson
+                path: config.json
+    EOF
+
+      echo "Build job created: $JOB_NAME"
+
+      # Update last commit
+      echo "$CURRENT_COMMIT" > "$LAST_COMMIT_FILE"
+      echo "Updated last commit reference"
+    else
+      echo "No new commits. Skipping build."
+    fi
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: build-trigger-data
+  namespace: default
+spec:
+  accessModes:
+    - ReadWriteOnce
+  resources:
+    requests:
+      storage: 100Mi
+---
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+  name: neon-vortex-build-trigger
+  namespace: default
+spec:
+  schedule: "*/2 * * * *"  # Check every 2 minutes
+  concurrencyPolicy: Forbid
+  successfulJobsHistoryLimit: 3
+  failedJobsHistoryLimit: 3
+  jobTemplate:
+    spec:
+      template:
+        spec:
+          serviceAccountName: build-trigger-sa
+          restartPolicy: Never
+          containers:
+          - name: trigger
+            image: bitnami/kubectl:latest
+            command: ["/bin/bash"]
+            args:
+            - -c
+            - |
+              apk add --no-cache git bash
+              /scripts/trigger-build.sh
+            volumeMounts:
+            - name: script
+              mountPath: /scripts
+            - name: data
+              mountPath: /data
+          volumes:
+          - name: script
+            configMap:
+              name: build-trigger-script
+              defaultMode: 0755
+          - name: data
+            persistentVolumeClaim:
+              claimName: build-trigger-data
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: build-trigger-sa
+  namespace: default
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: build-trigger-role
+  namespace: default
+rules:
+- apiGroups: ["batch"]
+  resources: ["jobs"]
+  verbs: ["create", "get", "list", "watch", "delete"]
+- apiGroups: [""]
+  resources: ["pods", "pods/log"]
+  verbs: ["get", "list", "watch"]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: build-trigger-binding
+  namespace: default
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: build-trigger-role
+subjects:
+- kind: ServiceAccount
+  name: build-trigger-sa
+  namespace: default
--- a/flux-helmrelease.yaml
+++ b/flux-helmrelease.yaml
@@ -17,5 +17,5 @@ spec:
    image:
      registry: images.caffeinetux.com
      repository: apps/neon-vortex
-      tag: latest # {"$imagepolicy": "flux-system:neon-vortex:tag"}
+      tag: latest
      pullPolicy: Always
--- a/flux-imagepolicy.yaml
+++ b/flux-imagepolicy.yaml
@@ -10,5 +10,5 @@ spec:
    semver:
      range: '>=1.0.0'
  filterTags:
-    pattern: '^v?(?P<version>[0-9]+\.[0-9]+\.[0-9]+)$'
+    pattern: '^v(?P<version>[0-9]+\.[0-9]+\.[a-f0-9]+)$'
    extract: '$version'
--- a/flux-kustomization-build.yaml
+++ b/flux-kustomization-build.yaml
@@ -0,0 +1,36 @@
+---
+apiVersion: kustomize.toolkit.fluxcd.io/v1
+kind: Kustomization
+metadata:
+  name: neon-vortex-build
+  namespace: flux-system
+spec:
+  interval: 5m
+  path: ./
+  prune: false
+  sourceRef:
+    kind: GitRepository
+    name: neon-vortex
+  targetNamespace: default
+  patches:
+    - patch: |
+        apiVersion: batch/v1
+        kind: Job
+        metadata:
+          name: neon-vortex-build
+          namespace: default
+        spec:
+          template:
+            metadata:
+              name: neon-vortex-build
+      target:
+        kind: Job
+        name: neon-vortex-build
+  postBuild:
+    substitute:
+      GIT_COMMIT: "unknown"
+  healthChecks:
+    - apiVersion: batch/v1
+      kind: Job
+      name: neon-vortex-build
+      namespace: default
--- a/flux-receiver.yaml
+++ b/flux-receiver.yaml
@@ -0,0 +1,47 @@
+---
+apiVersion: v1
+kind: Secret
+metadata:
+  name: gitea-webhook-token
+  namespace: flux-system
+type: Opaque
+stringData:
+  token: "change-me-to-random-string"
+---
+apiVersion: notification.toolkit.fluxcd.io/v1
+kind: Receiver
+metadata:
+  name: neon-vortex-receiver
+  namespace: flux-system
+spec:
+  type: gitea
+  events:
+    - "ping"
+    - "push"
+  secretRef:
+    name: gitea-webhook-token
+  resources:
+    - apiVersion: source.toolkit.fluxcd.io/v1
+      kind: GitRepository
+      name: neon-vortex
+      namespace: flux-system
+    - apiVersion: kustomize.toolkit.fluxcd.io/v1
+      kind: Kustomization
+      name: neon-vortex-build
+      namespace: flux-system
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: receiver
+  namespace: flux-system
+spec:
+  type: NodePort
+  selector:
+    app: notification-controller
+  ports:
+    - name: http
+      port: 80
+      protocol: TCP
+      targetPort: 9292
+      nodePort: 30082