From 473de53cffd0a8da88e892656055503d08c9e13a Mon Sep 17 00:00:00 2001 From: Neon Vortex Date: Sat, 22 Nov 2025 21:56:12 -0500 Subject: [PATCH] Add complete CI/CD pipeline with Tekton and Flux image automation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add Tekton pipeline and triggers for automated builds on git push - Add Flux ImageRepository to track Harbor registry images - Add Flux ImagePolicy for semantic versioning strategy - Add Flux ImageUpdateAutomation to auto-update HelmRelease - Update HelmRelease with image automation marker - Add comprehensive CI/CD pipeline setup documentation This enables automatic build and deployment when pushing to Gitea: 1. Gitea webhook triggers Tekton pipeline 2. Kaniko builds and pushes image to Harbor 3. Flux detects new image and updates deployment 4. Application automatically deploys to cluster 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- CICD_PIPELINE_SETUP.md | 260 ++++++++++++++++++++++++++++++++ flux-helmrelease.yaml | 2 +- flux-imagepolicy.yaml | 14 ++ flux-imagerepository.yaml | 10 ++ flux-imageupdateautomation.yaml | 28 ++++ tekton-pipeline.yaml | 225 +++++++++++++++++++++++++++ 6 files changed, 538 insertions(+), 1 deletion(-) create mode 100644 CICD_PIPELINE_SETUP.md create mode 100644 flux-imagepolicy.yaml create mode 100644 flux-imagerepository.yaml create mode 100644 flux-imageupdateautomation.yaml create mode 100644 tekton-pipeline.yaml diff --git a/CICD_PIPELINE_SETUP.md b/CICD_PIPELINE_SETUP.md new file mode 100644 index 0000000..62939a5 --- /dev/null +++ b/CICD_PIPELINE_SETUP.md @@ -0,0 +1,260 @@ +# Neon Vortex CI/CD Pipeline Setup + +This document describes the complete CI/CD pipeline that automatically builds and deploys Neon Vortex when you push to Gitea. + +## Pipeline Flow + +1. **Git Push** → Push code to Gitea repository +2. **Gitea Webhook** → Triggers Tekton EventListener +3. **Tekton Pipeline** → Builds container image with Kaniko +4. **Harbor Registry** → Stores the new image +5. **Flux Image Automation** → Detects new image and updates HelmRelease +6. **Flux CD** → Deploys updated application to cluster + +## Prerequisites + +Before setting up the pipeline, ensure you have: + +- Tekton Pipelines installed in your cluster +- Tekton Triggers installed in your cluster +- Flux CD installed and configured +- Harbor registry credentials configured as a Kubernetes secret + +## Installation Steps + +### 1. Install Tekton (if not already installed) + +```bash +# Install Tekton Pipelines +kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml + +# Install Tekton Triggers +kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/latest/release.yaml +kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/latest/interceptors.yaml +``` + +### 2. Ensure Harbor Registry Secret Exists + +The pipeline expects a secret named `harbor-registry` in the `default` namespace: + +```bash +# Check if secret exists +kubectl get secret harbor-registry -n default + +# If not, create it (replace with your Harbor credentials) +kubectl create secret docker-registry harbor-registry \ + --docker-server=images.caffeinetux.com \ + --docker-username=admin \ + --docker-password=YOUR_PASSWORD \ + --docker-email=admin@cluster.local \ + -n default +``` + +### 3. Update Webhook Secret + +Edit the `tekton-pipeline.yaml` file and change the webhook secret: + +```yaml +stringData: + secretToken: "your-random-secret-here" +``` + +Generate a random secret: +```bash +openssl rand -hex 32 +``` + +### 4. Apply All Configurations + +```bash +# Apply Tekton pipeline and triggers +kubectl apply -f tekton-pipeline.yaml + +# Apply Flux image automation +kubectl apply -f flux-imagerepository.yaml +kubectl apply -f flux-imagepolicy.yaml +kubectl apply -f flux-imageupdateautomation.yaml + +# Apply updated HelmRelease +kubectl apply -f flux-helmrelease.yaml +``` + +### 5. Configure Gitea Webhook + +1. Go to your Gitea repository: `http://192.168.1.49:13001/admin/neon-vortex` +2. Click **Settings** → **Webhooks** → **Add Webhook** → **Gitea** +3. Configure: + - **Target URL**: `http://el-neon-vortex-listener.default.svc.cluster.local:8080` + - Or if using NodePort: `http://:30081` + - **HTTP Method**: `POST` + - **POST Content Type**: `application/json` + - **Secret**: Use the same secret from step 3 + - **Trigger On**: `Push events` + - **Branch filter**: `main` +4. Click **Add Webhook** + +### 6. Enable Flux Write Access to Git Repository + +Flux needs write access to update the HelmRelease file. Update the GitRepository secret: + +```bash +# Create or update git credentials for Flux +kubectl create secret generic flux-git-auth \ + --from-literal=username=admin \ + --from-literal=password=YOUR_GITEA_TOKEN \ + -n flux-system \ + --dry-run=client -o yaml | kubectl apply -f - + +# Update the GitRepository to use the secret +kubectl patch gitrepository neon-vortex -n flux-system --type merge -p '{"spec":{"secretRef":{"name":"flux-git-auth"}}}' +``` + +## How It Works + +### Image Tagging Strategy + +The pipeline uses semantic versioning with git commit SHA: +- Pattern: `v1.0.` +- Example: `v1.0.67a6ae146f1f93aa5a58896419347e543b62fd88` + +### Flux Image Policy + +The `ImagePolicy` watches for new images matching the semver pattern `>=1.0.0` and automatically updates the HelmRelease when a new image is pushed. + +### Automatic Deployment + +When Flux detects a new image: +1. Updates `flux-helmrelease.yaml` with the new tag +2. Commits the change back to the repository +3. Reconciles the HelmRelease +4. Kubernetes pulls the new image and performs a rolling update + +## Monitoring the Pipeline + +### Check Tekton Pipeline Runs + +```bash +# List recent pipeline runs +kubectl get pipelinerun -n default + +# Watch a specific run +kubectl logs -f -n default +``` + +### Check Flux Image Automation + +```bash +# Check ImageRepository status +flux get image repository neon-vortex -n flux-system + +# Check ImagePolicy status +flux get image policy neon-vortex -n flux-system + +# Check ImageUpdateAutomation status +flux get image update neon-vortex -n flux-system +``` + +### Check HelmRelease Status + +```bash +# Check deployment status +flux get helmrelease neon-vortex -n default + +# Check pods +kubectl get pods -n default -l app.kubernetes.io/name=neon-vortex +``` + +## Troubleshooting + +### Webhook Not Triggering + +1. Check EventListener is running: +```bash +kubectl get pods -n default | grep neon-vortex-listener +``` + +2. Check EventListener logs: +```bash +kubectl logs -l eventlistener=neon-vortex-listener -n default +``` + +3. Test webhook from Gitea settings page + +### Build Failing + +Check the PipelineRun logs: +```bash +kubectl get pipelinerun -n default +kubectl logs -n default -f +``` + +### Image Not Updating + +1. Check ImageRepository can access Harbor: +```bash +flux get image repository neon-vortex -n flux-system +``` + +2. Check ImagePolicy is finding images: +```bash +kubectl describe imagepolicy neon-vortex -n flux-system +``` + +3. Verify Flux has write access to Git: +```bash +kubectl logs -n flux-system deployment/image-automation-controller +``` + +## Manual Trigger + +To manually trigger a build without pushing to Git: + +```bash +# Create a PipelineRun manually +kubectl create -f - <=1.0.0' + filterTags: + pattern: '^v?(?P[0-9]+\.[0-9]+\.[0-9]+)$' + extract: '$version' diff --git a/flux-imagerepository.yaml b/flux-imagerepository.yaml new file mode 100644 index 0000000..ff09f67 --- /dev/null +++ b/flux-imagerepository.yaml @@ -0,0 +1,10 @@ +apiVersion: image.toolkit.fluxcd.io/v1beta2 +kind: ImageRepository +metadata: + name: neon-vortex + namespace: flux-system +spec: + image: images.caffeinetux.com/apps/neon-vortex + interval: 1m + secretRef: + name: harbor-registry diff --git a/flux-imageupdateautomation.yaml b/flux-imageupdateautomation.yaml new file mode 100644 index 0000000..2e95511 --- /dev/null +++ b/flux-imageupdateautomation.yaml @@ -0,0 +1,28 @@ +apiVersion: image.toolkit.fluxcd.io/v1beta2 +kind: ImageUpdateAutomation +metadata: + name: neon-vortex + namespace: flux-system +spec: + interval: 1m + sourceRef: + kind: GitRepository + name: neon-vortex + namespace: flux-system + git: + checkout: + ref: + branch: main + commit: + author: + email: flux@cluster.local + name: Flux Image Automation + messageTemplate: | + Update image to {{range .Updated.Images}}{{println .}}{{end}} + + Automation: flux-system/neon-vortex + push: + branch: main + update: + path: ./flux-helmrelease.yaml + strategy: Setters diff --git a/tekton-pipeline.yaml b/tekton-pipeline.yaml new file mode 100644 index 0000000..462261a --- /dev/null +++ b/tekton-pipeline.yaml @@ -0,0 +1,225 @@ +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: tekton-build-sa + namespace: default +--- +apiVersion: v1 +kind: Secret +metadata: + name: gitea-webhook-secret + namespace: default +type: Opaque +stringData: + secretToken: "change-me-to-random-string" +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: tekton-build-role + namespace: default +rules: +- apiGroups: [""] + resources: ["secrets"] + verbs: ["get"] +- apiGroups: ["batch"] + resources: ["jobs"] + verbs: ["create", "get", "list", "watch", "delete"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: tekton-build-binding + namespace: default +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: tekton-build-role +subjects: +- kind: ServiceAccount + name: tekton-build-sa + namespace: default +--- +apiVersion: tekton.dev/v1beta1 +kind: Task +metadata: + name: kaniko-build + namespace: default +spec: + params: + - name: IMAGE + description: Name (reference) of the image to build + - name: DOCKERFILE + description: Path to the Dockerfile to build + default: ./Dockerfile + - name: CONTEXT + description: The build context used by Kaniko + default: ./ + - name: EXTRA_ARGS + default: "" + - name: GIT_URL + description: Git repository URL + - name: GIT_REVISION + description: Git revision to checkout + default: main + workspaces: + - name: source + - name: dockerconfig + mountPath: /kaniko/.docker + steps: + - name: git-clone + image: alpine/git:latest + script: | + #!/bin/sh + set -e + cd $(workspaces.source.path) + git clone $(params.GIT_URL) . + git checkout $(params.GIT_REVISION) + echo "Cloned $(params.GIT_URL) at revision $(params.GIT_REVISION)" + - name: build-and-push + image: gcr.io/kaniko-project/executor:latest + args: + - $(params.EXTRA_ARGS) + - --dockerfile=$(params.DOCKERFILE) + - --context=$(workspaces.source.path)/$(params.CONTEXT) + - --destination=$(params.IMAGE) + - --cache=true + - --cache-repo=$(params.IMAGE)/cache +--- +apiVersion: tekton.dev/v1beta1 +kind: Pipeline +metadata: + name: neon-vortex-build-pipeline + namespace: default +spec: + params: + - name: git-url + type: string + - name: git-revision + type: string + default: main + - name: image-name + type: string + - name: image-tag + type: string + workspaces: + - name: shared-workspace + - name: docker-credentials + tasks: + - name: build-image + taskRef: + name: kaniko-build + workspaces: + - name: source + workspace: shared-workspace + - name: dockerconfig + workspace: docker-credentials + params: + - name: IMAGE + value: "$(params.image-name):$(params.image-tag)" + - name: DOCKERFILE + value: "./htlm/Dockerfile" + - name: CONTEXT + value: "htlm" + - name: GIT_URL + value: "$(params.git-url)" + - name: GIT_REVISION + value: "$(params.git-revision)" +--- +apiVersion: triggers.tekton.dev/v1beta1 +kind: TriggerTemplate +metadata: + name: neon-vortex-trigger-template + namespace: default +spec: + params: + - name: git-revision + description: The git revision + - name: git-commit-message + description: The git commit message + - name: git-repo-url + description: The git repository url + resourcetemplates: + - apiVersion: tekton.dev/v1beta1 + kind: PipelineRun + metadata: + generateName: neon-vortex-build- + namespace: default + spec: + serviceAccountName: tekton-build-sa + pipelineRef: + name: neon-vortex-build-pipeline + workspaces: + - name: shared-workspace + volumeClaimTemplate: + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + - name: docker-credentials + secret: + secretName: harbor-registry + params: + - name: git-url + value: $(tt.params.git-repo-url) + - name: git-revision + value: $(tt.params.git-revision) + - name: image-name + value: images.caffeinetux.com/apps/neon-vortex + - name: image-tag + value: v1.0.$(tt.params.git-revision) +--- +apiVersion: triggers.tekton.dev/v1beta1 +kind: TriggerBinding +metadata: + name: neon-vortex-trigger-binding + namespace: default +spec: + params: + - name: git-repo-url + value: $(body.repository.clone_url) + - name: git-revision + value: $(body.after) + - name: git-commit-message + value: $(body.head_commit.message) +--- +apiVersion: triggers.tekton.dev/v1beta1 +kind: EventListener +metadata: + name: neon-vortex-listener + namespace: default +spec: + serviceAccountName: tekton-build-sa + triggers: + - name: gitea-push + interceptors: + - ref: + name: "gitea" + params: + - name: "secretRef" + value: + secretName: gitea-webhook-secret + secretKey: secretToken + - name: "eventTypes" + value: ["push"] + bindings: + - ref: neon-vortex-trigger-binding + template: + ref: neon-vortex-trigger-template +--- +apiVersion: v1 +kind: Service +metadata: + name: el-neon-vortex-listener + namespace: default +spec: + type: NodePort + ports: + - port: 8080 + targetPort: 8080 + nodePort: 30081 + selector: + eventlistener: neon-vortex-listener