Add complete CI/CD pipeline with Tekton and Flux image automation
Some checks failed
Build and Push to Harbor / build-and-push (push) Has been cancelled

- Add Tekton pipeline and triggers for automated builds on git push
- Add Flux ImageRepository to track Harbor registry images
- Add Flux ImagePolicy for semantic versioning strategy
- Add Flux ImageUpdateAutomation to auto-update HelmRelease
- Update HelmRelease with image automation marker
- Add comprehensive CI/CD pipeline setup documentation

This enables automatic build and deployment when pushing to Gitea:
1. Gitea webhook triggers Tekton pipeline
2. Kaniko builds and pushes image to Harbor
3. Flux detects new image and updates deployment
4. Application automatically deploys to cluster

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Neon Vortex
2025-11-22 21:56:12 -05:00
parent 67a6ae146f
commit 473de53cff
6 changed files with 538 additions and 1 deletions

260
CICD_PIPELINE_SETUP.md Normal file
View File

@@ -0,0 +1,260 @@
# Neon Vortex CI/CD Pipeline Setup
This document describes the complete CI/CD pipeline that automatically builds and deploys Neon Vortex when you push to Gitea.
## Pipeline Flow
1. **Git Push** → Push code to Gitea repository
2. **Gitea Webhook** → Triggers Tekton EventListener
3. **Tekton Pipeline** → Builds container image with Kaniko
4. **Harbor Registry** → Stores the new image
5. **Flux Image Automation** → Detects new image and updates HelmRelease
6. **Flux CD** → Deploys updated application to cluster
## Prerequisites
Before setting up the pipeline, ensure you have:
- Tekton Pipelines installed in your cluster
- Tekton Triggers installed in your cluster
- Flux CD installed and configured
- Harbor registry credentials configured as a Kubernetes secret
## Installation Steps
### 1. Install Tekton (if not already installed)
```bash
# Install Tekton Pipelines
kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml
# Install Tekton Triggers
kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/latest/release.yaml
kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/latest/interceptors.yaml
```
### 2. Ensure Harbor Registry Secret Exists
The pipeline expects a secret named `harbor-registry` in the `default` namespace:
```bash
# Check if secret exists
kubectl get secret harbor-registry -n default
# If not, create it (replace with your Harbor credentials)
kubectl create secret docker-registry harbor-registry \
--docker-server=images.caffeinetux.com \
--docker-username=admin \
--docker-password=YOUR_PASSWORD \
--docker-email=admin@cluster.local \
-n default
```
### 3. Update Webhook Secret
Edit the `tekton-pipeline.yaml` file and change the webhook secret:
```yaml
stringData:
secretToken: "your-random-secret-here"
```
Generate a random secret:
```bash
openssl rand -hex 32
```
### 4. Apply All Configurations
```bash
# Apply Tekton pipeline and triggers
kubectl apply -f tekton-pipeline.yaml
# Apply Flux image automation
kubectl apply -f flux-imagerepository.yaml
kubectl apply -f flux-imagepolicy.yaml
kubectl apply -f flux-imageupdateautomation.yaml
# Apply updated HelmRelease
kubectl apply -f flux-helmrelease.yaml
```
### 5. Configure Gitea Webhook
1. Go to your Gitea repository: `http://192.168.1.49:13001/admin/neon-vortex`
2. Click **Settings****Webhooks****Add Webhook****Gitea**
3. Configure:
- **Target URL**: `http://el-neon-vortex-listener.default.svc.cluster.local:8080`
- Or if using NodePort: `http://<NODE_IP>:30081`
- **HTTP Method**: `POST`
- **POST Content Type**: `application/json`
- **Secret**: Use the same secret from step 3
- **Trigger On**: `Push events`
- **Branch filter**: `main`
4. Click **Add Webhook**
### 6. Enable Flux Write Access to Git Repository
Flux needs write access to update the HelmRelease file. Update the GitRepository secret:
```bash
# Create or update git credentials for Flux
kubectl create secret generic flux-git-auth \
--from-literal=username=admin \
--from-literal=password=YOUR_GITEA_TOKEN \
-n flux-system \
--dry-run=client -o yaml | kubectl apply -f -
# Update the GitRepository to use the secret
kubectl patch gitrepository neon-vortex -n flux-system --type merge -p '{"spec":{"secretRef":{"name":"flux-git-auth"}}}'
```
## How It Works
### Image Tagging Strategy
The pipeline uses semantic versioning with git commit SHA:
- Pattern: `v1.0.<git-commit-sha>`
- Example: `v1.0.67a6ae146f1f93aa5a58896419347e543b62fd88`
### Flux Image Policy
The `ImagePolicy` watches for new images matching the semver pattern `>=1.0.0` and automatically updates the HelmRelease when a new image is pushed.
### Automatic Deployment
When Flux detects a new image:
1. Updates `flux-helmrelease.yaml` with the new tag
2. Commits the change back to the repository
3. Reconciles the HelmRelease
4. Kubernetes pulls the new image and performs a rolling update
## Monitoring the Pipeline
### Check Tekton Pipeline Runs
```bash
# List recent pipeline runs
kubectl get pipelinerun -n default
# Watch a specific run
kubectl logs -f <pipelinerun-name> -n default
```
### Check Flux Image Automation
```bash
# Check ImageRepository status
flux get image repository neon-vortex -n flux-system
# Check ImagePolicy status
flux get image policy neon-vortex -n flux-system
# Check ImageUpdateAutomation status
flux get image update neon-vortex -n flux-system
```
### Check HelmRelease Status
```bash
# Check deployment status
flux get helmrelease neon-vortex -n default
# Check pods
kubectl get pods -n default -l app.kubernetes.io/name=neon-vortex
```
## Troubleshooting
### Webhook Not Triggering
1. Check EventListener is running:
```bash
kubectl get pods -n default | grep neon-vortex-listener
```
2. Check EventListener logs:
```bash
kubectl logs -l eventlistener=neon-vortex-listener -n default
```
3. Test webhook from Gitea settings page
### Build Failing
Check the PipelineRun logs:
```bash
kubectl get pipelinerun -n default
kubectl logs <pipelinerun-name> -n default -f
```
### Image Not Updating
1. Check ImageRepository can access Harbor:
```bash
flux get image repository neon-vortex -n flux-system
```
2. Check ImagePolicy is finding images:
```bash
kubectl describe imagepolicy neon-vortex -n flux-system
```
3. Verify Flux has write access to Git:
```bash
kubectl logs -n flux-system deployment/image-automation-controller
```
## Manual Trigger
To manually trigger a build without pushing to Git:
```bash
# Create a PipelineRun manually
kubectl create -f - <<EOF
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: neon-vortex-build-manual-
namespace: default
spec:
serviceAccountName: tekton-build-sa
pipelineRef:
name: neon-vortex-build-pipeline
workspaces:
- name: shared-workspace
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
- name: docker-credentials
secret:
secretName: harbor-registry
params:
- name: git-url
value: http://192.168.1.49:13001/admin/neon-vortex.git
- name: git-revision
value: main
- name: image-name
value: images.caffeinetux.com/apps/neon-vortex
- name: image-tag
value: v1.0.0
EOF
```
## Security Considerations
1. **Webhook Secret**: Use a strong random secret for Gitea webhooks
2. **Registry Credentials**: Keep Harbor credentials in Kubernetes secrets
3. **Git Credentials**: Use tokens instead of passwords for Flux git access
4. **RBAC**: The pipeline uses minimal RBAC permissions
## Next Steps
- Set up image scanning with Trivy
- Add automated testing in the pipeline
- Configure notifications (Slack, email) for build status
- Implement multi-environment deployments (dev, staging, prod)

View File

@@ -17,5 +17,5 @@ spec:
image: image:
registry: images.caffeinetux.com registry: images.caffeinetux.com
repository: apps/neon-vortex repository: apps/neon-vortex
tag: latest tag: latest # {"$imagepolicy": "flux-system:neon-vortex:tag"}
pullPolicy: Always pullPolicy: Always

14
flux-imagepolicy.yaml Normal file
View File

@@ -0,0 +1,14 @@
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImagePolicy
metadata:
name: neon-vortex
namespace: flux-system
spec:
imageRepositoryRef:
name: neon-vortex
policy:
semver:
range: '>=1.0.0'
filterTags:
pattern: '^v?(?P<version>[0-9]+\.[0-9]+\.[0-9]+)$'
extract: '$version'

10
flux-imagerepository.yaml Normal file
View File

@@ -0,0 +1,10 @@
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageRepository
metadata:
name: neon-vortex
namespace: flux-system
spec:
image: images.caffeinetux.com/apps/neon-vortex
interval: 1m
secretRef:
name: harbor-registry

View File

@@ -0,0 +1,28 @@
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageUpdateAutomation
metadata:
name: neon-vortex
namespace: flux-system
spec:
interval: 1m
sourceRef:
kind: GitRepository
name: neon-vortex
namespace: flux-system
git:
checkout:
ref:
branch: main
commit:
author:
email: flux@cluster.local
name: Flux Image Automation
messageTemplate: |
Update image to {{range .Updated.Images}}{{println .}}{{end}}
Automation: flux-system/neon-vortex
push:
branch: main
update:
path: ./flux-helmrelease.yaml
strategy: Setters

225
tekton-pipeline.yaml Normal file
View File

@@ -0,0 +1,225 @@
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: tekton-build-sa
namespace: default
---
apiVersion: v1
kind: Secret
metadata:
name: gitea-webhook-secret
namespace: default
type: Opaque
stringData:
secretToken: "change-me-to-random-string"
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: tekton-build-role
namespace: default
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
- apiGroups: ["batch"]
resources: ["jobs"]
verbs: ["create", "get", "list", "watch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: tekton-build-binding
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: tekton-build-role
subjects:
- kind: ServiceAccount
name: tekton-build-sa
namespace: default
---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: kaniko-build
namespace: default
spec:
params:
- name: IMAGE
description: Name (reference) of the image to build
- name: DOCKERFILE
description: Path to the Dockerfile to build
default: ./Dockerfile
- name: CONTEXT
description: The build context used by Kaniko
default: ./
- name: EXTRA_ARGS
default: ""
- name: GIT_URL
description: Git repository URL
- name: GIT_REVISION
description: Git revision to checkout
default: main
workspaces:
- name: source
- name: dockerconfig
mountPath: /kaniko/.docker
steps:
- name: git-clone
image: alpine/git:latest
script: |
#!/bin/sh
set -e
cd $(workspaces.source.path)
git clone $(params.GIT_URL) .
git checkout $(params.GIT_REVISION)
echo "Cloned $(params.GIT_URL) at revision $(params.GIT_REVISION)"
- name: build-and-push
image: gcr.io/kaniko-project/executor:latest
args:
- $(params.EXTRA_ARGS)
- --dockerfile=$(params.DOCKERFILE)
- --context=$(workspaces.source.path)/$(params.CONTEXT)
- --destination=$(params.IMAGE)
- --cache=true
- --cache-repo=$(params.IMAGE)/cache
---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: neon-vortex-build-pipeline
namespace: default
spec:
params:
- name: git-url
type: string
- name: git-revision
type: string
default: main
- name: image-name
type: string
- name: image-tag
type: string
workspaces:
- name: shared-workspace
- name: docker-credentials
tasks:
- name: build-image
taskRef:
name: kaniko-build
workspaces:
- name: source
workspace: shared-workspace
- name: dockerconfig
workspace: docker-credentials
params:
- name: IMAGE
value: "$(params.image-name):$(params.image-tag)"
- name: DOCKERFILE
value: "./htlm/Dockerfile"
- name: CONTEXT
value: "htlm"
- name: GIT_URL
value: "$(params.git-url)"
- name: GIT_REVISION
value: "$(params.git-revision)"
---
apiVersion: triggers.tekton.dev/v1beta1
kind: TriggerTemplate
metadata:
name: neon-vortex-trigger-template
namespace: default
spec:
params:
- name: git-revision
description: The git revision
- name: git-commit-message
description: The git commit message
- name: git-repo-url
description: The git repository url
resourcetemplates:
- apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: neon-vortex-build-
namespace: default
spec:
serviceAccountName: tekton-build-sa
pipelineRef:
name: neon-vortex-build-pipeline
workspaces:
- name: shared-workspace
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
- name: docker-credentials
secret:
secretName: harbor-registry
params:
- name: git-url
value: $(tt.params.git-repo-url)
- name: git-revision
value: $(tt.params.git-revision)
- name: image-name
value: images.caffeinetux.com/apps/neon-vortex
- name: image-tag
value: v1.0.$(tt.params.git-revision)
---
apiVersion: triggers.tekton.dev/v1beta1
kind: TriggerBinding
metadata:
name: neon-vortex-trigger-binding
namespace: default
spec:
params:
- name: git-repo-url
value: $(body.repository.clone_url)
- name: git-revision
value: $(body.after)
- name: git-commit-message
value: $(body.head_commit.message)
---
apiVersion: triggers.tekton.dev/v1beta1
kind: EventListener
metadata:
name: neon-vortex-listener
namespace: default
spec:
serviceAccountName: tekton-build-sa
triggers:
- name: gitea-push
interceptors:
- ref:
name: "gitea"
params:
- name: "secretRef"
value:
secretName: gitea-webhook-secret
secretKey: secretToken
- name: "eventTypes"
value: ["push"]
bindings:
- ref: neon-vortex-trigger-binding
template:
ref: neon-vortex-trigger-template
---
apiVersion: v1
kind: Service
metadata:
name: el-neon-vortex-listener
namespace: default
spec:
type: NodePort
ports:
- port: 8080
targetPort: 8080
nodePort: 30081
selector:
eventlistener: neon-vortex-listener