Add complete CI/CD pipeline with Tekton and Flux image automation
Some checks failed
Build and Push to Harbor / build-and-push (push) Has been cancelled
Some checks failed
Build and Push to Harbor / build-and-push (push) Has been cancelled
- Add Tekton pipeline and triggers for automated builds on git push - Add Flux ImageRepository to track Harbor registry images - Add Flux ImagePolicy for semantic versioning strategy - Add Flux ImageUpdateAutomation to auto-update HelmRelease - Update HelmRelease with image automation marker - Add comprehensive CI/CD pipeline setup documentation This enables automatic build and deployment when pushing to Gitea: 1. Gitea webhook triggers Tekton pipeline 2. Kaniko builds and pushes image to Harbor 3. Flux detects new image and updates deployment 4. Application automatically deploys to cluster 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Neon Vortex
260
CICD_PIPELINE_SETUP.md
Normal file
260
CICD_PIPELINE_SETUP.md
Normal file
@@ -0,0 +1,260 @@
|
||||
# Neon Vortex CI/CD Pipeline Setup
|
||||
|
||||
This document describes the complete CI/CD pipeline that automatically builds and deploys Neon Vortex when you push to Gitea.
|
||||
|
||||
## Pipeline Flow
|
||||
|
||||
1. **Git Push** → Push code to Gitea repository
|
||||
2. **Gitea Webhook** → Triggers Tekton EventListener
|
||||
3. **Tekton Pipeline** → Builds container image with Kaniko
|
||||
4. **Harbor Registry** → Stores the new image
|
||||
5. **Flux Image Automation** → Detects new image and updates HelmRelease
|
||||
6. **Flux CD** → Deploys updated application to cluster
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before setting up the pipeline, ensure you have:
|
||||
|
||||
- Tekton Pipelines installed in your cluster
|
||||
- Tekton Triggers installed in your cluster
|
||||
- Flux CD installed and configured
|
||||
- Harbor registry credentials configured as a Kubernetes secret
|
||||
|
||||
## Installation Steps
|
||||
|
||||
### 1. Install Tekton (if not already installed)
|
||||
|
||||
```bash
|
||||
# Install Tekton Pipelines
|
||||
kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml
|
||||
|
||||
# Install Tekton Triggers
|
||||
kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/latest/release.yaml
|
||||
kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/latest/interceptors.yaml
|
||||
```
|
||||
|
||||
### 2. Ensure Harbor Registry Secret Exists
|
||||
|
||||
The pipeline expects a secret named `harbor-registry` in the `default` namespace:
|
||||
|
||||
```bash
|
||||
# Check if secret exists
|
||||
kubectl get secret harbor-registry -n default
|
||||
|
||||
# If not, create it (replace with your Harbor credentials)
|
||||
kubectl create secret docker-registry harbor-registry \
|
||||
--docker-server=images.caffeinetux.com \
|
||||
--docker-username=admin \
|
||||
--docker-password=YOUR_PASSWORD \
|
||||
--docker-email=admin@cluster.local \
|
||||
-n default
|
||||
```
|
||||
|
||||
### 3. Update Webhook Secret
|
||||
|
||||
Edit the `tekton-pipeline.yaml` file and change the webhook secret:
|
||||
|
||||
```yaml
|
||||
stringData:
|
||||
secretToken: "your-random-secret-here"
|
||||
```
|
||||
|
||||
Generate a random secret:
|
||||
```bash
|
||||
openssl rand -hex 32
|
||||
```
|
||||
|
||||
### 4. Apply All Configurations
|
||||
|
||||
```bash
|
||||
# Apply Tekton pipeline and triggers
|
||||
kubectl apply -f tekton-pipeline.yaml
|
||||
|
||||
# Apply Flux image automation
|
||||
kubectl apply -f flux-imagerepository.yaml
|
||||
kubectl apply -f flux-imagepolicy.yaml
|
||||
kubectl apply -f flux-imageupdateautomation.yaml
|
||||
|
||||
# Apply updated HelmRelease
|
||||
kubectl apply -f flux-helmrelease.yaml
|
||||
```
|
||||
|
||||
### 5. Configure Gitea Webhook
|
||||
|
||||
1. Go to your Gitea repository: `http://192.168.1.49:13001/admin/neon-vortex`
|
||||
2. Click **Settings** → **Webhooks** → **Add Webhook** → **Gitea**
|
||||
3. Configure:
|
||||
- **Target URL**: `http://el-neon-vortex-listener.default.svc.cluster.local:8080`
|
||||
- Or if using NodePort: `http://<NODE_IP>:30081`
|
||||
- **HTTP Method**: `POST`
|
||||
- **POST Content Type**: `application/json`
|
||||
- **Secret**: Use the same secret from step 3
|
||||
- **Trigger On**: `Push events`
|
||||
- **Branch filter**: `main`
|
||||
4. Click **Add Webhook**
|
||||
|
||||
### 6. Enable Flux Write Access to Git Repository
|
||||
|
||||
Flux needs write access to update the HelmRelease file. Update the GitRepository secret:
|
||||
|
||||
```bash
|
||||
# Create or update git credentials for Flux
|
||||
kubectl create secret generic flux-git-auth \
|
||||
--from-literal=username=admin \
|
||||
--from-literal=password=YOUR_GITEA_TOKEN \
|
||||
-n flux-system \
|
||||
--dry-run=client -o yaml | kubectl apply -f -
|
||||
|
||||
# Update the GitRepository to use the secret
|
||||
kubectl patch gitrepository neon-vortex -n flux-system --type merge -p '{"spec":{"secretRef":{"name":"flux-git-auth"}}}'
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
### Image Tagging Strategy
|
||||
|
||||
The pipeline uses semantic versioning with git commit SHA:
|
||||
- Pattern: `v1.0.<git-commit-sha>`
|
||||
- Example: `v1.0.67a6ae146f1f93aa5a58896419347e543b62fd88`
|
||||
|
||||
### Flux Image Policy
|
||||
|
||||
The `ImagePolicy` watches for new images matching the semver pattern `>=1.0.0` and automatically updates the HelmRelease when a new image is pushed.
|
||||
|
||||
### Automatic Deployment
|
||||
|
||||
When Flux detects a new image:
|
||||
1. Updates `flux-helmrelease.yaml` with the new tag
|
||||
2. Commits the change back to the repository
|
||||
3. Reconciles the HelmRelease
|
||||
4. Kubernetes pulls the new image and performs a rolling update
|
||||
|
||||
## Monitoring the Pipeline
|
||||
|
||||
### Check Tekton Pipeline Runs
|
||||
|
||||
```bash
|
||||
# List recent pipeline runs
|
||||
kubectl get pipelinerun -n default
|
||||
|
||||
# Watch a specific run
|
||||
kubectl logs -f <pipelinerun-name> -n default
|
||||
```
|
||||
|
||||
### Check Flux Image Automation
|
||||
|
||||
```bash
|
||||
# Check ImageRepository status
|
||||
flux get image repository neon-vortex -n flux-system
|
||||
|
||||
# Check ImagePolicy status
|
||||
flux get image policy neon-vortex -n flux-system
|
||||
|
||||
# Check ImageUpdateAutomation status
|
||||
flux get image update neon-vortex -n flux-system
|
||||
```
|
||||
|
||||
### Check HelmRelease Status
|
||||
|
||||
```bash
|
||||
# Check deployment status
|
||||
flux get helmrelease neon-vortex -n default
|
||||
|
||||
# Check pods
|
||||
kubectl get pods -n default -l app.kubernetes.io/name=neon-vortex
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Webhook Not Triggering
|
||||
|
||||
1. Check EventListener is running:
|
||||
```bash
|
||||
kubectl get pods -n default | grep neon-vortex-listener
|
||||
```
|
||||
|
||||
2. Check EventListener logs:
|
||||
```bash
|
||||
kubectl logs -l eventlistener=neon-vortex-listener -n default
|
||||
```
|
||||
|
||||
3. Test webhook from Gitea settings page
|
||||
|
||||
### Build Failing
|
||||
|
||||
Check the PipelineRun logs:
|
||||
```bash
|
||||
kubectl get pipelinerun -n default
|
||||
kubectl logs <pipelinerun-name> -n default -f
|
||||
```
|
||||
|
||||
### Image Not Updating
|
||||
|
||||
1. Check ImageRepository can access Harbor:
|
||||
```bash
|
||||
flux get image repository neon-vortex -n flux-system
|
||||
```
|
||||
|
||||
2. Check ImagePolicy is finding images:
|
||||
```bash
|
||||
kubectl describe imagepolicy neon-vortex -n flux-system
|
||||
```
|
||||
|
||||
3. Verify Flux has write access to Git:
|
||||
```bash
|
||||
kubectl logs -n flux-system deployment/image-automation-controller
|
||||
```
|
||||
|
||||
## Manual Trigger
|
||||
|
||||
To manually trigger a build without pushing to Git:
|
||||
|
||||
```bash
|
||||
# Create a PipelineRun manually
|
||||
kubectl create -f - <<EOF
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: PipelineRun
|
||||
metadata:
|
||||
generateName: neon-vortex-build-manual-
|
||||
namespace: default
|
||||
spec:
|
||||
serviceAccountName: tekton-build-sa
|
||||
pipelineRef:
|
||||
name: neon-vortex-build-pipeline
|
||||
workspaces:
|
||||
- name: shared-workspace
|
||||
volumeClaimTemplate:
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
- name: docker-credentials
|
||||
secret:
|
||||
secretName: harbor-registry
|
||||
params:
|
||||
- name: git-url
|
||||
value: http://192.168.1.49:13001/admin/neon-vortex.git
|
||||
- name: git-revision
|
||||
value: main
|
||||
- name: image-name
|
||||
value: images.caffeinetux.com/apps/neon-vortex
|
||||
- name: image-tag
|
||||
value: v1.0.0
|
||||
EOF
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Webhook Secret**: Use a strong random secret for Gitea webhooks
|
||||
2. **Registry Credentials**: Keep Harbor credentials in Kubernetes secrets
|
||||
3. **Git Credentials**: Use tokens instead of passwords for Flux git access
|
||||
4. **RBAC**: The pipeline uses minimal RBAC permissions
|
||||
|
||||
## Next Steps
|
||||
|
||||
- Set up image scanning with Trivy
|
||||
- Add automated testing in the pipeline
|
||||
- Configure notifications (Slack, email) for build status
|
||||
- Implement multi-environment deployments (dev, staging, prod)
|
||||
@@ -17,5 +17,5 @@ spec:
|
||||
image:
|
||||
registry: images.caffeinetux.com
|
||||
repository: apps/neon-vortex
|
||||
tag: latest
|
||||
tag: latest # {"$imagepolicy": "flux-system:neon-vortex:tag"}
|
||||
pullPolicy: Always
|
||||
|
||||
14
flux-imagepolicy.yaml
Normal file
14
flux-imagepolicy.yaml
Normal file
@@ -0,0 +1,14 @@
|
||||
apiVersion: image.toolkit.fluxcd.io/v1beta2
|
||||
kind: ImagePolicy
|
||||
metadata:
|
||||
name: neon-vortex
|
||||
namespace: flux-system
|
||||
spec:
|
||||
imageRepositoryRef:
|
||||
name: neon-vortex
|
||||
policy:
|
||||
semver:
|
||||
range: '>=1.0.0'
|
||||
filterTags:
|
||||
pattern: '^v?(?P<version>[0-9]+\.[0-9]+\.[0-9]+)$'
|
||||
extract: '$version'
|
||||
10
flux-imagerepository.yaml
Normal file
10
flux-imagerepository.yaml
Normal file
@@ -0,0 +1,10 @@
|
||||
apiVersion: image.toolkit.fluxcd.io/v1beta2
|
||||
kind: ImageRepository
|
||||
metadata:
|
||||
name: neon-vortex
|
||||
namespace: flux-system
|
||||
spec:
|
||||
image: images.caffeinetux.com/apps/neon-vortex
|
||||
interval: 1m
|
||||
secretRef:
|
||||
name: harbor-registry
|
||||
28
flux-imageupdateautomation.yaml
Normal file
28
flux-imageupdateautomation.yaml
Normal file
@@ -0,0 +1,28 @@
|
||||
apiVersion: image.toolkit.fluxcd.io/v1beta2
|
||||
kind: ImageUpdateAutomation
|
||||
metadata:
|
||||
name: neon-vortex
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 1m
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: neon-vortex
|
||||
namespace: flux-system
|
||||
git:
|
||||
checkout:
|
||||
ref:
|
||||
branch: main
|
||||
commit:
|
||||
author:
|
||||
email: flux@cluster.local
|
||||
name: Flux Image Automation
|
||||
messageTemplate: |
|
||||
Update image to {{range .Updated.Images}}{{println .}}{{end}}
|
||||
|
||||
Automation: flux-system/neon-vortex
|
||||
push:
|
||||
branch: main
|
||||
update:
|
||||
path: ./flux-helmrelease.yaml
|
||||
strategy: Setters
|
||||
225
tekton-pipeline.yaml
Normal file
225
tekton-pipeline.yaml
Normal file
@@ -0,0 +1,225 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: tekton-build-sa
|
||||
namespace: default
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: gitea-webhook-secret
|
||||
namespace: default
|
||||
type: Opaque
|
||||
stringData:
|
||||
secretToken: "change-me-to-random-string"
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: tekton-build-role
|
||||
namespace: default
|
||||
rules:
|
||||
- apiGroups: [""]
|
||||
resources: ["secrets"]
|
||||
verbs: ["get"]
|
||||
- apiGroups: ["batch"]
|
||||
resources: ["jobs"]
|
||||
verbs: ["create", "get", "list", "watch", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: tekton-build-binding
|
||||
namespace: default
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: tekton-build-role
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: tekton-build-sa
|
||||
namespace: default
|
||||
---
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: kaniko-build
|
||||
namespace: default
|
||||
spec:
|
||||
params:
|
||||
- name: IMAGE
|
||||
description: Name (reference) of the image to build
|
||||
- name: DOCKERFILE
|
||||
description: Path to the Dockerfile to build
|
||||
default: ./Dockerfile
|
||||
- name: CONTEXT
|
||||
description: The build context used by Kaniko
|
||||
default: ./
|
||||
- name: EXTRA_ARGS
|
||||
default: ""
|
||||
- name: GIT_URL
|
||||
description: Git repository URL
|
||||
- name: GIT_REVISION
|
||||
description: Git revision to checkout
|
||||
default: main
|
||||
workspaces:
|
||||
- name: source
|
||||
- name: dockerconfig
|
||||
mountPath: /kaniko/.docker
|
||||
steps:
|
||||
- name: git-clone
|
||||
image: alpine/git:latest
|
||||
script: |
|
||||
#!/bin/sh
|
||||
set -e
|
||||
cd $(workspaces.source.path)
|
||||
git clone $(params.GIT_URL) .
|
||||
git checkout $(params.GIT_REVISION)
|
||||
echo "Cloned $(params.GIT_URL) at revision $(params.GIT_REVISION)"
|
||||
- name: build-and-push
|
||||
image: gcr.io/kaniko-project/executor:latest
|
||||
args:
|
||||
- $(params.EXTRA_ARGS)
|
||||
- --dockerfile=$(params.DOCKERFILE)
|
||||
- --context=$(workspaces.source.path)/$(params.CONTEXT)
|
||||
- --destination=$(params.IMAGE)
|
||||
- --cache=true
|
||||
- --cache-repo=$(params.IMAGE)/cache
|
||||
---
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Pipeline
|
||||
metadata:
|
||||
name: neon-vortex-build-pipeline
|
||||
namespace: default
|
||||
spec:
|
||||
params:
|
||||
- name: git-url
|
||||
type: string
|
||||
- name: git-revision
|
||||
type: string
|
||||
default: main
|
||||
- name: image-name
|
||||
type: string
|
||||
- name: image-tag
|
||||
type: string
|
||||
workspaces:
|
||||
- name: shared-workspace
|
||||
- name: docker-credentials
|
||||
tasks:
|
||||
- name: build-image
|
||||
taskRef:
|
||||
name: kaniko-build
|
||||
workspaces:
|
||||
- name: source
|
||||
workspace: shared-workspace
|
||||
- name: dockerconfig
|
||||
workspace: docker-credentials
|
||||
params:
|
||||
- name: IMAGE
|
||||
value: "$(params.image-name):$(params.image-tag)"
|
||||
- name: DOCKERFILE
|
||||
value: "./htlm/Dockerfile"
|
||||
- name: CONTEXT
|
||||
value: "htlm"
|
||||
- name: GIT_URL
|
||||
value: "$(params.git-url)"
|
||||
- name: GIT_REVISION
|
||||
value: "$(params.git-revision)"
|
||||
---
|
||||
apiVersion: triggers.tekton.dev/v1beta1
|
||||
kind: TriggerTemplate
|
||||
metadata:
|
||||
name: neon-vortex-trigger-template
|
||||
namespace: default
|
||||
spec:
|
||||
params:
|
||||
- name: git-revision
|
||||
description: The git revision
|
||||
- name: git-commit-message
|
||||
description: The git commit message
|
||||
- name: git-repo-url
|
||||
description: The git repository url
|
||||
resourcetemplates:
|
||||
- apiVersion: tekton.dev/v1beta1
|
||||
kind: PipelineRun
|
||||
metadata:
|
||||
generateName: neon-vortex-build-
|
||||
namespace: default
|
||||
spec:
|
||||
serviceAccountName: tekton-build-sa
|
||||
pipelineRef:
|
||||
name: neon-vortex-build-pipeline
|
||||
workspaces:
|
||||
- name: shared-workspace
|
||||
volumeClaimTemplate:
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
- name: docker-credentials
|
||||
secret:
|
||||
secretName: harbor-registry
|
||||
params:
|
||||
- name: git-url
|
||||
value: $(tt.params.git-repo-url)
|
||||
- name: git-revision
|
||||
value: $(tt.params.git-revision)
|
||||
- name: image-name
|
||||
value: images.caffeinetux.com/apps/neon-vortex
|
||||
- name: image-tag
|
||||
value: v1.0.$(tt.params.git-revision)
|
||||
---
|
||||
apiVersion: triggers.tekton.dev/v1beta1
|
||||
kind: TriggerBinding
|
||||
metadata:
|
||||
name: neon-vortex-trigger-binding
|
||||
namespace: default
|
||||
spec:
|
||||
params:
|
||||
- name: git-repo-url
|
||||
value: $(body.repository.clone_url)
|
||||
- name: git-revision
|
||||
value: $(body.after)
|
||||
- name: git-commit-message
|
||||
value: $(body.head_commit.message)
|
||||
---
|
||||
apiVersion: triggers.tekton.dev/v1beta1
|
||||
kind: EventListener
|
||||
metadata:
|
||||
name: neon-vortex-listener
|
||||
namespace: default
|
||||
spec:
|
||||
serviceAccountName: tekton-build-sa
|
||||
triggers:
|
||||
- name: gitea-push
|
||||
interceptors:
|
||||
- ref:
|
||||
name: "gitea"
|
||||
params:
|
||||
- name: "secretRef"
|
||||
value:
|
||||
secretName: gitea-webhook-secret
|
||||
secretKey: secretToken
|
||||
- name: "eventTypes"
|
||||
value: ["push"]
|
||||
bindings:
|
||||
- ref: neon-vortex-trigger-binding
|
||||
template:
|
||||
ref: neon-vortex-trigger-template
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: el-neon-vortex-listener
|
||||
namespace: default
|
||||
spec:
|
||||
type: NodePort
|
||||
ports:
|
||||
- port: 8080
|
||||
targetPort: 8080
|
||||
nodePort: 30081
|
||||
selector:
|
||||
eventlistener: neon-vortex-listener
|
||||
Reference in New Issue
Block a user