Add complete CI/CD pipeline with Tekton and Flux image automation
Some checks failed
Build and Push to Harbor / build-and-push (push) Has been cancelled
Some checks failed
Build and Push to Harbor / build-and-push (push) Has been cancelled
- Add Tekton pipeline and triggers for automated builds on git push - Add Flux ImageRepository to track Harbor registry images - Add Flux ImagePolicy for semantic versioning strategy - Add Flux ImageUpdateAutomation to auto-update HelmRelease - Update HelmRelease with image automation marker - Add comprehensive CI/CD pipeline setup documentation This enables automatic build and deployment when pushing to Gitea: 1. Gitea webhook triggers Tekton pipeline 2. Kaniko builds and pushes image to Harbor 3. Flux detects new image and updates deployment 4. Application automatically deploys to cluster 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
260
CICD_PIPELINE_SETUP.md
Normal file
260
CICD_PIPELINE_SETUP.md
Normal file
@@ -0,0 +1,260 @@
|
||||
# Neon Vortex CI/CD Pipeline Setup
|
||||
|
||||
This document describes the complete CI/CD pipeline that automatically builds and deploys Neon Vortex when you push to Gitea.
|
||||
|
||||
## Pipeline Flow
|
||||
|
||||
1. **Git Push** → Push code to Gitea repository
|
||||
2. **Gitea Webhook** → Triggers Tekton EventListener
|
||||
3. **Tekton Pipeline** → Builds container image with Kaniko
|
||||
4. **Harbor Registry** → Stores the new image
|
||||
5. **Flux Image Automation** → Detects new image and updates HelmRelease
|
||||
6. **Flux CD** → Deploys updated application to cluster
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before setting up the pipeline, ensure you have:
|
||||
|
||||
- Tekton Pipelines installed in your cluster
|
||||
- Tekton Triggers installed in your cluster
|
||||
- Flux CD installed and configured
|
||||
- Harbor registry credentials configured as a Kubernetes secret
|
||||
|
||||
## Installation Steps
|
||||
|
||||
### 1. Install Tekton (if not already installed)
|
||||
|
||||
```bash
|
||||
# Install Tekton Pipelines
|
||||
kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml
|
||||
|
||||
# Install Tekton Triggers
|
||||
kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/latest/release.yaml
|
||||
kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/latest/interceptors.yaml
|
||||
```
|
||||
|
||||
### 2. Ensure Harbor Registry Secret Exists
|
||||
|
||||
The pipeline expects a secret named `harbor-registry` in the `default` namespace:
|
||||
|
||||
```bash
|
||||
# Check if secret exists
|
||||
kubectl get secret harbor-registry -n default
|
||||
|
||||
# If not, create it (replace with your Harbor credentials)
|
||||
kubectl create secret docker-registry harbor-registry \
|
||||
--docker-server=images.caffeinetux.com \
|
||||
--docker-username=admin \
|
||||
--docker-password=YOUR_PASSWORD \
|
||||
--docker-email=admin@cluster.local \
|
||||
-n default
|
||||
```
|
||||
|
||||
### 3. Update Webhook Secret
|
||||
|
||||
Edit the `tekton-pipeline.yaml` file and change the webhook secret:
|
||||
|
||||
```yaml
|
||||
stringData:
|
||||
secretToken: "your-random-secret-here"
|
||||
```
|
||||
|
||||
Generate a random secret:
|
||||
```bash
|
||||
openssl rand -hex 32
|
||||
```
|
||||
|
||||
### 4. Apply All Configurations
|
||||
|
||||
```bash
|
||||
# Apply Tekton pipeline and triggers
|
||||
kubectl apply -f tekton-pipeline.yaml
|
||||
|
||||
# Apply Flux image automation
|
||||
kubectl apply -f flux-imagerepository.yaml
|
||||
kubectl apply -f flux-imagepolicy.yaml
|
||||
kubectl apply -f flux-imageupdateautomation.yaml
|
||||
|
||||
# Apply updated HelmRelease
|
||||
kubectl apply -f flux-helmrelease.yaml
|
||||
```
|
||||
|
||||
### 5. Configure Gitea Webhook
|
||||
|
||||
1. Go to your Gitea repository: `http://192.168.1.49:13001/admin/neon-vortex`
|
||||
2. Click **Settings** → **Webhooks** → **Add Webhook** → **Gitea**
|
||||
3. Configure:
|
||||
- **Target URL**: `http://el-neon-vortex-listener.default.svc.cluster.local:8080`
|
||||
- Or if using NodePort: `http://<NODE_IP>:30081`
|
||||
- **HTTP Method**: `POST`
|
||||
- **POST Content Type**: `application/json`
|
||||
- **Secret**: Use the same secret from step 3
|
||||
- **Trigger On**: `Push events`
|
||||
- **Branch filter**: `main`
|
||||
4. Click **Add Webhook**
|
||||
|
||||
### 6. Enable Flux Write Access to Git Repository
|
||||
|
||||
Flux needs write access to update the HelmRelease file. Update the GitRepository secret:
|
||||
|
||||
```bash
|
||||
# Create or update git credentials for Flux
|
||||
kubectl create secret generic flux-git-auth \
|
||||
--from-literal=username=admin \
|
||||
--from-literal=password=YOUR_GITEA_TOKEN \
|
||||
-n flux-system \
|
||||
--dry-run=client -o yaml | kubectl apply -f -
|
||||
|
||||
# Update the GitRepository to use the secret
|
||||
kubectl patch gitrepository neon-vortex -n flux-system --type merge -p '{"spec":{"secretRef":{"name":"flux-git-auth"}}}'
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
### Image Tagging Strategy
|
||||
|
||||
The pipeline uses semantic versioning with git commit SHA:
|
||||
- Pattern: `v1.0.<git-commit-sha>`
|
||||
- Example: `v1.0.67a6ae146f1f93aa5a58896419347e543b62fd88`
|
||||
|
||||
### Flux Image Policy
|
||||
|
||||
The `ImagePolicy` watches for new images matching the semver pattern `>=1.0.0` and automatically updates the HelmRelease when a new image is pushed.
|
||||
|
||||
### Automatic Deployment
|
||||
|
||||
When Flux detects a new image:
|
||||
1. Updates `flux-helmrelease.yaml` with the new tag
|
||||
2. Commits the change back to the repository
|
||||
3. Reconciles the HelmRelease
|
||||
4. Kubernetes pulls the new image and performs a rolling update
|
||||
|
||||
## Monitoring the Pipeline
|
||||
|
||||
### Check Tekton Pipeline Runs
|
||||
|
||||
```bash
|
||||
# List recent pipeline runs
|
||||
kubectl get pipelinerun -n default
|
||||
|
||||
# Watch a specific run
|
||||
kubectl logs -f <pipelinerun-name> -n default
|
||||
```
|
||||
|
||||
### Check Flux Image Automation
|
||||
|
||||
```bash
|
||||
# Check ImageRepository status
|
||||
flux get image repository neon-vortex -n flux-system
|
||||
|
||||
# Check ImagePolicy status
|
||||
flux get image policy neon-vortex -n flux-system
|
||||
|
||||
# Check ImageUpdateAutomation status
|
||||
flux get image update neon-vortex -n flux-system
|
||||
```
|
||||
|
||||
### Check HelmRelease Status
|
||||
|
||||
```bash
|
||||
# Check deployment status
|
||||
flux get helmrelease neon-vortex -n default
|
||||
|
||||
# Check pods
|
||||
kubectl get pods -n default -l app.kubernetes.io/name=neon-vortex
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Webhook Not Triggering
|
||||
|
||||
1. Check EventListener is running:
|
||||
```bash
|
||||
kubectl get pods -n default | grep neon-vortex-listener
|
||||
```
|
||||
|
||||
2. Check EventListener logs:
|
||||
```bash
|
||||
kubectl logs -l eventlistener=neon-vortex-listener -n default
|
||||
```
|
||||
|
||||
3. Test webhook from Gitea settings page
|
||||
|
||||
### Build Failing
|
||||
|
||||
Check the PipelineRun logs:
|
||||
```bash
|
||||
kubectl get pipelinerun -n default
|
||||
kubectl logs <pipelinerun-name> -n default -f
|
||||
```
|
||||
|
||||
### Image Not Updating
|
||||
|
||||
1. Check ImageRepository can access Harbor:
|
||||
```bash
|
||||
flux get image repository neon-vortex -n flux-system
|
||||
```
|
||||
|
||||
2. Check ImagePolicy is finding images:
|
||||
```bash
|
||||
kubectl describe imagepolicy neon-vortex -n flux-system
|
||||
```
|
||||
|
||||
3. Verify Flux has write access to Git:
|
||||
```bash
|
||||
kubectl logs -n flux-system deployment/image-automation-controller
|
||||
```
|
||||
|
||||
## Manual Trigger
|
||||
|
||||
To manually trigger a build without pushing to Git:
|
||||
|
||||
```bash
|
||||
# Create a PipelineRun manually
|
||||
kubectl create -f - <<EOF
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: PipelineRun
|
||||
metadata:
|
||||
generateName: neon-vortex-build-manual-
|
||||
namespace: default
|
||||
spec:
|
||||
serviceAccountName: tekton-build-sa
|
||||
pipelineRef:
|
||||
name: neon-vortex-build-pipeline
|
||||
workspaces:
|
||||
- name: shared-workspace
|
||||
volumeClaimTemplate:
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
- name: docker-credentials
|
||||
secret:
|
||||
secretName: harbor-registry
|
||||
params:
|
||||
- name: git-url
|
||||
value: http://192.168.1.49:13001/admin/neon-vortex.git
|
||||
- name: git-revision
|
||||
value: main
|
||||
- name: image-name
|
||||
value: images.caffeinetux.com/apps/neon-vortex
|
||||
- name: image-tag
|
||||
value: v1.0.0
|
||||
EOF
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Webhook Secret**: Use a strong random secret for Gitea webhooks
|
||||
2. **Registry Credentials**: Keep Harbor credentials in Kubernetes secrets
|
||||
3. **Git Credentials**: Use tokens instead of passwords for Flux git access
|
||||
4. **RBAC**: The pipeline uses minimal RBAC permissions
|
||||
|
||||
## Next Steps
|
||||
|
||||
- Set up image scanning with Trivy
|
||||
- Add automated testing in the pipeline
|
||||
- Configure notifications (Slack, email) for build status
|
||||
- Implement multi-environment deployments (dev, staging, prod)
|
||||
Reference in New Issue
Block a user