# Simple CI/CD Pipeline for Neon Vortex This pipeline automatically builds and deploys your application when you push changes to Gitea. ## How It Works The pipeline uses a simple, reliable approach with Kubernetes CronJob: 1. **Git Push** → You push code to your Gitea repository 2. **CronJob Poll** → Every 2 minutes, a CronJob checks for new commits 3. **Kaniko Build** → When changes are detected, Kaniko builds a new container image 4. **Harbor Registry** → The new image is pushed to Harbor with tags: - `latest` - Always points to the most recent build - `v1.0.` - Specific version tag (e.g., `v1.0.473de53`) 5. **Flux CD** → Automatically deploys the `latest` tag to your cluster ## What Was Installed The following resources were created in your cluster: ```bash # Check the CronJob (runs every 2 minutes) kubectl get cronjob neon-vortex-build-trigger -n default # View recent build trigger jobs kubectl get jobs -n default -l app=neon-vortex # Check build logs kubectl logs -n default -l job-name= -c kaniko ``` ## Current Status ```bash # Verify the CronJob is active kubectl get cronjob -n default # Check when the next build check will run kubectl describe cronjob neon-vortex-build-trigger -n default | grep "Last Schedule" # See if any builds are currently running kubectl get jobs -n default ``` ## How to Trigger a Build ### Automatic (Recommended) Just push your code to Gitea: ```bash git add . git commit -m "Your changes" git push origin main ``` Within 2 minutes, the CronJob will detect the change and start a build automatically. ### Manual Trigger To manually trigger a build immediately: ```bash # Create a job from the CronJob kubectl create job --from=cronjob/neon-vortex-build-trigger manual-build-$(date +%s) -n default ``` ## Monitoring Builds ### Watch for New Builds ```bash # Watch jobs being created kubectl get jobs -n default -w # Follow build logs (replace JOB_NAME) kubectl logs -f -n default job/JOB_NAME -c kaniko ``` ### Check Build History ```bash # List all build jobs kubectl get jobs -n default -l app=neon-vortex # Get details of a specific job kubectl describe job -n default ``` ### Check Deployed Version ```bash # See current image in use kubectl get deployment neon-vortex -n default -o jsonpath='{.spec.template.spec.containers[0].image}' # Check HelmRelease status flux get helmrelease neon-vortex -n default ``` ## Image Tagging Strategy Each build creates two tags: - **`latest`**: Always updated, used by Flux for deployment - **`v1.0.`**: Permanent version tag for rollbacks Example: ``` images.caffeinetux.com/apps/neon-vortex:latest images.caffeinetux.com/apps/neon-vortex:v1.0.473de53 ``` ## Rolling Back To roll back to a previous version: ```bash # List available versions in Harbor # (You'll need to check your Harbor web UI or use Docker registry API) # Update the HelmRelease to use a specific version flux suspend helmrelease neon-vortex -n default kubectl patch helmrelease neon-vortex -n default --type merge -p ' { "spec": { "values": { "image": { "tag": "v1.0.67a6ae1" } } } }' flux resume helmrelease neon-vortex -n default flux reconcile helmrelease neon-vortex -n default ``` ## Adjusting Build Frequency The CronJob checks for changes every 2 minutes by default. To change this: ```bash # Edit the CronJob kubectl edit cronjob neon-vortex-build-trigger -n default # Change the schedule line: # */2 * * * * = every 2 minutes # */5 * * * * = every 5 minutes # */10 * * * * = every 10 minutes ``` ## Troubleshooting ### Builds Not Starting 1. Check if CronJob is running: ```bash kubectl get cronjob neon-vortex-build-trigger -n default ``` 2. Check recent CronJob executions: ```bash kubectl get jobs -n default -l app=neon-vortex ``` 3. View trigger logs: ```bash # Find the most recent trigger job TRIGGER_JOB=$(kubectl get jobs -n default -l cronjob=neon-vortex-build-trigger --sort-by=.metadata.creationTimestamp -o jsonpath='{.items[-1].metadata.name}') # View its logs kubectl logs job/$TRIGGER_JOB -n default ``` ### Build Failures 1. Check the build job status: ```bash kubectl get jobs -n default kubectl describe job -n default ``` 2. View build logs: ```bash kubectl logs -n default job/ -c git-clone kubectl logs -n default job/ -c kaniko ``` 3. Common issues: - **Harbor authentication**: Check `harbor-registry` secret exists - **Git access**: Ensure Gitea is accessible from cluster - **Dockerfile errors**: Check Dockerfile syntax in `htlm/Dockerfile` ### Harbor Authentication Issues Verify the Harbor registry secret: ```bash kubectl get secret harbor-registry -n default kubectl get secret harbor-registry -n default -o jsonpath='{.data.\.dockerconfigjson}' | base64 -d ``` Recreate if needed: ```bash kubectl delete secret harbor-registry -n default kubectl create secret docker-registry harbor-registry \ --docker-server=images.caffeinetux.com \ --docker-username=admin \ --docker-password=YOUR_PASSWORD \ -n default ``` ### Deployment Not Updating 1. Check if new image exists in Harbor 2. Verify HelmRelease is using `pullPolicy: Always`: ```bash kubectl get helmrelease neon-vortex -n default -o yaml | grep -A5 image ``` 3. Force Flux to reconcile: ```bash flux reconcile helmrelease neon-vortex -n default ``` 4. Force pods to restart and pull new image: ```bash kubectl rollout restart deployment neon-vortex -n default ``` ## Cleanup Old Build Jobs Jobs are automatically deleted after 1 hour (ttlSecondsAfterFinished: 3600). To manually clean up: ```bash # Delete all completed jobs older than 1 hour kubectl delete jobs -n default -l app=neon-vortex --field-selector status.successful=1 ``` ## Security Notes 1. **Git Access**: The CronJob pulls from Gitea without authentication (public repo) - If your repo is private, add credentials to the git clone command 2. **Harbor Credentials**: Stored securely in Kubernetes secret `harbor-registry` 3. **RBAC**: The build trigger has minimal permissions (only create/manage jobs) ## Next Steps Consider these enhancements: 1. **Add Tests**: Run tests before building the image 2. **Notifications**: Send Slack/email alerts on build success/failure 3. **Multi-stage Builds**: Optimize image size 4. **Image Scanning**: Add Trivy or Clair for vulnerability scanning 5. **Blue/Green Deployment**: Implement canary releases 6. **Prometheus Metrics**: Monitor build success rates ## Quick Reference ```bash # Check everything is running kubectl get cronjob,jobs,pods -n default -l app=neon-vortex # Trigger immediate build kubectl create job --from=cronjob/neon-vortex-build-trigger manual-$(date +%s) -n default # Watch deployment status watch kubectl get helmrelease,deployment,pods -n default # View recent changes flux get helmrelease neon-vortex -n default kubectl get events -n default --sort-by='.lastTimestamp' | grep neon-vortex ```