neon-vortex/SIMPLE_CICD_SETUP.md

# Simple CI/CD Pipeline for Neon Vortex

This pipeline automatically builds and deploys your application when you push changes to Gitea.

## How It Works

The pipeline uses a simple, reliable approach with Kubernetes CronJob:

1. **Git Push** → You push code to your Gitea repository
2. **CronJob Poll** → Every 2 minutes, a CronJob checks for new commits
3. **Kaniko Build** → When changes are detected, Kaniko builds a new container image
4. **Harbor Registry** → The new image is pushed to Harbor with tags:
   - `latest` - Always points to the most recent build
   - `v1.0.<commit-sha>` - Specific version tag (e.g., `v1.0.473de53`)
5. **Flux CD** → Automatically deploys the `latest` tag to your cluster

## What Was Installed

The following resources were created in your cluster:

```bash
# Check the CronJob (runs every 2 minutes)
kubectl get cronjob neon-vortex-build-trigger -n default

# View recent build trigger jobs
kubectl get jobs -n default -l app=neon-vortex

# Check build logs
kubectl logs -n default -l job-name=<job-name> -c kaniko
```

## Current Status

```bash
# Verify the CronJob is active
kubectl get cronjob -n default

# Check when the next build check will run
kubectl describe cronjob neon-vortex-build-trigger -n default | grep "Last Schedule"

# See if any builds are currently running
kubectl get jobs -n default
```

## How to Trigger a Build

### Automatic (Recommended)
Just push your code to Gitea:

```bash
git add .
git commit -m "Your changes"
git push origin main
```

Within 2 minutes, the CronJob will detect the change and start a build automatically.

### Manual Trigger
To manually trigger a build immediately:

```bash
# Create a job from the CronJob
kubectl create job --from=cronjob/neon-vortex-build-trigger manual-build-$(date +%s) -n default
```

## Monitoring Builds

### Watch for New Builds

```bash
# Watch jobs being created
kubectl get jobs -n default -w

# Follow build logs (replace JOB_NAME)
kubectl logs -f -n default job/JOB_NAME -c kaniko
```

### Check Build History

```bash
# List all build jobs
kubectl get jobs -n default -l app=neon-vortex

# Get details of a specific job
kubectl describe job <job-name> -n default
```

### Check Deployed Version

```bash
# See current image in use
kubectl get deployment neon-vortex -n default -o jsonpath='{.spec.template.spec.containers[0].image}'

# Check HelmRelease status
flux get helmrelease neon-vortex -n default
```

## Image Tagging Strategy

Each build creates two tags:
- **`latest`**: Always updated, used by Flux for deployment
- **`v1.0.<commit-sha>`**: Permanent version tag for rollbacks

Example:
```
images.caffeinetux.com/apps/neon-vortex:latest
images.caffeinetux.com/apps/neon-vortex:v1.0.473de53
```

## Rolling Back

To roll back to a previous version:

```bash
# List available versions in Harbor
# (You'll need to check your Harbor web UI or use Docker registry API)

# Update the HelmRelease to use a specific version
flux suspend helmrelease neon-vortex -n default

kubectl patch helmrelease neon-vortex -n default --type merge -p '
{
  "spec": {
    "values": {
      "image": {
        "tag": "v1.0.67a6ae1"
      }
    }
  }
}'

flux resume helmrelease neon-vortex -n default
flux reconcile helmrelease neon-vortex -n default
```

## Adjusting Build Frequency

The CronJob checks for changes every 2 minutes by default. To change this:

```bash
# Edit the CronJob
kubectl edit cronjob neon-vortex-build-trigger -n default

# Change the schedule line:
# */2 * * * * = every 2 minutes
# */5 * * * * = every 5 minutes
# */10 * * * * = every 10 minutes
```

## Troubleshooting

### Builds Not Starting

1. Check if CronJob is running:
```bash
kubectl get cronjob neon-vortex-build-trigger -n default
```

2. Check recent CronJob executions:
```bash
kubectl get jobs -n default -l app=neon-vortex
```

3. View trigger logs:
```bash
# Find the most recent trigger job
TRIGGER_JOB=$(kubectl get jobs -n default -l cronjob=neon-vortex-build-trigger --sort-by=.metadata.creationTimestamp -o jsonpath='{.items[-1].metadata.name}')

# View its logs
kubectl logs job/$TRIGGER_JOB -n default
```

### Build Failures

1. Check the build job status:
```bash
kubectl get jobs -n default
kubectl describe job <failed-job-name> -n default
```

2. View build logs:
```bash
kubectl logs -n default job/<job-name> -c git-clone
kubectl logs -n default job/<job-name> -c kaniko
```

3. Common issues:
   - **Harbor authentication**: Check `harbor-registry` secret exists
   - **Git access**: Ensure Gitea is accessible from cluster
   - **Dockerfile errors**: Check Dockerfile syntax in `htlm/Dockerfile`

### Harbor Authentication Issues

Verify the Harbor registry secret:
```bash
kubectl get secret harbor-registry -n default
kubectl get secret harbor-registry -n default -o jsonpath='{.data.\.dockerconfigjson}' | base64 -d
```

Recreate if needed:
```bash
kubectl delete secret harbor-registry -n default

kubectl create secret docker-registry harbor-registry \
  --docker-server=images.caffeinetux.com \
  --docker-username=admin \
  --docker-password=YOUR_PASSWORD \
  -n default
```

### Deployment Not Updating

1. Check if new image exists in Harbor
2. Verify HelmRelease is using `pullPolicy: Always`:
```bash
kubectl get helmrelease neon-vortex -n default -o yaml | grep -A5 image
```

3. Force Flux to reconcile:
```bash
flux reconcile helmrelease neon-vortex -n default
```

4. Force pods to restart and pull new image:
```bash
kubectl rollout restart deployment neon-vortex -n default
```

## Cleanup Old Build Jobs

Jobs are automatically deleted after 1 hour (ttlSecondsAfterFinished: 3600). To manually clean up:

```bash
# Delete all completed jobs older than 1 hour
kubectl delete jobs -n default -l app=neon-vortex --field-selector status.successful=1
```

## Security Notes

1. **Git Access**: The CronJob pulls from Gitea without authentication (public repo)
   - If your repo is private, add credentials to the git clone command
2. **Harbor Credentials**: Stored securely in Kubernetes secret `harbor-registry`
3. **RBAC**: The build trigger has minimal permissions (only create/manage jobs)

## Next Steps

Consider these enhancements:

1. **Add Tests**: Run tests before building the image
2. **Notifications**: Send Slack/email alerts on build success/failure
3. **Multi-stage Builds**: Optimize image size
4. **Image Scanning**: Add Trivy or Clair for vulnerability scanning
5. **Blue/Green Deployment**: Implement canary releases
6. **Prometheus Metrics**: Monitor build success rates

## Quick Reference

```bash
# Check everything is running
kubectl get cronjob,jobs,pods -n default -l app=neon-vortex

# Trigger immediate build
kubectl create job --from=cronjob/neon-vortex-build-trigger manual-$(date +%s) -n default

# Watch deployment status
watch kubectl get helmrelease,deployment,pods -n default

# View recent changes
flux get helmrelease neon-vortex -n default
kubectl get events -n default --sort-by='.lastTimestamp' | grep neon-vortex
```