neon-vortex/CICD_PIPELINE_SETUP.md

# Neon Vortex CI/CD Pipeline Setup

This document describes the complete CI/CD pipeline that automatically builds and deploys Neon Vortex when you push to Gitea.

## Pipeline Flow

1. **Git Push** → Push code to Gitea repository
2. **Gitea Webhook** → Triggers Tekton EventListener
3. **Tekton Pipeline** → Builds container image with Kaniko
4. **Harbor Registry** → Stores the new image
5. **Flux Image Automation** → Detects new image and updates HelmRelease
6. **Flux CD** → Deploys updated application to cluster

## Prerequisites

Before setting up the pipeline, ensure you have:

- Tekton Pipelines installed in your cluster
- Tekton Triggers installed in your cluster
- Flux CD installed and configured
- Harbor registry credentials configured as a Kubernetes secret

## Installation Steps

### 1. Install Tekton (if not already installed)

```bash
# Install Tekton Pipelines
kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml

# Install Tekton Triggers
kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/latest/release.yaml
kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/latest/interceptors.yaml
```

### 2. Ensure Harbor Registry Secret Exists

The pipeline expects a secret named `harbor-registry` in the `default` namespace:

```bash
# Check if secret exists
kubectl get secret harbor-registry -n default

# If not, create it (replace with your Harbor credentials)
kubectl create secret docker-registry harbor-registry \
  --docker-server=images.caffeinetux.com \
  --docker-username=admin \
  --docker-password=YOUR_PASSWORD \
  --docker-email=admin@cluster.local \
  -n default
```

### 3. Update Webhook Secret

Edit the `tekton-pipeline.yaml` file and change the webhook secret:

```yaml
stringData:
  secretToken: "your-random-secret-here"
```

Generate a random secret:
```bash
openssl rand -hex 32
```

### 4. Apply All Configurations

```bash
# Apply Tekton pipeline and triggers
kubectl apply -f tekton-pipeline.yaml

# Apply Flux image automation
kubectl apply -f flux-imagerepository.yaml
kubectl apply -f flux-imagepolicy.yaml
kubectl apply -f flux-imageupdateautomation.yaml

# Apply updated HelmRelease
kubectl apply -f flux-helmrelease.yaml
```

### 5. Configure Gitea Webhook

1. Go to your Gitea repository: `http://192.168.1.49:13001/admin/neon-vortex`
2. Click **Settings** → **Webhooks** → **Add Webhook** → **Gitea**
3. Configure:
   - **Target URL**: `http://el-neon-vortex-listener.default.svc.cluster.local:8080`
     - Or if using NodePort: `http://<NODE_IP>:30081`
   - **HTTP Method**: `POST`
   - **POST Content Type**: `application/json`
   - **Secret**: Use the same secret from step 3
   - **Trigger On**: `Push events`
   - **Branch filter**: `main`
4. Click **Add Webhook**

### 6. Enable Flux Write Access to Git Repository

Flux needs write access to update the HelmRelease file. Update the GitRepository secret:

```bash
# Create or update git credentials for Flux
kubectl create secret generic flux-git-auth \
  --from-literal=username=admin \
  --from-literal=password=YOUR_GITEA_TOKEN \
  -n flux-system \
  --dry-run=client -o yaml | kubectl apply -f -

# Update the GitRepository to use the secret
kubectl patch gitrepository neon-vortex -n flux-system --type merge -p '{"spec":{"secretRef":{"name":"flux-git-auth"}}}'
```

## How It Works

### Image Tagging Strategy

The pipeline uses semantic versioning with git commit SHA:
- Pattern: `v1.0.<git-commit-sha>`
- Example: `v1.0.67a6ae146f1f93aa5a58896419347e543b62fd88`

### Flux Image Policy

The `ImagePolicy` watches for new images matching the semver pattern `>=1.0.0` and automatically updates the HelmRelease when a new image is pushed.

### Automatic Deployment

When Flux detects a new image:
1. Updates `flux-helmrelease.yaml` with the new tag
2. Commits the change back to the repository
3. Reconciles the HelmRelease
4. Kubernetes pulls the new image and performs a rolling update

## Monitoring the Pipeline

### Check Tekton Pipeline Runs

```bash
# List recent pipeline runs
kubectl get pipelinerun -n default

# Watch a specific run
kubectl logs -f <pipelinerun-name> -n default
```

### Check Flux Image Automation

```bash
# Check ImageRepository status
flux get image repository neon-vortex -n flux-system

# Check ImagePolicy status
flux get image policy neon-vortex -n flux-system

# Check ImageUpdateAutomation status
flux get image update neon-vortex -n flux-system
```

### Check HelmRelease Status

```bash
# Check deployment status
flux get helmrelease neon-vortex -n default

# Check pods
kubectl get pods -n default -l app.kubernetes.io/name=neon-vortex
```

## Troubleshooting

### Webhook Not Triggering

1. Check EventListener is running:
```bash
kubectl get pods -n default | grep neon-vortex-listener
```

2. Check EventListener logs:
```bash
kubectl logs -l eventlistener=neon-vortex-listener -n default
```

3. Test webhook from Gitea settings page

### Build Failing

Check the PipelineRun logs:
```bash
kubectl get pipelinerun -n default
kubectl logs <pipelinerun-name> -n default -f
```

### Image Not Updating

1. Check ImageRepository can access Harbor:
```bash
flux get image repository neon-vortex -n flux-system
```

2. Check ImagePolicy is finding images:
```bash
kubectl describe imagepolicy neon-vortex -n flux-system
```

3. Verify Flux has write access to Git:
```bash
kubectl logs -n flux-system deployment/image-automation-controller
```

## Manual Trigger

To manually trigger a build without pushing to Git:

```bash
# Create a PipelineRun manually
kubectl create -f - <<EOF
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  generateName: neon-vortex-build-manual-
  namespace: default
spec:
  serviceAccountName: tekton-build-sa
  pipelineRef:
    name: neon-vortex-build-pipeline
  workspaces:
  - name: shared-workspace
    volumeClaimTemplate:
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
  - name: docker-credentials
    secret:
      secretName: harbor-registry
  params:
  - name: git-url
    value: http://192.168.1.49:13001/admin/neon-vortex.git
  - name: git-revision
    value: main
  - name: image-name
    value: images.caffeinetux.com/apps/neon-vortex
  - name: image-tag
    value: v1.0.0
EOF
```

## Security Considerations

1. **Webhook Secret**: Use a strong random secret for Gitea webhooks
2. **Registry Credentials**: Keep Harbor credentials in Kubernetes secrets
3. **Git Credentials**: Use tokens instead of passwords for Flux git access
4. **RBAC**: The pipeline uses minimal RBAC permissions

## Next Steps

- Set up image scanning with Trivy
- Add automated testing in the pipeline
- Configure notifications (Slack, email) for build status
- Implement multi-environment deployments (dev, staging, prod)