12 KiB

Raw Blame History

Getting Started

This document describes a getting started guide for jenkins-operator and an additional configuration.

First Steps
Deploy Jenkins
Configure Seed Jobs and Pipelines
Install Plugins
Configure Backup & Restore
Debugging

First Steps

Prepare your Kubernetes cluster and set up access. Once you have running Kubernetes cluster you can focus on installing jenkins-operator according to the Installation guide.

Deploy Jenkins

Once jenkins-operator is up and running let's deploy actual Jenkins instance. Create manifest ie. jenkins_instance.yaml with following data and save it on drive.

apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: example
spec:
  master:
   image: jenkins/jenkins
   readinessProbe:
     httpGet:
       path: /login
       port: 8080
       scheme: HTTP
     failureThreshold: 12
     initialDelaySeconds: 20
     periodSeconds: 10
     successThreshold: 1
     timeoutSeconds: 5
   livenessProbe:
     httpGet:
       path: /login
       port: 8080
       scheme: HTTP
     initialDelaySeconds: 20
     failureThreshold: 12
     periodSeconds: 10
     successThreshold: 1
     timeoutSeconds: 5
  seedJobs:
  - id: jenkins-operator
    targets: "cicd/jobs/*.jenkins"
    description: "Jenkins Operator repository"
    repositoryBranch: master
    repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git

Deploy Jenkins to K8s:

kubectl create -f jenkins_instance.yaml

Watch Jenkins instance being created:

kubectl get pods -w

Get Jenkins credentials:

kubectl get secret jenkins-operator-credentials-example -o 'jsonpath={.data.user}' | base64 -d
kubectl get secret jenkins-operator-credentials-example -o 'jsonpath={.data.password}' | base64 -d

Connect to Jenkins (minikube):

minikube service jenkins-operator-example --url

Pick up the first URL.

Connect to Jenkins (actual Kubernetes cluster):

kubectl describe svc jenkins-operator-example
kubectl port-forward jenkins-operator-example 8080:8080

Then open browser with address http://localhost:8080.

Configure Seed Jobs and Pipelines

Jenkins operator uses job-dsl and kubernetes-credentials-provider plugins for configuring jobs and deploy keys.

Prepare job definitions and pipelines

First you have to prepare pipelines and job definition in your GitHub repository using the following structure:

cicd/
├── jobs
│   └── build.jenkins
└── pipelines
    └── build.jenkins

cicd/jobs/build.jenkins it's a job definition:

#!/usr/bin/env groovy

pipelineJob('build-jenkins-operator') {
    displayName('Build jenkins-operator')

    definition {
        cpsScm {
            scm {
                git {
                    remote {
                        url('https://github.com/jenkinsci/kubernetes-operator.git')
                        credentials('jenkins-operator')
                    }
                    branches('*/master')
                }
            }
            scriptPath('cicd/pipelines/build.jenkins')
        }
    }
}

cicd/jobs/build.jenkins it's an actual Jenkins pipeline:

#!/usr/bin/env groovy

def label = "build-jenkins-operator-${UUID.randomUUID().toString()}"
def home = "/home/jenkins"
def workspace = "${home}/workspace/build-jenkins-operator"
def workdir = "${workspace}/src/github.com/jenkinsci/kubernetes-operator/"

podTemplate(label: label,
        containers: [
                containerTemplate(name: 'jnlp', image: 'jenkins/jnlp-slave:alpine'),
                containerTemplate(name: 'go', image: 'golang:1-alpine', command: 'cat', ttyEnabled: true),
        ]) {

    node(label) {
        dir(workdir) {
            stage('Init') {
                timeout(time: 3, unit: 'MINUTES') {
                    checkout scm
                }
                container('go') {
                    sh 'apk --no-cache --update add make git gcc libc-dev'
                }
            }

            stage('Build') {
                container('go') {
                    sh 'make build'
                }
            }
        }
    }
}

Configure Seed Jobs

Jenkins Seed Jobs are configured using Jenkins.spec.seedJobs section from your custom resource manifest:

apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: example
spec:
  master:
   image: jenkins/jenkins:lts
  seedJobs:
  - id: jenkins-operator
    targets: "cicd/jobs/*.jenkins"
    description: "Jenkins Operator repository"
    repositoryBranch: master
    repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git

jenkins-operator will automatically discover and configure all seed jobs.

You can verify if deploy keys were successfully configured in Jenkins Credentials tab.

You can verify if your pipelines were successfully configured in Jenkins Seed Job console output.

If your GitHub repository is private you have to configure SSH or username/password authentication.

SSH authentication

Configure seed job like:

apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: example
spec:
  master:
   image: jenkins/jenkins:lts
  seedJobs:
  - id: jenkins-operator-ssh
    credentialType: basicSSHUserPrivateKey
    credentialID: k8s-ssh
    targets: "cicd/jobs/*.jenkins"
    description: "Jenkins Operator repository"
    repositoryBranch: master
    repositoryUrl: git@github.com:jenkinsci/kubernetes-operator.git

and create Kubernetes Secret(name of secret should be the same from credentialID field):

apiVersion: v1
kind: Secret
metadata:
  name: k8s-ssh
data:
  privateKey: |
    -----BEGIN RSA PRIVATE KEY-----
    MIIJKAIBAAKCAgEAxxDpleJjMCN5nusfW/AtBAZhx8UVVlhhhIKXvQ+dFODQIdzO
    oDXybs1zVHWOj31zqbbJnsfsVZ9Uf3p9k6xpJ3WFY9b85WasqTDN1xmSd6swD4N8
    ...
  username: github_user_name

Username & password authentication

Configure seed job like:

apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: example
spec:
  master:
   image: jenkins/jenkins:lts
  seedJobs:
  - id: jenkins-operator-user-pass
    credentialType: usernamePassword
    credentialID: k8s-user-pass
    targets: "cicd/jobs/*.jenkins"
    description: "Jenkins Operator repository"
    repositoryBranch: master
    repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git

and create Kubernetes Secret(name of secret should be the same from credentialID field):

apiVersion: v1
kind: Secret
metadata:
  name: k8s-user-pass
data:
  username: github_user_name
  password: password_or_token

Jenkins Customisation

Jenkins can be customized using groovy scripts or configuration as code plugin. All custom configuration is stored in the jenkins-operator-user-configuration-example ConfigMap which is automatically created by jenkins-operator.

jenkins-operator creates jenkins-operator-user-configuration-example secret where user can store sensitive information used for custom configuration. If you have entry in secret named PASSWORD then you can use it in Configuration as Plugin as adminAddress: "${PASSWORD}".

kubectl get secret jenkins-operator-user-configuration-example -o yaml

apiVersion: v1
data:
  SECRET_JENKINS_ADMIN_ADDRESS: YXNkZgo=
kind: Secret
metadata:
  creationTimestamp: 2019-03-03T11:54:36Z
  labels:
    app: jenkins-operator
    jenkins-cr: example
    watch: "true"
  name: jenkins-operator-user-configuration-example
  namespace: default
type: Opaque

kubectl get configmap jenkins-operator-user-configuration-example -o yaml

apiVersion: v1
data:
  1-configure-theme.groovy: |2
    import jenkins.*
    import jenkins.model.*
    import hudson.*
    import hudson.model.*
    import org.jenkinsci.plugins.simpletheme.ThemeElement
    import org.jenkinsci.plugins.simpletheme.CssTextThemeElement
    import org.jenkinsci.plugins.simpletheme.CssUrlThemeElement

    Jenkins jenkins = Jenkins.getInstance()

    def decorator = Jenkins.instance.getDescriptorByType(org.codefirst.SimpleThemeDecorator.class)

    List<ThemeElement> configElements = new ArrayList<>();
    configElements.add(new CssTextThemeElement("DEFAULT"));
    configElements.add(new CssUrlThemeElement("https://cdn.rawgit.com/afonsof/jenkins-material-theme/gh-pages/dist/material-light-green.css"));
    decorator.setElements(configElements);
    decorator.save();

    jenkins.save()
  1-system-message.yaml: |2
    jenkins:
      systemMessage: "Configuration as Code integration works!!!"
      adminAddress: "${SECRET_JENKINS_ADMIN_ADDRESS}"
kind: ConfigMap
metadata:
  labels:
    app: jenkins-operator
    jenkins-cr: example
    watch: "true"
  name: jenkins-operator-user-configuration-example
  namespace: default

When jenkins-operator-user-configuration-example ConfigMap is updated Jenkins automatically runs the jenkins-operator-user-configuration Jenkins Job which executes all scripts then runs the jenkins-operator-user-configuration-casc Jenkins Job which applies Configuration as Code configuration.

Install Plugins

Via CR

Edit CR under spec.master.plugins:

apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: example
spec:
  master:
   image: jenkins/jenkins:lts
   plugins:
     simple-theme-plugin:0.5.1: []
     configuration-as-code:1.4:
     - configuration-as-code-support:1.4

Then jenkins-operator will automatically install plugins after Jenkins master pod restart.

Configure backup and restore

Backup and restore is done by container sidecar.

PVC

Create PVC

Save to file pvc.yaml:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: <pvc_name>
  namespace: <namesapce>
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 500Gi

Run command:

$ kubectl -n <namesapce> create -f pvc.yaml

Configure Jenkins CR

apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: <cr_name>
  namespace: <namespace>
spec:
  backup:
    action:
      exec:
        command:
        - /home/user/bin/backup.sh # this command is invoked on "backup" container to make backup, for example /home/user/bin/backup.sh <backup_number>, <backup_number> is passed by operator
    containerName: backup # container name is responsible for backup
    interval: 30 # how often make backup in seconds
    makeBackupBeforePodDeletion: true # make backup before pod deletion
  master:
    containers:
    - name: jenkins-master
      image: jenkins/jenkins:lts
    - name: backup # container responsible for backup and restore
      env:
      - name: BACKUP_DIR
        value: /backup
      - name: JENKINS_HOME
        value: /jenkins-home
      image: virtuslab/jenkins-operator-backup-pvc:v0.0.2 # look at backup/pvc directory
      imagePullPolicy: IfNotPresent
      volumeMounts:
      - mountPath: /jenkins-home # Jenkins home volume
        name: jenkins-home
      - mountPath: /backup # backup volume
        name: backup
    volumes:
    - name: backup # PVC volume where backups will be stored
      persistentVolumeClaim:
        claimName: <pvc_name>
  restore:
    action:
      exec:
        command:
        - /home/user/bin/restore.sh # this command is invoked on "backup" container to make restore backup, for example /home/user/bin/restore.sh <backup_number>, <backup_number> is passed by operator
    containerName: backup # container name is responsible for restore backup
    #recoveryOnce: <backup_number> # if want to restore specific backup configure this field and then Jenkins will be restarted and desired backup will be restored

Debugging

Turn on debug in jenkins-operator deployment:

sed -i 's|\(args:\).*|\1\ ["--debug"\]|' deploy/operator.yaml
kubectl apply -f deploy/operator.yaml

Watch Kubernetes events:

kubectl get events --sort-by='{.lastTimestamp}'

Verify Jenkins master logs:

kubectl logs -f jenkins-master-example

Verify jenkins-operator logs:

kubectl logs deployment/jenkins-operator

Troubleshooting

Delete Jenkins master pod and wait for the new one to come up:

kubectl delete pod jenkins-operator-example

12 KiB Raw Blame History