--- title: "Developer Guide" linkTitle: "Developer Guide" weight: 4 date: 2021-08-19 description: > Jenkins Operator for developers --- {{% pageinfo %}} This document explains how to setup your development environment. {{% /pageinfo %}} ## Prerequisites - [operator_sdk][operator_sdk] version 1.3.0 - [git][git_tool] - [go][go_tool] version 1.15.6 - [goimports, golint, checkmake and staticcheck][install_dev_tools] - [minikube][minikube] version 1.21.0 (preferred Hypervisor - [virtualbox][virtualbox]) (automatically downloaded) - [docker][docker_tool] version 17.03+ ## Clone repository and download dependencies ```bash git clone git@github.com:jenkinsci/kubernetes-operator.git cd kubernetes-operator make go-dependencies ``` ## Build and run with a minikube Start minikube instance configured for **Jenkins Operator**. Appropriate minikube version will be downloaded to bin folder. ```bash make minikube-start ``` Next run **Jenkins Operator** locally. ```bash make run ``` Console output indicating readiness of this phase: ```bash + build + run kubectl config use-context minikube Switched to context "minikube". Watching 'default' namespace bin/manager --jenkins-api-hostname=192.168.99.252 --jenkins-api-port=0 --jenkins-api-use-nodeport=true --cluster-domain=cluster.local 2021-02-08T14:14:45.263+0100 INFO cmd Version: v0.5.0 2021-02-08T14:14:45.263+0100 INFO cmd Git commit: 305dbeda-dirty-dirty 2021-02-08T14:14:45.264+0100 INFO cmd Go Version: go1.15.6 2021-02-08T14:14:45.264+0100 INFO cmd Go OS/Arch: darwin/amd64 2021-02-08T14:14:45.264+0100 INFO cmd Watch namespace: default 2021-02-08T14:14:45.592+0100 INFO controller-runtime.metrics metrics server is starting to listen {"addr": "0.0.0.0:8383"} 2021-02-08T14:14:45.599+0100 INFO cmd starting manager 2021-02-08T14:14:45.599+0100 INFO controller-runtime.manager starting metrics server {"path": "/metrics"} 2021-02-08T14:14:45.599+0100 INFO controller-runtime.manager.controller.jenkins Starting EventSource {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "source": "kind source: jenkins.io/v1alpha2, Kind=Jenkins"} 2021-02-08T14:14:45.700+0100 INFO controller-runtime.manager.controller.jenkins Starting EventSource {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "source": "kind source: /, Kind="} 2021-02-08T14:14:45.800+0100 INFO controller-runtime.manager.controller.jenkins Starting EventSource {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "source": "kind source: /, Kind="} 2021-02-08T14:14:45.901+0100 INFO controller-runtime.manager.controller.jenkins Starting EventSource {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "source": "kind source: /, Kind="} 2021-02-08T14:14:46.003+0100 INFO controller-runtime.manager.controller.jenkins Starting EventSource {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "source": "kind source: core/v1, Kind=Secret"} 2021-02-08T14:14:46.004+0100 INFO controller-runtime.manager.controller.jenkins Starting EventSource {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "source": "kind source: core/v1, Kind=ConfigMap"} 2021-02-08T14:14:46.004+0100 INFO controller-runtime.manager.controller.jenkins Starting EventSource {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "source": "kind source: jenkins.io/v1alpha2, Kind=Jenkins"} 2021-02-08T14:14:46.004+0100 INFO controller-runtime.manager.controller.jenkins Starting Controller {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins"} 2021-02-08T14:14:46.004+0100 INFO controller-runtime.manager.controller.jenkins Starting workers {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "worker count": 1} ``` Lastly apply Jenkins Custom Resource to minikube cluster: ```bash kubectl apply -f config/samples/jenkins.io_v1alpha2_jenkins.yaml {"level":"info","ts":1612790690.875426,"logger":"controller-jenkins","msg":"Setting default Jenkins container command","cr":"jenkins-example"} {"level":"info","ts":1612790690.8754492,"logger":"controller-jenkins","msg":"Setting default Jenkins container JAVA_OPTS environment variable","cr":"jenkins-example"} {"level":"info","ts":1612790690.875456,"logger":"controller-jenkins","msg":"Setting default operator plugins","cr":"jenkins-example"} {"level":"info","ts":1612790690.875463,"logger":"controller-jenkins","msg":"Setting default Jenkins master service","cr":"jenkins-example"} {"level":"info","ts":1612790690.875467,"logger":"controller-jenkins","msg":"Setting default Jenkins slave service","cr":"jenkins-example"} {"level":"info","ts":1612790690.881811,"logger":"controller-jenkins","msg":"*v1alpha2.Jenkins/jenkins-example has been updated","cr":"jenkins-example"} {"level":"info","ts":1612790691.252834,"logger":"controller-jenkins","msg":"Creating a new Jenkins Master Pod default/jenkins-jenkins-example","cr":"jenkins-example"} {"level":"info","ts":1612790691.322793,"logger":"controller-jenkins","msg":"Jenkins master pod restarted by operator:","cr":"jenkins-example"} {"level":"info","ts":1612790691.322817,"logger":"controller-jenkins","msg":"Jenkins Operator version has changed, actual '' new 'v0.5.0'","cr":"jenkins-example"} {"level":"info","ts":1612790691.3228202,"logger":"controller-jenkins","msg":"Jenkins CR has been replaced","cr":"jenkins-example"} {"level":"info","ts":1612790695.8789551,"logger":"controller-jenkins","msg":"Creating a new Jenkins Master Pod default/jenkins-jenkins-example","cr":"jenkins-example"} {"level":"warn","ts":1612790817.9423082,"logger":"controller-jenkins","msg":"Reconcile loop failed: couldn't init Jenkins API client: Get \"http://192.168.99.254:31998/api/json\": dial tcp 192.168.99.254:31998: connect: connection refused","cr":"jenkins-example"} {"level":"warn","ts":1612790817.9998221,"logger":"controller-jenkins","msg":"Reconcile loop failed: couldn't init Jenkins API client: Get \"http://192.168.99.254:31998/api/json\": dial tcp 192.168.99.254:31998: connect: connection refused","cr":"jenkins-example"} {"level":"info","ts":1612790818.581316,"logger":"controller-jenkins","msg":"base-groovy ConfigMap 'jenkins-operator-base-configuration-jenkins-example' name '1-basic-settings.groovy' running groovy script","cr":"jenkins-example"} ... {"level":"info","ts":1612790820.9473379,"logger":"controller-jenkins","msg":"base-groovy ConfigMap 'jenkins-operator-base-configuration-jenkins-example' name '8-disable-job-dsl-script-approval.groovy' running groovy script","cr":"jenkins-example"} {"level":"info","ts":1612790821.244055,"logger":"controller-jenkins","msg":"Base configuration phase is complete, took 2m6s","cr":"jenkins-example"} {"level":"info","ts":1612790821.7953842,"logger":"controller-jenkins","msg":"Waiting for Seed Job Agent `seed-job-agent`...","cr":"jenkins-example"} ... {"level":"info","ts":1612790851.843638,"logger":"controller-jenkins","msg":"Waiting for Seed Job Agent `seed-job-agent`...","cr":"jenkins-example"} {"level":"info","ts":1612790853.489524,"logger":"controller-jenkins","msg":"User configuration phase is complete, took 2m38s","cr":"jenkins-example"} Two log lines says that Jenkins Operator works correctly: * `Base configuration phase is complete` - ensures manifests, Jenkins pod, Jenkins configuration and Jenkins API token * `User configuration phase is complete` - ensures Jenkins restore, backup and seed jobs along with user configuration > Details about base and user phase can be found [here](https://jenkinsci.github.io/kubernetes-operator/docs/how-it-works/architecture-and-design/). ``` ```bash kubectl get jenkins -o yaml apiVersion: v1 items: - apiVersion: jenkins.io/v1alpha2 kind: Jenkins metadata: ... spec: backup: action: {} containerName: "" interval: 0 makeBackupBeforePodDeletion: false configurationAsCode: configurations: [] secret: name: "" groovyScripts: configurations: [] secret: name: "" jenkinsAPISettings: authorizationStrategy: createUser master: basePlugins: ... containers: - command: - bash - -c - /var/jenkins/scripts/init.sh && exec /sbin/tini -s -- /usr/local/bin/jenkins.sh env: - name: JAVA_OPTS value: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -Djenkins.install.runSetupWizard=false -Djava.awt.headless=true image: jenkins/jenkins:2.263.3-lts-alpine imagePullPolicy: Always livenessProbe: ... readinessProbe: ... resources: limits: cpu: 1500m memory: 3Gi requests: cpu: "1" memory: 500Mi disableCSRFProtection: false restore: action: {} containerName: "" getLatestAction: {} seedJobs: - additionalClasspath: "" bitbucketPushTrigger: false buildPeriodically: "" description: Jenkins Operator repository failOnMissingPlugin: false githubPushTrigger: false id: jenkins-operator ignoreMissingFiles: false pollSCM: "" repositoryBranch: master repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git targets: cicd/jobs/*.jenkins unstableOnDeprecation: false service: port: 8080 type: NodePort serviceAccount: {} slaveService: port: 50000 type: ClusterIP status: appliedGroovyScripts: - configurationType: base-groovy hash: 2ownqpRyBjQYmzTRttUx7axok3CKe2E45frI5iRwH0w= name: 1-basic-settings.groovy source: jenkins-operator-base-configuration-jenkins-example ... baseConfigurationCompletedTime: "2021-02-08T13:27:01Z" createdSeedJobs: - jenkins-operator operatorVersion: v0.5.0 provisionStartTime: "2021-02-08T13:24:55Z" userAndPasswordHash: nnfZsWmFfAYlYyVYeKhWW2KB4L8mE61JUfetAsr9IMM= userConfigurationCompletedTime: "2021-02-08T13:27:33Z" kind: List metadata: resourceVersion: "" selfLink: "" ``` ```bash kubectl get po NAME READY STATUS RESTARTS AGE jenkins-jenkins-example 1/1 Running 0 23m seed-job-agent-jenkins-example-758cc7cc5c-82hbl 1/1 Running 0 21m ``` ### Debug Jenkins Operator ```bash make run OPERATOR_EXTRA_ARGS="--debug" ``` ### Stop or delete minikube cluster To stop Kubernetes cluster running locally on minikube: ```bash minikube stop ``` To delete the cluster altogether: ```bash minikube delete ``` ## Build and run with a remote Kubernetes cluster You can also run the controller locally and make it listen to a remote Kubernetes server. ```bash make run NAMESPACE=default KUBECTL_CONTEXT=remote-k8s EXTRA_ARGS='--kubeconfig ~/.kube/config' ``` Once **Jenkins Operator** are up and running, apply Jenkins custom resource: ```bash kubectl --context remote-k8s --namespace default apply -f deploy/crds/jenkins_v1alpha2_jenkins_cr.yaml kubectl --context remote-k8s --namespace default get jenkins -o yaml kubectl --context remote-k8s --namespace default get po ``` ## Testing Tests are written using [Ginkgo](https://onsi.github.io/ginkgo/) with [Gomega](https://onsi.github.io/gomega/). Run unit tests with go fmt, lint, staticcheck, vet: ```bash make verify ``` Run unit tests only: ```bash make test ``` ### Running E2E tests Run e2e tests with minikube: ```bash make minikube-start make e2e ``` Run Helm e2e tests: ```bash eval $(bin/minikube docker-env) make helm-e2e ``` Run the specific e2e test: ```bash make e2e E2E_TEST_SELECTOR='^TestConfiguration$' ``` ### Building docker image on minikube To be able to work with the docker daemon on `minikube` machine run the following command before building an image: ```bash eval $(bin/minikube docker-env) ``` ### When `api/v1alpha2/jenkins_types.go` has changed Run: ```bash make manifests ``` ### Getting the Jenkins URL and basic credentials ```bash minikube service jenkins-operator-http- --url kubectl get secret jenkins-operator-credentials- -o 'jsonpath={.data.user}' | base64 -d kubectl get secret jenkins-operator-credentials- -o 'jsonpath={.data.password}' | base64 -d ``` ### Webhook To deploy the operator along with webhook, run : ```bash eval $(minikube docker-env) make deploy-webhook ``` It uses [cert-manager](https://cert-manager.io/) as an external dependency. ## Self-learning * [Tutorial: Deep Dive into the Operator Framework for... Melvin Hillsman, Michael Hrivnak, & Matt Dorn ](https://www.youtube.com/watch?v=8_DaCcRMp5I) * [Operator Framework Training By OpenShift](https://www.katacoda.com/openshift/courses/operatorframework) * [Operator SDK Tutorial for Go](https://sdk.operatorframework.io/docs/building-operators/golang/tutorial/) * [Kubebuilder Validating Webhook Implementation](https://book.kubebuilder.io/cronjob-tutorial/webhook-implementation.html) [dep_tool]:https://golang.github.io/dep/docs/installation.html [git_tool]:https://git-scm.com/downloads [go_tool]:https://golang.org/dl/ [operator_sdk]:https://github.com/operator-framework/operator-sdk [fork_guide]:https://help.github.com/articles/fork-a-repo/ [docker_tool]:https://docs.docker.com/install/ [kubectl_tool]:https://kubernetes.io/docs/tasks/tools/install-kubectl/ [minikube]:https://kubernetes.io/docs/tasks/tools/install-minikube/ [virtualbox]:https://www.virtualbox.org/wiki/Downloads [install_dev_tools]:https://jenkinsci.github.io/kubernetes-operator/docs/developer-guide/tools/