kubernetes-operator/_index.md at 9106582a9e23bb839b670df40636ce71ef09f18f

13 KiB

Raw Blame History

title	linkTitle	weight	date	description
Developer Guide	Developer Guide	60	2021-07-30	Jenkins Operator for developers

{{% pageinfo %}} This document explains how to setup your development environment. {{% /pageinfo %}}

Prerequisites

operator_sdk version 1.3.0
git
go version 1.15.6
goimports, golint, checkmake and staticcheck
minikube version 1.21.0 (preferred Hypervisor - virtualbox) (automatically downloaded)
docker version 17.03+

Clone repository and download dependencies

git clone git@github.com:jenkinsci/kubernetes-operator.git
cd kubernetes-operator
make go-dependencies

Build and run with a minikube

Start minikube instance configured for Jenkins Operator. Appropriate minikube version will be downloaded to bin folder.

make minikube-start

Next run Jenkins Operator locally.

make run

Console output indicating readiness of this phase:

+ build
+ run
kubectl config use-context minikube
Switched to context "minikube".
Watching 'default' namespace
bin/manager --jenkins-api-hostname=192.168.99.252 --jenkins-api-port=0 --jenkins-api-use-nodeport=true --cluster-domain=cluster.local 
2021-02-08T14:14:45.263+0100    INFO    cmd     Version: v0.5.0
2021-02-08T14:14:45.263+0100    INFO    cmd     Git commit: 305dbeda-dirty-dirty
2021-02-08T14:14:45.264+0100    INFO    cmd     Go Version: go1.15.6
2021-02-08T14:14:45.264+0100    INFO    cmd     Go OS/Arch: darwin/amd64
2021-02-08T14:14:45.264+0100    INFO    cmd     Watch namespace: default
2021-02-08T14:14:45.592+0100    INFO    controller-runtime.metrics      metrics server is starting to listen    {"addr": "0.0.0.0:8383"}
2021-02-08T14:14:45.599+0100    INFO    cmd     starting manager
2021-02-08T14:14:45.599+0100    INFO    controller-runtime.manager      starting metrics server {"path": "/metrics"}
2021-02-08T14:14:45.599+0100    INFO    controller-runtime.manager.controller.jenkins   Starting EventSource    {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "source": "kind source: jenkins.io/v1alpha2, Kind=Jenkins"}
2021-02-08T14:14:45.700+0100    INFO    controller-runtime.manager.controller.jenkins   Starting EventSource    {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "source": "kind source: /, Kind="}
2021-02-08T14:14:45.800+0100    INFO    controller-runtime.manager.controller.jenkins   Starting EventSource    {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "source": "kind source: /, Kind="}
2021-02-08T14:14:45.901+0100    INFO    controller-runtime.manager.controller.jenkins   Starting EventSource    {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "source": "kind source: /, Kind="}
2021-02-08T14:14:46.003+0100    INFO    controller-runtime.manager.controller.jenkins   Starting EventSource    {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "source": "kind source: core/v1, Kind=Secret"}
2021-02-08T14:14:46.004+0100    INFO    controller-runtime.manager.controller.jenkins   Starting EventSource    {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "source": "kind source: core/v1, Kind=ConfigMap"}
2021-02-08T14:14:46.004+0100    INFO    controller-runtime.manager.controller.jenkins   Starting EventSource    {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "source": "kind source: jenkins.io/v1alpha2, Kind=Jenkins"}
2021-02-08T14:14:46.004+0100    INFO    controller-runtime.manager.controller.jenkins   Starting Controller     {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins"}
2021-02-08T14:14:46.004+0100    INFO    controller-runtime.manager.controller.jenkins   Starting workers        {"reconciler group": "jenkins.io", "reconciler kind": "Jenkins", "worker count": 1}

Lastly apply Jenkins Custom Resource to minikube cluster:

kubectl apply -f config/samples/jenkins.io_v1alpha2_jenkins.yaml

{"level":"info","ts":1612790690.875426,"logger":"controller-jenkins","msg":"Setting default Jenkins container command","cr":"jenkins-example"}
{"level":"info","ts":1612790690.8754492,"logger":"controller-jenkins","msg":"Setting default Jenkins container JAVA_OPTS environment variable","cr":"jenkins-example"}
{"level":"info","ts":1612790690.875456,"logger":"controller-jenkins","msg":"Setting default operator plugins","cr":"jenkins-example"}
{"level":"info","ts":1612790690.875463,"logger":"controller-jenkins","msg":"Setting default Jenkins master service","cr":"jenkins-example"}
{"level":"info","ts":1612790690.875467,"logger":"controller-jenkins","msg":"Setting default Jenkins slave service","cr":"jenkins-example"}
{"level":"info","ts":1612790690.881811,"logger":"controller-jenkins","msg":"*v1alpha2.Jenkins/jenkins-example has been updated","cr":"jenkins-example"}
{"level":"info","ts":1612790691.252834,"logger":"controller-jenkins","msg":"Creating a new Jenkins Master Pod default/jenkins-jenkins-example","cr":"jenkins-example"}
{"level":"info","ts":1612790691.322793,"logger":"controller-jenkins","msg":"Jenkins master pod restarted by operator:","cr":"jenkins-example"}
{"level":"info","ts":1612790691.322817,"logger":"controller-jenkins","msg":"Jenkins Operator version has changed, actual '' new 'v0.5.0'","cr":"jenkins-example"}
{"level":"info","ts":1612790691.3228202,"logger":"controller-jenkins","msg":"Jenkins CR has been replaced","cr":"jenkins-example"}
{"level":"info","ts":1612790695.8789551,"logger":"controller-jenkins","msg":"Creating a new Jenkins Master Pod default/jenkins-jenkins-example","cr":"jenkins-example"}
{"level":"warn","ts":1612790817.9423082,"logger":"controller-jenkins","msg":"Reconcile loop failed: couldn't init Jenkins API client: Get \"http://192.168.99.254:31998/api/json\": dial tcp 192.168.99.254:31998: connect: connection refused","cr":"jenkins-example"}
{"level":"warn","ts":1612790817.9998221,"logger":"controller-jenkins","msg":"Reconcile loop failed: couldn't init Jenkins API client: Get \"http://192.168.99.254:31998/api/json\": dial tcp 192.168.99.254:31998: connect: connection refused","cr":"jenkins-example"}
{"level":"info","ts":1612790818.581316,"logger":"controller-jenkins","msg":"base-groovy ConfigMap 'jenkins-operator-base-configuration-jenkins-example' name '1-basic-settings.groovy' running groovy script","cr":"jenkins-example"}
...
{"level":"info","ts":1612790820.9473379,"logger":"controller-jenkins","msg":"base-groovy ConfigMap 'jenkins-operator-base-configuration-jenkins-example' name '8-disable-job-dsl-script-approval.groovy' running groovy script","cr":"jenkins-example"}
{"level":"info","ts":1612790821.244055,"logger":"controller-jenkins","msg":"Base configuration phase is complete, took 2m6s","cr":"jenkins-example"}
{"level":"info","ts":1612790821.7953842,"logger":"controller-jenkins","msg":"Waiting for Seed Job Agent `seed-job-agent`...","cr":"jenkins-example"}
...

{"level":"info","ts":1612790851.843638,"logger":"controller-jenkins","msg":"Waiting for Seed Job Agent `seed-job-agent`...","cr":"jenkins-example"}
{"level":"info","ts":1612790853.489524,"logger":"controller-jenkins","msg":"User configuration phase is complete, took 2m38s","cr":"jenkins-example"}

Two log lines says that Jenkins Operator works correctly:
 
* `Base configuration phase is complete` - ensures manifests, Jenkins pod, Jenkins configuration and Jenkins API token  
* `User configuration phase is complete` - ensures Jenkins restore, backup and seed jobs along with user configuration 

> Details about base and user phase can be found [here](https://jenkinsci.github.io/kubernetes-operator/docs/how-it-works/architecture-and-design/).

kubectl get jenkins -o yaml

apiVersion: v1
items:
- apiVersion: jenkins.io/v1alpha2
  kind: Jenkins
  metadata:
  ...
  spec:
    backup:
      action: {}
      containerName: ""
      interval: 0
      makeBackupBeforePodDeletion: false
    configurationAsCode:
      configurations: []
      secret:
        name: ""
    groovyScripts:
      configurations: []
      secret:
        name: ""
    jenkinsAPISettings:
      authorizationStrategy: createUser
    master:
      basePlugins:
      ...
      containers:
      - command:
        - bash
        - -c
        - /var/jenkins/scripts/init.sh && exec /sbin/tini -s -- /usr/local/bin/jenkins.sh
        env:
        - name: JAVA_OPTS
          value: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap
            -XX:MaxRAMFraction=1 -Djenkins.install.runSetupWizard=false -Djava.awt.headless=true
        image: jenkins/jenkins:2.263.3-lts-alpine
        imagePullPolicy: Always
        livenessProbe:
        ...
        readinessProbe:
        ...
        resources:
          limits:
            cpu: 1500m
            memory: 3Gi
          requests:
            cpu: "1"
            memory: 500Mi
      disableCSRFProtection: false
    restore:
      action: {}
      containerName: ""
      getLatestAction: {}
    seedJobs:
    - additionalClasspath: ""
      bitbucketPushTrigger: false
      buildPeriodically: ""
      description: Jenkins Operator repository
      failOnMissingPlugin: false
      githubPushTrigger: false
      id: jenkins-operator
      ignoreMissingFiles: false
      pollSCM: ""
      repositoryBranch: master
      repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git
      targets: cicd/jobs/*.jenkins
      unstableOnDeprecation: false
    service:
      port: 8080
      type: NodePort
    serviceAccount: {}
    slaveService:
      port: 50000
      type: ClusterIP
  status:
    appliedGroovyScripts:
    - configurationType: base-groovy
      hash: 2ownqpRyBjQYmzTRttUx7axok3CKe2E45frI5iRwH0w=
      name: 1-basic-settings.groovy
      source: jenkins-operator-base-configuration-jenkins-example
    ...
    baseConfigurationCompletedTime: "2021-02-08T13:27:01Z"
    createdSeedJobs:
    - jenkins-operator
    operatorVersion: v0.5.0
    provisionStartTime: "2021-02-08T13:24:55Z"
    userAndPasswordHash: nnfZsWmFfAYlYyVYeKhWW2KB4L8mE61JUfetAsr9IMM=
    userConfigurationCompletedTime: "2021-02-08T13:27:33Z"
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

kubectl get po

NAME                                              READY   STATUS              RESTARTS   AGE
jenkins-jenkins-example                           1/1     Running             0          23m
seed-job-agent-jenkins-example-758cc7cc5c-82hbl   1/1     Running             0          21m

Debug Jenkins Operator

make run OPERATOR_EXTRA_ARGS="--debug"

Stop or delete minikube cluster

To stop Kubernetes cluster running locally on minikube:

minikube stop

To delete the cluster altogether:

minikube delete

Build and run with a remote Kubernetes cluster

You can also run the controller locally and make it listen to a remote Kubernetes server.

make run NAMESPACE=default KUBECTL_CONTEXT=remote-k8s EXTRA_ARGS='--kubeconfig ~/.kube/config'

Once Jenkins Operator are up and running, apply Jenkins custom resource:

kubectl --context remote-k8s --namespace default apply -f deploy/crds/jenkins_v1alpha2_jenkins_cr.yaml
kubectl --context remote-k8s --namespace default get jenkins -o yaml
kubectl --context remote-k8s --namespace default get po

Testing

Tests are written using Ginkgo with Gomega.

Run unit tests with go fmt, lint, statickcheck, vet:

make verify

Run unit tests only:

make test

Running E2E tests

Run e2e tests with minikube:

make minikube-start
make e2e

Run the specific e2e test:

make e2e E2E_TEST_SELECTOR='^TestConfiguration$'

Building docker image on minikube

To be able to work with the docker daemon on minikube machine run the following command before building an image:

eval $(bin/minikube docker-env)

When `api/v1alpha2/jenkins_types.go` has changed

Run:

make manifests

Getting the Jenkins URL and basic credentials

minikube service jenkins-operator-http-<cr_name> --url
kubectl get secret jenkins-operator-credentials-<cr_name> -o 'jsonpath={.data.user}' | base64 -d
kubectl get secret jenkins-operator-credentials-<cr_name> -o 'jsonpath={.data.password}' | base64 -d

13 KiB Raw Blame History