mirror of
https://github.com/jenkinsci/kubernetes-operator.git
synced 2026-10-17 21:35:38 +02:00
Move docs
This commit is contained in:
Tomasz Sęk
@@ -1,116 +0,0 @@
|
||||
# Developer guide
|
||||
|
||||
This document explains how to setup your development environment.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- [operator_sdk][operator_sdk] version v0.8.1
|
||||
- [git][git_tool]
|
||||
- [go][go_tool] version v1.12+
|
||||
- [goimports, golint, checkmake and staticcheck][install_dev_tools]
|
||||
- [minikube][minikube] version v1.1.0+ (preferred Hypervisor - [virtualbox][virtualbox])
|
||||
- [docker][docker_tool] version 17.03+
|
||||
|
||||
## Clone repository and download dependencies
|
||||
|
||||
```bash
|
||||
mkdir -p $GOPATH/src/github.com/jenkinsci
|
||||
cd $GOPATH/src/github.com/jenkinsci/
|
||||
git clone git@github.com:jenkinsci/kubernetes-operator.git
|
||||
cd kubernetes-operator
|
||||
make go-dependencies
|
||||
```
|
||||
|
||||
## Build and run with a minikube
|
||||
|
||||
Build and run **jenkins-operator** locally:
|
||||
|
||||
```bash
|
||||
make minikube-run EXTRA_ARGS='--minikube --local'
|
||||
```
|
||||
|
||||
Once minikube and **jenkins-operator** are up and running, apply Jenkins custom resource:
|
||||
|
||||
```bash
|
||||
kubectl apply -f deploy/crds/jenkins_v1alpha2_jenkins_cr.yaml
|
||||
kubectl get jenkins -o yaml
|
||||
kubectl get po
|
||||
```
|
||||
|
||||
## Build and run with a remote Kubernetes cluster
|
||||
|
||||
You can also run the controller locally and make it listen to a remote Kubernetes server.
|
||||
|
||||
```bash
|
||||
make run NAMESPACE=default KUBECTL_CONTEXT=remote-k8s EXTRA_ARGS='--kubeconfig ~/.kube/config'
|
||||
```
|
||||
|
||||
Once minikube and **jenkins-operator** are up and running, apply Jenkins custom resource:
|
||||
|
||||
```bash
|
||||
kubectl --context remote-k8s --namespace default apply -f deploy/crds/jenkins_v1alpha2_jenkins_cr.yaml
|
||||
kubectl --context remote-k8s --namespace default get jenkins -o yaml
|
||||
kubectl --context remote-k8s --namespace default get po
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
Run unit tests:
|
||||
|
||||
```bash
|
||||
make test
|
||||
```
|
||||
|
||||
Run e2e tests with minikube:
|
||||
|
||||
```bash
|
||||
make start-minikube
|
||||
eval $(minikube docker-env)
|
||||
make e2e
|
||||
```
|
||||
|
||||
Run the specific e2e test:
|
||||
|
||||
```bash
|
||||
make e2e E2E_TEST_SELECTOR='^TestConfiguration$'
|
||||
```
|
||||
|
||||
## Tips & Tricks
|
||||
|
||||
### Building docker image on minikube (for e2e tests)
|
||||
|
||||
To be able to work with the docker daemon on `minikube` machine run the following command before building an image:
|
||||
|
||||
```bash
|
||||
eval $(minikube docker-env)
|
||||
```
|
||||
|
||||
### When `pkg/apis/jenkinsio/*/jenkins_types.go` has changed
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
make deepcopy-gen
|
||||
```
|
||||
|
||||
### Getting Jenkins URL and basic credentials
|
||||
|
||||
```bash
|
||||
minikube service jenkins-operator-http-<cr_name> --url
|
||||
kubectl get secret jenkins-operator-credentials-<cr_name> -o 'jsonpath={.data.user}' | base64 -d
|
||||
kubectl get secret jenkins-operator-credentials-<cr_name> -o 'jsonpath={.data.password}' | base64 -d
|
||||
```
|
||||
|
||||
|
||||
[dep_tool]:https://golang.github.io/dep/docs/installation.html
|
||||
[git_tool]:https://git-scm.com/downloads
|
||||
[go_tool]:https://golang.org/dl/
|
||||
[operator_sdk]:https://github.com/operator-framework/operator-sdk
|
||||
[fork_guide]:https://help.github.com/articles/fork-a-repo/
|
||||
[docker_tool]:https://docs.docker.com/install/
|
||||
[kubectl_tool]:https://kubernetes.io/docs/tasks/tools/install-kubectl/
|
||||
[minikube]:https://kubernetes.io/docs/tasks/tools/install-minikube/
|
||||
[virtualbox]:https://www.virtualbox.org/wiki/Downloads
|
||||
[jenkins-operator]:../README.md
|
||||
[install_dev_tools]:./install_dev_tools.md
|
||||
|
||||
@@ -1,699 +0,0 @@
|
||||
# Getting Started
|
||||
|
||||
This document describes a getting started guide for **jenkins-operator** and an additional configuration.
|
||||
|
||||
1. [First Steps](#first-steps)
|
||||
2. [Deploy Jenkins](#deploy-jenkins)
|
||||
3. [Configure Seed Jobs and Pipelines](#configure-seed-jobs-and-pipelines)
|
||||
4. [Pulling Docker images from private repositories](#pulling-docker-images-from-private-repositories)
|
||||
5. [Install Plugins](#install-plugins)
|
||||
6. [Configure Backup & Restore](#configure-backup-and-restore)
|
||||
7. [AKS](#aks)
|
||||
8. [Jenkins login credentials](#jenkins-login-credentials)
|
||||
9. [Override default Jenkins container command](#override-default-Jenkins-container-command)
|
||||
10. [Debugging](#debugging)
|
||||
|
||||
## First Steps
|
||||
|
||||
Prepare your Kubernetes cluster and set up access.
|
||||
Once you have running Kubernetes cluster you can focus on installing **jenkins-operator** according to the [Installation](installation.md) guide.
|
||||
|
||||
## Deploy Jenkins
|
||||
|
||||
Once jenkins-operator is up and running let's deploy actual Jenkins instance.
|
||||
Create manifest ie. **jenkins_instance.yaml** with following data and save it on drive.
|
||||
|
||||
```bash
|
||||
apiVersion: jenkins.io/v1alpha2
|
||||
kind: Jenkins
|
||||
metadata:
|
||||
name: example
|
||||
spec:
|
||||
master:
|
||||
containers:
|
||||
- name: jenkins-master
|
||||
image: jenkins/jenkins:lts
|
||||
imagePullPolicy: Always
|
||||
livenessProbe:
|
||||
failureThreshold: 12
|
||||
httpGet:
|
||||
path: /login
|
||||
port: http
|
||||
scheme: HTTP
|
||||
initialDelaySeconds: 80
|
||||
periodSeconds: 10
|
||||
successThreshold: 1
|
||||
timeoutSeconds: 5
|
||||
readinessProbe:
|
||||
failureThreshold: 3
|
||||
httpGet:
|
||||
path: /login
|
||||
port: http
|
||||
scheme: HTTP
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
successThreshold: 1
|
||||
timeoutSeconds: 1
|
||||
resources:
|
||||
limits:
|
||||
cpu: 1500m
|
||||
memory: 3Gi
|
||||
requests:
|
||||
cpu: "1"
|
||||
memory: 500Mi
|
||||
seedJobs:
|
||||
- id: jenkins-operator
|
||||
targets: "cicd/jobs/*.jenkins"
|
||||
description: "Jenkins Operator repository"
|
||||
repositoryBranch: master
|
||||
repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git
|
||||
```
|
||||
|
||||
Deploy Jenkins to K8s:
|
||||
|
||||
```bash
|
||||
kubectl create -f jenkins_instance.yaml
|
||||
```
|
||||
Watch Jenkins instance being created:
|
||||
|
||||
```bash
|
||||
kubectl get pods -w
|
||||
```
|
||||
|
||||
Get Jenkins credentials:
|
||||
|
||||
```bash
|
||||
kubectl get secret jenkins-operator-credentials-<cr_name> -o 'jsonpath={.data.user}' | base64 -d
|
||||
kubectl get secret jenkins-operator-credentials-<cr_name> -o 'jsonpath={.data.password}' | base64 -d
|
||||
```
|
||||
|
||||
Connect to Jenkins (minikube):
|
||||
|
||||
```bash
|
||||
minikube service jenkins-operator-http-<cr_name> --url
|
||||
```
|
||||
|
||||
Connect to Jenkins (actual Kubernetes cluster):
|
||||
|
||||
```bash
|
||||
kubectl port-forward jenkins-<cr_name> 8080:8080
|
||||
```
|
||||
Then open browser with address `http://localhost:8080`.
|
||||
|
||||
|
||||
## Configure Seed Jobs and Pipelines
|
||||
|
||||
Jenkins operator uses [job-dsl][job-dsl] and [kubernetes-credentials-provider][kubernetes-credentials-provider] plugins for configuring jobs
|
||||
and deploy keys.
|
||||
|
||||
## Prepare job definitions and pipelines
|
||||
|
||||
First you have to prepare pipelines and job definition in your GitHub repository using the following structure:
|
||||
|
||||
```
|
||||
cicd/
|
||||
├── jobs
|
||||
│ └── build.jenkins
|
||||
└── pipelines
|
||||
└── build.jenkins
|
||||
```
|
||||
|
||||
**cicd/jobs/build.jenkins** it's a job definition:
|
||||
|
||||
```
|
||||
#!/usr/bin/env groovy
|
||||
|
||||
pipelineJob('build-jenkins-operator') {
|
||||
displayName('Build jenkins-operator')
|
||||
|
||||
definition {
|
||||
cpsScm {
|
||||
scm {
|
||||
git {
|
||||
remote {
|
||||
url('https://github.com/jenkinsci/kubernetes-operator.git')
|
||||
credentials('jenkins-operator')
|
||||
}
|
||||
branches('*/master')
|
||||
}
|
||||
}
|
||||
scriptPath('cicd/pipelines/build.jenkins')
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**cicd/jobs/build.jenkins** it's an actual Jenkins pipeline:
|
||||
|
||||
```
|
||||
#!/usr/bin/env groovy
|
||||
|
||||
def label = "build-jenkins-operator-${UUID.randomUUID().toString()}"
|
||||
def home = "/home/jenkins"
|
||||
def workspace = "${home}/workspace/build-jenkins-operator"
|
||||
def workdir = "${workspace}/src/github.com/jenkinsci/kubernetes-operator/"
|
||||
|
||||
podTemplate(label: label,
|
||||
containers: [
|
||||
containerTemplate(name: 'jnlp', image: 'jenkins/jnlp-slave:alpine'),
|
||||
containerTemplate(name: 'go', image: 'golang:1-alpine', command: 'cat', ttyEnabled: true),
|
||||
],
|
||||
envVars: [
|
||||
envVar(key: 'GOPATH', value: workspace),
|
||||
],
|
||||
) {
|
||||
|
||||
node(label) {
|
||||
dir(workdir) {
|
||||
stage('Init') {
|
||||
timeout(time: 3, unit: 'MINUTES') {
|
||||
checkout scm
|
||||
}
|
||||
container('go') {
|
||||
sh 'apk --no-cache --update add make git gcc libc-dev'
|
||||
}
|
||||
}
|
||||
|
||||
stage('Dep') {
|
||||
container('go') {
|
||||
sh 'make dep'
|
||||
}
|
||||
}
|
||||
|
||||
stage('Test') {
|
||||
container('go') {
|
||||
sh 'make test'
|
||||
}
|
||||
}
|
||||
|
||||
stage('Build') {
|
||||
container('go') {
|
||||
sh 'make build'
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Configure Seed Jobs
|
||||
|
||||
Jenkins Seed Jobs are configured using `Jenkins.spec.seedJobs` section from your custom resource manifest:
|
||||
|
||||
```
|
||||
apiVersion: jenkins.io/v1alpha2
|
||||
kind: Jenkins
|
||||
metadata:
|
||||
name: example
|
||||
spec:
|
||||
seedJobs:
|
||||
- id: jenkins-operator
|
||||
targets: "cicd/jobs/*.jenkins"
|
||||
description: "Jenkins Operator repository"
|
||||
repositoryBranch: master
|
||||
repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git
|
||||
```
|
||||
|
||||
**jenkins-operator** will automatically discover and configure all seed jobs.
|
||||
|
||||
You can verify if deploy keys were successfully configured in Jenkins **Credentials** tab.
|
||||
|
||||
|
||||
|
||||
You can verify if your pipelines were successfully configured in Jenkins Seed Job console output.
|
||||
|
||||
|
||||
|
||||
If your GitHub repository is **private** you have to configure SSH or username/password authentication.
|
||||
|
||||
### SSH authentication
|
||||
|
||||
Configure seed job like:
|
||||
|
||||
```
|
||||
apiVersion: jenkins.io/v1alpha2
|
||||
kind: Jenkins
|
||||
metadata:
|
||||
name: example
|
||||
spec:
|
||||
seedJobs:
|
||||
- id: jenkins-operator-ssh
|
||||
credentialType: basicSSHUserPrivateKey
|
||||
credentialID: k8s-ssh
|
||||
targets: "cicd/jobs/*.jenkins"
|
||||
description: "Jenkins Operator repository"
|
||||
repositoryBranch: master
|
||||
repositoryUrl: git@github.com:jenkinsci/kubernetes-operator.git
|
||||
```
|
||||
|
||||
and create Kubernetes Secret(name of secret should be the same from `credentialID` field):
|
||||
|
||||
```
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: k8s-ssh
|
||||
data:
|
||||
privateKey: |
|
||||
-----BEGIN RSA PRIVATE KEY-----
|
||||
MIIJKAIBAAKCAgEAxxDpleJjMCN5nusfW/AtBAZhx8UVVlhhhIKXvQ+dFODQIdzO
|
||||
oDXybs1zVHWOj31zqbbJnsfsVZ9Uf3p9k6xpJ3WFY9b85WasqTDN1xmSd6swD4N8
|
||||
...
|
||||
username: github_user_name
|
||||
```
|
||||
|
||||
### Username & password authentication
|
||||
|
||||
Configure seed job like:
|
||||
|
||||
```
|
||||
apiVersion: jenkins.io/v1alpha2
|
||||
kind: Jenkins
|
||||
metadata:
|
||||
name: example
|
||||
spec:
|
||||
seedJobs:
|
||||
- id: jenkins-operator-user-pass
|
||||
credentialType: usernamePassword
|
||||
credentialID: k8s-user-pass
|
||||
targets: "cicd/jobs/*.jenkins"
|
||||
description: "Jenkins Operator repository"
|
||||
repositoryBranch: master
|
||||
repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git
|
||||
```
|
||||
|
||||
and create Kubernetes Secret(name of secret should be the same from `credentialID` field):
|
||||
|
||||
```
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: k8s-user-pass
|
||||
data:
|
||||
username: github_user_name
|
||||
password: password_or_token
|
||||
```
|
||||
|
||||
## Pulling Docker images from private repositories
|
||||
To pull Docker Image from private repository you can use `imagePullSecrets`.
|
||||
|
||||
Please follow the instructions on [creating a secret with a docker config](https://kubernetes.io/docs/concepts/containers/images/?origin_team=T42NTAGHM#creating-a-secret-with-a-docker-config).
|
||||
|
||||
### Docker Hub Configuration
|
||||
To use Docker Hub additional steps are required.
|
||||
|
||||
Edit the previously created secret:
|
||||
```bash
|
||||
kubectl -n <namespace> edit secret <name>
|
||||
```
|
||||
|
||||
The `.dockerconfigjson` key's value needs to be replaced with a modified version.
|
||||
|
||||
After modifications it needs to be encoded as Base64 value before setting the `.dockerconfigjson` key:q.
|
||||
|
||||
Example config file to modify and use:
|
||||
```
|
||||
{
|
||||
"auths":{
|
||||
"https://index.docker.io/v1/":{
|
||||
"username":"user",
|
||||
"password":"password",
|
||||
"email":"yourdockeremail@gmail.com",
|
||||
"auth":"base64 of string user:password"
|
||||
},
|
||||
"auth.docker.io":{
|
||||
"username":"user",
|
||||
"password":"password",
|
||||
"email":"yourdockeremail@gmail.com",
|
||||
"auth":"base64 of string user:password"
|
||||
},
|
||||
"registry.docker.io":{
|
||||
"username":"user",
|
||||
"password":"password",
|
||||
"email":"yourdockeremail@gmail.com",
|
||||
"auth":"base64 of string user:password"
|
||||
},
|
||||
"docker.io":{
|
||||
"username":"user",
|
||||
"password":"password",
|
||||
"email":"yourdockeremail@gmail.com",
|
||||
"auth":"base64 of string user:password"
|
||||
},
|
||||
"https://registry-1.docker.io/v2/": {
|
||||
"username":"user",
|
||||
"password":"password",
|
||||
"email":"yourdockeremail@gmail.com",
|
||||
"auth":"base64 of string user:password"
|
||||
},
|
||||
"registry-1.docker.io/v2/": {
|
||||
"username":"user",
|
||||
"password":"password",
|
||||
"email":"yourdockeremail@gmail.com",
|
||||
"auth":"base64 of string user:password"
|
||||
},
|
||||
"registry-1.docker.io": {
|
||||
"username":"user",
|
||||
"password":"password",
|
||||
"email":"yourdockeremail@gmail.com",
|
||||
"auth":"base64 of string user:password"
|
||||
},
|
||||
"https://registry-1.docker.io": {
|
||||
"username":"user",
|
||||
"password":"password",
|
||||
"email":"yourdockeremail@gmail.com",
|
||||
"auth":"base64 of string user:password"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Jenkins Customisation
|
||||
|
||||
Jenkins can be customized using groovy scripts or [configuration as code plugin](https://github.com/jenkinsci/configuration-as-code-plugin).
|
||||
By using [ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) you can create own **Jenkins** customized configuration.
|
||||
Then you must reference the *ConfigMap* in **Jenkins** pod customization file in `spec.groovyScripts` or `spec.configurationAsCode`
|
||||
|
||||
For example create *ConfigMap* with name `jenkins-operator-user-configuration`. Then, modify the **Jenkins** manifest to look like this:
|
||||
|
||||
```yaml
|
||||
apiVersion: jenkins.io/v1alpha2
|
||||
kind: Jenkins
|
||||
metadata:
|
||||
name: example
|
||||
spec:
|
||||
configurationAsCode:
|
||||
configurations:
|
||||
- name: jenkins-operator-user-configuration
|
||||
groovyScripts:
|
||||
configurations:
|
||||
- name: jenkins-operator-user-configuration
|
||||
```
|
||||
|
||||
Here is example of `jenkins-operator-user-configuration`:
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: jenkins-operator-user-configuration
|
||||
data:
|
||||
1-configure-theme.groovy: |
|
||||
import jenkins.*
|
||||
import jenkins.model.*
|
||||
import hudson.*
|
||||
import hudson.model.*
|
||||
import org.jenkinsci.plugins.simpletheme.ThemeElement
|
||||
import org.jenkinsci.plugins.simpletheme.CssTextThemeElement
|
||||
import org.jenkinsci.plugins.simpletheme.CssUrlThemeElement
|
||||
|
||||
Jenkins jenkins = Jenkins.getInstance()
|
||||
|
||||
def decorator = Jenkins.instance.getDescriptorByType(org.codefirst.SimpleThemeDecorator.class)
|
||||
|
||||
List<ThemeElement> configElements = new ArrayList<>();
|
||||
configElements.add(new CssTextThemeElement("DEFAULT"));
|
||||
configElements.add(new CssUrlThemeElement("https://cdn.rawgit.com/afonsof/jenkins-material-theme/gh-pages/dist/material-light-green.css"));
|
||||
decorator.setElements(configElements);
|
||||
decorator.save();
|
||||
|
||||
jenkins.save()
|
||||
1-system-message.yaml: |
|
||||
jenkins:
|
||||
systemMessage: "Configuration as Code integration works!!!"
|
||||
```
|
||||
|
||||
* *.groovy is Groovy script configuration
|
||||
* *.yaml is configuration as code
|
||||
|
||||
If you want to correct your configuration you can edit it while **jenkins-operator** is running.
|
||||
Jenkins will reconcile and apply new configuration.
|
||||
|
||||
### Using secrets inside Groovy script
|
||||
|
||||
If you configured `spec.groovyScripts.secret.name`, then this secret is available to use inside map Groovy scripts.
|
||||
The secrets are loaded to `secrets` map.
|
||||
|
||||
Create a [secret](https://kubernetes.io/docs/concepts/configuration/secret/) with for eg. `jenkins-conf-secrets` name.
|
||||
|
||||
```yaml
|
||||
kind: Secret
|
||||
apiVersion: v1
|
||||
type: Opaque
|
||||
metadata:
|
||||
name: jenkins-conf-secrets
|
||||
namespace: default
|
||||
data:
|
||||
SYSTEM_MESSAGE: SGVsbG8gd29ybGQ=
|
||||
```
|
||||
|
||||
Then modify the **Jenkins** pod manifest by changing `spec.groovyScripts.secret.name` to `jenkins-conf-secrets`.
|
||||
|
||||
```yaml
|
||||
apiVersion: jenkins.io/v1alpha2
|
||||
kind: Jenkins
|
||||
metadata:
|
||||
name: example
|
||||
spec:
|
||||
configurationAsCode:
|
||||
configurations:
|
||||
- name: jenkins-operator-user-configuration
|
||||
secret:
|
||||
name: jenkins-conf-secrets
|
||||
groovyScripts:
|
||||
configurations:
|
||||
- name: jenkins-operator-user-configuration
|
||||
secret:
|
||||
name: jenkins-conf-secrets
|
||||
```
|
||||
|
||||
Now you can test that the secret is mounted by applying this ConfigMap for Groovy script:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: jenkins-operator-user-configuration
|
||||
data:
|
||||
1-system-message.groovy: |
|
||||
import jenkins.*
|
||||
import jenkins.model.*
|
||||
import hudson.*
|
||||
import hudson.model.*
|
||||
Jenkins jenkins = Jenkins.getInstance()
|
||||
|
||||
jenkins.setSystemMessage(secrets["SYSTEM_MESSAGE"])
|
||||
jenkins.save()
|
||||
```
|
||||
|
||||
Or by applying configuration as code:
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: jenkins-operator-user-configuration
|
||||
data:
|
||||
1-system-message.yaml: |
|
||||
jenkins:
|
||||
systemMessage: ${SYSTEM_MESSAGE}
|
||||
```
|
||||
|
||||
|
||||
After this, you should see the `Hello world` system message at **Jenkins** homepage.
|
||||
|
||||
## Install Plugins
|
||||
|
||||
Edit CR under `spec.master.plugins`:
|
||||
|
||||
```
|
||||
apiVersion: jenkins.io/v1alpha2
|
||||
kind: Jenkins
|
||||
metadata:
|
||||
name: example
|
||||
spec:
|
||||
master:
|
||||
plugins:
|
||||
- name: simple-theme-plugin
|
||||
version: 0.5.1
|
||||
```
|
||||
|
||||
Then **jenkins-operator** will automatically install plugins after Jenkins master pod restart.
|
||||
|
||||
## Configure backup and restore
|
||||
|
||||
Backup and restore is done by container sidecar.
|
||||
|
||||
### PVC
|
||||
|
||||
#### Create PVC
|
||||
|
||||
Save to file pvc.yaml:
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: <pvc_name>
|
||||
namespace: <namesapce>
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 500Gi
|
||||
```
|
||||
|
||||
Run command:
|
||||
```bash
|
||||
$ kubectl -n <namesapce> create -f pvc.yaml
|
||||
```
|
||||
|
||||
#### Configure Jenkins CR
|
||||
|
||||
```yaml
|
||||
apiVersion: jenkins.io/v1alpha2
|
||||
kind: Jenkins
|
||||
metadata:
|
||||
name: <cr_name>
|
||||
namespace: <namespace>
|
||||
spec:
|
||||
master:
|
||||
securityContext:
|
||||
runAsUser: 1000
|
||||
fsGroup: 1000
|
||||
containers:
|
||||
- name: jenkins-master
|
||||
image: jenkins/jenkins:lts
|
||||
- name: backup # container responsible for backup and restore
|
||||
env:
|
||||
- name: BACKUP_DIR
|
||||
value: /backup
|
||||
- name: JENKINS_HOME
|
||||
value: /jenkins-home
|
||||
- name: BACKUP_COUNT
|
||||
value: "2" # keep only the 2 most recent backups
|
||||
image: virtuslab/jenkins-operator-backup-pvc:v0.0.5 # look at backup/pvc directory
|
||||
imagePullPolicy: IfNotPresent
|
||||
volumeMounts:
|
||||
- mountPath: /jenkins-home # Jenkins home volume
|
||||
name: jenkins-home
|
||||
- mountPath: /backup # backup volume
|
||||
name: backup
|
||||
volumes:
|
||||
- name: backup # PVC volume where backups will be stored
|
||||
persistentVolumeClaim:
|
||||
claimName: <pvc_name>
|
||||
backup:
|
||||
containerName: backup # container name is responsible for backup
|
||||
action:
|
||||
exec:
|
||||
command:
|
||||
- /home/user/bin/backup.sh # this command is invoked on "backup" container to make backup, for example /home/user/bin/backup.sh <backup_number>, <backup_number> is passed by operator
|
||||
interval: 30 # how often make backup in seconds
|
||||
makeBackupBeforePodDeletion: true # make backup before pod deletion
|
||||
restore:
|
||||
containerName: backup # container name is responsible for restore backup
|
||||
action:
|
||||
exec:
|
||||
command:
|
||||
- /home/user/bin/restore.sh # this command is invoked on "backup" container to make restore backup, for example /home/user/bin/restore.sh <backup_number>, <backup_number> is passed by operator
|
||||
#recoveryOnce: <backup_number> # if want to restore specific backup configure this field and then Jenkins will be restarted and desired backup will be restored
|
||||
```
|
||||
|
||||
## AKS
|
||||
|
||||
Azure AKS managed Kubernetes service adds to every pod the following envs:
|
||||
|
||||
```yaml
|
||||
- name: KUBERNETES_PORT_443_TCP_ADDR
|
||||
value:
|
||||
- name: KUBERNETES_PORT
|
||||
value: tcp://
|
||||
- name: KUBERNETES_PORT_443_TCP
|
||||
value: tcp://
|
||||
- name: KUBERNETES_SERVICE_HOST
|
||||
value:
|
||||
```
|
||||
|
||||
The operator is aware of it and omits these envs when checking if Jenkins pod envs have been changed. It prevents
|
||||
restart Jenkins pod over and over again.
|
||||
|
||||
## Jenkins login credentials
|
||||
|
||||
The operator automatically generate Jenkins user name and password and stores it in Kubernetes secret named
|
||||
`jenkins-operator-credentials-<cr_name>` in namespace where Jenkins CR has been deployed.
|
||||
|
||||
If you want change it you can override the secret:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: jenkins-operator-credentials-<cr-name>
|
||||
namespace: <namespace>
|
||||
data:
|
||||
user: <base64-encoded-new-username>
|
||||
password: <base64-encoded-new-password>
|
||||
```
|
||||
|
||||
If needed **jenkins-operator** will restart Jenkins master pod and then you can login with the new user and password
|
||||
credentials.
|
||||
|
||||
## Override default Jenkins container command
|
||||
|
||||
The default command for the Jenkins master container `jenkins/jenkins:lts` looks like:
|
||||
|
||||
```yaml
|
||||
command:
|
||||
- bash
|
||||
- -c
|
||||
- /var/jenkins/scripts/init.sh && /sbin/tini -s -- /usr/local/bin/jenkins.sh
|
||||
```
|
||||
|
||||
The script`/var/jenkins/scripts/init.sh` is provided be the operator and configures init.groovy.d(creates Jenkins user)
|
||||
and installs plugins.
|
||||
The `/sbin/tini -s -- /usr/local/bin/jenkins.sh` command runs the Jenkins master main process.
|
||||
|
||||
You can overwrite it in the following pattern:
|
||||
|
||||
```yaml
|
||||
command:
|
||||
- bash
|
||||
- -c
|
||||
- /var/jenkins/scripts/init.sh && <custom-code-here> && /sbin/tini -s -- /usr/local/bin/jenkins.sh
|
||||
```
|
||||
|
||||
## Debugging
|
||||
|
||||
Turn on debug in **jenkins-operator** deployment:
|
||||
|
||||
```bash
|
||||
sed -i 's|\(args:\).*|\1\ ["--debug"\]|' deploy/operator.yaml
|
||||
kubectl apply -f deploy/operator.yaml
|
||||
```
|
||||
|
||||
Watch Kubernetes events:
|
||||
|
||||
```bash
|
||||
kubectl get events --sort-by='{.lastTimestamp}'
|
||||
```
|
||||
|
||||
Verify Jenkins master logs:
|
||||
|
||||
```bash
|
||||
kubectl logs -f jenkins-<cr_name>
|
||||
```
|
||||
|
||||
Verify jenkins-operator logs:
|
||||
|
||||
```bash
|
||||
kubectl logs deployment/jenkins-operator
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
Delete Jenkins master pod and wait for the new one to come up:
|
||||
|
||||
```bash
|
||||
kubectl delete pod jenkins-<cr_name>
|
||||
```
|
||||
|
||||
[job-dsl]:https://github.com/jenkinsci/job-dsl-plugin
|
||||
[kubernetes-credentials-provider]:https://jenkinsci.github.io/kubernetes-credentials-provider-plugin/
|
||||
@@ -1,52 +0,0 @@
|
||||
# How it works
|
||||
|
||||
This document describes a high level overview how **jenkins-operator** works.
|
||||
|
||||
1. [Architecture and design](#architecture-and-design)
|
||||
2. [Operator State](#operator-state)
|
||||
3. [System Jenkins Jobs](#system-jenkins-jobs)
|
||||
3. [Jenkins Docker Images](#jenkins-docker-images)
|
||||
|
||||
## Architecture and design
|
||||
|
||||
The **jenkins-operator** design incorporates the following concepts:
|
||||
- watches any changes of manifests and maintain the desired state according to deployed custom resource manifest
|
||||
- implements the main reconciliation loop which consists of two smaller reconciliation loops - base and user
|
||||
|
||||
|
||||
|
||||
**Base** reconciliation loop takes care of reconciling base Jenkins configuration, which consists of:
|
||||
- Ensure Manifests - monitors any changes in manifests
|
||||
- Ensure Jenkins Pod - creates and verifies status of Jenkins master Pod
|
||||
- Ensure Jenkins Configuration - configures Jenkins instance including hardening, initial configuration for plugins, etc.
|
||||
- Ensure Jenkins API token - generates Jenkins API token and initialized Jenkins client
|
||||
|
||||
**User** reconciliation loop takes care of reconciling user provided configuration, which consists of:
|
||||
- Ensure Restore Job - creates Restore job and ensures that restore has been successfully performed
|
||||
- Ensure Seed Jobs - creates Seed Jobs and ensures that all of them have been successfully executed
|
||||
- Ensure User Configuration - executed user provided configuration, like groovy scripts, configuration as code or plugins
|
||||
- Ensure Backup Job - creates Backup job and ensures that backup has been successfully performed
|
||||
|
||||
|
||||
|
||||
## Operator State
|
||||
|
||||
Operator state is kept in custom resource status section, which is used for storing any configuration events or job statuses managed by the operator.
|
||||
It helps to maintain or recover desired state even after operator or Jenkins restarts.
|
||||
|
||||
## System Jenkins Jobs
|
||||
|
||||
The operator or Jenkins instance can be restarted at any time and any operation should not block the reconciliation loop.
|
||||
Taking this into account we implemented custom jobs API for executing system jobs (seed jobs, groovy scripts, etc.) according to the operator lifecycle.
|
||||
|
||||
Main assumptions are:
|
||||
- do not block reconciliation loop
|
||||
- fire job, requeue reconciliation loop and verify job status next time
|
||||
- handle retries if case of failure
|
||||
- handle build expiration (deadline)
|
||||
- keep state in the custom resource status section
|
||||
|
||||
## Jenkins Docker Images
|
||||
|
||||
**jenkins-operator** is fully compatible with **jenkins:lts** docker image and does not introduce any hidden changes there.
|
||||
If needed, the docker image can easily be changed in custom resource manifest as long as it supports standard Jenkins file system structure.
|
||||
@@ -1,50 +0,0 @@
|
||||
# Installation of the required development tools
|
||||
|
||||
This document explains how to install the Go tools used by the development process.
|
||||
|
||||
## Configure environment variables
|
||||
|
||||
```bash
|
||||
export GOPATH=/home/go # example value
|
||||
export GOROOT=/usr/lib/go-1.12 # example value
|
||||
export PATH=$GOPATH/bin:$PATH
|
||||
```
|
||||
|
||||
## goimports
|
||||
|
||||
```
|
||||
go get golang.org/x/tools/cmd/goimports
|
||||
cd $GOPATH/src/golang.org/x/tools/cmd/goimports
|
||||
go build
|
||||
go install
|
||||
```
|
||||
|
||||
## golint
|
||||
|
||||
```
|
||||
go get -u golang.org/x/lint/golint
|
||||
cd $GOPATH/src/golang.org/x/lint/golint
|
||||
go build
|
||||
go install
|
||||
```
|
||||
|
||||
## checkmake
|
||||
```
|
||||
go get github.com/mrtazz/checkmake
|
||||
cd $GOPATH/src/github.com/mrtazz/checkmake
|
||||
go build
|
||||
go install
|
||||
```
|
||||
|
||||
## staticcheck
|
||||
|
||||
```
|
||||
mkdir -p $GOPATH/src/github.com/dominikh/
|
||||
cd $GOPATH/src/github.com/dominikh/
|
||||
git clone https://github.com/dominikh/go-tools.git
|
||||
cd $GOPATH/src/github.com/dominikh/go-tools/staticcheck
|
||||
go build
|
||||
go install
|
||||
```
|
||||
|
||||
|
||||
@@ -1,37 +0,0 @@
|
||||
# Installation
|
||||
|
||||
This document describes installation procedure for **jenkins-operator**.
|
||||
All container images can be found at [virtuslab/jenkins-operator](https://hub.docker.com/r/virtuslab/jenkins-operator)
|
||||
|
||||
## Requirements
|
||||
|
||||
To run **jenkins-operator**, you will need:
|
||||
- running Kubernetes cluster version 1.11+
|
||||
- kubectl version 1.11+
|
||||
|
||||
## Configure Custom Resource Definition
|
||||
|
||||
Install Jenkins Custom Resource Definition:
|
||||
|
||||
```bash
|
||||
kubectl apply -f https://raw.githubusercontent.com/jenkinsci/kubernetes-operator/master/deploy/crds/jenkins_v1alpha2_jenkins_crd.yaml
|
||||
```
|
||||
|
||||
## Deploy jenkins-operator
|
||||
|
||||
Apply Service Account and RBAC roles:
|
||||
|
||||
```bash
|
||||
kubectl apply -f https://raw.githubusercontent.com/jenkinsci/kubernetes-operator/master/deploy/all-in-one-v1alpha2.yaml
|
||||
```
|
||||
|
||||
Watch **jenkins-operator** instance being created:
|
||||
|
||||
```bash
|
||||
kubectl get pods -w
|
||||
```
|
||||
|
||||
Now **jenkins-operator** should be up and running in `default` namespace.
|
||||
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,331 +0,0 @@
|
||||
# Migration guide from v1alpha1 to v1alpha2
|
||||
|
||||
Please not that **CRD manifests are global**, not namespaced, so every jenkins operator running on the cluster
|
||||
will be impacted by the new CRD manifest. Multiple operator instances with different versions *should* continue to work.
|
||||
|
||||
## Stop jenkins-operator pod
|
||||
|
||||
Run command:
|
||||
```bash
|
||||
$ kubectl -n <namespace> scale deployment.apps/jenkins-operator --replicas=0
|
||||
deployment.apps/jenkins-operator scaled
|
||||
```
|
||||
|
||||
Desired state:
|
||||
```bash
|
||||
$ kubectl -n <namespace> get po
|
||||
No resources found.
|
||||
```
|
||||
|
||||
## Stop Jenkins master pod
|
||||
|
||||
Run command:
|
||||
```bash
|
||||
$ kubectl -n <namespace> get po
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
jenkins-operator-<cr_name> 2/2 Running 0 3m35s
|
||||
$ kubectl -n <namespace> get delete po jenkins-operator-<cr_name>
|
||||
pod "jenkins-operator-<cr_name>" deleted
|
||||
```
|
||||
|
||||
Desired state:
|
||||
```bash
|
||||
$ kubectl -n <namespace> get po
|
||||
No resources found.
|
||||
```
|
||||
|
||||
## Save Jenkins CR to jenkins.yaml file
|
||||
|
||||
Run command:
|
||||
```bash
|
||||
$ kubectl -n <namespace> get jenkins <cr_name> -o yaml > jenkins.yaml
|
||||
```
|
||||
|
||||
## Modify jenkins.yaml file
|
||||
|
||||
Change apiVersion to `apiVersion: jenkins.io/v1alpha2`
|
||||
|
||||
New plugin format without dependent plugins:
|
||||
|
||||
- `spec.master.basePlugins` example:
|
||||
```
|
||||
spec:
|
||||
master:
|
||||
basePlugins:
|
||||
- name: a-plugin-name
|
||||
version: "1.0.0"
|
||||
...
|
||||
```
|
||||
- `spec.master.plugins` example:
|
||||
```
|
||||
|
||||
spec:
|
||||
master:
|
||||
plugins:
|
||||
- name: a-plugin-name
|
||||
version: "1.0.0"
|
||||
...
|
||||
```
|
||||
|
||||
Move Jenkins `master` container properties to `spec.master.containers[jenkins-master]` (non exhaustive list):
|
||||
- `spec.master.image` -> `spec.master.containers[jenkins-master].image`
|
||||
- `spec.master.imagePullPolicy` -> `spec.master.containers[jenkins-master].imagePullPolicy`
|
||||
- `spec.master.livenessProbe` -> `spec.master.containers[jenkins-master].livenessProbe`
|
||||
- `spec.master.readinessProbe` -> `spec.master.containers[jenkins-master].readinessProbe`
|
||||
- `spec.master.resources` -> `spec.master.containers[jenkins-master].resources`
|
||||
- `spec.master.env` -> `spec.master.containers[jenkins-master].env`
|
||||
|
||||
```
|
||||
spec:
|
||||
master:
|
||||
containers:
|
||||
- name: jenkins-master
|
||||
image: jenkins/jenkins:lts
|
||||
...
|
||||
```
|
||||
|
||||
See also the examples bellow for more details. For even more details please look at the source code.
|
||||
Code that defines the data structures can be found [here](jenkins-v1alpha2-scheme.md)
|
||||
|
||||
### Examples
|
||||
|
||||
Old format:
|
||||
```yaml
|
||||
apiVersion: jenkins.io/v1alpha1
|
||||
kind: Jenkins
|
||||
metadata:
|
||||
name: <cr_name>
|
||||
namespace: <namespace>
|
||||
spec:
|
||||
master:
|
||||
basePlugins:
|
||||
configuration-as-code:1.17:
|
||||
- configuration-as-code-support:1.17
|
||||
git:3.10.0:
|
||||
- apache-httpcomponents-client-4-api:4.5.5-3.0
|
||||
- credentials:2.1.19
|
||||
- display-url-api:2.3.1
|
||||
- git-client:2.7.7
|
||||
- jsch:0.1.55
|
||||
- junit:1.28
|
||||
- mailer:1.23
|
||||
- matrix-project:1.14
|
||||
- scm-api:2.4.1
|
||||
- script-security:1.59
|
||||
- ssh-credentials:1.16
|
||||
- structs:1.19
|
||||
- workflow-api:2.34
|
||||
- workflow-scm-step:2.7
|
||||
- workflow-step-api:2.19
|
||||
job-dsl:1.74:
|
||||
- script-security:1.59
|
||||
- structs:1.19
|
||||
kubernetes-credentials-provider:0.12.1:
|
||||
- credentials:2.1.19
|
||||
- structs:1.19
|
||||
- variant:1.2
|
||||
kubernetes:1.15.5:
|
||||
- apache-httpcomponents-client-4-api:4.5.5-3.0
|
||||
- cloudbees-folder:6.8
|
||||
- credentials:2.1.19
|
||||
- durable-task:1.29
|
||||
- jackson2-api:2.9.9
|
||||
- kubernetes-credentials:0.4.0
|
||||
- plain-credentials:1.5
|
||||
- structs:1.19
|
||||
- variant:1.2
|
||||
- workflow-step-api:2.19
|
||||
workflow-aggregator:2.6:
|
||||
- ace-editor:1.1
|
||||
- apache-httpcomponents-client-4-api:4.5.5-3.0
|
||||
- authentication-tokens:1.3
|
||||
- branch-api:2.5.2
|
||||
- cloudbees-folder:6.8
|
||||
- credentials-binding:1.18
|
||||
- credentials:2.1.19
|
||||
- display-url-api:2.3.1
|
||||
- docker-commons:1.15
|
||||
- docker-workflow:1.18
|
||||
- durable-task:1.29
|
||||
- git-client:2.7.7
|
||||
- git-server:1.7
|
||||
- handlebars:1.1.1
|
||||
- jackson2-api:2.9.9
|
||||
- jquery-detached:1.2.1
|
||||
- jsch:0.1.55
|
||||
- junit:1.28
|
||||
- lockable-resources:2.5
|
||||
- mailer:1.23
|
||||
- matrix-project:1.14
|
||||
- momentjs:1.1.1
|
||||
- pipeline-build-step:2.9
|
||||
- pipeline-graph-analysis:1.10
|
||||
- pipeline-input-step:2.10
|
||||
- pipeline-milestone-step:1.3.1
|
||||
- pipeline-model-api:1.3.8
|
||||
- pipeline-model-declarative-agent:1.1.1
|
||||
- pipeline-model-definition:1.3.8
|
||||
- pipeline-model-extensions:1.3.8
|
||||
- pipeline-rest-api:2.11
|
||||
- pipeline-stage-step:2.3
|
||||
- pipeline-stage-tags-metadata:1.3.8
|
||||
- pipeline-stage-view:2.11
|
||||
- plain-credentials:1.5
|
||||
- scm-api:2.4.1
|
||||
- script-security:1.59
|
||||
- ssh-credentials:1.16
|
||||
- structs:1.19
|
||||
- workflow-api:2.34
|
||||
- workflow-basic-steps:2.16
|
||||
- workflow-cps-global-lib:2.13
|
||||
- workflow-cps:2.69
|
||||
- workflow-durable-task-step:2.30
|
||||
- workflow-job:2.32
|
||||
- workflow-multibranch:2.21
|
||||
- workflow-scm-step:2.7
|
||||
- workflow-step-api:2.19
|
||||
- workflow-support:3.3
|
||||
workflow-job:2.32:
|
||||
- scm-api:2.4.1
|
||||
- script-security:1.59
|
||||
- structs:1.19
|
||||
- workflow-api:2.34
|
||||
- workflow-step-api:2.19
|
||||
- workflow-support:3.3
|
||||
image: jenkins/jenkins:lts
|
||||
imagePullPolicy: Always
|
||||
livenessProbe:
|
||||
failureThreshold: 12
|
||||
httpGet:
|
||||
path: /login
|
||||
port: 8080
|
||||
scheme: HTTP
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
successThreshold: 1
|
||||
timeoutSeconds: 5
|
||||
plugins:
|
||||
simple-theme-plugin:0.5.1: []
|
||||
slack:2.24:
|
||||
- workflow-step-api:2.19
|
||||
- credentials:2.1.19
|
||||
- display-url-api:2.3.1
|
||||
- junit:1.28
|
||||
- plain-credentials:1.5
|
||||
- script-security:1.59
|
||||
- structs:1.19
|
||||
- token-macro:2.8
|
||||
readinessProbe:
|
||||
failureThreshold: 12
|
||||
httpGet:
|
||||
path: /login
|
||||
port: 8080
|
||||
scheme: HTTP
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
successThreshold: 1
|
||||
timeoutSeconds: 5
|
||||
resources:
|
||||
limits:
|
||||
cpu: 1500m
|
||||
memory: 3Gi
|
||||
requests:
|
||||
cpu: "1"
|
||||
memory: 500Mi
|
||||
```
|
||||
|
||||
New format:
|
||||
```yaml
|
||||
apiVersion: jenkins.io/v1alpha2
|
||||
kind: Jenkins
|
||||
metadata:
|
||||
name: <cr_name>
|
||||
namespace: <namespace>
|
||||
spec:
|
||||
master:
|
||||
basePlugins:
|
||||
- name: kubernetes
|
||||
version: 1.15.7
|
||||
- name: workflow-job
|
||||
version: "2.32"
|
||||
- name: workflow-aggregator
|
||||
version: "2.6"
|
||||
- name: git
|
||||
version: 3.10.0
|
||||
- name: job-dsl
|
||||
version: "1.74"
|
||||
- name: configuration-as-code
|
||||
version: "1.19"
|
||||
- name: configuration-as-code-support
|
||||
version: "1.19"
|
||||
- name: kubernetes-credentials-provider
|
||||
version: 0.12.1
|
||||
containers:
|
||||
- name: jenkins-master
|
||||
image: jenkins/jenkins:lts
|
||||
imagePullPolicy: Always
|
||||
livenessProbe:
|
||||
failureThreshold: 12
|
||||
httpGet:
|
||||
path: /login
|
||||
port: http
|
||||
scheme: HTTP
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
successThreshold: 1
|
||||
timeoutSeconds: 5
|
||||
readinessProbe:
|
||||
failureThreshold: 3
|
||||
httpGet:
|
||||
path: /login
|
||||
port: http
|
||||
scheme: HTTP
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
successThreshold: 1
|
||||
timeoutSeconds: 1
|
||||
resources:
|
||||
limits:
|
||||
cpu: 1500m
|
||||
memory: 3Gi
|
||||
requests:
|
||||
cpu: "1"
|
||||
memory: 500Mi
|
||||
plugins:
|
||||
- name: simple-theme-plugin
|
||||
version: 0.5.1
|
||||
- name: slack
|
||||
version: 2.24
|
||||
```
|
||||
|
||||
## Update CRD to new version
|
||||
|
||||
New version of the Custom Resource definition for the operator needs to be applied:
|
||||
-[Jenkins CRD v1alpha2](https://github.com/jenkinsci/kubernetes-operator/blob/master/deploy/crds/jenkins_v1alpha2_jenkins_crd.yaml)
|
||||
|
||||
To use default CRD file:
|
||||
```
|
||||
kubectl -n <namespace> apply -f https://github.com/jenkinsci/kubernetes-operator/blob/master/deploy/crds/jenkins_v1alpha2_jenkins_crd.yaml
|
||||
```
|
||||
|
||||
## Update RBAC to new version
|
||||
|
||||
New operator version requires updated RBAC permissions:
|
||||
|
||||
To use default Role file:
|
||||
```
|
||||
$ kubectl -n <namespace> apply -f https://raw.githubusercontent.com/jenkinsci/kubernetes-operator/master/deploy/role.yaml
|
||||
```
|
||||
|
||||
## Deploy new operator manifests
|
||||
|
||||
Replace your modified operator configuration file:
|
||||
```bash
|
||||
$ kubectl -n <namespace> replace -f jenkins.yaml
|
||||
```
|
||||
|
||||
Update operator version in the deployment file to `image: virtuslab/jenkins-operator:v0.1.0` and scale up,
|
||||
or use the default deployment manifest:
|
||||
```
|
||||
$ kubectl -n <namespace> apply -f https://raw.githubusercontent.com/jenkinsci/kubernetes-operator/master/deploy/operator.yaml
|
||||
```
|
||||
@@ -1,94 +0,0 @@
|
||||
# Jenkins Security
|
||||
|
||||
By default **jenkins-operator** performs an initial security hardening of Jenkins instance via groovy scripts to prevent any security gaps.
|
||||
|
||||
## Jenkins Access Control
|
||||
|
||||
Currently **jenkins-operator** generates a username and random password and stores them in a Kubernetes Secret.
|
||||
However any other authorization mechanisms are possible and can be done via groovy scripts or configuration as code plugin.
|
||||
For more information take a look at [getting-started#jenkins-customization](getting-started.md#jenkins-customisation).
|
||||
|
||||
Any change to Security Realm or Authorization requires that user called `jenkins-operator` must have admin rights
|
||||
because **jenkins-operator** calls Jenkins API.
|
||||
|
||||
## Jenkins Hardening
|
||||
|
||||
The list below describes all the default security setting configured by the **jenkins-operator**:
|
||||
- basic settings - use `Mode.EXCLUSIVE` - Jobs must specify that they want to run on master node
|
||||
- enable CSRF - Cross Site Request Forgery Protection is enabled
|
||||
- disable usage stats - Jenkins usage stats submitting is disabled
|
||||
- enable master access control - Slave To Master Access Control is enabled
|
||||
- disable old JNLP protocols - `JNLP3-connect`, `JNLP2-connect` and `JNLP-connect` are disabled
|
||||
- disable CLI - CLI access of `/cli` URL is disabled
|
||||
- configure kubernetes-plugin - secure configuration for Kubernetes plugin
|
||||
|
||||
If you would like to dig a little bit into the code, take a look [here](../pkg/controller/jenkins/configuration/base/resources/base_configuration_configmap.go).
|
||||
|
||||
## Jenkins API
|
||||
|
||||
The **jenkins-operator** generates and configures Basic Authentication token for Jenkins go client and stores it in a Kubernetes Secret.
|
||||
|
||||
## Kubernetes
|
||||
|
||||
Kubernetes API permissions are limited by the following roles:
|
||||
- [jenkins-operator role](../deploy/role.yaml)
|
||||
- [Jenkins Master role](../pkg/controller/jenkins/configuration/base/resources/rbac.go)
|
||||
|
||||
Since **jenkins-operator** must be able to grant permission for its' deployed Jenkins masters to spawn pods (the `Jenkins Master role` above),
|
||||
the operator itself requires permission to create RBAC resources (the `jenkins-operator role` above).
|
||||
Deployed this way, any subject which may create a Pod (including a Jenkins job) may
|
||||
assume the `jenkins-operator` role by using its' ServiceAccount, create RBAC rules, and thus escape its granted permissions.
|
||||
Any namespace to which the `jenkins-operator` is deployed must be considered to implicitly grant all
|
||||
possible permissions to any subject which can create a Pod in that namespace.
|
||||
|
||||
To mitigate this issue **jenkins-operator** should be deployed in one namespace and the Jenkins CR should be created in separate namespace.
|
||||
To achieve it change watch namespace in https://github.com/jenkinsci/kubernetes-operator/blob/master/deploy/operator.yaml#L25
|
||||
|
||||
## Setup Jenkins Operator and Jenkins in separated namespaces
|
||||
|
||||
You need to create two namespaces, for example we'll call them **jenkins** for Jenkins and **jenkins-operator** for Jenkins Operator.
|
||||
```bash
|
||||
$ kubectl create ns jenkins-operator
|
||||
$ kubectl create ns jenkins
|
||||
```
|
||||
|
||||
Next, apply the RBAC manifests for **jenkins-operator** namespace
|
||||
```bash
|
||||
$ kubectl -n jenkins-operator apply -f deploy/service_account.yaml
|
||||
$ kubectl -n jenkins-operator apply -f deploy/role_binding.yaml
|
||||
```
|
||||
|
||||
Create file role_binding_jenkins.yaml in `deploy` folder:
|
||||
```yaml
|
||||
kind: RoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
metadata:
|
||||
name: jenkins-operator
|
||||
namespace: jenkins
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: jenkins-operator
|
||||
namespace: jenkins-operator
|
||||
roleRef:
|
||||
kind: Role
|
||||
name: jenkins-operator
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
```
|
||||
|
||||
Then, apply RBAC rules for **jenkins** namespace
|
||||
```bash
|
||||
$ kubectl -n jenkins apply -f deploy/role.yaml
|
||||
$ kubectl -n jenkins apply -f role_binding_jenkins.yaml
|
||||
```
|
||||
|
||||
Finally, you must create operator pod by:
|
||||
```bash
|
||||
$ kubectl -n jenkins -n jenkins-operator apply -f deploy/operator.yaml
|
||||
```
|
||||
|
||||
|
||||
## Report a Security Vulnerability
|
||||
|
||||
If you find a vulnerability or any misconfiguration in Jenkins, please report it in the [issues](https://github.com/jenkinsci/kubernetes-operator/issues).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user