--- title: "Configuring Seed Jobs and Pipelines" linkTitle: "Configuring Seed Jobs and Pipelines" weight: 4 date: 2021-01-25 description: > How to configure Jenkins with Operator --- ## Configure Seed Jobs and Pipelines Jenkins operator uses [job-dsl][job-dsl] and [kubernetes-credentials-provider][kubernetes-credentials-provider] plugins for configuring jobs and deploy keys. ## Prepare job definitions and pipelines First you have to prepare pipelines and job definition in your GitHub repository using the following structure: ``` cicd/ ├── jobs │   └── k8s.jenkins └── pipelines └── k8s.jenkins ``` **`cicd/jobs/k8s.jenkins`** is a job definition: ``` #!/usr/bin/env groovy pipelineJob('k8s-e2e') { displayName('Kubernetes Plugin E2E Test') logRotator { numToKeep(10) daysToKeep(30) } configure { project -> project / 'properties' / 'org.jenkinsci.plugins.workflow.job.properties.DurabilityHintJobProperty' { hint('PERFORMANCE_OPTIMIZED') } } definition { cpsScm { scm { git { remote { url('https://github.com/jenkinsci/kubernetes-operator.git') credentials('jenkins-operator') } branches('*/master') } } scriptPath('cicd/pipelines/k8s.jenkins') } } } ``` **`cicd/pipelines/k8s.jenkins`** is an actual Jenkins pipeline: ``` #!/usr/bin/env groovy def label = "k8s-${UUID.randomUUID().toString()}" def home = "/home/jenkins" def workspace = "${home}/workspace/build-jenkins-operator" def workdir = "${workspace}/src/github.com/jenkinsci/kubernetes-operator/" podTemplate(label: label, containers: [ containerTemplate(name: 'alpine', image: 'alpine:3.11', ttyEnabled: true, command: 'cat'), ], ) { node(label) { stage('Run shell') { container('alpine') { sh 'echo "hello world"' } } } } ``` ## Configure Seed Jobs Jenkins Seed Jobs are configured using `Jenkins.spec.seedJobs` section from your custom resource manifest: ``` apiVersion: jenkins.io/v1alpha2 kind: Jenkins metadata: name: example spec: seedJobs: - id: jenkins-operator targets: "cicd/jobs/*.jenkins" description: "Jenkins Operator repository" repositoryBranch: master repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git ``` **Jenkins Operator** will automatically discover and configure all the seed jobs. You can verify if deploy keys were successfully configured in the Jenkins **Credentials** tab. ![jenkins](/kubernetes-operator/img/jenkins-credentials.png) You can verify if your pipelines were successfully configured in the Jenkins Seed Job console output. ![jenkins](/kubernetes-operator/img/jenkins-seed.png) If your GitHub repository is **private** you have to configure SSH or username/password authentication. ### SSH authentication #### Generate SSH Keys There are two methods of SSH private key generation: ```bash $ openssl genrsa -out 2048 ``` or ```bash $ ssh-keygen -t rsa -b 2048 $ ssh-keygen -p -f -m pem ``` Then copy content from generated file. #### Public key If you want to upload your public key to your Git server you need to extract it. If key was generated by `openssl` then you need to type this to extract public key: ```bash $ openssl rsa -in -pubout > .pub ``` If key was generated by `ssh-keygen` the public key content is located in .pub and there is no need to extract public key #### Configure SSH authentication Configure a seed job like this: ``` apiVersion: jenkins.io/v1alpha2 kind: Jenkins metadata: name: example spec: seedJobs: - id: jenkins-operator-ssh credentialType: basicSSHUserPrivateKey credentialID: k8s-ssh targets: "cicd/jobs/*.jenkins" description: "Jenkins Operator repository" repositoryBranch: master repositoryUrl: git@github.com:jenkinsci/kubernetes-operator.git ``` and create a Kubernetes Secret (name of secret should be the same from `credentialID` field): ``` apiVersion: v1 kind: Secret metadata: name: k8s-ssh labels: "jenkins.io/credentials-type": "basicSSHUserPrivateKey" annotations: "jenkins.io/credentials-description" : "ssh github.com:jenkinsci/kubernetes-operator" stringData: privateKey: | -----BEGIN RSA PRIVATE KEY----- MIIJKAIBAAKCAgEAxxDpleJjMCN5nusfW/AtBAZhx8UVVlhhhIKXvQ+dFODQIdzO oDXybs1zVHWOj31zqbbJnsfsVZ9Uf3p9k6xpJ3WFY9b85WasqTDN1xmSd6swD4N8 ... username: github_user_name ``` ### Username & password authentication Configure the seed job like: ``` apiVersion: jenkins.io/v1alpha2 kind: Jenkins metadata: name: example spec: seedJobs: - id: jenkins-operator-user-pass credentialType: usernamePassword credentialID: k8s-user-pass targets: "cicd/jobs/*.jenkins" description: "Jenkins Operator repository" repositoryBranch: master repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git ``` and create a Kubernetes Secret (name of secret should be the same from `credentialID` field): ``` apiVersion: v1 kind: Secret metadata: name: k8s-user-pass stringData: username: github_user_name password: password_or_token ``` ### External authentication You can use `external` credential type if you want to configure authentication using Configuration As Code or Groovy Script. Example: ```yaml apiVersion: jenkins.io/v1alpha2 kind: Jenkins metadata: name: example spec: seedJobs: - id: jenkins-operator-external credentialType: external credentialID: k8s-external targets: "cicd/jobs/*.jenkins" description: "Jenkins Operator repository" repositoryBranch: master repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git ``` Remember that `credentialID` must match the id of the credentials configured in Jenkins. Consult the [Jenkins docs for using credentials][jenkins-using-credentials] for details. ## HTTP Proxy for downloading plugins To use forwarding proxy with an operator to download plugins you need to add the following environment variable to Jenkins Custom Resource (CR), e.g.: ```yaml spec: master: containers: - name: jenkins-master env: - name: CURL_OPTIONS value: -L -x ``` In `CURL_OPTIONS` var you can set additional arguments to `curl` command. ## Pulling Docker images from private repositories To pull a Docker Image from private repository you can use `imagePullSecrets`. Please follow the instructions on [creating a secret with a docker config](https://kubernetes.io/docs/concepts/containers/images/?origin_team=T42NTAGHM#creating-a-secret-with-a-docker-config). ### Docker Hub Configuration To use Docker Hub additional steps are required. Edit the previously created secret: ```bash kubectl -n edit secret ``` The `.dockerconfigjson` key's value needs to be replaced with a modified version. After modifications, it needs to be encoded as a Base64 value before setting the `.dockerconfigjson` key. Example config file to modify and use: ``` { "auths":{ "https://index.docker.io/v1/":{ "username":"user", "password":"password", "email":"yourdockeremail@gmail.com", "auth":"base64 of string user:password" }, "auth.docker.io":{ "username":"user", "password":"password", "email":"yourdockeremail@gmail.com", "auth":"base64 of string user:password" }, "registry.docker.io":{ "username":"user", "password":"password", "email":"yourdockeremail@gmail.com", "auth":"base64 of string user:password" }, "docker.io":{ "username":"user", "password":"password", "email":"yourdockeremail@gmail.com", "auth":"base64 of string user:password" }, "https://registry-1.docker.io/v2/": { "username":"user", "password":"password", "email":"yourdockeremail@gmail.com", "auth":"base64 of string user:password" }, "registry-1.docker.io/v2/": { "username":"user", "password":"password", "email":"yourdockeremail@gmail.com", "auth":"base64 of string user:password" }, "registry-1.docker.io": { "username":"user", "password":"password", "email":"yourdockeremail@gmail.com", "auth":"base64 of string user:password" }, "https://registry-1.docker.io": { "username":"user", "password":"password", "email":"yourdockeremail@gmail.com", "auth":"base64 of string user:password" } } } ``` [job-dsl]:https://github.com/jenkinsci/job-dsl-plugin [kubernetes-credentials-provider]:https://jenkinsci.github.io/kubernetes-credentials-provider-plugin/ [jenkins-using-credentials]:https://www.jenkins.io/doc/book/using/using-credentials/