Jenkins install on kubernetes in 3 simple steps

Introduction

Jenkins is the most popular open source automation server. This tool helps developers and devops engineers to automate phases relating to building, testing, and deploying, facilitating continuous integration and continuous delivery.

There are many ways to install Jenkins server depending on the use case, platform and the environment requirements.

Containerized install on Docker
Binary install on Windows, Mac and Linux
Install jenkins on public clouds
Install jenkins on kubernetes based environment

Jenkins can be installed on Kubernetes in multiple ways :

Install via yaml resources and Jenkins containerized image
Install via Helm chart and manage parameterised configs
Install via Operators

We would focus on the recommended and stable way of installing Jenkins on kubernetes platform via Helm chart.

Jenkins – helm chart

Helm being the package manager for Kubernetes, provides an elegant way to manage the deployment resources and configurations. It abstracts the install dependencies via “helm templates” and provides a single place for referring all the configs in “values.yaml”. Helm Charts provide “push button” deployment and deletion of apps, making adoption and development of Kubernetes apps easier for those with little container or microservices experience.

Jenkins helm chart’s are available from various repos.

$ helm search repo jenkins
NAME                       CHART VERSION APP VERSION DESCRIPTION
bitnami/jenkins           7.3.7         2.263.4     The leading open source automation server
incubator/jenkins-operator 0.3.1         0.3.0       DEPRECATED: A Helm chart for Kubernetes Jenkins...
jenkinsci/jenkins         3.2.4         2.277.1     Jenkins - Build great things at any scale! The ...
stable/jenkins             2.5.4         lts         DEPRECATED - Open source continuous integration...

I would recommend to follow the helm chart developed and maintained by the jenkins.io (i.e : jenkinsci/jenkins“)

Initial Setup

In order to install Jenkins on Kubernetes platform, Jenkins needs to be available as a containerized app. There are certain sub-components in Jenkins which need to be configured. The most important sub-component is “plugins“. Almost every capability of Jenkins server is managed by plugins. Thus, there are certain must have plugins, to be installed during the initial Jenkins setup.

Step1: Download the helm chart from the repo

$ helm fetch jenkinsci/jenkins
 
$ ls -l | grep jenkins
-rw-r--r--    1 user  user      67195 Apr 23 10:10 jenkins-3.2.4.tgz
 
$ tar -xzf jenkins-3.2.4.tgz
 
$ ls -l | grep jenkins
drwxr-xr-x   11 user  user        352 Apr 23 10:18 jenkins
-rw-r--r--    1 user  user      67195 Apr 23 10:10 jenkins-3.2.4.tgz
 
$ cd jenkins && ls -l
-rw-r--r--   1 user  user  41275 Mar 15 18:12 CHANGELOG.md
-rw-r--r--   1 user  user   1008 Mar 15 18:12 Chart.yaml
-rw-r--r--   1 user  user  29194 Mar 15 18:12 README.md
-rw-r--r--   1 user  user    205 Mar 15 18:12 Tiltfile
-rw-r--r--   1 user  user  36102 Mar 15 18:12 VALUES_SUMMARY.md
drwxr-xr-x  27 user  user    864 Apr 23 10:18 templates
drwxr-xr-x  20 user  user    640 Apr 23 10:18 tests
-rw-r--r--   1 user  user  32625 Mar 15 18:12 values.yamlshell

Step2: Prepare the configurations (values.yaml)

There are many configs related to Jenkins and kubernetes in the values.yaml, including below major ones :

Jenkins worker image
numExecutors: 0 (default values)
adminUser and it’s initial password
usePodSecurityContext: true
agentListenerPort: 50000
disabledAgentProtocols: [JNLP-connect, JNLP2-connect]
csrf – defaultCrumbIssuer – {enabled: true, proxyCompatability: true}
installPlugins – List of plugins to be install during Jenkins controller start
agentListenerServiceType: “ClusterIP”
ingress
- enabled
- hostName
- tls
persistence
networkPolicy
serviceAccount

By default, the application expose the service to “ClusterIP”. We can either change the serviceType to “NodePort” or LoadBalancer to expose the app on a NodeIP and higher range post or on a LB_IP on exposed application port. We can even create an ingress resource by configuring the values.yaml as mentioned below :

 
  ingress:
    enabled: true
    # Override for the default paths that map requests to the backend
    paths: []
    # - backend:
    #     serviceName: ssl-redirect
    #     servicePort: use-annotation
    # - backend:
    #     serviceName: >-
    #       {{ template "jenkins.fullname" . }}
    #     # Don't use string here, use only integer value!
    #     servicePort: 8080
    # For Kubernetes v1.14+, use 'networking.k8s.io/v1beta1'
    # For Kubernetes v1.19+, use 'networking.k8s.io/v1'
    apiVersion: "extensions/v1beta1"
    labels: {}
    annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
    # Set this path to jenkinsUriPrefix above or use annotations to rewrite path
    # path: "/jenkins"
    # configures the hostname e.g. jenkins.example.com
    hostName: jenkins.172.16.44.11.nip.io
    tls:
    - secretName: jenkins.tls
      hosts:
      - jenkins.172.16.44.11.nip.io

Step3: Prepare the Kubernetes environment for the installation

The best part for Helm installation that it’s platform agnostic, and works same on any kubernetes setup from any vendor or environment. So, we need to have a kubernetes setup and the cluster should be accessible from the host-VM being used to store helm charts and install Jenkins.

Step4: Prepare plugin installation for jenkins

Ideally, we can include the required and mandatory plugins in the values.yaml within the installPlugins section as below :

 
  # List of plugins to be install during Jenkins controller start
  installPlugins:
    - kubernetes:1.29.2
    - workflow-aggregator:2.6
    - git:4.6.0
    - configuration-as-code:1.47

But, this process, might have some issues depending on the environment setup. Actually, during init phase jenkins tries to download and install the plugins and may either hang or slow down the install process.

So, there is a better way, where we pre-build the Jenkins image with the plugins of our choice and use the image to install jenkins on kubernetes platform. It’s benefits are that it speeds up the install process and allows to maintain secure jenkins plugins setup separately.

To perform the alternative plugin setup, we follow below steps.

Assuming docker daemon is installed on Host-VM, we pull the jenkins image. (i.e: jenkins/jenkins:2.277.3-lts)
Prepare Dockerfile to use above image and pass instruction (script) to install plugins during docker build plase.

kerfile
 
FROM jenkins/jenkins:2.277.3-lts
COPY plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN jenkins-plugin-cli -f /usr/share/jenkins/ref/plugins.txt

As seen above, we are passing the list of the plugins via ” plugins.txt “. Thus, we need to prepare this file and put the list of all the mandatory and required plugins as shown in below example(Note: we can get the huge list of available plugins at plugins.jenkins.io

kubernetes:1.29.2
workflow-aggregator:2.6
git:4.6.0
configuration-as-code:1.47
blueocean:1.24.4
credentials:2.3.15
workflow-scm-step:2.12
junit:1.49
git-client:3.6.0
dependency-check-jenkins-plugin:5.1.1
pipeline-utility-steps:2.6.1
pipeline-stage-step:2.5
nexus-artifact-uploader:2.13
kubernetes-cli:1.10.0
sonar:2.13
performance:3.19
monitoring:1.86.0
email-ext:2.82
allure-jenkins-plugin:2.29.0
conditional-buildstep:1.4.1
parameterized-trigger:2.40

Then, we can perform docker build and create the custom-jenkins image, which can be pushed to an image registry.
In the values.yaml file, we can disable the plugin install step by updating as installPlugins: [] and. replacing the image:tag with the custom image create in the previous step.

Setup in devops environment

Step1: Jenkins install – kickoff

ell commands
 
$ kubectl create namespace jenkins
namespace/jenkins created
 
$ helm install jenkins ../jenkins -n jenkins
NAME: jenkins
LAST DEPLOYED: Fri Apr 23 12:51:06 2021
NAMESPACE: jenkins
STATUS: deployed
REVISION: 1
NOTES:
1. Get your 'admin' user password by running:
  kubectl exec --namespace jenkins -it svc/jenkins -c jenkins -- /bin/cat /run/secrets/chart-admin-password && echo
 
2. Visit http://jenkins.172.16.44.11.nip.io
 
3. Login with the password from step 1 and the username: admin
4. Configure security realm and authorization strategy
5. Use Jenkins Configuration as Code by specifying configScripts in your values.yaml file, see documentation: http://jenkins.172.16.44.11.nip.io/configuration-as-code and examples: https://github.com/jenkinsci/configuration-as-code-plugin/tree/master/demos
 
For more information on running Jenkins on Kubernetes, visit:
https://cloud.google.com/solutions/jenkins-on-container-engine
 
For more information about Jenkins Configuration as Code, visit:
https://jenkins.io/projects/jcasc/

Step2: Verify the installation in kubernetes

 
$ kubectl -n jenkins get pod -w
NAME        READY   STATUS     RESTARTS   AGE
jenkins-0   0/2     Init:0/1   0          8s
jenkins-0   0/2     PodInitializing   0          28s
jenkins-0   1/2     Running           0          93s
jenkins-0   2/2     Running           0          102s
 
$ kubectl get all,secret,cm,ing,pvc -n jenkins
NAME            READY   STATUS    RESTARTS   AGE
pod/jenkins-0   2/2     Running   0          3m15s
 
NAME                    TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)     AGE
service/jenkins         ClusterIP   10.105.111.5   <none>        8080/TCP    3m15s
service/jenkins-agent   ClusterIP   10.99.20.35    <none>        50000/TCP   3m15s
 
NAME                       READY   AGE
statefulset.apps/jenkins   1/1     3m15s
 
NAME                                   TYPE                                  DATA   AGE
secret/default-token-lv22d             kubernetes.io/service-account-token   3      83m
secret/jenkins                         Opaque                                2      3m15s
secret/jenkins-token-jgqnk             kubernetes.io/service-account-token   3      3m15s
secret/sh.helm.release.v1.jenkins.v1   helm.sh/release.v1                    1      3m15s
 
NAME                                     DATA   AGE
configmap/jenkins                        2      3m15s
configmap/jenkins-jenkins-jcasc-config   1      3m15s
configmap/kube-root-ca.crt               1      83m
 
NAME                                CLASS    HOSTS                         ADDRESS        PORTS     AGE
ingress.networking.k8s.io/jenkins   <none>   jenkins.172.16.44.11.nip.io   172.16.44.11   80, 443   3m15s
 
NAME                            STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/jenkins   Bound    pvc-45b0fbdd-1ce4-497c-94a5-3386e9fcf6f3   8Gi        RWO            standard       3m15s

Verification

Thus , we can confirm that the installation is successful. The application can be accessed with this app-hostname as jenkins.172.16.44.11.nip.io which is exposed via ingress resources.

At this point, we can create a kubernetes-secret using the Jenkins >> Manage Jenkins >> Manage Credentials which would be used during pipeline job execution.

We can also observe an error notification, which shows that the Jenkins application lacks privilege on the kubernetes cluster.

Failed to initialize Kubernetes secret providerio.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://10.96.0.1/api/v1/namespaces/jenkins/secrets?labelSelector=jenkins.io%2Fcredentials-type. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. secrets is forbidden: User "system:serviceaccount:jenkins:jenkins" cannot list resource "secrets" in API group "" in the namespace "jenkins".

In order to resolve, we can create a clusterrolebinding for Jenkins service account with a clusterrole.

 
$ k create clusterrolebinding jenkinsrolebinding --clusterrole=cluster-admin --serviceaccount=jenkins:jenkins
clusterrolebinding.rbac.authorization.k8s.io/jenkinsrolebinding created

Thus, the installation of Jenkins is completed and operational in Kubernetes cluster.