The [Kubernetes Credentials Provider][0] plugin for Jenkins allows
Jenkins to expose Kubernetes Secret resources as Jenkins Credentials.
Jobs can use them like normal Jenkins credentials, e.g. using
`withCredentials`, `sshagent`, etc. The only drawback is that every
credential exposed this way is available to every job, at least until
[PR #40][1] is merged. Fortunately, jobs managed by this Jenkins
instance are all trusted; no anonymous pull requests are possible, so
the risk is mitigated.
[0]: https://jenkinsci.github.io/kubernetes-credentials-provider-plugin/
[1]: https://github.com/jenkinsci/kubernetes-credentials-provider-plugin/pull/40
Setting the `imagePullSecrets` property on the default service account
for the *jenkins-jobs* namespace allows jobs to run from private
container images automatically, without additional configuration in the
pipeline definitions.
The Raspberry Pi usually has the most free RAM of all the Kubernetes
nodes, so pods tend to get assigned there even when it would not be
appropriate. Jenkins, for example definitely does not need to run
there, so let's force it to run on the bigger nodes.
I don't want Jenkins updating itself whenever the pod restarts, so I'm
going to pin it to a specific version. This way, I can be sure to take
a snapshot of the data volume before upgrading.
Setting a static SELinux level for the container allows CRI-O to skip
relabeling all the files in the persistent volume each time the
container starts. For this to work, the pod needs a special annotation,
and CRI-O itself has to be configured to respect it:
```toml
[crio.runtime.runtimes.runc]
allowed_annotations = ["io.kubernetes.cri-o.TrySkipVolumeSELinuxLabel"]
```
This *dramatically* improves the start time of the Jenkins container.
Instead of taking 5+ minutes, it now starts instantly.
https://github.com/cri-o/cri-o/issues/6185#issuecomment-1334719982
Running Jenkins in Kubernetes is relatively straightforward. The
Kubernetes plugin automatically discovers all the connection and
authentication configuration, so a `kubeconfig` file is no longer
necessary. I did set the *Jenkins tunnel* option, though, so that
agents will connect directly to the Jenkins JNLP port instead of going
through the ingress controller.
Jobs now run in pods in the *jenkins-job* namespace instead of the
*jenkins* namespace. The latter is now where the Jenkins controller
runs, and the controller should not have permission to modify its own
resources.
Jenkins doesn't really need full control of all resources in its
namespace. Rather, it only needs to be able to manage Pod and
PersistentVolumeClaim resources.
Dustin