infra

Resources for deploying and managing my personal Kubernetes cluster

Go to file

Dustin 8b8ae3df04 setup: Use kickstart instead of Ansible Kubernetes, or rather mostly Calico, does not play well on a machine with an immutable root filesyste. Specifically, Calico needs write access to a couple of paths on the root filesystem, such as `/etc/cni/net.d`, `/opt/cni/bin`, and `/usr/libexec/kubernetes/kubelet-plugins/volume`. Some of those paths can be configured, but doing so is quite cumbersome. While these paths could be made writable, e.g. using symlinks or bind mounts, it would add a lot of complexity to the kubelet Ansible role. After considering the options for a while, I decided that the best approach was probably to mount specific filesystems at these paths. Instead of using small LVM logical volumes for each one, I thought it would be better to use a single btrfs filesystem for all the mutable storage locations. This way, if I discover more paths that need to be writable, I can create subvolumes for them, without having to try to move or resize the existing volumes. Now that the Kubernetes nodes need their own special kickstart file for the disk layout, it also makes sense to handle the rest of the machine setup there, too. This eliminates the need for the kubelet Ansible role altogether. Any machine provisioned with this kickstart configuration is immediately ready to become a Kubernetes control plane or worker node.	2022-07-26 22:29:50 -05:00
jenkins	Add Jenkins setup resources	2022-07-25 17:52:55 -05:00
setup	setup: Use kickstart instead of Ansible	2022-07-26 22:29:50 -05:00
README.md	Add Jenkins setup resources	2022-07-25 17:52:55 -05:00

Dustin 8b8ae3df04 setup: Use kickstart instead of Ansible

Kubernetes, or rather mostly Calico, does not play well on a machine
with an immutable root filesyste.  Specifically, Calico needs write
access to a couple of paths on the root filesystem, such as
`/etc/cni/net.d`, `/opt/cni/bin`, and
`/usr/libexec/kubernetes/kubelet-plugins/volume`.  Some of those paths
can be configured, but doing so is quite cumbersome.  While these paths
could be made writable, e.g. using symlinks or bind mounts, it would add
a lot of complexity to the *kubelet* Ansible role.  After considering
the options for a while, I decided that the best approach was probably
to mount specific filesystems at these paths.  Instead of using small
LVM logical volumes for each one, I thought it would be better to use a
single *btrfs* filesystem for all the mutable storage locations.  This
way, if I discover more paths that need to be writable, I can create
subvolumes for them, without having to try to move or resize the
existing volumes.

Now that the Kubernetes nodes need their own special kickstart file for
the disk layout, it also makes sense to handle the rest of the machine
setup there, too.  This eliminates the need for the *kubelet* Ansible
role altogether.  Any machine provisioned with this kickstart
configuration is immediately ready to become a Kubernetes control plane
or worker node.

2022-07-26 22:29:50 -05:00

jenkins

Add Jenkins setup resources

2022-07-25 17:52:55 -05:00

setup

setup: Use kickstart instead of Ansible

2022-07-26 22:29:50 -05:00

README.md

Add Jenkins setup resources

2022-07-25 17:52:55 -05:00

README.md

Dustin's Kubernetes Cluster

This repository contains resources for deploying and managing my on-premises Kubernetes cluster

Cluster Setup

The cluster primarily consists of libvirt/QEMU+KVM virtual machines. The Control Plane nodes are VMs, as are the x86_64 worker nodes. Eventually, I would like to add Raspberry Pi or Pine64 machines as aarch64 nodes.

All machines run Fedora, using only Fedora builds of the Kubernetes components (kubeadm, kubectl, and kubeadm).

See Cluster Setup for details.

Jenkins Agents

One of the main use cases for the Kubernetes cluster is to provide dynamic agents for Jenkins. Using the Kubernetes Plugin, Jenkins will automatically launch worker nodes as Kubernetes pods.

See Jenkins Kubernetes Integration for details.