Dustin C. Hatch
8f088fb6ae
Since *mtrcs0.pyrocufflink.blue* (the Metrics Pi) seems to be dying, I decided to move monitoring and alerting into Kubernetes. I was originally planning to have a single, dedicated virtual machine for Victoria Metrics and Grafana, similar to how the Metrics Pi was set up, but running Fedora CoreOS instead of a custom Buildroot-based OS. While I was working on the Ignition configuration for the VM, it occurred to me that monitoring would be interrupted frequently, since FCOS updates weekly and all updates require a reboot. I would rather not have that many gaps in the data. Ultimately I decided that deploying a cluster with Kubernetes would probably be more robust and reliable, as updates can be performed without any downtime at all. I chose not to use the Victoria Metrics Operator, but rather handle the resource definitions myself. Victoria Metrics components are not particularly difficult to deploy, so the overhead of running the operator and using its custom resources would not be worth the minor convenience it provides.
Dustin's Kubernetes Cluster
This repository contains resources for deploying and managing my on-premises Kubernetes cluster
Cluster Setup
The cluster primarily consists of libvirt/QEMU+KVM virtual machines. The Control Plane nodes are VMs, as are the x86_64 worker nodes. Eventually, I would like to add Raspberry Pi or Pine64 machines as aarch64 nodes.
All machines run Fedora, using only Fedora builds of the Kubernetes components
(kubeadm, kubectl, and kubeadm).
See Cluster Setup for details.
Jenkins Agents
One of the main use cases for the Kubernetes cluster is to provide dynamic agents for Jenkins. Using the Kubernetes Plugin, Jenkins will automatically launch worker nodes as Kubernetes pods.
See Jenkins Kubernetes Integration for details.
Persistent Storage
Persistent storage for pods is provided by Longhorn. Longhorn runs within the cluster and provisions storage on worker nodes to make available to pods over iSCSI.
See Persistent Storage Using Longorn for details.