Initial commit

README.md

@@ -0,0 +1,16 @@
+# Dustin's Kubernetes Cluster
+
+This repository contains resources for deploying and managing my on-premises
+Kubernetes cluster
+
+
+## Cluster Setup
+
+The cluster primarily consists of libvirt/QEMU+KVM virtual machines.  The
+Control Plane nodes are VMs, as are the x86_64 worker nodes.  Eventually, I
+would like to add Raspberry Pi or Pine64 machines as aarch64 nodes.
+
+All machines run Fedora, using only Fedora builds of the Kubernetes components
+(`kubeadm`, `kubectl`, and `kubeadm`).
+
+See [Cluster Setup](setup/README.md) for details.

setup/README.md

@@ -0,0 +1,102 @@
+# Cluster Setup
+
+* Fedora 35
+* Fedora Kubernetes packages 1.22
+
+
+## Machine Setup
+
+Add to *pyrocufflink.blue* domain:
+
+```sh
+ansible-playbook \
+    -l k8s-amd64-ctrl0.pyrocufflink.blue \
+    remount.yml \
+    bootstrap.yml \
+    pyrocufflink.yml \
+    -e ansible_host=172.30.0.167/28 \
+    -u root \
+    -e @join.creds
+```
+
+Set up Kubernetes agent (`kubelet`):
+
+```sh
+ansible-playbook -l k8s-amd64-ctrl0.pyrocufflink.blue kubelet.yml -b
+```
+
+
+## Initialize cluster
+
+Run on *k8s-ctrl0.pyrocufflink.blue*:
+
+```sh
+kubeadm init \
+    --control-plane-endpoint kubernetes.pyrocufflink.blue \
+    --upload-certs \
+    --kubernetes-version=$(rpm -q --qf '%{V}' kubernetes-node) \
+    --pod-network-cidr=10.149.0.0/16
+```
+
+## Configure Pod Networking
+
+[Calico] seems to be the best choice, based on its feature completeness, and
+a couple of performance benchmarks put it basically at the top.
+
+```sh
+curl -fL\
+    -O 'https://projectcalico.docs.tigera.io/manifests/tigera-operator.yaml' \
+    -O 'https://projectcalico.docs.tigera.io/manifests/custom-resources.yaml'
+sed -i 's/192\.168\.0\.0\/16/10.149.0.0\/16/' custom-resources.yaml
+kubectl create -f tigera-operator.yaml
+kubectl create -f custom-resources.yaml
+```
+
+Wait for Calico to deploy completely, then restart CoreDNS:
+
+```sh
+kubectl wait -n calico-system --for=condition=ready \
+    $(kubectl get pods -n calico-system -l k8s-app=calico-node -o name)
+kubectl -n kube-system rollout restart deployment coredns
+unset calico_node
+```
+
+
+## Add Worker Nodes
+
+```sh
+kubeadm join kubernetes.pyrocufflink.blue:6443 \
+    --token xxxxxx.xxxxxxxxxxxxxxxx \
+    --discovery-token-ca-cert-hash sha256:…
+```
+
+
+## Add Control Plane Nodes
+
+```sh
+kubeadm join kubernetes.pyrocufflink.blue:6443 \
+    --token xxxxxx.xxxxxxxxxxxxxxxxx \
+    --discovery-token-ca-cert-hash sha256:… \
+    --control-plane \
+    --certificate-key …
+```
+
+[Calico]: https://projectcalico.docs.tigera.io/getting-started/kubernetes/self-managed-onprem/onpremises
+
+
+## Create Admin user
+
+```sh
+cat < kubeadm-user.yaml <<EOF
+apiVersion: kubeadm.k8s.io/v1beta3
+kind: ClusterConfiguration
+clusterName: kubernetes
+controlPlaneEndpoint: kubernetes.pyrocufflink.blue:6443
+certificatesDir: /etc/kubernetes/pki
+EOF
+kubeadm kubeconfig user \
+    --client-name dustin \
+    --config kubeadm-user.yaml \
+    --org system:masters \
+    > dustin.kubeconfig
+```