152 lines
3.7 KiB
Markdown
152 lines
3.7 KiB
Markdown
# Cluster Setup
|
|
|
|
* Fedora 35
|
|
* Fedora Kubernetes packages 1.22
|
|
|
|
|
|
## Installation
|
|
|
|
For control plane nodes, use the [`fedora-k8s-ctrl.ks`][0] kickstart file. For
|
|
worker nodes, use [`fedora-k8s-node.ks`][1].
|
|
|
|
[0]: fedora-k8s-ctrl.ks
|
|
[1]: fedora-k8s-node.ks
|
|
|
|
Use `virt-manager` to create the virtual machines.
|
|
|
|
### Control Plane
|
|
|
|
```sh
|
|
name=k8s-ctrl0; virt-install \
|
|
--name ${name} \
|
|
--memory 4096 \
|
|
--vcpus 2 \
|
|
--cpu host \
|
|
--location http://dl.fedoraproject.org/pub/fedora/linux/releases/35/Everything/x86_64/os \
|
|
--extra-args "ip=::::${name}::dhcp inst.ks=http://rosalina.pyrocufflink.blue/~dustin/kickstart/fedora-k8s-ctrl.ks" \
|
|
--os-variant fedora34 \
|
|
--disk pool=default,size=16,cache=none \
|
|
--network network=kube,model=virtio,mac=52:54:00:be:29:76 \
|
|
--sound none \
|
|
--redirdev none \
|
|
--rng /dev/urandom \
|
|
--noautoconsole \
|
|
--wait -1
|
|
```
|
|
|
|
|
|
### Worker
|
|
|
|
Be sure to set the correct MAC address for each node!
|
|
|
|
```sh
|
|
name=k8s-amd64-n0; virt-install \
|
|
--name ${name} \
|
|
--memory 4096
|
|
--vcpus 2 \
|
|
--cpu host \
|
|
--location http://dl.fedoraproject.org/pub/fedora/linux/releases/35/Everything/x86_64/os \
|
|
--extra-args "ip=::::${name}::dhcp inst.ks=http://rosalina.pyrocufflink.blue/~dustin/kickstart/fedora-k8s-node.ks" \
|
|
--os-variant fedora34 \
|
|
--disk pool=default,size=64,cache=none \
|
|
--disk pool=default,size=256,cache=none \
|
|
--network network=kube,model=virtio,mac=52:54:00:67:ce:35 \
|
|
--sound none \
|
|
--redirdev none \
|
|
--rng /dev/urandom \
|
|
--noautoconsole \
|
|
--wait -1
|
|
```
|
|
|
|
|
|
## Machine Setup
|
|
|
|
Add to *pyrocufflink.blue* domain:
|
|
|
|
```sh
|
|
ansible-playbook \
|
|
-l k8s-ctrl0.pyrocufflink.blue \
|
|
remount.yml \
|
|
base.yml \
|
|
hostname.yml \
|
|
pyrocufflink.yml \
|
|
-e ansible_host=172.30.0.170 \
|
|
-u root \
|
|
-e @join.creds
|
|
```
|
|
|
|
|
|
## Initialize cluster
|
|
|
|
Run on *k8s-ctrl0.pyrocufflink.blue*:
|
|
|
|
```sh
|
|
kubeadm init \
|
|
--control-plane-endpoint kubernetes.pyrocufflink.blue \
|
|
--upload-certs \
|
|
--kubernetes-version=$(rpm -q --qf '%{V}' kubernetes-node) \
|
|
--pod-network-cidr=10.149.0.0/16
|
|
```
|
|
|
|
## Configure Pod Networking
|
|
|
|
[Calico] seems to be the best choice, based on its feature completeness, and
|
|
a couple of performance benchmarks put it basically at the top.
|
|
|
|
```sh
|
|
curl -fL\
|
|
-O 'https://projectcalico.docs.tigera.io/manifests/tigera-operator.yaml' \
|
|
-O 'https://projectcalico.docs.tigera.io/manifests/custom-resources.yaml'
|
|
sed -i 's/192\.168\.0\.0\/16/10.149.0.0\/16/' custom-resources.yaml
|
|
kubectl create -f tigera-operator.yaml
|
|
kubectl create -f custom-resources.yaml
|
|
```
|
|
|
|
Wait for Calico to deploy completely, then restart CoreDNS:
|
|
|
|
```sh
|
|
kubectl wait -n calico-system --for=condition=ready \
|
|
$(kubectl get pods -n calico-system -l k8s-app=calico-node -o name)
|
|
kubectl -n kube-system rollout restart deployment coredns
|
|
```
|
|
|
|
|
|
## Add Worker Nodes
|
|
|
|
```sh
|
|
kubeadm join kubernetes.pyrocufflink.blue:6443 \
|
|
--token xxxxxx.xxxxxxxxxxxxxxxx \
|
|
--discovery-token-ca-cert-hash sha256:…
|
|
```
|
|
|
|
|
|
## Add Control Plane Nodes
|
|
|
|
```sh
|
|
kubeadm join kubernetes.pyrocufflink.blue:6443 \
|
|
--token xxxxxx.xxxxxxxxxxxxxxxxx \
|
|
--discovery-token-ca-cert-hash sha256:… \
|
|
--control-plane \
|
|
--certificate-key …
|
|
```
|
|
|
|
[Calico]: https://projectcalico.docs.tigera.io/getting-started/kubernetes/self-managed-onprem/onpremises
|
|
|
|
|
|
## Create Admin user
|
|
|
|
```sh
|
|
cat > kubeadm-user.yaml <<EOF
|
|
apiVersion: kubeadm.k8s.io/v1beta3
|
|
kind: ClusterConfiguration
|
|
clusterName: kubernetes
|
|
controlPlaneEndpoint: kubernetes.pyrocufflink.blue:6443
|
|
certificatesDir: /etc/kubernetes/pki
|
|
EOF
|
|
kubeadm kubeconfig user \
|
|
--client-name dustin \
|
|
--config kubeadm-user.yaml \
|
|
--org system:masters \
|
|
> dustin.kubeconfig
|
|
```
|