The Kubernetes root CA certificate is stored in a ConfigMap named
`kube-root-ca.crt` in every namespace. The _host-provisioner_ needs to
be able to read this ConfigMap in order to prepare control plane nodes,
as it is used by HAProxy to check the health of the API servers running
on each node.
We don't want to pull public container images that already exist. This
creates prevents pods from starting if there is any connectivity issue
with the upstream registry.
The `scrape-collectd` ConfigMap in the `default` namespace is used by
Victoria Metrics to identif the hosts from which it should scrape
collectd metrics. When deploying new machines that are _not_ part of
the Kubernetes cluster, we need to explicitly add them to this list.
The _host-provisioner_ can do this with an Ansible task, but it needs
the appropriate permissions to do so.
The _k8s-worker_ Ansible role in the configuration policy now uses the
Kubernetes API to create bootstrap tokens for adding worker nodes to the
cluster. For this to work, the pod running the host-provisioner must be
associated with a service account that has the correct permissions to
create secrets and access the `cluster-info` ConfigMap.
The *dch-webhooks* server now has a _POST /host/online_ hook that can
be triggered by a new machine when it first comes online. This hook
starts an automatic provisioning process by creating a Kubernetes Job
to run Ansible and publishing information about the host to provision
via AMQP. Thus, the server now needs access to the Kubernetes API in
order to create the Job and access to RabbitMQ in order to publish the
task parameters.
[ARA Records Ansible][0] is a results storage system for Ansible. It
provides a convenient UI for tracking Ansible playbooks and tasks. The
data are populated by an Ansible callback plugin.
ARA is a fairly simple Python+Django application. It needs a database
to store Ansible results, so we've connected it to the main PostgreSQL
database and configured it to connect and authenticate using mTLS.
Rather than mess with managing and distributing a static password for
ARA clients, I've configured Autheliad to allow anonymous access to
post data to the ARA API from within the private network or the
Kubernetes cluster. Access to the web UI does require authentication.
[0]: https://ara.recordsansible.org/
Dustin