The *GET /kubeadm/kubeconfig/<instance-id>* operation returns a
configuration document for `kubeadm` to add the node to the cluster as a
worker. The document is derived from the kubeconfig stored in the
`cluster-info` ConfigMap, which includes the external URL of the
Kubernetes API server and the root CA certificate used in the cluster.
The bootstrap token assigned to the specified instance is added to the
document for `kubeadm` to use for authentication. The kubeconfig is
stored in the ConfigMap as a string, so extracting data from it requires
deserializing the YAML document first.
In order to access the cluster information ConfigMap, the service
account bound to the pod running the provisioner service must have the
appropriate permissions.
The lifecycle of ephemeral Kubernetes worker nodes is driven by events
emitted by Amazon EventBridge and delivered via Amazon Simple
Notification Service. These events trigger the *dynk8s* provisioner to
take the appropriate action based on the state of an EC2 instance.
In order to add a node to the cluster using `kubeadm`, a "bootstrap
token" needs to be created. When manually adding a node, this would be
done e.g. using `kubeadm token create`. Since bootstrap tokens are just
a special type of Secret, they can be easily created programmatically as
well. When a new EC2 instance enters the "running" state, the
provisioner creates a new bootstrap token and associates it with the
instance by storing the instance ID in a label in the Secret resource's
metadata.
The initial implementation of the event handler is rather naïve. It
generates a token for every instance, though some instances may not be
intended to be used as Kubernetes workers. Ideally, the provisioner
would only allocate tokens for instances matching some configurable
criteria, such as AWS tags. Further, a token is allocated every time
the instance enters the running state, even if a token already exists or
is not needed.
This commit introduces the HTTP interface for the dynamic K8s node
provisioner. It will serve as the main communication point between the
ephemeral nodes in the cloud, sharing the keys and tokens they require
in order to join the Kubernetes cluster.
The initial functionality is simply an Amazon SNS notification receiver.
SNS notifications will be used to manage the lifecycle of the dynamic
nodes.
For now, the notification receiver handles subscription confirmation
messages by following the link provided to confirm the subscription.
All other messages are simply written to the filesystem; these will be
used to implement and test future functionality.
In order to prevent arbitrary clients from using the provisioner to
retrieve WireGuard keys and Kubernetes bootstrap tokens, access to those
resources *must* be restricted to the EC2 machines created by the
Kubernetes Cloud Autoscaler. The key to the authentication process will
be SNS notifications from AWS to indicate when new EC2 instances are
created; everything that the provisioner does will be associated with an
instance it discovered through an SNS notification.
SNS messages are signed using PKCS#1 v1.5 RSA-SHA1, with a public key
distributed in an X.509 certificate. To ensure that messages received
are indeed from AWS, the provisioner will need to verify those
signatures. Messages with missing or invalid signatures will be
considered unsafe and ignored.
The `model::sns` module includes the data structures that represent SNS
messages. The `sns::sig` module includes the primitive operations for
implementing signature verification.
Dustin