In order to join the on-premises Kubernetes cluster, EC2 instances will
need to first connect to the WireGuard VPN. The *dynk8s* provisioner
will provide keys to instances to configure their WireGuard clients.
WireGuard keys must be pre-configured on the server and stored in
Kubernetes as *dynk8s.du5t1n.me/wireguard-key* Secret resources. They
must also have a `dynk8s.du5t1n.me/ec2-instance-id` label. If this
label is empty, the key is available to be assigned to an instance.
When an EventBridge event is received indicating an instance is now
running, a WireGuard key is assigned to that instance (by setting the
`dynk8s.du5t1n.me/ec2-instance-id` label). Conversely, when an event is
received indicating that the instance is terminated, any WireGuard keys
assigned to that instance are freed.
The lifecycle of ephemeral Kubernetes worker nodes is driven by events
emitted by Amazon EventBridge and delivered via Amazon Simple
Notification Service. These events trigger the *dynk8s* provisioner to
take the appropriate action based on the state of an EC2 instance.
In order to add a node to the cluster using `kubeadm`, a "bootstrap
token" needs to be created. When manually adding a node, this would be
done e.g. using `kubeadm token create`. Since bootstrap tokens are just
a special type of Secret, they can be easily created programmatically as
well. When a new EC2 instance enters the "running" state, the
provisioner creates a new bootstrap token and associates it with the
instance by storing the instance ID in a label in the Secret resource's
metadata.
The initial implementation of the event handler is rather naïve. It
generates a token for every instance, though some instances may not be
intended to be used as Kubernetes workers. Ideally, the provisioner
would only allocate tokens for instances matching some configurable
criteria, such as AWS tags. Further, a token is allocated every time
the instance enters the running state, even if a token already exists or
is not needed.
Dustin