basementhud/pipeline/head This commit looks goodDetails
By default, the emergency shell prompts for the root password. If the
root account is locked, the emergency shell is unusable. This makes it
impossible to troubleshoot issues before the network is available.
Fortunately, systemd can be configured to skip the root password prompt
and allow access to the emergency shell with an environment variable.
basementhud/pipeline/head This commit looks goodDetails
NetworkManager is a memory hog. We don't need it to do anything except
renew the DHCP lease when it's time, so let's switch to a more
lightweight alternative. Unfortunately, Fedora does not include any
systemd units for `dhclient`, so we have to provide our own.
Running Voonex (Python), Playwight (Node.js), and Firefox simultaneously
takes way too much memory. Using the NBD swap prevents hard lockups and
crashes in OOM situations, but it makes the machine *incredibly* slow.
To avoid needing to push anything into the network swap, I want to try
and reduce the memory footprint as much as possible. To that end, we
will go back to running just Firefox. To control it, we will use the
Marionette protocol.
Firefox only allows control via Marionette over the loopback interface.
This is hard-coded in the Marionette server, and cannot be changed at
runtime. To allow a remote machine (running Voonex) to control it, we
need to expose the socket to the network. Fortunately, *systemd*
includes a tool for exactly this purpose: `systemd-socket-proxyd`.
Machine-specific data, such as SSH keys, should not be included in the
immutable root filesystem image, as this would prevent multiple machines
running from the same image. These few files can be stored locally, on
the SD card on eMMC flash.
The first time a machine boots up using this image, its local storage is
initialized. This involves creating a new filesystem on the block
device and generating SSH host keys. Once the storage is initialized,
it is remounted read-only. All subsequent mounts are read-only as well.
Using zram to compress pages did not provide enough memory to run
Firefox, Node, and Python for an extended period of time. Here's hoping
swap-on-NBD will be fast and reliable enough to work.
Note that systemd's default behavior with respect to swap areas is to
enable them all rather early in the boot process, before the network is
fully configured. As such, we have to use the *noauto* flag to disable
the default dependencies and add our own in a later target.
[Playwright] is a cross-browser automation framework, intended for
automating browser-based GUI application tests. It also works as a
general remote-control tool for automating basically anything having to
do with a browser.
Playwright requires a special Firefox binary that it manages itself. To
avoid downloading and installing Playwright and Firefox, and thus
storing them in memory, we need them to be already available in the
rootfs image. Since I want the control software to be updated easily,
without rebooting the machine, I decided to separate it into its own
project, [Voonex]. A short shell script to install/update and run it is
launched by the systemd user instance.
[Playwright]: https://playwright.dev/
[Voonex]: https://git.pyrocufflink.blue/dustin/voonex
While developing, it will be nice to have the serial console available
for troubleshooting, especially for issues that prevent network/SSH from
working correctly. I've temporarily connected a USB-TTL console cable
to *serial0* to control the serial console of the HUD machine.
Using `--no-W` (disable whole file transfer) saves some time and
bandwidth when uploading the root filesystem image, but it has the
negative side-effect of overwriting the destination file in place. If
any NBD clients are currently running using the image as their root
filesystem, they are likely to experience SquashFS corruption errors, as
the filesystem driver does not expect the underlying data to change once
it has been mounted. As such, we have to use the default file transfer
method to create a new file and atomically replace the old file once the
transfer is complete. The original file will be unlinked and will
eventually be deleted once no clients are using it.
Matchbox Window Manager, being designed for mobile devices, does not
handle multiple monitors well. It manages the monitors together as one
giant screen, which will make it extremely difficult to manage separate
Firefox windows on each screen. Openbox is almost as lightweight as
Matchbox, but it works significantly better for this use case.
Dustin