victoria-metrics: Send alerts via ntfy

I don't like having alerts sent by e-mail.  Since I don't get e-mail
notifications on my watch, I often do not see alerts for quite some
time.  They are also much harder to read in an e-mail client (Fastmail
web an K-9 Mail both display them poorly).  I would much rather have
them delivered via _ntfy_, just like all the rest of the ephemeral
notifications I receive.

Fortunately, it is easy enough to integrate Alertmanager and _ntfy_
using the webhook notifier in Alertmanager.  Since _ntfy_ does not
natively support the Alertmanager webhook API, though, a bridge is
necessary to translate from one data format to the other.  There are a
few options for this bridge, but I chose
[alexbakker/alertmanager-ntfy][0] because it looked the most complete
while also having the simplest configuration format.  Sadly, it does not
expose any Prometheus metrics itself, and since it's deployed in the
_victoria-metrics_ namespace, it needs to be explicitly excluded from
the VMAgent scrape configuration.

[0]: https://github.com/alexbakker/alertmanager-ntfy

victoria-metrics/scrape.yml

@@ -14,6 +14,10 @@ scrape_configs:
     separator: ':'
     regex: 'vmstorage:(vminsert|vmselect)'
     action: drop
+  - source_labels:
+    - __meta_kubernetes_pod_label_app_kubernetes_io_component
+    regex: alertmanager-ntfy
+    action: drop
   - source_labels:
     - __meta_kubernetes_pod_container_name
     regex: vmalert