infra
/
kubernetes
1
0
Fork 0
Code Pull Requests 4 Activity
kubernetes/victoria-metrics
History
Dustin 7158ff89df v-m/alerts: Ignore Restic alert for Purple Pi 
The Purple Pi is no more.  We want to keep it's backups around, though,
but we don't need alerts about them.
 		2025-09-12 07:25:21 -05:00
..
README.md v-m: Deploy (clustered) Victoria Metrics 2024-01-01 17:48:10 -06:00
alertmanager-ntfy.config.yml v-m/alertmanager-ntfy: Add labels to notifications 2024-05-22 15:20:27 -05:00
alertmanager-ntfy.yaml victoria-metrics: Send alerts via ntfy 2024-05-10 10:32:52 -05:00
alertmanager.config.yml v-m/alerts: Rework free disk space alert 2024-11-02 09:38:02 -05:00
alertmanager.yaml v-m/alertmanager: Bring up replicas in parallel 2025-09-07 10:42:50 -05:00
alerts.yml v-m/alerts: Ignore Restic alert for Purple Pi 2025-09-12 07:25:21 -05:00
blackbox-exporter.yaml v-m/blackbox: Switch to upstream container image 2024-01-15 18:45:25 -06:00
blackbox.yml v-m: Redo Internet/DNS connectivity checks (again) 2024-09-22 12:06:03 -05:00
ingress.yaml v-m: Deploy (clustered) Victoria Metrics 2024-01-01 17:48:10 -06:00
kustomization.yaml v-m/alertmanager: Migrate PVC to Synology 2025-09-07 10:42:13 -05:00
namespace.yaml v-m: Deploy (clustered) Victoria Metrics 2024-01-01 17:48:10 -06:00
recording.yml v-m/alerts: Rework free disk space alert 2024-11-02 09:38:02 -05:00
scrape.yml v-m/scrape: Scrape Music Assistant via Blackbox 2025-09-07 08:27:19 -05:00
secrets.yaml v-m: Deploy (clustered) Victoria Metrics 2024-01-01 17:48:10 -06:00
vmagent.yaml v-m/vmagent: Start in parallel 2025-09-07 10:49:22 -05:00
vmalert.yaml v-m: Switch to quay.io for container images 2025-07-07 08:43:20 -05:00
vminsert.yaml v-m: Switch to quay.io for container images 2025-07-07 08:43:20 -05:00
vmselect.yaml v-m: Switch to quay.io for container images 2025-07-07 08:43:20 -05:00
vmstorage-iscsi.yaml v-m: Redo vmstorage persistent volumes 2024-06-26 18:29:49 -05:00
vmstorage.yaml v-m: Switch to quay.io for container images 2025-07-07 08:43:20 -05:00

README.md

Victoria Metrics

Victoria Metrics is a powerful, scalable time-series database compatible with Prometheus and its ecosystem of metrics exporters.

Clustered Deployment

Victoria Metrics can run in a high-availability cluster, with the various functions of the TSDB split into independently-scalable processes:

  • vmstorage: Stores time series data.
  • vminsert: Ingests metrics in various formats (e.g. Prometheus) and sends them to one or more vmstorage nodes.
  • vmselect: Performs metrics queries, retrieving results from one or more vmstorage nodes.

The vmstorage processes are managed by a StatefulSet with a volume claim template for persistent storage. The number of replicas in the StatefulSet must be 2n-1 where n is the value of the replicationFactor setting for vminsert.

vminsert and vmselect processes are stateless and thus managed by a Deployment. There should be at least 2 replicas of each of these, so that restarts, etc. can be performed without any downtime.

vmagent

In a typical Victoria Metrics ecosystem, collecting metrics is handled separately from the TSDB. The vmagent process handles scraping and receiving metrics and passing them to vminsert. vmagent can cache received metrics locally, in case no vminsert process is available, so it requires persistent storage and is therefore managed by a StatefulSet. Because there are multiple vmagent processes scraping the same targets, the vminsert and vmstorage processes MUST have the dedup.minScrapeInterval setting set to match the vmagent scrape interval. Jobs with scrape intervals longer than the default will unfortunately have duplicate data points.

Blackbox Exporter

Many applications and web sites are monitored via the Blackbox Exporter, which makes arbitrary HTTP, TCP, ICMP, etc. requests and reports Prometheus metrics about them. This is a stateless process, managed by a Deployment.

vmalert

Victoria Metrics has a separate process for alerting, vmalert. This process periodically executes the queries defined in its alerting rules and creates alerts for matching results. Alerts are stored in the Victoria Metrics TSDB. Rules are defined in a YAML document, managed by a ConfigMap. Notifications are sent to Alertmanager.

Alertmanager

Alertmanager receives notifications from vmalert and sends e.g. email messages. Multiple instances can be run in a cluster; each node needs to know the host and port of every node in the cluster.