Prometheus provides a solution for collecting metrics from systems and applications and aggregating them into a central location.
Rules can be defined in the main server to send alerts to the operations team. A web GUI displays the alerts and graphs from the collected metrics.
Auto Discovery
Prometheus is cloud friendly, and integrates with cloud discovery components such as:
- DNS SRV records
- Consul
- AWS EV2
- Google Cloud
- Kubernetes
- Mesospher Marathon
- AirBNB Nerve
- Zooeeper
- Triton
- File based – scans a directory of files every few seconds for updates.
It is written in Go and contained in a single executable file.
Architecture
Prometheus aggregates the metrics from the servers to a central server using pull requests to the agents running on the servers.
Agents are called “exporters” and used export the metrics as HTTP using REST.
Below are the types of exporters:
- Embedded into the application, listening on a special port and exposing the metrics to Prometheus server.
There are libraries in most common languages, see https://prometheus.io/docs/instrumenting/clientlibs/ for the list of libraries. - External to application
This are exporters for applications that does not have built in support for Prometheus, such as: MySQL, Apache httpd, etc.
The list of exports can be found here: https://prometheus.io/docs/instrumenting/exporters/.
Time server data store
Prometheus uses its internal time series data store for the collected metrics.
Exporters
The list of current exports can be found here: https://prometheus.io/docs/instrumenting/exporters/.
Notable exporters:
- Node exporter
Comes as part of Prometheus. Exports OS related information such as CPU load, disk space, RAM, etc.
It can also track values in files. This is useful for short living jobs running from cron. - Script exporter
Collects metrics based on script exit codes and time taken to run. - Blackbox exporter
Collects metrics from HTTP, TCP, DNS and ICMP, based on pattern matching rules. - Consul
- CouchDB
- MySQL
- Redis
- RabbitMQ
- Ceph
- GlusterFS
- Apache httpd
- HAProxy
- Nginx
- AWS
- Docker
- Collectd
- AWS CloudWatch
- Graphite
- InfluxDB
- StatsD
Software that has built in exporters
Full list can be found here: https://prometheus.io/docs/instrumenting/exporters/.
- cAdvisor
- Doorman
- Etcd
- Kubernetes-Mesos
- Kubernetes
- RobustIRC
- SkyDNS
Notifications (Alerts)
Prometheus support alerting, and enables the administrator to define rules on the central server.
After passing the rules, the alerts are sent to a component called Alertsmanager, which is in charge of silencing, de-duplication, and sending the alerts via email, PagerDuty, Slack etc.