Prometheus Introduction and fundamentals (Part 2)

Prometheus Introduction and fundamentals (Part 2)

Job and Instance

Prometheus uses the name “instance” to represent an arbitrary independent data source (target). The collection of multiple and identical instances is called job, such as the example shown below:

- job: api-server - instance 1: 1.2.3.4:5670 - instance 2: 1.2.3.4:5671 - instance 3: 5.6.7.8:5670 - instance 4: 5.6.7.8:5671
  • Instance: The <host>:<port> part of the target’s URL that was scraped.
  • Job: an identical instance collection (mainly used to ensure scalability and reliability)

PromQL

PromQL (Prometheus Query Language) is the data query DSL language developed by Prometheus. It provides many built-in functions, which will be used in data visualization and rule alert. For more information, please refer to the official Querying page.

Prometheus Simple Deployment and Use

Prometheus official website provides a binary execution file for free download. Simply go to Download page and download it. All you have to do is selecting the one that matches and is compatible with your Operation System (OS). Take the following Linux as an example:

$ wget https://github.com/prometheus/prometheus/releases/download/v2.3.0/prometheus-2.3.0.linux-amd64.tar.gz $ tar xvfz prometheus-*.tar.gz $ tree prometheus-2.3.0.linux-amd64 ├── console_libraries # Web console templates │   ├── menu.lib │   └── prom.lib ├── consoles # Web console templates │   ├── index.html.example │   ├── node-cpu.html │   ├── node-disk.html │   ├── node.html │   ├── node-overview.html │   ├── prometheus.html │   └── prometheus-overview.html ├── LICENSE ├── NOTICE ├── prometheus # Prometheus 執行檔 ├── prometheus.yml # Prometheus 設定檔 └── promtool # 2.x+ 版本用來將一些 rules 格式轉成 YAML 用。

After uncompressing the file, edit the prometheus.yml to change settings.

global: scrape_interval: 15s # 設定預設 scrape 的拉取間隔時間 external_labels: # 外通溝通時標示在 time series 或 Alert 的 Labels。 monitor: 'codelab-monitor' scrape_configs: # 設定 scrape jobs - job_name: 'prometheus' scrape_interval: 5s # 若設定間隔時間,將會覆蓋 global 的預設時間。 static_configs: - targets: ['localhost:9090']

Upon completion of editing, execute prometheus directly to activate server.

$ ./prometheus --config.file=prometheus.yml --storage.tsdb.path /tmp/data ... level=info ts=2018-06-19T08:46:37.42756438Z caller=main.go:500 msg="Server is ready to receive web requests."

–storage.tsdb.path 預設會直接存放在./data底下。

After activation, browse to port:9090 to view the web-based console.

Or enter :9090/metrics to view export metrics information. You may also query in the console to find specific metrics, and present them in chart.

Prometheus provides Functional Expression Language for querying and aggregating time series data. For example, using sum(http_requests_total{method=”GET”} offset 5m) to sum up value of a specific time.

Prometheus provides third-party or self-developed exporter metrics to monitor data. The example below uses Go client to illustrate a simple deployment of Exporter.

$ git clone https://github.com/prometheus/client_golang.git $ cd client_golang/examples/random $ go get -d $ go build

Then, open three terminal windows respectively to activate the following exporter:

# terminal 1 $ ./random -listen-address=:8081 # terminal 2 $ ./random -listen-address=:8082 # terminal 3 $ ./random -listen-address=:8083

After activation, you may see the metric information via the port :8081

If there is no problem, modify prometheus.yml to add target, and reactivate Prometheus Server:

global: scrape_interval: 15s # 設定預設 scrape 的拉取間隔時間 external_labels: # 外通溝通時標示在 time series 或 Alert 的 Labels。 monitor: 'codelab-monitor' scrape_configs: # 設定 scrape jobs - job_name: 'prometheus' scrape_interval: 5s # 若設定間隔時間,將會覆蓋 global 的預設時間。 static_configs: - targets: ['localhost:9090'] - job_name: 'example-random' scrape_interval: 5s static_configs: - targets: ['localhost:8080', 'localhost:8081'] labels: group: 'production' - targets: ['localhost:8082'] labels: group: 'canary'

After reactivation, you may use the following line to query:

avg(rate(rpc_durations_seconds_count[5m])) by (job, service)

Prometheus also provides self-defined group rules to specify expression query as a metric. Here we create a file prometheus.rules.yml and add the following content:

groups: - name: example rules: - record: job_service:rpc_durations_seconds_count:avg_rate5m expr: avg(rate(rpc_durations_seconds_count[5m])) by (job, service)

Then modify prometheus.yml and add the following content, and then reactivate Prometheus Server:

global: ... scrape_configs: ... rule_files: - 'prometheus.rules.yml'

Do not modify global and scrape_configs , only add rule_files and be careful of the file paths.

After normal activation, you may see the new metric added.

Refererences

By 白凱仁 迎棧科技資深工程師

EDM

Select list(s)*

 

Loading