Connect a new exporting server to a running Prometheus instance
(Reminder only, especially no full explanation of how to set up a prometheus host.)
Exporting machine: Machine that is running the node exporter/Apache exporter.
Prometheus machine: Machine that is running the Prometheus server to collect the exported telemetry data. Also may be the host for Grafana to display the data in a nice way.
On the exporting machine:
adduser pexp --system --no-create-home
Install /opt/apache-exporter
and /opt/node-exporter
binaries.
On the Prometheus machine, execute script
#!/bin/bash
if [ -z "$1" ]
then
echo "No argument supplied"
else
sudo openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 -keyout $1_exporter.key -out $1_exporter.crt -subj "/C=DE/ST=Hamburg/L=Hamburg/O=org/OU=IT/CN=example.com" -addext "subjectAltName = IP:$1"
fi
with exporting machine public IP address as argument in order to get some transport encryption for the data between exporting machine and Prometheus machine.
Copy the [public_IP]_exporter.crt of the new exporter machine to /etc/prometheus
at the prometheus machine.
Copy (i.e. scp) .key
and .crt
files to exporting machine under /etc/node_exporter
. Chown to pexp:root
there.
On exporting machine, create /opt/node_exporter/web.yml
:
tls_server_config:
cert_file: /etc/node_exporter/[public_IP]_exporter.crt
key_file: /etc/node_exporter/[public_IP]_exporter.key
and chown to pexp:root
.
Under /lib/systemd/system
create an apacheexporter.service
[Unit]
Description=Apache Exporter for Prometheus
Documentation=https://github.com/Lusitaniae/apache_exporter
After=network-online.target
[Service]
User=pexp
Restart=on-failure
ExecStart=/opt/apache-exporter/apache_exporter --web.config=/etc/node_exporter/web.yml --scrape_uri=http://localhost/server-status/?auto --telemetry.address=0.0.0.0:9117 --telemetry.endpoint=/metrics
[Install]
WantedBy=multi-user.target
and a nodeexporter.service
:
[Unit]
Description=Prometheus Node Exporter
Documentation=https://prometheus.io/docs/guides/node-exporter/
After=network-online.target
[Service]
User=pexp
Restart=on-failure
ExecStart=/opt/node-exporter/node_exporter --web.config.file=/etc/node_exporter/web.yml
[Install]
WantedBy=multi-user.target
Enable both services:
cd /etc/systemd/system/multi-user.target.wants
ln -sf /lib/systemd/system/apacheexporter.service .
ln -sf /lib/systemd/system/nodeexporter.service .
Start the services
service apacheexporter start
service apacheexporter status
service nodeexporter start
service nodeexporter status
netstat -anp
should show the exporter processes listening at ports 9117 and 9100.
On the Prometheus machine, add the new config items to /etc/prometheus/prometheus.yml
:
- job_name: "new_exporter"
scheme: https
tls_config:
ca_file: /etc/prometheus/[public_IP]_exporter.crt
static_configs:
- targets: ["[public_IP]:9100"]
- job_name: "nextcloud-apache"
scheme: https
tls_config:
ca_file: /etc/prometheus/[public_IP]_exporter.crt
static_configs:
- targets: ["[public_IP]:9117"]
Then, restart the Prometheus service with service prometheus restart
.
Last, at the exporting machine, configure the firewall to enable incoming tcp access at ports 9100 and 9117 originating from the IP of the Prometheus host only.
In Grafana, add the new job to the node exporter charts.