...
- run-container.sh : shell script that will run our docker command to install / configure prometheus.
- prometheus.yml : single configuration file for prometheus;
Before we run the script (which will setup and configure Prometheus) let me point out a few things with these files:
See the Explanation of run-container.sh section for an explanation and several notes for these files.
Once you've cloned the repo, simply make the script executable (if it isn't already) and run the script with:
| Code Block |
|---|
|
chmod +x prometheus.sh
./prometheus.sh |
Docker will then download the latest official Prometheus Docker image, create and configure the container and then run said container.
If things worked as they should you should be able to visit the basic Prometheus web interface at http:\\<INTERNAL-SERVER-IP>:10090.
| Info |
|---|
Replace <INTERNAL-SERVER-IP> with the actual internal IP address of your server, e.g. 192.168.1.x, 10.0.0.x (or whatever it is). |
That's it! You will now have Prometheus running on port 10090.
| Info |
|---|
Note: port 10090 is not the default Prometheus port (which is 9090) - I have another service using that port (which I didn't want to change). See the Explanation of run-container.sh section for notes on changing the port. |
| Anchor |
|---|
| run-container.sh |
|---|
| run-container.sh |
|---|
|
(Optional Read) Explanation of run-container.shThis script simply creates (and/or runs) our Prometheus container. So what's going on here?
...
|
|---|
| View Git file |
|---|
| path | run-container.sh |
|---|
| repository-id | 8 |
|---|
| language | bash |
|---|
| branch | refs/tags/documented |
|---|
| linenumbers | true |
|---|
|
|
...
| Line(s) number | Comment |
|---|
| 1 | bash script header (tells shell that it's a bash script when it gets executed) | | 2 | removes / deletes the container. This deletes the already running container (if it exists) before creating a new one. | Tip |
|---|
Don't worry(!) our data won't be lost since we're persisting the Prometheus data with a Docker data volume! |
| | 3 | main docker run command. The -d flag means "detached" (i.e. run it detached or in the background) and the --name=prom names the container that will be created "prom". | | 4 | this data-binds our prometheus.yml config file to /etc/prometheus/prometheus.yml in the created Docker container. I.e. prometheus running in the container will use this file for it's configuration. | | 5 | this instructs Docker to create a data volume (for storing our persistent Prometheus data) and call it "prom-data". It's kind of similar to a data-bind but it's managed by Docker and provides many advantages. The actual files (and persistent data) can be found at:
/var/lib/docker/volumes/prom-data/_data | | 6 | |
|
...
Once you've cloned the repo, simply make the script executable (if it isn't already) and run the script with:
| Code Block |
|---|
|
chmod +x prometheus.sh
./prometheus.sh |
Docker will then download the latest
...
10090 on the host to post 10090 on the container (in other words exposes the prometheus port on the host). | Info |
|---|
Note: this is not the default port for Prometheus (default port is 9090). Change both of these to whatever port you would like to bind Prometheus to. Also see line number 9 which would also need to match the port entered here. You'll also need to modify the included prometheus.yml file and replace 10090 to whatever port you want to use. |
| | 7 | Tells Docker that this container should start/restart whenever Docker starts/restarts (for example on boot etc.). | | 8 | Points Docker to the official Prometheus image. Docker will download this image if it hasn't already. | | 9-13 | Commands for configuration sent to the Prometheus executable in the container. Most of these are pointing to the various locations were have previous bound with data-binds or data-volumes. | | 14 | Disabled. Uncomment to enable the admin-api (so can accept POST queries to modify / delete data etc.). |
|
| Anchor |
|---|
| prometheus.yml |
|---|
| prometheus.yml |
|---|
|
(Optional Read) Explanation of prometheus.ymlThis file is the main Prometheus.yml scrape configuration file. It defines things like how often Prometheus should scrape (query) data and from where (and how) it should scrape this data.
| Expand |
|---|
| View Git file |
|---|
| path | prometheus.yml |
|---|
| repository-id | 8 |
|---|
| branch | refs/tags/documented |
|---|
| linenumbers | true |
|---|
|
| Line(s) number | Comment |
|---|
| 1-5 | Default scraping config for how often Prometheus should scrape. Note, these can be overridden for specific scrape configs (see from line 22). | | 23-29 | Prometheus (be default) monitors itself and can report on scrape query duration, samples added to db, data compaction stats etc. | Info |
|---|
Note that localhost:10090 refers to the Prometheus docker container and not the server. |
| | 31-34 | Scrape configutation for node_exporter, an exporter which scrape server machine metrics. We haven't set this up yet but will in the following sections. | Info |
|---|
Note that 10.0.0.137:10091 refers to the internal IP address of the machine you want to monitor (and the port on which node_exporter is running). This is NOT localhost (localhost in this context is the Prometheus Docker container). |
| | 36-57 | Scrape configuration for blackbox, a prober that can probe endpoints over HTTP, HTTPS, DNS, TCP and ICMP. We haven't set this up yet but will in the following sections. | Info |
|---|
Note that 10.0.0.137:9115 refers to the internal IP address and port of the machine on which blackbox is running on. We'll be setting up blackbox on a docker container and then exposing port 9115 to the host. This is NOT localhost (localhost in this context is the Prometheus Docker container). |
|
|
Updating Prometheus with Docker
Updating with Docker is straight-forward. We just need to pull the (latest) image and then re-run our run-container.sh script.
| Code Block |
|---|
docker pull prom/prometheus
./run-container.sh |
Installing and configuring node_exporter (to monitor server stats)
Installing and configuring blackbox (to monitor endpoints)
Installing and configuring Grafana
...