| Status | ||||
|---|---|---|---|---|
|
Prometheus is a very nice open-source monitoring system for recording real-time metrics (and providing real-time alerts) in a time-series database for a variety of purposes.
...
Installing and configuring Grafana
Grafana is a great great platform for visualising data and metrics from large data sets. It can connect with a very large number of data sources and has native (built-in) prometheus support, which makes it extremely easy to integrate prometheus and provides an attractive and versatile front-end to view various prometheus metrics.
So let's go ahead and install grafana and do some basic setup.
Like previous approaches, we'll use a simple docker image and setup data-persistency via a docker-volume, set a port for grafana to use (in this example we use a custom port), as well as setup public (anonymous) viewing access.
We only need two files for all this - which you can get by cloning
https://gitlab.jaytaala.com/docker/grafana.git
| Code Block | ||
|---|---|---|
| ||
git clone https://gitlab.jaytaala.com/docker/grafana.git |
The repo contains:
- run-container.sh : shell script that will run our docker command to install / configure grafana.
- grafana.ini : single configuration file for grafana;
- dashboard-blackbox.json : blackbox exporter dashboard I've customised see Adding monitoring dashboards;
- dashboard-node_exporter.json : node_exporter dashboard I've customised see Adding monitoring dashboards;
See the Explanation of run-container.sh section for an explanation and several notes for these files.
Once you've cloned the repo, simply make the script executable (if it isn't already) and run the script with:
| Code Block | ||
|---|---|---|
| ||
chmod +x run-container.sh
./run-container.sh |
Docker will then download the latest official Prometheus Docker image, create and configure the container and then run said container.
If things worked as they should you should be able to visit the basic Prometheus web interface at http:\\<INTERNAL-SERVER-IP>:10090.
| Info |
|---|
Replace <INTERNAL-SERVER-IP> with the actual internal IP address of your server, e.g. 192.168.1.x, 10.0.0.x (or whatever it is). |
That's it! You will now have Prometheus running on port 10090.
| Info |
|---|
Note: port 10090 is not the default Prometheus port (which is 9090) - I have another service using that port (which I didn't want to change). See the Explanation of run-container.sh section for notes on changing the port. |
| Anchor | ||||
|---|---|---|---|---|
|
This script simply creates (and/or runs) our grafana container. So what's going on here?
| View Git file | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
| Line(s) number | Comment | ||
|---|---|---|---|
| 1 | bash script header (tells shell that it's a bash script when it gets executed) | ||
| 2 | removes / deletes the container. This deletes the already running container (if it exists) before creating a new one.
| ||
| 3 | main docker run command. The | ||
| 4 | this data-binds our grafana.ini config file to | ||
| 5 | this instructs Docker to create a data volume (for storing our persistent grafana data) and call it "graf-data". It's kind of similar to a data-bind but it's managed by Docker and provides many advantages. The actual files (and persistent data) can be found at: | ||
| 6 | binds port 4000 on the host to post 4000 on the container.
| ||
| 7 | Tells Docker that this container should start/restart whenever Docker starts/restarts (for example on boot etc.). | ||
| 8 | Points Docker to the official Grafana image. Docker will download this image if it hasn't already. |
| Anchor | ||||
|---|---|---|---|---|
|
This file is the main grafana config file for our setup. It defines things server related settings, as well as settings we'll use to enable public viewing access.
| View Git file | ||||||||
|---|---|---|---|---|---|---|---|---|
|
| Line(s) number | Comment |
|---|---|
| 1-3 | Defines server settings for our setup - including the port to bind/ and our domain name. |
| 5-10 | Settings to enable anonymous (public) viewer access to any dashboards you give |
| Info |
|---|
You'll note that I've defined my domain as |
Adding Prometheus as a data source to Grafana
Once you have your docker container running, access port 4000 on the server with a browser (e.g. visit http://192.168.0.1:4000 or whatever your internal address is for your server).
Grafana's login screen should appear - if you're the first user to login after creating this container then just enter any new credentials.
Once actually logged in, click on the "add data source" and select "Prometheus":
You can leave most setting as default and simply enter the server url for the Prometheus instance we have setup, for my case it looks like:
| Warning |
|---|
The URL should point to the server's internal IP and NOT localhost (or 127.0.0.1) since |
| Anchor | ||||
|---|---|---|---|---|
|
One of the great things with Grafana is that it is extremely easy to use (and publish) any dashboards that others have created. There's many (many) dashboards for different systems and many for prometheus and it's many exporters (like node_exporter, blackbox, etc.).
At mon.jaytaala.com I've used these dashboard (which I've then customised further to suit my needs):
- https://grafana.com/grafana/dashboards/7587 (for viewing blackbox http endpoint statuses);
- https://grafana.com/grafana/dashboards/1860 (for viewing server system stats);
You can see many other dasboards for grafana/prometheus here.
It's incredibly simple to add a dashboard to Grafana. Select the dashboards icon from the left menu and then select "Manage:
Now click the "import" button:
Either copy the link (e.g. https://grafana.com/grafana/dashboards/7587) or simply note the dashboard number (7587). Enter this into the "Grafana.com Dashboard" input and press the tab key:
| Tip |
|---|
You can also add dashboard via by uploading a dashboard .json. You'll note that I have two dashboard .json files in the repository - you can upload these by clicking the "Upload .json file" button. |
Grafana will grab the dashboard and after a second or so will fetch the dashboard ready for importing. Enter a few options (see arrows) and then click "Import":
Finally, we install and setup grafana
References
- https://prometheus.io/
- https://github.com/prometheus/node_exporter
- https://prometheus.io/download/#node_exporter
- https://github.com/prometheus/blackbox_exporter
- https://grafana.com/
...