Prometheus-server pod will be pulling metrics via Http endpoints, most typically the /metrics endpoint from various sources. Remember at the start of the article, we noted down the various Prometheus applications you will get for following this guide. We will keep it simple by only focusing on how we are retrieving node and application(running pods) metrics. I've attached an architecture diagram from the docs here for reference. To understand the architecture of Prometheus, check out their documentation. You may have wondered how all these metrics are available, even though you've simply deployed it, without configuring anything other than disk space. If there are too many errors, I will suggest finding another dashboard until you find one that works and fits your needs. We will have to edit the panel and debug the queries to fix them. You may have noticed the "N/A" in a few of the dashboard panels, this is a common problem in various dashboards, due to incompatible versions of Prometheus/Kubernetes with changes in metric labels, etc. You will have a dashboard that looks like this Select our Prometheus datasource we added in the previous step into the drop-down selection Mouse over the "+" icon and select "Import", paste the dashboard ID into the textbox and click "Load" Here are a few dashboards to get you started. Now that Prometheus is setup, and has started to collect metrics, we can start visualizing the data. Your setting should look like this.Ĭlick "Save & Test" and Grafana will tell you that the data source is working. This is the DNS A record of prometheus that will be resolvable for any pod in the cluster, including our Grafana pod. Since Grafana and Prometheus are hosted in the same cluster, we can simply use the assigned internal A record to let Grafana discover prometheus. If you've followed my steps, your Prometheus setup will create a service named prometheus-server in the prometheus namespace. Go to the datasource section under the settings wheel To access Grafana UI, run kubectl port-forward svc/grafana -n grafana 8080:80, go to and log in with the admin user, if you need the credentials, see the previous post on instructions. Prometheus-server-7cd7d486cb-c24lm 2/2 Running 0 67m Prometheus-pushgateway-567987c9fd-5mbdn 1/1 Running 0 67m Prometheus-node-exporter-l5pck 1/1 Running 0 67m Prometheus-node-exporter-8xw2r 1/1 Running 0 67m Prometheus-kube-state-metrics-685dccc6d8-h88dv 1/1 Running 0 67m Prometheus-alertmanager-78b5c64fd5-ch7hb 2/2 Running 0 67m With this out of the way, let us get our Prometheus application started. Note that I'm running Prometheus 2.x, that has an improved storage layer over Prometheus 1, and has shown to have reduced the storage usage and thus the lower need of disk space, see blog I think it is a good starting point and you can save the time and effort on calculating for storage provisioning and just start with this. If the price of 100GiB of storage is acceptable for you, in my region it is about USD12/month. So as a guide, I'll share with you my current setup and disk usage, so you can gauge how much of disk space you want to provision. This is difficult for me to calculate even with an existing setup, so if this is your first time setting up I can imagine it being even more so. Needed_disk_space = retention_time_seconds * ingested_samples_per_second * bytes_per_sample Note that we have set our PV to 100Gi in the following line of values.yaml. Prometheus server will be running with a persistent volume(PV) attached, and this volume will be used by the time-series database to store the various metrics it collects in the /data folder. We will deploy Prometheus using helm, and explain more along the way.īefore we begin, it is worth mentioning the file storage requirements of Prometheus. Knowing the resources our applications consume can help prevent disasters, such as when applications consume all the RAM on the node, causing it to no longer function, and we now have dead nodes and applications.įor us to view the metrics on our Grafana dashboard, we can integrate it into a Prometheus datasource, and have Prometheus collect metrics from our nodes and applications. While it provides us with many insights on AWS resources, it doesn't tell us how our applications are doing in our Kubernetes cluster. In the previous post, we've gotten Grafana up and running with a cloudwatch datasource.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |