Kubernetes Monitoring Made Visual: Grafana + Prometheus on K3s

Helm is the Kubernetes package manager. Think of it like apt on Ubuntu

Check the helm repo

helm repo list

Add grafana repo to helm

helm repo add grafana https://grafana.github.io/helm-charts

Install Grafana

helm install grafana grafana/grafana --namespace monitoring

the standalone grafana/grafana chart is fine for lab but In production you’d use kube-prometheus-stack which bundles everything together.

Get the admin password:

kubectl get secret --namespace monitoring grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

Run get pods to check the Grafana pod

Run get services output to see the ports exposed

kubectl get svc -A

Save the yaml file in the pod to a temp file

kubectl get svc grafana -n monitoring -o yaml > /tmp/grafana-svc.yaml

Open the yml to expose the UI

Expose UI via NodePort like we did for Prometheus by Change type: ClusterIP to type: NodePort and add nodePort: 30030 under ports.

Apply the pod config

Show all services in every namespace and port

Login as admin/secret passwd

In the Grafana UI, connect Grafana to Prometheus. I

Select Prometheus

Default DNS domain that Kubernetes automatically assigns to every service in the cluster is monitoring.svc.cluster.local so the connect it we just use prometheus-server.monitoring.svc.cluster.local

We will now add a dashboard to Grafana

New → Import > Import via grafana.com

box enter: 3119 for Kubernetes cluster monitoring so these are like our vRops templates

Add more ID as we go. box enter: 6417 for AWX

We can keep scaling as we go

Our Prometheus / Grafana dashboard is ready