If you read the previous article about kops and how to make a cluster, this article will be a continuation of it. You will need kubernetes cluster on which you will install Prometheus and Grafana. You might need to upgrade this cluster though, and I will show how in this article.
Upgrading cluster with Kops
For successful operation of Prometheus we need Kubernetes 1.7.3, and we are actually going to upgrade to 1.7.5.
To setup the upgrade, we need to edit yaml of the cluster with the following command:
kops edit cluster
Then we will change the line that says kubernetesVersion to look like this:
kubernetesVersion: 1.7.5
Next, we will try out configuration for validity
kops update cluster
And then actually update cluster by running the same command with flag --yes.
kops update cluster --yes
After the kops finish this process, you will probably be prompted to do rolling update as well. You can do it with following commands:
kops rolling-update cluster
Again you need to say --yes to commit the changes
kops rolling-update cluster --yes
This will reboot all the AWS instances, and when they are back up, you will want to validate cluster again to see if all went well.
kops validate cluster
Setting up Prometheus and Grafana
We are done with updating the cluster and we now need to install prometheus and grafana on top of it. We will start by cloning the repository.
git clone https://github.com/camilb/prometheus-kubernetes.git
Next , we will change directory into it.
cd prometheus-kubernetes/
We will edit service file of prometheus to change from ClusterIP to LoadBalancer
nano k8s/prometheus/prometheus.svc.ss.yaml
Just find clusterIP and change to LoadBalancer, to look similar to (but not exactly) like this:
apiVersion: v1 kind: Service metadata: annotations: prometheus.io/scrape: 'true' labels: name: prometheus name: prometheus namespace: monitoring spec: selector: app: prometheus type: LoadBalancer ports: - name: prometheus protocol: TCP port: 9090 targetPort: 9090
We will also edit the grafana service, but in this case we need need to add type field bellow spec field.
nano k8s/grafana/grafana.svc.de.yaml
spec:
type: LoadBalancer
ports:
- port: 3000
Now we are ready to go, Execute ./init.sh command and work the prompt as follows
Enter Grafana version [4.5.0]: Enter Prometheus version [v2.0.0-beta.3]: Enter Alert Manager version [v0.8.0]: Enter Node Exporter version [v0.14.0]: Enter Kube State Metrics version [v1.0.1]: Enter Dockerhub username []: yourdockerhubusername Do you want to set up an SMTP relay? Y/N [N]: n Do you want to set up slack alerts? Y/N [N]: n Do you want to monitor EC2 instances in your AWS account? Y/N [N]: n Creating 'monitoring' namespace. namespace "monitoring" created Using RBAC? [y/N]: y
Now wait few minutes and then look for grafana and prometheus services. When it is done, you need to get address of LoadBalancer in order to access the web UI. You can do it with this command
kubectl describe svc -n monitoring prometheus grafana
The lines you are looking are LoadBalancer Ingress and Port. Port for grafana is 3000 and for prometheus is 9090. You need them to form the url together with load balancer. For example like this
The password for start is admin on username admin. You will probably want to change that. As we see, prometheus is happily monitoring our kubernetes cluster and sending data to Grafana dashboard. When you decide to remove grafana and prometheus, you can do so by executing ./remove.sh script while being in the repository. How to remove kops cluster was already shown in the previous article, but let's say again, here is command
kops delete cluster --name cluster2.k8s.local --yes
Off course, change the name of the cluster for you actual. With that we conclude this tutorial and wish you good day.
