How to Create Kubernetes Cluster with Kops

In this article we will use kops to install production grade Kubernetes cluster on Amazon Web Services. For this guide we assume you have AWS account. Also you need kubectl installed, you probably already have it if you followed any of our previous Kubernetes articles. If not, we will provide installation step here along with other dependencies like awscli.

Installing kubectl, kops and awscli binaries

Lets kick off the game by fetching the kubectl

curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl && chmod +x kubectl 

sudo mv kubectl /usr/local/bin/

Next we will get kops binary

wget https://github.com/kubernetes/kops/releases/download/1.7.0/kops-linux-amd64

chmod +x kops-linux-amd64

sudo mv kops-linux-amd64 /usr/local/bin/kops

Next we need to use pip, the python's package manager to install awscli. You obviously need to have python installed.

sudo pip install awscli

Configuring awscli

You need to have access keys of your AWS account in order to use awscli. You can refer how to create root AWS access keys documentation.

Once you do it, we can move to configuration part

aws configure

Then work the prompt as follows

AWS Access Key ID [None]: youraccesskeyID
AWS Secret Access Key [None]: yourSecretAccessKey

Default region name [None]: us-west-2
Default output format [None]:

We will need later those keys as well so you need to output them to env variables.

export AWS_ACCESS_KEY_ID=youraccesskeyID
export AWS_SECRET_ACCESS_KEY=yourSecretAccessKey

You can also add those lines to ~/.bashrc and run source ~/.bashrc command so you make it stay across reboots. But be aware that everyone using your computer can get those keys.

Next let's create group and user kops and give it required permissions so kops can operate without hiccups.

aws iam create-group --group-name kops

aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonEC2FullAccess --group-name kops

aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonRoute53FullAccess --group-name kops

aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess --group-name kops

aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/IAMFullAccess --group-name kops

aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonVPCFullAccess --group-name kops

aws iam create-user --user-name kops

aws iam add-user-to-group --user-name kops --group-name kops

And at last we create access key for kops user

aws iam create-access-key --user-name kops

You can always check IAM users with following command

aws iam list-users

Creating the cluster

For cluster creating we need to add more env variables. NAME will be the name of the cluster, since we wont do DNS settings we will use gossip based DNS and for that cluster name needs to end with k8s.local and before that you can put almost anything.

export NAME=cluster.k8s.local

Next we need to create an aws bucket for the cluster and make a variable that will kops use for state store.

aws s3api create-bucket --bucket ${NAME}-state

export KOPS_STATE_STORE=s3://cluster.k8s.local-state

Next we will actually create cluster. We will use t2.micro instances because they are free tier eligible, and if you signed up for free tier (like me) you can pass with no costs. Otherwise, AWS will cost you some money.

kops create cluster \
 --name=${NAME} \
 --zones=us-west-2a \
 --master-size="t2.micro" \
 --node-size="t2.micro" \
 --node-count="3" \
 --ssh-public-key="~/.ssh/id_rsa.pub"

After this you will get configuration files for new cluster.

You can edit it with this command. It will use your default editor, which you can change by changing $EDITOR variable.

kops edit cluster ${NAME}

After reviewing the yaml file you can commit to creating the cluster with this command:

kops update cluster ${NAME} --yes

After this, you are in for some waiting

Following command can show you when cluster is ready.

kops validate cluster

You will likely have to do it few times until nodes are initialized. Here is how my cluster looks like when it is ready.

Deploying some apps

For starters we can install the dashboard.

kubectl create -f https://git.io/kube-dashboard

We can use dashboard with following command:

kubectl proxy

Next we will install good old sock shop microservices demo.

kubectl apply -n sock-shop -f "https://github.com/microservices-demo/microservices-demo/blob/master/deploy/kubernetes/complete-demo.yaml?raw=true"

kubectl -n sock-shop get svc front-end

You will get something similar to this

NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
 front-end 100.66.88.111 <nodes> 80:30001/TCP 35m

What we looking for is port 30001 in this case. We need to match this port with the IP if master server. It is needed to allow inbound traffic to this port in AWS security groups.

Next when we form the url from ip and port, we should get the sock-shop site.

When you are done with using the cluster, you can preview its deletion with this command:

kops delete cluster --name ${NAME}

And actually delete it with this one:

kops delete cluster --name ${NAME} --yes

I am not going to delete it yet because we are going to use kube-prompt on this cluster in the next article. With this, we end this article after successful creation of production grade kubernetes cluster with Kops. Kops stands for Kubernetes Operations and it sure makes operations easy, as long as you are deploying on AWS. If you need Google Cloud Engine, Bare Metal or private OpenStack Cloud, then Ansible based Kubespray is a thing for you and we will introduce it in some of next articles.