Deploying the Kubernetes cluster manually is a very complex, tedious, long, and error-prone process. Industry demands agility and wants to set up Kubernetes clusters as fast as possible. So, the Community has started the incubated project called Kubespray (Kargo) under the community umbrella. KubeSpray is an incubated Kubernetes community project for deploying K8s clusters on-premises or in the cloud.
Kubespray runs on bare metal and most clouds, using Ansible as its substrate for provisioning and orchestration. So, It is more flexible than other cluster orchestration tools Kops and Kubeadm in terms of platforms. It supports specific Linux distribution versions which are Centos 7, CoreoS (Container Linux), Ubuntu 16, and Debian Jessie. It supports the following types of infrastructure deployment:
- Baremetal
- Openstack
- Virtual Machines (KVM, Vmware, VirtualBox)
- Public clouds (AWS, Google Cloud etc)
I will set up 3 nodes Baremetal Kubernetes cluster. I have already set up three Nodes running on Centos 7. Now we will deploy the Kubernetes cluster using Kubespray.
PreRequisites for Kubespray
Ansible v2.3 (or newer)
To install the latest ansible on RPM-based distributions, run:
$ sudo yum install epel-release $ sudo yum install ansible
To install latest ansible on Debian based distributions, run:
$ sudo apt-get update $ sudo apt-get install software-properties-common $ sudo apt-add-repository ppa:ansible/ansible $ sudo apt-get update $ sudo apt-get install ansible
Jinja 2.9 (or newer)
To install Jinja 2.9 or upgrade existing Jinja to version 2.9, run:
$ easy_install pip $ pip2 install jinja2 --upgrade
Allow IPv4 forwarding
You can check IPv4 forwarding is enabled or disabled by executing the following command:
$ sudo sysctl net.ipv4.ip_forward
If the value is 0 then, IPv4 forwarding is disabled. Execute the following command to enable it.
$ sudo sysctl -w net.ipv4.ip_forward=1
Password Less authentication between the kubespray machine and all inventory nodes
Your machine ssh key must be copied to all the servers part of your inventory. If you are not aware of how to set up passwordless authentication between machines, You can refer article ssh-login-with-public-key
The firewalls should not be managed and The target servers must have access to the Internet
Kubespray CLI installation
You can also use kubespray without CLI by directory cloning its git repository. We will use it using CLI.
To install kubespray, run:
$ pip2 install kubespray
You can check the version of kubespray after successful completion of installation
$ kubespray -v
During installation, one new file ".kubespray.yml" is created in $HOME. It contains the information about your cloud credentials and various other options specific to different clouds. One new directory ".kubespray" is also created in $HOME. It is replica of kubespray git repository.
Inventory File setup
Create new inventory file at ~/.kubespray/inventory/inventory.cfg and Add the contents as shown below.
$ vi ~/.kubespray/inventory/inventory.cfg machine-01 ansible_ssh_host=192.168.0.144 http_proxy=http://genproxy:8080 machine-02 ansible_ssh_host=192.168.0.145 http_proxy=http://genproxy:8080 machine-03 ansible_ssh_host=192.168.0.146 http_proxy=http://genproxy:8080 [kube-master] machine-01 machine-02 [etcd] machine-01 machine-02 machine-03 [kube-node] machine-02 machine-03 [k8s-cluster:children] kube-node kube-master
Here, I am using 3 nodes for the cluster, You need to configure inventory file as per your architecture. Change IPs according to your environment, I am using the proxy server to access the internet, So I have added "http_proxy=http://genproxy:8080" in hosts declaration.
You can modify and remove as per your requirement. I am treating machine-01 and machine-02 as master, machine-02 and machine-03 as nodes, while all the three machine as etcd servers.
Let's start the cluster deployment.
Kubernetes Cluster Deployment Using Kubespray
Before, starting actual Deployment, Let's see what will be going behind the scenes and how painful manual installation task is executed smoothly. Kubespray will install kubernetes-api-server, etcd (key-value store), controller, Scheduler will be installed on master machines and kubelet, kube-proxy and Docker (or rkt) will be installed on node machines (minions). These all components will be installed and configured by ansible roles in kubespray. All, We need to do is to execute one command.
To start deployment of kubernetes cluster, run:
$ kubespray deploy
Based on the number of master and minions, It will take time to deploy the complete cluster. At the end of execution, you will get output something like shown below. If there are no failed task, Your deployment is successful.
PLAY RECAP ******************************************************************** 192.168.0.144 : ok=278 changed=89 unreachable=0 failed=0 192.168.0.145 : ok=287 changed=103 unreachable=0 failed=0 192.168.0.146 : ok=246 changed=78 unreachable=0 failed=0 localhost : ok=3 changed=1 unreachable=0 failed=0
To check that Everything went good and deployment was successful, you can login to master node and get all the worker node.
$ kubectl get nodes machine-02 Ready 4m machine-03 Ready 4m
List pods in all namespaces, type:
$ kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system dnsmasq-7yk3n 1/1 Running 0 5m kube-system dnsmasq-5vfh0j 1/1 Running 0 5m kube-system flannel-machine-02 2/2 Running 0 4m kube-system flannel-machine-03 2/2 Running 0 4m kube-system kube-apiserver-machine-01 1/1 Running 0 5m kube-system kube-controller-manager-machine-01 1/1 Running 0 5m kube-system kube-proxy-machine-02 1/1 Running 0 4m kube-system kube-proxy-machine-03 1/1 Running 0 4m kube-system kube-scheduler-machine-02 1/1 Running 0 5m kube-system kubedns-p8mk7 3/3 Running 0 4m kube-system nginx-proxy-machine-02 1/1 Running 0 2m kube-system nginx-proxy-machine-03 1/1 Running 0 2m
Seems all good. So, we have successfully deployed 3 node kubernetes cluster on bare metal server using kubespray. It can be also used to deploy cluster on Openstack, Vagrant and cloud like aws, google etc. Kubespray future seems to be very promising because of its ease of use. Please share your thoughts in the comments section. Bye, cheers..!
Can you give the exact steps for deploying the kubernetes simple cluster on aws. I did check the documentation, on the kubespray git repo, but seems tedious to configure vpc and stuff and then running the ansible.
The instruction is broken.
After running $ pip2 install kubespray
.kubespray directory is not in the $HOME.
And kubespray deploy results in error because cluster.yml is not there as well.