Docker is great peace of software, but Free Software world doesn't tolerate mono-cultures. Whenever some piece of software gets popular, the alternatives show up and each of them makes its own design changes in order to fill different niches. Containers are no exception. First there was LXC and OpenVZ, then Docker shown up, then CoreOS introduced Rocket or rkt, and then Canoincal introduced updated LXC in the face of LXD. Hat didn't twiddle thumbs either, they made their own fork of Docker called OCID.
And those are only container formats, we also have container clustering and orchestration technologies like Kubernetes. There is distribution of Kubernetes from Hat called OpenShift, and there are other competing technologies not based on Kubernetes like for example Marathon which manages Apache Mesos and DC/OS containers. Another competitor to Kubernetes is Docker's on project Docker swarm. In this article we will take a break from practical tutorials review all those alternatives to see what they offer. In some of the next articles we will show how to install and use some of them.
LXC and LXD
Lets start from Canonical's projects for container hypervisors. LXC and LXD are technically separate projects, but LXD builds upon LXC and it is more or less a front end of LXC. LXC is almost a decade old project which started as a cgroups front end for making jails better jails than chroot, and evolved into full machine container. All other container formats (except OpenVZ) are designed around a notion of encapsulating single process of one application, while LXC and LXD give you entire machine container, where you can run multiple services like you would on Virtual Machine like KVM.
The reason why LXD was created, is to change interface of LXC, something that would not be possible without breaking backward compatibility. So then new project was spawned that uses LXC core libraries and implements a cleaner interface and security model. For example LXD runs by design as non privileged user, unlike LXC which supports previously ran only as root, and latter added partial support for unprivileged containers. The goal of both LXD and LXC is to make containers as close as possible to real Linux install, like it would be in VM, except unlike the VM, they use host kernel. Features supported include live migration, snapshoting, device passtrough and many more.
For more information on LXC and LXD, vist the https://linuxcontainers.org/
OpenVZ is even older than LXC and it is very similar in its design. It is modeled after Solaris Containers and it uses patched Linux kernel. The company behind it, Virtuozzo, has vzLinux operating system that comes bundled with OpenVZ. Since OpneVZ 7.o, it uses KVM as hypervisor with the host kernel, meaning it is basically a VM that can only run Linux. Feature set is very similar to LXC and LXD except that Virtuozzo company offers Virtuozzo 7, a proprietary offering based on the OpenVZ technology.
That does it for full machine containers, now we look into application container solutions and leader in this space is Docker. Docker is produced by Docker Inc and it have recommended itself as de facto standard when it comes to application containers. That means that in docker you should generally run one process in foreground mode per container. For more processes, you just run more container and in those cases orchestration and clustering technology like Kubernetes is advisable. Docker runs on host machine as docker daemon and launches containers using chgroups namespace isolation, not unile LXC but it is no longer based on LXC core libraries like it was in early days. For getting started for docker, you can read the official site for documentation. Docker Inc is very popular with Dockercon has over 5K attendees.
OCID is a daemon launched to create industry standards around containers. It can be used by Kubernetes for running container images that are stored on Docker registries. According to Red Hat it was required as a stable enterprise-ready version of Docker because Docker is too fast moving and it breaks backwards compatibility way too often. OCID stands for Open Container Initiative Daemon, and it is developed as part of Kubernetes project. Currently it is in Kubernetes Incubator. The project regularly feeds code back to docker development and it does not aim to break away with Docker compatibility wise.
Rocket or rkt is CoreOS way of doing contaners. It is competitor to docker which unlike docker has no damon which runs all the time. Instead it has single rkt executable which can be invoked to create containers. The containers in rkt have App Container format, but rkt can execute containers made by Docker as well. Like docker, it is designed for running singe foregrount process in each container.
Now that we pinned down most popular container technologies, need to see how to use them at scale. We need orchestration and clustering technlogies. The leader here is Kubernetes.
Even before Docker, and even before LXC, Google ran its own version of cgroup containers on their own version of Linux kernel inside their data centers. And since they ran it on enormous scale, they had orchestration and clustering technology to go with it. With the rise of Docker, they decided to start new project called Kubernetes which will use standard containers in format of Docker or rkt. They used Go programming language and decided to supplant their own private in-house technology with newly founded Kubernetes project which quickly became one of top open source projects.
Kubernetes is built in a such a way that can use docker, rkt or OCID container format, while helping developers and administrators to quickly scale fault tolerant clusters running thousands of containers on many nodes. Kubernetes support self-healing of clusters by restarting the responsiveness processes, have replication controllers and support load-balancing and adding nodes on the fly.
Docker Swarm is Docker clustering solution from Docker inc. It can make several docker hosts act as one large host for running Docker containers. It uses same API as docker which means any application using docker API can use Docker Swarm as well in order to scale. Unfortunately, Docker Swarm is not nearly as popular as Kubernetes and it has no such big community, which is the main drawback of this clustering technology.
OpenShift is Kubernetes Distribution by Red Hat. The OpenShift was originally based on Makara technology that Red Hat created with same name company. But latter this was fully rewritten and made to work with Kubernetes. The front end of OpenShift is written in Ruby and Go, while the Kubernetes part is off course in Go. Red Hat is one of main contributors of Kubernetes Project, second to only Google itself.
CloudFoundry is competing PaaS offering to OpenShift. It was started by Pivotal, the VMware spin-off company which is now part of Dell along with VMware and EMC. It is written in Ruby and Go, same as OpenShift. Most contributions too it now comes from Dell and IBM employees, and only commercial offering on top of CloudFoundry is now from Pivotal, a subsidiary of Dell.
Apache Mesos and Marathon in DC/OS
Used by a few large organizations at massive scale (e.g. Twitter). This is very complex project or set of projects that make lower level clustering. Kubernetes is also deployed on top of Mesos and DC/OS, so it is not strictly competitor to Kubernetes, it is also a complimentary component. Mesos is written in C++ and it is modeled by Google's secret Omega scheduler. DC/OS comprises of Mesos, Kubernetes, Marathon and various other components. Marathon is PaaS offering which is deployed on top of DC/OS. It is written in Scala.
We have listed and described all notable container formats as well as orchestration and clustering technologies for them. We hope that this article helps you better see options for your container based data center solution. Thank you for reading and have a nice day.