A Look at Major Container Orchestration Engines
If you’re reading this article, you might be asking yourself what container orchestration engines are, what problems do they solve, and what are the differences between them. This article will attempt to give a high-level overview of Kubernetes, Docker Swarm, and Apache Mesos, as well as a few of their notable similarities and differences.
Container Orchestration Engines
While definitions vary, Kubernetes, Docker, and Swarm all fall into a class of DevOps infrastructure management tools known as Container Orchestration Engines (COEs). COEs provide an abstraction layer between pools of resources, and the application containers that run on those resources.
Along with containers, the major problem COEs solve is how to take multiple discrete resources in the cloud or datacenter and combine them into a single pool, onto which a variety of applications can be deployed. These applications can range from simple three-tier web architectures to large-scale data ingestion and processing, and everything in between.
Each of these tools provides different feature sets, and vary in their maturity, learning curves, and ease of use. Some high-level features that they can share in common are:
- Container scheduling, which consists of performing such functions as starting and stopping containers; distributing containers amongst the pooled resources; recovery of failed containers; rebalancing containers from failed hosts to healthy ones, and scaling applications via containers, whether manually or automatically.
- High availability of either the application and containers, or the orchestration system itself.
- Health checks to determine container or application health.
- Service discovery, which is used to determine where various services are located on a network in a distributed computing architecture.
- Load Balancing requests, whether internally generated in a cluster, or externally from outside clients.
- Attaching various types (network, local) of storage to containers in a cluster.
Note that this list is by no means exhaustive, and is meant to be representative of some of the high-level services provided by an orchestration engine. It’s also worth mentioning that while each of the tools discussed here will perform these functions to one degree or another, the implementations can vary quite a bit.
Kubernetes Container Orchestration Capabilities
Kubernetes (also known as “k8s”) was first released in June of 2014, and is written in Go. Translated from Ancient Greek, the word Kubernetes means “Helmsman.” The project originated from and was open-sourced by Google, and is based on their experience running containers at a massive scale.
In terms of features, it probably has the most natively integrated of the three options examined in this article. Very widely used, Kubernetes has a large community behind it. Google uses Kubernetes for its own Container as a Service (CaaS) offering, called Google Container Engine (GKE). There are a number of other platforms that support Kubernetes, including Red Hat OpenShift and Microsoft Azure.
Docker is the most ubiquitous container engine currently supported by Kubernetes, but CoreOS rkt (pronounced “rocket”) is also supported.
Kubernetes uses a YAML-based deployment model. In addition to scheduling containers on hosts, Kubernetes provides many other features.
Major features include built-in auto-scaling, load balancing, volume management, and secrets management. In addition, there is a web UI to help with managing and troubleshooting the cluster. With these features included, Kubernetes often requires less third-party software than Swarm or Mesos.
Also differentiating Kubernetes from Swarm and Mesos is the concept of “pods,” which are groups of containers that are scheduled together to make up a “service,” in Kubernetes terminology.
While it is possible to configure the Kubernetes master as a high-availability cluster, this is not as well supported as single-node master installations, and is an advanced use case with Kubernetes.
Kubernetes has a somewhat steeper learning curve, and can take more effort to configure than Swarm. Due in part to its tighter integration of features, Kubernetes is sometimes regarded as more “opinionated” than the other two orchestration engines discussed here.
Swarm Container Orchestration Capabilities
Docker Swarm is Docker’s native Container Orchestration Engine. Originally released in November of 2015, it is also written in Go. Swarmkit is the Docker native version of Swarm included as of version 1.12, which is the recommended version of Docker for those wishing to use Swarm.
Swarm is tightly integrated with the Docker API, making it well-suited for use with Docker. The same primitives that apply to a single host docker cluster are used with Swarm. This can simplify managing container infrastructure, as there is no need to configure a separate orchestration engine, or relearn Docker concepts in order to use Swarm.
Like Kubernetes, Swarm has a YAML-based deployment model using Docker Compose. Other features of note include auto-healing of clusters, overlay networks with DNS, high-availability through the use of multiple masters, and network security using TLS with a Certificate Authority.
As of this writing, Swarm does not yet support native auto-scaling or external load balancing. Scaling must be done manually or through third-party solutions. Along the same vein, Swarm includes ingress load balancing, but external load balancing would be done through the use of a third-party load balancer such as AWS ELB. Also notable is a lack of a web interface for Swarm.
Mesos Container Orchestration Capabilities
Apache Mesos version 1.0 was released in July of 2016, but it has roots going all the way back to 2009, when it was initially being developed by PhD students at UC Berkeley. Unlike Swarm and Kubernetes, Mesos is written in C++.
Mesos is somewhat different than the first two mentioned here, in that it takes more of a distributed approach to managing datacenter and cloud resources. Mesos can have multiple masters which use Zookeeper to keep track of the cluster state amongst the masters and form a high-availability cluster.
Other container management frameworks can be run on top of Mesos, including Kubernetes, Apache Aurora, Chronos, and Mesosphere Marathon. In addition, Mesosphere DC/OS, a distributed datacenter operating system, is based on Apache Mesos.
This means that Mesos takes a more modular approach to how containers should be managed, allowing users to have more flexibility in the types of applications and the scale on which they can run.
Mesos can scale to tens of thousands of nodes, and is used by the likes of Twitter, Airbnb, Yelp, and eBay. Apple even has its own proprietary framework based on Mesos called Jarvis, which is used to power Siri.
A few of the features available in Mesos that are worth mentioning are support for multiple types of container engines, including Docker and its own “Containerizer,” as well as a web UI, and the ability to run on multiple OSes, including Linux, OS X, and even Windows.
Due to its complexity and flexibility, Mesos has a steeper learning curve than Docker Swarm. But, that same flexibility and complexity are also strengths that allow companies like Twitter and Airbnb to use Mesos to manage their large-scale applications.
Choosing the Right COE for Your Needs
As you can see, the subject of cluster management and the associated tools can deepen quickly. We’ve only just touched upon some of the features and use cases for the different options presented here. Each has its own strengths and weaknesses, and a solid understanding of your own use case will dictate which is most suitable for your application implementation.
With that said, if you’re just looking to get up and running and test out using an orchestration engine, then Docker Swarm is probably a good choice. When you’re ready to delve further into the subject, or possibly deploy something leaning toward industrial grade, look to Kubernetes. If flexibility and massive scale are your goals, then consider Apache Mesos.
For those wanting to get up to speed on the basics of Kubernetes, we recommend taking a few minutes to watch The Illustrated Children’s Guide to Kubernetes.
When you’re ready to learn a bit more, take a look at Comparison of Container Schedulers, and Kubernetes, Mesos, and Swarm: Comparing the Rancher Orchestration Engine Options.