Why write this?

For a while I got the feeling that people were using Kubernetes to solve all their problems, I also got swept up in this trend. After 3 years of using Kubernetes professionally and privately, I can finally summarize my thoughts on whether or not it’s the right tool for the job.

What is your context?

Cloud, on-prem, VMware, oVirt, bare metal? This matters, because Kubernetes is really good at dynamic scaling of resources. Something you might want in a Cloud environment, or a shared environment, where resources are costly.

But if you’re using an on-prem, dedicated, hypervisor environment that you already own, you should rethink your decision of using Kubernetes.

Can you afford it?

Kubernetes has a lot of resource overhead. In most managed k8s I’ve used only 50% of RAM was usable on each node, due to services the cloud provider was running.

If it ain’t broke

Before containers we would host services in a “layered design”. One layer for web proxy, one for application servers, and one for database. This gave you separation between layers by using different subnets, and in each layer you simply installed your service in a very traditional way.

There’s no reason to abandon this design, most services could function just fine like this using containers instead of traditional services.

IaC, Gitops and containers

The big progression this last decade isn’t necessarily k8s, it’s containers and Infrastructure as Code. Those two things allow for streamlined development pipelines going straight from code into git repo to deployment on server.

While still keeping the responsibility of infrastructure and configuration separate, your developers will literally only control the container images while your ops team controls how they’re deployed on the container host.

K8s is amazing, but before you reach straight for the most complex container orchestration you should consider just using regular container hosts in a classic layered design, backed up by IaC and gitops.

A lot of scaling can be achieved manually by using IaC, Gitops and containers for your design. Most services or applications will do just fine with this type of manual scaling before they need the high end automated scaling of k8s.

The trick is to figure out how to handle state and idempotency of the provisioned environment.

So when should we use kubernetes?

Here are some generalizations that might help.

• Do you have the need to automatically and dynamically scale your resource usage?
• Do you have in-house developers, or at least an in-house lead developer?
• Do you want to trust your developers with parts of infrastructure and operations?
• After you’ve done enough capacity planning to confidently say you need the features of kubernetes to efficiently handle the predicted load.