Understanding Kubernetes architecture

Containers solved a real problem. They let you package an application together with everything it needs so that it runs the same on a laptop, in a test environment, and in production, with no more arguments about why it worked on one machine and broke on another. What containers did not solve is what happens once you have more than a handful of them. In any serious system you end up with hundreds of containers spread across dozens of machines, and now real questions appear. Which machine should each container run on. What happens when one crashes at three in the morning. How do you run more copies of a service at midday and fewer overnight. How does one container find and talk to another when both are moving around. Doing all of that by hand does not scale, and scripting it turns into a brittle pile of glue that breaks the moment reality diverges from the script.

Kubernetes is the system that took over that job. It sits between your applications and the raw machines, and it treats a pool of servers as a single place to run work rather than a set of individual boxes you manage one at a time. You hand it your containers and a description of how they should run, and it decides where each one goes, restarts the ones that fail, adds and removes copies as demand changes, gives them stable addresses and names so they can find each other, and keeps the whole arrangement matching what you asked for even as machines come and go. The machines themselves become interchangeable, because any healthy one can run any workload, and a machine that dies is simply replaced by its neighbors picking up the slack.

Kubernetes is a cluster, meaning a group of machines pooled into one logical system that you address as a whole, and it is a platform rather than a finished product. It gives you a powerful and consistent foundation, and it deliberately leaves gaps for you to fill with the pieces that suit your environment, which is a theme we return to when we reach what it takes to run this safely in an enterprise. That tradeoff, enormous capability in exchange for real operational responsibility, is the honest shape of the thing. Everything that follows explains how Kubernetes does that job, and it all rests on a single idea.

Kubernetes is a control system whose whole job is to make the actual state of your infrastructure match a desired state that you declared, continuously and without being asked. You do not command it to run a container. You declare that a container should exist, and the system takes ownership of making that true and keeping it true.

Every useful behavior follows from that one mechanism. Self-healing is reconciliation noticing a pod is missing and creating a replacement. Scaling is reconciliation noticing the requested count changed and adjusting. A rollout is reconciliation moving from one desired state to another in controlled steps. The habit you are replacing is the imperative one, where you run commands in order and fix failures by hand.

These control loops are level-triggered, not edge-triggered. A level-triggered system looks at the current state of the world and drives it toward the goal, regardless of which events it did or did not see. This is what makes Kubernetes hard to knock over, because a controller that was asleep, restarted, or briefly disconnected simply looks at reality when it wakes up and corrects whatever gap it finds. It does not need a perfect event history to do its job.

Underneath, Kubernetes is a database of objects with controllers watching it. Almost everything you touch is an API object, a pod, a service, a deployment, a node, and every object has the same shape. It has a spec, the desired state you declared, and a status, the observed state the system reports back. You write the spec. Controllers write the status. The gap between the two is the work the system is trying to do.

That uniformity is where the power comes from. Because everything is an object behind one consistent API, the same tools and the same access controls and the same reconciliation pattern apply to a database, a load balancer rule, and a custom object you invent yourself. When you later meet custom resources and operators, they are not a bolt-on. They are this same object-plus-controller pattern handed to you.

Objects live in versioned API groups, which is how Kubernetes evolves without breaking you. You will see identifiers like apps/v1 and networking.k8s.io/v1. The version signals the stability contract, from alpha that can change, through beta, to stable with strong compatibility promises.

People new to Kubernetes imagine components calling each other in a chain. They do not. For cluster state and coordination, every component talks to the API server and nothing else. The scheduler does not talk to the kubelet. The controllers do not reach into etcd. They all read from and write to one hub, and they stay current using a mechanism called a watch, a long-lived subscription the API server streams changes to. A component does still talk directly to the things it drives on its own machine, so the kubelet does call the container runtime, the network plugin, and the storage plugin on its node, but it learns what to do only from the API server. Keep those two ideas separate and the whole system stays legible.

Trace a deployment through this fabric and it becomes concrete. You submit a deployment. The API server validates it and writes it to etcd. The deployment controller sees the new object and creates a replica set. The replica-set controller sees it needs pods and creates them, still just records with no node yet. The scheduler sees unscheduled pods and assigns nodes. The kubelet on each chosen node sees its assignment and starts the containers. Every step is one component reacting to a change and writing its own change back. That is the whole coordination model.

A cluster is split into two layers, and getting that split straight in your head is the first step to understanding everything else. The control plane is the part that thinks. It makes every decision and holds the definitive record of what the cluster is supposed to look like. The worker nodes are the part that does the work, and they are where your actual applications run. What makes the distinction matter is how differently you treat the two. You guard the control plane carefully, because it holds the truth about the cluster, while the worker nodes are deliberately treated as disposable, since any healthy one can run any workload, so a node that dies gets replaced rather than nursed back to health.

API server

The API server is the front door to the whole cluster, and it is the only component allowed to read and write the cluster's records directly, so everything else has to go through it. Every request it receives runs the same fixed pipeline. First it works out who you are, which is authentication. Then it checks whether you are allowed to do what you are asking, which is authorization. Then it passes the request through admission control, which can adjust it or reject it outright, and only after all of that is anything written down. That same pipeline explains almost all of the access and policy behavior across the rest of the cluster.

etcd

etcd is the distributed database that holds the entire state of the cluster, which makes it the single most precious thing in the whole system. It keeps its copies in agreement using the Raft consensus algorithm, which needs a majority of members to agree before any change is accepted. That majority requirement is the reason you run an odd number of members, usually three or five, so the cluster can still form a majority when one member is lost. etcd also asks more of you operationally than most people expect. It has a storage limit, it needs periodic housekeeping to compact its history and reclaim space, and it needs backups that you have genuinely tested by restoring them.

Scheduler

The scheduler is the component that decides which node each new pod should run on, and it makes that decision in two passes. First it filters out every node that simply cannot run the pod, for reasons like too little memory or a rule that forbids placement there. Then it scores the nodes that survive that filter and places the pod on the best fit. You can steer these decisions with a handful of named tools. Taints and tolerations let a node repel pods unless a pod is explicitly marked to tolerate it, which is how you keep ordinary workloads off specialized machines. Node affinity and pod affinity pull pods toward particular nodes or toward each other, and their anti-affinity counterparts push them apart, so replicas do not all land on the same node. Topology spread keeps pods evenly distributed across failure zones such as separate racks or cloud availability zones. And priority lets a more important pod evict a less important one when a node is full. The scheduler places pods based on the resources you say they need, not on what they actually use. If you under-declare, it will happily pack too much onto a node and leave everything fighting for room, so treat your resource requests as a promise worth getting roughly right.

Controller manager and cloud controller manager

The controller manager is a single program that runs many small control loops side by side, each one responsible for reconciling one kind of thing, whether that is node health, the number of running replicas, the endpoints behind a service, or the lifecycle of a namespace. When Kubernetes runs on a cloud, a companion piece called the cloud controller manager handles anything provider-specific, such as asking the cloud to create a load balancer, which keeps the core of Kubernetes free of any one provider's details.

Running the control plane for real

A single control-plane node is fine for learning and unwise for production, because losing it means losing the ability to change the cluster. A highly available control plane runs three nodes, or five for the largest clusters, and there are two patterns at work. The API servers run active-active behind a load balancer, so any one of them can serve requests and a failure simply drops it from rotation. The scheduler and controller manager run active-passive through leader election, so only one instance acts at a time while the others stand ready to take over within seconds. etcd runs as a cluster of odd-numbered members, three or five, so it keeps its majority even when one member is lost. Managed Kubernetes services from the cloud providers run and guarantee this control-plane redundancy for you, which is a large part of why teams choose them.

If the control plane is the brain, the worker nodes are the muscle, and every node runs three services in constant coordination.

Kubelet

The kubelet is the agent that runs on every node, and it is the control plane's representative on the ground. It registers its node with the cluster, watches for the pods it has been given, and makes sure their containers are running and healthy, driving the runtime to start and restart them as needed. It reports back on the node's health, and it enforces the resource limits you set by leaning on the kernel. It only ever manages the containers that Kubernetes created, and ignores anything else running on the machine. It also handles node-pressure eviction. When a node starts running short on memory or disk, the kubelet begins evicting pods to protect the node itself, which means a perfectly well-behaved pod can be evicted simply for sharing a node with something greedier.

Container runtime and the CRI

The runtime is what actually runs containers, and the kubelet drives it through a standard interface, the Container Runtime Interface. That interface exists so Kubernetes is not tied to one runtime. The common choices today are containerd and CRI-O, with runc underneath. This is where the old Docker confusion lives. The built-in support for Docker as a runtime was removed from the core, and while Docker Engine can still be used through a separate adapter, the modern default is to run containerd directly. Either way, images built with Docker still run fine, because they follow the same open image format the modern runtimes understand.

Kube-proxy

Kube-proxy is the piece that makes services actually work on each node. Its job is to make sure that traffic sent to a service's virtual address ends up at a healthy pod behind it. It has done this a few different ways over the years, with iptables as the long-standing default, IPVS added later for efficiency in very large clusters, and a newer nftables mode as the modern replacement for both. IPVS is now deprecated in favor of nftables, while iptables stays the default for compatibility. More and more, a network plugin built on eBPF, a modern Linux kernel technology for running fast networking logic inside the kernel itself, takes this role over entirely with a faster dataplane rather than running as a mode of kube-proxy at all. A service address is virtual, with nothing actually listening on it. It is really just a set of forwarding rules, which is why it behaves differently from an ordinary machine address.

A pod is the smallest deployable unit, and it is not simply a container. It wraps one or more closely related containers that share a network namespace, which is a private networking sandbox the Linux kernel gives them, and can share storage, so they reach each other over localhost and share one IP. A network namespace is a low-level kernel feature, and despite the shared word it has nothing to do with a Kubernetes namespace. The first is a private network view for a group of containers, while the second is simply a way of grouping and isolating objects inside the cluster. The most important thing about a pod is that it is ephemeral by design. Pods crash, get evicted, and are replaced during scaling. You never nurse an individual pod back to health.

Requests and limits

Two numbers govern how a pod gets its share of a node, a request and a limit, and they do different jobs. A request is the amount a pod is guaranteed, and it is the figure the scheduler uses to decide where the pod fits. A limit is the ceiling the pod is not allowed to cross. Put in one line, requests drive scheduling and limits drive throttling. Throttling is the moment Kubernetes steps in once a pod reaches its limit, either by capping its access to more of a resource or, in the case of memory, by killing and restarting the container. A pod that pushes past its CPU limit is slowed rather than stopped, while a pod that pushes past its memory limit is terminated and restarted, because memory cannot be reclaimed the way CPU time can.

Health probes, three of them

Kubernetes needs some way to tell whether your application is actually working, and it cannot simply guess that from the outside, so you give it probes to check. There are three of them, and they answer three genuinely different questions. A liveness probe asks whether the container is alive at all, and failing it triggers a restart. A readiness probe asks whether the pod should be receiving traffic right now, and failing it quietly pulls the pod out of the service's routing without restarting it, which is exactly what you want while an application is still warming up. A startup probe holds the liveness checks back until a slow application has finished booting, so that a slow start is not mistaken for a failure and restarted over and over in a loop.

The controllers you actually use

You rarely create bare pods, because a bare pod that dies is gone. A Deployment manages stateless workloads with controlled rollouts and clean rollbacks. A StatefulSet gives each pod a stable identity and its own persistent storage, which is what databases need. A DaemonSet runs one pod per node for agents like log collectors. Jobs and CronJobs run batch work to completion. A PodDisruptionBudget keeps enough replicas alive during a node drain so an upgrade cannot take your whole service down at once.

Kubernetes lays down a network model but, quite deliberately, does not build it. The model itself is refreshingly simple. Every pod gets its own unique IP address, any pod can talk directly to any other pod across nodes without network address translation getting in the way, and the address a pod sees for itself is the same one everyone else uses to reach it. Kubernetes states those requirements and then hands the actual implementation to a pluggable component called the Container Network Interface, or CNI.

The four layers, from a container out

Kubernetes networking is easiest to hold in your head as four layers, working from the innermost outward. The innermost is the containers inside a single pod, which share one network namespace and talk to each other over loopback at 127.0.0.1, exactly as if they were processes on the same machine. One layer out, every pod has its own unique IP and can reach any other pod directly across the cluster with no address translation in the way, and this pod network is the layer the CNI builds. One layer out again, a service places a single stable virtual IP in front of a group of pods, so callers reach the group by one unchanging address even as the pods behind it come and go. The outermost layer is how the world outside gets in, through a node port, a load balancer, or an ingress.

Following a request the whole way in makes those layers concrete. A public load balancer takes traffic on a friendly port like 80 and forwards it to a node port, which is a high port in the 30000 to 32767 range that is opened on every node in the cluster and stays fixed unless the service is recreated. The node port is routed inward to the service's cluster IP, which is virtual and deliberately unreachable from outside, and the cluster IP in turn forwards to one of the healthy pods behind it. Inside that pod, the receiving container is reached over loopback. Every hop moves one layer further in, from the real host network, to the virtual service network, to the pod, to the container.

Choosing a network plugin

Plugins differ in how they move packets, and it matters. Some create an overlay that wraps pod traffic to carry it between nodes, simple and portable at a small cost. Others route natively for speed and add policy enforcement. Cilium uses eBPF for high-performance networking, fine-grained policy, and often replaces kube-proxy entirely. The CNI choice sets your performance, your security model, and which features you can even use.

Services, DNS, and getting traffic in

Above the raw network sit the pieces you work with. A Service gives a stable address to a set of pods, in types: ClusterIP for internal, NodePort on a fixed node port, LoadBalancer for external. CoreDNS gives every service a predictable name. An Ingress routes external traffic in and terminates TLS, with the Gateway API as its more capable successor.

The LoadBalancer type is worth a closer look, because where the load balancer actually comes from depends on where you run. On a cloud, the provider creates one for you automatically. On your own hardware, often called bare metal, there is no cloud load balancer to ask for, so you install a small add-on such as MetalLB or kube-vip to do that job, handing out and advertising the public addresses itself. These can work in one of two ways. The simpler is Layer 2 mode, where one node volunteers to answer for the address on the local network and passes incoming traffic on from there. It is easy to set up, but every connection to that service arrives through that one node. The other is BGP mode, named after the routing protocol that networks use to tell each other where addresses live. Here the cluster announces the address to your network's routers, which then spread incoming traffic across all the nodes and recover faster if one fails, though this does need routers that understand BGP.

Following a request end to end

It helps to trace a single request through all of this from end to end. When someone opens the application in a browser, the name they typed is resolved through DNS to the address of the load balancer standing in front of the cluster. The load balancer hands the request to an ingress controller inside the cluster, which reads the host and path and picks the right service to send it to. The service, through kube-proxy, forwards it to one healthy pod out of however many are running, and the container in that pod does the actual work and produces a response. The response then travels back out along the same path, through the service, the ingress, and the load balancer, to the browser that asked. Every piece you have met in this chapter has a place on that one journey.

Finding another service inside the cluster

So far we have followed a request coming in from outside, but pods also need to find and talk to each other inside the cluster, and they do it through the same DNS. This is what people mean by service discovery. Every service is automatically given a name of the form service.namespace.svc.cluster.local, and CoreDNS answers that name with the service's stable ClusterIP. A pod reaches another service simply by connecting to the name. Within the same namespace the short form, just the service name on its own, is enough. To reach a service in a different namespace, the pod adds the namespace, as in orders.payments, and CoreDNS resolves it exactly the same way, since names work across the whole cluster no matter which namespace they live in. From there the request follows the same internal hops as before, because the ClusterIP is virtual and kube-proxy passes the connection on to one healthy pod behind the service. The rule of thumb is to always reach other services by name and never by pod IP, since pod IPs change all the time while the service name and its ClusterIP stay put.

Two escape hatches, HostPort and HostNetwork

Two escape hatches sidestep this layering entirely, and both are worth recognizing when you see them. A HostPort binds a container's port straight to a port on the specific node the pod happens to land on, so it is reachable at that node's own IP rather than through a service. HostNetwork goes further and places the pod directly into the host's network namespace, so the pod uses the host's IP and sees every one of its interfaces. Both trade away the isolation the pod network normally gives you.

Storage follows the same philosophy as networking. Kubernetes defines the abstractions and delegates the actual storage integration to a plugin standard, the Container Storage Interface. It exists because storage systems are wildly diverse, and baking every driver into Kubernetes itself was unsustainable. The old in-tree drivers were pushed out precisely so vendors can ship and update their own drivers on their own schedule.

The pieces fit together in a way that keeps application teams away from hardware details entirely. A PersistentVolume is an actual piece of storage in the cluster. A PersistentVolumeClaim is a request for storage, and Kubernetes binds it to a volume that satisfies it. A StorageClass describes a type of storage and provisions it on demand, which is how storage is normally handled in production, so nobody has to pre-create disks by hand. Around these, access modes declare how a volume may be mounted, and the reclaim policy decides whether the underlying disk survives once the claim is deleted.

Kubernetes separates configuration from images so the same image runs everywhere. A ConfigMap holds non-sensitive configuration injected as environment variables or files. A Secret holds credentials and keys. Kubernetes' dirty little secret, though, is that secrets are not that secret by default.

Many enterprises keep secrets in an external vault as the source of truth and sync them in through an operator, so credentials are rotated centrally and never simply sit in the cluster in reversible form.

Scaling the pods

Scaling happens at three levels, and they must work together. The Horizontal Pod Autoscaler adds and removes pod replicas based on metrics. The Vertical Pod Autoscaler adjusts the CPU and memory given to existing pods. Both operate on pods, and pods can only grow until the nodes are full.

Scaling the nodes

At the infrastructure level, node autoscaling adds machines when pods cannot be placed and removes them when idle. The traditional tool is the Cluster Autoscaler, with newer approaches like Karpenter provisioning right-sized nodes more flexibly. For queue-based work, add-ons scale on external signals and can even scale to zero. The pattern to hold: pod scaling and node scaling are separate concerns, and both must be in place for real elasticity.

Kubernetes is extensible by design, which is what turns it from a product into a platform. The object-and-controller pattern the core is built from is handed to you, unchanged, so you can teach Kubernetes about things it has never heard of and have it manage them with the same machinery it uses for pods. Operators are a framework for teaching Kubernetes new tricks. Out of the box it knows how to run generic workloads and little about what is inside them, and an operator teaches it to run something specific and complicated and, from then on, to just know how. Almost every serious piece of software you will meet on Kubernetes, the databases, the certificate managers, the monitoring stacks, the service meshes, arrives as exactly this kind of taught trick.

Custom Resource Definitions

A Custom Resource Definition adds your own object type to the Kubernetes API. Once it is installed, a new kind of object exists alongside the built-in ones, and it behaves like a first-class citizen in every way that matters. You create and edit it with the same tools, you secure it with the same role-based access control, you watch it and version it like anything else, and it lives in the same store. A database, a message queue, or a whole application can become a named object you declare and manage through the standard API.

A CRD on its own is only a shape, though, and that limit is the point. Installing the definition teaches the API server to accept and store objects of the new type, and nothing more. It gives you a place to declare what you want, with no behavior behind it yet. The object will sit there, valid and inert, until something running in the cluster is watching it and acting on it. That something is a controller, and getting that controller into the cluster is what an operator is for.

Operators

Standing up a fully clustered, highly available MySQL by hand is real work that only an experienced database engineer gets right. You have to provision several instances, elect a primary and attach replicas to it, configure replication between them, wire up automatic failover so that a dying primary is replaced without losing data, schedule and verify backups, and know how to move between versions without taking the database down. A MySQL operator captures all of that judgment in code. You declare, in a few lines, that you want a three-node highly available MySQL cluster, and the operator does everything the engineer would have done, then keeps doing it, watching the cluster and correcting it whenever reality drifts from what you asked for. That is the trick, taught once and performed forever.

Underneath the metaphor, an operator is a package of things you install into the cluster. It bundles the custom resource definitions for the new object types, the access permissions they need, and, at its heart, a new controller that is deployed and runs as an ordinary workload alongside your applications. That controller is the part that actually knows how to do the job. It runs the same reconciliation loop from chapter one, pointed at the new object type, so it watches the custom resources, compares the desired state you declared against the observed state of the real world, and acts to close the gap, over and over. What makes it powerful is that the loop never stops. The trick is not a one-time script that deploys the database and walks away, it is a permanent caretaker that holds the database in the shape you asked for and steps in the moment it drifts, which is exactly the operational knowledge a skilled human would apply, captured and applied without anyone being paged.

The same pattern shows up everywhere once you look. A certificate manager operator watches for certificate objects and issues and renews the real certificates behind them, so the certificate expiry that the enterprise chapter warns about is handled for you. A monitoring operator stands up and configures an entire metrics stack from a handful of custom resources. Message queues, service meshes, and whole application platforms ship as operators too. In every case the shape is identical to the MySQL example above. You declare intent in a custom object, and the operator does the ongoing work.

Not all operators are equal

Operators vary widely in how much they actually take off your hands, and the Operator Framework's capability model is a useful way to talk about that maturity. It runs across five levels, from an operator that can only install and configure its software, through one that can upgrade it safely, one that can handle the full lifecycle including backup and failover, one that surfaces deep metrics and insights, and finally one that acts on those insights on its own by scaling, tuning, and healing without a human. The higher the level, the closer the operator gets to running the software as a genuine service. It is worth checking where an operator sits on this scale before you rely on it, because the label operator promises much more than some of them deliver.

Admission webhooks

The third extension point plugs into the request pipeline from chapter four rather than the reconciliation loop. An admission webhook lets your own code inspect every object as it is created or changed, and either modify it, which Kubernetes calls a mutating webhook, or reject it, which it calls a validating webhook, before it is stored. This is how defaults get injected, how sidecars get added automatically, and how the policy engines in the enterprise chapter enforce their rules. It is the same admission stage the core uses, opened up for you to hook into.

A fair question before any of this reaches production is what hardware it actually demands. The honest answer is that the floor is modest, the real figure scales with how much you run, and one component deserves special care. It is worth knowing the grounded numbers rather than guessing.

Sizing the nodes

The published minimum for a node bootstrapped with kubeadm is two CPUs and 2GB of RAM, and the installer will refuse to start below that. Treat that as a starting line, not a target. A control-plane node for real use wants more headroom, commonly four or more CPUs, 8GB or more of RAM, and 20GB or more of disk, growing with the number of nodes, pods, and objects the cluster holds, because the API server and controllers work harder as the cluster grows. A worker node carries no fixed requirement of its own beyond a small overhead for the kubelet, the runtime, and the networking agent. You size a worker for the workloads you intend to pack onto it and then add that system overhead on top, which in practice means a worker is as large as the applications you want it to hold.

The etcd exception

etcd is the exception here, and its needs are unusual. It is not especially CPU-hungry or memory-hungry, and its own guidance puts 8GB of memory as typically enough, rising to 16GB or more only for very large clusters. What etcd is acutely sensitive to is disk write latency. Its consensus protocol forces a majority of members to write every change to disk before it counts, so a slow disk directly slows every write to the whole cluster, and if writes lag far enough it triggers leader elections that destabilize the control plane. etcd's own recommendation is a modest 50 sequential IOPS for a light cluster and around 500 sequential IOPS for a heavily loaded one, delivered at low latency, which in practice means backing etcd with SSD or NVMe rather than spinning disks or network-attached storage.

Managed and lightweight options

These figures are the overhead of running Kubernetes itself, and they sit underneath whatever your applications need on top. A managed Kubernetes service hides the control-plane and etcd sizing entirely, because the provider runs that layer for you, so on a managed cluster you are mostly sizing workers. Knowing the underlying requirements still matters, because it tells you what you are paying the provider to handle, and what you must size yourself the day you run the control plane in-house.

If the footprint itself is the obstacle, a family of lightweight distributions exists to shrink it. Distributions like k3s, k0s, and microk8s package the whole control plane into a compact, self-contained install, strip out the legacy and cloud-specific components most clusters never use, and often swap the heavyweight etcd for something lighter, since k3s defaults to an embedded SQLite database and microk8s uses a distributed SQLite variant, which removes much of the disk-latency burden described above. They run in a fraction of the usual memory, often well under a gigabyte, which is what makes Kubernetes practical at the edge, on a developer laptop, or on hardware as small as a Raspberry Pi. KubeSolo goes further still and is more specialized, a single-node distribution with no quorum and no clustering at all, which drops the entire consensus and high-availability machinery, and with it the etcd disk-input-output sensitivity, for situations where one small node is all you have or all you need.

Production is not the tidy world of tutorials, and how Kubernetes handles failure is the point of running it. The philosophy is that failure is assumed rather than prevented, and the system is built to recover automatically.

When a node fails, the control plane stops receiving its heartbeat and marks it NotReady. After a grace period it marks the node's pods for eviction, and the controllers create replacements on healthy nodes, restoring the declared state with no one stepping in. A harder case is a node that is still running but cut off from the control plane. Kubernetes handles that cautiously, rescheduling the pods elsewhere for availability, and terminating the now-duplicate pods if the isolated node reconnects. Recovery is automatic but not instant, and it depends on the probes, requests, and disruption budgets you set.

When the control plane goes away

If the entire control plane goes down, your running applications keep running. The worker nodes and the pods on them do not depend on the control plane to keep serving traffic moment to moment. What you lose is not the workloads, it is the cluster's ability to notice change and react to it. The data plane keeps flowing while the management plane goes dark.

How long does this matter? As long as nothing else changes, a cluster can run in this frozen state for a surprisingly long time, because the data plane is self-sufficient. The danger is not the outage itself, it is a second failure landing during it. A node that dies while the control plane is down takes its pods with it and nothing reschedules them, and a traffic spike arrives with no autoscaling to meet it. The cluster does not fall over when the control plane goes, it quietly loses the ability to protect itself, which is exactly why the control plane is the thing you make redundant first.

Stock Kubernetes is a powerful engine with deliberate gaps, because it aims to be a flexible foundation rather than a finished product. Running it safely in an enterprise means filling those gaps.

Authentication

Kubernetes has no built-in user store and no concept of your employees. It authenticates workloads through service accounts and can validate certificates and tokens, but it does not know your people. You integrate an external identity provider, typically through OpenID Connect, so login flows through your corporate identity system. Without this you end up handing out static credentials, which meets no serious security standard.

Authorization

Role-based access control defines what each identity may do, scoped to a namespace or the whole cluster, bound to users, groups, or service accounts. Done well it enforces least privilege. Left open, it is one of the most common ways clusters get compromised.

Policy and admission control

Authentication and authorization settle who is allowed to act, but they say nothing about how the things people create must be configured. Enterprises need guardrails for that, so that every workload sets resource limits, no container runs with dangerous privileges, and images come only from approved registries. Policy engines such as OPA Gatekeeper and Kyverno provide those guardrails by checking every object as it is created and rejecting or adjusting anything that breaks the rules, while Pod Security Admission enforces baseline safety standards across whole namespaces. This is how a security requirement stops being a wiki page people ignore and becomes something the cluster enforces on its own.

Multi-tenancy, secrets, supply chain

A single cluster usually serves many teams at once, and Kubernetes gives you the tools to keep them out of each other's way, though you have to set them up yourself. Namespaces partition the cluster into separate areas, resource quotas stop any one team from starving the others, and network policies keep traffic segmented between them. Secrets need encryption at rest, and often an external vault sitting behind them. On top of that, you need real confidence that only trusted, scanned, and signed images ever run, which comes from approved registries and signature checks at admission, because Kubernetes will otherwise run whatever image you point it at.

Observability, backup, certificates, and day two

Because pods are ephemeral, you need centralized metrics, logs, and traces from day one, plus the API audit log that records who did what. etcd needs tested backups and volumes need their own backup strategy. Certificates expire, so their lifecycle has to be automated or an unnoticed lapse causes an outage. And a cluster is never finished, so upgrades of the control plane, nodes, and add-ons are continuous work, best managed through GitOps where the desired state lives in version control and is reconciled onto the cluster. This day-two operations work is where most of the real cost and risk of Kubernetes actually sits.

A handful of edges account for most of the trouble that shows up in early production.

The best way to cement this model is to run a cluster and watch these behaviors happen, breaking things on purpose and seeing the system recover. The official documentation at kubernetes.io is the authoritative reference for every object and mechanism described here, and it is the source to trust over any blog when the details matter.

Once you can read and write the specs for deployments, services, ingress, and persistent storage, once probes and resource requests are second nature, and once you understand which of the enterprise gaps your environment has filled and how, you have the foundation to work on real clusters with genuine confidence.

When you are ready to deploy real applications onto the cluster, the companion KubeSchool primer on how Helm works walks through how packaging and installing them onto Kubernetes works. The primer on how GitOps works then covers how that packaged application reaches the cluster under version control, and who is allowed to change it once it is there. Sitting outside the sequence, the aside on designing systems that solve the right problem covers the decision that should come before any of this, which is whether Kubernetes was the answer to a business problem somebody could name.