How to set up a local Kubernetes cluster and deploy a self-made microservice in less than 10 minutes

Nowadays, many server applications are not installed and run directly on physical hosts or virtual machines any more. Instead, application code is often built into container images, and run in so-called pods in a Kubernetes cluster. Kubernetes provides a standardized way to orchestrate applications and works the same way no matter where the cluster is running.

Kubernetes clusters are often hosted and managed by cloud providers. They can also be deployed on-premise though, and either be created and managed by a service provider, or with tools like Kubespray. Most clusters are long-lived, and consist of multiple nodes for redundancy.

However, sometimes a cluster that can be created and discarded quickly and easily on a single computer can be very useful:

• A developer might want to create a cluster on their development machine for playing around with Kubernetes, exploring the newest tools, or testing their newly developed code and Kubernetes resources. If they used a cluster that is shared with others or even with production workloads instead, they might get into the way of others, or worse, break things that should better not break.
• Another use case is automated testing of application deployments in Kubernetes, or testing of applications that interact with Kubernetes resources themselves. This works best in a dedicated cluster that is set up in a clean state just for this purpose, and thrown away after the tests.

There are a few solutions for setting up a local Kubernetes cluster. I choose k3d here because it is pretty easy to use - it is a single binary that requires only Docker to create its cluster in a container. Unlike kind, which I also like a lot and which follows the same approach, it has an ingress controller built in. This makes accessing the applications that run in the cluster easier, and allows testing of ingress resources. With kind, this can also be done, but only after an ingress controller has been installed.

This post shows how to set up a cluster with k3d, and deploy a self-made application. It does not assume any prior Kubernetes knowledge. If you have some Kubernetes experience, and you are interested in playing around with k3d, feel free to skip most of the text and just look at the commands 🙂 You can also download the Jupyter notebook that this post is based on, and play around with it.

I use Linux to work with k3d and Kubernetes in general, but most of what I do should work pretty much the same way on a Mac. Trying it on Windows might require more modifictions though.

Installing k3d and kubectl¶

We will need not only k3d, but also kubectl, which is the command-line interface for interacting with Kubernetes clusters.

There is a range of installation options. I prefer using arkade, which allows to install many tools that are related to Kubernetes easily.

We will use the arkade installation script as recommended in its README, but run it without root permissions:

We could now move the arkade binary to a directory in our PATH, or re-run the download script with root permissions. But we can also run the binary from the current directory and use it to install the tools that we need:

We will add the directory where arkade puts the downloaded tools to the PATH for convenience:

Creating a local Kubernetes cluster with k3d¶

Now that we have downloaded k3d, we can use it to create our first local Kubernetes cluster. We will give it the name test-cluster and forward port 80 on the host to port 80 in the container which matches the node filter "loadbalancer". This will enable easy HTTP/HTTPS access to apps running in the cluster via ingress routes, as we will see in the next post.

We can verify that our cluster is running and can be accessed via kubectl:

The cluster has a single node:¹

Note that the status of the node is NotReady. We can either wait for a few seconds, or use kubectl wait, which runs until the given condition is met:

Create a simple web service¶

Let's build a simple application that we will deploy and test in our cluster. We will do it in Python with FastAPI (but we could just as well use any other programming language and framework, of course).

The app accepts GET requests at the endpoint /greet/{name}, where a name can be substituted for name. It responds with a small JSON object that contains a greeting, and also the host name. This will be interesting later on when multiple instances of the application are running.

Build Docker image¶

The Dockerfile is quite simple. We just have to copy the Python file into a Python base image and install the dependencies fastapi and uvicorn. The latter is the web server that we will use.

Import the Docker image into the Kubernetes cluster¶

To run our service in the Kubernetes cluster, the nodes in the cluster need access to the image. This can be achieved in several ways: we could either

• push the image to a public registry,
• push the image to a private registry, and configure the cluster such that it has access to this registry, or
• load the image to the nodes in the cluster directly.

Usually, the third option is the easiest when we run tests and experiments on the local development machine, so we will use it here.²

Create a namespace for the application¶

Before deploying our application, we will create a new namespace. It will contain all Kubernetes resources that belong to, or interact with, our application.

While not stricly necessary, putting each independent application into its own namespace is a good practice. Advantages of this approach include:

• If anything is seriously wrong with the Kubernetes resources of an application, its namespace can be deleted, and one can start from scratch without affecting other applications running in the Kubernetes cluster.
• One can create a service account that has only the permissions to view, modify, or create resources in one namespace. Such an account can be used for automated interactions with the cluster, e.g., in continuous integration pipelines. Any accidental effects on applications running in other namespaces can then be avoided.
• Resource quotas can be assigned to namespaces to limit the amount of resources that the applications in a namespace can use.

Now we would like to deploy our application in the new namespace.

In Kubernetes, applications run in pods¶

In Kubernetes, the "smallest deployable units of computing that you can create and manage" are pods. A pod is essentially a set of one or more containers which can share some resources, like network and storage. In our case, a single container, which runs the image that we have just built and uploaded to the cluster, is sufficient.

Kubernetes resources are usually defined in YAML files, although it is possible to create some types of resources with kubectl create directly on the command line.³ A pod that runs our application, and that lives in the namespace test, could be defined like this:

Each Kubernetes resource definition has a number of top-level keys:

• The apiVersion tells what version the resource comes from.
• The kind tells what kind of resource is defined.
• The metadata include the name of the resource, the namespace that it belongs to, and labels. Labels can be used for multiple purposes, some of which will will se later.
• The spec defines the properties of the resource. In the case of a pod, this includes the containers that the pod consists of. Note that we have to set the imagePullPolicy to IfNotPresent here. Otherwise, Kubernetes would try to pull the image, which does not work unless it has been pushed to a registry that the cluster can access.

We could now create the pod in the cluster with kubectl apply -f k8s-resources/pod.yaml. However, the more common approach is to create Kubernetes resources which control the creation of pods, such as, e.g., deployments, jobs, or daemon sets.

Define a deployment that controls the pods running an application¶

Here will will use a deployment, which ensures that a certain number of pods of a given type are running in the cluster.⁴ Having more than one pod of a certain type, e.g., a web service that answers incoming requests, can have a number of advantages:

• Distributing the incoming traffic over multiple pods can be beneficial because a single pod might not be able to handle a sufficient number of simultaneous requests.
• It helps to improve the reliability of the system: if a single pod terminates for whatever reason, the other pods can take over request handling immediately.

If a pod terminates, e.g., because of a crash in the application or because the node on which it is running is shut down, a new pod is created automatically, possibly on another node. Moreover, deployments can do other useful things concerning the life cycle of pods. For example, updates of the image version or other parts of the pod spec can be done with zero downtime. The deployment ensures that pods are created and destroyed dynamically at a controlled rate.

Note that this may not work well if a pod needs a lot of internal state to do its work. Kubernetes works best with stateless applications.⁵

Let's see what a deployment looks like:

The spec of the deployent describes the pods that it should control:

• replicas: 3 tells Kubernetes that three pods should be running at all times.
• The template describes what each pod should look like. Note that this object looks a lot like the definition of the plain pod that we saw earlier.
• The selector describes how the deployment finds the pods that it controls. The matchLabels are compared with the labels in the metadata of all pods for this purpose.

To create the deployment in the cluster, we use this command:⁶

If we look at the list of pods in our namespace now, we get this result:

There are three pods because we set the number of replicas to three in the definition of the deployment. Moreover, all pods have the status ContainerCreating, so they are not active yet.

Now we can either wait for a few seconds, or use kubectl rollout status to wait until all pods are running:

Now all pods are running. We can also see that each pod got its own IP address:

Accessing the pods via HTTP within the Kubernetes cluster¶

The IP addresses which Kubernetes assigns to the pods are not reachable from outside the cluster. We will consider how to use a Kubernetes service and an ingress to make our application accessible for the outside world in the next post.

For the time being, we can use those IP addresses to connect to the pods from other pods in the cluster though, as we will show next.

To get the IP address of one of the pods, we could look at the output of kubectl -n test get pod -o wide above. We could also parse the output with shell commands to assign the IP to a variable. However, kubectl also offers other output formats which are easier to process:

• -o json outputs a JSON object that contains all information about pods.
• -o jsonpath=... allows to specify a path to the information we need inside the JSON object, and prints just that.

By looking for IP addresses in the JSON output (which I will not show here because it is rather lengthy), we conclude that the IP address of the first pod in the list can be obtained like this:

To verify that the application can be reached at this address from within the cluster, we create a new pod, which serves as the client that accesses our application. This can be done like this:

We have sucessfully made an HTTP request to our application! Note that the hostname in the output is indeed the name of the first pod, whose IP address we have used here.

The options to kubectl run have the following meaning:

• --rm ensures that the pod is deleted after it exits, such that it no longer occupies resources in the cluster.
• --attach attaches to the process in the pod, such that we can see its output in the terminal.
• -q suppresses output from kubectl - we are only interested in output from wget.
• --image=busybox sets the image that the single container in the pod will use. All we need is a way to make HTTP requests from the pod, so we will use the busybox image, which contains a variant of wget.
• --restart Never prevents that Kubernetes restarts the pod after termination.⁷
• my-test-pod is the name of the pod. This can be any name that is not taken yet in the namespace. Note that we don't use a namespace argument here, so our pod will be created in the default namespace.

Using pod IPs to access our application has a number of downsides though. This post is already a bit long though, so we will discuss them and look at the solutions that Kubernetes provides for this issue, namely, services and ingresses, in a future post.

Deleting the cluster¶

For the time being, we are done with our experiments. We could stop the cluster now with k3d cluster stop and restart it later with k3d cluster start. Everything in the cluster is easy to restore though, so we will delete it:

Note that all resources are removed despite the warning about the failed deletion of the Docker volume.

Summary¶

In this post, we created a local single-node Kubernetes cluster with k3d. Moreover, we deployed a small self-made application in the cluster, and connected to this application via HTTP from within the cluster.

In the next post, we will investigate how to access the application from outside the cluster.

1. If we wanted more server or worker nodes, we could specify the desired numbers with the --agents and --servers options to k3d cluster create, respectively. ↩

2. There is one thing to keep in mind though: by default, Kubernetes will try to pull the image from a registry even if it is available on the node running the application, and therefore, the application will fail to run. We will see in a minute how this can be prevented by setting a suitable imagePullPolicy for the pods using the image. ↩

3. We have already created a Kubernetes resource on the command line with kubectl create: the namespace for our application, which we have created with kubectl create namespace test. We could also have achieved this using a YAML document with the content below: ↩

apiVersion: v1
kind: Namespace
metadata:
  name: test

4. To be precise, the deployment does not control the number of pods directly. It achieves this with a replica set which is a lower-level object that has controlling the number of pods as its sole purpose. The deployment adds other useful functionality, such as, e.g., updating image versions. ↩

5. It is possible to work with stateful applications in Kubernetes though. Stateful sets cah help with that. ↩

6. Note that kubectl apply is not only useful for creating deployments and other resources. The same command can be used to make changes to the resource. A common example would be to modify a deployment such that the image version is updated. ↩

7. Restarting terminated pods is the default behavior because most applications running in Kubernetes clusters are services which should always be up. ↩