#selfhosting #homelab #kubernetes

This is in no way a guide on correct deployment of such solution. I'm learning this on the go, and my setup might include mistakes I'm not aware of that might make it insecure and dangerous in production environment. If you spot such mistakes, feel free to correct me.

Goals

The main goal is to build a reliable k8s cluster using industry standard technologies like RHEL, Ansible and set everything up in a declarative way, so bootstrapping new nodes or rebuilding the whole cluster is reliable and requires as little manual interaction as possible.

At the end, I'd like to use this cluster to deploy and run some services I want in our house and to have environment to locally test my projects.

I also want to provide infra to my DevOps housemate and friend, who wants a test environment for learning.

Hardware

One of most common hardware solutions in this space are refurbished thin clients and similar miniature PCs or ARM boards like Raspberry PI.

Another interesting solution is one of many Intel N100 and similar PCs that popped up lately.

My deciding factors were: – high enough performance to run everything comfortably – price – power consumption

Refurbished thin clients don't generally excel in power consumption (although it's not terrible) and Raspberry Pi and similar lack in performance. I ended up going for Intel N100, specifically Firebat T8 Pro Plus. It's reasonably priced, small, and it has bat in its name. I like bats, bats are cute. source of bat photo: Screenshot from Batzilla the bat video

Software

RHEL I wanted to use a Linux distribution that is industry standard and possible to find in professional setting. Candidates were NixOS, Debian, Suse, CentOS, Rocky, Alpine and RHEL.

As much as NixOS is my preferred OS, its steep learning curve would certainly not benefit my friend, and I already run NixOS for my public facing infrastructure like this blog, so it wouldn't teach me much more.

I don't like Debian, and I'm used to RHEL based ecosystems, so the final decision was between CentOS, Rocky, Alpine and RHEL. And since RedHat provides a reasonable number of free licenses for similar purposes, I just went with it, because it's a chance to try it.

Ansible I don't think there's much to say here. Ansible is at home with RHEL, and while I used SaltStack in past, it and other competing projects are not as widespread anymore. Notable exception is Terraform / OpenTofu, but its strength lies more in infrastructure deployment and if I used it, it would likely still be in tandem with Ansible for configuration management.

RKE2 The choice of K8S distribution is a first step out of RedHat ecosystem I'm making. Indeed, OKD would be more at home there, but I wanted a cleaner k8s experience while keeping some convenience of prepared solution.

ArgoCD From what I found, two biggest players in CD space are ArgoCD and Flux. I went mostly with feelings here, ArgoCD just looked cooler, although Flux might be overall a better solution for small deployments like this.

Longhorn Distributed block storage for my persistent volume needs. There are other competing solutions in this space, among the well known ones GlusterFS and Ceph, but I went with Longhorn for its ease of use and deployment.

Bind 9 We need to be able to easily access deployments on our cluster from our network. Bind 9 in combination with External DNS as authoritative server and .internal TLD solve this.

In my next post, I will go into details of my Ansible setup to configure and spin up RKE2 cluster :)