🧸 Smol K8s Lab

smol-k8s-lab leverages Argo CD and slim k8s distributions like K3s to create production-like environments via a declarative workflow. Batteries and 🦑 included.

About

smol-k8s-lab's declarative workflow, CLI, and TUI enable rapid iteration in production-like environments with minimal costs for failure. This makes it ideal for proof-of-concepts, prototyping, and benchmarking Kubernetes applications and distributions! 💙

By default, smol-k8s-lab deploys Argo CD + Argo CD Appset Secret Plugin which enables Argo CD to securely manage your lab via files in open-source Git repos. We can optionally make heavy use of Bitwarden. Additionally, a customized dark-theme is provided for Argo CD's incredibly useful web-interface.

Consider viewing my very long walk through if you like video walk-throughs:

Here's the same video with captions.

Getting Started

Please see our Getting Started guide.

Under the hood

Note: this project is not officially affiliated with any of the below tooling or applications.

Supported k8s distributions

We always install the latest version of Kubernetes that is available from the distro's startup script.

Distro	Description
k3s	The certified Kubernetes distribution built for IoT & Edge computing
k3d	K3d is k3s in Docker 🐳. ⚠️ testing
KinD	kind is a tool for running local Kubernetes clusters using Docker container “nodes”. kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI.

We tend to test first on k3s first, then the other distros. k3d support coming soon.

Default Installed Applications

Version is the helm chart version, or manifest version. See the Default Applications tab for more info on each application.

Application	Description	Initialization Supported
metallb	Loadbalancer and IP Address pool manager for metal	✅
ingress-nginx	The ingress-nginx controller allows access to the cluster remotely, needed for web traffic	❌
cert-manager	For SSL/TLS certificates	✅
Argo CD	Gitops - Continuous Deployment	✅
Argo CD Appset Secret Plugin	Gitops - Continuous Deployment	✅
ESO	external-secrets-operator integrates external secret management systems like Bitwarden or GitLab	❌
Bitwarden ESO Provider	Bitwarden external-secrets-operator provider	✅
ZITADEL	An identity provider and OIDC provider to provide SSO	✅
Vouch	Vouch proxy allows you to secure web pages that lack authentication e.g. prometheus	✅
Prometheus Stack	Prometheus monitoring and logging stack using loki/promtail, alert manager, and grafana	✅

Minor Notes:

All Default Applications can be disabled through your ~/.config/smol-k8s-lab/config.yaml file, except Argo CD. You can still choose not to install it, but if not installed, smol-k8s-lab will only install: metallb, nginx-ingress, and cert-manager

Optionally Installed Applications

Application/Tool	Description	Initialization Supported
Ciliumdemo	Kubernetes netflow visualizer and policy editor	✅
Home Assistant	Home Assistant, a self hosted, at home IoT management solution.	✅
Kyvernoalpha	Kubernetes native policy management to enforce policies on k8s resources	❌
kepler	Kepler (Kubernetes Efficient Power Level Exporter) uses eBPF to probe energy-related system stats and exports them as Prometheus metrics.	✅
k8up	Backups operator using restic to backup to s3 endpoints	✅
k8tz	Timezone environment variable injector for pods and cronjobs	✅
Netmaker	Netmaker is a self hosted vpn management tool	✅
Nextcloud	Nextcloud is a self hosted file server	✅
Mastodon	Mastodon is a self hosted federated social media network	✅
matrix	Matrix is a self hosted chat platform	✅
minio	Self hosted S3 Object Store operator	✅
seaweedfs	Self hosted S3 Object Store	✅
k9s	Terminal based dashboard for kubernetes	✅

Status

Development

smol-k8s-lab is written in Python and built and published using Poetry. You can check out the pyproject.toml for the versions of each library we install below:

• bcrypt (to pass a password to argocd and automatically update your Bitwarden)
• rich (this is what makes all the pretty formatted text in logs and --help)
• textual (this is the framework used for writing the TUI)
• ruamel.yaml (to handle the k8s yamls and configs while maintaining comments)
• click (handles arguments for the CLI)

We also utilize the Bitwarden cli, for a password manager so you never have to see/know your Argo CD password.

Things we don't handle (yet)

1. Port Forwarding

   If you want to access an app outside of port forwarding to test, you'll need to make sure your app's ingress is setup correctly and then you'll need to setup your router to port forward 80->80 and 443->443 for your WAN. Then, setup DNS for your domain if you want the wider internet to access this remotely.

2. High-Availability

   HA cluster design with K3s requires etcd or another external key-value store such as PostgreSQL. Smol-K8s-Lab deploys k3s in a single-node configuration using SQLite which can be used for multi-node configurations but is not suitable for high-availability.