We will attempt to build a kubernetes cluster from scratch in this blog. The whole deployment runs on a VMware workstation Linux on an old Centos laptop I got from Ebay very cheap so we can easily run most kube workloads and few AI models than spinning up my rack servers.

A quick way to install kubernetes is by downloading Rancher desktop so on everything on one machine, control plane and workloads together. Thou simple to set up but has no resilience, no scheduling across nodes, not how real clusters work so we will use a 3 node setup to mimic a production setup.

https://documentation.suse.com/cloudnative/rancher-manager/latest/en/installation-and-upgrade/installation-and-upgrade.html

Lab Hardware

VMware Workstation vmnet8 (NAT) configured as 172.16.77.0/24.

With Kuberenetes in place, the CP acts as the brain controlling all the pod lifecycle such as, scaling, upgrading, snapshots etc.

The configuration of each of my VM’s is as below

Name: k3s-cp-01/k3s-worker-01/k3s-worker-02
vCPU: 2
RAM: 4096MB
Disk: 40GB
Network: Bridged

To set static IP, sudo nano /etc/netplan/50-cloud-init.yaml and run sudo netplan apply to apply the netconfig

network:
  version: 2
  ethernets:
    ens33:
      dhcp4: no
      addresses:
        - 172.16.77.12/24
      routes:
        - to: default
          via: 172.16.77.2
      nameservers:
        addresses: [8.8.8.8, 8.8.4.4]

We are going to need a fair bit of space here because I’m going to setup persistent volumes (pvc) so we will just extend the root disk and extend the LV and grow the filesystem

sudo lvextend -l +100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv
sudo resize2fs /dev/mapper/ubuntu--vg-ubuntu--lv

Install control plane (k3s-cp-01)

Ok, First step it to download the K3s install script from the official K3s website and set one of our VM as the control VM and its a quick script from https://docs.k3s.io/quick-start

curl -sfL https://get.k3s.io | sh -

Run the get pods command to see what all are the default pods and namespaces

To configure worker nodes or to form a cluster, we need a token to be generated from the control plane VM

 sudo cat /var/lib/rancher/k3s/server/node-token

Install worker nodes (k3s-worker-01 and 02)

Lets join the worker node 02 to the K8s cluster

curl -sfL https://get.k3s.io | K3S_URL=https://172.16.77.10:6443 K3S_TOKEN=K102b93136dfe241806e96cfc99b14b2af08988436d31c66dd87e0ec27d290a96e8::server:99e09011c8b012984053cc0612489a75 sh -

Repeat the same for worker node 01

Check if kubernetes is running

sudo systemctl status k3s-agent

Run the k get nodes and we can see our kubernetes cluster now with 1 control VM and two worker nodes.

k get nodes 

Lets just label it nicely

kubectl label node k3s-worker-01 node-role.kubernetes.io/worker=worker
kubectl label node k3s-worker-02 node-role.kubernetes.io/worker=worker

Few routing rules if required so you can connect from outside world as VMNET8 is our NAT

Masquerade traffic going to vmnet8

sudo iptables -t nat -A POSTROUTING -o vmnet8 -j MASQUERADE
sudo iptables -A FORWARD -i vmnet8 -j ACCEPT
sudo iptables -A FORWARD -o vmnet8 -j ACCEPT

Add IP forwarding on /etc/sysctl.conf

net.ipv4.ip_forward=1

Packages and Configuration for a K8s envt

On the control plane , we will just some packages and do some additional configurations

sudo apt install tmux vim bash-completion -y

Then install k9s so we can get a UI for our cluster

wget https://github.com/derailed/k9s/releases/latest/download/k9s_linux_amd64.deb
sudo apt install ./k9s_linux_amd64.deb
rm k9s_linux_amd64.deb

Then set up your ~/.bashrc with the content above.

alias k='kubectl'
source /etc/bash_completion
source <(kubectl completion bash)
complete -o default -F __start_kubectl k
source <(helm completion bash)
source /etc/bash_completion
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
alias kubectl=kubecolor

Then set up your~/.vimrc with the content above.

vim uses " for comments not #.

" Enable filetype detection and plugins
filetype plugin on

" Enable filetype-based indentation
filetype indent on

" Set tab width to 2 spaces
set tabstop=2

" Set soft tab to 2 spaces
set softtabstop=2

" Set indent width to 2 spaces
set shiftwidth=2

" Use spaces instead of tabs
set expandtab

" Highlight trailing whitespace as error
autocmd BufRead,BufNewFile * match Error /\s\+$/

" Enable auto-indentation
set autoindent

" Enable syntax highlighting
syntax on

" Fix backspace behaviour
set backspace=indent,eol,start

When we now ssh into cp-01 and edit any YAML file with vim, indentation is automatically 2 spaces, tabs convert to spaces, trailing whitespace shows as red error.