Bootstrap K3s Cluster With Ignition

In this walkthrough, we configure our FCOS powered raspberry pis,
to bootstrap K3s at each reboot effectively ensurely durability and reliability.
Each reboot will get nodes back at their desired state, allowing us to separate
the application leval from the data since the configuration is fully declarative.

The issue with K3s is the embedded datastore :
the power of this configuration is that each reboot nodes are fresh new,
but K3s has no possible way to add new master nodes if deployed with embedded sqlite.
The solution is an external datastore, we can use a mysql container.
Keep in mind that compatibility is up to certain version, more on this on the documentation,
The 5.7.9 and it works for sure : Rancher K3s

A docker container can be configured with the following parameters :

MYSQL_VERSION : 5.7.9debian8
MYSQL_USER:  {{ USERNAME }}
MYSQL_PASSWORD: {{ PASSWORD }}
MYSQL_DATABASE: {{ DATABASE }}
MYSQL_ROOT_PASSWORD: {{ ROOT_PASSWORD }}

We will need the parameters to to write the ignition config.
First we need to download K3s, we are lucky is a single binary !
Important note, we need to verify the integrity of our binary,
simply use this command to extract the 512bit hash :

curl -O https://github.com/k3s-io/k3s/releases/download/v1.24.7%2Bk3s1/k3s-arm64
sha512sum  k3sarm

We need to store the hash in the igntion config file :

storage:
  files:

    - path: "/usr/local/bin/k3s"
      contents:
        source: https://github.com/k3s-io/k3s/releases/download/{{ K3S_VERSION }}%2Bk3s2/k3s
        verification:
          # Insert the generated hash
          hash: sha512-{{ HASH_512 }}
      mode: 0755
      user:
        name: root
      group:
        name: root

    - path: /usr/local/etc/k3s/token
      contents:
        source: data:text/plain;charset=utf-8;base64,YWQ4NDE1NTUtZGMyOS00YTZkLTk1NjQtN2E5NzQ0NGUwM2Ux
      mode: 0400
      user:
        name: root

The k3s binary and its token is downloaded now we need to run it as a systemd service.
Follows the config for the master nodes :

systemd:
  units:
    - name: k3s.service
      enabled: true
      contents: |
        [Unit]
        Description=Lightweight Kubernetes
        Documentation=https://k3s.io
        Wants=network-online.target
        After=network-online.target
        After=settimezone.service

        [Install]
        WantedBy=multi-user.target

        [Service]
        Type=notify
        KillMode=process
        Delegate=yes
        LimitNOFILE=1048576
        LimitNPROC=infinity
        LimitCORE=infinity
        TasksMax=infinity
        TimeoutStartSec=0
        Restart=always
        RestartSec=5s
        ExecStartPre=-/sbin/modprobe br_netfilter
        ExecStartPre=-/sbin/modprobe overlay
        ExecStart=/usr/local/bin/k3s \
          'server' \
          '--disable-selinux' \
          '--token-file=/usr/local/etc/k3s/token' \
          '--cluster-cidr=10.10.0.0/16' \
          '--service-cidr=10.20.0.0/16' \
          '--token=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'\
          '--datastore-endpoint=mysql://{{ USERNAME }}:{{ PASSWORD }}@tcp(10.10.130.100:12000)/{{ DATABASE }}'
Config for the worker nodes :

    - name: k3s.service
      enabled: true
      contents: |
        [Unit]
        Description=Lightweight Kubernetes
        Documentation=https://k3s.io
        Wants=network-online.target
        After=network-online.target
        After=settimezone.service

        [Install]
        WantedBy=multi-user.target

        [Service]
        Type=notify
        KillMode=process
        Delegate=yes
        LimitNOFILE=1048576
        LimitNPROC=infinity
        LimitCORE=infinity
        TasksMax=infinity
        TimeoutStartSec=0
        Restart=always
        RestartSec=5s
        ExecStartPre=-/sbin/modprobe br_netfilter
        ExecStartPre=-/sbin/modprobe overlay
        ExecStart=/usr/local/bin/k3s \
          'agent' \
          '--server=https://{{ API_SERVER }}:6443' \
          '--token=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'\
          '--node-name=worker' \
          '--with-node-id'

Generate the ignition configs with butane, and host it at the http-boot server.
Congratulations on your highly available rpi k3s cluster up and running!

Comments

Subscribe to our Newsletter