k0s 클러스터 구성

GitHub - k0sproject/k0s: k0s - The Zero Friction Kubernetes

k0s - The Zero Friction Kubernetes. Contribute to k0sproject/k0s development by creating an account on GitHub.

GitHubk0sproject

What is k0s?

• Docker Enterprise와 k8s lens를 인수한 Mirantis에서 만든 초경량 배포판

(Prerequisite) Node

• k0sctl (latest, 24-02-21, v0.17.4)
• VM 3EA
  • Controller Plane 1EA
  • Worker 2EA

• ssh-copy-id required

taking@kubernetes-01:~$ ssh-copy-id taking@kubernetes-01
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/taking/.ssh/id_rsa.pub"
The authenticity of host 'kubernetes-01 (127.0.1.1)' can't be established.
ED25519 key fingerprint is SHA256:S64c0qlwsvTKhAJDrad5ehaj++ZQwUFAcqO2wNGsf2M.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
taking@kubernetes-01's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'taking@kubernetes-01'"
and check to make sure that only the key(s) you wanted were added.

taking@kubernetes-01:~$ ssh-copy-id taking@kubernetes-02
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/taking/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
taking@kubernetes-02's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'taking@kubernetes-02'"
and check to make sure that only the key(s) you wanted were added.

taking@kubernetes-01:~$ ssh-copy-id taking@kubernetes-03
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/taking/.ssh/id_rsa.pub"
The authenticity of host 'kubernetes-03 (192.168.0.142)' can't be established.
ED25519 key fingerprint is SHA256:Jy80ILXXk3tOS4KxNk/DbDCl8Jqifxg5JqBt7ACSTiE.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
taking@kubernetes-03's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'taking@kubernetes-03'"
and check to make sure that only the key(s) you wanted were added.

k0s Install

curl get.k0s.sh | sudo sh > k0sctl

taking@kubernetes-01:~$ k0sctl version
version: v0.17.4
commit: 372a589

cat <<'EOF'> ./k0sctl.yaml
apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
  name: k0s-cluster
spec:
  hosts:
  - hostname: kubernetes-01
    role: controller+worker
    ssh:
      address: 192.168.0.140 # replace with the controller's IP address
      user: taking
      port: 22
      #keyPath: ~/.ssh/id_rsa
  - hostname: kubernetes-02
    role: controller+worker
    ssh:
      address: 192.168.0.141 # replace with the controller's IP address
      user: taking
      port: 22
      #keyPath: ~/.ssh/id_rsa
  - hostname: kubernetes-03
    role: worker
    ssh:
      address: 192.168.0.142 # replace with the worker's IP address
      user: taking
      port: 22
      #keyPath: ~/.ssh/id_rsa
  k0s:
    version: v1.29.1+k0s.1
    config:
      apiVersion: k0s.k0sproject.io/v1beta1
      kind: ClusterConfig
      metadata:
        name: my-k0s-cluster
      spec:
        images:
          calico:
            cni:
              image: calico/cni
              version: v3.27.0
EOF

k0sctl apply --config k0sctl.yaml

taking@kubernetes-01:~$ k0sctl apply --config k0sctl.yaml

⠀⣿⣿⡇⠀⠀⢀⣴⣾⣿⠟⠁⢸⣿⣿⣿⣿⣿⣿⣿⡿⠛⠁⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀█████████ █████████ ███
⠀⣿⣿⡇⣠⣶⣿⡿⠋⠀⠀⠀⢸⣿⡇⠀⠀⠀⣠⠀⠀⢀⣠⡆⢸⣿⣿⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀███          ███    ███
⠀⣿⣿⣿⣿⣟⠋⠀⠀⠀⠀⠀⢸⣿⡇⠀⢰⣾⣿⠀⠀⣿⣿⡇⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀███          ███    ███
⠀⣿⣿⡏⠻⣿⣷⣤⡀⠀⠀⠀⠸⠛⠁⠀⠸⠋⠁⠀⠀⣿⣿⡇⠈⠉⠉⠉⠉⠉⠉⠉⠉⢹⣿⣿⠀███          ███    ███
⠀⣿⣿⡇⠀⠀⠙⢿⣿⣦⣀⠀⠀⠀⣠⣶⣶⣶⣶⣶⣶⣿⣿⡇⢰⣶⣶⣶⣶⣶⣶⣶⣶⣾⣿⣿⠀█████████    ███    ██████████
k0sctl v0.17.4 Copyright 2023, k0sctl authors.
Anonymized telemetry of usage will be sent to the authors.
By continuing to use k0sctl you agree to these terms:
https://k0sproject.io/licenses/eula
INFO ==> Running phase: Connect to hosts 
INFO [ssh] 192.168.0.140:22: connected            
INFO [ssh] 192.168.0.142:22: connected            
INFO [ssh] 192.168.0.141:22: connected            
INFO ==> Running phase: Detect host operating systems 
INFO [ssh] 192.168.0.140:22: is running Ubuntu 22.04.3 LTS 
INFO [ssh] 192.168.0.142:22: is running Ubuntu 22.04.3 LTS 
INFO [ssh] 192.168.0.141:22: is running Ubuntu 22.04.3 LTS 
INFO ==> Running phase: Acquire exclusive host lock 
INFO ==> Running phase: Prepare hosts    
INFO ==> Running phase: Gather host facts 
INFO [ssh] 192.168.0.142:22: using kubernetes-03 from configuration as hostname 
INFO [ssh] 192.168.0.140:22: using kubernetes-01 from configuration as hostname 
INFO [ssh] 192.168.0.141:22: using kubernetes-02 from configuration as hostname 
INFO [ssh] 192.168.0.142:22: discovered eth0 as private interface 
INFO [ssh] 192.168.0.140:22: discovered eth0 as private interface 
INFO [ssh] 192.168.0.141:22: discovered eth0 as private interface 
INFO ==> Running phase: Validate hosts   
INFO ==> Running phase: Gather k0s facts 
INFO ==> Running phase: Validate facts   
INFO [ssh] 192.168.0.140:22: validating configuration 
INFO [ssh] 192.168.0.141:22: validating configuration 
INFO ==> Running phase: Configure k0s    
INFO [ssh] 192.168.0.141:22: installing new configuration 
INFO [ssh] 192.168.0.140:22: installing new configuration 
INFO ==> Running phase: Initialize the k0s cluster 
INFO [ssh] 192.168.0.140:22: installing k0s controller 
INFO [ssh] 192.168.0.140:22: waiting for the k0s service to start 
INFO [ssh] 192.168.0.140:22: waiting for kubernetes api to respond 
INFO ==> Running phase: Install controllers 
INFO [ssh] 192.168.0.141:22: validating api connection to https://192.168.0.140:6443 
INFO [ssh] 192.168.0.140:22: generating token     
INFO [ssh] 192.168.0.141:22: writing join token   
INFO [ssh] 192.168.0.141:22: installing k0s controller 
INFO [ssh] 192.168.0.141:22: starting service     
INFO [ssh] 192.168.0.141:22: waiting for the k0s service to start 
INFO [ssh] 192.168.0.141:22: waiting for kubernetes api to respond 
INFO ==> Running phase: Install workers  
INFO [ssh] 192.168.0.142:22: validating api connection to https://192.168.0.140:6443 
INFO [ssh] 192.168.0.140:22: generating a join token for worker 1 
INFO [ssh] 192.168.0.142:22: writing join token   
INFO [ssh] 192.168.0.142:22: installing k0s worker 
INFO [ssh] 192.168.0.142:22: starting service     
INFO [ssh] 192.168.0.142:22: waiting for node to become ready 
INFO ==> Running phase: Release exclusive host lock 
INFO ==> Running phase: Disconnect from hosts 
INFO ==> Finished in 1m0s                
INFO k0s cluster version v1.29.1+k0s.1 is now installed 
INFO Tip: To access the cluster you can now fetch the admin kubeconfig using: 
INFO      k0sctl kubeconfig

k0s settings

sudo mkdir -p $HOME/.kube
sudo chown -f -R $USER $HOME/.kube
sudo su - $USER

k0sctl kubeconfig > $HOME/.kube/config

sudo tee /usr/local/bin/kubectl <<'EOF'
#!/bin/bash
exec sudo /usr/local/bin/k0s kubectl $@
EOF

sudo chmod a+x /usr/local/bin/kubectl

kubectl get nodes -o wide -n show-labels

k0s uninstall

• 모든 노드에 다음 명령어를 실행하여, k0s 서비스를 종료 및 리셋한다.

k0s stop
k0s reset

k0sctl reset --config k0sctl.yaml

taking@kubernetes-01:~$ k0sctl reset --config k0sctl.yaml
k0sctl v0.17.4 Copyright 2023, k0sctl authors.
Anonymized telemetry of usage will be sent to the authors.
By continuing to use k0sctl you agree to these terms:
https://k0sproject.io/licenses/eula
? Going to reset all of the hosts, which will destroy all configuration and data, Are you sure? Yes
INFO ==> Running phase: Connect to hosts 
INFO [ssh] 192.168.0.140:22: connected            
INFO [ssh] 192.168.0.142:22: connected            
INFO [ssh] 192.168.0.141:22: connected            
INFO ==> Running phase: Detect host operating systems 
INFO [ssh] 192.168.0.142:22: is running Ubuntu 22.04.3 LTS 
INFO [ssh] 192.168.0.141:22: is running Ubuntu 22.04.3 LTS 
INFO [ssh] 192.168.0.140:22: is running Ubuntu 22.04.3 LTS 
INFO ==> Running phase: Acquire exclusive host lock 
INFO ==> Running phase: Prepare hosts    
INFO ==> Running phase: Gather k0s facts 
INFO [ssh] 192.168.0.141:22: found existing configuration 
INFO [ssh] 192.168.0.140:22: found existing configuration 
INFO ==> Running phase: Reset workers    
INFO [ssh] 192.168.0.142:22: reset                
INFO ==> Running phase: Reset controllers 
INFO [ssh] 192.168.0.140:22: reset                
INFO [ssh] 192.168.0.141:22: reset                
INFO ==> Running phase: Reset leader     
INFO [ssh] 192.168.0.140:22: reset                
INFO ==> Running phase: Release exclusive host lock 
INFO ==> Running phase: Disconnect from hosts 
INFO ==> Finished in 1s                  
taking@kubernetes-01:~$ 

(trouble shoot)

k0s uninstall

• 각 노드별로 /opt/cni 폴더를 삭제해줘야 한다.

INFO * Running clean-up for phase: Acquire exclusive host lock 
INFO ==> Apply failed                    
FATA apply failed - log file saved to /home/taking/.cache/k0sctl/k0sctl.log: failed on 3 hosts:
 - [ssh] 192.168.0.141:22: download k0s: command failed: client exec: ssh session wait: Process exited with status 22
 - [ssh] 192.168.0.142:22: download k0s: command failed: client exec: ssh session wait: Process exited with status 22
 - [ssh] 192.168.0.140:22: download k0s: command failed: client exec: ssh session wait: Process exited with status 22

k0s permission denied

taking@kubernetes-01:~$ k0s kubectl get nodes
error: error loading config file "/var/lib/k0s/pki/admin.conf": open /var/lib/k0s/pki/admin.conf: permission denied