Merge pull request #97 from rdxmb/kubeadm-kubelet-config-file

configure kubeadm and kubelet through config-files

README.md

@@ -6,7 +6,7 @@ An Ansible Role that installs [Kubernetes](https://kubernetes.io) on Linux.
 
 ## Requirements
 
-Requires Docker; recommended role for Docker installation: `geerlingguy.docker`.
+Requires Docker or another [Container Runtime](https://kubernetes.io/docs/setup/production-environment/container-runtimes) ; recommended role for Docker installation: `geerlingguy.docker`.
 
 ## Role Variables
 
@@ -24,8 +24,8 @@ Available variables are listed below, along with default values (see `defaults/m
 
 Kubernetes packages to be installed on the server. You can either provide a list of package names, or set `name` and `state` to have more control over whether the package is `present`, `absent`, `latest`, etc.
 
-    kubernetes_version: '1.17'
-    kubernetes_version_rhel_package: '1.17.2'
+    kubernetes_version: '1.20'
+    kubernetes_version_rhel_package: '1.20.4'
 
 The minor version of Kubernetes to install. The plain `kubernetes_version` is used to pin an apt package version on Debian, and as the Kubernetes version passed into the `kubeadm init` command (see `kubernetes_version_kubeadm`). The `kubernetes_version_rhel_package` variable must be a specific Kubernetes release, and is used to pin the version on Red Hat / CentOS servers.
 
@@ -33,10 +33,44 @@ The minor version of Kubernetes to install. The plain `kubernetes_version` is us
 
 Whether the particular server will serve as a Kubernetes `master` (default) or `node`. The master will have `kubeadm init` run on it to intialize the entire K8s control plane, while `node`s will have `kubeadm join` run on them to join them to the `master`.
 
+### Variables to configure kubeadm and kubelet with `kubeadm init` through a config file (recommended)
+
+With this role, `kubeadm init` will be run with `--config <FILE>`.
+
+    kubernetes_kubeadm_kubelet_config_file_path: '/etc/kubernetes/kubeadm-kubelet-config.yaml'
+
+Path for `<FILE>`. If the directory does not exist, this role will create it.
+
+The following variables are parsed as options to <FILE>. To understand its syntax, see https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/kubelet-integration and https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/#config-file . The skeleton (`apiVersion`, `kind`) of the config file will be created by this role, so do not define them within the variables. (See `templates/kubeadm-kubelet-config.yaml`).
+
+    kubernetes_config_init_configuration:
+      localAPIEndpoint:
+        advertiseAddress: "{{ kubernetes_apiserver_advertise_address | default(ansible_default_ipv4.address, true) }}"
+
+Defines the options under `kind: InitConfiguration`. Including `kubernetes_apiserver_advertise_address` here is for backward-compatibilty to older versions of this role, where `kubernetes_apiserver_advertise_address` was used with a command-line-option.
+
+    kubernetes_config_cluster_configuration:
+      networking:
+        podSubnet: "{{ kubernetes_pod_network.cidr }}"
+      kubernetesVersion: "{{ kubernetes_version_kubeadm }}"
+
+Options under `kind: ClusterConfiguration`. Including `kubernetes_pod_network.cidr` and `kubernetes_version_kubeadm` here are for backward-compatibilty to older versions of this role, where they were used with command-line-options.
+
+    kubernetes_config_kubelet_configuration:
+      cgroupDriver: cgroupfs
+
+Options to configure kubelet on any nodes in your cluster through the `kubeadm init` process. To get the syntax of this options see https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file and https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/kubelet-integration.
+
+NOTE: This is the recommended way to do the kubelet-configuration. Most command-line-options are deprecated.
+
+NOTE: The recommended cgroupDriver depends on your [Container Runtime](https://kubernetes.io/docs/setup/production-environment/container-runtimes). When using this role with containerd instead of docker, this value should be changed to `systemd`.
+
+### Variables to configure kubeadm and kubelet through command-line-options
+
     kubernetes_kubelet_extra_args: ""
     kubernetes_kubelet_extra_args_config_file: /etc/default/kubelet
 
-Extra args to pass to `kubelet` during startup. E.g. to allow `kubelet` to start up even if there is swap is enabled on your server, set this to: `"--fail-swap-on=false"`. Or to specify the node-ip advertised by `kubelet`, set this to `"--node-ip={{ ansible_host }}"`.
+Extra args to pass to `kubelet` during startup. E.g. to allow `kubelet` to start up even if there is swap is enabled on your server, set this to: `"--fail-swap-on=false"`. Or to specify the node-ip advertised by `kubelet`, set this to `"--node-ip={{ ansible_host }}"`. *This is deprecated. Please use `kubernetes_config_kubelet_configuration` instead.*
 
     kubernetes_kubeadm_init_extra_opts: ""
 
@@ -46,6 +80,8 @@ Extra args to pass to `kubeadm init` during K8s control plane initialization. E.
 
 Extra args to pass to the generated `kubeadm join` command during K8s node initialization. E.g. to ignore certain preflight errors like swap being enabled, set this to: `--ignore-preflight-errors=Swap`
 
+### Additional variables
+
     kubernetes_allow_pods_on_master: true
 
 Whether to remove the taint that denies pods from being deployed to the Kubernetes master. If you have a single-node cluster, this should definitely be `True`. Otherwise, set to `False` if you want a dedicated Kubernetes master which doesn't run any other pods.

defaults/main.yml

@@ -9,15 +9,16 @@ kubernetes_packages:
   - name: kubernetes-cni
     state: present
 
-kubernetes_version: '1.19'
-kubernetes_version_rhel_package: '1.19.0'
+kubernetes_version: '1.20'
+kubernetes_version_rhel_package: '1.20.4'
 
 kubernetes_role: master
 
+# This is deprecated. Please use kubernetes_config_kubelet_configuration instead.
 kubernetes_kubelet_extra_args: ""
+
 kubernetes_kubeadm_init_extra_opts: ""
 kubernetes_join_command_extra_opts: ""
-
 kubernetes_allow_pods_on_master: true
 kubernetes_enable_web_ui: true
 kubernetes_web_ui_manifest_file: https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml
@@ -30,6 +31,23 @@ kubernetes_pod_network:
   # cni: 'calico'
   # cidr: '192.168.0.0/16'
 
+kubernetes_kubeadm_kubelet_config_file_path: '/etc/kubernetes/kubeadm-kubelet-config.yaml'
+kubernetes_config_kubelet_configuration:
+  cgroupDriver: "cgroupfs"
+
+kubernetes_config_init_configuration:
+  localAPIEndpoint:
+    advertiseAddress: "{{ kubernetes_apiserver_advertise_address | default(ansible_default_ipv4.address, true) }}"
+# if you use the next lines, remove the command line argument below
+# nodeRegistration:
+#    ignorePreflightErrors:
+#      - all
+
+kubernetes_config_cluster_configuration:
+  networking:
+    podSubnet: "{{ kubernetes_pod_network.cidr }}"
+  kubernetesVersion: "{{ kubernetes_version_kubeadm }}"
+
 kubernetes_apiserver_advertise_address: ''
 kubernetes_version_kubeadm: 'stable-{{ kubernetes_version }}'
 kubernetes_ignore_preflight_errors: 'all'

tasks/kubelet-setup.yml

@@ -1,35 +1,42 @@
 ---
-- name: Check for existence of kubelet environment file.
+
+# ---- DEPRECATED ----------------
+#
+# Most of the kubernetes_kubelet_extra_args are deprecated. See https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet for details.
+# Use the kubernetes_kubelet_config variable instead, which will be used to create the kubelet config file.
+
+- name: Check for existence of kubelet environment file. (deprecated)
   stat:
     path: '{{ kubelet_environment_file_path }}'
   register: kubelet_environment_file
 
-- name: Set facts for KUBELET_EXTRA_ARGS task if environment file exists.
+- name: Set facts for KUBELET_EXTRA_ARGS task if environment file exists. (deprecated)
   set_fact:
     kubelet_args_path: '{{ kubelet_environment_file_path }}'
     kubelet_args_line: "{{ 'KUBELET_EXTRA_ARGS=' + kubernetes_kubelet_extra_args }}"
     kubelet_args_regexp: '^KUBELET_EXTRA_ARGS='
   when: kubelet_environment_file.stat.exists
 
-- name: Set facts for KUBELET_EXTRA_ARGS task if environment file doesn't exist.
+- name: Set facts for KUBELET_EXTRA_ARGS task if environment file doesn't exist. (deprecated)
   set_fact:
     kubelet_args_path: '/etc/systemd/system/kubelet.service.d/10-kubeadm.conf'
     kubelet_args_line: "{{ 'Environment=\"KUBELET_EXTRA_ARGS=' + kubernetes_kubelet_extra_args + '\"' }}"
     kubelet_args_regexp: '^Environment="KUBELET_EXTRA_ARGS='
   when: not kubelet_environment_file.stat.exists
 
-- name: Configure KUBELET_EXTRA_ARGS.
+- name: Configure KUBELET_EXTRA_ARGS. (deprecated)
   lineinfile:
     path: '{{ kubelet_args_path }}'
     line: '{{ kubelet_args_line }}'
     regexp: '{{ kubelet_args_regexp }}'
     state: present
     mode: 0644
-  register: kubelet_config_file
+  register: kubelet_extra_args
+  when: kubernetes_kubelet_extra_args|length > 0
 
-- name: Reload systemd unit if args were changed.
+- name: Reload systemd unit if args were changed. (deprecated)
   systemd:
     state: restarted
     daemon_reload: true
     name: kubelet
-  when: kubelet_config_file is changed
+  when: kubelet_extra_args is changed

tasks/main.yml

@@ -20,7 +20,8 @@
 
 - include_tasks: sysctl-setup.yml
 
-- include_tasks: kubelet-setup.yml
+- include_tasks: kubelet-setup.yml  # deprecated
+  when: kubernetes_kubelet_extra_args|length > 0
 
 - name: Ensure kubelet is started and enabled at boot.
   service:

tasks/master-setup.yml

@@ -1,14 +1,30 @@
 ---
-- name: Initialize Kubernetes master with kubeadm init.
+- name: Create the directory for the kubernetes_config_file
+  file:
+    path: "{{ kubernetes_kubeadm_kubelet_config_file_path | dirname }}"
+    state: directory
+
+- name: Deploy the config-file for kubeadm and kubelet
+  template:
+    src: "kubeadm-kubelet-config.j2"
+    dest: "{{ kubernetes_kubeadm_kubelet_config_file_path }}"
+
+- name: Initialize Kubernetes master with kubeadm init
   command: >
     kubeadm init
-    --pod-network-cidr={{ kubernetes_pod_network.cidr }}
-    --apiserver-advertise-address={{ kubernetes_apiserver_advertise_address | default(ansible_default_ipv4.address, true) }}
-    --kubernetes-version {{ kubernetes_version_kubeadm }}
+    --config {{ kubernetes_kubeadm_kubelet_config_file_path }}
+    {{ kubernetes_kubeadm_init_extra_opts }}
+  register: kubeadmin_init
+  when: (not kubernetes_init_stat.stat.exists) and (kubernetes_ignore_preflight_errors is not defined)
+
+- name: Initialize Kubernetes master with kubeadm init and ignore_preflight_errors
+  command: >
+    kubeadm init
+    --config {{ kubernetes_kubeadm_kubelet_config_file_path }}
     --ignore-preflight-errors={{ kubernetes_ignore_preflight_errors }}
     {{ kubernetes_kubeadm_init_extra_opts }}
   register: kubeadmin_init
-  when: not kubernetes_init_stat.stat.exists
+  when: (not kubernetes_init_stat.stat.exists) and (kubernetes_ignore_preflight_errors is defined)
 
 - name: Print the init output to screen.
   debug:

templates/kubeadm-kubelet-config.j2

@@ -0,0 +1,14 @@
+---
+apiVersion: kubeadm.k8s.io/v1beta2
+kind: InitConfiguration
+{{ kubernetes_config_init_configuration | to_nice_yaml }}
+---
+kind: ClusterConfiguration
+apiVersion: kubeadm.k8s.io/v1beta2
+{{ kubernetes_config_cluster_configuration | to_nice_yaml }}
+---
+{% if kubernetes_config_kubelet_configuration|length > 0 %}
+apiVersion: kubelet.config.k8s.io/v1beta1
+kind: KubeletConfiguration
+{{ kubernetes_config_kubelet_configuration | to_nice_yaml }}
+{% endif %}