how to enable resource EphemeralContainers in Rancher kubernetes

Question

I want to enable EphemeralContainers by setting this --feature-gates=EphemeralContainers=true in below files based on the steps given in this link https://www.shogan.co.uk/kubernetes/enabling-and-using-ephemeral-containers-on-kubernetes-1-16/

• /etc/kubernetes/manifests/kube-apiserver.yaml
• /etc/kubernetes/manifests/kube-scheduler.yaml

But there is no such files under etc/kubernetes , Kubernetes is setup through Rancher. Any idea where i can set the value in rancher k8

Answer 1

You need to pass extra_args by editing cluster.yml.

Here is a sample snippet:

services:
    kube-api:
      service_node_port_range: 30000-32767
      pod_security_policy: false
      extra_args:
        feature-gates: "EphemeralContainers=true"
    kube-scheduler:
      extra_args:
        feature-gates: "EphemeralContainers=true"

Reference:

• https://rancher.com/docs/rke/latest/en/example-yamls/

Answer 2

Rancher may have similar setting. Try with your own cluster. For bare metal kubernetes add the arguement EphemeralContainers=true to flag --feature-gates for kube-apiserver/kube-scheduler/kubelet

your probably need to change following config files

on each node, cat /var/lib/kubelet/kubeadm-flags.env

KUBELET_KUBEADM_ARGS="--cgroup-driver=systemd --network-plugin=cni --pod-infra-container-image=k8s.gcr.io/pause:3.1 --feature-gates=EphemeralContainers=true"

on master node: cat /etc/kubernetes/manifests/kube-apiserver.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    component: kube-apiserver
    tier: control-plane
  name: kube-apiserver
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-apiserver
    - --advertise-address=10.252.90.44
    - --allow-privileged=true
    - --authorization-mode=Node,RBAC
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --enable-admission-plugins=NodeRestriction
    - --enable-bootstrap-token-auth=true
    - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
    - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
    - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
    - --etcd-servers=https://127.0.0.1:2379
    - --feature-gates=TTLAfterFinished=true,EphemeralContainers=true #change me
    - --insecure-port=0
    - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
    - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
    - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
    - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt
    - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key
    - --requestheader-allowed-names=front-proxy-client
    - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
    - --requestheader-extra-headers-prefix=X-Remote-Extra-
    - --requestheader-group-headers=X-Remote-Group
    - --requestheader-username-headers=X-Remote-User
    - --secure-port=6443
    - --service-account-key-file=/etc/kubernetes/pki/sa.pub
    - --service-cluster-ip-range=10.96.0.0/12
    - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
    - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
    - --cors-allowed-origins=http://localhost:3000,http://127.0.0.1:3000
    image: k8s.gcr.io/kube-apiserver:v1.16.0
    imagePullPolicy: IfNotPresent
    ...

and cat /etc/kubernetes/manifests/kube-scheduler.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    component: kube-scheduler
    tier: control-plane
  name: kube-scheduler
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-scheduler
    - --authentication-kubeconfig=/etc/kubernetes/scheduler.conf
    - --authorization-kubeconfig=/etc/kubernetes/scheduler.conf
    - --bind-address=127.0.0.1
    - --feature-gates=TTLAfterFinished=true,EphemeralContainers=true #change me
    - --kubeconfig=/etc/kubernetes/scheduler.conf
    - --leader-elect=true
    image: k8s.gcr.io/kube-scheduler:v1.16.0
    imagePullPolicy: IfNotPresent
    ...

Answer 3

For me in rancher v2.5.9, I had to tweak the settings slightly to get it to work. Use scheduler instead of kube-scheduler, and you need to add the extra_args to the kubelet stanza as well.

      services:
        kubelet:
          extra_args:
            feature-gates: "EphemeralContainers=true"
        kube-api:
          extra_args:
            feature-gates: "EphemeralContainers=true"
        scheduler:
          extra_args:
            feature-gates: "EphemeralContainers=true"