Skip to content

K8s security best practices

General

Basics

  • update your kubernetes environment version early and othen
  • don't use the cluster amdin user for your daily work, treat it like root
  • if ou can, use a managed K8s services (AKS, EKS, GKE)
  • Check out the CIS Kubernetes Benchmark documents for more security best practises

RBAC Authorization

  • RBAC is a standard Kubernetes authorization mode, and can easily be used to authorize use of policies.
  • Allows to configure who can access what is the cluster
  • Enabled by default on the latestK8s
  • --authorization-mode=Node,RBAC
  • Has 4 different objects: Role, RoleBinding, ClusterRole and ClusterRoleBinding
  • https://kubernetes.io/docs/reference/access-authn-authz/rbac/ 
    kubectl api-versions | grep rbac 

    kubectl cluster-info dump | grep authorization-mode

TLS Everywhere

TLS should be enabled for every component that supports it to prevent traffic sniffing, verify the identity of the server, and (for mutual TLS) verify the identity of the client. more

Kubelet API

Kubelet expose HTTPS endpoints which grant powerful control over the node and containers. By default, Kubelets allow unauthenticated access to this API. Production clusters should enable Kubelet authentication and authorization.

  • https://127.0.0.1:10250/runningpods → list running pods
  • https://127.0.0.1:10250/exec → runs a command in a container
  • k8s-hacking-10250

Audit Logs

  • audit logs are not enabled by default
  • highly recommended enabling them for security and troubleshooting
  • need at least tow things: a log path and a policy file
  • set those up on the kube-apiserver configuration

ETCS

  • main data storage location for your cluster
  • all the cluster objects are saved here
  • use authentication and firewalls to restrict access to etcs
  • encrypt data in etcs (ecryption at rest)

Encryption in transit - default

ps -ef | grep etcd
# --cert-file=/etc/kubernetes/pkietcd/server.crt
# --key-file= /etc/kubernetes/pki/server,key

Encryption at rest - NOT default

  • By default, it's all stored in plain text
  • Create an EncryptionConfiguration object

Pods

Limit Resources

CPU & Memory 

apiVersion: v1
kind: ResourceQuota
metadata:
  name: pods-low
spec:
  hard:
    cpu: "5"
    memory: 10Gi
    pods: "10"

Network Policies

  • by default, all pods can communicate with any other pod in the cluster
  • make sure you create a proper network policy for your cluster
  • does the front-end pod really nned to talk to the DB pod ?
  • what if an attacker can access the pods on the kube-system namespace ?
  • Restrict access to database to only the Pods that require it
  • Specify access via labels
  • If you want to control traffic flow at the IP address or port level (OSI layer 3 or 4), then you might consider using Kubernetes NetworkPolicies for particular applications in your cluster. NetworkPolicies are an application-centric construct which allow you to specify how a pod is allowed to communicate with various network "entities" (we use the word "entity" here to avoid overloading the more common terms such as "endpoints" and "services", which have specific Kubernetes connotations) over the network. NetworkPolicies apply to a connection with a pod on one or both ends, and are not relevant to other connections.
  • https://editor.cilium.io
  • more 
    kind: NetworkPolicy
...
spec:
  ingress:
    - form:
      - podSelector:
          matchLabels:
            name: guestbook

Run as non-root

RunAsNotRoot - Least Privilege - Only give as much permission/privilege as is absolutely necessary - In the configuration file, the runAsUser field specifies that for any Containers in the Pod, all processes run with user ID 1000. - more 

spec:
  securityContext:
    runAsUser: 1000

Read only root filesystem

ReadOnlyRootFilesystem - Requires that containers must run with a read-only root filesystem (i.e. no writable layer) more 

spec:
  securityContext:
    readOnlyRootFilesystem: true

Privilege Escalation (no_new_privs)

AllowPrivilegeEscalation - Gates whether or not a user is allowed to set the security context of a container to allowPrivilegeEscalation=true. This defaults to allowed so as to not break setuid binaries. Setting it to false ensures that no child process of a container can gain more privileges than its parent. more 

spec:
  securityContext:
    allowPrivilegeEscalation: false

SecComp

  • filters a process's system calls 
    apiVersion: v1
kind: Pod
metadata:
  name: mypod
  annotation:
    seccomp.security.alpha.kubernetes.io/pod: runtime/defailt

AppArmor

  • uses program profiles to restrict the capabilities of individual programs 
    apiVersion: v1
kind: Pod
metadata:
  name: mypod
  annotation:
    apparmor.security.beta.kubernetes.io/hello: runtime/defailt
spec:
  containers:
    - name: hello

SELinux

  • applies security labels to objects and evaluates all security-relevant interactions via the security policy 
    apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
securityContext:
  seLinuxOptions:
    level: "s0:c123,c456"

Cluster

Open Policy Agent

In Kubernetes, Admission Controllers enforce semantic validation of objects during create, update, and delete operations. With OPA you can enforce custom policies on Kubernetes objects without recompiling or reconfiguring the Kubernetes API server.

Resources