K8s security best practices

General

Basics

update your kubernetes environment version early and othen
don't use the cluster amdin user for your daily work, treat it like root
if ou can, use a managed K8s services (AKS, EKS, GKE)
Check out the CIS Kubernetes Benchmark documents for more security best practises

RBAC Authorization

RBAC is a standard Kubernetes authorization mode, and can easily be used to authorize use of policies.
Allows to configure who can access what is the cluster
Enabled by default on the latestK8s
--authorization-mode=Node,RBAC
Has 4 different objects: Role, RoleBinding, ClusterRole and ClusterRoleBinding

https://kubernetes.io/docs/reference/access-authn-authz/rbac/

kubectl api-versions | grep rbac

kubectl cluster-info dump | grep authorization-mode

TLS Everywhere

TLS should be enabled for every component that supports it to prevent traffic sniffing, verify the identity of the server, and (for mutual TLS) verify the identity of the client. more

Kubelet API

Kubelet expose HTTPS endpoints which grant powerful control over the node and containers. By default, Kubelets allow unauthenticated access to this API. Production clusters should enable Kubelet authentication and authorization.

https://127.0.0.1:10250/runningpods → list running pods
https://127.0.0.1:10250/exec → runs a command in a container
K8S hacking 10250

Audit Logs

audit logs are not enabled by default
highly recommended enabling them for security and troubleshooting
need at least tow things: a log path and a policy file
set those up on the kube-apiserver configuration

ETCS

main data storage location for your cluster
all the cluster objects are saved here
use authentication and firewalls to restrict access to etcs
encrypt data in etcs (ecryption at rest)

Encryption in transit - default

ps -ef | grep etcd
# --cert-file=/etc/kubernetes/pkietcd/server.crt
# --key-file= /etc/kubernetes/pki/server,key

Encryption at rest - NOT default

By default, it's all stored in plain text
Create an EncryptionConfiguration object

Pods

Limit Resources

CPU & Memory

apiVersion: v1
kind: ResourceQuota
metadata:
  name: pods-low
spec:
  hard:
    cpu: "5"
    memory: 10Gi
    pods: "10"

Network Policies

by default, all pods can communicate with any other pod in the cluster
make sure you create a proper network policy for your cluster
does the front-end pod really nned to talk to the DB pod ?
what if an attacker can access the pods on the kube-system namespace ?
Restrict access to database to only the Pods that require it
Specify access via labels
If you want to control traffic flow at the IP address or port level (OSI layer 3 or 4), then you might consider using Kubernetes NetworkPolicies for particular applications in your cluster. NetworkPolicies are an application-centric construct which allow you to specify how a pod is allowed to communicate with various network "entities" (we use the word "entity" here to avoid overloading the more common terms such as "endpoints" and "services", which have specific Kubernetes connotations) over the network. NetworkPolicies apply to a connection with a pod on one or both ends, and are not relevant to other connections.
https://editor.cilium.io

more

kind: NetworkPolicy
...
spec:
  ingress:
    - form:
      - podSelector:
          matchLabels:
            name: guestbook

Run as non-root

RunAsNotRoot - Least Privilege - Only give as much permission/privilege as is absolutely necessary - In the configuration file, the runAsUser field specifies that for any Containers in the Pod, all processes run with user ID 1000. - more

spec:
  securityContext:
    runAsUser: 1000

Read only root filesystem

ReadOnlyRootFilesystem - Requires that containers must run with a read-only root filesystem (i.e. no writable layer) more

spec:
  securityContext:
    readOnlyRootFilesystem: true

Privilege Escalation (no_new_privs)

AllowPrivilegeEscalation - Gates whether or not a user is allowed to set the security context of a container to allowPrivilegeEscalation=true. This defaults to allowed so as to not break setuid binaries. Setting it to false ensures that no child process of a container can gain more privileges than its parent. more

spec:
  securityContext:
    allowPrivilegeEscalation: false

SecComp

filters a process's system calls

apiVersion: v1
kind: Pod
metadata:
  name: mypod
  annotation:
    seccomp.security.alpha.kubernetes.io/pod: runtime/defailt

AppArmor

uses program profiles to restrict the capabilities of individual programs

apiVersion: v1
kind: Pod
metadata:
  name: mypod
  annotation:
    apparmor.security.beta.kubernetes.io/hello: runtime/defailt
spec:
  containers:
    - name: hello

SELinux

applies security labels to objects and evaluates all security-relevant interactions via the security policy

apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
securityContext:
  seLinuxOptions:
    level: "s0:c123,c456"

Cluster

Open Policy Agent

In Kubernetes, Admission Controllers enforce semantic validation of objects during create, update, and delete operations. With OPA you can enforce custom policies on Kubernetes objects without recompiling or reconfiguring the Kubernetes API server.