Completek8s Classroom notes 21/Oct/2023


  • Out-of-the-box Kubernetes Security
  • Investigating Cluster-Security
  • Understanding RBAC
  • kubernetes resource security
  • Kubernetes Secrets

Out of the Box K8s Security

  • Nothing is 100% secure out of box.
  • Regardless of how much time you spend to secure an environment, it will never be 100%
  • Goal of security is to mitigate as much risk as possible, but you will never be able to mitigate 100% of the risk.
  • Nortol states in recent blog Refer Here that there are roughly 2200 cyber security attacks per day.

Kubernetes Security

  • Refer Here for the state of k8s security report from Redhat
    • 93% of respondents experienced atleast 1 security incident in their k8s environment in the last 12 months
    • More that half of respondents (55%) have had to delay an application rollout because of security concerns
    • Around 70% of security issues in k8s are due to misconfiguration
  • It looks like security is a huge issue in k8s space according to above statistics
  • Because of everchanging k8s, the landscape of k8s security is a mess.
  • There’s some light at the end of tunnel though. As with all platforms & environments there are best practices which we can follow.
  • Goal: To mitigate as many security risks as possible.
  • Kubernetes Security Attack surfaces

Cluster Hardening and benchmarks

  • The CIS (Center for Internet Security) has been defacto standard of hardening for years
  • CIS benchmarks are set of globally identified standards and best practices when it comes to helping engineers set up the security defences.
  • CIS hardened images for AWS Images Refer Here GCP images Refer Here and Refer Here for Azure VM Images
  • Refer Here for Apple iOS CIS Benchmark
  • Refer Here for CIS benchmarks of Securing kubernetes and Refer Here
  • Generally every benchmark is around 200+ pages pdf document. Going through every column and fixing issues might be difficult, We have lot of toools and platforms for using CIS Benchmarks.
  • Platforms such Checkov, kube-bench, kubescape scan the Security space against CIS and National vulnerability Database (NVD)

Lets create a kubeadm cluster using ubuntu 20.04 images

  • 1 master
  • 1 node

System Scanning

  • This is not k8s specific
  • Download the CIS-CAT Lite tool Refer Here
  • Extract the zip file and run the Accessor-GUI binary Assessor-GUI.exe
  • Now select the Advanced option
  • Fill the connectivity options


  • Refer Here for kube-bench docs
  • Kube-bench can be executed directly or by using trivy
  • Follow the getting started guide and deploy nopcommmerce or pitstop and get the kube-bench report
  • Exercise: Get me a security report

On-Prem Kubernetes Reality Check

  • Sizing Considerations
    • Standard workers
    • Memory intensive workers
    • CPU intensive workers
    • Special cases: (GPU)
  • Where to run
    • kubeadm
    • Openstack
    • Rancher
    • Kubespray
  • Operating System
    • Run a bare-metal & have OS run directly on server
    • Have a virtualized Hypervisor (ESXi/Hyperv)
    • Linux
    • Windows
  • Server logs
    • For control plane logs
      • /var/log/kube-apiserver.log
      • /var/log/kube-scheduler.log
      • /var/log/kube-controller-manager.log
    • For worker nodes
      • /var/log/kubelet.log
      • /var/log/kube-proxy.log
  • Ensure metrics server is up
  • crictl: helps you troubleshoot the container runtime Refer Here

Hybrid Services

  • Azure Stack/Azure Arc/ Azure Kubernetes Edge
  • Kubernetes Anywhere
  • Google Anthos

Virtualized bare metal

  • Equinix: Allows you to run k8s from UI as well as Automated space (using terraform) Refer Here
  • Open Metal: Refer Here

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

About continuous learner

devops & cloud enthusiastic learner