Completek8s Classroom notes 21/Oct/2023

Security

• Out-of-the-box Kubernetes Security
• Investigating Cluster-Security
• Understanding RBAC
• kubernetes resource security
• Kubernetes Secrets

Out of the Box K8s Security

• Nothing is 100% secure out of box.
• Regardless of how much time you spend to secure an environment, it will never be 100%
• Goal of security is to mitigate as much risk as possible, but you will never be able to mitigate 100% of the risk.
• Nortol states in recent blog Refer Here that there are roughly 2200 cyber security attacks per day.

Kubernetes Security

• Refer Here for the state of k8s security report from Redhat
  • 93% of respondents experienced atleast 1 security incident in their k8s environment in the last 12 months
  • More that half of respondents (55%) have had to delay an application rollout because of security concerns
  • Around 70% of security issues in k8s are due to misconfiguration
• It looks like security is a huge issue in k8s space according to above statistics
• Because of everchanging k8s, the landscape of k8s security is a mess.
• There’s some light at the end of tunnel though. As with all platforms & environments there are best practices which we can follow.
• Goal: To mitigate as many security risks as possible.
• Kubernetes Security Attack surfaces
  

Cluster Hardening and benchmarks

• The CIS (Center for Internet Security) has been defacto standard of hardening for years
• CIS benchmarks are set of globally identified standards and best practices when it comes to helping engineers set up the security defences.
• CIS hardened images for AWS Images Refer Here GCP images Refer Here and Refer Here for Azure VM Images
• Refer Here for Apple iOS CIS Benchmark
• Refer Here for CIS benchmarks of Securing kubernetes and Refer Here
• Generally every benchmark is around 200+ pages pdf document. Going through every column and fixing issues might be difficult, We have lot of toools and platforms for using CIS Benchmarks.
• Platforms such Checkov, kube-bench, kubescape scan the Security space against CIS and National vulnerability Database (NVD)

Lets create a kubeadm cluster using ubuntu 20.04 images

• 1 master
• 1 node

System Scanning

• This is not k8s specific
• Download the CIS-CAT Lite tool Refer Here
• Extract the zip file and run the Accessor-GUI binary Assessor-GUI.exe
• Now select the Advanced option
  
  
• Fill the connectivity options
  
  

kube-bench

• Refer Here for kube-bench docs
• Kube-bench can be executed directly or by using trivy
  • Refer Here and Refer Here for getting started and run the scan using Refer Here
• Follow the getting started guide and deploy nopcommmerce or pitstop and get the kube-bench report
• Exercise: Get me a security report

On-Prem Kubernetes Reality Check

• Sizing Considerations
  • Standard workers
  • Memory intensive workers
  • CPU intensive workers
  • Special cases: (GPU)
• Where to run
  • kubeadm
  • Openstack
  • Rancher
  • Kubespray
• Operating System
  • Run a bare-metal & have OS run directly on server
  • Have a virtualized Hypervisor (ESXi/Hyperv)
  • Linux
  • Windows
• Server logs
  • For control plane logs
    • /var/log/kube-apiserver.log
    • /var/log/kube-scheduler.log
    • /var/log/kube-controller-manager.log
  • For worker nodes
    • /var/log/kubelet.log
    • /var/log/kube-proxy.log
• Ensure metrics server is up
• crictl: helps you troubleshoot the container runtime Refer Here

Hybrid Services

• Azure Stack/Azure Arc/ Azure Kubernetes Edge
• Kubernetes Anywhere
• Google Anthos

Virtualized bare metal

• Equinix: Allows you to run k8s from UI as well as Automated space (using terraform) Refer Here
• Open Metal: Refer Here