DevOps Classroom notes 08/Mar/2025

Authentication and Authorization in k8s

• Kubernetes doesnot have an in built authentication system i.e. k8s will not store users.
• For k8s authentication is done externally
• k8s relies on
  • oidc providers
  • user signed by trusted CA (Certificate authority)
• Once the user is authenticated then we can assign roles (permissions), here we have
  • role
  • clusterrole

Certificate Authority in k8s

Overview of Certificate Authority in Kubernetes

In Kubernetes, a Certificate Authority (CA) plays a crucial role in securing communication between various components of the cluster. It acts as a trusted third party that issues digital certificates to validate the identities of entities interacting with the cluster, such as nodes, users, and services.

Role of Certificate Authority

• Issuing Certificates: The CA issues certificates to different components of the Kubernetes infrastructure, such as the Kubernetes API server, kubelet, and etcd. These certificates are used to establish secure communication via TLS (Transport Layer Security) between these components[1][2].
• Authentication and Authorization: Certificates issued by the CA are used for authentication and authorization within the cluster. Client certificates, for example, are used to verify the identity of users or processes interacting with the cluster[2][3].
• Trust Establishment: The CA helps establish trust between different entities in the cluster by ensuring that only trusted entities can interact with the Kubernetes infrastructure and its resources[5].

Types of Certificates in Kubernetes

• Server Certificates: Used by components like the Kubernetes API server to secure communication with clients. For instance, the API server uses a server certificate to establish secure connections with clients like kubectl[2].
• Client Certificates: Used by clients (users or processes) to authenticate with the Kubernetes API server. These certificates contain information such as the client’s name and group membership[2][3].

Certificate Management in Kubernetes

Managing certificates in Kubernetes can be challenging due to the dynamic nature of the environment. Tools like cert-manager help automate the provisioning, renewal, and revocation of certificates, making it easier to manage certificate lifecycles[5].

Steps for Generating Certificates

1. Create a Certificate Authority (CA): Set up a CA to sign and validate certificates.
2. Generate a Private Key: Create a private key for the certificate.
3. Create a Certificate Signing Request (CSR): Generate a CSR with necessary details.
4. Sign the CSR: Submit the CSR to the CA for signing.
5. Distribute Certificates: Deploy the signed certificates to relevant components.
6. Configure Components: Update configurations to use the new certificates[2].

Importance of Certificate Authority

• Security: Ensures secure communication and prevents unauthorized access.
• Trust: Establishes trust between entities in the cluster.
• Authentication and Authorization: Facilitates strong authentication and access control[5].

Citations:
[1] https://jvns.ca/blog/2017/08/05/how-kubernetes-certificates-work/
[2] https://everythingdevops.dev/securing-your-kubernetes-environment-a-comprehensive-guide-to-server-and-client-certificates-in-kubernetes/
[3] https://www.youtube.com/watch?v=gXz4cq3PKdg
[4] https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/
[5] https://www.appviewx.com/education-center/importance-of-pki-and-tls-certificates-in-kubernetes/
[6] https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/
[7] https://kubernetes.io/docs/setup/best-practices/certificates/
[8] https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md

Certificate Location

In a Kubernetes cluster, certificates for the master node are typically stored in the /etc/kubernetes/pki directory. This directory contains various certificates and keys used for secure communication between components of the cluster.

Key Files and Their Locations

• CA Certificate: /etc/kubernetes/pki/ca.crt
• CA Private Key: /etc/kubernetes/pki/ca.key
• API Server Certificate: /etc/kubernetes/pki/apiserver.crt
• API Server Private Key: /etc/kubernetes/pki/apiserver.key
• Other Certificates: Other certificates like apiserver-etcd-client.crt, apiserver-etcd-client.key, etc., are also stored in this directory[4][5].

Purpose of These Certificates

• CA Certificate (ca.crt): Acts as the root of trust for all certificates in the cluster.
• API Server Certificate (apiserver.crt): Used by the Kubernetes API server to establish secure connections with clients.
• API Server Private Key (apiserver.key): Corresponds to the API server certificate and is used for decryption.

These certificates are crucial for maintaining the security and integrity of the cluster by ensuring that all communication is encrypted and authenticated.

Managing Certificates

Kubernetes provides tools like kubeadm for managing certificates during cluster initialization and joining nodes. Additionally, the certificates.k8s.io API allows for dynamic management of TLS certificates within the cluster[6].

Renewing Certificates

Certificates have a limited lifespan and need to be renewed periodically. This can be done manually or using tools like cert-manager to automate the process[1].

Citations:
[1] https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/renew-expiring-kubernetes-cluster-certificates
[2] https://kubernetes.io/docs/tasks/administer-cluster/certificates/
[3] https://stackoverflow.com/questions/65186710/how-to-add-extra-nodes-to-the-certificate-authority-data-from-a-self-signed-k8s
[4] https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/
[5] https://kubernetes.io/docs/setup/best-practices/certificates/
[6] https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/
[7] https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/
[8] https://github.com/kubernetes/kubeadm/issues/521

Create a user in kubeadm cluster

Here are the steps to create a user in Kubernetes with reader permission on pods using Role-Based Access Control (RBAC):

Step 1: Create a Certificate for the User

To authenticate a human user in Kubernetes, you need to create a certificate. You can use tools like OpenSSL to generate a certificate signing request (CSR) and then sign it with your Certificate Authority (CA).

1. Generate Private Key and CSR:
   bash
openssl genrsa -out thor.key 2048
openssl req -new -key thor.key -out thor.csr -subj "/CN=thor"
2. Sign the CSR with CA:
   bash
openssl x509 -req -in thor.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out thor.crt -days 365

Step 2: Configure Kubeconfig for the User

1. Create a Kubeconfig File:
   bash
kubectl config set-credentials thor --client-certificate=thor.crt --client-key=thor.key --embed-certs
2. Set the Cluster and Context:
   “`bash
   kubectl config set-cluster –server=https://172.30.1.2:6443 –certificate-authority=/etc/kubernetes/pki/ca.crt
   kubectl config set-context thor-context –cluster= –user=thor

“`

<ol>
<li><strong>Use the Context</strong>:
<code>bash
kubectl config use-context thor-context</code></li>
</ol>

<h3>Step 3: Create a Role for Pod Reader</h3>

<ol>
<li><strong>Define a Role</strong>:
Create a YAML file named <code>pod-reader-role.yaml</code> with the following content:
“`yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: pod-reader
rules:

apiGroups: [“”] # “” indicates the core API group
resources: [“pods”]
verbs: [“get”, “watch”, “list”]
“`

Apply the Role:
bash
kubectl apply -f pod-reader-role.yaml

Step 4: Create a RoleBinding

1. Define a RoleBinding:
   Create a YAML file named pod-reader-rolebinding.yaml with the following content:
   “`yaml
   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
   name: pod-reader-binding
   namespace: default
   subjects:
2. kind: User
   name: thor
   apiGroup: rbac.authorization.k8s.io
   roleRef:
   kind: Role
   name: pod-reader
   apiGroup: rbac.authorization.k8s.io
   “`
3. Apply the RoleBinding:
   bash
kubectl apply -f pod-reader-rolebinding.yaml

Step 5: Verify Permissions

1. Switch to the User Context:
   bash
kubectl config use-context user-context
2. List Pods:
   bash
kubectl get pods

You should be able to list pods successfully.

1. Try Creating a Pod:
   bash
kubectl create pod --image=nginx

This should fail because the user only has read permissions.

These steps ensure that the user can read pods but cannot perform any write operations.

Citations:
[1] https://dev.to/thenjdevopsguy/kubernetes-roles-and-users-rbac-for-k8s-44n4
[2] https://kubernetes.io/docs/reference/access-authn-authz/rbac/
[3] https://shalb.com/devops-methods-practices/how-to-grant-user-access-to-particular-namespace/
[4] https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
[5] https://stackoverflow.com/questions/51657393/kubernetes-rbac-user-with-access-to-specific-pods
[6] https://discuss.kubernetes.io/t/how-to-create-user-in-kubernetes-cluster-and-give-it-access/9101
[7] http://kubernetes-tutorial.schoolofdevops.com/configuring_authentication_and_authorization/
[8] https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/

Kubernetes RBAC Lets Learn about authorization

• Refer Here for official docs
• To deal with authorization in k8s we have
  • roles
  • cluster roles
  • role binding
  • cluster roler binding
• Role and cluster role Refer Here
• RoleBinding and ClusterRoleBinding Refer Here

Lets write a Role which gives read permissions

• namespace: dev
• to read pods, rs, deployments, service
• Assuming our users are thor, hulk
• Lets create a role => dev-reader

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: dev-reader
  namespace: dev
rules:
  - apiGroups: 
      - ""
      - "apps"
    verbs:
      - get
      - list
      - watch
    resources:
      - pods
      - deployments
      - replicasets
      - services

• Lets apply a role binding to users thor and hulk

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: dev-reader
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: dev-reader
subjects:
  - apiGroup: "rbac.authorization.k8s.io"
    kind: "User"
    name: thor
  - apiGroup: "rbac.authorization.k8s.io"
    kind: "User"
    name: hulk

• Exercise: Create a cluster role for reader and assign that to users hulk and thor

Ingress in kubernetes

• Consider the following architecture
  
• Now exposing individual services is not a good idea
• We would use ingress, which needs an ingress controller. k8s doesnot have default ingress controller so we can use ingress controllers of our choice Refer Here
  

Popular ingress controllers

Here are some of the most popular Kubernetes ingress controllers as of 2025:

1. NGINX Ingress Controller

• Overview: One of the most widely used ingress controllers, known for its stability and community support.
• Key Features: SSL termination, flexible routing, Lua scripting for extensibility.
• Use Case: Suitable for stable, well-documented environments.

2. Traefik

• Overview: A dynamic ingress controller designed for microservices architectures.
• Key Features: Auto-discovery of services, real-time updates, native Let’s Encrypt support for SSL.
• Use Case: Ideal for teams needing modern, easy-to-use ingress with dynamic service discovery.

3. HAProxy Ingress

• Overview: Known for high performance and reliability, HAProxy is a long-standing load balancer.
• Key Features: Advanced Layer 7 routing, robust monitoring and observability.
• Use Case: Best for environments requiring efficient traffic management.

4. Envoy

• Overview: An open-source edge and service proxy that offers advanced traffic routing and observability.
• Key Features: Support for retries, circuit breaking, rate limiting, and integrated observability.
• Use Case: Suitable for teams needing a feature-rich ingress controller capable of complex traffic management.

5. Istio Ingress Gateway

• Overview: Part of the Istio service mesh, it provides advanced traffic management features.
• Key Features: Intelligent routing, mTLS security, integrated observability.
• Use Case: Ideal for organizations already using Istio for service mesh purposes.

6. Contour

• Overview: An ingress controller based on Envoy that supports modern protocols like HTTP/2 and gRPC.
• Key Features: High performance with path-based routing and header manipulation capabilities.
• Use Case: Well-suited for organizations needing a lightweight ingress controller.

7. Pomerium

• Overview: An identity-aware ingress controller focused on security and access control.
• Key Features: Integrates with identity providers, zero trust security model, centralized access control.
• Use Case: Best for organizations prioritizing security in their Kubernetes environments.

Additional Options

Other notable ingress controllers include:
– Kong Ingress Controller
– Gloo
– Kusk Gateway
– OpenNJet Ingress Controller

Choosing the right ingress controller depends on your specific requirements regarding performance, security, ease of use, and integration with existing systems[1][2][3][4].

Citations:
[1] https://www.pomerium.com/blog/best-ingress-controllers-for-kubernetes
[2] https://www.solo.io/topics/kubernetes-api-gateway/kubernetes-ingress-controller
[3] https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
[4] https://tetrate.io/blog/best-ingress-controller-for-kubernetes/
[5] https://www.loft.sh/blog/advanced-guide-to-kubernetes-ingress-controllers
[6] https://kubernetes.io/docs/concepts/services-networking/ingress/
[7] https://github.com/kubernetes/ingress-nginx
[8] https://blog.palark.com/comparing-ingress-controllers-for-kubernetes/

• For complete list Refer Here