Custom Resource Definitions (CRD) & Operators

• CRDs extend the k8s api by allowing users to define new resource types.
  • CRDs define the schema for custom resources
  • CRDs expose API Endpoints
  • CRD’s are yaml specs
• Operators use those resources to automate lifecycl management of applications & infra
  • Operatars implement the logic for thos custom resources actively monitoring the state and taking actions to reconcile. Operators use a control loop
  • Operators can be developed in python or Go
• Refer Here for docs of CRDs with controllers
• Refer Here for operators
• A Sample CRD YAML

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: sampledb.qt.example.com
spec:
  group: qt.example.com
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                dbType:
                  type: string
                replicas:
                  type: integer
                storageSize:
                  type: string
  scoped: Namespaced
  names:
    plural: sampledbs
    singular: sampledb
    kind: SampleDB
    shortNames:
      - sdb

• The custom Resource spec will be as shown below

apiVersion: sampledb.qt.example.com/v1
kind: SampleDB
metadata:
  name: my-sampledb
spec:
  dbType: postgres
  replicas: 1
  storageSize: "1Gi"

• Now we need to build th operator/controller using k8s operator sdk Refer Here and kopf a framework in python
• Popular operators

Operator Name	Purpose / Functionality
Prometheus Operator	Automates deploying and managing Prometheus, Alertmanager, ServiceMonitors, etc. (Medium, CNCF)
Strimzi (Kafka Operator)	Manages Kafka clusters, topics, users, MirrorMaker, etc. (Medium, adaltas.com)
Elastic Cloud on Kubernetes (ECK)	Automates Elasticsearch, Kibana, APM deployments. (CNCF, @knowledgehut)
PostgreSQL Operator	Manages PostgreSQL clusters with backups, scaling, failover. (Medium)
MySQL Operator	Automates MySQL deployment, backups, replication. (Medium)
Vault Operator	Manages HashiCorp Vault for secrets, rotation, policies. (Medium)
Cert-manager Operator	Automates TLS certificate issuance and renewal. (CNCF)
Crossplane	Manages cloud resources (e.g., AWS, GCP) via Kubernetes CRDs. (CNCF)
Kong Gateway Operator	Deploys and configures Kong API gateway via Kubernetes. (Wikipedia)
TiDB Operator	Deploys and manages TiDB (MySQL-compatible HTAP database) in Kubernetes. (Wikipedia)

• Basic usage of some operators
• Prometheus Operator – Quickstart & Usage
• Strimzi (Kafka Operator) – Quickstart & Usage

Prep for Service Mesh

• Create an AKS Cluster and deploy AKS Store Apply

NS=shop
kubectl create namespace $NS
kubectl apply -f https://raw.githubusercontent.com/Azure-Samples/aks-store-demo/refs/heads/main/aks-store-all-in-one.yaml -n $NS

• Architecture of this application
  

Service Mesh

• A service mesh is an infra layer that manages how microservice communicate with each other, Instead of puhsing networking and observability logic into application code it delegates these responsibiliteis to meesh of sidecar proxies (envoy proxy) that run alongside servers
• Main features of Service mesh
  • Traffic Management:
    • Load Balancing, request routing, retries, timeouts
  • Service Discovery
  • Security:
    • mTLS
    • Automcatic Certificate & key management
    • Access Policies
  • Observability:
    • Collect metrics
  • Policy & Governance
  • Resilience
    • Circuit breaking
    • Fault Injection
• Popular Service Meshes
  • istio
  • linkerd
• Istio
  • Traffic management:
    • Advanced routing
      • HTTP headers
      • weights
      • regex
    • Canary
    • A/B
    • mirroring
  • Security:
    • mTLS
    • integrates with enterprise identity products
    • fine-grained RBAC
  • Observability:
    • Native INtegration with Prometheus and grafan, jaeger, Kaili
    • Rich telemetry
  • Extensibility
• Linker d:
  • Traffic Management
    • retries, timeouts
    • less advanced routing than istion
  • security:
    • mTLS
  • Observability:
    • Golden metrics (latency, traffic, error,saturation)

Istio Architecture

• Components

Installation

• Refer Here for installation options
• In cloud prefer native installation.
• Azure istio addon

az aks mesh enable --resource-group $MY_RESOURCE_GROUP_NAME --name $MY_AKS_CLUSTER_NAME
kubectl label ns shop istio.io/rev=asm-1-25 --overwrite

# restart workloads to get envoy sidecars
kubectl -n shop rollout restart deployment

• Enable mTLS in shop

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: shop # Apply to the 'default' namespace
spec:
  mtls:
    mode: STRICT

Put north-south traffic (external-to-internal) through ingress gateway

• lets patch svc store-front to use clusterIp

kubectl -n shop patch svc store-front  -p '{"spec": { "type": "ClusterIP" }}'

• Lets create a gateway and virtual service

apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: shop-gateway
  namespace: shop
spec:
  selector:
    istio: aks-istio-ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - *
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: store-front
  namespace: shop
spec:
  hosts:
  - *
  http:
  - route:
    - destination:
        host: store-front.shop.svc.cluster.local
        port:
          number: 80

• 90/10 traffic split

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-service
spec:
  hosts:
  - my-service
  http:
  - route:
    - destination:
        host: my-service
        subset: v1
      weight: 90
    - destination:
        host: my-service
        subset: v2
      weight: 10
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: my-service
spec:
  host: my-service
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

• Retry

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: my-service
spec:
  hosts:
  - my-service
  http:
  - route:
    - destination:
        host: my-service
        subset: v1
    retries:
      attempts: 3
      perTryTimeout: 2s
      retryOn: 5xx,connect-failure

• Refer Here for deploying ingress gateways Refer Here for securing gateways.