Workshop Classroom Notes – 19/Dec/2022

CI/CD Pipelines For Container Based Applications

• Project:
  • Opensource Ecommerce Application Developed in Python
  • Refer Here for the Project Website
  • Refer Here for the code repository
• Target Platform: Kubernetes with Docker
• Environments:
  • Dev
  • ST (System Test)
  • UAT (User Acceptance Test)/Staging/Pre-Prod
  • PROD

Setup

1. Cloud Account
2. GitHub Account
3. Visual Studio Code
4. gist.github.com/shaikkhajaibrahim

Goal

• Create a CI/CD pipeline to deploy the application
  • on every commit done by developer to Dev Environment
  • Every day at a fixed time deploy the application to ST environment
  • once in two weeks deploy to UAT environment
  • for every two sprints deploy to PROD

Architecture of Appliction

• Refer Here for the architecture document.

• Technologies

  • Python Django
  • Next Js (Enterprise React framework)
  • Storage:
    • PostgreSql (RDBMS)
    • Redis (Cache)
  • Connectivity: over http by using GraphQL

Manually run the application

• To manually run the application, lets create a ubuntu vm with atleast 2 vcpus and 4/8 GB of RAM

• Install docker

• Try to bring up the application.

• Refer Here for the steps to run the application

• Steps:

git clone https://github.com/saleor/saleor-platform.git --recursive --jobs 3
cd saleor-platform
docker compose build
docker compose run --rm api python3 manage.py migrate
docker compose run --rm api python3 manage.py collectstatic --noinput
docker compose run --rm api python3 manage.py populatedb
docker compose run --rm api python3 manage.py createsuperuser
docker compose up -d

• We need a instance with atleast 30 GB of disk space.

Activity 1: Understand the docker compose files

• We got the code from git clone https://github.com/saleor/saleor-platform.git --recursive --jobs 3
• Lets review the docker-compose.yaml and list all the network and volume requirements
• Database:
  • Volumes:

  - saleor-db:/var/lib/postgresql/data
  - ./replica_user.sql:/docker-entrypoint-initdb.d/replica_user.sql
  

• Write down for the api (saleor core), storefront and dashboard
  • volumes
  • environmental variables

Activity 2: Lets try to figureout manual steps for dashboard.

• Steps

clone the dashboard
build the docker image 
push the docker image to docker hub

• Manual commands

git clone https://github.com/WorkshopsByKhaja/saleor-dashboard.git
cd saleor-dashboard
docker image build -t shaikkhajaibrahim/saleor-dashboar:<ENV>-<DDMMYYYHHMM> .
docker image push shaikkhajaibrahim/saleor-dashboar:<ENV>-<DDMMYYYHHMM>

• We have built the jenkins pipeline with the following

pipeline {
    agent any
    triggers {
        pollSCM('* * * * *')
    }
    stages {
        stage('vcs') {
            steps {
                git branch: 'main', url: 'https://github.com/WorkshopsByKhaja/saleor-dashboard.git'
            }
        }
        stage('docker image build') {
            steps {
                sh 'docker image build -t shaikkhajaibrahim/saleor-dashboar:DEV .'
            }
        }
        stage('push image to registry') {
            steps {
                sh 'docker image push shaikkhajaibrahim/saleor-dashboar:DEV'
            }
        }
    }
}

Activity 3: Create a Basic Jenkinsfile to perform the below manual steps

• Manual commands

git clone https://github.com/WorkshopsByKhaja/saleor-dashboard.git
cd saleor-dashboard
docker image build -t shaikkhajaibrahim/saleor-dashboar:<ENV>-<DDMMYYYHHMM> .
docker image push shaikkhajaibrahim/saleor-dashboar:<ENV>-<DDMMYYYHHMM>

• I need jenkins to be setup.

• Refer Here for pipeline syntax

• Refer Here for jenkins pipeline steps

• Refer Here for the git repo

• Configuration:

  • Install docker on a node & add the user with which we login from ci/cd to docker user group
  • Any changes made in the docker after the node/master is added restarting services will help on master and reconnecting will help on nodes.

Activity 3: Create a Basic azure-pipelines to perform the below manual steps

• Create a azure-pipelines.yaml with necessary agent configuration in https://github.com/WorkshopsByKhaja/saleor-dashboard.git

• Setup Azure DevOps Pipeline Refer Here

• Azure-Pipeline to build the docker image whenever any change pushed to main branch

---
# where to run the ci/cd pipeline
pool:
  vmImage: ubuntu-22.04

# when to run
trigger:
  - main

# what has to be done
steps:
  - task: Docker@2
    inputs:
      command: build
      Dockerfile: "./Dockerfile"
      tags: "shaikkhajaibrahim/saleordashboard:AZDEV"
      

• Configured CI pipeline to build the docker image on every code commit
• We need to push the image to docker hub Refer Here

---
# where to run the ci/cd pipeline
pool:
  vmImage: ubuntu-22.04

# when to run
trigger:
  - main

# what has to be done
steps:
  - task: Docker@2
    inputs:
      command: buildAndPush
      Dockerfile: "./Dockerfile"
      tags: "shaikkhajaibrahim/saleordashboard:AZDEV"
      containerRegistry: DockerHub

Activity 4: Create a Saleor Core image build from jenkins and Azure devops

• Consider the following section

services:
  api:
    ports:
      - 8000:8000
    build:
      context: ./saleor
      dockerfile: ./Dockerfile
      args:
        STATIC_URL: "/static/"
    restart: unless-stopped
    networks:
      - saleor-backend-tier
    stdin_open: true
    tty: true
    depends_on:
      - db
      - redis
      - jaeger
    volumes:
      - ./saleor/saleor/:/app/saleor:Z
      - ./saleor/templates/:/app/templates:Z
      - ./saleor/tests/:/app/tests
      # shared volume between worker and api for media
      - saleor-media:/app/media
    command: python manage.py runserver 0.0.0.0:8000
    env_file:
      - common.env
      - backend.env
    environment:
      - JAEGER_AGENT_HOST=jaeger
      - STOREFRONT_URL=http://localhost:3000/
      - DASHBOARD_URL=http://localhost:9000/
      - ALLOWED_HOSTS=localhost,api

• Create a Jenkinsfile and azure-pipelines.yaml for saleor

• Refer Here for changes

• Exercise: Try to repeat the same for saleor-storefront

Next Steps

• Activities