Workshop Classroom Notes – 19/Dec/2022

CI/CD Pipelines For Container Based Applications

  • Project:
    • Opensource Ecommerce Application Developed in Python
    • Refer Here for the Project Website
    • Refer Here for the code repository
  • Target Platform: Kubernetes with Docker
  • Environments:
    • Dev
    • ST (System Test)
    • UAT (User Acceptance Test)/Staging/Pre-Prod
    • PROD

Setup

  1. Cloud Account
  2. GitHub Account
  3. Visual Studio Code
  4. gist.github.com/shaikkhajaibrahim

Goal

  • Create a CI/CD pipeline to deploy the application
    • on every commit done by developer to Dev Environment
    • Every day at a fixed time deploy the application to ST environment
    • once in two weeks deploy to UAT environment
    • for every two sprints deploy to PROD

Architecture of Appliction

  • Refer Here for the architecture document.

  • Technologies

    • Python Django
    • Next Js (Enterprise React framework)
    • Storage:
      • PostgreSql (RDBMS)
      • Redis (Cache)
    • Connectivity: over http by using GraphQL Preview

Manually run the application

  • To manually run the application, lets create a ubuntu vm with atleast 2 vcpus and 4/8 GB of RAM

  • Install docker

  • Try to bring up the application.

  • Refer Here for the steps to run the application

  • Steps:

git clone https://github.com/saleor/saleor-platform.git --recursive --jobs 3
cd saleor-platform
docker compose build
docker compose run --rm api python3 manage.py migrate
docker compose run --rm api python3 manage.py collectstatic --noinput
docker compose run --rm api python3 manage.py populatedb
docker compose run --rm api python3 manage.py createsuperuser
docker compose up -d
  • We need a instance with atleast 30 GB of disk space.

Activity 1: Understand the docker compose files

  • We got the code from git clone https://github.com/saleor/saleor-platform.git --recursive --jobs 3
  • Lets review the docker-compose.yaml and list all the network and volume requirements
  • Database:
    • Volumes:
    - saleor-db:/var/lib/postgresql/data
    - ./replica_user.sql:/docker-entrypoint-initdb.d/replica_user.sql
    
  • Write down for the api (saleor core), storefront and dashboard
    • volumes
    • environmental variables

Activity 2: Lets try to figureout manual steps for dashboard.

  • Steps
clone the dashboard
build the docker image 
push the docker image to docker hub
  • Manual commands
git clone https://github.com/WorkshopsByKhaja/saleor-dashboard.git
cd saleor-dashboard
docker image build -t shaikkhajaibrahim/saleor-dashboar:<ENV>-<DDMMYYYHHMM> .
docker image push shaikkhajaibrahim/saleor-dashboar:<ENV>-<DDMMYYYHHMM>
  • We have built the jenkins pipeline with the following
pipeline {
    agent any
    triggers {
        pollSCM('* * * * *')
    }
    stages {
        stage('vcs') {
            steps {
                git branch: 'main', url: 'https://github.com/WorkshopsByKhaja/saleor-dashboard.git'
            }
        }
        stage('docker image build') {
            steps {
                sh 'docker image build -t shaikkhajaibrahim/saleor-dashboar:DEV .'
            }
        }
        stage('push image to registry') {
            steps {
                sh 'docker image push shaikkhajaibrahim/saleor-dashboar:DEV'
            }
        }
    }
}

Preview

Activity 3: Create a Basic Jenkinsfile to perform the below manual steps

  • Manual commands
git clone https://github.com/WorkshopsByKhaja/saleor-dashboard.git
cd saleor-dashboard
docker image build -t shaikkhajaibrahim/saleor-dashboar:<ENV>-<DDMMYYYHHMM> .
docker image push shaikkhajaibrahim/saleor-dashboar:<ENV>-<DDMMYYYHHMM>
  • I need jenkins to be setup. Preview Preview

  • Refer Here for pipeline syntax

  • Refer Here for jenkins pipeline steps

  • Refer Here for the git repo

  • Configuration:

    • Install docker on a node & add the user with which we login from ci/cd to docker user group
    • Any changes made in the docker after the node/master is added restarting services will help on master and reconnecting will help on nodes.

Activity 3: Create a Basic azure-pipelines to perform the below manual steps

---
# where to run the ci/cd pipeline
pool:
  vmImage: ubuntu-22.04

# when to run
trigger:
  - main

# what has to be done
steps:
  - task: Docker@2
    inputs:
      command: build
      Dockerfile: "./Dockerfile"
      tags: "shaikkhajaibrahim/saleordashboard:AZDEV"
      
  • Configured CI pipeline to build the docker image on every code commit Preview
  • We need to push the image to docker hub Refer Here
---
# where to run the ci/cd pipeline
pool:
  vmImage: ubuntu-22.04

# when to run
trigger:
  - main

# what has to be done
steps:
  - task: Docker@2
    inputs:
      command: buildAndPush
      Dockerfile: "./Dockerfile"
      tags: "shaikkhajaibrahim/saleordashboard:AZDEV"
      containerRegistry: DockerHub

Activity 4: Create a Saleor Core image build from jenkins and Azure devops

  • Consider the following section
services:
  api:
    ports:
      - 8000:8000
    build:
      context: ./saleor
      dockerfile: ./Dockerfile
      args:
        STATIC_URL: "/static/"
    restart: unless-stopped
    networks:
      - saleor-backend-tier
    stdin_open: true
    tty: true
    depends_on:
      - db
      - redis
      - jaeger
    volumes:
      - ./saleor/saleor/:/app/saleor:Z
      - ./saleor/templates/:/app/templates:Z
      - ./saleor/tests/:/app/tests
      # shared volume between worker and api for media
      - saleor-media:/app/media
    command: python manage.py runserver 0.0.0.0:8000
    env_file:
      - common.env
      - backend.env
    environment:
      - JAEGER_AGENT_HOST=jaeger
      - STOREFRONT_URL=http://localhost:3000/
      - DASHBOARD_URL=http://localhost:9000/
      - ALLOWED_HOSTS=localhost,api
  • Create a Jenkinsfile and azure-pipelines.yaml for saleor

  • Refer Here for changes

  • Exercise: Try to repeat the same for saleor-storefront

Next Steps

  • Activities Preview

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

About continuous learner

devops & cloud enthusiastic learner