Background

這篇文章主要介紹 docker-compose 來完成整個 deploy 的 Flow。 整個實作的流程如下:

Push code to Github
        |
        V
CI automatically pulls repo (CircleCI)
        |
        V         failed
CI test code       -->      rejected
        |  pass
        V
CI build images for deploying
        |
        V
CI pushes built images to image repo (AWS ECR)
        |
        V
CI create deployment to AWS codeploy
        |
        V
Pull images from AWS ECR to target ec2 instances via AWS codedeploy
        |
        V
Restart Services

Docker-compose

我們這邊用之前 flask-vue-crud 的程式碼當作範例。

整個資料結構如下:

.
├── LICENSE
├── README.md
├── appspec.yml
├── .env
├── .gitignore
├── .circleci
│   └── config.yml
├── client
│   ├── Dockerfile
│   ├── README.md
│   ├── babel.config.js
│   ├── dist
│   ├── node_modules
│   ├── package-lock.json
│   ├── package.json
│   ├── public
│   └── src
├── docker-compose.yml
├── nginx
│   ├── Dockerfile
│   └── default.conf
├── scripts
│   ├── after-install.sh
│   └── before-install.sh
└── server
    ├── Dockerfile
    ├── README.md
    ├── __pycache__
    ├── config.py
    ├── locust
    ├── requirements.txt
    ├── run.py
    ├── src
    └── tests

我們解釋幾個比較重要的資料夾&檔案的意義:

  • .circleci: for circleci service
  • appspec.yml: for aws codedeploy service
  • .env: environment variables for docker-compose
  • client: vue.js service
  • nginx: nginx service
  • server: flask service
  • scripts: for aws codedeploy service
  • docker-compose.yml: for docker-compose tool

對應到 Background 的流程圖的話,我們把程式 commit 到 Github 時候 trigger CircleCI 進行測試和部署,部署的部分則是會透過 AWS codedeploy 來幫助。

docker-compose.yml

version: '3'
services:
  nginx:
    image: ${DOCKER_REGISTRY}/nginx:${BRANCH}
    restart: always
    build:
      dockerfile: Dockerfile
      context: ./nginx
    ports:
      - '3000:3000'
      - '5000:5000'
  api:
    image: ${DOCKER_REGISTRY}/server:${BRANCH}
    build:
      dockerfile: Dockerfile
      context: ./server
      args:
        ENV: ${BRANCH}
  client:
    image: ${DOCKER_REGISTRY}/client:${BRANCH}
    build:
      dockerfile: Dockerfile
      context: ./client
      args:
        ENV: ${BRANCH}

稍微解釋一下 docker-compose.yml,整個 application 包含了三個 services,分別是 nginx, api, client,對應到的資料夾分別是 nginx, server, client。基本的架構如下:

docker-compose--1--1

那每個資料夾的 Dockerfile 內容我就不細說了,有興趣的麻煩自己看一下原始碼。

P.S. ${DOCKER_REGISTRY} 和 ${BRANCH} 都是 environment variable 放在 .env 中。

CircleCI

.circleci/config.yml

version: 2
jobs:
  build-server:
    working_directory: ~/app
    docker:
      - image: circleci/python:3.6.4
        environment:
          PIPENV_VENV_IN_PROJECT: true
    steps:
      - checkout
      - run: sudo apt-get install python3-pip
      - run: sudo pip3 install --upgrade pip
      - run:
          command: |
            cd ~/app/server
            python3 -m venv venv
            . venv/bin/activate
            sudo pip3 install -r requirements.txt
      - save_cache:
          key: deps1-{{ .Branch }}-{{ checksum "server/requirements.txt" }}
          paths:
              -  ~/app/server/venv
      - run:
          command: |
            set -e
            export FLASK_APP=/home/circleci/app/server/run.py
            cd ~/app/server
            pytest -x

  build-client:
    working_directory: ~/app
    docker:
      - image: circleci/node:latest
    steps:
      - checkout
      - run:
          command: |
            cd ~/app/client 
            npm install
      - save_cache:
          key: deps1-{{ .Branch }}-{{ checksum "client/package.json" }}
          paths:
              -  ~/app/client/node_modules
      - run: echo "run test command"

  build-puhs-images:
    machine: true
    working_directory: ~/app
    steps:
      - checkout
      - restore_cache:
          key: deps1-{{ .Branch }}-{{ checksum "server/requirements.txt" }}
      - restore_cache:
          key: deps1-{{ .Branch }}-{{ checksum "client/package.json" }}
      - run:
          name: Show current branch
          command: echo ${CIRCLE_BRANCH}
      - run:
          name: Show git commit id
          command: echo ${CIRCLE_SHA1}
      - run:
          name: Install aws cli
          command: sudo pip install awscli --upgrade --user
      - run: docker-compose version
      - run: aws --version
      - run:
          name: "Log in to AWS ECR"
          command: eval $(aws ecr get-login --no-include-email)
      - run: 
          name: set environment
          command: |
            if [ "${CIRCLE_BRANCH}" == "master" ]; then
              echo BRANCH=prd >> .env
            elif [ "${CIRCLE_BRANCH}" == "qa" ]; then
              echo BRANCH=${CIRCLE_BRANCH} >> .env
            elif [ "${CIRCLE_BRANCH}" == "stg" ]; then
              echo BRANCH=${CIRCLE_BRANCH} >> .env
            fi
      - run: echo DOCKER_REGISTRY=${ECR_URL}/flask-vue-crud >> .env
      #- run: cat .env
      - run: docker-compose build
      - run: docker-compose push
      - run:
          name: AWS codedeploy
          command: |
            if [ "${CIRCLE_BRANCH}" == "qa" ]; then
                aws deploy create-deployment --application-name flask-vue-crud --deployment-group-name flask-vue-crud-qa --auto-rollback-configuration enabled=true,events="DEPLOYMENT_FAILURE" --file-exists-behavior OVERWRITE --github-location repository="TaikerLiang/flask-vue-crud",commitId="$CIRCLE_SHA1"
            fi
workflows:
  version: 2
  build_and_test:
    jobs:
      - build-server
      - build-client  
      - build-puhs-images:
          requires:
            - build-server
            - build-client  

解釋一下 circleci 的 config file。主要分成三個部分 build-server, build-clientbuild-puhs-images

  • build-server: 建立 server side 的環境 & 通過 Test。
  • build-client: 建立 client sied 的環境 & 通過 Test。
  • build-puhs-images: 上面兩個部分都要通過才會進入到這個部分(可以發現有requres的要求)。這部分就是建立 images 接著 push 到指定的 repository,最後我們透過 aws cli 的指令來呼叫 codedeploy 服務。

整個 build-puhs-images 中比較特別的部分是我們會根據 branch name 來做相對應的動作,比如說 qa 的 branch 通過後就 deploy 到 qa 的機器這樣。

P.S. ${ECR_URL} 這個環境變數需到 circleci 的 console 中設定。

AWS Codedeploy

appspec.yml

version: 0.0
os: linux
files:
  - source: /
    destination: /home/ubuntu/flask-vue-crud
hooks:
  BeforeInstall:
    - location: scripts/before-install.sh
      runas: ubuntu
      timeout: 300
  AfterInstall:
    - location: scripts/after-install.sh
      runas: ubuntu
      timeout: 300

scripts/before-install.sh

#!/bin/bash

cd /home/ubuntu/flask-vue-crud
docker-compose down
sudo cp -r .env ../
cd ~
sudo rm -fr /home/ubuntu/flask-vue-crud

before-install.sh 基本上就是把服務暫停資料夾清空然後下載新的程式碼下來。(這邊的重點是我們會保留 .env 檔案,裡面有環境變數的資料。)

scripts/after-install.sh

#!/bin/bash
sudo chown -R ubuntu:root /home/ubuntu/flask-vue-crud
cd /home/ubuntu/flask-vue-crud
sudo mv ../.env ./.env
docker-compose pull
docker-compose up -d

先給予資料夾先對應的權限接著把剛剛保留 .env 檔案塞回資料夾中,接著更新 images 檔案,最後啟動服務。

Code

因為我是開一台EC2的機器來測試整個流程,因此在 client 資料夾中的 src/config.js 需要做需相對應的設定。

let config;

console.log(process.env);

if (process.env.VUE_APP_ENV === "dev") {
  config = {
    $api_url: "http://localhost",
    timeoutDuration: 30000,
  };
} else {
  config = {
    $api_url: "http://54.248.53.18",
    timeoutDuration: 1000,
  };
}

export { config }

可以看到有設定一個環境變數 VUE_APP_ENV 相關資訊可以到 client/.env 中查看,基本上就是根據 VUE_APP_ENV 來決定 api_url

Start

最後我們來啟動整個服務,記得如果你是在EC2啟動相關的port要開啟。

$ docker-compose up

Screen-Shot-2019-08-19-at-2.02.28-PM

Reference