Devops

https://docs.docker.com/engine/swarm/ Docker Swarm Node Manager Node 초기화 https://docs.docker.com/engine/reference/commandline/swarm_init/ 초기화 Initialize a swarm docker swarm init $ docker swarm init Swarm initialized: current node (bvz81updecsj6wjz393c09vti) is now a manager. To add a worker to this swarm, run the following command: docker swarm join \ --token SWMTKN-1-3pu6hszjas19xyp7ghgosyx9k8atbfcr8p2is99znpy26u2lkl-1awxwuwd3z9j1z3puu7rcgdbx \ 172.17.0.2:2377 To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions. Initialize a swarm with advertised address docker swarm init --advertise-addr <ip|interface>[:port] $ docker swarm init --advertise-addr 192.168.99.121 Swarm initialized: current node (bvz81updecsj6wjz393c09vti) is now a manager. To add a worker to this swarm, run the following command: docker swarm join \ --token SWMTKN-1-3pu6hszjas19xyp7ghgosyx9k8atbfcr8p2is99znpy26u2lkl-1awxwuwd3z9j1z3puu7rcgdbx \ 172.17.0.2:2377 To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions. 노드 추가 Join Node Join a swarm as a node and/or manager https://docs.docker.com/engine/reference/commandline/swarm_join/ ...

Github Actions 101 Github 에서 제공하는 Workflow 툴 GitHub-hosted Runner or Self-Hosted Runner 에서 실행 Actions 탭을 통해서 Template을 선택하고 Yaml 파일로 Task 내용을 기술 .github/workflows 디렉토리 밑에 위치 Runner 종류 GitHub-hosted Runner : MS Azure 가상머신에서 실행 Public Repository : 무료 Private Repository : 2000분/월 무료 Self-Hosted Runner : 자체 머신을 통해 Runner Hosting https://help.github.com/en/actions/hosting-your-own-runners/adding-self-hosted-runners{:target="_blank"} Actions Basic Actions Tab Workflow syntax for GitHub Actions https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions{:target="_blank"} awesome-actions https://github.com/sdras/awesome-actions{:target="_blank"} Workflow runs-on: Virtual machine ubuntu, macos, windows server 제공 기본 Package or apps 가 등록되어 있음 : https://github.com/actions/virtual-environments{:target="_blank"} Ubuntu : https://github.com/actions/virtual-environments/blob/master/images/linux/Ubuntu1804-README.md{:target="_blank"} Windows Server : https://github.com/actions/virtual-environments/blob/master/images/win/Windows2019-Readme.md{:target="_blank"} steps : uses: 예약된 Actions 실행이나 Apps 통합을 통해 Apps 사용 환경 구성 ex> uses: actions/checkout@v2 : git checkout 실행 ex> uses: nuget/setup-nuget@v1 : nuget apps setup ex> uses: microsoft/setup-msbuild@v1 : msbuild setup run: run command 지정 name: CI # workflow 이름 on: # Triggers Event push: branches: [ master ] pull_request: branches: [ master ] jobs: build: # Single job name runs-on: ubuntu-latest # virtual machine steps: - uses: actions/checkout@v2 - name: Run a one-line script run: echo Hello, world! - name: Run a multi-line script run: | echo Add other actions to build, echo test, and deploy your project. Actions 예제 Docker Build & Registry Push ubuntu-latest 이미지에는 Docker Daemon 활성화됨 secrets 변수 : [Settings] - [Secrets] 에서 변수 세팅 (DOCKERHUB_PASS) ${{ secrets.DOCKERHUB_PASS }} name: Docker Image CI on: push: branches: [ master ] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Build the Docker image run: docker build . --tag cdecl/gcc-boost - name: docker login run: echo '${{ secrets.DOCKERHUB_PASS }}' | docker login -u cdecl --password-stdin - name: docker push run: docker push cdecl/gcc-boost ...

Gitlab CI/CD 101 Gitlab 에서 제공하는 CI/CD 목적의 Workflow 툴 Auto DevOps or gitlab-runner 에서 실행 Setup CI/CD 를 통해 세팅 .gitlab-ci.yml 파일에 기술 Gitlab-Runner gitlab-runner : .gitlab-ci.yml 기반 파이프 라인 구성 Shared Runners : gitlab.com 에서 hosting 해주는 Runner Self hosting Runners : 별도 머신을 통해 Runner 설치 Gitlab-Runner 세팅 (Self hosting) Installing the Runner https://docs.gitlab.com/runner/install/linux-repository.html{:target="_blank"} Registering Runners https://docs.gitlab.com/runner/register/index.html{:target="_blank"} Interactive register runner $ sudo gitlab-runner register Runtime platform arch=amd64 os=linux pid=120146 revision=c5874a4b version=12.10.2 Running in system-mode. Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/): http://hqgit.inpark.kr/ Please enter the gitlab-ci token for this runner: xxxxxxxxxxxxxxxxxxxxxxxxxxx Please enter the gitlab-ci description for this runner: ci-test runner Please enter the gitlab-ci tags for this runner (comma separated): centos24,ci-test,cdecl Registering runner... succeeded runner=WpQDakzK Please enter the executor: shell, kubernetes, parallels, docker, docker-ssh, ssh, virtualbox, docker+machine, docker-ssh+machine, custom: shell Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! # inline sudo gitlab-runner register \ --non-interactive \ --url "https://gitlab.com/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ --executor "docker" \ --docker-image alpine:latest \ --description "docker-runner" \ --tag-list "docker" \ sudo gitlab-runner register \ --non-interactive \ --url "http://centos.cdecl.net/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ --executor "docker" \ --docker-image alpine \ --description "docker-runner" \ --tag-list "docker" \ --env "DOCKER_TLS_CERTDIR=" \ --docker-privileged=true \ --docker-volumes "/ansible:/ansible" \ --docker-extra-hosts "centos.cdecl.net:192.168.0.20" Pipeline Configuration Basic GitLab CI/CD Pipeline Configuration Reference https://docs.gitlab.com/ee/ci/yaml/{:target="_blank"} Pipeline 기본적으로 git checkout 실행 Github Action 과 다르게 매뉴얼 실행 버튼 존재 image: ubuntu stages: # statge 정의 - build - test - deploy before_script: # - echo "Before script section" - echo "For example you might run an update here or install a build dependency" - echo "Or perhaps you might print out some debugging details" after_script: - echo "After script section" - echo "For example you might do some cleanup here" build_stage: stage: build script: - echo "Do your build here" test_stage1: stage: test script: - echo "Do a test here" - echo "For example run a test suite" test_stage2: stage: test script: - echo "Do another parallel test here" - echo "For example run a lint test" deploy_stage: stage: deploy script: - echo "Do your deploy here" ...

Kubernetes 설치 및 운영 101 사전 준비 Kubernetes 설치 전 서버 구성 변경 참고 : https://www.mirantis.com/blog/how-install-kubernetes-kubeadm/{:target="_blank"} Swap 영역을 비활성화 # 일시적인 설정 $ sudo swapoff -a # 영구적인 설정, 아래 swap 파일 시스템을 주석처리 $ sudo vi /etc/fstab ... # /dev/mapper/kube--master--vg-swap_1 none swap sw 0 0 SELinux Disable # 임시 $ sudo setenforce 0 # 영구 $ sudo vi /etc/sysconfig/selinux ... SELinux=disabled 방화벽 Disable $ sudo systemctl disable firewalld $ sudo systemctl stop firewalld 브릿지 네트워크 할성화 # Centos $ sudo vim /etc/sysctl.d/k8s.conf net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 # Ubuntu $ sudo vim /etc/ufw/sysctl.conf net/bridge/bridge-nf-call-ip6tables = 1 net/bridge/bridge-nf-call-iptables = 1 net/bridge/bridge-nf-call-arptables = 1 Docker Install Centos Install : https://docs.docker.com/engine/install/centos/{:target="_blank"} Cgroup 드라이버 이슈 최신 Kubernetes는 docker cgroup driver를 cgroupfs → systemd 변경 필요 Master Init 및 Worker Join 시 WARNING 발생 https://kubernetes.io/ko/docs/setup/production-environment/container-runtimes/{:target="_blank"} kubeadm init --pod-network-cidr 10.244.0.0/16 ... [init] Using Kubernetes version: v1.19.3 [preflight] Running pre-flight checks [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". ... ... 드라이버 변경 작업 /etc/docker/daemon.json 파일 작성 $ cat <<EOF | sudo tee /etc/docker/daemon.json { "exec-opts": ["native.cgroupdriver=systemd"], "log-driver": "json-file", "log-opts": { "max-size": "100m" }, "storage-driver": "overlay2", "storage-opts": [ "overlay2.override_kernel_check=true" ] } EOF # 도커 재시작 $ sudo systemctl restart docker # 확인 $ sudo docker info | grep -i cgroup Cgroup Driver: systemd Kubernetes (kubeadm, kubelet, kubectl) 설치 참고 : https://kubernetes.io/docs/setup/independent/install-kubeadm/{:target="_blank"} Kubernetes 설치 : Centos7 기준 Docker 설치 sudo yum install -y docker sudo systemctl enable docker && systemctl start docker sudo usermod -aG docker $USER kubeadm, kubelet, kubectl : Repo 추가 및 패키지 설치 $ cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg EOF $ sudo yum install -y kubelet kubeadm kubectl $ sudo systemctl enable kubelet && systemctl start kubelet # 버전이 안맞을 경우 지정 # sudo yum install kubelet-[version] kubeadm-[version] kubectl-[version] kubectl 자동완성 # sh source <(kubectl completion sh) echo "source <(kubectl completion sh)" >> ~/.shrc # zsh source <(kubectl completion zsh) echo "if [ $commands[kubectl] ]; then source <(kubectl completion zsh); fi" >> ~/.zshrc Master Node Init 및 Worker Node Join Master Node 설정 Master 초기화 네트워크 클래스 대역을 설정 필요 : --pod-network-cidr 10.244.0.0/16 sudo kubeadm init --pod-network-cidr 10.244.0.0/16 Kubectl 사용 : To start using your cluster.. 아래 항목 3줄 실행 [init] Using Kubernetes version: v1.10.5 ... To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config ... You can now join any number of machines by running the following on each node as root: kubeadm join 192.168.28.15:6443 --token 1ovd36.ft4mefr909iotg0a --discovery-token-ca-cert-hash sha256:82953a3ed178aa8c511792d0e21d9d3283e7575f3d3350a00bea3e34c2b87d29 Pod 상태 확인 coredns STATUS → Pending (∵ Overlay network 미설치) $ kubectl get po -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-66bff467f8-ktvsz 0/1 Pending 0 19s kube-system coredns-66bff467f8-nvvjz 0/1 Pending 0 19s kube-system etcd-node1 1/1 Running 0 29s kube-system kube-apiserver-node1 1/1 Running 0 29s kube-system kube-controller-manager-node1 1/1 Running 0 29s kube-system kube-proxy-s582x 1/1 Running 0 19s kube-system kube-scheduler-node1 1/1 Running 0 29s Overlay network : Calico 설치 Overlay network 종류 https://kubernetes.io/docs/concepts/cluster-administration/networking/{:target="_blank"} Install Calico for on-premises deployments https://docs.projectcalico.org/getting-started/kubernetes/self-managed-onprem/onpremises{:target="_blank"} # Install Calico for on-premises deployments $ kubectl apply -f https://docs.projectcalico.org/manifests/calico-typha.yaml coredns 서비스가 정상적으로 Running $ kubectl get po -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-799fb94867-bcntz 0/1 CrashLoopBackOff 3 2m6s kube-system calico-node-jtcmt 0/1 Running 1 2m7s kube-system calico-typha-6bc9dd6468-x2hjj 0/1 Pending 0 2m6s kube-system coredns-66bff467f8-ktvsz 0/1 Running 0 3m23s kube-system coredns-66bff467f8-nvvjz 0/1 Running 0 3m23s kube-system etcd-node1 1/1 Running 0 3m33s kube-system kube-apiserver-node1 1/1 Running 0 3m33s kube-system kube-controller-manager-node1 1/1 Running 0 3m33s kube-system kube-proxy-s582x 1/1 Running 0 3m23s kube-system kube-scheduler-node1 1/1 Running 0 3m33s Worker Node 추가 (Join) Worker Node 실행 # Join 명령 가져오기 $ kubeadm token create --print-join-command kubeadm join 192.168.28.15:6443 --token 1ovd36.ft4mefr909iotg0a --discovery-token-ca-cert-hash sha256:82953a3ed178aa8c511792d0e21d9d3283e7575f3d3350a00bea3e34c2b87d29 # Worker node 에서 실행 $ kubeadm join 192.168.28.15:6443 --token 1ovd36.ft4mefr909iotg0a --discovery-token-ca-cert-hash sha256:82953a3ed178aa8c511792d0e21d9d3283e7575f3d3350a00bea3e34c2b87d29 노드 상태 확인 > kubectl get node NAME STATUS ROLES AGE VERSION node1 Ready master 8m50s v1.18.6 node2 Ready <none> 16s v1.18.6 node3 Ready <none> 16s v1.18.6 서비스 배포 : 명령어(CLI) 기반 ...

Ansible-101 Ansible Install Centos 7 기본적으로 Python 2.7 기반으로 설치 됨 추후 pywinrm 패키지가 필요할때, python2-pip 패키지 추가 설치 필요 sudo yum install ansible 대안으로 pip 으로 설치 하는 방법 (user) pip install ansible # --user 용어 정의 Inventory : 관리하는 원격 서버 목록 지정하지 않으면 Default Path 의 hosts 파일을 참조 /etc/ansible/hosts Module : Task 를 실행하는 방법 (모듈) https://docs.ansible.com/ansible/latest/modules/modules_by_category.html{:target="_blank"} e.g. - command, shell, copy, service Playbook : Task 실행 프로세스 정의서 Yaml 파일로 작성 멱등성(idempotence) : 연산을 여러 번 적용하더라도 결과가 달라지지 않는 성질을 의미한다. Ansbile Config Path : /etc/ansible/ansible.cfg Ignore ansible ssh authenticity : ssh 최초 접속시 인증을 host key 체크 생략 https://stackoverflow.com/questions/32297456/how-to-ignore-ansible-ssh-authenticity-checking{:target="_blank"} [defaults] host_key_checking = False Callback plugins minimal stdout [defaults] stdout_callback = minimal 참고 : How to disable strict host key checking in ssh? https://askubuntu.com/questions/87449/how-to-disable-strict-host-key-checking-in-ssh{:target="_blank"} Inventory 구성 Public key 등록 사용 방법 : 키를 생성하고 public 키를 관리되는 서버에 등록 후 사용 # Key 생성 ssh-keygen # Public key 등록 ssh-copy-id [server-ip] Inventory 서버 등록 : /etc/ansible/hosts 기본디렉토리의 Host 파일을 사용하지 않는다면 -i 옵션으로 Inventory 파일 지정 https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html#list-of-behavioral-inventory-parameters{:target="_blank"} # ansible host 파일 예제 192.168.1.5 [db] 192.168.1.10 [webservers] 192.168.1.100 192.168.1.110 # hosts 파일에 등록된 경우 [apiserver] apiserver1 apiserver2 # host명으로 등록 server1 ansible_host=192.168.1.50 server2 ansible_host=192.168.1.51 Host 지정 방법 all : 해당 Inventory의 모든 서버 webservers : webservers 라고 정의된 서버 목록 192.168.1.100 : 192.168.1.100 서버 Ad-hook Execute ansible 명령을 통한 inline 실행 주요 인수 Usage: ansible <host-pattern> [options] -a MODULE_ARGS, --args=MODULE_ARGS module arguments -e EXTRA_VARS, --extra-vars=EXTRA_VARS set additional variables as key=value or YAML/JSON, if filename prepend with @ -f FORKS, --forks=FORKS specify number of parallel processes to use (default=5) -h, --help show this help message and exit -i INVENTORY, --inventory=INVENTORY, --inventory-file=INVENTORY specify inventory host path or comma separated host list. --inventory-file is deprecated -m MODULE_NAME, --module-name=MODULE_NAME module name to execute (default=command) -v, --verbose verbose mode (-vvv for more, -vvvv to enable connection debugging) --version show program's version number, config file location, configured module search path, module location, executable location and exit -b, --become run operations with become (does not imply password prompting) - Example hostname 확인 # webservers 목록 실행 # command 모듈, "hostname" 인수 ansible webservers -m command -a "hostname" # default module (생략가능) : -m command ansible webservers -a "hostname" # 모든 서버 ansible all -a "hostname" 기타 모듈 사용 # ping 모듈 ansible webservers -m ping # copy 모듈 : local → host ansible webservers -m copy -a "src=file.txt dest=/home/cdecl/web/" # service 모듈 : kubelet 서비스를 시작 ansible webservers -m service -a "name=kubelet state=started" Playbook 사용 일련의 Task(작업)를 기술하여, 프로세스를 실행하는 방법 ...