Gitlab CI/CD 101 Gitlab 에서 제공하는 CI/CD 목적의 Workflow 툴 Auto DevOps or gitlab-runner 에서 실행 Setup CI/CD 를 통해 세팅 .gitlab-ci.yml 파일에 기술 Gitlab-Runner gitlab-runner : .gitlab-ci.yml 기반 파이프 라인 구성 Shared Runners : gitlab.com 에서 hosting 해주는 Runner Self hosting Runners : 별도 머신을 통해 Runner 설치 Gitlab-Runner 세팅 (Self hosting) Installing the Runner https://docs.gitlab.com/runner/install/linux-repository.html{:target="_blank"} Registering Runners https://docs.gitlab.com/runner/register/index.html{:target="_blank"} Interactive register runner $ sudo gitlab-runner register Runtime platform arch=amd64 os=linux pid=120146 revision=c5874a4b version=12.10.2 Running in system-mode. Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/): http://hqgit.inpark.kr/ Please enter the gitlab-ci token for this runner: xxxxxxxxxxxxxxxxxxxxxxxxxxx Please enter the gitlab-ci description for this runner: ci-test runner Please enter the gitlab-ci tags for this runner (comma separated): centos24,ci-test,cdecl Registering runner... succeeded runner=WpQDakzK Please enter the executor: shell, kubernetes, parallels, docker, docker-ssh, ssh, virtualbox, docker+machine, docker-ssh+machine, custom: shell Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! # inline sudo gitlab-runner register \ --non-interactive \ --url "https://gitlab.com/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ --executor "docker" \ --docker-image alpine:latest \ --description "docker-runner" \ --tag-list "docker" \ sudo gitlab-runner register \ --non-interactive \ --url "http://centos.cdecl.net/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ --executor "docker" \ --docker-image alpine \ --description "docker-runner" \ --tag-list "docker" \ --env "DOCKER_TLS_CERTDIR=" \ --docker-privileged=true \ --docker-volumes "/ansible:/ansible" \ --docker-extra-hosts "centos.cdecl.net:192.168.0.20" Pipeline Configuration Basic GitLab CI/CD Pipeline Configuration Reference https://docs.gitlab.com/ee/ci/yaml/{:target="_blank"} Pipeline 기본적으로 git checkout 실행 Github Action 과 다르게 매뉴얼 실행 버튼 존재 image: ubuntu stages: # statge 정의 - build - test - deploy before_script: # - echo "Before script section" - echo "For example you might run an update here or install a build dependency" - echo "Or perhaps you might print out some debugging details" after_script: - echo "After script section" - echo "For example you might do some cleanup here" build_stage: stage: build script: - echo "Do your build here" test_stage1: stage: test script: - echo "Do a test here" - echo "For example run a test suite" test_stage2: stage: test script: - echo "Do another parallel test here" - echo "For example run a lint test" deploy_stage: stage: deploy script: - echo "Do your deploy here" ...

Kubernetes 설치 및 운영 101 사전 준비 Kubernetes 설치 전 서버 구성 변경 참고 : https://www.mirantis.com/blog/how-install-kubernetes-kubeadm/{:target="_blank"} Swap 영역을 비활성화 # 일시적인 설정 $ sudo swapoff -a # 영구적인 설정, 아래 swap 파일 시스템을 주석처리 $ sudo vi /etc/fstab ... # /dev/mapper/kube--master--vg-swap_1 none swap sw 0 0 SELinux Disable # 임시 $ sudo setenforce 0 # 영구 $ sudo vi /etc/sysconfig/selinux ... SELinux=disabled 방화벽 Disable $ sudo systemctl disable firewalld $ sudo systemctl stop firewalld 브릿지 네트워크 할성화 # Centos $ sudo vim /etc/sysctl.d/k8s.conf net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 # Ubuntu $ sudo vim /etc/ufw/sysctl.conf net/bridge/bridge-nf-call-ip6tables = 1 net/bridge/bridge-nf-call-iptables = 1 net/bridge/bridge-nf-call-arptables = 1 Docker Install Centos Install : https://docs.docker.com/engine/install/centos/{:target="_blank"} Cgroup 드라이버 이슈 최신 Kubernetes는 docker cgroup driver를 cgroupfs → systemd 변경 필요 Master Init 및 Worker Join 시 WARNING 발생 https://kubernetes.io/ko/docs/setup/production-environment/container-runtimes/{:target="_blank"} kubeadm init --pod-network-cidr 10.244.0.0/16 ... [init] Using Kubernetes version: v1.19.3 [preflight] Running pre-flight checks [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". ... ... 드라이버 변경 작업 /etc/docker/daemon.json 파일 작성 $ cat <<EOF | sudo tee /etc/docker/daemon.json { "exec-opts": ["native.cgroupdriver=systemd"], "log-driver": "json-file", "log-opts": { "max-size": "100m" }, "storage-driver": "overlay2", "storage-opts": [ "overlay2.override_kernel_check=true" ] } EOF # 도커 재시작 $ sudo systemctl restart docker # 확인 $ sudo docker info | grep -i cgroup Cgroup Driver: systemd Kubernetes (kubeadm, kubelet, kubectl) 설치 참고 : https://kubernetes.io/docs/setup/independent/install-kubeadm/{:target="_blank"} Kubernetes 설치 : Centos7 기준 Docker 설치 sudo yum install -y docker sudo systemctl enable docker && systemctl start docker sudo usermod -aG docker $USER kubeadm, kubelet, kubectl : Repo 추가 및 패키지 설치 $ cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg EOF $ sudo yum install -y kubelet kubeadm kubectl $ sudo systemctl enable kubelet && systemctl start kubelet # 버전이 안맞을 경우 지정 # sudo yum install kubelet-[version] kubeadm-[version] kubectl-[version] kubectl 자동완성 # sh source <(kubectl completion sh) echo "source <(kubectl completion sh)" >> ~/.shrc # zsh source <(kubectl completion zsh) echo "if [ $commands[kubectl] ]; then source <(kubectl completion zsh); fi" >> ~/.zshrc Master Node Init 및 Worker Node Join Master Node 설정 Master 초기화 네트워크 클래스 대역을 설정 필요 : --pod-network-cidr 10.244.0.0/16 sudo kubeadm init --pod-network-cidr 10.244.0.0/16 Kubectl 사용 : To start using your cluster.. 아래 항목 3줄 실행 [init] Using Kubernetes version: v1.10.5 ... To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config ... You can now join any number of machines by running the following on each node as root: kubeadm join 192.168.28.15:6443 --token 1ovd36.ft4mefr909iotg0a --discovery-token-ca-cert-hash sha256:82953a3ed178aa8c511792d0e21d9d3283e7575f3d3350a00bea3e34c2b87d29 Pod 상태 확인 coredns STATUS → Pending (∵ Overlay network 미설치) $ kubectl get po -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-66bff467f8-ktvsz 0/1 Pending 0 19s kube-system coredns-66bff467f8-nvvjz 0/1 Pending 0 19s kube-system etcd-node1 1/1 Running 0 29s kube-system kube-apiserver-node1 1/1 Running 0 29s kube-system kube-controller-manager-node1 1/1 Running 0 29s kube-system kube-proxy-s582x 1/1 Running 0 19s kube-system kube-scheduler-node1 1/1 Running 0 29s Overlay network : Calico 설치 Overlay network 종류 https://kubernetes.io/docs/concepts/cluster-administration/networking/{:target="_blank"} Install Calico for on-premises deployments https://docs.projectcalico.org/getting-started/kubernetes/self-managed-onprem/onpremises{:target="_blank"} # Install Calico for on-premises deployments $ kubectl apply -f https://docs.projectcalico.org/manifests/calico-typha.yaml coredns 서비스가 정상적으로 Running $ kubectl get po -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-799fb94867-bcntz 0/1 CrashLoopBackOff 3 2m6s kube-system calico-node-jtcmt 0/1 Running 1 2m7s kube-system calico-typha-6bc9dd6468-x2hjj 0/1 Pending 0 2m6s kube-system coredns-66bff467f8-ktvsz 0/1 Running 0 3m23s kube-system coredns-66bff467f8-nvvjz 0/1 Running 0 3m23s kube-system etcd-node1 1/1 Running 0 3m33s kube-system kube-apiserver-node1 1/1 Running 0 3m33s kube-system kube-controller-manager-node1 1/1 Running 0 3m33s kube-system kube-proxy-s582x 1/1 Running 0 3m23s kube-system kube-scheduler-node1 1/1 Running 0 3m33s Worker Node 추가 (Join) Worker Node 실행 # Join 명령 가져오기 $ kubeadm token create --print-join-command kubeadm join 192.168.28.15:6443 --token 1ovd36.ft4mefr909iotg0a --discovery-token-ca-cert-hash sha256:82953a3ed178aa8c511792d0e21d9d3283e7575f3d3350a00bea3e34c2b87d29 # Worker node 에서 실행 $ kubeadm join 192.168.28.15:6443 --token 1ovd36.ft4mefr909iotg0a --discovery-token-ca-cert-hash sha256:82953a3ed178aa8c511792d0e21d9d3283e7575f3d3350a00bea3e34c2b87d29 노드 상태 확인 > kubectl get node NAME STATUS ROLES AGE VERSION node1 Ready master 8m50s v1.18.6 node2 Ready <none> 16s v1.18.6 node3 Ready <none> 16s v1.18.6 서비스 배포 : 명령어(CLI) 기반 ...

Flask 에서 SQLAlchemy 사용시 Query 팁 설정 Connection 기본 DBMS 연결 정보 # MySQL app.config['SQLALCHEMY_DATABASE_URI'] = r'mysql+pymysql://user:passwd@address:3306/db_name?charset=UTF8MB4' db = SQLAlchemy() db.init_app(app) Bind 추가 기본 DBMS 이외, 추가적인 DB 정보 추가 app.config['SQLALCHEMY_BINDS'] = { 'dbms1': r'mysql+pymysql://user:passwd@address1:3306/db_name?charset=UTF8MB4', 'dbms2': r'mysql+pymysql://user:passwd@address2:3306/db_name?charset=UTF8MB4' } # Model Example class Network(db.Model): __tablename__ = 'network' __bind_key__ = 'dbms1' ip = db.Column(db.String(128), primary_key=True) switch = db.Column(db.String(128)) doc = db.Column(db.JSON) AlchemyEncoder SQLAlchemy JSON Encoder class AlchemyEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj.__class__, DeclarativeMeta): # an SQLAlchemy class fields = {} for field in [x for x in dir(obj) if not x.startswith('_') and x != 'metadata']: data = obj.__getattribute__(field) try: json.dumps(data) # this will fail on non-encodable values, like other classes fields[field] = data except TypeError: if isinstance(data, datetime): fields[field] = data.strftime('%Y-%m-%d %H:%M:%S') elif isinstance(data, date): fields[field] = data.strftime('%Y-%m-%d') else: fields[field] = None return fields return json.JSONEncoder.default(self, obj) Flask Basic Route @app.route('/api/network/') def api_network(): cur = db.session.query(Network) return json.dumps({ 'data': cur.all() }, cls=AlchemyEncoder) DB Model Generate # 해당 DB 전체 Table 대상 $ ./venv/bin/flask-sqlacodegen 'mysql+pymysql://user:passwd@address2:3306/db_name' --flask # 해당 DB 테이블 지정 $ ./venv/bin/flask-sqlacodegen 'mysql+pymysql://user:passwd@address2:3306/db_name' --flask --table network,other_table Query Sample Select All cur = db.session.query(Network) Select Filter, Sort # Filter cur = db.session.query(network).filter(network.switch == switch_name) cur = db.session.query(network).filter(network.ip.like('{}%'.format(ip))) cur = db.session.query(network).filter(network.switch.in_(('AA', 'BB'))) # Text filter, json cur = db.session.query(network).filter(text(r'''doc->>"$.name" <> '' ''')) # OR cur = db.session.query(network).filter(or_(network.ip == ip, network.switch == switch_name)) # AND cur = db.session.query(network).filter(and_(network.ip == ip, network.switch == switch_name)) # SORT cur = db.session.query(network).filter(network.switch == switch_name) \ .order_by(network.ip.desc()) Join # Join cur = db.session.query(Network, NetworkSwitch) \ .filter(Network.ip == NetworkSwitch.ip, Network.switch == NetworkSwitch.switch) # Outer Join cur = db.session.query(Network) \ .outerjoin(NetworkSwitch, and_(Network.ip == NetworkSwitch.ip, Network.switch == NetworkSwitch.switch)) # Self Join, Table Alias NetworkAlias = aliased(Network) cur = db.session.query(Network, NetworkAlias) \ .filter(Network.ip == NetworkAlias.ip, Network.switch == NetworkAlias.switch)

Ansible-101 Ansible Install Centos 7 기본적으로 Python 2.7 기반으로 설치 됨 추후 pywinrm 패키지가 필요할때, python2-pip 패키지 추가 설치 필요 sudo yum install ansible 대안으로 pip 으로 설치 하는 방법 (user) pip install ansible # --user 용어 정의 Inventory : 관리하는 원격 서버 목록 지정하지 않으면 Default Path 의 hosts 파일을 참조 /etc/ansible/hosts Module : Task 를 실행하는 방법 (모듈) https://docs.ansible.com/ansible/latest/modules/modules_by_category.html{:target="_blank"} e.g. - command, shell, copy, service Playbook : Task 실행 프로세스 정의서 Yaml 파일로 작성 멱등성(idempotence) : 연산을 여러 번 적용하더라도 결과가 달라지지 않는 성질을 의미한다. Ansbile Config Path : /etc/ansible/ansible.cfg Ignore ansible ssh authenticity : ssh 최초 접속시 인증을 host key 체크 생략 https://stackoverflow.com/questions/32297456/how-to-ignore-ansible-ssh-authenticity-checking{:target="_blank"} [defaults] host_key_checking = False Callback plugins minimal stdout [defaults] stdout_callback = minimal 참고 : How to disable strict host key checking in ssh? https://askubuntu.com/questions/87449/how-to-disable-strict-host-key-checking-in-ssh{:target="_blank"} Inventory 구성 Public key 등록 사용 방법 : 키를 생성하고 public 키를 관리되는 서버에 등록 후 사용 # Key 생성 ssh-keygen # Public key 등록 ssh-copy-id [server-ip] Inventory 서버 등록 : /etc/ansible/hosts 기본디렉토리의 Host 파일을 사용하지 않는다면 -i 옵션으로 Inventory 파일 지정 https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html#list-of-behavioral-inventory-parameters{:target="_blank"} # ansible host 파일 예제 192.168.1.5 [db] 192.168.1.10 [webservers] 192.168.1.100 192.168.1.110 # hosts 파일에 등록된 경우 [apiserver] apiserver1 apiserver2 # host명으로 등록 server1 ansible_host=192.168.1.50 server2 ansible_host=192.168.1.51 Host 지정 방법 all : 해당 Inventory의 모든 서버 webservers : webservers 라고 정의된 서버 목록 192.168.1.100 : 192.168.1.100 서버 Ad-hook Execute ansible 명령을 통한 inline 실행 주요 인수 Usage: ansible <host-pattern> [options] -a MODULE_ARGS, --args=MODULE_ARGS module arguments -e EXTRA_VARS, --extra-vars=EXTRA_VARS set additional variables as key=value or YAML/JSON, if filename prepend with @ -f FORKS, --forks=FORKS specify number of parallel processes to use (default=5) -h, --help show this help message and exit -i INVENTORY, --inventory=INVENTORY, --inventory-file=INVENTORY specify inventory host path or comma separated host list. --inventory-file is deprecated -m MODULE_NAME, --module-name=MODULE_NAME module name to execute (default=command) -v, --verbose verbose mode (-vvv for more, -vvvv to enable connection debugging) --version show program's version number, config file location, configured module search path, module location, executable location and exit -b, --become run operations with become (does not imply password prompting) - Example hostname 확인 # webservers 목록 실행 # command 모듈, "hostname" 인수 ansible webservers -m command -a "hostname" # default module (생략가능) : -m command ansible webservers -a "hostname" # 모든 서버 ansible all -a "hostname" 기타 모듈 사용 # ping 모듈 ansible webservers -m ping # copy 모듈 : local → host ansible webservers -m copy -a "src=file.txt dest=/home/cdecl/web/" # service 모듈 : kubelet 서비스를 시작 ansible webservers -m service -a "name=kubelet state=started" Playbook 사용 일련의 Task(작업)를 기술하여, 프로세스를 실행하는 방법 ...