Devops

Docker 내부에서 Docker 실행 Docker in Docker : DinD Docker 내부에서 Docker 를 실행 하기 위해서는 추가적인 호스트 머신의 권한을 획득 해야함 --privileged : Give extended privileges to this container 호스트 머신의 커널 기능 및 장치에 접근 가능하게 됨 --privileged 의 문제점 안전하지 않은 컨테이너로 인한 호스트 머신의 커널이나 장치를 활용하여 취약점에 노출되게 됨 http://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/{:target="_blank"} 서비스 환경에서의 여러가지 약점은 가지고 있지만, Workflow 등의 내부 Devops 툴로서의 유용하다고 판단 Docker in Docker 실행 alpine 이미지 활용 테스트 $ docker run -it --rm --privileged --name=dind alpine Docker 설치 및 실행 # 설치 $ apk add docker fetch http://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz fetch http://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz (1/13) Installing ca-certificates (20191127-r5) ... (13/13) Installing docker (20.10.7-r2) Executing docker-20.10.7-r2.pre-install Executing busybox-1.33.1-r3.trigger Executing ca-certificates-20191127-r5.trigger OK: 253 MiB in 27 packages # 실행 $ dockerd 2> /dev/null & $ ps -ef PID USER TIME COMMAND 1 root 0:00 /bin/sh 125 root 0:00 dockerd 133 root 0:00 containerd --config /var/run/docker/containerd/containerd.toml --log-le 260 root 0:00 ps -ef Docker in Docker 테스트 $ cat /etc/*-release 3.14.2 NAME="Alpine Linux" ID=alpine VERSION_ID=3.14.2 PRETTY_NAME="Alpine Linux v3.14" HOME_URL="https://alpinelinux.org/" BUG_REPORT_URL="https://bugs.alpinelinux.org/" $ docker run --rm ubuntu cat /etc/*-release cat: /etc/alpine-release: No such file or directory NAME="Ubuntu" VERSION="20.04.3 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.3 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal $ docker run --rm centos cat /etc/*-release cat: /etc/alpine-release: No such file or directory NAME="CentOS Linux" VERSION="8" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="8" PLATFORM_ID="platform:el8" PRETTY_NAME="CentOS Linux 8" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:8" HOME_URL="https://centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-8" CENTOS_MANTISBT_PROJECT_VERSION="8" Docker out of Docker : DooD --privileged 사용의 피하기위한 방법 CLI 명령 API인 docker.sock 을 활용, 볼륨을 통해 호스트 머신의 명령을 전달하여 호스트 머신에서 실행 하는 방법 /var/run/docker.sock /var/run/docker.sock docker daemon 에서 인터페이스용으로 노출한 unix domain socket Docker ouf of Docker 실행 # volume 으로 docker.sock 을 연결 $ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock --name=dood alpine Docker 설치 및 실행 # 설치 $ apk add docker fetch http://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz fetch http://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz (1/13) Installing ca-certificates (20191127-r5) ... (13/13) Installing docker (20.10.7-r2) Executing docker-20.10.7-r2.pre-install Executing busybox-1.33.1-r3.trigger Executing ca-certificates-20191127-r5.trigger OK: 253 MiB in 27 packages # 호스트머신에서 명령을 실행 하므로 자신의 컨테이너가 보임 $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1a391d12768c alpine "/bin/sh" 2 minutes ago Up 2 minutes dood $ docker run -d --rm --name=ubuntu ubuntu sleep 10 d14cfc9ff9cd2b4faecfed5af9e1e1b6dd4a7d1497bed4a6d34faaad64a442f9 # 자신의 컨테이너와 동일한 환경에서의 별도 docker 실행 확인 $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d14cfc9ff9cd ubuntu "sleep 10" 3 seconds ago Up 2 seconds ubuntu 1a391d12768c alpine "/bin/sh" 6 minutes ago Up 6 minutes dood

Kubernetes Job 을 활용한 동시작업 Kubernetes Job https://kubernetes.io/ko/docs/concepts/workloads/controllers/job/{:target="_blank"} Pod 를 생성하고, Pod를 통해 성공적으로 종료할떄까지의 일련의 작업실행 Job : 단일 Job 테스트 alpine pod 실행 및 ip 명령어로 IP 확인 command : 명령어 (배열) restartPolicy : Always, OnFailure, Never (default Always) 배치 작업이므로 재시작 하면 안됨 : Never backoffLimit : 실패시 재시작 횟수 (defalut: 6) # time.yml apiVersion: batch/v1 kind: Job metadata: name: ip spec: template: metadata: name: ip spec: containers: - name: ip image: alpine command: ["ip", "a"] restartPolicy: Never backoffLimit: 0 $ kubectl apply -f ip.yml job.batch/ip created $ kubectl get pod NAME READY STATUS RESTARTS AGE ip-5x8qm 0/1 Completed 0 14s 로그 확인 $ kubectl logs ip-5x8qm 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 3: eth0@if58: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP link/ether 9a:f9:d3:9f:32:eb brd ff:ff:ff:ff:ff:ff inet 10.42.0.37/24 brd 10.42.0.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::98f9:d3ff:fe9f:32eb/64 scope link valid_lft forever preferred_lft forever Job : Parallel 동시작업 wrk 를 활용 http 퍼포먼스 테스트 : https://github.com/wg/wrk{:target="_blank"} Image : cdecl/asb{:target="_blank"} Parallel 실행 1 parallelism : 동시 실행 Pod 개수 (default: 1) completions : Pod 완료로 판단하는 개수 (default: parallelism ) apiVersion: batch/v1 kind: Job metadata: name: wrk spec: completions: 4 parallelism: 4 template: metadata: name: wrk spec: containers: - name: wrk image: cdecl/asb command: ["wrk", "-d5", "http://httpbin.org/get"] restartPolicy: Never backoffLimit: 0 실행, 로그 확인 $ kubectl apply -f wrk.yml ...

Kubespray 를 이용한 Kubernetes 설치 (Baremetal) 사전준비 서버 노드 준비 3대 Node 192.168.28.15 192.168.28.16 192.168.28.17 서버 노드 환경설정 : kubernetes-101 참고 Swap 영역을 비활성화 SELinux Disable 방화벽 Disable 브릿지 네트워크 할성화 설치 노드에서의 ssh 접속 허용 : SSH 키복사 ssh-copy-id 설치 준비 Git : Repository Clone Python3 : Inventory 및 환경 설정을 위한 스크립트 실행 Ansible : 원격 실행(설치) ansible-playbook Repository clone 및 Python package install $ git clone https://github.com/kubernetes-sigs/kubespray $ cd kubespray # Package install $ pip3 install -r requirements.txt 설치 방법 1 : inventory_builder inventory_builder Python script 활용 inventory 구성 및 설치 ...

WSL2 에 Docker 설치 (without Docker Desktop) Docker Desktop Docker Desktop 유료화{:target="_blank"} Docker Desktop remains free for small businesses (fewer than 250 employees AND less than $10 million in annual revenue), personal use, education, and non-commercial open source projects. WSL2 이후, Docker Desktop 의 기능을 따로 사용하지 않아도 CLI 기능으로도 충분 개인적으로 Hyper-v VM 사용 권고 Docker install without docker desktop WSL2 Install Windows 10에 Linux용 Windows 하위 시스템 설치 가이드{:target="_blank"} Docker install https://newbedev.com/shell-install-docker-on-wsl-without-docker-desktop-code-example{:target="_blank"} # Update the apt package list. $ sudo apt-get update -y # Install Docker's package dependencies. $ sudo apt-get install -y \ apt-transport-https \ ca-certificates \ curl \ software-properties-common # Download and add Docker's official public PGP key. $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - # Verify the fingerprint. $ sudo apt-key fingerprint 0EBFCD88 # Add the `stable` channel's Docker upstream repository. # # If you want to live on the edge, you can change "stable" below to "test" or # "nightly". I highly recommend sticking with stable! $ sudo add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) \ stable" # Update the apt package list (for the new apt repo). $ sudo apt-get update -y # Install the latest version of Docker CE. $ sudo apt-get install -y docker-ce # Allow your user to access the Docker CLI without needing root access. $ sudo usermod -aG docker $USER # START DOCKER DAEMON $ sudo service docker start $ sudo service docker status $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

Docker 이미지 다중 아키텍처(Multi-Architecture) 빌드 가이드 Docker build Docker 이미지를 빌드할 때는 기본적으로 빌드하는 머신의 CPU 아키텍처(Platform)에 맞춰 빌드됩니다. 이는 Docker의 기본 동작이며, 호스트 시스템의 아키텍처와 일치하는 이미지를 생성합니다. 아키텍처의 중요성: 컨테이너는 호스트 OS의 커널을 공유하지만, CPU 아키텍처에 맞는 바이너리가 필요합니다. 예를 들어, ARM64용으로 빌드된 이미지는 AMD64 시스템에서 직접 실행할 수 없습니다. macOS arm64 테스트 $ uname -a Darwin glass 20.6.0 Darwin Kernel Version 20.6.0: Wed Jun 23 00:26:27 PDT 2021; \ root:xnu-7195.141.2~5/RELEASE_ARM64_T8101 arm64 # Dockerfile FROM alpine RUN uname -a # no-cache, plain output $ docker build . -t cdecl/alpine --no-cache --progress=plain ... #5 [2/2] RUN uname -a #5 sha256:bb35af1757e6b002e99344411e8e8fc700e8760f29f48b4de0f0bb7276ead75d #5 0.238 Linux buildkitsandbox 5.10.25-linuxkit #1 SMP PREEMPT Tue Mar 23 09:24:45 UTC 2021 aarch64 Linux #5 DONE 0.2s ... $ docker image inspect cdecl/alpine -f '{{.Architecture}}' arm64 Platform 변경하여 이미지 빌드 Docker는 --platform 플래그를 통해 타겟 아키텍처를 지정할 수 있습니다. QEMU를 통한 에뮬레이션으로 크로스 플랫폼 빌드가 가능합니다. ...

netstat Host 머신에서 netstat 명령으로 docker container의 네트워크 상태가 확인 안됨 물론 container 내부에서 실행하면 되지만… docker container는 bridge 네트워크 기반으로 운영이 되므로 Host Network 에서는 노출이 안됨 # docker 실행 $ docker run -d -p 8081:80 --name=mvcapp cdecl/mvcapp 4fafaf418f84bf6541a1301b4422f825c58fa20b11d1190e87a3e23eea7a6825 # Host 에서는 publsh port (listen) 정보만 노출 $ netstat -ntl | grep 8081 tcp6 0 0 :::8081 :::* LISTEN # ip 현황 $ ip a ... 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:15:5d:3a:4a:00 brd ff:ff:ff:ff:ff:ff inet 192.168.137.100/24 brd 192.168.137.255 scope global noprefixroute eth0 valid_lft forever preferred_lft forever 4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default <-- docker bridge link/ether 02:42:58:c6:1b:23 brd ff:ff:ff:ff:ff:ff inet 10.1.0.1/24 brd 10.1.0.255 scope global docker0 valid_lft forever preferred_lft forever ... # bridge network $ docker inspect -f '{{.NetworkSettings.Gateway}}' mvcapp 10.1.0.1 # 라우팅 정보 $ netstat -r Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface default gateway 0.0.0.0 UG 0 0 0 eth0 10.1.0.0 0.0.0.0 255.255.255.0 U 0 0 0 docker0 <-- docker bridge 10.42.0.0 0.0.0.0 255.255.255.0 U 0 0 0 cni0 192.168.137.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 nsenter https://github.com/jpetazzo/nsenter{:target="_blank"} 격리되어 있는 namespace enter 진입하는 명령 It is a small tool allowing to enter into namespaces. Technically, it can enter existing namespaces, or spawn a process into a new set of namespaces. ...

Docker Private Registry 구성 Registry 실행 docker-compose.yml version: '3' services: registry: image: registry container_name: registry restart: always ports: - 5000:5000 environment: REGISTRY_HTTP_ADDR: 0.0.0.0:5000 volumes: - registry:/var/lib/registry - ./config.yml:/etc/docker/registry/config.yml volumes: registry: # 실행 $ docker-compose up -d $ curl -s http://localhost:5000/v2/_catalog {"repositories":[]} SSL 적용 version: '3' services: registry: image: registry container_name: private_registry restart: always ports: - 5000:5000 environment: REGISTRY_HTTP_ADDR: 0.0.0.0:5000 REGISTRY_HTTP_TLS_CERTIFICATE: /certs/domain-pem.pem REGISTRY_HTTP_TLS_KEY: /certs/domain-pem.key volumes: - registry:/var/lib/registry - ./config.yml:/etc/docker/registry/config.yml - ./certs:/certs volumes: registry: Image Push 테스트용 이미지 태깅 # get alpine images $ docker pull alpine $ docker pull alpine:3.13 # tagging $ docker tag alpine localhost:5000/cdecl/alpine:latest $ docker tag alpine:3.13 localhost:5000/cdecl/alpine:3.13 $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE alpine latest ae607a46d002 2 weeks ago 5.33MB localhost:5000/cdecl/alpine latest ae607a46d002 2 weeks ago 5.33MB alpine 3.13 81efc9693413 2 months ago 5.35MB localhost:5000/cdecl/alpine 3.13 81efc9693413 2 months ago 5.35MB Image Push # image push $ docker push localhost:5000/cdecl/alpine:3.13 The push refers to repository [localhost:5000/cdecl/alpine] c55d5dbdab40: Layer already exists 3.13: digest: sha256:55fc95a51d97f7b76b124f3b581a58ebf5555d12574f16087de3971a12724dd4 size: 528 $ docker push localhost:5000/cdecl/alpine:latest The push refers to repository [localhost:5000/cdecl/alpine] 5bfa694cc00a: Layer already exists latest: digest: sha256:bd9137c3bb45dbc40cde0f0e19a8b9064c2bc485466221f5e95eb72b0d0cf82e size: 528 # Catalog list $ curl -s http://localhost:5000/v2/_catalog {"repositories":["cdecl/alpine"]} # Tag 정보 $ curl -s http://localhost:5000/v2/cdecl/alpine/tags/list {"name":"cdecl/alpine","tags":["latest","3.13"]} Image Pull $ docker pull localhost:5000/cdecl/alpine Using default tag: latest latest: Pulling from cdecl/alpine Digest: sha256:bd9137c3bb45dbc40cde0f0e19a8b9064c2bc485466221f5e95eb72b0d0cf82e Status: Downloaded newer image for localhost:5000/cdecl/alpine:latest localhost:5000/cdecl/alpine:latest Image Delete Tag 확인 -> Manifests 확인 -> Manifests 이미지 삭제 # Tag 정보 확인 $ curl -s http://localhost:5000/v2/cdecl/alpine/tags/list {"name":"cdecl/alpine","tags":["latest","3.13"]} # Tag의 Manifest 정보확인 - headers 확인 # <registry-url>/v2/<image-path-name>/manifests/<tag> $ curl -s -I -H "Accept: application/vnd.docker.distribution.manifest.v2+json" \ http://localhost:5000/v2/cdecl/alpine/manifests/latest HTTP/1.1 200 OK Content-Length: 528 Content-Type: application/vnd.docker.distribution.manifest.v2+json Docker-Content-Digest: sha256:bd9137c3bb45dbc40cde0f0e19a8b9064c2bc485466221f5e95eb72b0d0cf82e Docker-Distribution-Api-Version: registry/2.0 Etag: "sha256:bd9137c3bb45dbc40cde0f0e19a8b9064c2bc485466221f5e95eb72b0d0cf82e" # Manifest 정보로 이미지 삭제 # Docker-Content-Digest: <sha256:로 시작하는 해쉬 문자열> $ curl -s -XDELETE http://localhost:5000/v2/cdecl/alpine/manifests/sha256:bd9137c3bb45dbc40cde0f0e19a8b9064c2bc485466221f5e95eb72b0d0cf82e # 확인 $ curl -s http://localhost:5000/v2/cdecl/alpine/tags/list {"name":"cdecl/alpine","tags":["3.13"]} Delete marked manifests $ curl -sS http://localhost:5000/v2/cdecl/alpine/tags/list {"name":"cdecl/alpine","tags":null} $ docker exec -it registry bin/registry garbage-collect /etc/docker/registry/config.yml cdecl/alpine 0 blobs marked, 3 blobs and 0 manifests eligible for deletion blob eligible for deletion: sha256:81efc9693413c6292235ea2c29ea07c149701140b98df6c1998bb91d41acf802 INFO[0000] Deleting blob: /docker/registry/v2/blobs/sha256/81/81efc9693413c6292235ea2c29ea07c149701140b98df6c1998bb91d41acf802 go.version=go1.11.2 instance.id=d8bbbd10-232a-457a-9ceb-312b7317da5f service=registry blob eligible for deletion: sha256:55fc95a51d97f7b76b124f3b581a58ebf5555d12574f16087de3971a12724dd4 INFO[0000] Deleting blob: /docker/registry/v2/blobs/sha256/55/55fc95a51d97f7b76b124f3b581a58ebf5555d12574f16087de3971a12724dd4 go.version=go1.11.2 instance.id=d8bbbd10-232a-457a-9ceb-312b7317da5f service=registry blob eligible for deletion: sha256:595b0fe564bb9444ebfe78288079a01ee6d7f666544028d5e96ba610f909ee43 INFO[0000] Deleting blob: /docker/registry/v2/blobs/sha256/59/595b0fe564bb9444ebfe78288079a01ee6d7f666544028d5e96ba610f909ee43 go.version=go1.11.2 instance.id=d8bbbd10-232a-457a-9ceb-312b7317da5f service=registry 위 작업으로는 물리적인 데이터가 삭제 안됨 # 물리적인 파일 삭제 $ docker exec -it registry rm -rf /var/lib/registry/docker/registry/v2/repositories/cdecl Script registry-image-delete.sh #!/bin/bash URI="$1" NAME="$2" TAG="$3" if [[ -z "$URI" ]]; then echo "Usage: $0 <URI> <NAME> [<TAG>]" exit -1 fi if [[ -z "$NAME" ]]; then echo "Usage: $0 $URI <NAME> [<TAG>]" echo curl -sS $URI/v2/_catalog | jq -r '.repositories[]' echo exit -1 fi if [[ -z "$TAG" ]]; then echo "Usage: $0 $URI $NAME [<TAG>]" echo curl -sS $URI/v2/$NAME/tags/list | jq -r '.tags[]' echo exit -1 fi MANIFESTS=$(curl -sS -I -H "Accept: application/vnd.docker.distribution.manifest.v2+json" \ $URI/v2/$NAME/manifests/$TAG | grep -i Docker-Content-Digest | awk '{print $2}') if [[ -z "$MANIFESTS" ]]; then echo "No manifest found for $NAME:$TAG" exit -1 fi echo $MANIFESTS echo echo curl -sS -XDELETE $URI/v2/$NAME/manifests/$MANIFESTS echo

oh-my-zsh 이 느린 경우 해결 방법 (특히 WSL) oh-my-zsh so slow Theme 가 느린 경우는 Pass git 관리 디렉토리 진입시 현저히 느려지는 경우 $ git config --global oh-my-zsh.hide-status 1 $ git config --global oh-my-zsh.hide-dirty 1 $ git config --global -l oh-my-zsh.hide-status=1 oh-my-zsh.hide-dirty=1

Mac 설치 어플리케이션 brew 패키지 관리자 https://brew.sh/index_ko{:target="_blank"} # 설치 $ brew search iterm2 $ brew install iterm2 karabiner-elements brew install brew search karabiner https://karabiner-elements.pqrs.org/{:target="_blank"} 오른쪽 command 버튼 한영키로 매핑 iTerm2 터미널 brew install iterm2 Zsh, Oh My Zsh https://github.com/ohmyzsh/ohmyzsh{:target="_blank"} # zsh install $ brew install zsh # oh my zsh VSCode https://code.visualstudio.com/{:target="_blank"} Docker https://www.docker.com/get-started{:target="_blank"} Dbeaver DB Tool brew install dbeaver-community IINA 동영상 플레이어 brew install iina https://iina.io/{:target="_blank"} Mounty NTFS Mount brew search mounty https://mounty.app/{:target="_blank"} Keka 압축프로그램 brew install keka https://www.keka.io/ko/{:target="_blank"}

vim 최소 설정 .vimrc set ts=4 set autoindent set cindent set hlsearch set showmatch if has("syntax") syntax on endif