containerd

github

containerd 是行业标准的容器运行时,注重简单性、稳健性和可移植性。它可用作 Linux 和 Windows 的守护进程,可以管理其主机系统的完整容器生命周期:镜像传输和存储、容器执行和监督、低级存储和网络附件等。

containerd 是 CNCF 的一名“毕业”成员。

containerd 旨在嵌入到更大的系统中,而不是由开发人员或最终用户直接使用。

NAME:
   containerd -
                    __        _                     __
  _________  ____  / /_____ _(_)___  ___  _________/ /
 / ___/ __ \/ __ \/ __/ __ `/ / __ \/ _ \/ ___/ __  /
/ /__/ /_/ / / / / /_/ /_/ / / / / /  __/ /  / /_/ /
\___/\____/_/ /_/\__/\__,_/_/_/ /_/\___/_/   \__,_/

high performance container runtime


USAGE:
   containerd [global options] command [command options] [arguments...]

VERSION:
   1.6.26

DESCRIPTION:

containerd is a high performance container runtime whose daemon can be started
by using this command. If none of the *config*, *publish*, or *help* commands
are specified, the default action of the **containerd** command is to start the
containerd daemon in the foreground.


A default configuration is used if no TOML configuration is specified or located
at the default file location. The *containerd config* command can be used to
generate the default configuration for containerd. The output of that command
can be used and modified as necessary as a custom configuration.

COMMANDS:
   config    information on the containerd config
   publish   binary to publish events to containerd
   oci-hook  provides a base for OCI runtime hooks to allow arguments to be injected.
   help, h   Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --config value, -c value     path to the configuration file (default: "/etc/containerd/config.toml")
   --log-level value, -l value  set the logging level [trace, debug, info, warn, error, fatal, panic]
   --address value, -a value    address for containerd's GRPC server
   --root value                 containerd root directory
   --state value                containerd state directory
   --help, -h                   show help
   --version, -v                print the version

安装

The official binary releases of containerd are available for the amd64 (also known as x86_64) and arm64 (also known as aarch64) architectures.

Typically, you will have to install runc and CNI plugins from their official sites too.

Step 1: Installing containerd

Download the containerd-<VERSION>-<OS>-<ARCH>.tar.gz archive from https://github.com/containerd/containerd/releases , verify its sha256sum, and extract it under /usr/local:

tar Cxzvf /usr/local containerd-1.6.*-linux-amd64.tar.gz

The cri-containerd-... archives are deprecated, do not work on old Linux distributions, and will be removed in containerd 2.0.

# 1.6 可以取代Setup 1, Setup 2
tar Cxzvf / cri-containerd-1.6.*-linux-amd64.tar.gz

systemd

If you intend to start containerd via systemd, you should also download the containerd.service unit file from https://raw.githubusercontent.com/containerd/containerd/main/containerd.service into /usr/local/lib/systemd/system/containerd.service, and run the following commands:

systemctl daemon-reload
systemctl enable --now containerd
systemctl restart containerd
systemctl status containerd

Step 2: Installing runc

Download the runc.<ARCH> binary from https://github.com/opencontainers/runc/releases , verify its sha256sum, and install it as /usr/local/sbin/runc.

install -m 755 runc.amd64 /usr/local/sbin/runc

Step 3: Installing CNI plugins

Download the cni-plugins-<OS>-<ARCH>-<VERSION>.tgz archive from https://github.com/containernetworking/plugins/releases , verify its sha256sum, and extract it under /opt/cni/bin:

mkdir -p /opt/cni/bin
tar Cxzvf /opt/cni/bin cni-plugins-linux-amd64-v1.*.*.tgz

通过 CLI 与 containerd 交互

有多个用于与 containerd 交互的命令行界面 (CLI) 项目:

Name Community API Target Web site
ctr containerd Native For debugging only (None, see ctr --help to learn the usage)
nerdctl containerd (non-core) Native General-purpose https://github.com/containerd/nerdctl
crictl Kubernetes SIG-node CRI For debugging only https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/crictl.md

虽然该ctr工具与 containerd 捆绑在一起,但需要注意的是,该ctr工具仅用于调试 containerd。该nerdctl工具提供稳定且人性化的用户体验。

Example (ctr):

ctr images pull docker.io/library/redis:alpine
ctr run docker.io/library/redis:alpine redis

Example (nerdctl):

nerdctl run --name redis redis:alpine

自定义containerd

containerd 使用位于 /etc/containerd/config.toml 的配置文件来指定守护进程级别的选项。示例配置文件可在此处 here 找到。

mv -f config.toml /etc/containerd/

可以通过生成默认配置。

mkdir /etc/containerd/
containerd config default > /etc/containerd/config.toml

CLI

ctr

ctrcontainerd的客户端工具。

USAGE:
   ctr [global options] command [command options] [arguments...]

VERSION:
   1.6.26

DESCRIPTION:

ctr is an unsupported debug and administrative client for interacting
with the containerd daemon. Because it is unsupported, the commands,
options, and operations are not guaranteed to be backward compatible or
stable from release to release of the containerd project.

COMMANDS:
   plugins, plugin            provides information about containerd plugins
   version                    print the client and server versions
   containers, c, container   manage containers
   content                    manage content
   events, event              display containerd events
   images, image, i           manage images
   leases                     manage leases
   namespaces, namespace, ns  manage namespaces
   pprof                      provide golang pprof outputs for containerd
   run                        run a container
   snapshots, snapshot        manage snapshots
   tasks, t, task             manage tasks
   install                    install a new package
   oci                        OCI tools
   deprecations
   shim                       interact with a shim directly
   help, h                    Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug                      enable debug output in logs
   --address value, -a value    address for containerd's GRPC server (default: "/run/containerd/containerd.sock") [$CONTAINERD_ADDRESS]
   --timeout value              total timeout for ctr commands (default: 0s)
   --connect-timeout value      timeout for connecting to containerd (default: 0s)
   --namespace value, -n value  namespace to use with commands (default: "default") [$CONTAINERD_NAMESPACE]
   --help, -h                   show help
   --version, -v                print the version

常用命令

# help
ctr c help

alias ctr='ctr -n k8s.io'

# 查看容器
ctr c list

# 查看镜像
ctr image list
ctr image pull ${IMAGE}:${TAG}
ctr image push ${IMAGE}:${TAG}
ctr image tag ${IMAGE}:${TAG} ${REPOSITORY}${IMAGE}:${TAG}

ctr image export ${IMAGE}_${TAG}.img ${IMAGE}:${TAG}
ctr image import ${IMAGE}_${TAG}.img

# 只打印导出语句
crictl images 2>/dev/null | awk -F '  *' '{print "ctr image export "$1"_"$2".img "$1":"$2}'
# 推送所有镜像
crictl images 2>/dev/null | awk -F '  *' '{print "ctr image push "$1":"$2}'

nerdctl

nerdctl是一个与 Docker 兼容的 CLI,适用于containerd

✅ Same UI/UX as docker

✅ Supports Docker Compose (nerdctl compose up)

✅ [Optional] Supports rootless mode, without slirp overhead (bypass4netns)

✅ [Optional] Supports lazy-pulling (Stargz, Nydus, OverlayBD)

✅ [Optional] Supports encrypted images (ocicrypt)

✅ [Optional] Supports P2P image distribution (IPFS) (*1)

✅ [Optional] Supports container image signing and verifying (cosign)

crictl

crictl.md

crictl 是 CRI 兼容的容器运行时命令行接口。 你可以使用它来检查和调试 Kubernetes 节点上的容器运行时和应用程序。 crictl 和它的源代码在 cri-tools 代码库。

crictl 需要带有 CRI 运行时的 Linux 操作系统。

你可以从 cri-tools 发布页面 下载一个压缩的 crictl 归档文件,用于几种不同的架构。 下载与你的 kubernetes 版本相对应的版本。 提取它并将其移动到系统路径上的某个位置,例如 /usr/local/bin/

crictl 命令示例

以下示例展示了一些 crictl 命令及其示例输出。

打印 Pod 清单

打印所有 Pod 的清单:

crictl pods

输出类似于:

POD ID              CREATED              STATE               NAME                         NAMESPACE           ATTEMPT
926f1b5a1d33a       About a minute ago   Ready               sh-84d7dcf559-4r2gq          default             0
4dccb216c4adb       About a minute ago   Ready               nginx-65899c769f-wv2gp       default             0
a86316e96fa89       17 hours ago         Ready               kube-proxy-gblk4             kube-system         0
919630b8f81f1       17 hours ago         Ready               nvidia-device-plugin-zgbbv   kube-system         0

根据名称打印 Pod 清单:

crictl pods --name nginx-65899c769f-wv2gp

输出类似于这样:

POD ID              CREATED             STATE               NAME                     NAMESPACE           ATTEMPT
4dccb216c4adb       2 minutes ago       Ready               nginx-65899c769f-wv2gp   default             0

根据标签打印 Pod 清单:

crictl pods --label run=nginx

输出类似于这样:

POD ID              CREATED             STATE               NAME                     NAMESPACE           ATTEMPT
4dccb216c4adb       2 minutes ago       Ready               nginx-65899c769f-wv2gp   default             0

打印镜像清单

打印所有镜像清单:

crictl images

输出类似于这样:

IMAGE                                     TAG                 IMAGE ID            SIZE
busybox                                   latest              8c811b4aec35f       1.15MB
k8s-gcrio.azureedge.net/hyperkube-amd64   v1.10.3             e179bbfe5d238       665MB
k8s-gcrio.azureedge.net/pause-amd64       3.1                 da86e6ba6ca19       742kB
nginx                                     latest              cd5239a0906a6       109MB

根据仓库打印镜像清单:

crictl images nginx

输出类似于这样:

IMAGE               TAG                 IMAGE ID            SIZE
nginx               latest              cd5239a0906a6       109MB

只打印镜像 ID:

crictl images -q

输出类似于这样:

sha256:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a
sha256:e179bbfe5d238de6069f3b03fccbecc3fb4f2019af741bfff1233c4d7b2970c5
sha256:da86e6ba6ca197bf6bc5e9d900febd906b133eaa4750e6bed647b0fbe50ed43e
sha256:cd5239a0906a6ccf0562354852fae04bc5b52d72a2aff9a871ddb6bd57553569

pull

crictl pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6

tag

crictl 不修改镜像标签,可以通过 crictl 来查找镜像的 ID,然后使用 dockerctr 等工具进行标记。

打印容器清单

打印所有容器清单:

crictl ps -a

输出类似于这样:

CONTAINER ID        IMAGE                                                                                                             CREATED             STATE               NAME                       ATTEMPT
1f73f2d81bf98       busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47                                   7 minutes ago       Running             sh                         1
9c5951df22c78       busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47                                   8 minutes ago       Exited              sh                         0
87d3992f84f74       nginx@sha256:d0a8828cccb73397acb0073bf34f4d7d8aa315263f1e7806bf8c55d8ac139d5f                                     8 minutes ago       Running             nginx                      0
1941fb4da154f       k8s-gcrio.azureedge.net/hyperkube-amd64@sha256:00d814b1f7763f4ab5be80c58e98140dfc69df107f253d7fdd714b30a714260a   18 hours ago        Running             kube-proxy                 0

打印正在运行的容器清单:

crictl ps

输出类似于这样:

CONTAINER ID        IMAGE                                                                                                             CREATED             STATE               NAME                       ATTEMPT
1f73f2d81bf98       busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47                                   6 minutes ago       Running             sh                         1
87d3992f84f74       nginx@sha256:d0a8828cccb73397acb0073bf34f4d7d8aa315263f1e7806bf8c55d8ac139d5f                                     7 minutes ago       Running             nginx                      0
1941fb4da154f       k8s-gcrio.azureedge.net/hyperkube-amd64@sha256:00d814b1f7763f4ab5be80c58e98140dfc69df107f253d7fdd714b30a714260a   17 hours ago        Running             kube-proxy                 0

在正在运行的容器上执行命令

crictl exec -i -t 1f73f2d81bf98 ls

输出类似于这样:

bin   dev   etc   home  proc  root  sys   tmp   usr   var

获取容器日志

获取容器的所有日志:

crictl logs 87d3992f84f74

输出类似于这样:

10.240.0.96 - - [06/Jun/2018:02:45:49 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
10.240.0.96 - - [06/Jun/2018:02:45:50 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
10.240.0.96 - - [06/Jun/2018:02:45:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"

获取最近的 N 行日志:

crictl logs --tail=1 87d3992f84f74

输出类似于这样:

10.240.0.96 - - [06/Jun/2018:02:45:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"

CRI-O

安装

Add the CRI-O repository
cat <<EOF | tee /etc/yum.repos.d/cri-o.repo
[cri-o]
name=CRI-O
baseurl=https://download.opensuse.org/repositories/isv:/cri-o:/stable:/$CRIO_VERSION/rpm/
enabled=1
gpgcheck=1
gpgkey=https://download.opensuse.org/repositories/isv:/cri-o:/stable:/$CRIO_VERSION/rpm/repodata/repomd.xml.key
EOF
Install package dependencies from the official repositories
dnf install -y container-selinux
Install the packages
dnf install -y cri-o kubelet kubeadm kubectl
Start CRI-O
systemctl start crio.service
Bootstrap a cluster
swapoff -a
modprobe br_netfilter
sysctl -w net.ipv4.ip_forward=1

kubeadm init

Docker

cri-docker

https://www.zhihu.com/collection/845458021

在Kubernetes早期版本中,Docker作为默认容器运行时,并在Kubelet程序中开发了一个名为“Dockershim”的代理程序,负责Kubelet与Docker通信。

随着Kubernetes生态系统的发展,涌现出多种容器运行时,例如containerd、cri-o、rkt等。为了支持这些容器运行时,Kubernetes引入CRI(Container Runtime Interface,容器运行时接口)标准,使得第三方容器运行时只需对接CRI即可与Kubernetes集成。

后来,在Kubernetes 1.20版本发布时宣布:为了优化核心代码,减少维护负担,将在1.24版本中正式移除“Dockershim”,而当时Docker又不支持CRI,这就意味着Kubernetes无法再Docker作为容器运行时。Docker官方为了解决这个问题,与Mirantis公司合作,开发了一个名为“cri-dockerd”的代理程序,负责Kubelet与Docker通信。

因此,从Kubernetes 1.24版本及更高版本开始,使用Docker作为容器运行时,需要安装cri-dockerd。可以在GitHub Releases页面(https://github.com/Mirantis/cri-dockerd/releases)找到适用于你系统平台版本的安装包,下载后上传到所有节点上并进行安装: