kubernetes CRI とは

概要

CRI(Container Runtime Interface) は、kubelet(kubernetes のコンポーネント) がコンテナランタイムを操作するためのインタフェース。
コンテナランタイムは kubelet からの命令を受けて、イメージを取得したりコンテナを実行するもの。メジャーなコンテナランタイムには、containerd や crio や dockershim(v1.20 からは非推奨)がある。
- 実際にコンテナを実行する部分は、OCI という別の規格が存在する。
- メジャーな OCI ランタイムには、runc(docker も内部で利用しているやつ。シンプルにカーネルの分離機能を利用しているやつ)や、gVisor(サンドボックス上で実行するやつ。カーネルをアプリケーションレベルで再実装した的なやつ) や、Kata Container(KVM 上で実行するやつ)などがある。
- コンテナランタイムと OCI ランタイムの組み合わせは、サポート状況を確認する必要がある。
CRI があることで、ランタイムを任意のものに差し替えられる(kubelet の --container-runtime-endpoint で変えられる)。また、kubernetes 本体とは独立して開発・リリースできる。

ざっくりと以下のようになっている。

kubelet
↓
CRI(gRPC)
↓
(High level runtime)containerd/crio/dockershim
↓
OCI(バイナリ実行)
↓
(Low level runtime)runc/gVisor/Kata Container

このリポジトリには、gRPC で通信するための proto ファイルが配置されている。 Go 言語による実装が多いからか、proto ファイルを基に生成した pb.go ファイルも配置されていて(pb.go は protoc コマンドで proto から生成するらしい。参考 Protocol Buffer Basics: Go)。kubelet や containerd は pb.go をインポートして利用していた。

たとえば、クライアント側のkubelet では、次のように利用している。

import (
        ...
    runtimeapi "k8s.io/cri-api/pkg/apis/runtime/v1alpha2"
        ...
)

func (r *remoteImageService) PullImage(image *runtimeapi.ImageSpec, auth *runtimeapi.AuthConfig, podSandboxConfig *runtimeapi.PodSandboxConfig) (string, error) {
    ctx, cancel := getContextWithCancel()
    defer cancel()

    resp, err := r.imageClient.PullImage(ctx, &runtimeapi.PullImageRequest{
        Image:         image,
        Auth:          auth,
        SandboxConfig: podSandboxConfig,
    })

また、サーバ側のcontainerdでは、次のように利用している。

import (
        ...
        runtime "k8s.io/cri-api/pkg/apis/runtime/v1alpha2"
        ...
)

func (c *criService) PullImage(ctx context.Context, r *runtime.PullImageRequest) (*runtime.PullImageResponse, error) {
        imageRef := r.GetImage().GetImage()
        namedRef, err := distribution.ParseDockerRef(imageRef)
        if err != nil {
                return nil, errors.Wrapf(err, "failed to parse image reference %q", imageRef)
        }
        ref := namedRef.String()
        if ref != imageRef {
                lo

gRPC でどういうリクエストを流しているのか(たとえば PullImageRequest)は、v1alpha2/api.pb.goに定義されている。

type PullImageRequest struct {
        // Spec of the image.
        Image *ImageSpec `protobuf:"bytes,1,opt,name=image,proto3" json:"image,omitempty"`
        // Authentication configuration for pulling the image.
        Auth *AuthConfig `protobuf:"bytes,2,opt,name=auth,proto3" json:"auth,omitempty"`
        // Config of the PodSandbox, which is used to pull image in PodSandbox context.
        SandboxConfig        *PodSandboxConfig `protobuf:"bytes,3,opt,name=sandbox_config,json=sandboxConfig,proto3" json:"sandbox_config,omitempty"`
        XXX_NoUnkeyedLiteral struct{}          `json:"-"`
        XXX_sizecache        int32             `json:"-"`
}

CRI に定義されている API

api.proto には、RuntimeService と ImageService というサービスが定義されている。名前のまんまだけど、RuntimeService が PodSandbox およびコンテナのライフサイクルを管理するもの、ImageService がコンテナイメージを管理するもの。

proto の定義を以下に抜き出してみる。実際の proto ファイルにはコメントが沢山記載されていてどんな関数か大体わかるようになってるが、紙面(紙じゃないけど)の都合で関数名のみ抜粋する。

// Runtime service defines the public APIs for remote container runtimes
service RuntimeService {
    rpc Version(VersionRequest) returns (VersionResponse) {}
    rpc RunPodSandbox(RunPodSandboxRequest) returns (RunPodSandboxResponse) {}
    rpc StopPodSandbox(StopPodSandboxRequest) returns (StopPodSandboxResponse) {}
    rpc RemovePodSandbox(RemovePodSandboxRequest) returns (RemovePodSandboxResponse) {}
    rpc PodSandboxStatus(PodSandboxStatusRequest) returns (PodSandboxStatusResponse) {}
    rpc ListPodSandbox(ListPodSandboxRequest) returns (ListPodSandboxResponse) {}

    rpc CreateContainer(CreateContainerRequest) returns (CreateContainerResponse) {}
    rpc StartContainer(StartContainerRequest) returns (StartContainerResponse) {}
    rpc StopContainer(StopContainerRequest) returns (StopContainerResponse) {}
    rpc RemoveContainer(RemoveContainerRequest) returns (RemoveContainerResponse) {}
    rpc ListContainers(ListContainersRequest) returns (ListContainersResponse) {}
    rpc ContainerStatus(ContainerStatusRequest) returns (ContainerStatusResponse) {}
    rpc UpdateContainerResources(UpdateContainerResourcesRequest) returns (UpdateContainerResourcesResponse) {}
    rpc ReopenContainerLog(ReopenContainerLogRequest) returns (ReopenContainerLogResponse) {}

    rpc ExecSync(ExecSyncRequest) returns (ExecSyncResponse) {}
    rpc UpdateContainerResources(UpdateContainerResourcesRequest) returns (UpdateContainerResourcesResponse) {}
    rpc ReopenContainerLog(ReopenContainerLogRequest) returns (ReopenContainerLogResponse) {}

    rpc ExecSync(ExecSyncRequest) returns (ExecSyncResponse) {}
    rpc Exec(ExecRequest) returns (ExecResponse) {}
    rpc Attach(AttachRequest) returns (AttachResponse) {}
    rpc PortForward(PortForwardRequest) returns (PortForwardResponse) {}

    rpc ContainerStats(ContainerStatsRequest) returns (ContainerStatsResponse) {}
    rpc ListContainerStats(ListContainerStatsRequest) returns (ListContainerStatsResponse) {}

    rpc UpdateRuntimeConfig(UpdateRuntimeConfigRequest) returns (UpdateRuntimeConfigResponse) {}

    rpc Status(StatusRequest) returns (StatusResponse) {}
}


// ImageService defines the public APIs for managing images.
service ImageService {
    rpc ListImages(ListImagesRequest) returns (ListImagesResponse) {}
    rpc ImageStatus(ImageStatusRequest) returns (ImageStatusResponse) {}
    rpc PullImage(PullImageRequest) returns (PullImageResponse) {}
    rpc RemoveImage(RemoveImageRequest) returns (RemoveImageResponse) {}
    rpc ImageFsInfo(ImageFsInfoRequest) returns (ImageFsInfoResponse) {}
}

ところで RuntimeService の PodSandbox ってなに？Pod じゃないの？という感じだけど、そういうもんらしい。 Kubernetes では Pod が最小単位であり Pod 単位でリソースが分離される(Pod に複数のコンテナが存在できる)。そしてその分離機能は、ランタイムごとに仮想マシンだったり namespace で実現される。これを PodSandbox と言ってるっぽい。

コンテナ作成～削除の流れとしては、だいたい以下のようになってた。詳細は kubelet.go が利用している runtime manager あたりが参考になる。

RuntimeService.RunPodSandbox で Pod 用の Sandbox を作成する。
- このときに CNI を実行してネットワークの設定もする。
ImageService.PullImage でコンテナイメージを取得する。
RuntimeService.CreateContainer で PodSandbox 内にコンテナを作成する。
RuntimeService.StartContainer でコンテナを開始する。
RuntimeService.StopContainer でコンテナを停止する。冪等。
RuntimeService.StopPodSandbox で PodSandbox 上のプロセスを停止する。コンテナがまだ動いてれば強制で停止する。冪等なので Status チェックしながら何回か呼び出すっぽい。
RuntimeService.RemoveContainer でコンテナを削除する。コンテナがまだ動いてれば強制で停止する。冪等なので Status チェックしながら何回か呼び出すっぽい。
RuntimeService.RemovePodSandbox で Pod 用の Sandbox を削除する。コンテナがまだ動いてれば強制で停止する。冪等なので Status チェックしながら何回か呼び出すっぽい。

microk8s で containerd を利用したときの containerd のログを以下に貼り付ける。

microk8s.kubectl create deployment --image=nginx app を実行したときのログ。

# journalctl -f -u snap.microk8s.daemon-containerd.service | grep -v "Finish piping " | grep -v "ExecSync" | grep -v "Exec process"

May 07 11:35:37 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:35:37.338956445Z" level=info msg="RunPodsandbox for &PodSandboxMetadata{Name:app-65f6b9d4d9-g26wm,Uid:691bca13-bd51-415a-af3a-9dc02e2477ad,Namespace:default,Attempt:0,}"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679262]: 2021-05-07 11:35:37.421 [INFO][679262] plugin.go 129: Extracted identifiers EndpointIDs=&utils.WEPIdentifiers{Namespace:"default", WEPName:"", WorkloadEndpointIdentifiers:names.WorkloadEndpointIdentifiers{Node:"test-machine", Orchestrator:"k8s", Endpoint:"eth0", Workload:"", Pod:"app-65f6b9d4d9-g26wm", ContainerID:"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"}}
May 07 11:35:37 test-machine microk8s.daemon-containerd[679262]: 2021-05-07 11:35:37.441 [INFO][679262] plugin.go 220: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {test--machine-k8s-app--65f6b9d4d9--g26wm-eth0 app-65f6b9d4d9- default  691bca13-bd51-415a-af3a-9dc02e2477ad 50704 0 2021-05-07 11:35:36 +0000 UTC <nil> <nil> map[app:app pod-template-hash:65f6b9d4d9 projectcalico.org/namespace:default projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:default] map[] [] []  []} {k8s  test-machine  app-65f6b9d4d9-g26wm eth0 [] []   [kns.default ksa.default.default] cali99de102e059  []}} ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Namespace="default" Pod="app-65f6b9d4d9-g26wm" WorkloadEndpoint="test--machine-k8s-app--65f6b9d4d9--g26wm-"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679262]: 2021-05-07 11:35:37.441 [INFO][679262] k8s.go 60: Extracted identifiers for CmdAddK8s ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Namespace="default" Pod="app-65f6b9d4d9-g26wm" WorkloadEndpoint="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679262]: 2021-05-07 11:35:37.446 [INFO][679262] k8s.go 775: namespace info &Namespace{ObjectMeta:{default   /api/v1/namespaces/default 899bb9d7-0106-4cb4-a1b4-a32c71df8383 75 0 2021-05-06 23:48:04 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-apiserver Update v1 2021-05-06 23:48:04 +0000 UTC FieldsV1 FieldsV1{Raw:*[123 34 102 58 115 116 97 116 117 115 34 58 123 34 102 58 112 104 97 115 101 34 58 123 125 125 125],}}]},Spec:NamespaceSpec{Finalizers:[kubernetes],},Status:NamespaceStatus{Phase:Active,Conditions:[]NamespaceCondition{},},}
May 07 11:35:37 test-machine microk8s.daemon-containerd[679262]: 2021-05-07 11:35:37.448 [INFO][679262] k8s.go 784: pod info &Pod{ObjectMeta:{app-65f6b9d4d9-g26wm app-65f6b9d4d9- default /api/v1/namespaces/default/pods/app-65f6b9d4d9-g26wm 691bca13-bd51-415a-af3a-9dc02e2477ad 50704 0 2021-05-07 11:35:36 +0000 UTC <nil> <nil> map[app:app pod-template-hash:65f6b9d4d9] map[] [{apps/v1 ReplicaSet app-65f6b9d4d9 83e34521-a6bd-40c4-9fdb-8e053459ee42 0xc00031ed67 0xc00031ed68}] []  [{kube-controller-manager Update v1 2021-05-07 11:35:36 +0000 UTC FieldsV1 FieldsV1{Raw:*[123 34 102 58 109 101 116 97 100 97 116 97 34 58 123 34 102 58 103 101 110 101 114 97 116 101 78 97 109 101 34 58 123 125 44 34 102 58 108 97 98 101 108 115 34 58 123 34 46 34 58 123 125 44 34 102 58 97 112 112 34 58 123 125 44 34 102 58 112 111 100 45 116 101 109 112 108 97 116 101 45 104 97 115 104 34 58 123 125 125 44 34 102 58 111 119 110 101 114 82 101 102 101 114 101 110 99 101 115 34 58 123 34 46 34 58 123 125 44 34 107 58 123 92 34 117 105 100 92 34 58 92 34 56 51 101 51 52 53 50 49 45 97 54 98 100 45 52 48 99 52 45 57 102 100 98 45 56 101 48 53 51 52 53 57 101 101 52 50 92 34 125 34 58 123 34 46 34 58 123 125 44 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 98 108 111 99 107 79 119 110 101 114 68 101 108 101 116 105 111 110 34 58 123 125 44 34 102 58 99 111 110 116 114 111 108 108 101 114 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 125 125 44 34 102 58 115 112 101 99 34 58 123 34 102 58 99 111 110 116 97 105 110 101 114 115 34 58 123 34 107 58 123 92 34 110 97 109 101 92 34 58 92 34 110 103 105 110 120 92 34 125 34 58 123 34 46 34 58 123 125 44 34 102 58 105 109 97 103 101 34 58 123 125 44 34 102 58 105 109 97 103 101 80 117 108 108 80 111 108 105 99 121 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 114 101 115 111 117 114 99 101 115 34 58 123 125 44 34 102 58 116 101 114 109 105 110 97 116 105 111 110 77 101 115 115 97 103 101 80 97 116 104 34 58 123 125 44 34 102 58 116 101 114 109 105 110 97 116 105 111 110 77 101 115 115 97 103 101 80 111 108 105 99 121 34 58 123 125 125 125 44 34 102 58 100 110 115 80 111 108 105 99 121 34 58 123 125 44 34 102 58 101 110 97 98 108 101 83 101 114 118 105 99 101 76 105 110 107 115 34 58 123 125 44 34 102 58 114 101 115 116 97 114 116 80 111 108 105 99 121 34 58 123 125 44 34 102 58 115 99 104 101 100 117 108 101 114 78 97 109 101 34 58 123 125 44 34 102 58 115 101 99 117 114 105 116 121 67 111 110 116 101 120 116 34 58 123 125 44 34 102 58 116 101 114 109 105 110 97 116 105 111 110 71 114 97 99 101 80 101 114 105 111 100 83 101 99 111 110 100 115 34 58 123 125 125 125],}} {kubelet Update v1 2021-05-07 11:35:37 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 115 116 97 116 117 115 34 58 123 34 102 58 99 111 110 100 105 116 105 111 110 115 34 58 123 34 107 58 123 92 34 116 121 112 101 92 34 58 92 34 67 111 110 116 97 105 110 101 114 115 82 101 97 100 121 92 34 125 34 58 123 34 46 34 58 123 125 44 34 102 58 108 97 115 116 80 114 111 98 101 84 105 109 101 34 58 123 125 44 34 102 58 108 97 115 116 84 114 97 110 115 105 116 105 111 110 84 105 109 101 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 116 97 116 117 115 34 58 123 125 44 34 102 58 116 121 112 101 34 58 123 125 125 44 34 107 58 123 92 34 116 121 112 101 92 34 58 92 34 73 110 105 116 105 97 108 105 122 101 100 92 34 125 34 58 123 34 46 34 58 123 125 44 34 102 58 108 97 115 116 80 114 111 98 101 84 105 109 101 34 58 123 125 44 34 102 58 108 97 115 116 84 114 97 110 115 105 116 105 111 110 84 105 109 101 34 58 123 125 44 34 102 58 115 116 97 116 117 115 34 58 123 125 44 34 102 58 116 121 112 101 34 58 123 125 125 44 34 107 58 123 92 34 116 121 112 101 92 34 58 92 34 82 101 97 100 121 92 34 125 34 58 123 34 46 34 58 123 125 44 34 102 58 108 97 115 116 80 114 111 98 101 84 105 109 101 34 58 123 125 44 34 102 58 108 97 115 116 84 114 97 110 115 105 116 105 111 110 84 105 109 101 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 116 97 116 117 115 34 58 123 125 44 34 102 58 116 121 112 101 34 58 123 125 125 125 44 34 102 58 99 111 110 116 97 105 110 101 114 83 116 97 116 117 115 101 115 34 58 123 125 44 34 102 58 104 111 115 116 73 80 34 58 123 125 44 34 102 58 115 116 97 114 116 84 105 109 101 34 58 123 125 125 125],}}]},Spec:PodSpec{Volumes:[]Volume{Volume{Name:default-token-cdkn6,VolumeSource:VolumeSource{HostPath:nil,EmptyDir:nil,GCEPersistentDisk:nil,AWSElasticBlockStore:nil,GitRepo:nil,Secret:&SecretVolumeSource{SecretName:default-token-cdkn6,Items:[]KeyToPath{},DefaultMode:*420,Optional:nil,},NFS:nil,ISCSI:nil,Glusterfs:nil,PersistentVolumeClaim:nil,RBD:nil,FlexVolume:nil,Cinder:nil,CephFS:nil,Flocker:nil,DownwardAPI:nil,FC:nil,AzureFile:nil,ConfigMap:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Projected:nil,StorageOS:nil,CSI:nil,},},},Containers:[]Container{Container{Name:nginx,Image:nginx,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:default-token-cdkn6,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,},},RestartPolicy:Always,TerminationGracePeriodSeconds:*30,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:default,DeprecatedServiceAccount:default,NodeName:test-machine,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,},ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{Toleration{Key:node.kubernetes.io/not-ready,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},Toleration{Key:node.kubernetes.io/unreachable,Operator:Exists,Value:,Effect:NoExecute,TolerationSeconds:*300,},},HostAliases:[]HostAlias{},PriorityClassName:,Priority:*0,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:*true,PreemptionPolicy:*PreemptLowerPriority,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},},Status:PodStatus{Phase:Pending,Conditions:[]PodCondition{PodCondition{Type:Initialized,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-05-07 11:35:37 +0000 UTC,Reason:,Message:,},PodCondition{Type:Ready,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-05-07 11:35:37 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [nginx],},PodCondition{Type:ContainersReady,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-05-07 11:35:37 +0000 UTC,Reason:ContainersNotReady,Message:containers with unready status: [nginx],},PodCondition{Type:PodScheduled,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2021-05-07 11:35:36 +0000 UTC,Reason:,Message:,},},Message:,Reason:,HostIP:192.168.11.21,PodIP:,StartTime:2021-05-07 11:35:37 +0000 UTC,ContainerStatuses:[]ContainerStatus{ContainerStatus{Name:nginx,State:ContainerState{Waiting:&ContainerStateWaiting{Reason:ContainerCreating,Message:,},Running:nil,Terminated:nil,},LastTerminationState:ContainerState{Waiting:nil,Running:nil,Terminated:nil,},Ready:false,RestartCount:0,Image:nginx,ImageID:,ContainerID:,Started:*false,},},QOSClass:BestEffort,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{},EphemeralContainerStatuses:[]ContainerStatus{},},}
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.463 [INFO][679267] ipam_plugin.go 211: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.472 [INFO][679267] ipam_plugin.go 223: Calico CNI IPAM handle=k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22 ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.473 [INFO][679267] ipam_plugin.go 233: Auto assigning IP ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc000694810), Attrs:map[string]string{"namespace":"default", "node":"test-machine", "pod":"app-65f6b9d4d9-g26wm"}, Hostname:"test-machine", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0}
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.473 [INFO][679267] ipam.go 87: Auto-assign 1 ipv4, 0 ipv6 addrs for host 'test-machine'
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.478 [INFO][679267] ipam.go 313: Looking up existing affinities for host handle="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" host="test-machine"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.480 [INFO][679267] ipam.go 381: Trying affinity for 10.1.0.0/26 handle="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" host="test-machine"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.482 [INFO][679267] ipam.go 135: Attempting to load block cidr=10.1.0.0/26 host="test-machine"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.485 [INFO][679267] ipam.go 212: Affinity is confirmed and block has been loaded cidr=10.1.0.0/26 host="test-machine"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.485 [INFO][679267] ipam.go 804: Attempting to assign 1 addresses from block block=10.1.0.0/26 handle="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" host="test-machine"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.487 [INFO][679267] ipam.go 1265: Creating new handle: k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.502 [INFO][679267] ipam.go 827: Writing block in order to claim IPs block=10.1.0.0/26 handle="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" host="test-machine"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.542 [INFO][679267] ipam.go 840: Successfully claimed IPs: [10.1.0.10/26] block=10.1.0.0/26 handle="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" host="test-machine"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.542 [INFO][679267] ipam.go 413: Block '10.1.0.0/26' provided addresses: [10.1.0.10/26] handle="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" host="test-machine"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.552 [INFO][679267] ipam.go 569: Auto-assigned 1 out of 1 IPv4s: [10.1.0.10/26] handle="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" host="test-machine"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.552 [INFO][679267] ipam_plugin.go 235: Calico CNI IPAM assigned addresses IPv4=[10.1.0.10/26] IPv6=[] ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679267]: 2021-05-07 11:35:37.552 [INFO][679267] ipam_plugin.go 261: IPAM Result ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0" result.IPs=[]*current.IPConfig{(*current.IPConfig)(0xc0006836e0)}
May 07 11:35:37 test-machine microk8s.daemon-containerd[679262]: 2021-05-07 11:35:37.554 [INFO][679262] k8s.go 358: Populated endpoint ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Namespace="default" Pod="app-65f6b9d4d9-g26wm" WorkloadEndpoint="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"test--machine-k8s-app--65f6b9d4d9--g26wm-eth0", GenerateName:"app-65f6b9d4d9-", Namespace:"default", SelfLink:"", UID:"691bca13-bd51-415a-af3a-9dc02e2477ad", ResourceVersion:"50704", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63755984136, loc:(*time.Location)(0x299c340)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app":"app", "pod-template-hash":"65f6b9d4d9", "projectcalico.org/namespace":"default", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"default"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"test-machine", ContainerID:"", Pod:"app-65f6b9d4d9-g26wm", Endpoint:"eth0", IPNetworks:[]string{"10.1.0.10/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.default", "ksa.default.default"}, InterfaceName:"cali99de102e059", MAC:"", Ports:[]v3.EndpointPort(nil)}}
May 07 11:35:37 test-machine microk8s.daemon-containerd[679262]: 2021-05-07 11:35:37.554 [INFO][679262] k8s.go 359: Calico CNI using IPs: [10.1.0.10/32] ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Namespace="default" Pod="app-65f6b9d4d9-g26wm" WorkloadEndpoint="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679262]: 2021-05-07 11:35:37.555 [INFO][679262] network_linux.go 76: Setting the host side veth name to cali99de102e059 ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Namespace="default" Pod="app-65f6b9d4d9-g26wm" WorkloadEndpoint="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679262]: 2021-05-07 11:35:37.556 [INFO][679262] network_linux.go 396: Disabling IPv4 forwarding ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Namespace="default" Pod="app-65f6b9d4d9-g26wm" WorkloadEndpoint="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:35:37 test-machine microk8s.daemon-containerd[679262]: 2021-05-07 11:35:37.594 [INFO][679262] k8s.go 385: Added Mac, interface name, and active container ID to endpoint ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Namespace="default" Pod="app-65f6b9d4d9-g26wm" WorkloadEndpoint="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"test--machine-k8s-app--65f6b9d4d9--g26wm-eth0", GenerateName:"app-65f6b9d4d9-", Namespace:"default", SelfLink:"", UID:"691bca13-bd51-415a-af3a-9dc02e2477ad", ResourceVersion:"50704", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63755984136, loc:(*time.Location)(0x299c340)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app":"app", "pod-template-hash":"65f6b9d4d9", "projectcalico.org/namespace":"default", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"default"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"test-machine", ContainerID:"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22", Pod:"app-65f6b9d4d9-g26wm", Endpoint:"eth0", IPNetworks:[]string{"10.1.0.10/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.default", "ksa.default.default"}, InterfaceName:"cali99de102e059", MAC:"2a:99:f6:89:68:36", Ports:[]v3.EndpointPort(nil)}}
May 07 11:35:37 test-machine microk8s.daemon-containerd[679262]: 2021-05-07 11:35:37.620 [INFO][679262] k8s.go 417: Wrote updated endpoint to datastore ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Namespace="default" Pod="app-65f6b9d4d9-g26wm" WorkloadEndpoint="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:35:37 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:35:37.736424325Z" level=info msg="starting signal loop" namespace=k8s.io path=/var/snap/microk8s/common/run/containerd/io.containerd.runtime.v2.task/k8s.io/b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22 pid=679297
May 07 11:35:37 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:35:37.910046157Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:app-65f6b9d4d9-g26wm,Uid:691bca13-bd51-415a-af3a-9dc02e2477ad,Namespace:default,Attempt:0,} returns sandbox id \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\""

May 07 11:35:37 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:35:37.915613381Z" level=info msg="PullImage \"nginx:latest\""
May 07 11:35:52 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:35:52.687147850Z" level=info msg="ImageCreate event &ImageCreate{Name:docker.io/library/nginx:latest,Labels:map[string]string{io.cri-containerd.image: managed,},XXX_unrecognized:[],}"
May 07 11:35:55 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:35:55.107834144Z" level=info msg="ImageCreate event &ImageCreate{Name:sha256:62d49f9bab67f7c70ac3395855bf01389eb3175b374e621f6f191bf31b54cd5b,Labels:map[string]string{io.cri-containerd.image: managed,},XXX_unrecognized:[],}"
May 07 11:35:55 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:35:55.145625929Z" level=info msg="ImageUpdate event &ImageUpdate{Name:docker.io/library/nginx:latest,Labels:map[string]string{io.cri-containerd.image: managed,},XXX_unrecognized:[],}"
May 07 11:35:55 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:35:55.177274297Z" level=info msg="ImageCreate event &ImageCreate{Name:docker.io/library/nginx@sha256:75a55d33ecc73c2a242450a9f1cc858499d468f077ea942867e662c247b5e412,Labels:map[string]string{io.cri-containerd.image: managed,},XXX_unrecognized:[],}"
May 07 11:35:55 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:35:55.179785063Z" level=info msg="PullImage \"nginx:latest\" returns image reference \"sha256:62d49f9bab67f7c70ac3395855bf01389eb3175b374e621f6f191bf31b54cd5b\""

May 07 11:35:55 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:35:55.193218280Z" level=info msg="CreateContainer within sandbox \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\" for container &ContainerMetadata{Name:nginx,Attempt:0,}"
May 07 11:35:58 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:35:58.084015766Z" level=info msg="CreateContainer within sandbox \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\" for &ContainerMetadata{Name:nginx,Attempt:0,} returns container id \"3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1\""

May 07 11:35:58 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:35:58.089472692Z" level=info msg="StartContainer for \"3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1\""
May 07 11:35:58 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:35:58.108036739Z" level=info msg="starting signal loop" namespace=k8s.io path=/var/snap/microk8s/common/run/containerd/io.containerd.runtime.v2.task/k8s.io/3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1 pid=679689
May 07 11:35:58 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:35:58.411439002Z" level=info msg="StartContainer for \"3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1\" returns successfully"

microk8s.kubectl delete deployment app を実行したときのログ。

# journalctl -f -u snap.microk8s.daemon-containerd.service | grep -v "Finish piping " | grep -v "ExecSync" | grep -v "Exec process"

May 07 11:36:55 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:55.272037988Z" level=info msg="StopContainer for \"3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1\" with timeout 30 (s)"
May 07 11:36:55 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:55.272491981Z" level=info msg="Stop container \"3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1\" with signal quit"
May 07 11:36:55 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:55.303729656Z" level=info msg="TaskExit event &TaskExit{ContainerID:3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1,ID:3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1,Pid:679716,ExitStatus:0,ExitedAt:2021-05-07 11:36:55.30340146 +0000 UTC,XXX_unrecognized:[],}"
May 07 11:36:55 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:55.891335751Z" level=info msg="shim disconnected" id=3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1
May 07 11:36:55 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:55.943793036Z" level=info msg="StopContainer for \"3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1\" returns successfully"

May 07 11:36:55 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:55.944720024Z" level=info msg="StopPodSandbox for \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\""
May 07 11:36:55 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:55.944791623Z" level=info msg="Container to stop \"3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1\" must be in running or unknown state, current state \"CONTAINER_EXITED\""
May 07 11:36:55 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:55.986814150Z" level=info msg="TaskExit event &TaskExit{ContainerID:b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22,ID:b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22,Pid:679318,ExitStatus:137,ExitedAt:2021-05-07 11:36:55.986234558 +0000 UTC,XXX_unrecognized:[],}"
May 07 11:36:56 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:56.431914787Z" level=info msg="shim disconnected" id=b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22
May 07 11:36:56 test-machine microk8s.daemon-containerd[680840]: 2021-05-07 11:36:56.473 [INFO][680840] plugin.go 496: Extracted identifiers ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Node="test-machine" Orchestrator="k8s" WorkloadEndpoint="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680840]: 2021-05-07 11:36:56.480 [WARNING][680840] workloadendpoint.go 77: Operation Delete is not supported on WorkloadEndpoint type
May 07 11:36:56 test-machine microk8s.daemon-containerd[680840]: 2021-05-07 11:36:56.480 [INFO][680840] k8s.go 467: Endpoint deletion will be handled by Kubernetes deletion of the Pod. ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"test--machine-k8s-app--65f6b9d4d9--g26wm-eth0", GenerateName:"app-65f6b9d4d9-", Namespace:"default", SelfLink:"", UID:"691bca13-bd51-415a-af3a-9dc02e2477ad", ResourceVersion:"50811", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63755984136, loc:(*time.Location)(0x299c340)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app":"app", "pod-template-hash":"65f6b9d4d9", "projectcalico.org/namespace":"default", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"default"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"test-machine", ContainerID:"", Pod:"app-65f6b9d4d9-g26wm", Endpoint:"eth0", IPNetworks:[]string{"10.1.0.10/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.default", "ksa.default.default"}, InterfaceName:"cali99de102e059", MAC:"", Ports:[]v3.EndpointPort(nil)}}
May 07 11:36:56 test-machine microk8s.daemon-containerd[680840]: 2021-05-07 11:36:56.481 [INFO][680840] k8s.go 474: Cleaning up netns ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680840]: 2021-05-07 11:36:56.482 [INFO][680840] network_linux.go 450: Calico CNI deleting device in netns /var/run/netns/cni-7a0a0250-add3-603e-b7e3-2a885529d18e ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680840]: 2021-05-07 11:36:56.498 [INFO][680840] network_linux.go 467: Calico CNI deleted device in netns /var/run/netns/cni-7a0a0250-add3-603e-b7e3-2a885529d18e ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680840]: 2021-05-07 11:36:56.498 [INFO][680840] k8s.go 481: Releasing IP address(es) ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680840]: 2021-05-07 11:36:56.498 [INFO][680840] utils.go 168: Calico CNI releasing IP address ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680848]: 2021-05-07 11:36:56.517 [INFO][680848] ipam_plugin.go 302: Releasing address using handleID ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680848]: 2021-05-07 11:36:56.518 [INFO][680848] ipam.go 1166: Releasing all IPs with handle 'k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22'
May 07 11:36:56 test-machine microk8s.daemon-containerd[680848]: 2021-05-07 11:36:56.627 [INFO][680848] ipam_plugin.go 314: Released address using handleID ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680848]: 2021-05-07 11:36:56.628 [INFO][680848] ipam_plugin.go 323: Releasing address using workloadID ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680848]: 2021-05-07 11:36:56.629 [INFO][680848] ipam.go 1166: Releasing all IPs with handle 'default.app-65f6b9d4d9-g26wm'
May 07 11:36:56 test-machine microk8s.daemon-containerd[680840]: 2021-05-07 11:36:56.639 [INFO][680840] k8s.go 487: Teardown processing complete. ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:56 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:56.647007457Z" level=info msg="TearDown network for sandbox \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\" successfully"
May 07 11:36:56 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:56.647195754Z" level=info msg="StopPodSandbox for \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\" returns successfully"

May 07 11:36:56 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:56.700702925Z" level=info msg="StopPodSandbox for \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\""
May 07 11:36:56 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:56.700932022Z" level=info msg="Container to stop \"3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1\" must be in running or unknown state, current state \"CONTAINER_EXITED\""
May 07 11:36:56 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:56.703736184Z" level=info msg="RemoveContainer for \"3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1\""
May 07 11:36:56 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:56.741609568Z" level=info msg="RemoveContainer for \"3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1\" returns successfully"
May 07 11:36:56 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:56.742874451Z" level=error msg="ContainerStatus for \"3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1\" failed" error="an error occurred when try to find container \"3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1\": does not exist"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680880]: 2021-05-07 11:36:56.759 [INFO][680880] plugin.go 496: Extracted identifiers ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Node="test-machine" Orchestrator="k8s" WorkloadEndpoint="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680880]: 2021-05-07 11:36:56.767 [WARNING][680880] workloadendpoint.go 77: Operation Delete is not supported on WorkloadEndpoint type
May 07 11:36:56 test-machine microk8s.daemon-containerd[680880]: 2021-05-07 11:36:56.768 [INFO][680880] k8s.go 467: Endpoint deletion will be handled by Kubernetes deletion of the Pod. ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"test--machine-k8s-app--65f6b9d4d9--g26wm-eth0", GenerateName:"app-65f6b9d4d9-", Namespace:"default", SelfLink:"", UID:"691bca13-bd51-415a-af3a-9dc02e2477ad", ResourceVersion:"50818", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63755984136, loc:(*time.Location)(0x299c340)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app":"app", "pod-template-hash":"65f6b9d4d9", "projectcalico.org/namespace":"default", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"default"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"test-machine", ContainerID:"", Pod:"app-65f6b9d4d9-g26wm", Endpoint:"eth0", IPNetworks:[]string{"10.1.0.10/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.default", "ksa.default.default"}, InterfaceName:"cali99de102e059", MAC:"", Ports:[]v3.EndpointPort(nil)}}
May 07 11:36:56 test-machine microk8s.daemon-containerd[680880]: 2021-05-07 11:36:56.768 [INFO][680880] k8s.go 474: Cleaning up netns ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680880]: 2021-05-07 11:36:56.768 [INFO][680880] k8s.go 481: Releasing IP address(es) ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680880]: 2021-05-07 11:36:56.768 [INFO][680880] utils.go 168: Calico CNI releasing IP address ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680888]: 2021-05-07 11:36:56.786 [INFO][680888] ipam_plugin.go 302: Releasing address using handleID ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680888]: 2021-05-07 11:36:56.787 [INFO][680888] ipam.go 1166: Releasing all IPs with handle 'k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22'
May 07 11:36:56 test-machine microk8s.daemon-containerd[680888]: 2021-05-07 11:36:56.793 [WARNING][680888] ipam_plugin.go 312: Asked to release address but it doesn't exist. Ignoring ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680888]: 2021-05-07 11:36:56.793 [INFO][680888] ipam_plugin.go 323: Releasing address using workloadID ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:56 test-machine microk8s.daemon-containerd[680888]: 2021-05-07 11:36:56.793 [INFO][680888] ipam.go 1166: Releasing all IPs with handle 'default.app-65f6b9d4d9-g26wm'
May 07 11:36:56 test-machine microk8s.daemon-containerd[680880]: 2021-05-07 11:36:56.796 [INFO][680880] k8s.go 487: Teardown processing complete. ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:56 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:56.797595905Z" level=info msg="TearDown network for sandbox \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\" successfully"
May 07 11:36:56 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:56.797746503Z" level=info msg="StopPodSandbox for \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\" returns successfully"

May 07 11:36:57 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:57.080765548Z" level=error msg="ContainerStatus for \"3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1\" failed" error="an error occurred when try to find container \"3f04c94a1aee8d64fb455fb486bbfe5db985fc355c2723948e91e8166fa8bbf1\": does not exist"
May 07 11:36:57 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:57.082487724Z" level=info msg="StopPodSandbox for \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\""
May 07 11:36:57 test-machine microk8s.daemon-containerd[680920]: 2021-05-07 11:36:57.131 [INFO][680920] plugin.go 496: Extracted identifiers ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Node="test-machine" Orchestrator="k8s" WorkloadEndpoint="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:57 test-machine microk8s.daemon-containerd[680920]: 2021-05-07 11:36:57.140 [WARNING][680920] workloadendpoint.go 77: Operation Delete is not supported on WorkloadEndpoint type
May 07 11:36:57 test-machine microk8s.daemon-containerd[680920]: 2021-05-07 11:36:57.141 [INFO][680920] k8s.go 467: Endpoint deletion will be handled by Kubernetes deletion of the Pod. ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"test--machine-k8s-app--65f6b9d4d9--g26wm-eth0", GenerateName:"app-65f6b9d4d9-", Namespace:"default", SelfLink:"", UID:"691bca13-bd51-415a-af3a-9dc02e2477ad", ResourceVersion:"50818", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63755984136, loc:(*time.Location)(0x299c340)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app":"app", "pod-template-hash":"65f6b9d4d9", "projectcalico.org/namespace":"default", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"default"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"test-machine", ContainerID:"", Pod:"app-65f6b9d4d9-g26wm", Endpoint:"eth0", IPNetworks:[]string{"10.1.0.10/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.default", "ksa.default.default"}, InterfaceName:"cali99de102e059", MAC:"", Ports:[]v3.EndpointPort(nil)}}
May 07 11:36:57 test-machine microk8s.daemon-containerd[680920]: 2021-05-07 11:36:57.141 [INFO][680920] k8s.go 474: Cleaning up netns ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:57 test-machine microk8s.daemon-containerd[680920]: 2021-05-07 11:36:57.141 [INFO][680920] k8s.go 481: Releasing IP address(es) ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:57 test-machine microk8s.daemon-containerd[680920]: 2021-05-07 11:36:57.141 [INFO][680920] utils.go 168: Calico CNI releasing IP address ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:57 test-machine microk8s.daemon-containerd[680926]: 2021-05-07 11:36:57.158 [INFO][680926] ipam_plugin.go 302: Releasing address using handleID ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:57 test-machine microk8s.daemon-containerd[680926]: 2021-05-07 11:36:57.158 [INFO][680926] ipam.go 1166: Releasing all IPs with handle 'k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22'
May 07 11:36:57 test-machine microk8s.daemon-containerd[680926]: 2021-05-07 11:36:57.165 [WARNING][680926] ipam_plugin.go 312: Asked to release address but it doesn't exist. Ignoring ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:57 test-machine microk8s.daemon-containerd[680926]: 2021-05-07 11:36:57.165 [INFO][680926] ipam_plugin.go 323: Releasing address using workloadID ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:57 test-machine microk8s.daemon-containerd[680926]: 2021-05-07 11:36:57.165 [INFO][680926] ipam.go 1166: Releasing all IPs with handle 'default.app-65f6b9d4d9-g26wm'
May 07 11:36:57 test-machine microk8s.daemon-containerd[680920]: 2021-05-07 11:36:57.168 [INFO][680920] k8s.go 487: Teardown processing complete. ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:57 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:57.169294342Z" level=info msg="TearDown network for sandbox \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\" successfully"
May 07 11:36:57 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:57.169441340Z" level=info msg="StopPodSandbox for \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\" returns successfully"

May 07 11:36:57 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:57.703731261Z" level=info msg="StopPodSandbox for \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\""
May 07 11:36:57 test-machine microk8s.daemon-containerd[680957]: 2021-05-07 11:36:57.782 [INFO][680957] plugin.go 496: Extracted identifiers ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Node="test-machine" Orchestrator="k8s" WorkloadEndpoint="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:57 test-machine microk8s.daemon-containerd[680957]: 2021-05-07 11:36:57.791 [WARNING][680957] workloadendpoint.go 77: Operation Delete is not supported on WorkloadEndpoint type
May 07 11:36:57 test-machine microk8s.daemon-containerd[680957]: 2021-05-07 11:36:57.791 [INFO][680957] k8s.go 467: Endpoint deletion will be handled by Kubernetes deletion of the Pod. ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"test--machine-k8s-app--65f6b9d4d9--g26wm-eth0", GenerateName:"app-65f6b9d4d9-", Namespace:"default", SelfLink:"", UID:"691bca13-bd51-415a-af3a-9dc02e2477ad", ResourceVersion:"50818", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63755984136, loc:(*time.Location)(0x299c340)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app":"app", "pod-template-hash":"65f6b9d4d9", "projectcalico.org/namespace":"default", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"default"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"test-machine", ContainerID:"", Pod:"app-65f6b9d4d9-g26wm", Endpoint:"eth0", IPNetworks:[]string{"10.1.0.10/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.default", "ksa.default.default"}, InterfaceName:"cali99de102e059", MAC:"", Ports:[]v3.EndpointPort(nil)}}
May 07 11:36:57 test-machine microk8s.daemon-containerd[680957]: 2021-05-07 11:36:57.791 [INFO][680957] k8s.go 474: Cleaning up netns ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:57 test-machine microk8s.daemon-containerd[680957]: 2021-05-07 11:36:57.791 [INFO][680957] k8s.go 481: Releasing IP address(es) ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:57 test-machine microk8s.daemon-containerd[680957]: 2021-05-07 11:36:57.791 [INFO][680957] utils.go 168: Calico CNI releasing IP address ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:57 test-machine microk8s.daemon-containerd[680963]: 2021-05-07 11:36:57.804 [INFO][680963] ipam_plugin.go 302: Releasing address using handleID ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:57 test-machine microk8s.daemon-containerd[680963]: 2021-05-07 11:36:57.815 [INFO][680963] ipam.go 1166: Releasing all IPs with handle 'k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22'
May 07 11:36:57 test-machine microk8s.daemon-containerd[680963]: 2021-05-07 11:36:57.824 [WARNING][680963] ipam_plugin.go 312: Asked to release address but it doesn't exist. Ignoring ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:57 test-machine microk8s.daemon-containerd[680963]: 2021-05-07 11:36:57.824 [INFO][680963] ipam_plugin.go 323: Releasing address using workloadID ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:36:57 test-machine microk8s.daemon-containerd[680963]: 2021-05-07 11:36:57.824 [INFO][680963] ipam.go 1166: Releasing all IPs with handle 'default.app-65f6b9d4d9-g26wm'
May 07 11:36:57 test-machine microk8s.daemon-containerd[680957]: 2021-05-07 11:36:57.827 [INFO][680957] k8s.go 487: Teardown processing complete. ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:36:57 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:57.828427362Z" level=info msg="TearDown network for sandbox \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\" successfully"
May 07 11:36:57 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:36:57.828584960Z" level=info msg="StopPodSandbox for \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\" returns successfully"

May 07 11:37:14 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:37:14.938782201Z" level=info msg="StopPodSandbox for \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\""
May 07 11:37:15 test-machine microk8s.daemon-containerd[681313]: 2021-05-07 11:37:15.015 [INFO][681313] plugin.go 496: Extracted identifiers ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Node="test-machine" Orchestrator="k8s" WorkloadEndpoint="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:37:15 test-machine microk8s.daemon-containerd[681313]: 2021-05-07 11:37:15.018 [WARNING][681313] k8s.go 444: WorkloadEndpoint does not exist in the datastore, moving forward with the clean up ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" WorkloadEndpoint="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:37:15 test-machine microk8s.daemon-containerd[681313]: 2021-05-07 11:37:15.018 [INFO][681313] k8s.go 474: Cleaning up netns ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:37:15 test-machine microk8s.daemon-containerd[681313]: 2021-05-07 11:37:15.018 [INFO][681313] k8s.go 481: Releasing IP address(es) ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:37:15 test-machine microk8s.daemon-containerd[681313]: 2021-05-07 11:37:15.018 [INFO][681313] utils.go 168: Calico CNI releasing IP address ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:37:15 test-machine microk8s.daemon-containerd[681319]: 2021-05-07 11:37:15.032 [INFO][681319] ipam_plugin.go 302: Releasing address using handleID ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:37:15 test-machine microk8s.daemon-containerd[681319]: 2021-05-07 11:37:15.032 [INFO][681319] ipam.go 1166: Releasing all IPs with handle 'k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22'
May 07 11:37:15 test-machine microk8s.daemon-containerd[681319]: 2021-05-07 11:37:15.038 [WARNING][681319] ipam_plugin.go 312: Asked to release address but it doesn't exist. Ignoring ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:37:15 test-machine microk8s.daemon-containerd[681319]: 2021-05-07 11:37:15.039 [INFO][681319] ipam_plugin.go 323: Releasing address using workloadID ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" HandleID="k8s-pod-network.b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22" Workload="test--machine-k8s-app--65f6b9d4d9--g26wm-eth0"
May 07 11:37:15 test-machine microk8s.daemon-containerd[681319]: 2021-05-07 11:37:15.039 [INFO][681319] ipam.go 1166: Releasing all IPs with handle 'default.app-65f6b9d4d9-g26wm'
May 07 11:37:15 test-machine microk8s.daemon-containerd[681313]: 2021-05-07 11:37:15.041 [INFO][681313] k8s.go 487: Teardown processing complete. ContainerID="b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22"
May 07 11:37:15 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:37:15.042750085Z" level=info msg="TearDown network for sandbox \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\" successfully"
May 07 11:37:15 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:37:15.043042281Z" level=info msg="StopPodSandbox for \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\" returns successfully"

May 07 11:37:15 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:37:15.043531974Z" level=info msg="RemovePodSandbox for \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\""
May 07 11:37:15 test-machine microk8s.daemon-containerd[19412]: time="2021-05-07T11:37:15.077320614Z" level=info msg="RemovePodSandbox \"b0a1903ca73fc036f66ad08d705f010e0edfdf40f7e03696313c2a05ff0c3f22\" returns successfully"

kubernetes CRI とは

概要

CRI って実際なに？

CRI に定義されている API

参考