Skip to content

Instantly share code, notes, and snippets.

@smreed

smreed/docker ps -a

Last active August 29, 2015 14:13
Show Gist options
  • Save smreed/6e6ebfab11d8ec7933a3 to your computer and use it in GitHub Desktop.
Save smreed/6e6ebfab11d8ec7933a3 to your computer and use it in GitHub Desktop.
Download ZIP
cadvisor problem on k8s 0.8.0
Raw
also in kubelet.log
I think this is unrelated
I0112 19:11:37.511627 4877 server.go:324] GET /stats/: (514.058us) 500
goroutine 913 [running]:
github.com/GoogleCloudPlatform/kubernetes/pkg/httplog.(*respLogger).WriteHeader(0xc20817de60, 0x1f4)
/go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/httplog/log.go:182 +0x9c
net/http.Error(0x7f8a04dcd680, 0xc20817de60, 0xc208304000, 0x47, 0x1f4)
/usr/src/go/src/pkg/net/http/server.go:1244 +0x96
github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet.(*Server).error(0xc208124340, 0x7f8a04dcd680, 0xc20817de60, 0x7f8a04dbf1b0, 0xc208149ce0)
/go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet/server.go:108 +0xf5
github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet.(*Server).serveStats(0xc208124340, 0x7f8a04dcd680, 0xc20817de60, 0xc2084f4dd0)
/go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet/server.go:370 +0x34f
github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet.(*Server).handleStats(0xc208124340, 0x7f8a04dcd680, 0xc20817de60, 0xc2084f4dd0)
/go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet/server.go:249 +0x45
github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet.*Server.(github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet.handleStats)·fm(0x7f8a04dcd680, 0xc20817de60, 0xc2084f4dd0)
/go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet/server.go:94 +0x44
net/http.HandlerFunc.ServeHTTP(0xc2081224d0, 0x7f8a04dcd680, 0xc20817de60, 0xc2084f4dd0)
/usr/src/go/src/pkg/net/http/server.go:1235 +0x40
net/http.(*ServeMux).ServeHTTP(0xc208114b10, 0x7f8a04dcd680, 0xc20817de60, 0xc2084f4dd0)
/usr/src/go/src/pkg/net/http/server.go:1511 +0x1a3
github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet.(*Server).ServeHTTP(0xc208124320, 0x7f8a04dcd648, 0xc2083ab180, 0xc2084f4dd0)
/go/src/github.com/GoogleCloudPlatform/kuber
Raw
docker ps -a
sreed@kubernetes-minion-4:~$ sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6f01f5894d23 kubernetes/pause:go "/pause" 2 seconds ago k8s_net.1509ae25_cadvisor-agent.default.file_cadvisormanifes12uqn2ohido76855gdecd9roadm7l0_3af445a9
46b0e8b9a850 kubernetes/pause:go "/pause" 12 seconds ago k8s_net.1509ae25_cadvisor-agent.default.file_cadvisormanifes12uqn2ohido76855gdecd9roadm7l0_a1ca4847
7a22f825d2f8 kubernetes/pause:go "/pause" 22 seconds ago k8s_net.1509ae25_cadvisor-agent.default.file_cadvisormanifes12uqn2ohido76855gdecd9roadm7l0_96ddebf3
e9d97fa93634 kubernetes/pause:go "/pause" 32 seconds ago k8s_net.1509ae25_cadvisor-agent.default.file_cadvisormanifes12uqn2ohido76855gdecd9roadm7l0_28217a17
54ebd236816a kubernetes/pause:go "/pause" 42 seconds ago k8s_net.1509ae25_cadvisor-agent.default.file_cadvisormanifes12uqn2ohido76855gdecd9roadm7l0_af3e061a
6dcc1cf7febc kubernetes/pause:go "/pause" 52 seconds ago k8s_net.1509ae25_cadvisor-agent.default.file_cadvisormanifes12uqn2ohido76855gdecd9roadm7l0_3c3dbe83
853f1101f25d kubernetes/pause:go "/pause" About a minute ago k8s_net.1509ae25_cadvisor-agent.default.file_cadvisormanifes12uqn2ohido76855gdecd9roadm7l0_7757773c
ac8def2f6b23 kubernetes/pause:go "/pause" 19 minutes ago Up 19 minutes k8s_net.61d3abb6_bd6bc202-9a8b-11e4-af56-42010af0dcb6.default.etcd_bd6bc202-9a8b-11e4-af56-42010af0dcb6_46bf27eb
29de1bbd873c kubernetes/pause:go "/pause" 20 minutes ago Up 20 minutes k8s_net.61d3abb6_bc1262aa-9a8b-11e4-af56-42010af0dcb6.default.etcd_bc1262aa-9a8b-11e4-af56-42010af0dcb6_65916080
f655f9838f00 kubernetes/pause:go "/pause" 23 minutes ago Up 23 minutes 0.0.0.0:6379->6379/tcp k8s_net.d21fae8c_redis-3.default.etcd_3475f3f6-9a8b-11e4-af56-42010af0dcb6_c88ec0d6
41307dc8ed5a kubernetes/fluentd-elasticsearch:latest "/bin/sh -c '/usr/sb 29 minutes ago Up 29 minutes k8s_fluentd-es.e2ed314_fluentd-to-elasticsearch.default.file_fluentdesmanife2u464h05heqcpotoddodpnehjaqsde_45ab2df2
817af3651074 kubernetes/pause:go "/pause" 30 minutes ago Up 30 minutes k8s_net.77108a9f_fluentd-to-elasticsearch.default.file_fluentdesmanife2u464h05heqcpotoddodpnehjaqsde_2ac610db
db083aa6b1b4 kubernetes/pause:go "/pause" 30 minutes ago Up 30 minutes 0.0.0.0:4194->8080/tcp k8s_net.a0f18f6e_cadvisormanifes12uqn2ohido76855gdecd9roadm7l0.default.file_cadvisormanifes12uqn2ohido76855gdecd9roadm7l0_aa970084
sreed@kubernetes-minion-4:~$
Raw
kubelet.log
I0112 19:11:34.627407 4877 kubelet.go:793] Network container doesn't exist for pod "cadvisor-agent.default.file", killing and re-creating the pod
I0112 19:11:34.629292 4877 event.go:92] Event(api.ObjectReference{Kind:"BoundPod", Namespace:"default", Name:"cadvisor-agent", UID:"cadvisormanifes12uqn2ohido76855gdecd9roadm7l0", APIVersion:"v1beta2", ResourceVersion:"", FieldPath:"im
plicitly required container net"}): status: 'waiting', reason: 'pulled' Successfully pulled image "kubernetes/pause:latest"
I0112 19:11:34.732351 4877 event.go:92] Event(api.ObjectReference{Kind:"BoundPod", Namespace:"default", Name:"cadvisor-agent", UID:"cadvisormanifes12uqn2ohido76855gdecd9roadm7l0", APIVersion:"v1beta2", ResourceVersion:"", FieldPath:"im
plicitly required container net"}): status: 'waiting', reason: 'created' Created with docker id db93e0b55d1193bd7921e276b0592f5aa7ff2b85f2707c5680c1639237f0df42
E0112 19:11:34.780088 4877 kubelet.go:800] Failed to introspect network container: API error (500): Cannot start container db93e0b55d1193bd7921e276b0592f5aa7ff2b85f2707c5680c1639237f0df42: Bind for 0.0.0.0:4194 failed: port is already
allocated
; Skipping pod "cadvisor-agent.default.file"
E0112 19:11:34.780145 4877 kubelet.go:1023] Error syncing pod, skipping: API error (500): Cannot start container db93e0b55d1193bd7921e276b0592f5aa7ff2b85f2707c5680c1639237f0df42: Bind for 0.0.0.0:4194 failed: port is already allocated
I0112 19:11:34.780370 4877 event.go:92] Event(api.ObjectReference{Kind:"BoundPod", Namespace:"default", Name:"cadvisor-agent", UID:"cadvisormanifes12uqn2ohido76855gdecd9roadm7l0", APIVersion:"v1beta2", ResourceVersion:"", FieldPath:"im
plicitly required container net"}): status: 'failed', reason: 'failed' Failed to start with docker id db93e0b55d1193bd7921e276b0592f5aa7ff2b85f2707c5680c1639237f0df42 with error: API error (500): Cannot start container db93e0b55d1193bd792
1e276b0592f5aa7ff2b85f2707c5680c1639237f0df42: Bind for 0.0.0.0:4194 failed: port is already allocated
I0112 19:11:34.780411 4877 event.go:92] Event(api.ObjectReference{Kind:"BoundPod", Namespace:"default", Name:"cadvisor-agent", UID:"cadvisormanifes12uqn2ohido76855gdecd9roadm7l0", APIVersion:"v1beta2", ResourceVersion:"", FieldPath:""}
): status: '', reason: 'failedSync' Error syncing pod, skipping: API error (500): Cannot start container db93e0b55d1193bd7921e276b0592f5aa7ff2b85f2707c5680c1639237f0df42: Bind for 0.0.0.0:4194 failed: port is already allocated
@aullrich
Copy link

Steve, did you report this already somewhere? I have the same issue. As workaround I ended up killing the running cadvisor container (last line in your docker ps -a output).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment