Skip to content

Instantly share code, notes, and snippets.

@alpeb

alpeb/gateway service dies

Created May 20, 2020 17:43
Show Gist options
  • Save alpeb/edc97c8c3cd164af230d6f955daeef12 to your computer and use it in GitHub Desktop.
Save alpeb/edc97c8c3cd164af230d6f955daeef12 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gateway service dies
time="2020-05-20T16:36:03Z" level=debug msg="Gateway is healthy" probe-key=linkerd-multicluster-linkerd-gateway-macbook
I0520 16:36:05.363727 1 reflector.go:268] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:105: forcing resync
time="2020-05-20T16:36:05Z" level=debug msg="Received: OnUpdateCalled: {svc: Service: {name: linkerd-prometheus, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=prometheus,linkerd.io/control-pla
ne-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: OnUpdateCalled: {svc: Service: {name: linkerd-web, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=web,linkerd.io/control-plane-ns=linkerd]
]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook time="2020-05-20T16:36:05Z" level=debug msg="Received: OnUpdateCalled: {svc: Service: {name: kube-dns, namespace: kube-system, annotations: [[]], labels [[]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: OnUpdateCalled: {svc: Service: {name: linkerd-identity, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=identity,linkerd.io/control-plane-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: ConsiderGatewayUpdateDispatch: {maybeGateway: Service: {name: linkerd-prometheus, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=prometheus,linkerd.io/control-plane-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: OnUpdateCalled: {svc: Service: {name: backend-one-svc, namespace: multicluster-test, annotations: [[mirror.linkerd.io/gateway-name=linkerd-gateway,mirror.linkerd.io/gateway-ns=linkerd-multicluster]], labels [[]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: OnUpdateCalled: {svc: Service: {name: linkerd-controller-api, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=controller,linkerd.io/control-plane-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: OnUpdateCalled: {svc: Service: {name: backend-two-svc, namespace: multicluster-test, annotations: [[]], labels [[]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook time="2020-05-20T16:36:05Z" level=debug msg="Received: OnUpdateCalled: {svc: Service: {name: linkerd-sp-validator, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=sp-validator,linkerd.io/control
-plane-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook time="2020-05-20T16:36:05Z" level=debug msg="Received: OnUpdateCalled: {svc: Service: {name: linkerd-tap, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=tap,linkerd.io/control-plane-ns=linkerd]
]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook time="2020-05-20T16:36:05Z" level=debug msg="Received: OnUpdateCalled: {svc: Service: {name: linkerd-gateway, namespace: linkerd-multicluster, annotations: [[mirror.linkerd.io/probe-path=/health,mirror.linkerd.io/probe-period=3,mirror.linkerd.io/probe-port=4181,linkerd.io/c
reated-by=linkerd/cli git-8b0de4a6,mirror.linkerd.io/gateway-identity=linkerd-gateway.linkerd-multicluster.serviceaccount.identity.linkerd.cluster.local]], labels [[]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook time="2020-05-20T16:36:05Z" level=debug msg="Received: OnUpdateCalled: {svc: Service: {name: kubernetes, namespace: default, annotations: [[]], labels [[]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: OnUpdateCalled: {svc: Service: {name: linkerd-proxy-injector, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=proxy-injector,linkerd.io/control-plane-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: OnUpdateCalled: {svc: Service: {name: linkerd-grafana, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=grafana,linkerd.io/control-plane-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: OnUpdateCalled: {svc: Service: {name: linkerd-dst, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=destination,linkerd.io/control-plane-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: ConsiderGatewayUpdateDispatch: {maybeGateway: Service: {name: linkerd-web, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=web,linkerd.io/control-plane-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: ConsiderGatewayUpdateDispatch: {maybeGateway: Service: {name: kube-dns, namespace: kube-system, annotations: [[]], labels [[]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook time="2020-05-20T16:36:05Z" level=debug msg="Received: ConsiderGatewayUpdateDispatch: {maybeGateway: Service: {name: linkerd-identity, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=identity,li
nkerd.io/control-plane-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook time="2020-05-20T16:36:05Z" level=debug msg="Received: ConsiderGatewayUpdateDispatch: {maybeGateway: Service: {name: linkerd-controller-api, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=contr
oller,linkerd.io/control-plane-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook time="2020-05-20T16:36:05Z" level=debug msg="Received: ConsiderGatewayUpdateDispatch: {maybeGateway: Service: {name: backend-two-svc, namespace: multicluster-test, annotations: [[]], labels [[]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: ConsiderGatewayUpdateDispatch: {maybeGateway: Service: {name: linkerd-sp-validator, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-ns=linkerd,linkerd.io/control-plane-component=sp-validator]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: ConsiderGatewayUpdateDispatch: {maybeGateway: Service: {name: linkerd-tap, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=tap,linkerd.io/control-plane-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: RemoteServiceDeleted: {name: linkerd-gateway, namespace: linkerd-multicluster, gatewayData: gatewayMetadata: {name: , namespace: }}" apiAddress="https://192.168.0.227:6443" cluster=macbook time="2020-05-20T16:36:05Z" level=debug msg="Deleting mirrored service linkerd-multicluster/linkerd-gateway-macbook and its corresponding Endpoints" apiAddress="https://192.168.0.227:6443" cluster=macbook
I0520 16:36:05.383369 1 round_trippers.go:443] DELETE https://10.96.0.1:443/api/v1/namespaces/linkerd-multicluster/services/linkerd-gateway-macbook 200 OK in 17 milliseconds I0520 16:36:05.388345 1 round_trippers.go:443] DELETE https://10.96.0.1:443/api/v1/namespaces/linkerd-multicluster/endpoints/linkerd-gateway-macbook 404 Not Found in 4 milliseconds
time="2020-05-20T16:36:05Z" level=debug msg="Successfully deleted Service: linkerd-multicluster/linkerd-gateway-macbook" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: ConsiderGatewayUpdateDispatch: {maybeGateway: Service: {name: kubernetes, namespace: default, annotations: [[]], labels [[]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: ConsiderGatewayUpdateDispatch: {maybeGateway: Service: {name: linkerd-proxy-injector, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=proxy-injector,linkerd.io/control-plane-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Probe Manager: received event: MirroredServiceUnpaired: {serviceName: linkerd-gateway-macbook, serviceNamespace: linkerd-multicluster, gatewayName: , gatewayNs: , clusterName: macbook}"
time="2020-05-20T16:36:05Z" level=debug msg="Received: ConsiderGatewayUpdateDispatch: {maybeGateway: Service: {name: linkerd-grafana, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=grafana,link
erd.io/control-plane-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Received: ConsiderGatewayUpdateDispatch: {maybeGateway: Service: {name: linkerd-dst, namespace: linkerd, annotations: [[linkerd.io/created-by=linkerd/cli edge-20.5.3]], labels [[linkerd.io/control-plane-component=destination,link
erd.io/control-plane-ns=linkerd]]}}" apiAddress="https://192.168.0.227:6443" cluster=macbook
time="2020-05-20T16:36:05Z" level=debug msg="Could not find a worker for --macbook while handling MirroredServiceUnpaired event"
I0520 16:36:06.411619 1 reflector.go:418] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:105: Watch close - *v1.Service total 1 items received
I0520 16:36:06.413448 1 round_trippers.go:443] GET https://10.96.0.1:443/api/v1/services?allowWatchBookmarks=true&resourceVersion=26340&timeout=8m8s&timeoutSeconds=488&watch=true 200 OK in 1 milliseconds
time="2020-05-20T16:36:08Z" level=debug msg="Gateway returned unexpected status 503. Marking as unhealthy" probe-key=linkerd-multicluster-linkerd-gateway-macbook
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment