Reproducer for skupper-router #864

README.md

Reproducer for skupper-router #864

skupperproject/skupper-router#864

On the main branch, a deployment is created to serve openssl s_server, which is then exported by Skupper.

The script then repeatedly connects to that server using openssl s_client -reconnect, until the issue on #864 occurs.

There are two branches, for control groups: in local, the connections are done directly on the container running the service, pointing to localhost.

On k8s-service, a pure Kubernetes service is used to expose the deployment and skupper is not engaged on the test.

These two control groups serve to confirm that the issue happens only when Skupper is the part handling the TLS connections.

Skupper, however, is initialized and a service is created with --enable-tls in all three scenarios. For the control group scenarios, that's only to use the certificates created by Skupper for the service when starting openssl s_server.

tls.sh

#!/bin/bash -x

TIMEOUT=5s
WAIT=9s

# echoes something, then sleeps for some time, to keep the
# stdout open as stdin for the next process in the pipe
echo_wait () {
  echo "$@"
  sleep $WAIT
}

kubectl create namespace test-tls

# just one router, to get the issue faster; it happens also with two,
# but it may take longer
skupper init --namespace test-tls --routers 1
skupper service create --namespace test-tls ssl-server 4433 --enable-tls
kubectl create --namespace test-tls -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: ssl-server
  name: ssl-server
spec:
  # replicas: 1
  selector:
    matchLabels:
      app: ssl-server
  template:
    metadata:
      labels:
        app: ssl-server
    spec:
      containers:
      - command:
        - sh
        - -c
        - microdnf install openssl ; openssl s_server -cert /cert/tls.crt -key /cert/tls.key -brief -rev
        image: quay.io/skupper/skupper-tests:master
        imagePullPolicy: IfNotPresent
        name: skupper-tests
        resources: {}
      # restartPolicy: Always
        volumeMounts:
          - mountPath: /cert
            name: cert
      volumes:
        - name: cert
          secret:
            secretName: skupper-tls-ssl-server


EOF

skupper --namespace test-tls service bind ssl-server deployment ssl-server

skupper --namespace test-tls version

count=0
while true
do
  (( count=count+1 ))
  echo "Try $count"
  date
  time echo_wait "reconnect $count - - - - - - -" | kubectl --namespace test-tls exec -i deploy/ssl-server -- openssl s_client -quiet -no_ign_eof -reconnect ssl-server.test-tls.svc.cluster.local:4433 | timeout -v $TIMEOUT grep -B 100 -m 1 '^- - - '
  ret=$?
  if [ "$ret" -ne 0 ]
  then
    echo "Reconnect failed with response $ret"
    if [ "$ret" -eq 124 ]
    then
      break
    fi
  fi

  date
  time echo_wait "Simple $count - - - - - - -" | kubectl --namespace test-tls exec -i deploy/ssl-server -- openssl s_client -quiet -no_ign_eof ssl-server.test-tls.svc.cluster.local:4433 | timeout -v $TIMEOUT grep -B 100 -m 1 '^- - - '
  ret=$?
  if [ "$ret" -ne 0 ]
  then
    echo_wait "Simple failed with response $ret"
    if [ "$ret" -eq 124 ]
    then
      break
    fi
  fi

  sleep 1

done

read -r -p "Type enter to delete namespace and finish"
kubectl delete namespace test-tls