Skip to content

Instantly share code, notes, and snippets.

@libesz
Last active November 9, 2023 16:12
Show Gist options
  • Save libesz/804ad714d47d57ac44c8f2d8dd7ba1f6 to your computer and use it in GitHub Desktop.
Save libesz/804ad714d47d57ac44c8f2d8dd7ba1f6 to your computer and use it in GitHub Desktop.
Egress routing solution with Istio, using a single Envoy listener. Ref: https://github.com/istio/istio.io/issues/14126
# Use-case: have a set of (possibly wildcard) external target hostnames
# that are approached with TLS (HTTPS) by mesh applications. Traffic towards
# these targets are routed through an egress gateway with the mesh mTLS.
# Egress gateway checks the hostname and if allowed, dynamically forwards the
# traffic to the exact destination that is in the SNI property.
#
# The very last patch requires Istio 1.20+. Before 1.20, the istio fork based
# forward_downstream_sni EnvoyFilter implementation can be used.
apiVersion: v1
kind: Service
metadata:
name: egressgateway
namespace: istio-egress
spec:
type: ClusterIP
selector:
istio: egressgateway
ports:
- port: 443
name: tls-egress
targetPort: 8443
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: istio-egressgateway
namespace: istio-egress
spec:
selector:
matchLabels:
istio: egressgateway
template:
metadata:
annotations:
inject.istio.io/templates: gateway
labels:
istio: egressgateway
sidecar.istio.io/inject: "true"
spec:
containers:
- name: istio-proxy
image: auto # The image will automatically update each time the pod starts.
securityContext:
capabilities:
drop:
- ALL
runAsUser: 1337
runAsGroup: 1337
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: istio-egressgateway-sds
namespace: istio-egress
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "watch", "list"]
- apiGroups:
- security.openshift.io
resourceNames:
- anyuid
resources:
- securitycontextconstraints
verbs:
- use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: istio-egressgateway-sds
namespace: istio-egress
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: istio-egressgateway-sds
subjects:
- kind: ServiceAccount
name: default
---
apiVersion: v1
kind: List
items:
- apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: egressgateway
namespace: istio-system
spec:
selector:
istio: egressgateway
servers:
- port:
number: 8443
name: tls-egress
protocol: TLS
hosts:
- "*.foo.com"
tls:
mode: ISTIO_MUTUAL
- apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-wildcard-through-egress-gateway
namespace: istio-system
spec:
hosts:
- "*.foo.com"
gateways:
- mesh
tls:
- match:
- gateways:
- mesh
port: 443
sniHosts:
- "*.foo.com"
route:
- destination:
host: egressgateway.istio-egress.svc.cluster.local
subset: wildcard
- apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-wildcard-through-egress-gateway2
namespace: istio-system
spec:
hosts:
- "*" # not effective, but necessary
gateways:
- egressgateway
tcp:
- match:
- gateways:
- egressgateway
port: 8443
route:
- destination:
host: "dummy.local"
port:
number: 443
weight: 100
- apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: egressgateway
namespace: istio-system
spec:
host: egressgateway.istio-egress.svc.cluster.local
subsets:
- name: wildcard
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
- apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: wildcard
namespace: istio-system
spec:
hosts:
- "*.foo.com"
ports:
- number: 443
name: tls
protocol: TLS
- apiVersion: telemetry.istio.io/v1alpha1
kind: Telemetry
metadata:
name: mesh-default
namespace: istio-system
spec:
accessLogging:
- providers:
- name: envoy
- apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: sni-magic
namespace: istio-system
spec:
configPatches:
- applyTo: CLUSTER
match:
context: GATEWAY
patch:
operation: ADD
value:
name: dynamic_forward_proxy_cluster
lb_policy: CLUSTER_PROVIDED
cluster_type:
name: envoy.clusters.dynamic_forward_proxy
typed_config:
"@type": type.googleapis.com/envoy.extensions.clusters.dynamic_forward_proxy.v3.ClusterConfig
dns_cache_config:
name: dynamic_forward_proxy_cache_config
dns_lookup_family: V4_ONLY
- applyTo: NETWORK_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: "envoy.filters.network.tcp_proxy"
patch:
operation: INSERT_BEFORE
value:
name: envoy.filters.network.sni_dynamic_forward_proxy
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.sni_dynamic_forward_proxy.v3.FilterConfig
port_value: 443
dns_cache_config:
name: dynamic_forward_proxy_cache_config
dns_lookup_family: V4_ONLY
- applyTo: NETWORK_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: "envoy.filters.network.tcp_proxy"
patch:
operation: MERGE
value:
name: envoy.tcp_proxy
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
stat_prefix: tcp
cluster: dynamic_forward_proxy_cluster
- applyTo: NETWORK_FILTER
match:
context: SIDECAR_OUTBOUND
listener:
name: "0.0.0.0_443"
filterChain:
filter:
name: "envoy.filters.network.tcp_proxy"
patch:
operation: INSERT_BEFORE
value:
name: forward_downstream_sni
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.set_filter_state.v3.Config
on_new_connection:
- object_key: envoy.network.upstream_server_name
format_string:
text_format_source:
inline_string: "%REQUESTED_SERVER_NAME%"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment