Skip to content

Instantly share code, notes, and snippets.

@aungkyawminn
Last active October 30, 2025 04:20
Show Gist options
  • Save aungkyawminn/e562bfc329dfd49d90dcd8dba021145b to your computer and use it in GitHub Desktop.
Save aungkyawminn/e562bfc329dfd49d90dcd8dba021145b to your computer and use it in GitHub Desktop.
Brief about mTLS (Mutual Transport Layer Security)

Mutual TLS (mTLS) Authentication β€” General Overview

Mutual TLS (mTLS) is an extension of Transport Layer Security (TLS) where both client and server authenticate each other using X.509 digital certificates.
It provides a stronger trust model than standard TLS, where only the server presents a certificate.

mTLS is widely used across ecosystems such as:

  • API Gateways (WSO2, Kong, Apigee, AWS API Gateway)
  • Payment and Financial Networks (Visa VCAS, Mastercard MIP, banking integrations)
  • Microservices and Service Meshes (Envoy, Istio, Linkerd, AWS App Mesh)
  • Cloud and VPN Security (PrivateLink, Zero Trust Networks, Client VPNs)

πŸ” 1️⃣ TLS vs Mutual TLS

Feature TLS (One-way) Mutual TLS (Two-way)
Server Authenticates βœ… Yes βœ… Yes
Client Authenticates ❌ No βœ… Yes
Authentication Method Username/Password or Token Digital Certificate
Security Level Good Very Strong
Use Case Websites, public APIs Financial, corporate, and secure APIs

🧠 2️⃣ How Mutual TLS Works

sequenceDiagram
    participant C as Client
    participant S as Server

    Note over C,S: TLS Handshake with Mutual Authentication

    C->>S: "Client Hello" (supported ciphers, TLS version)
    S->>C: "Server Hello" + Server Certificate
    C->>C: Validate Server Certificate (CA or pinned cert)
    S->>C: Request Client Certificate
    C->>S: Send Client Certificate + Proof (signature)
    S->>S: Validate Client Certificate (CA or whitelist)
    Note over C,S: Both verified β†’ Encrypted session established
    C-->>S: Application data securely transmitted
Loading

In mTLS, both sides present and verify certificates issued by a trusted Certificate Authority (CA).
This ensures identity, confidentiality, and integrity across both directions.


πŸͺͺ 3️⃣ Certificate Management Models

There are two main ways organizations manage mTLS certificates:

Model Description Example Use
Shared Certificate Both peers (client & server) use the same certificate pair. Simple but less granular. Private trusted networks, staging setups
PKI-based Certificates Each peer has its own unique certificate issued by a trusted CA. Financial networks, production-grade systems
flowchart LR
    subgraph "PKI Trust Model"
    CA["Root Certificate Authority"]:::ca
    Server["Server Certificate<br/>(Issued by CA)"]:::server
    Client["Client Certificate<br/>(Issued by CA)"]:::client
    CA --> Server
    CA --> Client
    Client <--> Server
    end

    classDef ca fill:#FFF3E0,stroke:#F57C00,color:#000
    classDef server fill:#E0F7FA,stroke:#006064,color:#000
    classDef client fill:#E8F5E9,stroke:#2E7D32,color:#000
Loading

βœ… Recommended: PKI model with certificate rotation and CA-based validation
βš™οΈ Tools: OpenSSL, AWS ACM, HashiCorp Vault, Entrust, DigiCert, etc.


🧩 4️⃣ mTLS Enforcement Modes

Across systems like Envoy, Istio, WSO2 API Manager, and Visa Network, mTLS enforcement defines how strictly certificates are required for connections.

Mode Description Typical Use
Strict Only mTLS traffic is allowed. Plaintext connections are rejected. Financial transactions, production APIs
Permissive Both mTLS and plaintext traffic are accepted. Migration or hybrid environments
Passthrough Gateway forwards encrypted traffic without terminating TLS. Edge proxy, Visa/3DS message routing
flowchart TB
    subgraph Strict_Mode
        Client1["Client (mTLS Only)"]:::client --> Proxy1["Server / Gateway (Strict Mode)"]:::server
    end

    subgraph Permissive_Mode
        Client2a["Client (mTLS)"]:::client --> Proxy2["Server / Gateway (Permissive Mode)"]:::server
        Client2b["Client (Plaintext)"]:::client --> Proxy2
    end

    subgraph Passthrough_Mode
        Client3["Client (Encrypted TLS)"]:::client --> Proxy3["Proxy (Passthrough Mode)"]:::server --> Backend["Backend System"]:::service
    end

    classDef client fill:#E3F2FD,stroke:#1565C0,color:#000
    classDef server fill:#FFF8E1,stroke:#F9A825,color:#000
    classDef service fill:#E8F5E9,stroke:#2E7D32,color:#000
Loading

🏦 5️⃣ Common Real-world Uses

Industry / Platform mTLS Role Example
Financial Networks Secure issuer ↔ network ↔ acquirer communication Visa VCAS ↔ Issuer CMS (e.g., CardZone)
API Gateways Client authentication at ingress WSO2 API Manager, Kong, AWS API Gateway
Service Mesh Service-to-service encryption Istio / Envoy / App Mesh
Cloud Interconnects VPC or PrivateLink encryption AWS ACM + NLB with client auth
Mobile / IoT Devices Device identity Certificate-based device onboarding

🧾 6️⃣ Key Benefits

Benefit Description
Mutual Trust Both sides verify each other’s identity.
Zero Passwords Authentication without credentials.
Resistance to MITM Prevents man-in-the-middle attacks.
Compliance Ready Meets PCI DSS, PSD2, and Zero Trust requirements.

🧭 7️⃣ Summary

  • TLS secures communication β€” mTLS adds mutual identity verification.
  • Certificates can be shared (simple) or PKI-based (scalable, secure).
  • Enforcement modes (strict, permissive, passthrough) control runtime policy.
  • mTLS underpins security in banking, payments, APIs, IoT, and service mesh ecosystems.

πŸ“š Recommended References


mTLS vs Application-Layer Authentication β€” Layered Security in TCP/IP Architecture

Understanding where different authentication mechanisms operate in the TCP/IP protocol suite is key to designing secure systems.
This document compares Mutual TLS (mTLS) β€” a transport-level authentication β€” with token-based, session-based, and credential-based authentication at the application layer.


🌐 1️⃣ TCP/IP and OSI Layer Reference

OSI Layer TCP/IP Equivalent Example Protocols Purpose
7️⃣ Application Application HTTP, HTTPS, SMTP, gRPC, SOAP Application logic, APIs, user data
6️⃣ Presentation β€” TLS, SSL Encryption, compression, encoding
5️⃣ Session β€” TLS Session, WebSocket Manage session keys and re-use
4️⃣ Transport Transport TCP, UDP End-to-end data delivery
3️⃣ Network Internet IP, ICMP Routing packets
2️⃣ Data Link Link Ethernet, Wi-Fi Frame transmission
1️⃣ Physical Physical Cable, Radio Bit transmission

πŸ” 2️⃣ Mutual TLS (mTLS) β€” Transport Layer Authentication

Mutual TLS (mTLS) authenticates both client and server using digital certificates before any application data is exchanged.

It occurs between the Transport and Application layers β€” typically during the TLS handshake right after TCP connection establishment.

🧩 How mTLS Works

sequenceDiagram
    participant Client
    participant Server

    Client->>Server: TCP SYN (Layer 4)
    Server->>Client: SYN/ACK
    Client->>Server: ACK
    Note over Client,Server: βœ… TCP connection established

    Client->>Server: TLS Handshake Begins
    Server->>Client: Sends Server Certificate
    Client->>Server: Sends Client Certificate
    Note over Client,Server: πŸ” Mutual Authentication Complete

    Client-->>Server: HTTPS Request (Encrypted Application Data)
Loading

βœ… What It Provides

  • Authentication: Both parties prove identity with certificates.
  • Encryption: All data encrypted after handshake.
  • Integrity: Prevents tampering during transit.

🧠 Used In

  • Visa ↔ Issuer or Acquirer secure channels (VCAS, 3DS, MIP)
  • WSO2 / Kong / Apigee gateways for inbound or outbound APIs
  • Bank-to-Bank ISO 20022 or Host-to-Host communication
  • Cloud PrivateLink, VPN, and zero-trust networks

🧩 3️⃣ Token, Session, and API-Key Based Authentication β€” Application Layer

Once a secure channel (TLS/mTLS) is established, application-level authentication ensures the right user, app, or system can access specific resources.

These operate inside the TLS tunnel β€” at the Application Layer (Layer 7).

Mechanism Example Description
Bearer Token (OAuth2, JWT) Authorization: Bearer eyJhbGci... Authenticates the user or service logic
API Key x-api-key: 12345 Identifies calling system or app
Session Cookie Set-Cookie: session_id=abc123 Maintains user login session
HMAC Signature Header-based cryptographic signature Ensures message integrity and authenticity
Client ID / Secret OAuth2 credential exchange Issues tokens based on registered credentials

βœ… What It Provides

  • Access Control β€” Determines who can access which API or resource.
  • Authorization Context β€” Links user, roles, or scopes.
  • Auditing β€” Enables tracking of API consumers or users.

🧭 4️⃣ Layered Relationship β€” Connection vs Application

graph TD
    subgraph Transport_Security["Transport Security (TLS / mTLS)"]
        A["TCP Handshake"] --> B["TLS Handshake (mTLS)"]
        B --> C["Encrypted Channel Established"]
    end

    subgraph Application_Security["Application Security (HTTP / Business Layer)"]
        C --> D["Send Token / Session / API Key"]
        D --> E["Server Verifies Application Credentials"]
    end

    classDef transport fill:#E0F7FA,stroke:#006064,color:#000
    classDef app fill:#E8F5E9,stroke:#2E7D32,color:#000

    class A,B,C transport
    class D,E app
Loading
Concept Layer Authenticates Timing Examples
mTLS Transport Device / Channel Before data transfer Visa ↔ Issuer, API Gateway ↔ Backend
Token / Session Application User / App / Process After connection established OAuth2, JWT, Cookie Session

🧩 mTLS secures the connection β€” tokens secure the intent.


🏦 5️⃣ Real-World Example: Visa ↔ Issuer Communication

Step Security Layer Purpose
1️⃣ Transport Layer (mTLS) Ensures Visa connects only to the legitimate Issuer host.
2️⃣ Application Layer (Token/Signature) Authorizes a specific transaction or risk message.

βœ… Visa verifies the channel identity first (mTLS).
βœ… Then verifies message authenticity (token/signature).


πŸ” 6️⃣ Summary

Layer Mechanism Purpose Scope
Transport Layer mTLS Authenticate and encrypt the channel System / Host Identity
Application Layer Token / Session / API Key Authorize user or app actions Business Logic

πŸ”’ mTLS β€” validates who you are connecting to.
πŸ”‘ Application Authentication β€” validates what you can do once connected.


πŸ“š References


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment