traefik: node-red + mosquitto using letsencrypt

docker-compose.yaml

version: "3.3"

services:

  traefik:
    image: "traefik:v2.2"
    container_name: "traefik"
    command:
      - "--api=true"
      - "--api.dashboard=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
        
        # Entrypoints 
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--entrypoints.mqtt.address=:8883"
      
        # Redirect http to https
      - "--entrypoints.web.http.redirections.entrypoint.to=websecure"
      - "--entrypoints.web.http.redirections.entrypoint.scheme=https"
      
        # Let's encrypt configuration
      - "--certificatesresolvers.myresolver.acme.tlschallenge=true"
      - "--certificatesresolvers.myresolver.acme.email=email@host.com"
      - "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
    ports:
      - "80:80"
      - "443:443"
      - "8883:8883"
    volumes:
      - "./letsencrypt:/letsencrypt"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
    labels:
      - "traefik.enable=true"
      
      - "traefik.http.routers.dashboard.rule=Host(`traefik.zoo.ocean.mofa.studio`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
      - "traefik.http.routers.dashboard.entrypoints=websecure"
      - "traefik.http.routers.dashboard.service=api@internal"
      - "traefik.http.routers.dashboard.tls.certresolver=myresolver"
      - "traefik.http.routers.dashboard.middlewares=auth"
      - "traefik.http.middlewares.auth.basicauth.users=test:$$apr1$$H6uskkkW$$IgXLP6ewTrSuBkTrqE8wj/,test2:$$apr1$$d9hr9HBB$$4HxwgUir3HP4EsggP/QNo0"

  whoami:
    image: "containous/whoami"
    container_name: "simple-service"
    labels:
      - "traefik.enable=true"

      - "traefik.http.routers.whoami.rule=Host(`whoami.zoo.ocean.mofa.studio`)"
      - "traefik.http.routers.whoami.entrypoints=websecure"
      - "traefik.http.routers.whoami.tls.certresolver=myresolver"
  
  mqtt:
    image: "eclipse-mosquitto"
    container_name: "mosquitto"
    expose:
      - "8883"
      - "9001"
    volumes:
      - "./mosquitto.conf:/mosquitto/config/mosquitto.conf"
    labels:
      - "traefik.enable=true"

      - "traefik.http.routers.mqtt.rule=Host(`mqtt.zoo.ocean.mofa.studio`)"
      - "traefik.http.routers.mqtt.entrypoints=websecure"
      - "traefik.http.routers.mqtt.tls.certresolver=myresolver"

      - "traefik.tcp.routers.mqtt.rule=HostSNI(`*`)"
      - "traefik.tcp.routers.mqtt.tls.certresolver=myresolver"
      - "traefik.tcp.services.mqtt.loadbalancer.server.port=8883"
      
      - "traefik.tcp.routers.mqtt.entrypoints=mqtt"

      - "traefik.http.services.mqtt.loadbalancer.server.port=9001"

  nodered:
    image: "nodered/node-red"
    container_name: "nodered"
    labels:
      - "traefik.enable=true"

      - "traefik.http.routers.nodered.rule=Host(`nodered.zoo.ocean.mofa.studio`)"
      - "traefik.http.routers.nodered.entrypoints=websecure"
      - "traefik.http.routers.nodered.tls.certresolver=myresolver"

      - "traefik.http.services.nodered.loadbalancer.server.port=1880"

mosquitto.conf

port 8883

listener 9001
protocol websockets

Can you show the main traefik config ?

Sure:

docker compose piece:

  traefik:
    image: traefik:v2.4
    container_name: traefik
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true
    networks:
      - public_facing
    ports:
      - "80:80"
      - "443:443"
      - "8883:8883"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ${DOCKER}/traefik/acme.json:/acme.json
      - ${DOCKER}/traefik/traefik.yml:/traefik.yml:ro
      - ${DOCKER}/traefik/users:/users:ro
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.entrypoints=http"
      - "traefik.http.routers.traefik.rule=Host(`traefik.${DOMAINNAME}`)"
      - "traefik.http.middlewares.traefik-auth.basicauth.usersfile=/users"
      - "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
      - "traefik.http.routers.traefik-secure.entrypoints=https"
      - "traefik.http.routers.traefik-secure.rule=Host(`traefik.${DOMAINNAME}`)"
      - "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
      - "traefik.http.routers.traefik-secure.tls=true"
      - "traefik.http.routers.traefik-secure.tls.certresolver=http"
      - "traefik.http.routers.traefik-secure.service=api@internal"

traefik.yml

api:
  dashboard: true

entryPoints:
  http:
    address: ":80"
  https:
    address: ":443"
  mqtt:
    address: ":8883"

providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false

certificatesResolvers:
  http:
    acme:
      email: fake@email.com
      storage: acme.json
      httpChallenge:
        entryPoint: http

Can you try using my examples to see if it works ? Yours are quite modified so I'm not sure where the issue could be....

Nice docker script. But for using TCP TLS with 8883 port you should change protocol to mqtt instead websocket into mosquitto.conf .

The configuration works for mosquitto 1.x but the ´port’ configuration is deprecated so I will update it.

The goal is to have both TCP and websockets, that’s why both configurations are there.

In this way, I'm guessing some malicious client can also pub/sub to your mqtt broker if somebody knows your server URI.
Am I correct??

I'm trying to setup client certificate type of authentication but with no luck. mqtt broker is behind Traefik as you do.
a reference of client certificate type of authentication is here:
https://primalcortex.wordpress.com/2016/11/08/mqtt-mosquitto-broker-client-authentication-and-client-certificates/
Do you have any experience on this??

I use mosquitto authentication to secure the broker, could this be enough for your use case ?

That may work, I'll give it a try

…

On Thu, May 19, 2022, 6:40 AM Nathanaël Lécaudé ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ I use mosquitto authentication to secure the broker, could this be enough for your use case ? — Reply to this email directly, view it on GitHub <https://gist.github.com/ed8253a34e7b87d879baabeba82cb846#gistcomment-4172110>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMO6L75PZQAOA7NACKWG3TTVKYSDZANCNFSM4T7ORYGQ> . You are receiving this because you commented.Message ID: ***@***.***>

Thank you for your help!
Very short answer is No.
Why I am doing this because I'd like to run eclipse-mosquitto broker (mqqts) on 443/tcp to make some firewalls happy.
My broker is a part of Netmaker (https://github.com/gravitl/netmaker) system that needs client certificate authentication.
I tried tls.passthrough with hostSNI("*") but no luck,,

I believe this could work but I doubt you'll be able to make it work using let's encrypt, you'll probably need you own certificates.

Yes, without reverse proxy, client cert authentication works perfect with own ca. but behind traefik it doesn’t work,,
Thank you

Hi, a doubt..in your docker-compose file why are you using the entrypoint for 8883 if you are using the websecure entrypoint in the mqtt section config?

@iboluda The 8883 entry point is for TCP connections while the 443 is for web sockets, does that answer your question ?

Thanks for clarify me that point. Yes that answer my question