Nginx Reverse Proxy for Nexus Docker Registries

README.md

Overview

This is a solution of a common problem with Nexus Docker repositories. The administrator has to expose port for "pull", another port for "push", other ports for each hosted repository. This solution is about leveraging Nginx reverse proxy to avoid using these ports.

How it works ?

Given :

• Nexus hostname is "nexus.example.com"
• Nexus web port is 8081
• A hosted repository is named "docker-hosted"
• A group repository is named "docker-group"
• Your nginx (with the nginx.conf of this gist) will run for example under cregistry.example.com

The following Nginx configuration file is for a reverse proxy without the need to expose connector ports from nexus :

• docker pull cregistry.example.com/myimage lets Nginx forward the request to "docker-group"
• docker push cregistry.example.com/myimage lets Nginx forward the request to "docker-hosted"

Notes

• If you have more than one hosted repository, create another Nginx reverse proxy for it, then aggregate them using a parent Nginx reverse proxy that forwards the request according to certain criteria (.i.e: Host header).

• All Nexus repositories must have consistent configuration of authentication: Either all require authentication, or all don't.

• If TLS is enabled with Nexus, change proxy_set_header X-Forwarded-Proto "http"; by proxy_set_header X-Forwarded-Proto "https";

docker-compose.yaml

version: "3"

services:
  web:
    image: nginx:1.15
    hostname: cregistry.example.com
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
    ports:
      - "80:80"
  nexus:
    image: sonatype/nexus3
    hostname: nexus.example.com
    volumes:
      - "nexus-data:/nexus-data"
    ports:
      - "8081:8081"
 
volumes:
  nexus-data: {}

nginx.conf

events {
  worker_connections  1024;
}

http {
  
  proxy_send_timeout 120;
  proxy_read_timeout 300;
  proxy_buffering    off;
  keepalive_timeout  5 5;
  tcp_nodelay        on;
  # disable any limits to avoid HTTP 413 for large image uploads
  client_max_body_size 0;


  server {
      listen   *:80;

      location ~ ^/(v1|v2)/[^/]+/?[^/]+/blobs/ {
          if ($request_method ~* (POST|PUT|DELETE|PATCH|HEAD) ) {
              rewrite ^/(.*)$ /repository/docker-hosted/$1 last;
          }
          rewrite ^/(.*)$ /repository/docker-group/$1 last;
      }

      location ~ ^/(v1|v2)/ {
          if ($request_method ~* (POST|PUT|DELETE|PATCH) ) {
              rewrite ^/(.*)$ /repository/docker-hosted/$1 last;
          }
          rewrite ^/(.*)$ /repository/docker-group/$1 last;
      }

      location / {
          proxy_pass http://nexus.example.com:8081/;
          proxy_set_header Host $host;
          proxy_set_header X-Real-IP $remote_addr;
          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
          proxy_set_header X-Forwarded-Proto "http";
      }
  }
} 

@MauricioPenteado hi, unfortunately it's mandatory for configuration. You must create repos but not need used for store your images.

Unfortunately, when pushing a docker image to the repo, I have gotten the following error:

"unexpected status from POST request to https://cregistry.example.com/v2/nginx/blobs/uploads/: 403 Deploying to groups is a PRO-licensed feature. See https://links.sonatype.com/product-nexus-repository
"

Is there a way to bypass it?
I am using Nexus OSS.
Could anyone help?

@MauricioPenteado i think you not correct set image URL or not fully complete instructions, this behaviour not happen when all done

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nexus
  namespace: nexusrepo
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: '0'
    nginx.ingress.kubernetes.io/proxy-read-timeout: '600'
    nginx.ingress.kubernetes.io/proxy-request-buffering: 'off'
    nginx.ingress.kubernetes.io/proxy-send-timeout: '600'
    nginx.ingress.kubernetes.io/server-snippet: >
      location ~
      ^/(?!repository\/(?:docker-private|docker-autotest|docker-public))(v1|v2)/[^/]+/?[^/]+/blobs/
      {
          if ($uri ~* repository/docker-private) {
              rewrite ^/(.*)$ /repository/docker-private/$1 last;
          }

          if ($uri ~* repository/docker-autotest) {
              rewrite ^/(.*)$ /repository/docker-autotest/$1 last;
          }
          rewrite ^/(.*)$ /repository/docker-public/$1 last;
      }


      location ~
      ^/(?!repository\/(?:docker-private|docker-autotest|docker-public))(v1|v2)/
      {
          if ($uri ~* repository/docker-private) {
              rewrite ^/(.*)$ /repository/docker-private/$1 last;
          }

          if ($uri ~* repository/docker-autotest) {
              rewrite ^/(.*)$ /repository/docker-autotest/$1 last;
          }
          rewrite ^/(.*)$ /repository/docker-public/$1 last;
      }
    nginx.ingress.kubernetes.io/ssl-redirect: 'true'
    nginx.ingress.kubernetes.io/x-forwarded-proto: https
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - 
  rules:
    - host: 
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: nexus-nxrm-ha-hl
                port:
                  number: 8081

Thanks for the config! I wrote the config similarly, but through proxy_pass instead of rewrite (could share if will be needed), and there's a nuance:
Let's say my Nexus has a domain name mycompany.com. There are three standard Docker repositories: docker-hosted, docker-proxy, docker-group. The docker-group includes both docker-hosted and docker-proxy. Nexus 3.77.

Let's say there's this dockerfile:
FROM memcached:1.6.38
RUN do something
...

The server itself is configured to use mycompany.com as a mirror-registry, so in the FROM line we don't need to explicitly write mycompany.com/memcached:1.6.38, the image will be pulled through docker-proxy and saved.

Then I execute "docker build -t mycompany.com/memcached:v1.0"
After building "docker push mycompany.com/memcached:v1.0"

And the push crashes, the layers which has base FROM image (mmecached:1.6.38) never gets pushed to the docker-hosted repo.

Here's the problem: Docker tries to mount the layer (Docker's internal optimization mechanism) of the base image from FROM, which is located in the docker-proxy repository, to the docker-hosted repository. It can't do this (because these are different repositories but the image name is the same), it keeps trying again until timeout. I managed to fix it by removing the base image from the docker-proxy repo and pushing again.

It's clear that the most obvious solution is: simply don't name the resulting image based on the base image name, i.e., if the base is memcached:1.6.38, name yours, for example, mycompany-memcached:1.6.38 and push it.

I want to understand if everyone really adheres to this approach, or if I'm just missing something in my configuration.

TLDR Why I can't build and push to docker-hosted repo an image with the same name, as one of the images in the docker-proxy (i.e. I used FROM node:14 , docker-proxy now has node:14 saved, built an image and named it node:14, and tried to push it, it will fail for me, because docker tried to get node:14 layers from proxy-repo into the hosted-repo, is this a normal behaviour?).