Scribble issue when documented bindings are provided by multiple modules

0-Scribble issue.md

I've set this gist up to illustrate an issue with Scribble docs.

In my package, I have the structure shown below. The actual files are also included in the gist below, but for the diagram I’ve stripped them down to their essential elements:

mole/main.rkt══════════════════════════╗    
║                                      ║    
║(require mole/func)                   ║    
║                                      ║    
║(provide (all-from-out mole/func))    ║    
╚══════════════════════════════════════╝    
   ▲                                        
   │                                        
  mole/func.rkt═══════════════════════════╗ 
  ║                                       ║ 
  ║ (require mole/private/text)           ║ 
  ║ (provide                              ║ 
  ║   [contract-out (cloak (-> ...))])    ║ 
  ╚═══════════════════════════════════════╝ 
    ▲                                       
    │                                       
   mole/private/text.rkt═══════════════════╗
   ║ (provide cloak)                       ║
   ║                                       ║
   ║ (define (cloak ...))                  ║
   ║                                       ║
   ╚═══════════════════════════════════════╝

The Scribble docs are set up this way:

mole.scrbl──────────────────────────────────────────┐
│                                                   │
│ @defmodule[mole]                                  │
│                                                   │
│ @include-section["tutorial.scrbl"]                │
│ @include-section["reference.scrbl"]               │
│                                                   │
└───────────────────────────────────────────────────┘
                                                     
reference.scrbl─────────────────────────────────────┐
│                                                   │
│ @(require [for/label mole])                       │
│                                                   │
│ @defmodule[mole/func]                             │
│                                                   │
│ These bindings also provided by                   │
│ @racktetmodname[mole].                            │
│                                                   │
│                                                   │
│ @defproc[(cloak ...)]{...}                        │
└───────────────────────────────────────────────────┘

What I want to happen with the Scribble docs

• No warnings when built with raco setup
• No warnings when built with scribble +m ...
• (stretch goal) cloak is documented as being provided from both mole and mole/func

What actually happens

Running raco setup works fine:

$ raco setup mole
[...]
raco setup: --- creating launchers ---                             [14:02:21]
raco setup: --- installing man pages ---                           [14:02:21]
raco setup: --- building documentation ---                         [14:02:21]
raco setup: running: <pkgs>/mole/scribblings/mole.scrbl
raco setup: rendering: <pkgs>/mole/scribblings/mole.scrbl
raco setup: --- installing collections ---                         [14:02:22]
raco setup: --- post-installing collections ---                    [14:02:22]

However, when rendering separately with scribble there are warnings:

$ scribble +m -l mole/scribblings/mole.scrbl
 [Output to mole.html]
Warning: some cross references may be broken due to undefined tags:
 (dep ((lib "mole/main.rkt") cloak))
 (dep ((lib "mole/main.rkt") uncloak))

Functions in reference.scrbl are shown as being provided by mole/func but not mole:

Note

It looks like in order to get multiple module paths to show up in that tooltip, I might need to do something similar to what the Racket Reference docs do with make-collect-element etc. I'm unable to find any documentation on the use of (collect-put! ci `(racket-extra-lib ,'lib) ...) except some notes on the collect-info struct that indicate it's not intended for public use.

Note on module/file structure

The intent here is that private/ contains implementation modules without contracts, func re-exports a subset of all functions in the package with contracts, and main re-exports everything with contracts.

In actual packages where I have this issue, there is more than one module alongside func in that middle layer. Ideally all of their functions would also appear in the docs as being provided from both the

What I’ve tried

Here are some things I've done that have had no effect.

Add #:use-sources

; mole.scrbl
-@defmodule[mole]
+@defmodule[mole #:use-sources (mole/private/text)]

; reference.scrbl
-@defmodule[mole/func]
+@defmodule[mole/func #:use-sources (mole/private/text)]

I've also tried all permutations of including mole and/or mole/func alongside mole/private/text in either or both of these files. Sometimes the result is that scribble no longer reports any warnings, but then raco setup does report warnings.

For example, leaving the line in mole.scrbl as defmodule[mole] and changing reference.scrbl:

; reference.scrbl
-@defmodule[mole/func]
+@defmodule[mole/func #:use-sources (mole/func mole/private/text)]

…results in no warnings from scribble +m … but then warnings appear in raco setup:

raco setup: running: <pkgs>/mole/scribblings/mole.scrbl
raco setup: WARNING: undefined tag in <pkgs>/mole/scribblings/mole.scrbl:
raco setup:  (def ((lib "mole/main.rkt") cloak))
raco setup:  (def ((lib "mole/main.rkt") uncloak))

Invoke scribble with -l

From mflatt on Discord today:

Ah, my guess is that the difference is whether you refer to the documentation via raco scribble as a filesystem path or a collection-based path using -l. When raco setup renders documentation, it refers to them via collection paths, and so relative paths like "../some-path-to-module.rkt" are also collection-relative. But if you use a filesystem path to refer to the document, then the relative path is also a filesystem path, and so there's a disconnect with a collection-based module name used elsewhere.

However, I finbd that the result is the same whether I use scribble +m mole.scrbl or scribble +m -l mole/scribblings/mole.scrbl.

func.rkt

#lang racket/base

(require racket/contract)

(require mole/private/text)

(provide
 [contract-out
  [cloak (-> string? #:in string? string?)]
  [uncloak (-> string? string?)]])

info.rkt

#lang info
(define collection "mole")
(define deps '("threading-lib"
               "base"))
(define build-deps '("scribble-lib" "racket-doc" "rackunit-lib"))
(define scribblings '(("scribblings/mole.scrbl" ())))
(define pkg-desc "Hide text in other text")
(define version "0.0")
(define pkg-authors '(joel))
(define license '(Apache-2.0 OR MIT))

main.rkt

#lang racket/base

(require mole/func)

(provide (all-from-out mole/func))

private⁄text.rkt

#lang racket/base
; note: filename simulates subfolder with unicode fraction slash since gists can't have folders

(require net/base64
         threading)

(provide (all-defined-out))

(define FIRST-INVISIBLE-CHAR 917760)

(define (invis-encode str)
  (list->string
   (for/list ([c (in-list (string->list str))]
              #:do [(define cnum (char->integer c))]
              #:when (<= cnum 127))
     (integer->char (+ cnum FIRST-INVISIBLE-CHAR)))))

(define (invis-decode str)
  (list->string
   (for/list ([c (in-list (string->list str))]
              #:do [(define plaintxt-c (- (char->integer c) FIRST-INVISIBLE-CHAR))]
              #:when (> plaintxt-c 0))
     (integer->char plaintxt-c))))

(define (cloak secret #:in plain)
  (~> (string->bytes/utf-8 secret)
      (base64-encode #"")         ; use #"" vs #"\r\n" to prevent line-wrapping
      (bytes->string/utf-8)
      (invis-encode)
      (string-append plain _)))

(define (uncloak ciphertext)
  (~> (invis-decode ciphertext)
      (string->bytes/utf-8)
      (base64-decode)
      (bytes->string/utf-8)))

(module+ test
  (require rackunit)
  (define secret "this is a s3cret message. ssh")
  (define plaintext "Hey you, nothing to see here.")
  (define to-share (cloak secret #:in plaintext))

  (check-equal? (string-length to-share) 69)         ; count of bytes
  (check-equal? (string-grapheme-count to-share) 29) ; 29 actually-visible graphemes
  (check-equal? secret (uncloak to-share)))

scribblings⁄mole.scrbl

#lang scribble/manual
@require[@for-label[mole
                    racket/base]]

@title{mole}
@author{joel}

@defmodule[mole]

@include-section["tutorial.scrbl"]
@include-section["reference.scrbl"]

@local-table-of-contents[]

scribblings⁄reference.scrbl

#lang scribble/manual

@(require (for-label mole racket/base))

@title{Reference}

@defmodule[mole/func]

@defproc[(cloak [secret string?] [#:in plaintext string?]) string?]{

Encodes @racket[secret] as invisible Unicode characters and appends them to @racket[plaintext].

}

@defproc[(uncloak [ciphertext string?]) string?]{

Decodes invisible Unicode characters in @racket[ciphertext].

}

scribblings⁄tutorial.scrbl

#lang scribble/manual

@(require scribble/examples
          (for-label mole
                     racket/base))

@(define e (make-base-eval #:lang 'racket/base))
@(e '(require mole))

@title{Tutorial}

@examples[#:eval e
          (define c (cloak "SECRET" #:in "a normal string"))
          c
          (string-length c)
          (string-grapheme-count c)
          (uncloak c)]