Static Pages monorepo: build, merge, and deploy architecture

static-pages-monorepo-architecture.md

How This Monorepo Builds, Merges, and Ships Static Pages

This repo is a Turborepo monorepo (npm workspaces + Turbo task orchestration). It does not run a long-lived Node server in production. Everything is pre-rendered at build time into plain HTML/CSS/JS files, bundled into a tarball, uploaded to S3, and served by nginx on DuckDuckGo's edge.

---

The Big Picture

flowchart LR
  subgraph monorepo [Monorepo]
    SP["apps/static-pages<br/>Pages Router"]
    NS["nextssg<br/>App Router"]
    PKGS["packages<br/>polyfills, inline-scripts, design-system, utils"]
  end

  subgraph build [Build]
    TB["turbo run merge-output"]
    OUT["apps/static-pages/out/"]
  end

  subgraph ci [CI on main]
    TAR["build-output.tar.gz"]
    S3["S3 artifacts/SHA.tar.gz"]
  end

  subgraph prod [Production]
    CACHE["/usr/local/ddg.cache/static-pages/"]
    NGINX["nginx rewrites URLs to .html files"]
  end

  PKGS --> SP
  PKGS --> NS
  SP --> TB
  NS --> TB
  TB --> OUT
  OUT --> TAR --> S3 --> CACHE --> NGINX

Loading

There are two Next.js apps that both emit static files, plus shared packages they depend on. At the end of a full build, there is one deployable directory: apps/static-pages/out/.

---

Monorepo Layout

Piece	Role
apps/static-pages	Main site — hundreds of marketing, subscription, and in-browser pages (/about, /pro, /app, DBP, etc.)
apps/nextssg	Performance-focused homepage rebuild (currently the noai homepage only)
apps/benchmark	Lighthouse regression tooling for nextssg only — not part of production output
packages/*	Shared code: design system components, polyfill bootloader, inline IIFE scripts, utilities, static file server

Turbo wires dependency order: app build tasks depend on ^build (build workspace dependencies first). Packages like @staticpages/polyfills and @staticpages/inline-scripts are compiled with esbuild/vite before the Next apps consume them.

---

How Individual Pages Get Built

Both apps use Next.js with output: "export" in production — meaning next build walks the route tree and writes finished HTML files to an out/ folder. No SSR at request time.

static-pages (the bulk of the site)

• Router: Next.js Pages Router (src/pages/)
• Route types:
  • Unlocalized pages at the top level (src/pages/about.tsx → /about.html)
  • Localized pages under src/pages/[locale]/ (/en-US/app.html, /es-ES/app.html, …)
  • Experiment variants as nested dynamic routes (e.g. experiment_appbranding/control.html)
• SSG hooks: Pages export getStaticPaths / getStaticProps via helpers makeGetStaticPaths() and makeGetStaticProps(). At build time, Next enumerates every locale × variant combination and renders each page to HTML.
• i18n: Strings go through react-intl + FormatJS. Before build, npm run i18n:compile turns Smartling JSON into compiled AST message files in text-compiled/.
• Build command: npm run build → compile i18n → next build --no-lint → writes to apps/static-pages/out/

nextssg (homepage rebuild)

• Router: Next.js App Router (src/app/)
• Scope: Small — noai homepage, mobile homepage, a few related routes
• Asset prefix: Production assets are prefixed with /static-pages/turbo so they don't collide with the main app's _next/ bundles
• Build command: next build → browser-compat post-processing → writes to apps/nextssg/out/
• Extra packages: Pulls in polyfills and inline-scripts more aggressively for perf

---

The Merge Step (Two Apps → One Artifact)

The two apps build independently, then get stitched together:

npm run merge-output
# equivalent to:
# rm -rf apps/static-pages/out/turbo
# cp -r apps/nextssg/out/ apps/static-pages/out/turbo/

In Turbo, this is the root task //#merge-output, which depends on both static-pages#build and nextssg#build. The convenience wrapper is:

just build          # runs: npx turbo run merge-output
npm run build       # runs: turbo run build (apps only, no merge)

Resulting layout:

apps/static-pages/out/
├── en-US/
│   ├── app.html
│   ├── pro/login.html
│   └── ...
├── about.html
├── _next/              ← static-pages JS/CSS bundles
├── static-assets/      ← images, sprites, etc.
└── turbo/              ← entire nextssg output, mounted at /static-pages/turbo/
    ├── en-US/
    │   └── index.html  ← noai homepage
    └── _next/          ← nextssg JS/CSS bundles (prefixed paths)

Nginx decides which tree to serve for a given URL — the main site for most routes, the turbo/ subtree for the optimized homepage.

---

Command Sequence (Typical Flows)

Local development

just install          # npm install
just dev              # turbo run dev — static-pages on :3000

nextssg dev runs separately via its workspace (ddg-serve out 3001 --base-path /static-pages/turbo after a build, or next dev in that app).

Full production-like build

just build            # turbo merge-output (builds both apps + merges)
just compress         # gzip + brotli every .html/.js/.css in out/
just deploy-from-dev  # rsync out/ → /usr/local/ddg.cache/static-pages/

Quality gates (what CI runs)

npx turbo run lint check-types lint:css format-check test
npx turbo run merge-output
npx turbo run test:integration --filter=static-pages
just test-smoke       # lightweight node tests against deployed HTML

---

CI Pipeline → S3 Artifact

On every PR and push to main, GitHub Actions roughly does:

1. Build job: npx turbo run merge-output
2. Tarball: tar -czf build-output.tar.gz -C ./apps/static-pages/out .
3. Upload artifact (retained ~10 days for PR deploys and test jobs)
4. Lint, unit tests, integration tests, nextssg e2e, smoke tests run in parallel / downstream
5. On main only, after all green: download artifact → just out-upload which compresses and uploads {git-sha}.tar.gz to s3://ddg-static-pages/artifacts/

The tarball contents are exactly what's in apps/static-pages/out/ — HTML files, _next/ chunks, pre-compressed .gz and .br siblings, static assets, and the turbo/ subtree.

---

What Gets Served in Production

Question	Answer
What folder on disk?	/usr/local/ddg.cache/static-pages/ (rsync'd from the tarball)
Who serves it?	nginx (config lives in the separate ddg repo, not this one)
How do URLs map?	nginx rewrites friendly URLs to file paths, e.g. /app → /static-pages/en-US/app.html. Locale is chosen via an nginx variable ($static_pages_locale).
Homepage routing	The optimized nextssg homepage is served from the turbo/ prefix under /static-pages/turbo/…
Is Node running?	No — pure static files. Pre-compressed gzip/brotli variants let nginx serve compressed responses without on-the-fly compression.

Release cadence

This repo builds and uploads artifacts. Actual production rollout is driven by the ddg repo's release workflow:

• Finds a recent commit whose tarball exists in S3
• Creates a GitHub release tag like static-pages_20240916_172053_ET
• Passes that ref to rex deploy --static-pages-ref=… on staging/canary/production
• Updates s3://ddg-static-pages/latest.txt to point at the deployed SHA

Rollback is also orchestrated from ddg (redeploy a previous tag, mark bad releases).

---

Shared Packages (High Level)

These aren't pages themselves, but they shape what ends up in the HTML:

Package	Purpose
design-system	Shared Button, Text, Heading, Card — imported by both apps
polyfills	Legacy-browser bootloader + per-locale intl chunks (esbuild-built IIFEs)
inline-scripts	Tiny IIFEs inlined in <head> (theme, device classes, error handler, pixels)
browser-compat	Post-build checks/fixes for ES5 compatibility
ddg-serve	Dev static server with autocomplete proxy (nextssg + benchmark)
utils	Shared helpers

---

Mental Model for a Monorepo Like This

1. Multiple apps can coexist if their outputs are namespaced (turbo/ subdirectory + asset prefix).
2. Turbo orchestrates build order and caching across workspaces — one merge-output task is the "release build."
3. The final artifact is boring on purpose: a directory of HTML + hashed JS/CSS + compressed copies. No container, no runtime.
4. Routing is split: Next.js decides file paths at build time; nginx decides public URLs at serve time.
5. CI produces immutable SHA-keyed tarballs; deployment is "pick a tarball, rsync to cache dir, nginx serves it."