Proton Photos bulk export toolkit (unofficial) - browser console automation + local validator

README.md

Proton Photos bulk export toolkit

Unofficial tools for getting everything out of Proton Photos. Proton Drive/Photos has no bulk "download my whole library" button and no export API, so these drive the web UI itself from the browser console: select a batch of photos, click download, wait for the zip, repeat - with a resumable, conservative-by-design state model, and a way to independently verify nothing got missed.

Not affiliated with or endorsed by Proton. This works by reading Proton Photos' current web UI (data-testid attributes, DOM structure, CSS classes on the transfers panel). If Proton changes that UI, these scripts can break - possibly silently. That's exactly why the validator exists: don't trust a large run blind, check it.

The three pieces

1. proton-photos-export.js - the exporter. Run in the browser console, walks your Photos library from the top (or wherever it left off) downloading fixed-size batches as zips. See the comment block at the top of the file for the design rationale (resumable state, why fixed batch size instead of per-month, why the next batch prefetches during the current download).

2. proton-photos-inventory.js - a separate, read-only scanner. It never clicks or downloads anything - it just scrolls the whole library and records every item it can see (uid + filename), then downloads that as proton-photos-manifest.json. This is independent ground truth: it can't inherit a bug from the exporter because it doesn't read the exporter's state at all.

3. validate-proton-export.py - run locally (not in the browser). Cross-checks the manifest against every zip actually sitting in your export folder, by filename. Reports anything missing, any unreadable/corrupt zip, and any zip with an unexpected item count. If it finds anything missing, it automatically writes a fourth script, proton-photos-redownload-targets.js, pre-loaded with exactly the missing items - drag that onto the console and run window.__redownloadTargets() to fetch just those, then re-run the validator to confirm.

Workflow

1. Drag proton-photos-export.js onto the DevTools console (Proton Photos tab).
2. window.__startProtonExport()
   - let it run; it logs progress and resumes automatically if interrupted
   - window.__stopExport = true to pause after the current batch
   - window.__protonExportState() to check progress
3. Once it reports "Reached the end of the library. Done.":
     Drag proton-photos-inventory.js onto the console.
     window.__scanProtonLibrary()
   -> downloads proton-photos-manifest.json
4. python3 validate-proton-export.py
   -> PASS, or writes proton-photos-redownload-targets.js if anything's short
5. If it wrote a fix script: drag it onto the console, window.__redownloadTargets(),
   then go back to step 4.

Why drag-and-drop delivery, not paste

These scripts are long enough that some browsers' console input silently truncates or corrupts a pasted script. Proton's Content-Security-Policy also blocks eval()-based delivery (so bookmarklets don't work here). Dragging a .js file onto the DevTools console panel is treated as a file load rather than a paste or an eval, and sidesteps both problems.

Caveats

• Filename-based verification has a real limit. Zip contents carry no Proton-internal id, so the validator can't always tell which specific photo is missing when several items share a generic auto-generated name (proxy-image.png, VIDEO0004.mp4, etc.) - it flags the shortfall and, when generating the fix script, includes every item sharing that name rather than guessing. Re-downloading an item that already succeeded is harmless (a small extra zip, easy to spot-check and delete).
• Selectors will drift. These were written against Proton Photos' UI as of the time of writing. If something stops working, open DevTools, inspect the relevant element, and update the data-testid/class selectors near the top of the script.
• Batch size and memory. MAX_ITEMS_PER_BATCH in proton-photos-export.js defaults to 20. Raise it for fewer/faster batches if your machine has memory to spare for larger in-browser zip generation; lower it if you're on a constrained machine (this whole project exists because an earlier per-month-batch version OOM-killed the browser on a large month).
• No warranty. Read the scripts before running them - they're short enough to review, and you should always be able to see exactly what browser automation is about to do before you run it.

proton-photos-export.js

// proton-photos-export.js
//
// Bulk-export tool for Proton Photos, run from the browser DevTools console. Proton Drive/Photos
// has no official bulk-export API or "download everything" button - this automates the web UI
// itself: select a batch of photos, click download, wait for the zip, repeat, tracking progress
// so it can resume if interrupted.
//
// Design choices that aren't obvious from the code alone:
//
// - STATE IS RESUMABLE AND CONSERVATIVE. Progress is tracked in localStorage as a set of
//   "confirmed downloaded" uids. An item only enters that set once this script has personally
//   watched its zip download complete - never on a timeout, an exception, or a best-effort
//   guess. Any failure mode (selection not registering, a stuck transfer, a timeout) stops the
//   whole run rather than marking-and-continuing, so re-running window.__startProtonExport()
//   always retries exactly what wasn't confirmed - nothing is silently written off as "probably
//   fine."
//
// - FIXED BATCH SIZE, NOT PER-MONTH. An earlier version grouped by calendar month, but a month
//   can be 3 photos or 3,000 - the latter can produce a zip large enough to strain a normal
//   machine's memory (this is what motivated the rewrite - an overnight run got OOM-killed).
//   A fixed item count per batch (MAX_ITEMS_PER_BATCH) keeps each zip's size bounded and
//   predictable.
//
// - PREFETCHING THE NEXT BATCH WHILE THE CURRENT ONE DOWNLOADS. Proton's timeline is virtualized
//   and its thumbnails are decrypted client-side after they render, so finding the next batch of
//   candidates can involve real scrolling + waiting. That scan never clicks anything, so it's run
//   concurrently with waiting for the current download to finish (see the main loop) instead of
//   serially after it - this alone cut multi-second idle gaps between batches down to near zero
//   in practice.
//
// - RE-RESOLVING PREFETCHED CARDS AGAINST THE LIVE DOM. Because that prefetch can sit around for
//   the length of a download, Proton's virtualization can unmount/replace a card that was
//   captured earlier. Clicking a stale/detached checkbox silently no-ops, which - left unchecked
//   - can produce a short zip that still passes the item-count sanity check (both sides of that
//   check would just agree on the same wrong number). Every prefetched batch is re-resolved
//   against the current DOM right before it's touched; any mismatch discards the whole prefetch
//   and re-collects fresh rather than risking a bad click.
//
// - FILENAMES ARE LOGGED PER BATCH so what actually landed in a zip is independently checkable
//   against the console log, rather than trusting an opaque success count.
//
// Caveats:
// - This is unofficial, community-written browser automation - not affiliated with or endorsed
//   by Proton. It works by driving Proton Photos' current web UI via CSS selectors
//   (`data-testid` attributes); if Proton changes that UI, this WILL break, possibly silently.
//   Spot-check a handful of the zips it produces before trusting a large run, and see the
//   companion validate-proton-export.py / proton-photos-inventory.js for a way to independently
//   verify a completed run against your actual library.
// - Must be delivered by DRAGGING this file onto the DevTools console panel, not pasted. Long
//   scripts get silently truncated/corrupted by paste in some browsers' console input, and
//   Proton's Content-Security-Policy blocks eval()-based delivery methods (e.g. bookmarklets).
//   Dragging a .js file onto the console is treated as a file load, not a paste, and sidesteps
//   both problems.
//
// Usage:
//   1. Open Proton Photos in your browser, open DevTools, go to the Console tab.
//   2. Drag this file onto the console panel.
//   3. window.__startProtonExport() - starts from the top of the library (or wherever it left
//      off, if resuming).
//   4. window.__stopExport = true - stop after the current batch finishes.
//   5. window.__protonExportState() - see the current confirmed-downloaded count.
//   6. window.__protonExportReset() - wipe all progress, start over from the top.

(() => {
  const STATE_KEY = `protonExportStateV2`;
  const CARD_SELECTOR = `[data-testid="photos-card"]`;
  const CARD_CHECKBOX_SELECTOR = `[data-testid="photos-card-checkbox"]`;
  const THUMBNAIL_SELECTOR = `[data-testid="photos-card-thumbnail"]`;
  const SELECTED_COUNT_SELECTOR = `[data-testid="photos-selected-count"] span`;
  const DOWNLOAD_SELECTOR = `[data-testid="toolbar-download-selection"]`;
  const CLEAR_SELECTOR = `[data-testid="toolbar-clear-selection"]`;
  const PROGRESS_SELECTOR = `progress.tm-progress`;
  const CANCEL_SELECTOR = `[data-testid="drive-transfers-manager:header-controls-cancel"]`;
  const CLOSE_SELECTOR = `[data-testid="drive-transfers-manager:close"]`;

  const MAX_ITEMS_PER_BATCH = 20;
  const WAIT_MS = 400;
  const CLICK_GAP_MS = 40;
  const POLL_MS = 250;
  const MAX_SCROLL_STALLS = 6;
  const RENDER_RETRY_ATTEMPTS = 5; // per ensureMoreRendered() call - see its comment
  const MAX_PENDING_WAIT_CYCLES = 120; // 120 x POLL_MS = 30s to let the CURRENT backlog of still-decrypting cards clear before giving up on it and scrolling for more
  const PER_BATCH_TIMEOUT_MS = 20 * 60 * 1000;
  const IDLE_WAIT_TIMEOUT_MS = 20 * 60 * 1000;
  const GRACE_MS = 6000;

  const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
  const log = (msg) => console.log(`[proton-export] ${msg}`);

  function loadState() {
    try {
      const raw = localStorage.getItem(STATE_KEY);
      if (!raw) return { downloadedUids: [] };
      const parsed = JSON.parse(raw);
      return {
        downloadedUids: Array.isArray(parsed.downloadedUids) ? parsed.downloadedUids : [],
      };
    } catch (e) {
      log(`Warning: couldn't parse saved state (${e}) - starting empty.`);
      return { downloadedUids: [] };
    }
  }

  function saveState(downloadedUids) {
    try {
      localStorage.setItem(STATE_KEY, JSON.stringify({
        downloadedUids: Array.from(downloadedUids),
      }));
    } catch (e) {
      log(`ERROR: failed to persist state (${e}) - this batch's progress is held in memory for` +
          ` the rest of THIS run but was NOT saved to localStorage. If this keeps happening, the` +
          ` origin's localStorage quota is likely full.`);
    }
  }

  function getCardUid(card) {
    const img = card.querySelector(THUMBNAIL_SELECTOR);
    return img?.dataset?.nodeUid || null;
  }

  function getCardFilename(card) {
    const img = card.querySelector(THUMBNAIL_SELECTOR);
    const alt = img?.getAttribute(`alt`) || ``;
    const parts = alt.split(` - `);
    return parts.length ? parts[parts.length - 1] : `(unknown filename)`;
  }

  const lastCard = () => {
    const c = document.querySelectorAll(CARD_SELECTOR);
    return c.length ? c[c.length - 1] : null;
  };

  // Scrolls the last-rendered card into view and waits for the virtualized list to mount more.
  // Retries with growing delays (up to RENDER_RETRY_ATTEMPTS) before giving up on THIS call, since
  // Proton's app can pause well over a second fetching/decrypting the next range - a single short
  // wait was seen to report a false "no progress" mid-scan. The caller's own MAX_SCROLL_STALLS is
  // still what decides "reached the true end" - this only makes each individual attempt patient.
  async function ensureMoreRendered() {
    const before = lastCard();
    if (!before) return false;
    before.scrollIntoView({ block: `end`, behavior: `instant` });
    for (let attempt = 1; attempt <= RENDER_RETRY_ATTEMPTS; attempt++) {
      await sleep(WAIT_MS * attempt);
      const after = lastCard();
      if (after && after !== before) return true;
    }
    return false;
  }

  function isTransferActive() {
    return !!document.querySelector(CANCEL_SELECTOR);
  }

  async function waitUntilIdle(timeoutMs) {
    const start = Date.now();
    while (Date.now() - start < timeoutMs) {
      if (!isTransferActive()) return true;
      await sleep(POLL_MS);
    }
    return false;
  }

  async function waitForDownloadComplete(timeoutMs) {
    const start = Date.now();
    let sawRunning = false;
    while (Date.now() - start < timeoutMs) {
      if (window.__stopExport) return false;
      const progress = document.querySelector(PROGRESS_SELECTOR);
      const cancelBtn = document.querySelector(CANCEL_SELECTOR);
      const closeBtn = document.querySelector(CLOSE_SELECTOR);

      if (cancelBtn || (progress && progress.classList.contains(`progress-bar--running`))) {
        sawRunning = true;
      }

      const successByBar = !!(progress && progress.classList.contains(`progress-bar--success`) && progress.value >= progress.max);
      const successByControls = !!(closeBtn && !cancelBtn);
      const success = successByBar || successByControls;
      const elapsed = Date.now() - start;

      if (success && (sawRunning || elapsed > GRACE_MS)) return true;

      await sleep(POLL_MS);
    }
    return false;
  }

  function readSelectedCount() {
    const el = document.querySelector(SELECTED_COUNT_SELECTOR);
    if (!el) return 0;
    const m = el.textContent.match(/\d+/);
    return m ? parseInt(m[0], 10) : 0;
  }

  // Returns a per-loop async decision function for "is it safe to scroll for more content yet?".
  // Scrolling while cards already on screen are still mid-decrypt (no uid) only adds MORE
  // unresolved cards on top of the existing backlog. Wait for the CURRENT backlog to fully clear
  // (up to MAX_PENDING_WAIT_CYCLES) before scrolling for more, rather than advancing on a fixed
  // short timeout regardless of whether the backlog is shrinking or growing.
  function makeBacklogWaiter() {
    let cycles = 0;
    return async (pendingUidCount) => {
      if (pendingUidCount === 0) {
        cycles = 0;
        return `ready`;
      }
      if (cycles < MAX_PENDING_WAIT_CYCLES) {
        cycles++;
        await sleep(POLL_MS);
        return `wait`;
      }
      cycles = 0;
      return `giveup`;
    };
  }

  // Scans currently-rendered cards (scrolling to render more as needed) for the next `batchSize`
  // items whose uid isn't already in `excludeUids`. Never scrolls for more while any rendered
  // card is still mid-decrypt (see makeBacklogWaiter above). Returns fewer than batchSize only
  // once the whole library has genuinely been walked to its end.
  // May run concurrently with a download (see the main loop) - it only reads the DOM and scrolls,
  // it never clicks anything, so it's safe to overlap. Bails early (returning whatever's been
  // found so far) if __stopExport flips mid-scan, so a stop during the overlap window doesn't have
  // to wait out a full backlog/stall cycle.
  async function collectNextBatch(excludeUids, batchSize) {
    let stalls = 0;
    const waitForBacklog = makeBacklogWaiter();
    while (true) {
      if (window.__stopExport) return [];
      const allCards = Array.from(document.querySelectorAll(CARD_SELECTOR));
      const candidates = [];
      let pendingUidCount = 0;
      for (const card of allCards) {
        const uid = getCardUid(card);
        if (!uid) { pendingUidCount++; continue; }
        if (!excludeUids.has(uid)) {
          candidates.push({ card, uid });
          if (candidates.length >= batchSize) break;
        }
      }
      if (candidates.length >= batchSize) return candidates;

      const decision = await waitForBacklog(pendingUidCount);
      if (decision === `wait`) continue;
      if (decision === `giveup`) {
        log(`Warning: ${pendingUidCount} rendered card(s) still have no uid after ${(MAX_PENDING_WAIT_CYCLES * POLL_MS / 1000).toFixed(0)}s - skipping them this pass, they'll be re-checked next pass.`);
      }

      const progressed = await ensureMoreRendered();
      if (!progressed) {
        stalls++;
        if (stalls >= MAX_SCROLL_STALLS) return candidates;
      } else {
        stalls = 0;
      }
    }
  }

  function currentCardsByUid() {
    const map = new Map();
    for (const card of document.querySelectorAll(CARD_SELECTOR)) {
      const uid = getCardUid(card);
      if (uid) map.set(uid, card);
    }
    return map;
  }

  window.__protonExportState = () => {
    const s = loadState();
    log(`downloaded=${s.downloadedUids.length}`);
    return s;
  };

  window.__protonExportReset = () => {
    localStorage.removeItem(STATE_KEY);
    log(`State cleared. Next window.__startProtonExport() call will start from item 0.`);
  };

  window.__stopExport = false;

  window.__startProtonExport = async function () {
    window.__stopExport = false;
    const s = loadState();
    const downloadedUids = new Set(s.downloadedUids);

    log(`Starting. ${downloadedUids.size} item(s) already confirmed downloaded (batch size ${MAX_ITEMS_PER_BATCH}).`);

    document.querySelector(CLEAR_SELECTOR)?.click();
    await sleep(WAIT_MS);
    const idleAtStart = await waitUntilIdle(IDLE_WAIT_TIMEOUT_MS);
    if (!idleAtStart) {
      log(`A transfer looked active at startup and never cleared - stopping before touching anything. Check the Proton tab manually.`);
      return;
    }
    document.querySelector(CLOSE_SELECTOR)?.click();
    await sleep(WAIT_MS);

    let batch = await collectNextBatch(downloadedUids, MAX_ITEMS_PER_BATCH);
    let gapTimings = null; // set at the end of a confirmed batch, read+logged right before the next download click - see its use below

    while (!window.__stopExport) {
      if (!batch.length) {
        log(`No more undone items found. Reached the end of the library. Done. Total downloaded: ${downloadedUids.size}.`);
        break;
      }

      // Prefetched batches were collected from a snapshot of the DOM taken WHILE the previous
      // download was still in flight. Proton keeps re-rendering during that window (thumbnails
      // finishing decrypt, the transfers-manager panel opening/closing), and virtualization can
      // unmount/replace a card that was captured earlier. Re-resolve every uid against the CURRENT
      // DOM right before touching it; if any is missing, throw the whole prefetch away and
      // re-collect fresh rather than clicking a stale/detached checkbox (which silently no-ops and
      // would produce a short zip that still passes the count check below, since both sides of
      // that check would agree on the same wrong number).
      const staleCheckStart = Date.now();
      const liveCards = currentCardsByUid();
      const resolved = [];
      let stale = false;
      for (const item of batch) {
        const liveCard = liveCards.get(item.uid);
        if (!liveCard) { stale = true; break; }
        resolved.push({ card: liveCard, uid: item.uid });
      }
      if (stale) {
        log(`Prefetched batch had a stale card reference (DOM changed during the previous download) - discarding and re-collecting fresh.`);
        batch = await collectNextBatch(downloadedUids, MAX_ITEMS_PER_BATCH);
        gapTimings = null;
        continue;
      }
      batch = resolved;
      const staleCheckMs = Date.now() - staleCheckStart;

      const batchLabel = `batch of ${batch.length}`;
      const idle = await waitUntilIdle(IDLE_WAIT_TIMEOUT_MS);
      if (!idle) {
        log(`${batchLabel}: a previous transfer never went idle - STOPPING. Nothing in this batch was marked downloaded. Re-run window.__startProtonExport() after checking the Proton tab manually.`);
        break;
      }

      const selectStart = Date.now();
      const confirmed = [];
      for (const item of batch) {
        const cb = item.card.querySelector(CARD_CHECKBOX_SELECTOR);
        if (!cb) continue;
        if (!cb.checked) cb.click();
        await sleep(CLICK_GAP_MS);
        if (cb.checked) confirmed.push(item);
      }
      const selectMs = Date.now() - selectStart;

      if (!confirmed.length) {
        log(`${batchLabel}: selection didn't register (0 confirmed checked) - STOPPING. Nothing marked downloaded.`);
        break;
      }
      const uiCount = readSelectedCount();
      if (uiCount !== confirmed.length) {
        log(`${batchLabel}: UI reports ${uiCount} selected, checkbox state confirms ${confirmed.length} - STOPPING rather than trusting a mismatch. Nothing marked downloaded.`);
        break;
      }

      const filenames = confirmed.map((item) => getCardFilename(item.card));
      log(`Selected ${confirmed.length} item(s) (${batchLabel}): ${JSON.stringify(filenames)}`);

      const downloadBtn = document.querySelector(DOWNLOAD_SELECTOR);
      if (!downloadBtn) {
        log(`${batchLabel}: download button missing right after selecting - STOPPING. Nothing marked downloaded.`);
        break;
      }

      if (gapTimings) {
        const totalGapMs = gapTimings.prefetchOverheadMs + gapTimings.clearCloseMs + staleCheckMs + selectMs;
        log(`Gap since previous download completed: prefetch-still-running=${gapTimings.prefetchOverheadMs}ms,` +
            ` clear/close=${gapTimings.clearCloseMs}ms, stale-check=${staleCheckMs}ms, selection=${selectMs}ms,` +
            ` total=${totalGapMs}ms.`);
      }

      downloadBtn.click();

      // Overlap: scan ahead for the NEXT batch (scrolling + waiting out any decrypt backlog)
      // while THIS batch's zip is generating/downloading, instead of after. downloadedUids only
      // gets this batch's uids once the download below is CONFIRMED complete, so pass them as an
      // extra exclude set here to keep the prefetch from re-selecting still-in-flight items.
      const inFlightExclude = new Set(downloadedUids);
      for (const item of confirmed) inFlightExclude.add(item.uid);

      let downloadDoneAt = null;
      let prefetchDoneAt = null;
      const [ok, prefetched] = await Promise.all([
        waitForDownloadComplete(PER_BATCH_TIMEOUT_MS).then((r) => { downloadDoneAt = Date.now(); return r; }),
        collectNextBatch(inFlightExclude, MAX_ITEMS_PER_BATCH).then((r) => { prefetchDoneAt = Date.now(); return r; }),
      ]);

      if (!ok) {
        log(`${batchLabel}: did not confirm completion (timed out, or you stopped it) - STOPPING. Nothing in this batch was marked downloaded, it will be retried from the top of this batch next run.`);
        break;
      }
      log(`${batchLabel}: download complete. Verify the zip that just landed has ${confirmed.length} file(s) matching the names above.`);
      for (const item of confirmed) downloadedUids.add(item.uid);
      saveState(downloadedUids);

      // prefetchOverheadMs is 0 if the prefetch scan finished before (or same time as) the
      // download - i.e. it was fully hidden behind the download and cost nothing. It's positive
      // only when the scroll/decrypt-backlog wait for the NEXT batch outlasted the download of
      // THIS one - that's genuine leftover idle time even with the overlap, worth watching for.
      const prefetchOverheadMs = Math.max(0, (prefetchDoneAt ?? Date.now()) - (downloadDoneAt ?? Date.now()));

      const clearCloseStart = Date.now();
      document.querySelector(CLEAR_SELECTOR)?.click();
      await sleep(WAIT_MS);
      document.querySelector(CLOSE_SELECTOR)?.click();
      await sleep(WAIT_MS);
      const clearCloseMs = Date.now() - clearCloseStart;

      gapTimings = { prefetchOverheadMs, clearCloseMs };
      batch = prefetched;
    }

    log(`Stopped. Confirmed downloaded so far: ${downloadedUids.size}.`);
    log(`Re-running window.__startProtonExport() resumes from here - only confirmed downloads count toward that number.`);
  };

  const initial = loadState();
  log(`Loaded. Confirmed downloaded so far: ${initial.downloadedUids.length}.`);
  log(`Call window.__startProtonExport() to begin/resume.`);
})();

proton-photos-inventory.js

// proton-photos-inventory.js — read-only full-library scan for Proton Photos.
//
// Independent ground truth for validating proton-photos-export.js. It does NOT read the export
// script's state, does NOT touch selection, and does NOT download anything - it only scrolls
// the timeline from the top and records {uid, filename} for every card it can see. That makes
// it safe to run any time (mid-export, after "Done", in a second tab) and, critically, immune
// to whatever bug the export script itself might have - it can't inherit a bookkeeping mistake
// it never looks at.
//
// Scrolling/render-patience logic is copied from proton-photos-export.js's collectNextBatch(),
// since that patience tuning (retry with growing waits, don't scroll while cards are still
// mid-decrypt) was already tuned and verified there against a large real library.
//
// Usage:
//   1. Drag this file onto the DevTools console (not copy-paste).
//   2. window.__scanProtonLibrary() - scans from the top, downloads proton-photos-manifest.json
//      when done. Takes a while for a large library (it's pacing itself the same as the export
//      script does) - let it run, don't interact with the Photos tab while it's scrolling.
//   3. Run it a second time and compare counts - if the count didn't change, the scan converged.

(() => {
  const CARD_SELECTOR = `[data-testid="photos-card"]`;
  const THUMBNAIL_SELECTOR = `[data-testid="photos-card-thumbnail"]`;

  const WAIT_MS = 400;
  const POLL_MS = 250;
  const MAX_SCROLL_STALLS = 6;
  const RENDER_RETRY_ATTEMPTS = 5;
  const MAX_PENDING_WAIT_CYCLES = 120; // 120 x POLL_MS = 30s

  const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
  const log = (msg) => console.log(`[proton-inventory] ${msg}`);

  function getCardUid(card) {
    const img = card.querySelector(THUMBNAIL_SELECTOR);
    return img?.dataset?.nodeUid || null;
  }

  function getCardFilename(card) {
    const img = card.querySelector(THUMBNAIL_SELECTOR);
    const alt = img?.getAttribute(`alt`) || ``;
    const parts = alt.split(` - `);
    return parts.length ? parts[parts.length - 1] : null;
  }

  const lastCard = () => {
    const c = document.querySelectorAll(CARD_SELECTOR);
    return c.length ? c[c.length - 1] : null;
  };

  async function ensureMoreRendered() {
    const before = lastCard();
    if (!before) return false;
    before.scrollIntoView({ block: `end`, behavior: `instant` });
    for (let attempt = 1; attempt <= RENDER_RETRY_ATTEMPTS; attempt++) {
      await sleep(WAIT_MS * attempt);
      const after = lastCard();
      if (after && after !== before) return true;
    }
    return false;
  }

  function makeBacklogWaiter() {
    let cycles = 0;
    return async (pendingUidCount) => {
      if (pendingUidCount === 0) { cycles = 0; return `ready`; }
      if (cycles < MAX_PENDING_WAIT_CYCLES) { cycles++; await sleep(POLL_MS); return `wait`; }
      cycles = 0;
      return `giveup`;
    };
  }

  window.__scanProtonLibrary = async function () {
    const seen = new Map(); // uid -> filename
    let unresolvedTotal = 0;
    let stalls = 0;
    const waitForBacklog = makeBacklogWaiter();

    log(`Starting full-library scan. Read-only - this never clicks or downloads anything.`);

    while (true) {
      const cards = document.querySelectorAll(CARD_SELECTOR);
      let pendingUidCount = 0;
      for (const card of cards) {
        const uid = getCardUid(card);
        if (!uid) { pendingUidCount++; continue; }
        if (!seen.has(uid)) seen.set(uid, getCardFilename(card));
      }

      const decision = await waitForBacklog(pendingUidCount);
      if (decision === `wait`) continue;
      if (decision === `giveup`) {
        log(`Warning: ${pendingUidCount} rendered card(s) still have no uid after ${(MAX_PENDING_WAIT_CYCLES * POLL_MS / 1000).toFixed(0)}s - moving on without them this pass.`);
        unresolvedTotal += pendingUidCount;
      }

      const progressed = await ensureMoreRendered();
      if (!progressed) {
        stalls++;
        if (stalls >= MAX_SCROLL_STALLS) break;
      } else {
        stalls = 0;
      }
    }

    const items = Array.from(seen.entries()).map(([uid, filename]) => ({ uid, filename }));
    log(`Scan complete. ${items.length} item(s) resolved, ${unresolvedTotal} card-encounter(s) never resolved a uid along the way.`);

    const manifest = {
      scannedAt: new Date().toISOString(),
      count: items.length,
      unresolvedEncounters: unresolvedTotal,
      items,
    };
    const blob = new Blob([JSON.stringify(manifest, null, 2)], { type: `application/json` });
    const url = URL.createObjectURL(blob);
    const a = document.createElement(`a`);
    a.href = url;
    a.download = `proton-photos-manifest.json`;
    document.body.appendChild(a);
    a.click();
    a.remove();
    URL.revokeObjectURL(url);

    log(`Downloaded proton-photos-manifest.json (${items.length} items). Run validate-proton-export.py against it.`);
    return manifest;
  };

  log(`Loaded. Call window.__scanProtonLibrary() to scan the whole library and download a manifest.`);
})();

validate-proton-export.py

#!/usr/bin/env python3
"""Cross-check proton-photos-manifest.json (from proton-photos-inventory.js, the independent
ground-truth browser scan) against the zips actually sitting in the export directory produced by
proton-photos-export.js. Matching is by FILENAME, not uid, because zip contents carry no
Proton-internal id - so it can't distinguish two different photos that happen to share a generic
camera/app-generated name (e.g. "proxy-image.png", "VIDEO0004.mp4"). Counts are compared
per-filename to at least catch a shortfall even when names collide.

When anything is missing, this also writes a ready-to-run proton-photos-redownload-targets.js
next to the export directory - drag it onto the console and run window.__redownloadTargets() to
fetch exactly the missing items, then re-run this validator to confirm.

Usage:
    python3 validate-proton-export.py [--manifest PATH] [--export-dir PATH] [--batch-size N]
"""
import argparse
import json
import sys
import zipfile
from collections import Counter
from pathlib import Path


def load_manifest(path: Path) -> list[dict]:
    with path.open() as f:
        data = json.load(f)
    items = data.get("items", [])
    if not items:
        sys.exit(f"Manifest at {path} has no items - did the scan actually run?")
    return items


FIX_SCRIPT_TEMPLATE = '''// proton-photos-redownload-targets.js — auto-generated by validate-proton-export.py on {generated_at}.
// Re-downloads a specific, known set of {count} uid(s) that came up short in the last validation
// pass (an unreadable zip, or a per-filename count shortfall). This is NOT a full re-scan - it
// scrolls the timeline looking only for these targets, and stops once every one is found and
// downloaded (or the true end of the library is reached, in which case whatever's left over is
// logged so you can look it up manually).
//
// Note on scope: when a shortfall couldn't be pinned to one specific uid (multiple items share a
// generic name like "proxy-image.png"), ALL uids sharing that name were included here rather than
// guessing which one actually failed. Re-downloading an item that already succeeded is harmless -
// it lands in a small extra zip, safe to delete after you've checked it against the names logged
// below.
//
// Reuses the same scrolling/backlog-wait/checkbox-click-and-verify/download-completion logic as
// proton-photos-export.js.
//
// Usage:
//   1. Drag this file onto the DevTools console (not copy-paste).
//   2. window.__redownloadTargets() - finds and downloads these {count} item(s) in batches.
//   3. Re-run validate-proton-export.py afterward to confirm they're now accounted for.

(() => {{
  const TARGET_UIDS = new Set({target_uids_json});

  const CARD_SELECTOR = `[data-testid="photos-card"]`;
  const CARD_CHECKBOX_SELECTOR = `[data-testid="photos-card-checkbox"]`;
  const THUMBNAIL_SELECTOR = `[data-testid="photos-card-thumbnail"]`;
  const SELECTED_COUNT_SELECTOR = `[data-testid="photos-selected-count"] span`;
  const DOWNLOAD_SELECTOR = `[data-testid="toolbar-download-selection"]`;
  const CLEAR_SELECTOR = `[data-testid="toolbar-clear-selection"]`;
  const PROGRESS_SELECTOR = `progress.tm-progress`;
  const CANCEL_SELECTOR = `[data-testid="drive-transfers-manager:header-controls-cancel"]`;
  const CLOSE_SELECTOR = `[data-testid="drive-transfers-manager:close"]`;

  const MAX_ITEMS_PER_BATCH = {batch_size};
  const WAIT_MS = 400;
  const CLICK_GAP_MS = 40;
  const POLL_MS = 250;
  const MAX_SCROLL_STALLS = 6;
  const RENDER_RETRY_ATTEMPTS = 5;
  const MAX_PENDING_WAIT_CYCLES = 120;
  const PER_BATCH_TIMEOUT_MS = 20 * 60 * 1000;
  const IDLE_WAIT_TIMEOUT_MS = 20 * 60 * 1000;
  const GRACE_MS = 6000;

  const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
  const log = (msg) => console.log(`[proton-redownload-targets] ${{msg}}`);

  function getCardUid(card) {{
    const img = card.querySelector(THUMBNAIL_SELECTOR);
    return img?.dataset?.nodeUid || null;
  }}

  function getCardFilename(card) {{
    const img = card.querySelector(THUMBNAIL_SELECTOR);
    const alt = img?.getAttribute(`alt`) || ``;
    const parts = alt.split(` - `);
    return parts.length ? parts[parts.length - 1] : `(unknown filename)`;
  }}

  const lastCard = () => {{
    const c = document.querySelectorAll(CARD_SELECTOR);
    return c.length ? c[c.length - 1] : null;
  }};

  async function ensureMoreRendered() {{
    const before = lastCard();
    if (!before) return false;
    before.scrollIntoView({{ block: `end`, behavior: `instant` }});
    for (let attempt = 1; attempt <= RENDER_RETRY_ATTEMPTS; attempt++) {{
      await sleep(WAIT_MS * attempt);
      const after = lastCard();
      if (after && after !== before) return true;
    }}
    return false;
  }}

  function isTransferActive() {{
    return !!document.querySelector(CANCEL_SELECTOR);
  }}

  async function waitUntilIdle(timeoutMs) {{
    const start = Date.now();
    while (Date.now() - start < timeoutMs) {{
      if (!isTransferActive()) return true;
      await sleep(POLL_MS);
    }}
    return false;
  }}

  async function waitForDownloadComplete(timeoutMs) {{
    const start = Date.now();
    let sawRunning = false;
    while (Date.now() - start < timeoutMs) {{
      if (window.__stopExport) return false;
      const progress = document.querySelector(PROGRESS_SELECTOR);
      const cancelBtn = document.querySelector(CANCEL_SELECTOR);
      const closeBtn = document.querySelector(CLOSE_SELECTOR);

      if (cancelBtn || (progress && progress.classList.contains(`progress-bar--running`))) {{
        sawRunning = true;
      }}

      const successByBar = !!(progress && progress.classList.contains(`progress-bar--success`) && progress.value >= progress.max);
      const successByControls = !!(closeBtn && !cancelBtn);
      const success = successByBar || successByControls;
      const elapsed = Date.now() - start;

      if (success && (sawRunning || elapsed > GRACE_MS)) return true;

      await sleep(POLL_MS);
    }}
    return false;
  }}

  function readSelectedCount() {{
    const el = document.querySelector(SELECTED_COUNT_SELECTOR);
    if (!el) return 0;
    const m = el.textContent.match(/\\d+/);
    return m ? parseInt(m[0], 10) : 0;
  }}

  function makeBacklogWaiter() {{
    let cycles = 0;
    return async (pendingUidCount) => {{
      if (pendingUidCount === 0) {{ cycles = 0; return `ready`; }}
      if (cycles < MAX_PENDING_WAIT_CYCLES) {{ cycles++; await sleep(POLL_MS); return `wait`; }}
      cycles = 0;
      return `giveup`;
    }};
  }}

  // Same shape as proton-photos-export.js's collectNextBatch(), but collects cards whose uid IS
  // a member of remainingTargets, instead of cards whose uid is NOT in a downloaded set.
  async function collectNextTargetBatch(remainingTargets, batchSize) {{
    let stalls = 0;
    const waitForBacklog = makeBacklogWaiter();
    while (true) {{
      if (window.__stopExport || remainingTargets.size === 0) return [];
      const allCards = Array.from(document.querySelectorAll(CARD_SELECTOR));
      const candidates = [];
      let pendingUidCount = 0;
      for (const card of allCards) {{
        const uid = getCardUid(card);
        if (!uid) {{ pendingUidCount++; continue; }}
        if (remainingTargets.has(uid)) {{
          candidates.push({{ card, uid }});
          if (candidates.length >= batchSize) break;
        }}
      }}
      if (candidates.length >= batchSize) return candidates;

      const decision = await waitForBacklog(pendingUidCount);
      if (decision === `wait`) continue;
      if (decision === `giveup`) {{
        log(`Warning: ${{pendingUidCount}} rendered card(s) still have no uid after ${{(MAX_PENDING_WAIT_CYCLES * POLL_MS / 1000).toFixed(0)}}s - skipping them this pass.`);
      }}

      const progressed = await ensureMoreRendered();
      if (!progressed) {{
        stalls++;
        if (stalls >= MAX_SCROLL_STALLS) return candidates;
      }} else {{
        stalls = 0;
      }}
    }}
  }}

  window.__stopExport = false;

  window.__redownloadTargets = async function () {{
    window.__stopExport = false;
    const remaining = new Set(TARGET_UIDS);
    let downloadedCount = 0;

    log(`Starting. Looking for ${{remaining.size}} specific item(s).`);

    document.querySelector(CLEAR_SELECTOR)?.click();
    await sleep(WAIT_MS);
    const idleAtStart = await waitUntilIdle(IDLE_WAIT_TIMEOUT_MS);
    if (!idleAtStart) {{
      log(`A transfer looked active at startup and never cleared - stopping. Check the Proton tab manually.`);
      return;
    }}
    document.querySelector(CLOSE_SELECTOR)?.click();
    await sleep(WAIT_MS);

    let batch = await collectNextTargetBatch(remaining, MAX_ITEMS_PER_BATCH);

    while (!window.__stopExport && remaining.size > 0) {{
      if (!batch.length) {{
        log(`Reached the end of the library with ${{remaining.size}} target(s) still not found: ${{JSON.stringify(Array.from(remaining))}}`);
        log(`These weren't rendered anywhere in the library on this pass - they may need a manual look (could be deleted from Proton, or just needed more scroll patience).`);
        break;
      }}

      const batchLabel = `batch of ${{batch.length}}`;
      const idle = await waitUntilIdle(IDLE_WAIT_TIMEOUT_MS);
      if (!idle) {{
        log(`${{batchLabel}}: a previous transfer never went idle - STOPPING.`);
        break;
      }}

      const confirmed = [];
      for (const item of batch) {{
        const cb = item.card.querySelector(CARD_CHECKBOX_SELECTOR);
        if (!cb) continue;
        if (!cb.checked) cb.click();
        await sleep(CLICK_GAP_MS);
        if (cb.checked) confirmed.push(item);
      }}

      if (!confirmed.length) {{
        log(`${{batchLabel}}: selection didn't register - STOPPING.`);
        break;
      }}
      const uiCount = readSelectedCount();
      if (uiCount !== confirmed.length) {{
        log(`${{batchLabel}}: UI reports ${{uiCount}} selected, checkboxes confirm ${{confirmed.length}} - STOPPING.`);
        break;
      }}

      const filenames = confirmed.map((item) => getCardFilename(item.card));
      log(`Selected ${{confirmed.length}} item(s) (${{batchLabel}}): ${{JSON.stringify(filenames)}}`);

      const downloadBtn = document.querySelector(DOWNLOAD_SELECTOR);
      if (!downloadBtn) {{
        log(`${{batchLabel}}: download button missing right after selecting - STOPPING.`);
        break;
      }}
      downloadBtn.click();

      const ok = await waitForDownloadComplete(PER_BATCH_TIMEOUT_MS);
      if (!ok) {{
        log(`${{batchLabel}}: did not confirm completion - STOPPING. Re-run window.__redownloadTargets() to retry.`);
        break;
      }}

      log(`${{batchLabel}}: download complete. Verify the zip has ${{confirmed.length}} file(s) matching the names above.`);
      for (const item of confirmed) {{ remaining.delete(item.uid); downloadedCount++; }}

      document.querySelector(CLEAR_SELECTOR)?.click();
      await sleep(WAIT_MS);
      document.querySelector(CLOSE_SELECTOR)?.click();
      await sleep(WAIT_MS);

      batch = await collectNextTargetBatch(remaining, MAX_ITEMS_PER_BATCH);
    }}

    if (remaining.size === 0) {{
      log(`Done. All ${{downloadedCount}} target(s) downloaded.`);
    }} else {{
      log(`Stopped. ${{downloadedCount}} downloaded, ${{remaining.size}} still remaining: ${{JSON.stringify(Array.from(remaining))}}`);
    }}
  }};

  log(`Loaded. ${{TARGET_UIDS.size}} target uid(s) to find. Call window.__redownloadTargets() to begin.`);
}})();
'''


def write_fix_script(target_items: list[dict], output_path: Path, batch_size: int) -> None:
    from datetime import datetime, timezone

    target_uids = sorted({item["uid"] for item in target_items})
    content = FIX_SCRIPT_TEMPLATE.format(
        generated_at=datetime.now(timezone.utc).isoformat(),
        count=len(target_uids),
        target_uids_json=json.dumps(target_uids),
        batch_size=batch_size,
    )
    output_path.write_text(content)


def scan_zips(export_dir: Path, batch_size: int) -> tuple[Counter, list[tuple[Path, int]], list[tuple[Path, str]]]:
    zip_filenames: Counter = Counter()
    non_batch_size = []
    unreadable = []

    for zip_path in sorted(export_dir.glob("*.zip")):
        try:
            with zipfile.ZipFile(zip_path) as z:
                names = z.namelist()
        except Exception as e:
            unreadable.append((zip_path, str(e)))
            continue
        if len(names) != batch_size:
            non_batch_size.append((zip_path, len(names)))
        zip_filenames.update(names)

    return zip_filenames, non_batch_size, unreadable


def main():
    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument("--manifest", type=Path, default=Path.home() / "Downloads" / "proton-photos-manifest.json")
    parser.add_argument("--export-dir", type=Path, default=Path.home() / "Downloads" / "proton-photos-export")
    parser.add_argument("--batch-size", type=int, default=20, help="MAX_ITEMS_PER_BATCH from proton-photos-export.js (default 20) - used to flag zips with an unexpected item count")
    parser.add_argument("--show-missing", type=int, default=50, help="max missing filenames to print (default 50, 0 for all)")
    args = parser.parse_args()

    if not args.manifest.exists():
        sys.exit(f"Manifest not found: {args.manifest}\nRun proton-photos-inventory.js in the browser first (window.__scanProtonLibrary()).")
    if not args.export_dir.is_dir():
        sys.exit(f"Export directory not found: {args.export_dir}")

    manifest_items = load_manifest(args.manifest)
    manifest_filenames = Counter(item["filename"] for item in manifest_items if item.get("filename"))
    unnamed_in_manifest = sum(1 for item in manifest_items if not item.get("filename"))

    zip_filenames, non_batch_size, unreadable = scan_zips(args.export_dir, args.batch_size)

    zip_total = sum(zip_filenames.values())
    manifest_total = sum(manifest_filenames.values())

    print(f"Manifest:    {len(manifest_items)} item(s) scanned from Proton ({manifest_total} with a usable filename, {unnamed_in_manifest} without)")
    print(f"Zips:        {zip_total} file(s) across {len(list(args.export_dir.glob('*.zip')))} zip(s) ({len(unreadable)} unreadable)")
    print()

    if unreadable:
        print(f"UNREADABLE ZIPS ({len(unreadable)}) - these contributed 0 files to the count above:")
        for path, err in unreadable:
            print(f"  {path.name}: {err}")
        print()

    if non_batch_size:
        print(f"ZIPS WITH != {args.batch_size} ENTRIES ({len(non_batch_size)}) - expected for at most the very last batch, suspicious otherwise:")
        for path, n in non_batch_size:
            print(f"  {path.name}: {n} entries")
        print()

    # Per-filename shortfall: manifest says N copies of this name should exist, zips have M.
    missing = []
    for name, manifest_count in manifest_filenames.items():
        zip_count = zip_filenames.get(name, 0)
        if zip_count < manifest_count:
            missing.append((name, manifest_count - zip_count, manifest_count, zip_count))

    surplus = []
    for name, zip_count in zip_filenames.items():
        manifest_count = manifest_filenames.get(name, 0)
        if zip_count > manifest_count:
            surplus.append((name, zip_count - manifest_count, manifest_count, zip_count))

    total_shortfall = sum(s for _, s, _, _ in missing)

    if missing:
        print(f"MISSING (best-effort, by filename): {total_shortfall} item(s) across {len(missing)} distinct filename(s) short.")
        limit = len(missing) if args.show_missing == 0 else args.show_missing
        for name, shortfall, mcount, zcount in missing[:limit]:
            print(f"  {name}: manifest has {mcount}, zips have {zcount} (short {shortfall})")
        if len(missing) > limit:
            print(f"  ... and {len(missing) - limit} more (use --show-missing 0 to print all)")
        print()

        missing_names = {name for name, _, _, _ in missing}
        target_items = [item for item in manifest_items if item.get("filename") in missing_names]
        fix_script_path = args.export_dir.parent / "proton-photos-redownload-targets.js"
        write_fix_script(target_items, fix_script_path, args.batch_size)
        print(f"Wrote {fix_script_path} - {len({i['uid'] for i in target_items})} target uid(s)" +
              f" (covers all {len(missing_names)} short filename(s), including collision groups in full).")
        print(f"Drag it onto the DevTools console, then run window.__redownloadTargets(). Re-run this" +
              f" validator afterward to confirm.")
        print()
    else:
        print("MISSING: none - every filename in the manifest is fully accounted for in the zips.")
        print()

    if surplus:
        print(f"SURPLUS (zips have MORE copies of a name than the manifest does - {len(surplus)} filename(s), likely benign generic-name collisions, but check if any count is large):")
        for name, extra, mcount, zcount in surplus[:20]:
            print(f"  {name}: manifest has {mcount}, zips have {zcount} (+{extra})")
        if len(surplus) > 20:
            print(f"  ... and {len(surplus) - 20} more")
        print()

    ok = not missing and not unreadable and not non_batch_size
    print("RESULT:", "PASS - nothing missing, no bad zips." if ok else "ISSUES FOUND - see above.")
    sys.exit(0 if ok else 1)


if __name__ == "__main__":
    main()