Skip to content

Instantly share code, notes, and snippets.

@majones919
Created June 11, 2026 14:44
Show Gist options
  • Select an option

  • Save majones919/b7e3dc23cd750875bdd5e556fa182f82 to your computer and use it in GitHub Desktop.

Select an option

Save majones919/b7e3dc23cd750875bdd5e556fa182f82 to your computer and use it in GitHub Desktop.
QA test plan — deployment/2026-06-11 (6 PRs + 1 direct commit)

QA test plan — deployment/2026-06-11

6 merged PRs + 1 direct commit (Ahmed reapplying #5104). Two of the six are bugs Megan reported on QA this month (the "seekers" email copy and the mobile photo crop); the other four cover fraud-signal accuracy, the unread-reminder feature itself, an ops gate fix, and the pricing-experiment collapse.

PR Ticket What it does
#5172 #5146 Reminder email now says "travelers," not "seekers"
#5158 #5150 Fraud Risk Review shows real last-login, not "Never"
#5104 #5007 New: one-a-day unread-inquiry reminder email to listers
#5173 #5155 Mobile listing photos show full image, no crop
#5167 #5162 Ops: keeps the QA smoke harness from silently dying
#5165 #5159 Pricing page collapses to one final layout (Quarterly removed)

#5172 — Email copy: "travelers" not "seekers" (#5146)

What it fixes: Megan flagged this on QA in #issues on 2026-06-09 — the new P1 unread-reminder email called the inquiring side "seekers" in the subject line and both email bodies, but RR copy everywhere else says "travelers." Gaurav confirmed the mockup's copy was never reviewed. This swaps the noun in all three places (subject + HTML body + plain-text body), and handles singular correctly ("from a traveler" when there's one inquiry). The "Rotating Room" two-word footer in the issue screenshot was only in the mockup draft — the real templates already render "RotatingRoom" correctly, so nothing else changed.

How to test: This is part of the unread-reminder feature (#5104 below), so it only renders when that email fires. Easiest path: have Gaurav/Ahmed flip the p1-unread-reminder PostHog flag onto your lister account, trigger a reminder (see #5104 steps), and confirm the email subject and body read "travelers" / "a traveler" — the word "seeker" should appear nowhere.


#5158 — Fraud Risk Review shows real last-login (#5150)

What it fixes: Megan reported on 2026-05-31 that the admin Fraud Risk Review preview always showed "Last login: never" for every user. The column that feeds it was never being populated. This adds a backfill (pulls each user's most-recent credential login from the activity log so the signal works on day one) and starts stamping the real login time going forward. It also makes sure admin "browse-as-user" impersonation doesn't pollute the signal — impersonating someone won't overwrite their real last-login.

How to test:

  1. In the admin panel, open Fraud Risk Review and preview a few users who have logged in recently → confirm they show a real date/time, not "Never."
  2. Find a user you know has never logged in → should still correctly show "Never."
  3. (Optional) Impersonate a user, then re-check their last-login in Risk Review → it should not have jumped to "just now" from your impersonation session.

#5104 — P1 lister unread-message reminders (#5007)

What it fixes: Brand-new feature. When a lister leaves an inquiry unread for 6+ hours, they get one reminder email per day summarizing all their unread inquiries, nudging them to reply. It's behind a 50/50 PostHog A/B flag (p1-unread-reminder) with suppression rules so nobody gets spammed (max one email per lister per 24h, respects a one-click "stop reminders" link, etc.). Also bundles a real bug fix (#5131): the listener that stamps login times was crashing every Backpack admin login with a 500 once this migration ran — that's now fixed and guarded by a regression test.

How to test:

  1. Admin login regression check (highest priority): log into the Backpack admin panel → it should load normally, no 500 error. This is the bug that cascaded ~30 smoke failures, so verify it first.
  2. The reminder email is flag-gated and not really self-serve — coordinate with Ahmed/Gaurav to put your lister account on the p1-unread-reminder allowlist, leave an inquiry unread >6h (or use the --dry-run command Ahmed runs), and confirm: the email arrives, the "Open your inbox" CTA works, and the "stop these reminders" link works.
  3. Confirm a lister does not get more than one such email in a 24h window.

#5173 — Mobile carousel shows full photo (#5155)

What it fixes: Megan escalated this from a first-time Annual Pro host — his listing photos were cropped to ~50% on phone browsers. Portrait phone photos in the gallery's landscape container were being center-cropped. One-line CSS fix so mobile photos now fit fully (letterboxed with a cream/navy bar instead of cropped). Same fix covers both the inline gallery and the tap-to-expand lightbox.

How to test:

  1. On your phone, open listing 46618 (the one from the report) on QA → the photo should display in full, no top/bottom crop. You may see thin letterbox bars on portrait photos — that's intended.
  2. Tap a photo to expand the lightbox → also full, no crop.
  3. Spot-check 2–3 other active listings with portrait photos → confirm nothing looks worse.
  4. Desktop is unchanged — quick glance to confirm no regression there.

#5165 — Pricing experiment collapsed to final layout (#5159)

What it fixes: The pricing-abc-pro-v3 A/B test is over. Two layouts tied on conversion (~26.5%); the winner (pro-old-layout) is now the only layout everyone sees. Quarterly billing is removed from the plan-selection / payment / upgrade flows — Annual and Standard durations remain. No new visual design; it's the layout most traffic already saw, with the experiment scaffolding stripped out. (Per the PR's top correction: the shipped variant is pro-old-layout, Quarterly removed — earlier text in the PR describing "Quarterly universal" is superseded.)

How to test:

  1. Go through the post-room → plans flow on QA (a couple of times / different sessions) → you should always see the same layout, with the Pro tier offered and no Quarterly option.
  2. Pick a plan → on the payment page confirm only Standard/Annual durations appear, billing line is correct, and you can proceed to checkout.
  3. Check the upgrade path (/account/listings → upgrade) → no Quarterly, Pro upsell shows for non-Pro users.
  4. Regression: public /pricing page is intentionally unchanged (still Standard + Premium only) — confirm it still renders normally.

#5167 — QA env-gate hardening (#5162) — Not user-testable

Why: This is pure ops infrastructure. On 2026-06-09 the QA Forge .env flag that enables the smoke-test harness silently flipped off twice, disabling smoke testing for ~2 hours each time. This change makes APP_ENV=qa a first-class gate so the harness no longer depends on that fragile flag, and fixes a cached-config "split-brain" read. There's no user-facing surface.

How to monitor: The proof is the smoke harness staying up — the next /qa-smoke qa run should not show the wholesale /testing/* 405 failures that hit on 2026-06-09. If smoke runs cleanly against QA, this did its job. No clicking required.


⚠ Direct commit by Ahmed Essam — "Reapply #5104: restore P1 lister unread reminders"

⚠ Flag: This commit landed directly on the QA branch without its own PR. It reapplies the full #5104 feature (17 files — the eligibility service, command, email templates, migrations, tests). It's not net-new unreviewed code — it's restoring already-reviewed PR #5104 content (likely the merge didn't carry the files cleanly) — but it did bypass the PR review hooks, and it includes one migration (...000002_add_reminder_scan_index_concurrent) not in #5104's original file list. Worth a 30-second confirm with Ahmed that the reapply was intentional and complete.

How to test: Covered by the #5104 steps above — the admin-login-loads check and the reminder-email behavior both exercise this code. If #5104 tests pass, the reapply is good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment