99 monitoring launches reviewed (10 deep-dived, 89 one-liners). 3 ready to graduate (blog-content-sitemap, email-pii-removal, map-z-index-ordering). 4 need human decisions (abc pricing experiment dead control arm, two mobile conversion launches below baseline, map-markers with no KPI after 62d). 25 off-track. 20 envelope-broken (no data after 62+ days).
- ABC Pro Plan Pricing — Control arm dead (n=46, deprecated 37d ago). Decision needed: end experiment and pick pro-old-layout as winner on directional signal, or reintroduce small control. Payment failure rate on pro-new-design (8.8%) needs investigation.
- Map Markers Grey Blocks — 62 days with no KPI registered. inquiry_submitted shows +3.2x improvement. Register a real metric or graduate by observation.
- Mobile Address Input Styling — iOS completion 40.4% vs Mac 57%. Collector may be misconfigured (counting all mobile, not just iPhone). Needs device-segment verification.
- Blog Migration — Working and accelerating (impressions +157% Jan-Jun). But target=100 is a placeholder. Set a real 150k/month target.
- Refund Copy Terms Match — Refund volume -34% post-launch, on track. 20 days left in soak. Set a numeric target before soak ends.
| Launch | Age | Confidence | Note |
|---|---|---|---|
| blog-content-sitemap | 86d | 0.85 | Target 40 exceeded (45 indexed). Impressions +104%, clicks +95%. Evergreen acquisition channel. |
| email-pii-removal | 82d | 0.93 | 9 tests, 82 days clean. Zero PII recurrence. Dead BroadcastInquiryEmail arg to clean up. |
| map-z-index-ordering | 34d | 0.80 | Fix stable 34d, no regression reports. Measurement pipeline broken (testDir mismatch) but underlying spec healthy. |
Hypothesis: Honest copy disclosure of refund-eligibility conditions reduces policy-confusion refund requests.
Stripe confirms declining trend: customer-requested refunds fell from 10.2/week pre-launch (Mar 1-Apr 12) to 6.6/week post-launch (May 10-Jun 7), a -34% decline. KPI 60-day MA at 46 (down from 58 at Apr 29).
Conversion impact: checkout-to-purchase CVR jumped +13.6pp post-launch (32.7% to 46.3%) but checkout_started volume dropped -24% and purchase volume fell -9.7%. Disclosure appears to filter intent-mismatched users before checkout.
No support-ticket reason data available. No target set. 20 days remain in 60-day soak.
PostHog experiment: ABC Pro Plan Pricing v4 (ID 367162) | Feature flag: pricing-abc-pro-v3 | Status: running | Age: 44 days since v5 reset.
Control was deprecated to 0% traffic on 2026-05-08 after only 46 exposures (6 days). Current allocations: control 0%, pro-old-layout 40%, pro-old-w-quarterly 40%, pro-new-design 20%.
Primary metric (Plan Page to Purchase, 7-day funnel): control 17.24% (5/29), pro-old-layout 25.16% (118/469), pro-old-w-quarterly 26.86% (137/510), pro-new-design 24.14% (56/232). All variants 40-56% above control but PostHog returns chance_to_win: null because control has not-enough-exposures.
Annual billing: pro-old-layout 37.2% vs pro-old-w-quarterly 24.0%. Adding quarterly back appears to cannibalize annual, a meaningful LTV signal.
Payment failures: pro-new-design 8.8% vs control 7.1% vs pro-old-layout 1.5%. New design has elevated failure rate.
Pro plan adoption: pro-old-layout 8.0%, pro-old-w-quarterly 3.6%, pro-new-design 4.5%. Only pro-old-layout above the 5% threshold.
Age: 68 days | Objective: Conversion | Owner: unassigned
Unclear. The hypothesis (Google Places autocomplete looks native on mobile) is a visual/UX claim with no direct measurement. The KPI (mobile listing creation completion rate) is a downstream outcome affected by many confounders.
The 14-day series shows a V-shape: 26 to 19 (Jun 2-8 trough) to 32 peak (Jun 13) to 29 today. PostHog absolute listing_created counts show the same V in both mobile and desktop, confirming this is a platform-wide demand signal, not mobile-specific.
Mobile funnel conversion (post-a-room to listing_created): Pre-launch mobile 8.34% vs Post-launch mobile 7.93% vs Post-launch desktop 12.75% (PostHog funnel, Jun 2-15). Mobile conversion is 4.82 ppts below desktop and 0.41 ppts below its own pre-launch baseline.
The metric-hypothesis mismatch is total. Mobile completion trails desktop by 38%. The V-shape is symmetric across devices. No target after 68 days.
Status: Target exceeded, ready for graduation.
| Metric | Baseline | Target | Current | Delta |
|---|---|---|---|---|
| Blog pages indexed | 18 | 40 | 45 | +150% from baseline, +12.5% above target |
| Blog impressions/wk | 12,471 | (none set) | ~25,500 (Jun 9-15 est.) | +104% |
| Blog clicks/wk | 168 | (none set) | ~328 (Jun 9-15 est.) | +95% |
The launch hit its primary target (40 indexed pages) around week 4 and has held at 42-45 since early June. Jun 5 dip to 25 and Jun 7 dip to 31 are GSC API sampling artifacts, not real deindexation.
Blog impressions doubled from ~12,500/wk baseline to ~25,500/wk. Clicks followed, up from 168/wk to ~328/wk. CTR remains flat at ~1.3%. Top queries are 'furnished finder' competitor comparisons (7 of top 10).
Duplicate URL indexation: 19 of 25 articles still have both old date-format and new slug-format URLs in GSC. Old URLs account for 25% of impressions and 31% of clicks. Google chose the new URL as canonical but hasn't fully consolidated.
Both guardrails have null current_value after 86 days (never wired to a collector). Decision matrix is null. Secondary metrics have no targets.
Recommendation: Graduate as evergreen. Blog is a permanent acquisition channel. Remaining URL consolidation and CTR optimization belong in standing SEO work.
Verdict: Working and accelerating. Target registration is broken.
Blog pages grew from ~1,400 GSC impressions/day in January to ~3,600/day in June, a 157% increase (source: GSC). Monthly totals: Jan 43k, Feb 42k, Mar 59k, Apr 55k, May 70k, Jun pace ~110k.
The registered current=137,246 is a rolling aggregate, not daily. target=100 is a placeholder. The day-to-day volatility (19k-153k) is an artifact of inconsistent GSC query windows.
Top pages by clicks (Jun 1-15): /blog/what-are-alternatives-to-furnished-finder (243 clicks, 8,956 impressions, pos 6.9), /blog/furnished-finder-alternatives-2026 (80 clicks). Blog has become a Furnished Finder comparison funnel.
Blog drives 17% of site impressions but only 4.5% of clicks. CTR 1.24% vs site-wide 4.7%. subletting-vs-subleasing: 5,055 impressions, 0.1% CTR at pos 9.7.
Status: working as intended. Recommend graduation.
PR #3267 removed the inquirer's personal email from three inquiry email templates. All now route replies through token addresses. Direct grep of templates confirmed: zero occurrences of personal email properties.
9 regression tests (Unit + Feature) assert PII absence across all templates. The pre-fix error ($replyToEmail undefined, 3 occurrences, last 2026-03-19) has not recurred in 82 days.
Guardrail gap: Email delivery rate unchanged was never measured. Postmark bounce errors in Rollbar are normal bounce handling, not a regression.
Comparable graduated launches confirm the pattern: qualitative pass/fail check, continuously green, no further numeric tracking needed.
Hypothesis: Internal links between content hubs distribute page authority, improving rankings for city and topic pages.
Verdict: Working signal exists but attribution is unmeasurable. Primary KPI was never instrumented.
The primary KPI ("Internal links per page / crawl depth") has returned null for all 81 days since launch. No crawl-depth measurement was ever collected (source: registered envelope, single data point 2026-04-17 = null).
What GSC actually shows for /sublets/ city pages:
Pre-launch 90-day window (Dec 25 2025 to Mar 25 2026, top 30 city pages):
- Total impressions: ~335,186
- Total clicks: ~8,727
- Avg position for top cities: Chicago 7.8, Boston 8.0, Atlanta 6.5, Philadelphia 5.9
Post-launch 81-day window (Mar 25 to Jun 14 2026, top 30 city pages):
- Total impressions: ~365,925 (+9% vs pre, despite covering 9 fewer days)
- Total clicks: ~8,538 (-2% vs pre)
- Avg position for top cities: Chicago 9.7, Boston 9.4, Atlanta 9.7, Philadelphia 7.4
Most city pages declined in ranking position post-launch: Chicago -1.9, Boston -1.4, New York -3.7, Houston -2.8, Atlanta -3.2. Washington DC improved slightly (+1.4). Impressions grew +9% over a shorter window. CTR compressed from ~3.3% to ~2.9%.
Sitewide position improved from ~12.4 (late Mar) to ~10.0 (early Jun), a +2.4 position gain, but cannot be attributed specifically to internal linking.
Guardrail: crawl errors at 137,246, trend declining. Page load time: null, never measured.
Launch: map-markers-grey-blocks | PR #3858 | Launched 2026-04-14 | Age: 62 days | Objective: conversion
Unknown for the stated hypothesis. The hypothesis claims map renders normal-sized colored pins with working click tooltips -- a visual rendering assertion -- but no map-specific events exist in PostHog (no map_pin_clicked, no map_tooltip_opened, no map URL pageviews tracked). Direct verification is impossible without instrumentation.
What CAN be said: the week of launch (Apr 14), search_select_listing went from 0 to 1,633 events, then stabilized at 9,391-10,772 per week over the following 8 weeks (PostHog, weekly trends, Apr 19 - Jun 7). This event did not exist prior to the week of April 12, strongly suggesting it was introduced alongside or as part of the map/search UI rework in PR #3858. inquiry_submitted rose from ~962/week (week of Apr 5) to a steady 3,000-3,263/week by May-June -- a 3.2x lift that has held for 8 consecutive weeks (PostHog).
The search volume baseline grew modestly (4,123 searches/week in mid-March to 6,884-7,556 in April-May, a ~60-80% lift), so the inquiry gain outpaces the search growth materially.
- search_select_listing did not exist before Apr 12 (PostHog event schema): 0 events for 4 weeks prior, then 1,633 on launch week, then 9,391-10,772 steady-state.
- listing_viewed collapsed from ~9,775/week (Mar 29) to near-zero by Apr 19 (PostHog trends). Likely an event rename on the same PR, never reconciled.
- inquiry_submitted up 3.2x and holding 8 weeks post-launch.
- Zero KPI instrumentation after 62 days on a conversion-objective launch.
Needs human input. The proxy signals are positive but the launch was never instrumented and the hypothesis (visual rendering) cannot be evaluated from analytics alone.
Age: 34 days (launched 2026-05-12, 3-day soak). PR: #4656. Issue: #4553 (closed).
Primary metric has zero data points. The spec is registered with source=playwright and query path in tests/playwright/journeys/, but kpi-refresh.js runs specs under tests/playwright/kpi-checks/ (testDir mismatch). Playwright silently returns 'No tests found' and the collector posts 0 or skips.
The spec itself is healthy (10/10 tests pass in 1.5s) but it is a unit-style spec testing pure JS helper functions, not a prod-check that loads /need-a-room as the hypothesis describes.
Decision matrix says graduate at day 7 if Playwright passes 7 consecutive days + zero reports. Cannot confirm (no data). However: the fix has been in production 34 days with no regression reports, the unit spec provides strong regression coverage, and issue #4553 is closed.
objective is null (should be conversion per issue labels). owner is null.
Recommendation: Graduate. Fix is stable, regression guard exists, but measurement pipeline is broken and hypothesis describes a test that does not exist in the claimed form.
Hypothesis: Address input on iPhone displays with correct font, colors, and sizing matching the form.
The collector is likely misconfigured. PostHog data (Jun 2-15) shows iOS users converted at 40.4% (89 starters, 36 completions), vs 57.0% for Mac OS X. iOS is 16.6pp below Mac. If the fix had materially improved iPhone UX, we would expect iOS conversion to be closer to desktop parity.
Registered KPI values (14-30) do not match any computable PostHog metric: daily iOS unique listing_created peaked at 5, all-OS daily peaked at 19. The values appear to reflect a rolling sum or total-count, not the stated completion rate.
14-day trend: 20, 15, 15, 14, 14, 15, 15, 18, 19, 25, 25, 30, 29, 27. V-shape tracks all-OS volume, not iPhone-specific rate. Pattern identical to sibling launch address-autocomplete-mobile-styling, suggesting overlapping populations.
No target after 66 days. No linked PR or issue.
All launched Jun 11. Too early to evaluate.
- stripe_refund_reconciliation — Stripe refund reconciliation (#5032)
- rooms-location-coord-desync-fix — Rooms.location coord sync (#5046)
- 4852-aol-reply-threading — AOL/provider email reply threading
- 4932-hide-pending-convos-from-hosts — Pending conversations hidden from host inbox
- 4976-listing-analytics-data-quality — Listing Analytics data quality fixes
- 5110-city-price-stats-varchar — city_price_stats varchar overflow fix
Notable:
- lcp-optimization — LCP p75 mobile at 10,698ms vs 2,500ms target. DECLINING.
- fraud-lister-only-contact-gate — 2,026 RiskReviews/day vs 45 target. Producer appears ungated.
- auth-users-active-on-register-complete — 4,718 users in half-state (register_complete=true, active=false). Needs backfill decision.
- ci-test-fixes-round-2 — CI pass rate 3% vs 100% target.
- slack-issues-skill-incremental-state — 100 duplicate issues filed vs 0 target.
Remaining 20 off-track launches are boolean/count metrics stuck at 0 vs target 1 (likely broken collectors, not feature failures): send-queries-null-state-gate, posthog-payment-trigger-mobile-checkout, listing-analytics-dashboard-refresh, attribution-phase-a-referral-utm, posthog-session-recording-re-enable, pro-video-tour, 4646-hide-listings-stripe-payment-in-review, 4630-post-room-address-field-stuck-disabled, 4552-messaging-widget-design-restore, 3898-message-form-name-after-profile-complete, 3701-room-availability-key-guard, 3726-payment-selectedplan-cookie-fallback, 4259-messaging-logged-out, 4338-systemic-checkbox-trait, 4316-room-under-review-uncheck, 4319-labs-nav-kpi-smoke, edu-fraud-prevention, fraud-mark-admin-and-activation-gate, prebook-duplicate-charges-fix, null-location-fix.
All meeting targets. Notable performers:
- posthog-messaging-instrumentation — 7,410 events vs target 1 (instrumentation live)
- 4554-off-platform-contact-detection — 1,644 RiskReviews created vs target 5 (fraud detection live)
- geo-radius-listing-price-stats — 49.9% coverage vs 40% target
- travel-nurse-landing — Ranking 14.95 vs target 20 (better = lower)
Remaining 20 green launches are bugfix zero-error metrics holding at 0.
Metrics registered but 0 data points collected (32-68 days):
- display-name-emails-fix (68d)
- 4636-riskreview-status-filter-regression (40d)
- 4647-riskreview-default-filter-all (40d)
- 4501-data-deletion-v1 (40d)
- chat-pending-inquiry-banner (34d)
- messaging-widget-design-restore (32d)
All 62+ days old with no primary metric data. Bulk from deployment-2026-04-14 cluster. Need /write-kpis or removal from monitoring.
Full list (20)
profile-ownership-sweep, broadcast-distance-validation, subscription-rate-limiter-fix, payment-rate-limiting, fraud-stripe-cancel, supply-chain-hardening, email-reply-read, softdeleted-user-conversations, haversine-distance, pending-listing-ui, edit-listing-redirect, listing-photo-formdata, listing-image-overflow, verification-save-order, faq-instructions-fix, draft-modal-fixes, subscription-type-null-guard, admin-chat-query, admin-boolean-fields, incomplete-account-flow
| Bucket | Count | % |
|---|---|---|
| Tier-1 deep-dived | 10 | 10% |
| First appraisal | 6 | 6% |
| Off-track | 25 | 25% |
| Stable green | 24 | 24% |
| Pending data | 6 | 6% |
| Envelope broken | 20 | 20% |
| Other | 8 | 8% |
| Total monitoring | 99 | 100% |
Cross-launch note: mobile-address-input-styling and address-autocomplete-mobile-styling share the same underlying metric (mobile listing creation completion rate) but are registered as separate launches. Consider merging into one launch to avoid double-counting.