Skip to content

Instantly share code, notes, and snippets.

@betatim
Last active October 27, 2026 12:59
Show Gist options
  • Select an option

  • Save betatim/5b92228b6cc7fffd459bf0e04e5c2f67 to your computer and use it in GitHub Desktop.

Select an option

Save betatim/5b92228b6cc7fffd459bf0e04e5c2f67 to your computer and use it in GitHub Desktop.
A skill that helps collect information about an issue/PR to help judge if it is "worth it". The last three files should be in the `scripts/` directory of the skill (no way to do directories in a gist)

Script reference

Run all of these from the root of the scikit-learn checkout, with required_permissions: ["all"]. The default sandbox blocks writes to agents/, which is a symlink to a sibling repository.

Paths below are relative to ~/.cursor/skills/pr-impact-brief/.

scripts/fetch_open_prs.py

Fetches every open PR into a JSON cache via GraphQL. Cheap: the whole repository costs about 23 rate-limit points out of 5000.

python3 ~/.cursor/skills/pr-impact-brief/scripts/fetch_open_prs.py
       [--repo OWNER/NAME]        default scikit-learn/scikit-learn
       [--cache PATH]             default agents/impact/.cache/open-prs-<repo>.json
       [--max-age-hours N]        reuse cache if younger than N (default 24)
       [--force]                  refetch regardless of age
       [--quiet]

Safe to call at the start of every session: it prints cache fresh (Nh old) and exits without network access when the cache is young enough.

Page size is fixed at 25. Larger pages exceed GitHub's node limit and return an empty body rather than an error, so do not raise it.

scripts/filter_prs.py

Applies the worklist filter and draws the next PR.

python3 ~/.cursor/skills/pr-impact-brief/scripts/filter_prs.py
       [--featurey | --all]       narrow to public-surface additions, or widen
       [--pick N] [--seed N]      draw N unbriefed PRs at random
       [--brief-dir PATH]         default agents/impact, searched recursively
       [--include-briefed]        do not skip PRs that already have a brief
       [--json]                   machine-readable, includes the brief_path to write
       [--counts]                 print filter cardinalities and exit
       [--cache PATH] [--repo OWNER/NAME]

Modes, with cardinalities as of July 2026:

Mode Meaning Count
default touches scikit-learn source, no core-dev has commented or reviewed 106
--featurey the above, restricted to ENH/FEA/API/PRF or a feature label 22
--all every human non-draft PR 465

--json is the form to use in a session: it carries brief_path, so the filename is decided by the script and stays stable for a given PR. brief_path always points at the top level of --brief-dir, which is where new briefs go.

The briefed check is recursive, so a brief the user has filed under agents/impact/done/ still counts and its PR is not drawn again. Archiving is therefore safe; only deleting a brief puts a PR back in the draw.

Exit code 3 means the worklist is exhausted — every PR in that mode already has a brief. Say so rather than widening the mode unasked.

scripts/pr_context.py

Builds the deterministic context bundle for one PR.

python3 ~/.cursor/skills/pr-impact-brief/scripts/pr_context.py NUMBER
       [--repo OWNER/NAME]
       [--out PATH]               write to a file instead of stdout
       [--json]                   raw bundle instead of rendered markdown
       [--diff-lines N]           cap on diff lines (default 2000, 0 for none)
       [--no-prior-art]           skip the history search (saves ~12s)
       [--cache PATH]             open-PR cache, for overlap detection

Sections produced:

  1. Metadata and PR body.
  2. Changed files with per-file line counts.
  3. Public API surface delta — base and head of each changed .py file are parsed with ast and compared. Reports new and removed public callables, new and removed parameters, and changed defaults. __init__ counts as public because for an estimator its parameters are the public API. Caveat: per-file and per-class, so it does not follow inheritance. A method moved to a new base class appears as both removed and added. Check the diff before calling anything removed.
  4. Linked issues with full discussion, from closingIssuesReferences plus Fixes/Closes/Resolves in the body. Usually the only place user impact is stated at all.
  5. PR discussion.
  6. Prior art — title searches of repository history for identifiers derived from the PR. Identifier-like terms (containing _ or a capital) get up to 25 hits; bare English words are capped at 4 most recent, with a note giving the true total, because a word like "categorical" matches 126 titles and buries the one result that matters. Throttled to 2.5s between queries, at most 5 queries, to stay under the search API's 30/minute limit. Search failures are reported inline rather than swallowed.
  7. Other open PRs touching the same files — read from the local cache at zero API cost. This is how overlapping and duplicate PRs get noticed.

Take the whole diff into account, but do not review it. Correctness and style are out of scope.

#!/usr/bin/env python3
"""Fetch every open PR of a repository into a local JSON cache via the GitHub GraphQL API.
Only bulk metadata is fetched here: enough to apply the worklist filter without
touching the REST search API, which is capped at 30 requests/minute. Per-PR
detail (body, diff, discussion) is the job of pr_context.py.
Page size is deliberately 25. Larger pages combined with the nested `files` and
`comments` selections exceed GitHub's node limit, and the failure mode is an
empty response body rather than an error, so it is easy to mistake for success.
"""
from __future__ import annotations
import argparse
import datetime as dt
import json
import pathlib
import subprocess
import sys
import time
DEFAULT_REPO = "scikit-learn/scikit-learn"
PAGE_SIZE = 25
QUERY = """
query($owner: String!, $name: String!, $size: Int!, $cursor: String) {
rateLimit { cost remaining resetAt }
repository(owner: $owner, name: $name) {
pullRequests(states: OPEN, first: $size, after: $cursor) {
totalCount
pageInfo { hasNextPage endCursor }
nodes {
number title url createdAt updatedAt isDraft
additions deletions changedFiles
author { login }
authorAssociation
labels(first: 20) { nodes { name } }
comments(last: 40) { totalCount nodes { authorAssociation } }
reviews(last: 40) { totalCount nodes { authorAssociation state } }
files(first: 60) { totalCount nodes { path } }
closingIssuesReferences(first: 5) { nodes { number title url } }
}
}
}
}
"""
class FetchError(RuntimeError):
pass
def run_graphql(owner: str, name: str, cursor: str | None, size: int) -> dict:
"""Execute one GraphQL page, with retries for transient and node-limit failures."""
args = [
"gh", "api", "graphql",
"-f", f"query={QUERY}",
"-F", f"owner={owner}",
"-F", f"name={name}",
"-F", f"size={size}",
]
if cursor:
args += ["-F", f"cursor={cursor}"]
last_problem = "unknown"
for attempt in range(4):
proc = subprocess.run(args, capture_output=True, text=True)
# An empty body is the signature of exceeding the node limit. It is not
# reported as an error, so it must be checked explicitly.
if not proc.stdout.strip():
last_problem = f"empty response body (exit {proc.returncode}): {proc.stderr.strip()[:300]}"
else:
try:
payload = json.loads(proc.stdout)
except json.JSONDecodeError as exc:
last_problem = f"unparseable response: {exc}"
else:
if "errors" in payload:
last_problem = "GraphQL errors: " + json.dumps(payload["errors"])[:300]
elif not payload.get("data", {}).get("repository"):
last_problem = "response contained no repository data"
else:
return payload
if attempt < 3:
time.sleep(2 * (attempt + 1))
raise FetchError(last_problem)
def fetch_all(owner: str, name: str, size: int, quiet: bool) -> tuple[list[dict], dict]:
nodes: list[dict] = []
cursor: str | None = None
cost = 0
rate: dict = {}
while True:
payload = run_graphql(owner, name, cursor, size)
rate = payload["data"]["rateLimit"]
cost += rate["cost"]
page = payload["data"]["repository"]["pullRequests"]
nodes += page["nodes"]
if not quiet:
print(
f" fetched {len(nodes)}/{page['totalCount']} open PRs",
file=sys.stderr,
)
if not page["pageInfo"]["hasNextPage"]:
break
cursor = page["pageInfo"]["endCursor"]
return nodes, {"cost": cost, "remaining": rate.get("remaining")}
def cache_age_hours(path: pathlib.Path) -> float | None:
"""Age of the cache in hours, or None if it is absent or unreadable."""
if not path.exists():
return None
try:
stamp = json.loads(path.read_text())["fetched_at"]
fetched = dt.datetime.fromisoformat(stamp)
except (json.JSONDecodeError, KeyError, ValueError, OSError):
return None
if fetched.tzinfo is None:
fetched = fetched.replace(tzinfo=dt.timezone.utc)
delta = dt.datetime.now(dt.timezone.utc) - fetched
return delta.total_seconds() / 3600.0
def default_cache_path(repo: str) -> pathlib.Path:
slug = repo.replace("/", "-")
return pathlib.Path("agents/impact/.cache") / f"open-prs-{slug}.json"
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--repo", default=DEFAULT_REPO, help=f"owner/name (default {DEFAULT_REPO})")
parser.add_argument("--cache", type=pathlib.Path, help="cache file path")
parser.add_argument(
"--max-age-hours",
type=float,
default=24.0,
help="reuse the cache if it is younger than this (default 24)",
)
parser.add_argument("--force", action="store_true", help="refetch regardless of cache age")
parser.add_argument("--quiet", action="store_true", help="suppress progress output")
args = parser.parse_args()
if "/" not in args.repo:
print(f"error: --repo must be owner/name, got {args.repo!r}", file=sys.stderr)
return 2
owner, name = args.repo.split("/", 1)
cache = args.cache or default_cache_path(args.repo)
age = cache_age_hours(cache)
if not args.force and age is not None and age < args.max_age_hours:
data = json.loads(cache.read_text())
print(f"cache fresh ({age:.1f}h old): {data['count']} open PRs at {cache}")
return 0
if not args.quiet:
reason = "no usable cache" if age is None else f"cache {age:.1f}h old"
print(f"fetching {args.repo} ({reason})...", file=sys.stderr)
try:
nodes, usage = fetch_all(owner, name, PAGE_SIZE, args.quiet)
except FetchError as exc:
print(f"error: fetch failed: {exc}", file=sys.stderr)
return 1
cache.parent.mkdir(parents=True, exist_ok=True)
cache.write_text(
json.dumps(
{
"fetched_at": dt.datetime.now(dt.timezone.utc).isoformat(),
"repo": args.repo,
"count": len(nodes),
"prs": nodes,
},
indent=1,
)
)
truncated = [n["number"] for n in nodes if n["files"]["totalCount"] > 60]
print(f"wrote {len(nodes)} open PRs to {cache} (rate-limit cost {usage['cost']}, {usage['remaining']} remaining)")
if truncated:
print(f"note: file lists truncated at 60 for PRs {truncated}")
return 0
if __name__ == "__main__":
sys.exit(main())
#!/usr/bin/env python3
"""Apply the worklist filter to the open-PR cache and draw the next PR to brief.
The default worklist is PRs that touch scikit-learn source and that no core
developer has ever commented on or reviewed. Those are the PRs where the
question "should we carry this?" has demonstrably never been asked.
Core-dev engagement is detected via `authorAssociation` of MEMBER or OWNER on
any comment or review. PRs authored by a core dev are deliberately kept when no
*other* core dev has engaged: an unattended core-dev PR carries the same
unanswered question.
"""
from __future__ import annotations
import argparse
import json
import pathlib
import random
import re
import sys
CORE_ASSOCIATIONS = {"MEMBER", "OWNER"}
FEATURE_TITLE = re.compile(r"^\s*(ENH|FEA|API|PRF)\b", re.IGNORECASE)
FEATURE_LABELS = {"New Feature", "Enhancement", "API"}
DOC_SUFFIXES = (".rst", ".md", ".txt")
# Leading commit-tag conventions stripped when building a filename slug.
TITLE_TAG = re.compile(
r"^\s*(ENH|FEA|FIX|DOC|MAINT|CI|TST|BLD|PRF|API|RFC|CLN|BUG|POC|WIP)[:\s]+",
re.IGNORECASE,
)
def is_bot(pr: dict) -> bool:
login = (pr.get("author") or {}).get("login") or ""
return login.endswith("[bot]") or login in {"scikit-learn-bot", "dependabot"}
def core_engaged(pr: dict) -> bool:
for comment in pr["comments"]["nodes"]:
if comment["authorAssociation"] in CORE_ASSOCIATIONS:
return True
for review in pr["reviews"]["nodes"]:
if review["authorAssociation"] in CORE_ASSOCIATIONS:
return True
return False
def core_approved(pr: dict) -> bool:
return any(
r["state"] == "APPROVED" and r["authorAssociation"] in CORE_ASSOCIATIONS
for r in pr["reviews"]["nodes"]
)
def touches_sklearn_source(pr: dict) -> bool:
for f in pr["files"]["nodes"]:
path = f["path"]
if not path.startswith("sklearn/"):
continue
if "/tests/" in path or path.startswith("sklearn/tests/"):
continue
if path.endswith((".py", ".pyx", ".pxd", ".pyx.tp")):
return True
return False
def docs_only(pr: dict) -> bool:
paths = [f["path"] for f in pr["files"]["nodes"]]
if not paths:
return False
return all(
p.startswith(("doc/", "examples/")) or p.endswith(DOC_SUFFIXES) for p in paths
)
def featurey(pr: dict) -> bool:
labels = {node["name"] for node in pr["labels"]["nodes"]}
return bool(FEATURE_TITLE.match(pr["title"]) or (labels & FEATURE_LABELS))
def slug_for(pr: dict, words: int = 7) -> str:
"""Deterministic filename slug, so the same PR always maps to the same file."""
title = TITLE_TAG.sub("", pr["title"])
cleaned = re.sub(r"[^a-z0-9]+", "-", title.lower()).strip("-")
parts = [part for part in cleaned.split("-") if part]
return "-".join(parts[:words]) or "untitled"
def modules_of(pr: dict) -> list[str]:
return sorted(
node["name"].removeprefix("module:")
for node in pr["labels"]["nodes"]
if node["name"].startswith("module:")
)
def build_worklist(prs: list[dict], mode: str) -> list[dict]:
candidates = [p for p in prs if not p["isDraft"] and not is_bot(p)]
if mode == "all":
return candidates
selected = [
p
for p in candidates
if not core_engaged(p)
and touches_sklearn_source(p)
and not docs_only(p)
and not core_approved(p)
]
if mode == "featurey":
selected = [p for p in selected if featurey(p)]
return selected
def already_briefed(brief_dir: pathlib.Path) -> set[int]:
"""PR numbers that already have a brief. Existence of the file is the state.
Recursive, so filing a brief under `done/` still counts as briefed. Without
that, archiving a brief would put its PR back in the draw.
"""
numbers: set[int] = set()
if not brief_dir.exists():
return numbers
for path in brief_dir.rglob("*.md"):
head = path.stem.split("-", 1)[0]
if head.isdigit():
numbers.add(int(head))
return numbers
def print_table(rows: list[dict], briefed: set[int]) -> None:
for pr in rows:
mark = "*" if pr["number"] in briefed else " "
mods = ",".join(modules_of(pr)) or "-"
print(
f"{mark}#{pr['number']:<6} {pr['createdAt'][:7]} "
f"+{pr['additions']:<5}/-{pr['deletions']:<5} "
f"{pr['authorAssociation'][:22]:<22} {mods[:24]:<24} {pr['title'][:70]}"
)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--cache", type=pathlib.Path, help="open-PR cache path")
parser.add_argument("--repo", default="scikit-learn/scikit-learn")
group = parser.add_mutually_exclusive_group()
group.add_argument(
"--featurey",
action="store_const",
const="featurey",
dest="mode",
help="narrow to public-surface additions only",
)
group.add_argument(
"--all",
action="store_const",
const="all",
dest="mode",
help="widen to every human non-draft PR",
)
parser.add_argument(
"--brief-dir",
type=pathlib.Path,
default=pathlib.Path("agents/impact"),
help="directory of existing briefs (default agents/impact)",
)
parser.add_argument("--pick", type=int, metavar="N", help="draw N unbriefed PRs at random")
parser.add_argument("--seed", type=int, help="seed the random draw for reproducibility")
parser.add_argument("--include-briefed", action="store_true", help="do not skip briefed PRs")
parser.add_argument("--json", action="store_true", help="emit JSON instead of a table")
parser.add_argument("--counts", action="store_true", help="print filter cardinalities and exit")
parser.set_defaults(mode="default")
args = parser.parse_args()
cache = args.cache or pathlib.Path("agents/impact/.cache") / f"open-prs-{args.repo.replace('/', '-')}.json"
if not cache.exists():
print(f"error: no cache at {cache}; run fetch_open_prs.py first", file=sys.stderr)
return 1
data = json.loads(cache.read_text())
prs = data["prs"]
if args.counts:
print(f"cache fetched {data['fetched_at']} repo {data['repo']}")
print(f"{'total open':46s} {len(prs)}")
print(f"{' drafts':46s} {sum(1 for p in prs if p['isDraft'])}")
print(f"{' bot-authored':46s} {sum(1 for p in prs if is_bot(p))}")
for label, mode in [
("--all (human, non-draft)", "all"),
("default (unengaged + touches sklearn)", "default"),
("--featurey", "featurey"),
]:
print(f"{label:46s} {len(build_worklist(prs, mode))}")
return 0
worklist = build_worklist(prs, args.mode)
worklist.sort(key=lambda p: p["createdAt"])
briefed = already_briefed(args.brief_dir)
pool = worklist if args.include_briefed else [p for p in worklist if p["number"] not in briefed]
if args.pick is not None:
if not pool:
print(
f"worklist exhausted: all {len(worklist)} PRs in mode '{args.mode}' already have briefs "
f"in {args.brief_dir}",
file=sys.stderr,
)
return 3
rng = random.Random(args.seed)
pool = rng.sample(pool, min(args.pick, len(pool)))
if args.json:
print(
json.dumps(
[
{
"number": p["number"],
"title": p["title"],
"url": p["url"],
"slug": slug_for(p),
"brief_path": str(args.brief_dir / f"{p['number']}-{slug_for(p)}.md"),
"created_at": p["createdAt"],
"updated_at": p["updatedAt"],
"author": (p.get("author") or {}).get("login"),
"author_association": p["authorAssociation"],
"additions": p["additions"],
"deletions": p["deletions"],
"changed_files": p["changedFiles"],
"labels": [n["name"] for n in p["labels"]["nodes"]],
"modules": modules_of(p),
"featurey": featurey(p),
"comment_count": p["comments"]["totalCount"],
"review_count": p["reviews"]["totalCount"],
"linked_issues": [
{"number": c["number"], "title": c["title"], "url": c["url"]}
for c in p["closingIssuesReferences"]["nodes"]
],
"already_briefed": p["number"] in briefed,
}
for p in pool
],
indent=1,
)
)
return 0
print_table(pool, briefed)
suffix = "" if args.include_briefed else f", {len(briefed & {p['number'] for p in worklist})} already briefed and skipped"
print(f"\n{len(pool)} PRs shown (mode '{args.mode}', worklist {len(worklist)}{suffix})")
return 0
if __name__ == "__main__":
sys.exit(main())
#!/usr/bin/env python3
"""Assemble the deterministic context bundle for one PR.
Everything here is mechanical and sourced. The judgment that this bundle feeds
-- reach, counterfactual availability, cost, and the questions the decision
hinges on -- is the model's job, not this script's.
Four things are gathered:
1. PR metadata, body, and the diff.
2. Linked issues with their discussion, which is usually where the only
statement of user impact lives.
3. The public API surface delta, computed by parsing the base and head versions
of each changed Python file with `ast` and comparing them. Path heuristics do
not work for scikit-learn, whose public names live in privately-named files
such as `_classification.py`, so the surface is derived from the code itself.
4. Prior art: earlier issues and PRs proposing the same thing, plus other open
PRs touching the same files. The latter comes from the local cache at zero
API cost and is how overlapping PRs get noticed.
"""
from __future__ import annotations
import argparse
import ast
import base64
import json
import pathlib
import re
import subprocess
import sys
import time
DEFAULT_REPO = "scikit-learn/scikit-learn"
# The REST search API allows 30 requests/minute. Stay well under it.
SEARCH_SLEEP_S = 2.5
MAX_SEARCH_QUERIES = 5
MAX_SURFACE_FILES = 12
MAX_LINKED_ISSUES = 6
PR_QUERY = """
query($owner: String!, $name: String!, $number: Int!) {
repository(owner: $owner, name: $name) {
pullRequest(number: $number) {
number title url body state isDraft createdAt updatedAt
additions deletions changedFiles
author { login }
authorAssociation
baseRefName headRefName baseRefOid headRefOid
isCrossRepository
headRepository { nameWithOwner }
labels(first: 25) { nodes { name } }
files(first: 100) { totalCount nodes { path additions deletions } }
comments(first: 60) {
totalCount
nodes { author { login } authorAssociation createdAt body }
}
reviews(first: 40) {
totalCount
nodes { author { login } authorAssociation state submittedAt body }
}
closingIssuesReferences(first: 10) { nodes { number } }
}
}
}
"""
ISSUE_QUERY = """
query($owner: String!, $name: String!, $number: Int!) {
repository(owner: $owner, name: $name) {
issue(number: $number) {
number title url state stateReason createdAt closedAt body
author { login }
reactions { totalCount }
labels(first: 25) { nodes { name } }
comments(first: 40) {
totalCount
nodes { author { login } authorAssociation createdAt body }
}
}
}
}
"""
STOPWORDS = {
"add", "adds", "added", "adding", "allow", "allows", "and", "any", "are", "avoid",
"based", "better", "between", "bug", "but", "can", "case", "cases", "change",
"changes", "check", "class", "common", "correct", "data", "default", "deprecate",
"doc", "docs", "does", "dont", "during", "else", "ensure", "error", "example",
"examples", "fail", "failing", "fails", "fix", "fixed", "fixes", "for", "from",
"function", "handle", "handling", "has", "have", "improve", "improved", "input",
"instead", "into", "issue", "make", "makes", "method", "methods", "missing",
"more", "new", "not", "now", "only", "option", "parameter", "parameters", "pass",
"raise", "raises", "remove", "removed", "return", "returns", "same", "set",
"should", "small", "some", "support", "test", "tests", "that", "the", "their",
"then", "there", "this", "throw", "use", "used", "using", "value", "values",
"when", "where", "which", "with", "without", "wrong",
}
# --------------------------------------------------------------------------- gh
def gh(args: list[str]) -> str:
proc = subprocess.run(args, capture_output=True, text=True)
if proc.returncode != 0 or not proc.stdout.strip():
raise RuntimeError(f"{' '.join(args[:3])} failed: {proc.stderr.strip()[:300]}")
return proc.stdout
def graphql(query: str, **variables) -> dict:
args = ["gh", "api", "graphql", "-f", f"query={query}"]
for key, value in variables.items():
args += ["-F", f"{key}={value}"]
payload = json.loads(gh(args))
if "errors" in payload:
raise RuntimeError("GraphQL errors: " + json.dumps(payload["errors"])[:300])
return payload["data"]
def fetch_blob(repo: str, path: str, ref: str) -> str | None:
"""File contents at a ref. Works for fork head SHAs via the base repo."""
try:
raw = gh(["gh", "api", f"repos/{repo}/contents/{path}?ref={ref}"])
except RuntimeError:
return None
try:
node = json.loads(raw)
if node.get("encoding") != "base64":
return None
return base64.b64decode(node["content"]).decode("utf-8", errors="replace")
except (json.JSONDecodeError, KeyError, ValueError):
return None
# ----------------------------------------------------------- public API surface
def _params(node: ast.FunctionDef | ast.AsyncFunctionDef) -> dict[str, str]:
"""Parameter name -> rendered default ('' when the parameter is required)."""
out: dict[str, str] = {}
pos = list(node.args.posonlyargs) + list(node.args.args)
defaults = list(node.args.defaults)
pad = len(pos) - len(defaults)
for index, arg in enumerate(pos):
if arg.arg == "self":
continue
default = defaults[index - pad] if index >= pad else None
out[arg.arg] = _render(default)
if node.args.vararg:
out["*" + node.args.vararg.arg] = ""
for arg, default in zip(node.args.kwonlyargs, node.args.kw_defaults):
out[arg.arg] = _render(default)
if node.args.kwarg:
out["**" + node.args.kwarg.arg] = ""
return out
def _render(node: ast.AST | None) -> str:
if node is None:
return ""
try:
return ast.unparse(node)
except Exception:
return "?"
def public_surface(source: str) -> dict:
"""Public functions, classes and methods with their parameters.
`__init__` is treated as public because for a scikit-learn estimator its
parameters *are* the public API.
"""
try:
tree = ast.parse(source)
except SyntaxError:
return {}
surface: dict[str, dict[str, str]] = {}
for node in tree.body:
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
if not node.name.startswith("_"):
surface[node.name] = _params(node)
elif isinstance(node, ast.ClassDef) and not node.name.startswith("_"):
surface[node.name + " (class)"] = {}
for item in node.body:
if not isinstance(item, (ast.FunctionDef, ast.AsyncFunctionDef)):
continue
if item.name.startswith("_") and item.name != "__init__":
continue
surface[f"{node.name}.{item.name}"] = _params(item)
return surface
def surface_delta(base_src: str | None, head_src: str | None) -> dict:
"""Added/removed callables, and added/removed/retyped parameters."""
base = public_surface(base_src) if base_src else {}
head = public_surface(head_src) if head_src else {}
delta: dict[str, list] = {
"added_callables": sorted(set(head) - set(base)),
"removed_callables": sorted(set(base) - set(head)),
"added_params": [],
"removed_params": [],
"changed_defaults": [],
}
for name in sorted(set(base) & set(head)):
before, after = base[name], head[name]
for param in sorted(set(after) - set(before)):
default = after[param]
delta["added_params"].append(
f"{name}({param}{'=' + default if default else ''})"
)
for param in sorted(set(before) - set(after)):
delta["removed_params"].append(f"{name}({param})")
for param in sorted(set(before) & set(after)):
if before[param] != after[param]:
delta["changed_defaults"].append(
f"{name}({param}): {before[param] or '<required>'} -> {after[param] or '<required>'}"
)
return delta
# ------------------------------------------------------------------ prior art
def search_terms(title: str, surface: dict) -> list[str]:
"""Distinctive identifiers worth searching history for."""
terms: list[str] = []
for match in re.findall(r"`([^`]+)`", title):
terms.append(match.strip())
for match in re.findall(r"\b([A-Z][a-zA-Z0-9]*[a-z][a-zA-Z0-9]*[A-Z][a-zA-Z0-9]*)\b", title):
terms.append(match)
for entry in surface.get("added_params", []):
name = re.search(r"\(([^=)]+)", entry)
if name:
terms.append(name.group(1))
for entry in surface.get("added_callables", []):
terms.append(entry.replace(" (class)", "").split(".")[-1])
for word in re.findall(r"\b[a-z][a-z_]{4,}\b", title.lower()):
if word not in STOPWORDS:
terms.append(word)
seen: list[str] = []
for term in terms:
term = term.strip().strip("`'\"")
if len(term) < 4 or term.lower() in STOPWORDS:
continue
if term.lower() not in {t.lower() for t in seen}:
seen.append(term)
return seen[:MAX_SEARCH_QUERIES]
def is_identifier_like(term: str) -> bool:
"""Code identifiers are high-precision search terms; bare English words are not."""
return "_" in term or any(c.isupper() for c in term)
def prior_art(repo: str, number: int, terms: list[str]) -> tuple[list[dict], list[str]]:
"""Earlier issues and PRs mentioning the same identifiers.
Broad single-word terms are capped hard. Searching scikit-learn history for
a word like "categorical" returns dozens of unrelated hits that bury the one
result that matters, which is usually a previous attempt at the same change.
"""
hits: dict[int, dict] = {}
notes: list[str] = []
# Search the precise terms first, so the per-term budget is spent well.
ordered = sorted(terms, key=lambda t: not is_identifier_like(t))
for index, term in enumerate(ordered):
if index:
time.sleep(SEARCH_SLEEP_S)
broad = not is_identifier_like(term)
cap = 4 if broad else 25
query = f'repo:{repo} "{term}" in:title'
try:
raw = gh([
"gh", "api", "-X", "GET", "search/issues",
"-f", f"q={query}", "-f", "per_page=30",
"-f", "sort=created", "-f", "order=desc",
])
except RuntimeError as exc:
notes.append(f"search for {term!r} failed ({str(exc)[:80]}) -- prior art may be incomplete")
continue
try:
payload = json.loads(raw)
items = payload.get("items", [])
except json.JSONDecodeError:
notes.append(f"search for {term!r} returned unparseable JSON")
continue
if broad and payload.get("total_count", 0) > cap:
notes.append(
f"{term!r} is a broad term with {payload['total_count']} title matches; "
f"showing the {cap} most recent only"
)
for item in items[:cap]:
if item["number"] == number:
continue
hits.setdefault(item["number"], {
"number": item["number"],
"title": item["title"],
"url": item["html_url"],
"kind": "pr" if "pull_request" in item else "issue",
"state": item["state"],
"merged": bool(item.get("pull_request", {}).get("merged_at")),
"created_at": item["created_at"][:10],
"closed_at": (item.get("closed_at") or "")[:10],
"matched": term,
})
return sorted(hits.values(), key=lambda h: h["created_at"]), notes
def overlapping_open_prs(cache: pathlib.Path, number: int, paths: set[str]) -> list[dict]:
"""Other open PRs touching the same files. Read from the cache, no API cost."""
if not cache.exists():
return []
try:
prs = json.loads(cache.read_text())["prs"]
except (json.JSONDecodeError, KeyError, OSError):
return []
out = []
for pr in prs:
if pr["number"] == number:
continue
shared = paths & {f["path"] for f in pr["files"]["nodes"]}
if shared:
out.append({
"number": pr["number"],
"title": pr["title"],
"url": pr["url"],
"created_at": pr["createdAt"][:10],
"is_draft": pr["isDraft"],
"shared_files": sorted(shared),
})
return sorted(out, key=lambda p: -len(p["shared_files"]))
# --------------------------------------------------------------------- render
def truncate(text: str | None, limit: int) -> str:
text = (text or "").strip()
if len(text) <= limit:
return text or "(empty)"
return text[:limit] + f"\n... [truncated, {len(text) - limit} more characters]"
def render(bundle: dict, diff_lines: int) -> str:
pr = bundle["pr"]
out: list[str] = []
add = out.append
add(f"# Context bundle: PR #{pr['number']} -- {pr['title']}")
add("")
add(f"- URL: {pr['url']}")
add(f"- Author: {(pr.get('author') or {}).get('login')} ({pr['authorAssociation']})")
add(f"- Opened {pr['createdAt'][:10]}, last updated {pr['updatedAt'][:10]}")
add(f"- Size: +{pr['additions']}/-{pr['deletions']} across {pr['changedFiles']} files")
add(f"- Labels: {', '.join(n['name'] for n in pr['labels']['nodes']) or 'none'}")
add(f"- Base {pr['baseRefName']} @ {pr['baseRefOid'][:9]}, head {pr['headRefName']} @ {pr['headRefOid'][:9]}")
add(f"- Comments: {pr['comments']['totalCount']}, reviews: {pr['reviews']['totalCount']}")
add("")
add("## PR body")
add("")
add(truncate(pr["body"], 4000))
add("")
add("## Changed files")
add("")
for f in pr["files"]["nodes"]:
add(f"- `{f['path']}` +{f['additions']}/-{f['deletions']}")
if pr["files"]["totalCount"] > len(pr["files"]["nodes"]):
add(f"- ... {pr['files']['totalCount'] - len(pr['files']['nodes'])} more files")
add("")
add("## Public API surface delta")
add("")
add("Computed by parsing base and head with `ast`. Empty means this PR adds no public surface.")
add("")
add("Caveat: this is a per-file, per-class view and does not follow inheritance. A method "
"moved to a new base class shows up as both removed and added, which is a refactor "
"rather than a removal of public API. Check the diff before describing anything as removed.")
add("")
surface = bundle["surface"]
if not any(surface.get(k) for k in ("added_callables", "removed_callables", "added_params", "removed_params", "changed_defaults")):
add("No public API surface change detected.")
else:
for key, label in [
("added_callables", "New public callables"),
("removed_callables", "Removed public callables"),
("added_params", "New parameters"),
("removed_params", "Removed parameters"),
("changed_defaults", "Changed defaults (behaviour change for existing users)"),
]:
if surface.get(key):
add(f"**{label}:**")
for entry in surface[key]:
add(f"- `{entry}`")
add("")
for note in bundle["surface_notes"]:
add(f"- note: {note}")
add("")
add("## Linked issues")
add("")
if not bundle["issues"]:
add("No linked issue. The PR body is the only statement of motivation.")
for issue in bundle["issues"]:
state = issue["state"] + (f"/{issue['stateReason']}" if issue.get("stateReason") else "")
add(f"### #{issue['number']} {issue['title']} [{state}]")
add("")
add(f"{issue['url']} -- opened {issue['createdAt'][:10]} by "
f"{(issue.get('author') or {}).get('login')}, "
f"{issue['reactions']['totalCount']} reactions, "
f"{issue['comments']['totalCount']} comments")
add(f"Labels: {', '.join(n['name'] for n in issue['labels']['nodes']) or 'none'}")
add("")
add(truncate(issue["body"], 2500))
add("")
if issue["comments"]["nodes"]:
add("Discussion:")
add("")
for comment in issue["comments"]["nodes"]:
who = (comment.get("author") or {}).get("login")
add(f"- **{who}** ({comment['authorAssociation']}, {comment['createdAt'][:10]}): "
f"{truncate(comment['body'], 700)}")
add("")
add("## PR discussion")
add("")
if not pr["comments"]["nodes"] and not pr["reviews"]["nodes"]:
add("None. Nobody has said anything on this PR.")
for comment in pr["comments"]["nodes"]:
who = (comment.get("author") or {}).get("login")
add(f"- **{who}** ({comment['authorAssociation']}, {comment['createdAt'][:10]}): "
f"{truncate(comment['body'], 700)}")
for review in pr["reviews"]["nodes"]:
who = (review.get("author") or {}).get("login")
add(f"- **{who}** review {review['state']} ({review['authorAssociation']}, "
f"{(review.get('submittedAt') or '')[:10]}): {truncate(review['body'], 700)}")
add("")
add("## Prior art: history mentioning the same identifiers")
add("")
add(f"Search terms: {', '.join(bundle['terms']) or 'none derived'}")
add("")
if not bundle["prior_art"]:
add("No earlier issues or PRs found for these terms.")
for hit in bundle["prior_art"]:
status = "merged" if hit["merged"] else hit["state"]
closed = f", closed {hit['closed_at']}" if hit["closed_at"] else ""
add(f"- [{hit['kind']} #{hit['number']}]({hit['url']}) [{status}] "
f"created {hit['created_at']}{closed} -- {hit['title']} (matched `{hit['matched']}`)")
for note in bundle["prior_art_notes"]:
add(f"- note: {note}")
add("")
add("## Other open PRs touching the same files")
add("")
if not bundle["overlaps"]:
add("None.")
for overlap in bundle["overlaps"]:
draft = " (draft)" if overlap["is_draft"] else ""
add(f"- [#{overlap['number']}]({overlap['url']}){draft} opened {overlap['created_at']} -- "
f"{overlap['title']} -- shares {', '.join('`' + p + '`' for p in overlap['shared_files'])}")
add("")
add("## Diff")
add("")
diff = bundle["diff"].splitlines()
if len(diff) > diff_lines:
add(f"Truncated to the first {diff_lines} of {len(diff)} lines.")
add("")
diff = diff[:diff_lines]
add("```diff")
out.extend(diff)
add("```")
return "\n".join(out)
# ----------------------------------------------------------------------- main
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("number", type=int, help="PR number")
parser.add_argument("--repo", default=DEFAULT_REPO)
parser.add_argument("--out", type=pathlib.Path, help="write to this path instead of stdout")
parser.add_argument("--json", action="store_true", help="emit the raw bundle as JSON")
parser.add_argument("--diff-lines", type=int, default=2000, help="cap on diff lines (default 2000)")
parser.add_argument("--no-prior-art", action="store_true", help="skip the throttled history search")
parser.add_argument("--cache", type=pathlib.Path, help="open-PR cache, for overlap detection")
args = parser.parse_args()
owner, name = args.repo.split("/", 1)
try:
pr = graphql(PR_QUERY, owner=owner, name=name, number=args.number)["repository"]["pullRequest"]
except RuntimeError as exc:
print(f"error: could not fetch PR #{args.number}: {exc}", file=sys.stderr)
return 1
if pr is None:
print(f"error: PR #{args.number} not found in {args.repo}", file=sys.stderr)
return 1
try:
diff = gh(["gh", "pr", "diff", str(args.number), "--repo", args.repo])
except RuntimeError as exc:
diff = f"(diff unavailable: {exc})"
# Public API surface delta, per changed Python file.
surface: dict[str, list] = {
"added_callables": [], "removed_callables": [],
"added_params": [], "removed_params": [], "changed_defaults": [],
}
surface_notes: list[str] = []
py_files = [
f["path"] for f in pr["files"]["nodes"]
if f["path"].endswith(".py") and "/tests/" not in f["path"]
]
if len(py_files) > MAX_SURFACE_FILES:
surface_notes.append(
f"surface computed for the first {MAX_SURFACE_FILES} of {len(py_files)} changed Python files"
)
py_files = py_files[:MAX_SURFACE_FILES]
for path in py_files:
base_src = fetch_blob(args.repo, path, pr["baseRefOid"])
head_src = fetch_blob(args.repo, path, pr["headRefOid"])
if base_src is None and head_src is None:
surface_notes.append(f"could not read either version of `{path}`")
continue
delta = surface_delta(base_src, head_src)
for key, entries in delta.items():
surface[key].extend(f"{path}: {entry}" for entry in entries)
# Linked issues: explicit closing references plus Fixes/Closes in the body.
numbers = [node["number"] for node in pr["closingIssuesReferences"]["nodes"]]
for match in re.findall(r"(?:fix(?:e[sd])?|close[sd]?|resolve[sd]?)\s*:?\s*#(\d+)", pr["body"] or "", re.I):
if int(match) not in numbers:
numbers.append(int(match))
issues = []
for issue_number in numbers[:MAX_LINKED_ISSUES]:
try:
issue = graphql(ISSUE_QUERY, owner=owner, name=name, number=issue_number)["repository"]["issue"]
except RuntimeError:
issue = None
if issue:
issues.append(issue)
terms = [] if args.no_prior_art else search_terms(pr["title"], surface)
hits, prior_notes = prior_art(args.repo, args.number, terms) if terms else ([], [])
if args.no_prior_art:
prior_notes = ["prior-art search skipped (--no-prior-art)"]
cache = args.cache or pathlib.Path("agents/impact/.cache") / f"open-prs-{args.repo.replace('/', '-')}.json"
overlaps = overlapping_open_prs(cache, args.number, {f["path"] for f in pr["files"]["nodes"]})
bundle = {
"pr": pr, "diff": diff, "surface": surface, "surface_notes": surface_notes,
"issues": issues, "terms": terms, "prior_art": hits,
"prior_art_notes": prior_notes, "overlaps": overlaps,
}
text = json.dumps(bundle, indent=1) if args.json else render(bundle, args.diff_lines)
if args.out:
args.out.parent.mkdir(parents=True, exist_ok=True)
args.out.write_text(text)
print(f"wrote context bundle to {args.out}")
else:
print(text)
return 0
if __name__ == "__main__":
sys.exit(main())
name pr-impact-brief
description Assemble source-linked evidence on whether a scikit-learn PR is worth carrying, one PR at a time, ending in the questions the decision hinges on and never in a verdict. Use when the user says "let's review some PRs for worth", asks to review or triage PRs for worth or impact, asks whether a PR is worth it or worth merging or worth carrying, or asks for an impact brief on a PR number or URL.

PR Impact Brief

Walk one unattended scikit-learn PR at a time and present the evidence a maintainer needs to answer "should we carry this?" — then stop. The human decides.

This skill exists because the cost side of a change is invisible by default. A PR that is small, correct, and green still adds permanent public surface, consumes review time, and displaces bigger work. Nobody asks whether it is worth that, so the question goes unanswered for years.

Hard rules

These are not style preferences. Violating any one of them defeats the purpose of the skill.

  1. Never write to GitHub. No comments, no labels, no reviews, no closing, no reopening. gh is read-only here: view, api with -X GET, pr diff, search. Never gh pr comment, gh pr close, gh issue edit, or any -X POST/PATCH/PUT/DELETE.
  2. Never state a verdict or recommendation. Not in the brief, not in the follow-up discussion, not when asked directly. If the user asks "would you merge it?", answer by naming what is still unresolved. A brief that concludes has failed, because the conclusion is the user's to reach.
  3. Every factual claim carries a source — a URL, or a file path with a line number. The one exception is the reach bucket, which is explicitly an estimate and is labelled as such.
  4. Unsourced statements go in the Unsupported claims section. If a claim cannot be sourced and cannot be dropped, it gets listed there. This makes the sourcing rule visible by construction rather than dependent on the reader's vigilance.
  5. Do not review the code. Correctness, style, and test coverage are out of scope. Other tools do that. The only question here is whether the change is worth carrying.
  6. Do not fill gaps with plausible prose. "None found" and "could not determine" are valid, informative answers. A confident paragraph about an issue you did not read is worse than an admission.
  7. Never relay the PR's own claims as findings. The author's description of what is broken, who is affected, and why it matters is a claim to check, not evidence. Check every load-bearing one against the source. In the pilot this changed the conclusion on three PRs out of four: a "security fix" turned out to guard a branch unreachable since Python 3.11.4, a "docs fix" turned out to correct text that was already accurate, and a workaround a core dev had endorsed turned out to be broken.

Permissions

The default sandbox blocks writes outside the workspace, and agents/ is a symlink to a sibling repository. So:

  • Run every script in scripts/ with required_permissions: ["all"].
  • Write brief files with required_permissions: ["all"], or via the file-writing tool if it has access.

Scripts need network access for gh, which the allowlist already covers.

Scripts

Deterministic work belongs in these scripts, never in the model. They own all GitHub access and everything countable.

Script Purpose
scripts/fetch_open_prs.py Paginated GraphQL fetch of every open PR into a JSON cache
scripts/filter_prs.py Applies the worklist filter; draws the next PR at random
scripts/pr_context.py Per-PR context bundle: diff, linked issues, public-surface delta, prior art

Full usage for each is in reference.md.

Workflow

  1. Ensure the worklist. Run fetch_open_prs.py; it refreshes only if the cache is stale, so it is cheap to call every session.
  2. Pick a PR. Run filter_prs.py --pick 1, which draws at random and skips anything already briefed. If the user named a specific PR, skip the draw.
  3. Gather context. Run pr_context.py <number>. Read the linked issue's labels first: a Needs Decision or Needs Triage label means the project has formally recorded that the worth question is open, which outranks everything else in the brief. AGENTS.md also forbids contributing against such issues.
  4. Read scikit-learn's source. This is the part no script can do, and the reason this is a skill. Two jobs here, and the second is easy to skip:
    • Answer the counterfactual. Does a sibling estimator already take this parameter, does an existing composition already achieve it? Cite file and line.
    • Verify the PR's central claim. Find the code the PR says is wrong, or the code path it says is vulnerable, and confirm it behaves as described. Check the version gate on any compatibility shim, and check whether a validating call upstream already makes the failure unreachable.
  5. Write the brief to agents/impact/<number>-<slug>.md, before presenting it.
  6. Present it in chat.
  7. Stay on the PR for follow-up questions.
  8. Advance only when told to.

Steps 5 through 8 are detailed under Session loop.

Brief template

Write to agents/impact/<number>-<slug>.md. Take <slug> from filter_prs.py --json, which supplies brief_path, so a given PR always maps to the same filename. Finished briefs live in agents/impact/done/, but the user files them there — see Resuming.

Keep it to roughly a page. Omit a section entirely when it has nothing to say, rather than padding it.

# PR #NNNNN: <title>

<author> (<association>) | opened YYYY-MM | +A/-D across F files | <labels>
Kind: public-surface addition | bug fix | performance | API change |
      documentation correctness fix | documentation consistency edit |
      security hardening

## What it changes

Two to four sentences, in terms of observable behaviour rather than
implementation. Cite the files and line numbers.

## Reach — ESTIMATE, NOT SOURCED

Bucket: most users | a common workflow | a niche workflow | one reporter

Affected population: who exactly hits this, under what conditions. This is
the stated basis for the bucket, so that a reader who knows the domain can
falsify the bucket instead of having to trust it.

## Severity

For bug-shaped PRs. One of: silently wrong results | crash or exception |
wrong documentation | annoyance with a workaround | cosmetic. Link the
reproducer, the failing test, or the issue comment that establishes it.
Omit this section for pure feature additions.

Two kinds fall outside that ladder; say so rather than forcing a rung:

- Security. Separate the theoretical impact from the exploitability, and
  source the exploitability. A scanner's severity rating is the scanner's
  opinion about a pattern, not a finding about this code.
- Documentation. Distinguish text that is *wrong* from text that is merely
  *inconsistent*. If the current wording misleads nobody, the honest rung is
  cosmetic, however plausible the PR's framing.

## Counterfactual — can users get this today without us?

For features: name the existing scikit-learn API that already covers it,
with file and line, or show the user-side workaround as code, or name the
third-party package. If nothing covers it, say so.

For fixes: is there a workaround, and what does it cost the user — a few
lines, a subclass, or reimplementing an estimator.

"None found" is a valid and informative answer. Do not invent a workaround
you have not checked against the source.

## Cost

- Public surface added: new parameters, classes, methods, modules, with file
  and line. Take this from the surface delta, not from the diff by eye.
- Backward compatibility: changed defaults, changed behaviour for existing
  callers, deprecations. Changed defaults are the highest-risk item here.
- Review effort: diff size, Cython involvement, new dependency, whether it
  touches shared core code many estimators rely on.
- Permanent maintenance: what has to keep working forever once merged.

## Prior art

Earlier issues and PRs proposing the same thing, and what happened to them.
Links and outcomes only. A previous attempt closed unmerged is the single
most decision-relevant fact this tool can surface — state it plainly and let
the reader draw the conclusion.

Watch for a *cluster* of near-identical titles: several attempts at one change
closed within weeks of each other means the project keeps deciding something,
and this PR's fate probably turns on that decision rather than on its diff.
Note that `pr_context.py` gives you titles, dates and merged-or-not, but not
close *reasons* — so the reason belongs in a hinge question, not in a claim.

Also note any other open PR touching the same files, since two PRs that
overlap are one decision rather than two.

## What this hinges on

Two or three specific questions a maintainer must answer to decide this.

They must be specific to this PR. "Is this worth the maintenance cost?" is
not a hinge question, it is a restatement of the problem. "Does `sample_weight`
on an encoder have a defined meaning when categories are unseen?" is one.

No recommendation, no confidence score, no verdict.

## Unsupported claims

Every statement above that has no link or file reference. The reach bucket
always belongs here. If this section is empty and reach is stated, something
has been dressed up as evidence.

Filling the four dimensions

Reach is the only unsourced field. Estimate it, label it, and name the population it applies to. Do not smuggle in confidence by citing download counts or code-search totals: per-API reach is not measurable this way. GitHub code search returns roughly 2,500 import matches for the obscure SelectFwe and only 17x more for LogisticRegression, because the counts are approximate and dominated by vendored copies. Reactions are near-empty on this repository too — commonly zero on issues with ten comments. Neither is evidence.

Severity comes from the linked issue and the reproducer, not from the PR author's framing. "Silently wrong results" is a much stronger claim than "raises an exception" and needs the stronger evidence.

Counterfactual is the dimension that most often decides the question, and the only one that requires reading scikit-learn's source. Actually go and look: does the sibling estimator already take this parameter, does Pipeline plus an existing transformer already compose it, is it three lines in user code. This is the work; do not skip it and assert an answer.

When the workaround is runnable, run it rather than describing it. Activate the environment as .cursorrules specifies, then execute it and paste the real output. A workaround that a core developer posted years ago may since have rotted, and only running it will tell you. In the pilot, the endorsed workaround for #29952 turned out to reject string labels outright and to report R² in place of accuracy — neither of which was visible from reading the thread, and both of which changed the picture.

Check upstream too. If a newer version of Python, NumPy or SciPy already fixes it, the counterfactual is "upgrade", and the PR is only relevant for the window of versions the project still supports. Get that window from requires-python in pyproject.toml, and get the exact fix version from the upstream changelog rather than from memory.

Cost is largely mechanical — take it from the surface delta. Two judgment parts: what the project has to keep working forever, and whether the PR bundles a change with a wider blast radius than its title suggests. A feature PR that also renames an existing public parameter is two decisions, and the reader should be told they can be separated.

Weigh maintenance against remaining lifetime. Code behind a version shim expires when the shim does, so a compatibility backport with one release cycle left is a different proposition from a permanent estimator, even at the same line count.

Apparent engagement

The worklist selects PRs no core developer has touched, but such a PR can still carry approving reviews from non-core contributors and bots, and so look attended to. If a review carries APPROVED from an author whose association is not MEMBER or OWNER, note it — the reader is otherwise likely to read the thread as "already reviewed" when nobody with merge authority has looked.

Session loop

This runs as a conversation, one PR at a time. Never brief several PRs in one turn.

Starting

Triggered by things like "let's review some PRs for worth", "review PRs for impact", or "is #34204 worth it". Then:

  1. Run fetch_open_prs.py. It no-ops on a fresh cache, so this is cheap.
  2. Say how large the worklist is and how many already have briefs, in one line. This is the only status output the user needs.
  3. Go straight to the first PR. Do not ask which mode to use or how many to review; default mode, one PR, and adjust if asked.

If the user named a specific PR, skip the draw and brief that one, whether or not it is in the worklist.

Picking

Run filter_prs.py --pick 1 --json. The draw is random and skips anything already briefed — do not impose an ordering of your own, and do not seed it unless asked, because the point of randomness is an unbiased view of what is actually in the worklist.

On exit code 3 the worklist is exhausted. Say so and stop. Do not widen to --all or --include-briefed without being asked.

Briefing

Run pr_context.py <number>, read scikit-learn's source for the counterfactual, then write the brief file before presenting it. Writing first means an interrupted session loses nothing.

Then present it in chat. Present the substance — do not just say "written to " and make the user open a file. Do not re-summarise the brief after presenting it either.

Discussing

Stay on this PR. Answer follow-ups by going and looking: read more source, fetch the diff of a prior attempt, pull an issue's full discussion. Do not answer by rephrasing what the brief already said, and do not manufacture a new consideration to seem responsive — "the brief covers what I found; the open question is still X" is a fine answer.

Expect the hardest pushback on the counterfactual, and be ready to show the actual lines that support it.

If the user asks for a verdict — "so would you merge this?", "is it worth it?" — do not give one. Answer with what is still unresolved and what would settle it. This will feel unhelpful; it is the design. The user chose it deliberately so that the brief informs their judgment instead of anchoring it.

If the user states their own conclusion, do not record it anywhere and do not offer to draft a comment. There is no verdict capture in this tool.

Advancing

Only on an explicit "next", "another", or equivalent. Then draw again. A follow-up question is not a request to advance.

Resuming

The presence of a brief file anywhere under agents/impact/ is the entire state. There is no separate bookkeeping file.

Two directories, one meaning:

  • agents/impact/ — briefs still in play. Always write new briefs here, never straight into done/.
  • agents/impact/done/ — briefs the user has finished with. Filed by hand, so the folder does not fill up.

Both count as briefed, so a PR filed under done/ is not drawn again. filter_prs.py searches recursively for this reason; do not "fix" it back to a flat glob.

  • "Show me that one again" means reading the existing file wherever it sits — check done/ too before concluding a brief does not exist. Do not regenerate it.
  • Never move a brief into done/ yourself. That is the user's signal that they are finished, and it is not yours to give.
  • "Start over" or "review those again" means deleting the files, and only the ones named. Moving a brief no longer resets anything. Confirm before deleting.
  • A brief older than a few weeks may be stale; note the file's date rather than silently trusting it.

Style rules

Follow the conventions in ~/.cursor/skills/pr-walkthrough/SKILL.md:

  • Link to code with relative markdown links from the brief's location, e.g. [_encoders.py](../../sklearn/preprocessing/_encoders.py), and put line numbers in the prose next to the link. Do not use vscode://file/ URIs or #L<n> fragments; neither renders in Cursor.
  • Be concise. Every line earns its space. A brief is a page, not an essay.
  • No flattery, no filler, no marketing tone. No emoji.
  • Show code only when prose cannot carry the point — most often when demonstrating a counterfactual workaround.
  • Prefer specifics over hedged generalities. "Affects users who pass sample_weight to OneHotEncoder.fit" beats "affects some users".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment