RB-Y1 Box Pickup Swept Volume

RB-Y1 Box Pickup Swept Volume.md

Box-transport swept-motion figure

Code for a chronophotography-style still: a mobile dual-arm robot carrying a cardboard box horizontally across a table, composited from a single fixed-camera video. Dense semi-transparent ghosts of the arms, a handful of box positions, no overlaid path graphics.

RB-Y1 design notes.md covers why it looks this way, what was measured, and which approaches were tried and rejected — including a few that failed and why.

What's here

File	What it is
step1_track_box.py	Lucas–Kanade box tracker → box_track.json
step2_prep.py	Background plate + sharpness table → plate.png, sharp.npy
step3_compose.py	The compositor — produces the figure
render_options.sh	Renders four variants (7 or 9 poses, two opacity levels)
RB-Y1 design notes.md	Design rationale and measurements

Getting the video

The source video is not included — it is 161 MB, well past the gist limit. Everything here needs it. It is on YouTube:

https://www.youtube.com/watch?v=4xJldqOGSdQ

Easiest is to download it manually at 1080p (any of the browser extensions or online downloaders will do) and save it beside the scripts as 20260811_174657.mp4.

If you would rather do it from the shell, yt-dlp works. Format 137 is the 1080p video-only stream; it returns HTTP 403 on its own, so pair it with an audio stream and let yt-dlp merge — the scripts ignore the audio track:

yt-dlp -f "137+140/bv*[height=1080]+ba/b[height<=1080]" \
       --merge-output-format mp4 \
       -o "20260811_174657.%(ext)s" \
       "https://www.youtube.com/watch?v=4xJldqOGSdQ"

Use a current yt-dlp — older versions silently fall back to 360p, at which point the figure still renders but comes out around 380 x 290 and the lidar clean-up stops firing.

A note on fidelity: YouTube re-encodes, so a download does not reproduce the figures bit-for-bit. Frame count and nominal frame rate differ slightly from the original (2623 @ 30 fps vs 2622 @ 29.99), which shifts the chosen poses by up to ~0.15 s and the auto-crop by a pixel. The result is visually equivalent — the sweep, the box trail and the lidar removal all come through — but if you need the exact PNGs, ask me for the original file.

Running

pip install opencv-python-headless numpy
chmod +x render_options.sh          # gists do not preserve the executable bit
./render_options.sh

That tracks the box, builds the background plate and sharpness table, and writes sweep_p{7,9}_{low,med}.png. First run takes about three minutes; afterwards the derived files are cached and re-rendering is seconds.

One variant at a time, with your own opacities:

python3 step3_compose.py <poses> <out.png> <arm_lo> <arm_hi> <box_lo> <box_hi> <arm_final> <box_final>
python3 step3_compose.py 9 mine.png 0.30 0.42 0.80 0.92 0.65 0.97

The scripts expect the video beside them as 20260811_174657.mp4; override with SWEEP_VIDEO=/path/to.mp4.

The four variants

All 1152 x 862 from the original file (a pixel shorter from a YouTube copy), cropped to content, from the same 58.2–66.5 s segment.

Output	Poses	Arm α (sweep → final)	Box α (sweep → final)
sweep_p7_low.png	7	0.20 → 0.30, 0.52	0.72 → 0.86, 0.94
sweep_p7_med.png	7	0.30 → 0.42, 0.65	0.80 → 0.92, 0.97
sweep_p9_low.png	9	0.20 → 0.30, 0.52	0.72 → 0.86, 0.94
sweep_p9_med.png	9	0.30 → 0.42, 0.65	0.80 → 0.92, 0.97

7 poses keeps the box positions discrete with table visible between them; 9 reads as a more continuous swept region. Low keeps the arm sweep a soft backdrop with the box dominant; medium makes individual arm links traceable.

How it works

1. Track the box with pyramidal Lucas–Kanade — median flow of features inside the box, forward/backward consistency checks, reseeding when features drop off.
2. Segment the trajectory by speed to isolate the horizontal transport (58.2–66.5 s); the lift before and the set-down after are excluded.
3. Pick poses evenly spaced along the box path rather than in time, each the sharpest frame within ±3 frames, which rejects motion blur.
4. Split each pose into cardboard (by HSV colour) and arm (everything else in the motion mask), so the white gripper stays with the arm rather than being drawn at the box's opacity.
5. Composite in temporal order over a median background plate, uniform opacity per object per pose.

The camera is static to well under a pixel, so this happens in raw image coordinates with no stabilisation.

Adapting this

Most of it is generic, but these are specific to this footage and would need re-deriving for another clip — all in step3_compose.py unless noted:

• T0, T1 = 58.2, 66.5 — the segment, found from the speed profile of the track.
• The seed box SEED_BOX_720 and window in step1_track_box.py.
• The cardboard HSV range (H 7–20, S 85–185, V 60–180) separating box from arm.
• The lidar-module footprint painted out of each frame, and the x > 1150 cutoff that keeps passers-by out of the crop.

Pixel constants are authored at 720p and scaled by K = width/1280, so the code runs at either resolution unchanged.

RB-Y1 design notes.md

Design notes — box-transport sweep figure

Rationale, measurements, and dead ends behind sweep_p9_med.png. Written so the figure can be re-tuned later without rediscovering all this.

Reference style

Modelled on the reference figures supplied for this task (degrees_of_freedom.pdf, control_subordinate_arms.pdf, eef_transform_fixed.pdf, since removed from this directory): dense semi-transparent ghosts of the whole arm, endpoints not dramatically stronger than the sweep, and no overlaid path graphics (an early version had a trajectory polyline with sample dots — dropped).

Segment selection

The box track was segmented into phases by speed in image coordinates:

Phase	Time	Motion
lift	51–56 s	+138 px vertical, +83 px horizontal — mostly vertical
still	56–58.2 s	—
transport	58.2–66.5 s	−428 px horizontal, −81 px vertical
still	66.5–72 s	—
set down	72.5–74 s	small, mostly vertical

Only the transport phase is in the figure, per the request for the horizontal part. Values are 720p pixels; multiply by 1.5 for the 1080p source.

The camera is static — an early mistake worth recording

The first attempt estimated camera motion by chaining frame-to-frame affine transforms. Drift accumulated to ~1000 px over the video, which looked like a panning camera and produced a garbage "stabilised" trajectory and a smeared background plate. Direct SIFT matching between frames shows median displacement of 0–2 px across 48–83.5 s — a tripod. Everything now works in raw image coordinates with no warping. If motion ever needs re-measuring, match directly against a reference frame; do not chain.

Compositing decisions

Pose spacing is by arc length along the box path, not by time. The box decelerates into place, so equal time spacing clumps poses at the end.

Each pose uses the sharpest frame within ±3 frames (Laplacian variance near the box), which rejects motion-blurred frames during acceleration. A mild unsharp mask is applied per layer.

Opacity is uniform per object per pose. An earlier version varied alpha spatially within a frame — the gripper fell inside the box ellipse so it drew at box opacity, making the wrist visibly sharper than the elbow. There is now no spatial variation within a frame.

The arm stays in front of the box, in natural temporal order. Drawing box layers on top of arm ghosts was tried and rejected: it looks wrong. The box is kept legible instead by lowering arm opacity and by the split below.

The box/arm split is by colour, not geometry. Cardboard is HSV H 7–20, S 85–185, V 60–180 within a locality ellipse; everything else in the motion mask is "arm". The white/silver wrist and gripper fail the saturation test and stay with the arm, which is the whole point — an earlier geometric (elliptical) split captured the wrist and caused the uneven-opacity problem above.

Background is the median plate, not the first frame. When the first frame was the canvas, the start pose was opaque by construction, giving a jarring step from sweep to final state. Every pose is now a ghost over an empty scene, and the final pose is 0.52–0.65 rather than 1.0.

The lidar module

The base-mounted sensor sits exactly where the box trail passes once the base parks (~63 s onward), so ghosts painted it over the cardboard. Diagnosis order mattered here: forcing the box to full opacity did not remove it, proving it was drawn over the box rather than bleeding through from the plate.

Fix: fill the sensor's footprint with the surrounding base colour (median of dark pixels in a ring, ~BGR 39/35/37) in the plate and in each frame, before masking, so the fill propagates consistently.

Two refinements were required:

• Size. Covering the whole mounting plate put a black blob over the arm in several poses. Only the finned sensor with the gold lens is filled; the white mounting plate stays and reads as ordinary hardware.
• Gating. NCC ≥ 0.40 against a template from the 66.46 s frame, measured over the sensor's own footprint. Gating on the whole module was too permissive — 0.45–0.53 even when the arm occluded the sensor, so it painted the arm. Sensor-only NCC separates cleanly: 0.19 / 0.28 occluded versus 0.46 / 0.59 / 1.00 visible.

Residual: in two poses the arm partially overlaps the sensor while it is still visible, so the fill grazes the arm edge. Not noticeable at these opacities.

Rejected approaches

• Masked-median plate (exclude moving pixels rather than out-vote them) to remove the robot base from the background. Doesn't work: the base never leaves that area in the usable window, so 130k pixels had no clean sample and fell back to the plain median. The base residue under the sweep is accepted.
• Per-frame template tracking of the lidar. Unreliable — occluded by the box in most poses (best scores 0.35–0.48, landing on cardboard). A fixed footprint plus a visibility gate is correct because the base is parked.
• Box drawn at every pose while arms were dense (19). The box overlaps ~90% of its own width at that spacing and smears into a solid cardboard band.

Resolution

Authored against a 720p copy, then moved to the 1080p source — same take, identical 2622 frames and 87.43 s duration. All pixel constants derive from K = width/1280, so the code works at either resolution.

The track was initially computed at 720p and scaled by 1.5. It is now tracked natively at 1080p, so the pipeline needs only the one video. Two consequences:

• The tracker seeds at t = 64 s and runs both ways, so it buffers frames from the window start up to the seed (~420 MB) and streams forward from there. Memory is bounded by the tracked window (57–69 s), not the video length — the old version preloaded all 2622 frames (~5.4 GB at 1080p).
• Native tracking shifts a few poses by 0.1–0.25 s versus the scaled track, because pose spacing is by arc length along a slightly different trajectory and the sharpness ROI moves with it. Mean absolute pixel difference against the earlier renders is 0.9–1.1 (7 poses) and 3.3–3.9 (9 poses); the figures are visually equivalent.

Output is 1152 x 862 — roughly 350 dpi in a 3.3-inch column, ~160 dpi across a full two-column width. Native; more pixels would require upscaling.

Other cleanups

• People walking through the right edge are replaced with the background plate, feathered over a 40 px ramp (720p scale), and excluded from the crop bbox.
• The crop is computed from the union of the ghost masks, padded asymmetrically (46 left, 24 right/bottom, 22 top at 720p scale), which drops the empty half of the table and the floor right of the robot.

render_options.sh

#!/usr/bin/env bash
# Renders the four sweep options next to this script.
# Args to step3_compose.py:
#   <poses> <output> <arm_lo> <arm_hi> <box_lo> <box_hi> <arm_final> <box_final>
set -e
cd "$(dirname "$0")"

# step2_prep.py builds the background plate and the sharpness table. They are
# derived files - rebuild them if they are not here (~2 min).
if [ ! -f plate.png ] || [ ! -f sharp.npy ]; then
    echo "cache missing - running step2_prep.py"
    python3 step2_prep.py
fi

#            poses  output            arm_lo arm_hi box_lo box_hi arm_fin box_fin
python3 step3_compose.py 7 sweep_p7_low.png 0.20   0.30   0.72   0.86   0.52    0.94
python3 step3_compose.py 7 sweep_p7_med.png 0.30   0.42   0.80   0.92   0.65    0.97
python3 step3_compose.py 9 sweep_p9_low.png 0.20   0.30   0.72   0.86   0.52    0.94
python3 step3_compose.py 9 sweep_p9_med.png 0.30   0.42   0.80   0.92   0.65    0.97

echo "wrote sweep_p{7,9}_{low,med}.png"

step1_track_box.py

"""Track the cardboard box -> box_track.json

Pyramidal Lucas-Kanade on features seeded inside the box: median flow per step,
forward/backward consistency check, reseeding when features drop off.

Runs at whatever resolution the video is (the seed box is authored at 720p and
scaled by K = width/1280). Memory is bounded by the tracked window, not the
video: frames from WIN_T0 up to the seed are buffered so the backward pass can
walk them in reverse, then the forward pass streams with one frame in hand.
"""
import cv2, numpy as np, json, os

HERE = os.path.dirname(os.path.abspath(__file__))
S = HERE + "/"
VID = os.environ.get("SWEEP_VIDEO", os.path.join(HERE, "20260811_174657.mp4"))

WIN_T0, WIN_T1 = 57.0, 69.0      # tracked window; the figure needs 58.2 - 66.5
SEED_T = 64.0                    # hand-placed box, verified against the frame
SEED_BOX_720 = (500, 285, 185, 215)

cap = cv2.VideoCapture(VID)
fps = cap.get(cv2.CAP_PROP_FPS)
W = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
K = W / 1280.0
x, y, w, h = [v * K for v in SEED_BOX_720]
f_lo, f_seed, f_hi = (int(round(t * fps)) for t in (WIN_T0, SEED_T, WIN_T1))
print(f"res {W}x{int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))}  scale {K}  "
      f"frames {f_lo}..{f_hi}  seed {f_seed}")

lk = dict(winSize=(int(31 * K) | 1, int(31 * K) | 1), maxLevel=4,
          criteria=(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 30, 0.01))
MIN_DIST = max(3, int(6 * K))

def seed_points(img, cx, cy):
    m = np.zeros_like(img)
    x0, y0 = int(cx - w / 2), int(cy - h / 2)
    m[max(0, y0):int(y0 + h), max(0, x0):int(x0 + w)] = 255
    return cv2.goodFeaturesToTrack(img, 300, 0.01, MIN_DIST, mask=m)

def step(prev, nxt, p, cx, cy):
    """One LK step. Returns (points, cx, cy) or None if the track is lost."""
    p1, st, _ = cv2.calcOpticalFlowPyrLK(prev, nxt, p, None, **lk)
    p0r, _, _ = cv2.calcOpticalFlowPyrLK(nxt, prev, p1, None, **lk)
    good = (np.linalg.norm(p - p0r, axis=2).ravel() < 1.0 * K) & (st.ravel() == 1)
    if good.sum() < 8:
        return None
    d = np.median((p1[good] - p[good]).reshape(-1, 2), axis=0)
    cx, cy = cx + d[0], cy + d[1]
    p = p1[good].reshape(-1, 1, 2)
    if len(p) < 25:                                   # reseed in the new window
        pn = seed_points(nxt, cx, cy)
        if pn is not None:
            p = pn
    return p, cx, cy

# ---- read the window: buffer up to the seed, then stream ----
cap.set(cv2.CAP_PROP_POS_FRAMES, f_lo)
buf, track = [], {}
for i in range(f_lo, f_hi + 1):
    ok, fr = cap.read()
    if not ok:
        break
    g = cv2.cvtColor(fr, cv2.COLOR_BGR2GRAY)
    if i <= f_seed:
        buf.append((i, g))
    else:
        if i == f_seed + 1:                           # switch to streaming
            prev_i, prev_g = buf[-1]
            cx, cy = x + w / 2, y + h / 2
            p = seed_points(prev_g, cx, cy)
        r = step(prev_g, g, p, cx, cy)
        if r is None:
            print(f"forward track lost at frame {i}")
            break
        p, cx, cy = r
        track[i] = (cx, cy)
        prev_g = g
print(f"buffered {len(buf)} frames ({(len(buf) * buf[0][1].nbytes) >> 20} MB), "
      f"streamed {len(track)}")

# ---- backward pass over the buffer ----
cx, cy = x + w / 2, y + h / 2
track[f_seed] = (cx, cy)
p = seed_points(buf[-1][1], cx, cy)
for k in range(len(buf) - 1, 0, -1):
    r = step(buf[k][1], buf[k - 1][1], p, cx, cy)
    if r is None:
        print(f"backward track lost at frame {buf[k - 1][0]}")
        break
    p, cx, cy = r
    track[buf[k - 1][0]] = (cx, cy)

data = [[int(i), float(track[i][0]), float(track[i][1])] for i in sorted(track)]
json.dump(data, open(S + "box_track.json", "w"))
print(f"tracked {len(data)} frames, {data[0][0]} -> {data[-1][0]}")
for k in range(0, len(data), 30):
    i, cx, cy = data[k]
    print(f"t={i / fps:6.2f} x={cx:7.1f} y={cy:7.1f}")

step2_prep.py

"""Background plate + sharpness table -> plate.png, sharp.npy

The plate is a temporal median of the scene from 44 s on (the camera is static
throughout), used as the empty-room background the ghosts are drawn over. The
sharpness table is the Laplacian variance around the box for every frame of the
transport window, so the compositor can pick the crispest frame per pose.
"""
import cv2, numpy as np, json, os

HERE = os.path.dirname(os.path.abspath(__file__))
S = HERE + "/"
VID = os.environ.get("SWEEP_VIDEO", os.path.join(HERE, "20260811_174657.mp4"))

cap = cv2.VideoCapture(VID)
fps = cap.get(cv2.CAP_PROP_FPS)
W = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)); H = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
N = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
K = W / 1280.0
trk = {i: (x, y) for i, x, y in json.load(open(S + "box_track.json"))}
print("res", W, H, "fps", fps, "N", N, "scale", K)

# ---- median background plate, in row chunks to bound memory ----
idx = set(range(int(44 * fps), N, 6))
buf = []
cap.set(cv2.CAP_PROP_POS_FRAMES, min(idx)); i = min(idx)
while True:
    ok, f = cap.read()
    if not ok: break
    if i in idx: buf.append(f)
    i += 1
print("plate frames", len(buf))
plate = np.zeros((H, W, 3), np.uint8)
CH = 90
for r in range(0, H, CH):
    sl = np.array([b[r:r + CH] for b in buf], np.uint8)
    plate[r:r + CH] = np.median(sl, axis=0).astype(np.uint8)
del buf
cv2.imwrite(S + "plate.png", plate)

# ---- per-frame sharpness over the transport window ----
f0, f1 = int(58.2 * fps), int(66.5 * fps)
cap.set(cv2.CAP_PROP_POS_FRAMES, f0)
out = []
R = int(200 * K)
for i in range(f0, f1 + 1):
    ok, fr = cap.read()
    if not ok: break
    cx, cy = trk[i]
    x0, x1 = int(max(0, cx - R)), int(min(W, cx + R))
    y0, y1 = int(max(0, cy - R)), int(min(H, cy + R))
    g = cv2.cvtColor(fr[y0:y1, x0:x1], cv2.COLOR_BGR2GRAY)
    out.append((i, cv2.Laplacian(g, cv2.CV_64F).var()))
np.save(S + "sharp.npy", np.array(out))
print("sharpness frames", len(out))

step3_compose.py

import cv2, numpy as np, json, sys
import os
HERE = os.path.dirname(os.path.abspath(__file__))
S = HERE + "/"
VID = os.environ.get("SWEEP_VIDEO", os.path.join(HERE, "20260811_174657.mp4"))

cap = cv2.VideoCapture(VID); fps = cap.get(cv2.CAP_PROP_FPS)
W = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)); H = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
K = W / 1280.0                                   # everything below is authored at 720p scale
plate = cv2.imread(S + "plate.png")
trk = {i: (x, y) for i, x, y in json.load(open(S + "box_track.json"))}
sharp = dict(np.load(S + "sharp.npy"))
odd = lambda v: int(round(v * K)) | 1
px = lambda v: int(round(v * K))

T0, T1 = 58.2, 66.5
f0, f1 = int(T0 * fps), int(T1 * fps)
NG      = int(sys.argv[1]);   OUT     = sys.argv[2]
ARM_LO  = float(sys.argv[3]); ARM_HI  = float(sys.argv[4])
BOX_LO  = float(sys.argv[5]); BOX_HI  = float(sys.argv[6])
FIN_ARM = float(sys.argv[7]); FIN_BOX = float(sys.argv[8])
DBG     = len(sys.argv) > 9

frames = {}
cap.set(cv2.CAP_PROP_POS_FRAMES, f0)
for i in range(f0, f1 + 1):
    ok, fr = cap.read()
    if not ok: break
    frames[i] = fr

idx = sorted(frames)
P = np.array([trk[i] for i in idx])
dcum = np.r_[0, np.cumsum(np.linalg.norm(np.diff(P, axis=0), axis=1))]
picks = []
for t in np.linspace(0, dcum[-1], NG):
    j = int(np.argmin(np.abs(dcum - t)))
    lo, hi = max(0, j - 3), min(len(idx) - 1, j + 3)
    j = max(range(lo, hi + 1), key=lambda c: sharp.get(idx[c], 0))
    if not picks or idx[j] != picks[-1]: picks.append(idx[j])
print("poses:", [round(i / fps, 2) for i in picks])

def fg_mask(img, cx, cy):
    d = cv2.absdiff(img.astype(np.int16), plate.astype(np.int16)).astype(np.uint8)
    g = cv2.GaussianBlur(cv2.cvtColor(d, cv2.COLOR_BGR2GRAY), (odd(5), odd(5)), 0)
    m = ((g > 20).astype(np.uint8)) * 255
    m = cv2.morphologyEx(m, cv2.MORPH_CLOSE, np.ones((px(15), px(15)), np.uint8))
    m = cv2.morphologyEx(m, cv2.MORPH_OPEN, np.ones((px(5), px(5)), np.uint8))
    n, lab, st, cent = cv2.connectedComponentsWithStats(m, 8)
    keep = np.zeros_like(m)
    for k in range(1, n):
        x, y, w, h, a = st[k]
        if a < 1200 * K * K or x > px(1150): continue
        if abs(cent[k][0] - cx) > px(760) or abs(cent[k][1] - cy) > px(520): continue
        keep[lab == k] = 255
    return cv2.morphologyEx(keep, cv2.MORPH_CLOSE, np.ones((px(25), px(25)), np.uint8))

def cardboard(img, cx, cy, fg):
    """the box itself, by colour - the white/silver wrist and gripper stay with the arm"""
    hsv = cv2.cvtColor(cv2.GaussianBlur(img, (odd(5), odd(5)), 0),
                       cv2.COLOR_BGR2HSV).astype(np.int16)
    Hh, Sa, V = hsv[..., 0], hsv[..., 1], hsv[..., 2]
    c = ((Hh >= 7) & (Hh <= 20) & (Sa >= 85) & (Sa <= 185) &
         (V >= 60) & (V <= 180)).astype(np.uint8) * 255
    loc = np.zeros_like(c)
    cv2.ellipse(loc, (int(cx), int(cy)), (px(190), px(180)), 0, 0, 360, 255, -1)
    m = cv2.bitwise_and(cv2.bitwise_and(c, loc), fg)
    m = cv2.morphologyEx(m, cv2.MORPH_CLOSE, np.ones((px(11), px(11)), np.uint8))
    return cv2.morphologyEx(m, cv2.MORPH_OPEN, np.ones((px(5), px(5)), np.uint8))

# --- paint the base-mounted lidar module out at source ---------------------
# it is rigidly mounted and lands on the box trail; where it is visible we fill
# its footprint with the surrounding base colour before any masking happens
LID = dict(c=(px(781), px(403)), ax=(px(19), px(18)), ang=0)
_tf = frames[max(frames)] if frames else None
_TX0, _TY0, _TX1, _TY1 = px(765), px(388), px(799), px(420)   # sensor only
_tmpl = cv2.cvtColor(_tf[_TY0:_TY1, _TX0:_TX1], cv2.COLOR_BGR2GRAY).astype(np.float32)

def _ncc(a, b):
    a = a - a.mean(); b = b - b.mean()
    return float((a * b).sum() / (np.sqrt((a * a).sum() * (b * b).sum()) + 1e-9))

lid_m = np.zeros((H, W), np.float32)
cv2.ellipse(lid_m, LID["c"], LID["ax"], LID["ang"], 0, 360, 1.0, -1)
lid_m = cv2.GaussianBlur(lid_m, (odd(11), odd(11)), 0)[..., None]
_ring = np.zeros((H, W), np.uint8)
cv2.ellipse(_ring, LID["c"], (LID["ax"][0] * 3, LID["ax"][1] * 3), LID["ang"], 0, 360, 255, -1)
cv2.ellipse(_ring, LID["c"], (LID["ax"][0] + px(6), LID["ax"][1] + px(6)), LID["ang"], 0, 360, 0, -1)

def hide_lidar(img, force=False):
    g = cv2.cvtColor(img[_TY0:_TY1, _TX0:_TX1], cv2.COLOR_BGR2GRAY).astype(np.float32)
    if not force and _ncc(g, _tmpl) < 0.40:
        return img                                   # occluded here - nothing to hide
    hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
    sel = (_ring > 0) & (hsv[..., 2] < 90)
    col = np.median(img[sel], axis=0) if sel.sum() > 500 else np.array([39., 35., 37.])
    return (img * (1 - lid_m) + col[None, None, :] * lid_m).astype(np.uint8)

plate = hide_lidar(plate, force=True)
frames = {i: hide_lidar(f) for i, f in frames.items()}

def crisp(img):
    b = cv2.GaussianBlur(img, (0, 0), 1.6 * K)
    return np.clip(cv2.addWeighted(img, 1.45, b, -0.45, 0), 0, 255)

soft = lambda m: cv2.GaussianBlur(m, (odd(7), odd(7)), 0).astype(np.float32) / 255.0
layers = []
for n, i in enumerate(picks):
    cx, cy = trk[i]
    fg = fg_mask(frames[i], cx, cy)
    bx = cardboard(frames[i], cx, cy, fg)
    arm = cv2.bitwise_and(fg, cv2.bitwise_not(bx))
    if DBG:
        v = frames[i].copy(); v[bx > 0] = (0, 0, 255)
        cv2.imwrite(S + f"hd_boxmask_{n}.png", v[::2, ::2])
    layers.append((crisp(frames[i].astype(np.float32)), soft(arm), soft(bx)))

last = len(layers) - 1
canvas = plate.astype(np.float32)
for k, (img, marm, mbox) in enumerate(layers):
    frac = k / last
    a_arm = ARM_LO + (ARM_HI - ARM_LO) * frac ** 1.2
    a_box = BOX_LO + (BOX_HI - BOX_LO) * frac ** 1.2
    if k == last: a_arm, a_box = FIN_ARM, FIN_BOX
    A = np.clip(marm * a_arm + mbox * a_box, 0, 1)[..., None]
    canvas = canvas * (1 - A) + img * A

union = np.max([l[1] + l[2] for l in layers], axis=0); union[:, px(1150):] = 0
ys, xs = np.where(union > 0.15)
x0, x1c = max(0, xs.min() - px(46)), min(W, xs.max() + px(24))
y0, y1c = max(0, ys.min() - px(22)), min(H, ys.max() + px(24))
out = np.clip(canvas, 0, 255).astype(np.uint8)
ramp = np.zeros((H, W), np.float32)
ramp[:, px(1195):] = 1.0
ramp[:, px(1155):px(1195)] = np.linspace(0, 1, px(1195) - px(1155))[None, :]
out = np.clip(out * (1 - ramp[..., None]) + plate.astype(np.float32) * ramp[..., None],
              0, 255).astype(np.uint8)[y0:y1c, x0:x1c]
cv2.imwrite(S + OUT, out)
print("wrote", OUT, out.shape[1], "x", out.shape[0])