Skip to content

Instantly share code, notes, and snippets.

@blink1073
blink1073 / pymongo-settings-extra-proposal.md
Last active August 4, 2026 19:32
Proposal: pymongo[settings] extra using pydantic-settings

Proposal: pymongo[settings] extra using pydantic-settings

Motivation

Users configuring MongoClient from environment variables or .env files currently do it by hand: os.environ.get(...), manual type coercion, no validation. This proposes an optional, typed, validated settings layer built on pydantic-settings, shipped as an opt-in extra so it never becomes a hard dependency for users who don't want pydantic.

Proposed API

New module: pymongo/pydantic_settings.py

@blink1073
blink1073 / ty_unused_awaitable_investigation.md
Last active July 30, 2026 00:41
Using ty's unused-awaitable rule to find missing awaits in pymongo

Using ty's unused-awaitable rule to find missing awaits in pymongo

Context

Investigated whether Astral's ty type checker's unused-awaitable rule catches missing await calls that mypy strict misses.

ty isn't a dependency of this repo, but runs with no install step using uvx ty (v0.0.65 as of 2026-07-29).

@blink1073
blink1073 / python-5947-plan.md
Last active July 22, 2026 00:28
PYTHON-5947: Add OpenTelemetry operation and transaction support - plan

PYTHON-5947: OpenTelemetry Operation and Transaction Support

Context

PYTHON-5947 extends the command-span support added in PYTHON-5945 (PR #2946) with:

  1. Operation-level spans — one span per public-API call (find_one, insert_one, aggregate, etc.), with command spans nested as children (including one per retry attempt).
@blink1073
blink1073 / 1-findings.md
Last active June 25, 2026 18:27
mongoproxy 1ms delay benchmark: GIL vs no-GIL Python

mongoproxy Latency Benchmark

Measures the effect of a 1ms proxy delay on find_one latency stability under GIL and no-GIL Python.

Test Setup

  • Host: RHEL 9 x86_64 (EC2 spawn host)
  • MongoDB: 7.0.14, standalone, localhost:27017
  • Driver: pymongo 4.17.0
  • Proxy: mongoproxy (Go 1.24) with --delay-ms 1, localhost:28017
@blink1073
blink1073 / memoized-exploring-pillow.md
Created May 27, 2026 12:58
PYTHON-5745: Consolidate logging and monitoring into a single internal API

Plan: PYTHON-5745 — Consolidate logging and monitoring into a single internal API

Context

The driver currently duplicates code at every telemetry call site: each event (command started/succeeded/failed, connection pool lifecycle, server selection, heartbeats) has two parallel blocks — one calling _debug_log(...) for structured logging and one calling listeners.publish_*() for APM event publishing. This duplication clutters the codebase and will make adding OpenTelemetry spans in PYTHON-5052 very painful (every call site would need a third block).

The solution is a unified telemetry API where a single call handles all telemetry channels simultaneously. A PoC exists in PR #2720 (command events only) using a context manager pattern.

Constraints:

  • No external behavior changes (logging output and APM events must remain identical)
@blink1073
blink1073 / wondrous-chasing-leaf.md
Last active May 26, 2026 22:55
Plan: Remove data files from scikit-image repo and serve from CDN

Plan: Remove Data Files from Repo and Serve from CDN

TL;DR

  1. GitLab data repo — consolidate all files under a canonical directory structure and create a tagged release (v1)
  2. _registry.py — replace the 65-line hardcoded registry_urls dict with {k: _DATA_REPO_BASE + k for k in registry}
  3. meson.build — remove the 31 bundled legacy data files from the wheel
  4. _fetchers.py — drop the legacy bundled-file fallback; add a stdlib urllib fallback for when pooch isn't installed; improve error messages; clean up the GitHub URL logic
  5. git rm — remove all binary data files from the repo (no history rewrite)
  6. tools/download_data.py — new stdlib-only script for Linux distros to pre-download data and set SKIMAGE_DATADIR
@blink1073
blink1073 / binary-jumping-newell.md
Created May 19, 2026 12:29
scikit-image limited ABI (abi3) implementation plan

Plan: Python Limited ABI (abi3) Support for scikit-image

Context

The goal is to compile scikit-image extension modules targeting Python's stable ABI (PEP 384, aka "abi3"), so that a single compiled wheel works across Python 3.12+ without per-version recompilation. This is done via Meson's limited_api keyword argument on py3.extension_module() calls, which:

  • Defines Py_LIMITED_API=0x030C0000 and injects CYTHON_LIMITED_API for Cython sources
  • Names resulting shared libraries module.abi3.so (Linux/macOS) instead of module.cpython-3XX-...so

Prerequisites already met:

  • Meson >= 1.5.0 required (limited_api available since 1.3.0)
import ctypes
import ctypes.wintypes as wintypes
# Load the IP Helper library
IPHLPAPI = ctypes.WinDLL('Iphlpapi.dll')
# Based on the following documentation:
# https://learn.microsoft.com/en-us/windows/win32/api/iphlpapi/nf-iphlpapi-getadaptersaddresses
# https://learn.microsoft.com/en-us/windows/win32/api/iptypes/ns-iptypes-ip_adapter_addresses_lh
# https://learn.microsoft.com/en-us/windows/win32/api/iptypes/ns-iptypes-ip_adapter_dns_server_address_xp
@blink1073
blink1073 / pymongo_greenlet.md
Created June 16, 2023 00:12
Async PyMongo with Greenlet

Async PyMongo with Greenlet

June 12, 2023

Proposal

Use greenletio to add asyncio support in PyMongo, with minimal changes to existing classes.

Background

The reason that Motor is not "truly asynchronous" is that we have blocking calls in PyMongo that do not enable cooperative multitasking. These include acquiring thread locks and socket i/o. Even if we were to use non-blocking versions of these, the event loop would still be blocked.

The previous proposal required rewriting the entirety of PyMongo to be asyncio-friendly, and gave a small performance hit for synchronous code, especially noticeable for small documents.

@blink1073
blink1073 / extension_types.py
Last active January 20, 2023 19:16
Pandas Extension Types for BSON
from __future__ import annotations
from bson import ObjectId, Decimal128, Binary
from pandas.api.extensions import ExtensionDtype, ExtensionArray, register_extension_dtype
from pandas._typing import type_t
import pyarrow as pa
from typing import Union, Any
import numpy as np
import pandas as pd
import numbers