Skip to content

Instantly share code, notes, and snippets.

View blink1073's full-sized avatar

Steven Silvester blink1073

595 followers · 41 following
View GitHub Profile
@blink1073
blink1073 / 1-findings.md
Last active June 25, 2026 18:27
mongoproxy 1ms delay benchmark: GIL vs no-GIL Python

mongoproxy Latency Benchmark

Measures the effect of a 1ms proxy delay on find_one latency stability under GIL and no-GIL Python.

Test Setup

  • Host: RHEL 9 x86_64 (EC2 spawn host)
  • MongoDB: 7.0.14, standalone, localhost:27017
  • Driver: pymongo 4.17.0
  • Proxy: mongoproxy (Go 1.24) with --delay-ms 1, localhost:28017
@blink1073
blink1073 / memoized-exploring-pillow.md
Created May 27, 2026 12:58
PYTHON-5745: Consolidate logging and monitoring into a single internal API

Plan: PYTHON-5745 — Consolidate logging and monitoring into a single internal API

Context

The driver currently duplicates code at every telemetry call site: each event (command started/succeeded/failed, connection pool lifecycle, server selection, heartbeats) has two parallel blocks — one calling _debug_log(...) for structured logging and one calling listeners.publish_*() for APM event publishing. This duplication clutters the codebase and will make adding OpenTelemetry spans in PYTHON-5052 very painful (every call site would need a third block).

The solution is a unified telemetry API where a single call handles all telemetry channels simultaneously. A PoC exists in PR #2720 (command events only) using a context manager pattern.

Constraints:

  • No external behavior changes (logging output and APM events must remain identical)
@blink1073
blink1073 / wondrous-chasing-leaf.md
Last active May 26, 2026 22:55
Plan: Remove data files from scikit-image repo and serve from CDN

Plan: Remove Data Files from Repo and Serve from CDN

TL;DR

  1. GitLab data repo — consolidate all files under a canonical directory structure and create a tagged release (v1)
  2. _registry.py — replace the 65-line hardcoded registry_urls dict with {k: _DATA_REPO_BASE + k for k in registry}
  3. meson.build — remove the 31 bundled legacy data files from the wheel
  4. _fetchers.py — drop the legacy bundled-file fallback; add a stdlib urllib fallback for when pooch isn't installed; improve error messages; clean up the GitHub URL logic
  5. git rm — remove all binary data files from the repo (no history rewrite)
  6. tools/download_data.py — new stdlib-only script for Linux distros to pre-download data and set SKIMAGE_DATADIR
@blink1073
blink1073 / binary-jumping-newell.md
Created May 19, 2026 12:29
scikit-image limited ABI (abi3) implementation plan

Plan: Python Limited ABI (abi3) Support for scikit-image

Context

The goal is to compile scikit-image extension modules targeting Python's stable ABI (PEP 384, aka "abi3"), so that a single compiled wheel works across Python 3.12+ without per-version recompilation. This is done via Meson's limited_api keyword argument on py3.extension_module() calls, which:

  • Defines Py_LIMITED_API=0x030C0000 and injects CYTHON_LIMITED_API for Cython sources
  • Names resulting shared libraries module.abi3.so (Linux/macOS) instead of module.cpython-3XX-...so

Prerequisites already met:

  • Meson >= 1.5.0 required (limited_api available since 1.3.0)
@blink1073
blink1073 / get-dns-namespaces.py
Last active May 2, 2025 11:53
import ctypes
import ctypes.wintypes as wintypes
# Load the IP Helper library
IPHLPAPI = ctypes.WinDLL('Iphlpapi.dll')
# Based on the following documentation:
# https://learn.microsoft.com/en-us/windows/win32/api/iphlpapi/nf-iphlpapi-getadaptersaddresses
# https://learn.microsoft.com/en-us/windows/win32/api/iptypes/ns-iptypes-ip_adapter_addresses_lh
# https://learn.microsoft.com/en-us/windows/win32/api/iptypes/ns-iptypes-ip_adapter_dns_server_address_xp
@blink1073
blink1073 / pymongo_greenlet.md
Created June 16, 2023 00:12
Async PyMongo with Greenlet

Async PyMongo with Greenlet

June 12, 2023

Proposal

Use greenletio to add asyncio support in PyMongo, with minimal changes to existing classes.

Background

The reason that Motor is not "truly asynchronous" is that we have blocking calls in PyMongo that do not enable cooperative multitasking. These include acquiring thread locks and socket i/o. Even if we were to use non-blocking versions of these, the event loop would still be blocked.

The previous proposal required rewriting the entirety of PyMongo to be asyncio-friendly, and gave a small performance hit for synchronous code, especially noticeable for small documents.

@blink1073
blink1073 / extension_types.py
Last active January 20, 2023 19:16
Pandas Extension Types for BSON
from __future__ import annotations
from bson import ObjectId, Decimal128, Binary
from pandas.api.extensions import ExtensionDtype, ExtensionArray, register_extension_dtype
from pandas._typing import type_t
import pyarrow as pa
from typing import Union, Any
import numpy as np
import pandas as pd
import numbers
@blink1073
blink1073 / hatch_comparison_script.py
Last active May 28, 2022 21:47
"""Compare the dist file created by a migrated package to one created by the original."""
import argparse
import glob
import os
import shutil
import subprocess
import sys
import tarfile
import zipfile
@blink1073
blink1073 / LICENSE
Last active October 31, 2025 07:26
Wrap an Asynchronous Class
Copyright (c) 2022 Steven Silvester
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
@blink1073
blink1073 / meta.yml
Last active December 20, 2021 18:39
prototype conda recipe for pymongoarrow
{% set name = "pymongoarrow" %}
{% set version = "0.2.0.dev0" %}
package:
name: {{ name|lower }}
version: {{ version }}
source:
url: file:///home/runner/work/mongo-arrow/mongo-arrow/bindings/python/dist/pymongoarrow-{{ version }}.tar.gz