PgDog: prepared statements cache retains gigabytes of RSS after clients disconnect (repro)

README.md

PgDog: prepared statements cache retains gigabytes after clients disconnect

Reproduces a memory retention issue in PgDog's global prepared statements cache (observed on v0.1.50, mechanism present on main).

Mechanism

Long-lived clients that keep preparing unique SQL statements (report jobs, ORMs interpolating values into SQL, cron batches) grow the global cache (frontend/prepared_statements/global_cache.rs) to millions of entries. Entries are only freed after the client closes the statement or disconnects — that part works. The problem:

1. HashMap/HashSet never return capacity to the allocator. After a spike of N entries the three tables (statements, names, unused) keep ~2N slots forever: for 1M statements that is ~0.8 GiB of empty buckets at 100 live entries, and it scales linearly from there.
2. The prepared_statements_memory_used metric sums live entries only (MemoryUsage for HashMap iterates), so it reports ~0 while RSS holds gigabytes.

Reproduction

Requires docker compose and python3 with psycopg[binary].

docker compose up -d
pip install 'psycopg[binary]'

# 1M unique statements, ~7 minutes at ~2.5k statements/s
python3 loadgen.py --host 127.0.0.1 --port 6432 --conns 16 \
  --total-qps 100000 --unique-fraction 1.0 --hold-seconds 60

Watch during and after the run:

watch -n 5 'curl -s localhost:9090/metrics | grep -E "^prepared_statements"; \
  docker stats pgdog --no-stream --format "RSS {{.MemUsage}}"'

Observed (v0.1.50, 1M unique statements, then all clients disconnect)

phase	prepared_statements	memory_used	container RSS
load peak	1 000 015	416 MiB	1 725 MiB
clients disconnected	100	43 KiB	1 462 MiB
10 minutes later, stable	100	43 KiB	971 MiB

The maintenance sweep correctly trims entries to prepared_statements_limit, but ~1 GiB of RSS never recovers (a fresh instance idles at ~25 MiB), and the metric claims the cache is empty. The retention scales with the spike size: the same run with 4.7M unique statements leaves a 4.8 GiB plateau.

Notes

• The loadgen.py client mimics drivers that never send Close for prepared statements while the connection lives (psycopg with prepared_max effectively unbounded).

docker-compose.yml

services:
  postgres:
    image: postgres:18
    environment:
      POSTGRES_USER: pgdog
      POSTGRES_PASSWORD: pgdog
      POSTGRES_DB: pgdog

  pgdog:
    image: ghcr.io/pgdogdev/pgdog:0.1.50
    container_name: pgdog
    command: ["pgdog", "--config", "/etc/pgdog/pgdog.toml", "--users", "/etc/pgdog/users.toml"]
    volumes:
      - ./pgdog.toml:/etc/pgdog/pgdog.toml:ro
      - ./users.toml:/etc/pgdog/users.toml:ro
    ports:
      - "6432:6432"
      - "9090:9090"
    depends_on:
      - postgres

loadgen.py

#!/usr/bin/env python3
"""Reproduce PgDog prepared statements cache memory retention.

Long-lived connections prepare unique statements (Parse without Close),
mimicking drivers/apps that never deallocate prepared statements while
the connection lives.
"""
import argparse
import random
import string
import threading
import time

import psycopg

ap = argparse.ArgumentParser()
ap.add_argument("--host", required=True)
ap.add_argument("--port", type=int, default=6432)
ap.add_argument("--db", default="pgdog")
ap.add_argument("--user", default="pgdog")
ap.add_argument("--password", default="pgdog")
ap.add_argument("--conns", type=int, default=8)
ap.add_argument("--total-qps", type=int, default=2500)
ap.add_argument("--unique-fraction", type=float, default=1.0,
                help="fraction of queries with a unique SQL text")
ap.add_argument("--target-unique", type=int, default=1_000_000)
ap.add_argument("--pad-bytes", type=int, default=260,
                help="approximate SQL text size")
ap.add_argument("--hold-seconds", type=int, default=300,
                help="keep connections open after reaching the target")
args = ap.parse_args()

stop = threading.Event()
counters_lock = threading.Lock()
unique_total = 0
exec_total = 0


def pad(n):
    return "".join(random.choices(string.ascii_lowercase, k=n))


def worker(idx):
    global unique_total, exec_total
    conn = psycopg.connect(
        host=args.host, port=args.port, dbname=args.db,
        user=args.user, password=args.password, autocommit=True,
        prepare_threshold=0,
    )
    # never deallocate prepared statements
    conn.prepared_max = 100_000_000
    qps = args.total_qps / args.conns
    interval = 1.0 / qps
    recent = []
    seq = 0
    next_at = time.monotonic()
    while not stop.is_set():
        with counters_lock:
            done = unique_total >= args.target_unique
        make_unique = not done and (random.random() < args.unique_fraction or not recent)
        if make_unique:
            seq += 1
            sql = (f"SELECT {seq} AS n, '{pad(args.pad_bytes - 60)}' AS p "
                   f"-- c{idx}s{seq}")
            recent.append(sql)
            if len(recent) > 500:
                recent = recent[-250:]
        else:
            sql = random.choice(recent)
        try:
            conn.execute(sql, prepare=True)
        except psycopg.Error as e:
            print(f"[conn{idx}] error: {e}", flush=True)
            time.sleep(1)
            continue
        with counters_lock:
            exec_total += 1
            if make_unique:
                unique_total += 1
        next_at += interval
        delay = next_at - time.monotonic()
        if delay > 0:
            time.sleep(delay)
        else:
            next_at = time.monotonic()
    conn.close()


threads = [threading.Thread(target=worker, args=(i,), daemon=True)
           for i in range(args.conns)]
start = time.time()
for t in threads:
    t.start()

held_since = None
try:
    while True:
        time.sleep(10)
        with counters_lock:
            u, e = unique_total, exec_total
        elapsed = time.time() - start
        print(f"[{elapsed:7.0f}s] unique={u} exec={e} qps={e/elapsed:.0f}",
              flush=True)
        if u >= args.target_unique:
            if held_since is None:
                held_since = time.time()
                print(f"target reached: {u} unique statements; holding "
                      f"connections for {args.hold_seconds}s", flush=True)
            elif time.time() - held_since >= args.hold_seconds:
                break
except KeyboardInterrupt:
    pass

print("disconnecting all clients", flush=True)
stop.set()
for t in threads:
    t.join(timeout=15)
print("done", flush=True)

pgdog.toml

[general]
# metrics endpoint to observe the cache (disabled by default)
openmetrics_port = 9090
# default is unlimited; a small limit shows the leak survives the
# maintenance sweep that trims entries down to it
prepared_statements_limit = 100

[[databases]]
name = "pgdog"
host = "postgres"
port = 5432

users.toml

[[users]]
name = "pgdog"
database = "pgdog"
password = "pgdog"