Skip to content

Instantly share code, notes, and snippets.

@matej21
Created October 22, 2026 20:48
Show Gist options
  • Select an option

  • Save matej21/84c1a6d5884e0097cf65c413a3cda3fe to your computer and use it in GitHub Desktop.

Select an option

Save matej21/84c1a6d5884e0097cf65c413a3cda3fe to your computer and use it in GitHub Desktop.
Aurora PostgreSQL 16.15 / 17.11 / 18.6: idle connections use 2× memory, and new connections use 3× CPU (isolated test report)

Aurora PostgreSQL 16.15 / 17.11 / 18.6: idle connections use 2× memory, and new connections use 3× CPU

This is a report of isolated tests on new, empty Amazon Aurora PostgreSQL clusters, run on 2026-10-22 in eu-west-1.

Summary

The engine release of 2026-09-29 (16.15, 17.11, 18.6) changes the cost of each client connection compared with the previous minor versions:

  • Memory: each idle connection costs about 3.5–4 MiB more.
    • The memory is allocated when the backend starts and stays in use until the backend exits.
    • It does not show in pg_backend_memory_contexts.
  • CPU: setting up a new TLS connection costs about 3× more server CPU (≈ 30 ms instead of ≈ 9 ms).
  • Latency and capacity:
    • Opening a connection takes about 20 ms longer.
    • A db.r8g.xlarge is CPU-bound at about 134 new connections/s, against about 390/s on 18.4.

RDS for PostgreSQL and community PostgreSQL of the same major versions do not show the memory increase.

Setup

Each cluster had:

  • One writer instance, db.r8g.xlarge (4 vCPU, 32 GiB), unless stated otherwise.
  • Default cluster and instance parameter groups (default.aurora-postgresqlNN).
  • Storage encryption on, Enhanced Monitoring at 1–5 s, everything else default.

Settings that affect memory were the same on every Aurora version:

  • max_connections = 3449
  • max_worker_processes = 8
  • max_wal_senders = 20
  • shared_buffers = 2681446 (8 kB pages)
  • huge_pages = on

All client connections:

  • use TLS (sslmode=require);
  • connect as the master user to the postgres database;
  • run only SELECT 1.

Test 1: memory per idle connection

Method (memtest.ts):

  1. Measure CloudWatch FreeableMemory for 4 minutes.
  2. Open 300 connections and run SELECT 1 on each one once a minute.
  3. Hold them for 8 minutes.
  4. Close them and measure for another 4 minutes.

The result is (average before − average during) / 300. Repeated runs agreed within ±0.3 MiB.

Engine MiB per idle connection
Aurora 14.17 2.7
Aurora 16.8 3.0
Aurora 16.15 6.0
Aurora 17.4 3.1
Aurora 17.5 3.3
Aurora 17.6 3.4
Aurora 17.7 3.4
Aurora 17.9 3.5
Aurora 17.10 3.4
Aurora 17.11 6.9
Aurora 18.4 3.4–3.6
Aurora 18.6 7.3
RDS for PostgreSQL 16.15 (db.r8g.large, default parameter group) 2.0
RDS for PostgreSQL 17.11 (db.r8g.large, default parameter group) 2.0
RDS for PostgreSQL 18.6 (db.r8g.large, default parameter group) 2.1
Community PostgreSQL 16.9 / 17.8 / 18.6 (Docker, huge_pages=off, delta of cgroup anon) 1.6 / 1.5 / 1.7

Settings that make no difference

Change MiB per connection
Aurora 16.15 with rds.enable_memory_management = 0 6.2
Aurora 18.6, ≈ 3,750 backend slots (max_connections 3470, max_worker_processes 150, max_wal_senders 110) 7.3
Aurora 18.6, ≈ 1,050 backend slots (max_connections 1000, max_worker_processes 20, max_wal_senders 10) 7.1

The extra memory is outside PostgreSQL memory contexts

These are the pg_backend_memory_contexts totals of a fresh backend:

Version Memory contexts Cost per connection (Test 1)
17.10 2,545 kB 3.4 MiB
17.11 2,617 kB 6.9 MiB
18.4 3,231 kB 3.4–3.6 MiB
18.6 3,303 kB 7.3 MiB

The contexts are almost the same on the good and the bad versions.

Enhanced Monitoring

16.8 17.11 / 18.6
inactive growth per 300 connections +0.75 GB +1.9 GB
Page tables growth +80–110 MB on all versions

Test 2: memory pressure (Aurora 18.6, db.r6g.xlarge)

The test cluster had 32 GiB of RAM and 8 GiB of swap. It added idle connections in steps of 100. After each step it ran SELECT 1 on every connection.

Connections FreeableMemory Swap used Note
0 7.7 GB 0
600 3.4 GB 0
1,000 0.97 GB 0
1,300 0.3–0.5 GB 1.9 GB swap-in ≈ 0
1,500 0.37 GB 4.2 GB swap-in ≈ 0–80 kB/s
2,000 0.33 GB – all connections alive
2,100 – – one backend crashed; the postmaster terminated all backends (57P02)

What happens to the swapped-out pages

Second run with 1,300 connections:

Phase Swap used Swap-in Latency p50 / p99
Idle, 4 min 1.84 → 1.91 GB 0–28 kB/s –
SELECT 1 on every connection 2.39 GB 1 kB/s 31 / 131 ms (client outside the VPC)
Catalog query (pg_class ⋈ pg_attribute …) on every connection 3.30 GB 13 kB/s 39 / 271 ms
Close 500 connections 3.38 → 1.21 GB short 3.3 MB/s, then 0 –

Results:

  • Backends that run queries do not page the swapped memory back in.
  • Closing backends frees their swapped pages: about 4.3 MiB per closed connection.

Test 3: connection setup latency and CPU (18.4 vs 18.6)

Setup (conn.ts):

  • Two new clusters, 18.4 and 18.6, both db.r8g.xlarge.
  • The client was a pod in the same VPC. Query round-trip time on an open connection: 0.5 ms on both.
  • Sequential connects alternated between the two clusters, 8 × 100 each.
  • Storms ran for 60 s each, one cluster at a time.
  • CPU numbers come from Enhanced Monitoring (1 s).
18.4 18.6
TLS connect + first query, sequential, n=800 (p50 / p90 / p99) 13.3 / 14.6 / 16.6 ms 33.1 / 34.9 / 37.1 ms
Storm, 20 concurrent clients 378 conn/s, CPU avg 81 % 134 conn/s, CPU 100 %
Storm, 60 concurrent clients 395 conn/s, CPU avg 89 % 134 conn/s, CPU 100 %
Server CPU per new connection (CPU × 4 vCPU / rate) ≈ 9 ms ≈ 30 ms
Idle CPU 3–4 % 3–4 %

Possibly related release note items (2026-09-29 release)

These items appear in the release notes of 17.11 (and of the 16.15 and 18.6 releases of the same date):

  • "Added a new column, worker_threads_cpu_usage_percent, to the aurora_stat_resource_usage() function. This column reports the CPU usage percentage of a backend process's worker threads."
  • "Fixed an issue that could cause a database instance to restart due to a memory management issue in Aurora Storage connection handling."

Worker threads started in every backend would match all three results:

  • thread stacks or per-thread malloc arenas that are touched once and never used again;
  • higher CPU and latency to set up a connection.

This is an inference. Nothing in the documentation confirms it.

Reproduction

The scripts use Bun and the postgres driver.

  • memtest.ts also needs the AWS CLI. It reads FreeableMemory from CloudWatch.
  • conn.ts only needs network access to both clusters.
// Connection setup latency and maximum connection rate of two clusters, run from a client in the same VPC.
// Server CPU for the printed storm windows comes from Enhanced Monitoring (RDSOSMetrics).
// usage: HOST_A=<endpoint> HOST_B=<endpoint> PGUSER=... PGPASSWORD=... bun conn.ts
import postgres from 'postgres'
const { HOST_A, HOST_B, PGUSER, PGPASSWORD } = process.env
if (!HOST_A || !HOST_B || !PGUSER || !PGPASSWORD) throw new Error('set HOST_A, HOST_B, PGUSER and PGPASSWORD')
const hosts = [HOST_A, HOST_B]
const options = (host: string) => ({ host, user: PGUSER, password: PGPASSWORD, database: 'postgres', ssl: 'require' as const, max: 1, idle_timeout: 0, connect_timeout: 30 })
const log = (...args: unknown[]) => console.log(new Date().toISOString(), ...args)
const percentile = (values: number[], p: number) => [...values].sort((a, b) => a - b)[Math.floor(values.length * p)].toFixed(2)
for (const host of hosts) {
const sql = postgres(options(host))
await sql`select 1`
const times: number[] = []
for (let i = 0; i < 200; i++) {
const start = performance.now()
await sql`select 1`
times.push(performance.now() - start)
}
await sql.end()
log(host, `query round trip p50 ${percentile(times, 0.5)} p99 ${percentile(times, 0.99)} ms`)
}
// alternate between the hosts, so that drift affects both equally
const connectTimes = new Map<string, number[]>(hosts.map(host => [host, []]))
for (let round = 0; round < 8; round++) {
for (const host of hosts) {
for (let i = 0; i < 100; i++) {
const sql = postgres(options(host))
const start = performance.now()
await sql`select 1`
connectTimes.get(host)!.push(performance.now() - start)
await sql.end()
}
}
}
for (const [host, times] of connectTimes) {
log(host, `connect + first query n=${times.length}: p50 ${percentile(times, 0.5)} p90 ${percentile(times, 0.9)} p99 ${percentile(times, 0.99)} ms`)
}
for (const concurrency of [20, 60]) {
for (const host of hosts) {
await Bun.sleep(45_000)
const from = Date.now()
const times: number[] = []
let errors = 0
await Promise.all(
Array.from({ length: concurrency }, async () => {
while (Date.now() - from < 60_000) {
const sql = postgres(options(host))
const start = performance.now()
try {
await sql`select 1`
times.push(performance.now() - start)
} catch {
errors++
}
await sql.end()
}
}),
)
const to = Date.now()
log(host, `storm concurrency=${concurrency} window ${from}-${to} ms: ${times.length} ok, ${errors} errors, ${(times.length / ((to - from) / 1000)).toFixed(0)} conn/s, connect p50 ${percentile(times, 0.5)} p99 ${percentile(times, 0.99)} ms`)
}
}
// Memory per idle connection on an Aurora PostgreSQL instance, measured from CloudWatch FreeableMemory.
// usage: PGHOST=<cluster endpoint> PGUSER=... PGPASSWORD=... INSTANCE=<db instance id> AWS_REGION=... bun memtest.ts [count=300] [holdSeconds=480] [baselineSeconds=240]
import postgres from 'postgres'
import { $ } from 'bun'
import { lookup } from 'dns/promises'
const [count = 300, hold = 480, baseline = 240] = process.argv.slice(2).map(Number)
const { PGHOST, PGUSER, PGPASSWORD, INSTANCE, AWS_REGION } = process.env
if (!PGHOST || !PGUSER || !PGPASSWORD || !INSTANCE || !AWS_REGION) throw new Error('set PGHOST, PGUSER, PGPASSWORD, INSTANCE and AWS_REGION')
// resolve once, so that opening hundreds of connections does not depend on the local resolver
const { address } = await lookup(PGHOST)
const options = { host: address, user: PGUSER, password: PGPASSWORD, database: 'postgres', ssl: 'require' as const, max: 1, idle_timeout: 0, connect_timeout: 30 }
const log = (...args: unknown[]) => console.log(new Date().toISOString().slice(11, 19), ...args)
const freeableGiB = async () => {
const start = new Date(Date.now() - 120_000).toISOString()
const end = new Date().toISOString()
const out = await $`aws cloudwatch get-metric-statistics --region ${AWS_REGION} --namespace AWS/RDS --metric-name FreeableMemory --dimensions Name=DBInstanceIdentifier,Value=${INSTANCE} --start-time ${start} --end-time ${end} --period 60 --statistics Average --query 'sort_by(Datapoints,&Timestamp)[-1].Average' --output text`.text()
return Number(out) / 2 ** 30
}
const sample = async (seconds: number) => {
const values: number[] = []
for (let t = 0; t < seconds; t += 60) {
await Bun.sleep(60_000)
values.push(await freeableGiB())
}
return values
}
const mean = (values: number[]) => values.reduce((sum, value) => sum + value, 0) / values.length
const probe = postgres(options)
const settings = await probe`select name, setting from pg_settings where name in ('server_version', 'max_connections', 'max_worker_processes', 'max_wal_senders', 'shared_buffers', 'huge_pages')`
log(Object.fromEntries(settings.map(row => [row.name, row.setting])))
const [contexts] = await probe`select sum(total_bytes)::bigint as total from pg_backend_memory_contexts`
log('fresh backend memory contexts kB', Math.round(Number(contexts.total) / 1024))
const before = await sample(baseline)
log('before GiB', before.map(value => value.toFixed(3)).join(' '))
const clients = Array.from({ length: count }, () => postgres(options))
for (const sql of clients) await sql`select 1`
log('opened', count)
const during: number[] = []
for (let t = 0; t < hold; t += 60) {
await Bun.sleep(60_000)
await Promise.all(clients.map(sql => sql`select 1`))
during.push(await freeableGiB())
}
log('during GiB', during.map(value => value.toFixed(3)).join(' '))
await Promise.all(clients.map(sql => sql.end()))
const after = await sample(baseline)
log('after GiB', after.map(value => value.toFixed(3)).join(' '))
const held = mean(during.slice(-3))
log(`MiB per connection: vs before ${(((mean(before.slice(-2)) - held) * 1024) / count).toFixed(2)}, vs after ${(((mean(after.slice(-2)) - held) * 1024) / count).toFixed(2)}`)
await probe.end()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment