Skip to content

Instantly share code, notes, and snippets.

@airhorns
Created August 17, 2026 16:07
Show Gist options
  • Select an option

  • Save airhorns/dc676921e02272cc41f8aa15da040e45 to your computer and use it in GitHub Desktop.

Select an option

Save airhorns/dc676921e02272cc41f8aa15da040e45 to your computer and use it in GitHub Desktop.
Ruby 4.0 x86_64-linux: Errno::NOERROR from connect(2) in a non-main Ractor with 2+ threads
# frozen_string_literal: true
# Does a Ruby thread inside a non-main Ractor migrate between native threads?
#
# The errno-0 hypothesis rests on it: errno is per-native-thread, so a Ruby thread that issues a
# syscall and resumes on a different native thread would read an unrelated thread's errno. If no
# migration ever happens, that mechanism is wrong and the conclusion needs rebuilding.
#
# /proc/thread-self resolves to <pid>/task/<tid> for the calling *native* thread, which is readable
# from a non-main Ractor (Fiddle.dlopen is not — it raises Ractor::UnsafeError). Linux only.
require "socket"
Warning[:experimental] = false
module Gettid
class << self
def tid
File.readlink("/proc/thread-self").split("/").last.to_i
end
# HELPER=cpu matches the arrangement that actually fails; sleeping is the gentler control.
def helper_loop(running)
if ENV.fetch("HELPER", "sleep") == "cpu"
2000.times { |i| i * i } while running[0]
else
sleep(0.001) while running[0]
end
end
# Sample the native thread id around whatever might preempt a Ruby thread: connect(2), accept(2),
# a read/write ping-pong, and sleep.
def sample(second_thread:)
tids = [tid]
running = [true]
helper = (Thread.new { helper_loop(running) } if second_thread)
server = TCPServer.new("127.0.0.1", 0)
accepter = Thread.new { server.accept }
socket = TCPSocket.new("127.0.0.1", server.addr.fetch(1))
tids << tid
peer = accepter.value
tids << tid
30.times do
socket.write("ping\n")
peer.gets
peer.write("pong\n")
socket.gets
tids << tid
end
sleep 0.01
tids << tid
Ractor.make_shareable(tids.uniq)
ensure
running[0] = false
socket&.close
peer&.close
server&.close
helper&.kill
accepter&.kill
end
def in_ractor(&block)
ractor = Ractor.new(&block)
ractor.respond_to?(:value) ? ractor.value : ractor.take
rescue Ractor::RemoteError => error
cause = error.cause
raise error if cause.nil?
raise cause.class, cause.message, cause.backtrace
end
end
end
rounds = Integer(ARGV[0] || 5)
puts RUBY_DESCRIPTION
puts "RUBY_MAX_CPU=#{ENV.fetch("RUBY_MAX_CPU", "(unset)")} HELPER=#{ENV.fetch("HELPER", "sleep")} " \
"nprocessors=#{`nproc`.strip} main_tid=#{Gettid.tid}"
puts
settings = {
"main Ractor, 1 thread" => -> { Gettid.sample(second_thread: false) },
"main Ractor, 2 threads" => -> { Gettid.sample(second_thread: true) },
"non-main Ractor, 1 thread" => -> { Gettid.in_ractor { Gettid.sample(second_thread: false) } },
"non-main Ractor, 2 threads" => -> { Gettid.in_ractor { Gettid.sample(second_thread: true) } },
}
settings.each do |description, run|
observations = Array.new(rounds) do
run.call
rescue => error
["ERROR #{error.class}: #{error.message}"]
end
migrated = observations.count { |tids| tids.length > 1 }
puts "#{description}: #{migrated}/#{rounds} rounds ran on more than one native thread"
observations.each_with_index { |tids, i| puts " round #{i}: #{tids.length} distinct #{tids.inspect}" }
puts
end

Errno::NOERROR from connect(2), and [BUG] rb_sys_fail_path_in(io_fillbuf) - errno == 0, once a non-main Ractor has more than one Ruby thread

Summary

On x86_64-linux, socket IO in a non-main Ractor fails spuriously as soon as that Ractor contains a second Ruby thread. The connecting thread raises Errno::NOERROR — errno 0 surfaced as a SystemCallError, whose message is literally Success:

Errno::NOERROR: Failed to open TCP connection to 127.0.0.1:38975
                (Success - connect(2) for "127.0.0.1" port 38975)

With a raw TCPSocket instead of Net::HTTP the VM can abort outright:

[BUG] rb_sys_fail_path_in(io_fillbuf, fd:6 ) - errno == 0

reported with Total ractor count: 2 / Ruby thread count for this ractor: 2.

What the second thread does is irrelevant — a thread that only calls sleep, one that only burns CPU, one doing pipe IO, and one doing socket IO all trigger it. Its mere existence in that Ractor is the trigger. Nothing else I varied matters: a Ractor with a single thread never fails, no matter how much socket work it does or how loaded the machine is, and a second thread in the main Ractor never causes it.

This is not a synthetic concern: it made a CI job flaky in roughly 10% of builds, where the "second thread" was nothing more exotic than a stub HTTP server standing in for an upstream service alongside the client under test.

Reproduction

ractor_factors.rb (attached). No gems and no network — every arrangement talks to a stub HTTP server on 127.0.0.1.

ruby ractor_factors.rb interleave 200

Sample the arrangements interleaved, not in blocks. On a shared machine the failure rate swings enormously with host load: run in blocks, the same arrangement scored 196/200 in one build and 0/200 in the next, so blocked counts are not comparable to each other and a 0 says nothing. The interleave mode runs one iteration of each arrangement per round with a rotating order, so every arrangement is sampled across the same load windows. The numbers below come from a single such run — 200 rounds on a shared 16-core x86_64-linux CI host, 1-minute load average 3.1 rising to 5.9.

arrangement threads in the non-main Ractor bad / 200
two client threads in the Ractor 2 145
client + a thread doing pipe IO (no socket) 2 132
client + a thread that only burns CPU 2 89
client + stub-server thread (the original CI failure) 2 49
client + a thread that only sleeps 2 15
client alone in the Ractor, stub server in the main Ractor 1 0
stub server alone in the Ractor (accept(2) inside it), client in main 1 0
both connection ends in the Ractor, single thread (listen, connect, accept, exchange) 1 0
client alone in the Ractor, a CPU-burning thread in the main Ractor 1 0
client + stub-server thread, both in the main Ractor 0
client + a CPU-burning thread, both in the main Ractor 0

The split is exactly "does the non-main Ractor hold two or more Ruby threads". Every failure in that run carried the identical Errno::NOERROR ... connect(2) signature. Raw log in observed-on-ci.txt, including the VM abort's control-frame and threading information.

What it is not

  • Not Net::HTTP specific. Raw TCPSocket fails too, and is the case that aborts the VM.
  • Not "IO in a Ractor is unsupported". A single-threaded non-main Ractor drove the identical request 200 times without a failure, in the same run, in the same load window — including one arrangement that owns the listening socket and calls accept(2) inside the Ractor, and one that owns both ends of the connection.
  • Not the second thread's IO. A neighbour that never touches a file descriptor fails at 89/200.
  • Not CPU contention as such. The same CPU-burning thread placed in the main Ractor instead of the non-main one gives 0/200, in the same run.
  • Not Happy Eyeballs v2. With Socket.tcp_fast_fallback = false the pattern reproduces unchanged: 49, 49, 32, 20, 3 bad out of 66 for the five two-thread arrangements, 0 for all six others.
  • Not M:N native-thread migration — see below.
  • Not a native extension. The reproduction is pure stdlib. I first saw this through a Rust extension's Ruby HTTP transport, but the extension is absent from the script.
  • Not the connection failing for a real reason. The stub server is listening on the port the same iteration just obtained from TCPServer#addr, and connect(2) reports success while raising.

Native-thread migration is ruled out

The obvious guess is that errno, being per-native-thread, is read on a different native thread than the one that set it — Ruby threads in non-main Ractors being scheduled M:N. Migration is directly observable: /proc/thread-self resolves to <pid>/task/<tid> for the calling native thread and is readable from a non-main Ractor (Fiddle.dlopen is not — it raises Ractor::UnsafeError). gettid_migration.rb is attached. Three observations kill the theory:

  • On the x86_64 host where the bug fires constantly, migration was observed 0 of 6 rounds in every setting, with both a sleeping and a CPU-burning neighbour, in the same build as the failures above.
  • On aarch64-linux migration happens readily — 3 of 4 rounds for a two-thread non-main Ractor, tids moving e.g. 16 → 17 and 17 → 18 → 19 — and the bug never reproduces there at all.
  • RUBY_MAX_CPU=1 leaves one native thread to migrate between, so migration is impossible by construction (and observably stops on arm64, where it otherwise happens constantly) — the failure persists anyway, 9/200.

So migration and the failure are, if anything, anti-correlated. Whatever loses errno here, it is not a thread waking on a different native thread.

Architecture

I have only reproduced this on x86_64-linux. On aarch64-linux — official ruby:4.0, byte-identical revision 03b6d3f889 — roughly 1500 exchanges of the failing arrangements are clean, including 300 of the arrangement that fails 89/200 on x86, under 1- and 2-CPU quotas, with competing busy loops, and with 8 neighbour threads. arm64-darwin is clean too. So expect to need an x86_64 Linux host, ideally a loaded one.

Environment

ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [x86_64-linux]
Socket.tcp_fast_fallback = true (default; false reproduces identically)
RUBY_MAX_CPU unset (=1 does not help)
shared CI host, nproc 16, 1-minute load average 3.1-5.9 during the run

Relation to Bug #21195

Bug #21195 is the same shape — errno lost around io_internal_wait, fixed for 3.3 and 3.4 in 2025 — and the errno == 0 assertion text is identical. This is a live path in 4.0.6, reached through ordinary socket connect and read, with or without Happy Eyeballs, so I am filing it separately rather than commenting there. I could not find an existing report for the Ractor variant.

Workaround

Keep every non-main Ractor that does socket IO single-threaded. In my case the fix was to move the stub server out of the Ractor under test and pass only its port across the boundary, leaving one thread in that Ractor. Note that "move the other socket work out" is not sufficient advice — an unrelated thread that never touches an fd is enough to break it.

Ruby 4.0.6 (revision 03b6d3f889) +PRISM [x86_64-linux], shared CI host, nproc 16, 1-minute load
average 3.1 -> 5.9 over the run (per-round values below).
One iteration of every arrangement per round, order rotated each round, so all arrangements are
sampled across the same host-load windows. Blocked sampling on this host is not comparable: the
same arrangement scored 196/200 in one build and 0/200 in the next.
nproc=16 loadavg=3.64 4.58 6.63
==============================================================================
interleaved, tcp_fast_fallback=true (default)
==============================================================================
round 25/200 loadavg=3.13 4.38 6.51 bad={"inner_stub" => 11, "idle_thread" => 1, "cpu_thread" => 11, "pipe_thread" => 18, "two_clients" => 22}
round 50/200 loadavg=3.47 4.35 6.44 bad={"inner_stub" => 18, "idle_thread" => 1, "cpu_thread" => 23, "pipe_thread" => 37, "two_clients" => 43}
round 75/200 loadavg=4.26 4.47 6.42 bad={"inner_stub" => 22, "idle_thread" => 3, "cpu_thread" => 34, "pipe_thread" => 47, "two_clients" => 60}
round 100/200 loadavg=4.86 4.60 6.41 bad={"inner_stub" => 31, "idle_thread" => 9, "cpu_thread" => 48, "pipe_thread" => 61, "two_clients" => 78}
round 125/200 loadavg=5.11 4.68 6.39 bad={"inner_stub" => 34, "idle_thread" => 11, "cpu_thread" => 56, "pipe_thread" => 77, "two_clients" => 93}
round 150/200 loadavg=5.60 4.82 6.39 bad={"inner_stub" => 40, "idle_thread" => 14, "cpu_thread" => 69, "pipe_thread" => 95, "two_clients" => 110}
round 175/200 loadavg=5.91 4.96 6.39 bad={"inner_stub" => 44, "idle_thread" => 14, "cpu_thread" => 82, "pipe_thread" => 113, "two_clients" => 126}
round 200/200 loadavg=5.53 4.95 6.35 bad={"inner_stub" => 49, "idle_thread" => 15, "cpu_thread" => 89, "pipe_thread" => 132, "two_clients" => 145}
main ------ 0 bad / 200 client + stub-server thread, both in the MAIN Ractor
outer_stub ------ 0 bad / 200 client alone in a Ractor, stub-server thread in the MAIN Ractor
inner_stub TSBLE- 49 bad / 200 client + stub-server thread, both inside ONE Ractor (the CI failure)
idle_thread T----- 15 bad / 200 client in a Ractor + a second thread that only sleeps
cpu_thread T----- 89 bad / 200 client in a Ractor + a second thread that only burns CPU
pipe_thread T-B--- 132 bad / 200 client in a Ractor + a second thread doing pipe IO (no socket)
two_clients TSB--- 145 bad / 200 two client threads in a Ractor, stub server in the MAIN Ractor
inner_listener ---L-- 0 bad / 200 stub server accept(2)ing inside a Ractor (sole thread), client in MAIN
raw_serial_inner ---LE- 0 bad / 200 raw, ONE thread in a Ractor: listen, connect, accept, exchange
outer_cpu -----M 0 bad / 200 client alone in a Ractor, a CPU-burning thread in the MAIN Ractor
main_cpu -----M 0 bad / 200 client + a CPU-burning thread, both in the MAIN Ractor
145x two_clients | Errno::NOERROR: Success - Failed to open TCP connection to N.N.N.N:N (Success - connect(N) for "N.N.N.N" p
132x pipe_thread | Errno::NOERROR: Success - Failed to open TCP connection to N.N.N.N:N (Success - connect(N) for "N.N.N.N" p
89x cpu_thread | Errno::NOERROR: Success - Failed to open TCP connection to N.N.N.N:N (Success - connect(N) for "N.N.N.N" p
49x inner_stub | Errno::NOERROR: Success - Failed to open TCP connection to N.N.N.N:N (Success - connect(N) for "N.N.N.N" p
15x idle_thread | Errno::NOERROR: Success - Failed to open TCP connection to N.N.N.N:N (Success - connect(N) for "N.N.N.N" p
==============================================================================
interleaved, tcp_fast_fallback=false
==============================================================================
round 25/66 loadavg=5.41 4.97 6.32 bad={"inner_stub" => 8, "cpu_thread" => 16, "pipe_thread" => 17, "two_clients" => 19}
round 50/66 loadavg=5.33 4.99 6.29 bad={"inner_stub" => 14, "idle_thread" => 2, "cpu_thread" => 28, "pipe_thread" => 35, "two_clients" => 37}
main ------ 0 bad / 66 client + stub-server thread, both in the MAIN Ractor
outer_stub ------ 0 bad / 66 client alone in a Ractor, stub-server thread in the MAIN Ractor
inner_stub TSBLE- 20 bad / 66 client + stub-server thread, both inside ONE Ractor (the CI failure)
idle_thread T----- 3 bad / 66 client in a Ractor + a second thread that only sleeps
cpu_thread T----- 32 bad / 66 client in a Ractor + a second thread that only burns CPU
pipe_thread T-B--- 49 bad / 66 client in a Ractor + a second thread doing pipe IO (no socket)
two_clients TSB--- 49 bad / 66 two client threads in a Ractor, stub server in the MAIN Ractor
inner_listener ---L-- 0 bad / 66 stub server accept(2)ing inside a Ractor (sole thread), client in MAIN
raw_serial_inner ---LE- 0 bad / 66 raw, ONE thread in a Ractor: listen, connect, accept, exchange
outer_cpu -----M 0 bad / 66 client alone in a Ractor, a CPU-burning thread in the MAIN Ractor
main_cpu -----M 0 bad / 66 client + a CPU-burning thread, both in the MAIN Ractor
49x pipe_thread | Errno::NOERROR: Success - Failed to open TCP connection to N.N.N.N:N (Success - connect(N) for "N.N.N.N" p
49x two_clients | Errno::NOERROR: Success - Failed to open TCP connection to N.N.N.N:N (Success - connect(N) for "N.N.N.N" p
32x cpu_thread | Errno::NOERROR: Success - Failed to open TCP connection to N.N.N.N:N (Success - connect(N) for "N.N.N.N" p
20x inner_stub | Errno::NOERROR: Success - Failed to open TCP connection to N.N.N.N:N (Success - connect(N) for "N.N.N.N" p
3x idle_thread | Errno::NOERROR: Success - Failed to open TCP connection to N.N.N.N:N (Success - connect(N) for "N.N.N.N" p
==============================================================================
native thread migration, HELPER=sleep
==============================================================================
RUBY_MAX_CPU=(unset) HELPER=sleep nprocessors=16 main_tid=2988
main Ractor, 1 thread: 0/6 rounds ran on more than one native thread
round 0: 1 distinct [2988]
round 1: 1 distinct [2988]
round 2: 1 distinct [2988]
round 3: 1 distinct [2988]
round 4: 1 distinct [2988]
round 5: 1 distinct [2988]
main Ractor, 2 threads: 0/6 rounds ran on more than one native thread
round 0: 1 distinct [2988]
round 1: 1 distinct [2988]
round 2: 1 distinct [2988]
round 3: 1 distinct [2988]
round 4: 1 distinct [2988]
round 5: 1 distinct [2988]
non-main Ractor, 1 thread: 0/6 rounds ran on more than one native thread
round 0: 1 distinct [3009]
round 1: 1 distinct [3009]
round 2: 1 distinct [3009]
round 3: 1 distinct [3009]
round 4: 1 distinct [3009]
round 5: 1 distinct [3009]
non-main Ractor, 2 threads: 0/6 rounds ran on more than one native thread
round 0: 1 distinct [3009]
round 1: 1 distinct [3009]
round 2: 1 distinct [3009]
round 3: 1 distinct [3009]
round 4: 1 distinct [3009]
round 5: 1 distinct [3009]
==============================================================================
native thread migration, HELPER=cpu
==============================================================================
RUBY_MAX_CPU=(unset) HELPER=cpu nprocessors=16 main_tid=3010
main Ractor, 1 thread: 0/6 rounds ran on more than one native thread
round 0: 1 distinct [3010]
round 1: 1 distinct [3010]
round 2: 1 distinct [3010]
round 3: 1 distinct [3010]
round 4: 1 distinct [3010]
round 5: 1 distinct [3010]
main Ractor, 2 threads: 0/6 rounds ran on more than one native thread
round 0: 1 distinct [3010]
round 1: 1 distinct [3010]
round 2: 1 distinct [3010]
round 3: 1 distinct [3010]
round 4: 1 distinct [3010]
round 5: 1 distinct [3010]
non-main Ractor, 1 thread: 0/6 rounds ran on more than one native thread
round 0: 1 distinct [3031]
round 1: 1 distinct [3031]
round 2: 1 distinct [3031]
round 3: 1 distinct [3031]
round 4: 1 distinct [3031]
round 5: 1 distinct [3031]
non-main Ractor, 2 threads: 0/6 rounds ran on more than one native thread
round 0: 1 distinct [3031]
round 1: 1 distinct [3031]
round 2: 1 distinct [3031]
round 3: 1 distinct [3031]
round 4: 1 distinct [3031]
round 5: 1 distinct [3031]
==============================================================================
VM abort seen in the raw-TCPSocket arrangement (earlier build of the same job)
==============================================================================
ractor-flake-probe.rb below is an earlier harness, not attached; the arrangement is the
raw_inner variant of ractor_factors.rb (client + stub-server thread in one Ractor, raw TCPSocket).
ractor-flake-probe.rb:68: [BUG] rb_sys_fail_path_in(io_fillbuf, fd:6 ) - errno == 0
ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [x86_64-linux]
-- Control frame information -----------------------------------------------
c:0005 p:---- s:0031 e:000030 l:y b:0001 CFUNC :gets
c:0004 p:0093 s:0027 e:000026 l:y b:0001 METHOD ractor-flake-probe.rb:68
c:0003 p:0100 s:0018 e:000017 l:y b:0001 METHOD ractor-flake-probe.rb:81
c:0002 p:0005 s:0007 e:000006 l:n b:---- BLOCK ractor-flake-probe.rb:113 [FINISH]
c:0001 p:---- s:0003 e:000002 l:y b:---- DUMMY [FINISH]
-- Ruby level backtrace information ----------------------------------------
ractor-flake-probe.rb:113:in 'block (2 levels) in <main>'
ractor-flake-probe.rb:81:in 'exercise'
ractor-flake-probe.rb:68:in 'raw_socket_request'
ractor-flake-probe.rb:68:in 'gets'
-- Threading information ---------------------------------------------------
Total ractor count: 2
Ruby thread count for this ractor: 2
# frozen_string_literal: true
# Which property of a non-main Ractor breaks its own socket IO on Ruby 4.0 / x86_64-linux, raising
# Errno::NOERROR ("Success") from connect(2) and sometimes aborting the VM with
# [BUG] rb_sys_fail_path_in(io_fillbuf) - errno == 0?
#
# Answer, measured: the Ractor holding two or more Ruby threads, and nothing else below. Each variant
# varies one factor so the cause is measured rather than assumed, and the letters name the factors:
#
# T a second Ruby thread exists in the non-main Ractor <- the only one that matters
# S that second thread does *socket* IO irrelevant
# B that second thread does blocking IO of any kind irrelevant
# L a listening socket is accept(2)ed inside the non-main Ractor irrelevant
# E both ends of the connection are owned by the non-main Ractor irrelevant
# M a CPU-burning thread runs in the MAIN Ractor instead irrelevant
#
# Failure rates on a shared host swing enormously with load — the same arrangement scored 196/200 and
# 0/200 in consecutive runs — so use the interleaved mode, which samples every arrangement across the
# same load windows and makes their counts comparable. A 0 from a *block* of iterations establishes
# nothing unless a known-bad arrangement scored non-zero in the same window.
#
# usage: ruby ractor_factors.rb [variant|all|interleave] [iterations|rounds]
# NO_HEV2=1 disables Happy Eyeballs (Socket.tcp_fast_fallback); the failure survives it.
require "etc"
require "json"
require "net/http"
require "socket"
Warning[:experimental] = false
module RactorFactors
RESPONSE_BODY = JSON.generate("data" => { "ok" => true }).freeze
REQUEST_BODY = JSON.generate("query" => "mutation { doThing(input: { name: \"probe\" }) { id } }").freeze
# Factor letters each variant carries; "-" means the variant is a control for that factor.
VARIANTS = {
"main" => ["------", "client + stub-server thread, both in the MAIN Ractor"],
"outer_stub" => ["------", "client alone in a Ractor, stub-server thread in the MAIN Ractor"],
"inner_stub" => ["TSBLE-", "client + stub-server thread, both inside ONE Ractor (the CI failure)"],
"idle_thread" => ["T-----", "client in a Ractor + a second thread that only sleeps"],
"cpu_thread" => ["T-----", "client in a Ractor + a second thread that only burns CPU"],
"pipe_thread" => ["T-B---", "client in a Ractor + a second thread doing pipe IO (no socket)"],
"two_clients" => ["TSB---", "two client threads in a Ractor, stub server in the MAIN Ractor"],
"inner_listener" => ["---L--", "stub server accept(2)ing inside a Ractor (sole thread), client in MAIN"],
"raw_inner" => ["TSBLE-", "as inner_stub, raw TCPSocket instead of Net::HTTP"],
"raw_serial_inner" => ["---LE-", "raw, ONE thread in a Ractor: listen, connect, accept, exchange"],
"outer_cpu" => ["-----M", "client alone in a Ractor, a CPU-burning thread in the MAIN Ractor"],
"main_cpu" => ["-----M", "client + a CPU-burning thread, both in the MAIN Ractor"],
}.freeze
class << self
def run(variant)
case variant
when "main" then with_stub { |port| post(port) }
when "outer_stub" then with_stub { |port| in_ractor(port) { |p| RactorFactors.post(p) } }
when "inner_stub" then in_ractor { RactorFactors.with_stub { |p| RactorFactors.post(p) } }
when "idle_thread" then with_stub { |port| in_ractor(port) { |p| RactorFactors.with_helper(:sleep) { RactorFactors.post(p) } } }
when "cpu_thread" then with_stub { |port| in_ractor(port) { |p| RactorFactors.with_helper(:cpu) { RactorFactors.post(p) } } }
when "pipe_thread" then with_stub { |port| in_ractor(port) { |p| RactorFactors.with_helper(:pipe) { RactorFactors.post(p) } } }
when "two_clients" then with_stub(2) { |port| in_ractor(port) { |p| RactorFactors.two_clients(p) } }
when "inner_listener" then inner_listener
when "raw_inner" then in_ractor { RactorFactors.with_stub { |p| RactorFactors.raw_post(p) } }
when "raw_serial_inner" then in_ractor { RactorFactors.serial_exchange }
when "outer_cpu" then with_stub { |port| with_helper(:cpu) { in_ractor(port) { |p| RactorFactors.post(p) } } }
when "main_cpu" then with_stub { |port| with_helper(:cpu) { post(port) } }
else raise ArgumentError, "unknown variant #{variant}"
end
end
# --- arrangements ----------------------------------------------------------------------------
# Stub upstream: one accept thread per expected request, in whichever Ractor calls this.
def with_stub(requests = 1)
server = TCPServer.new("127.0.0.1", 0)
threads = Array.new(requests) { Thread.new { serve_one(server) } }
result = yield(server.addr.fetch(1))
threads.each { |thread| thread.join(2) }
result
ensure
threads&.each(&:kill)
server&.close
end
# The whole server side — bind, listen, accept(2), exchange — runs inside the Ractor on its only
# thread; the client runs in the main Ractor. Only the port number crosses, over a Ractor::Port.
def inner_listener
out = Ractor::Port.new
ractor = Ractor.new(out) do |channel|
server = TCPServer.new("127.0.0.1", 0)
channel.send(server.addr.fetch(1))
RactorFactors.serve_one(server)
ensure
server&.close
end
result = post(out.receive)
ractor_value(ractor)
result
end
def two_clients(port)
results = Array.new(2) { Thread.new { post(port) } }.map(&:value)
raise "unexpected #{results.inspect}" unless results.all? { |result| result == :ok }
:ok
end
def with_helper(kind)
running = [true]
helper = Thread.new { helper_loop(kind, running) }
yield
ensure
running[0] = false
helper&.kill
end
def helper_loop(kind, running)
case kind
when :sleep then (sleep(0.001) while running[0])
when :cpu then (2000.times { |i| i * i } while running[0])
when :pipe
reader, writer = IO.pipe
while running[0]
writer.write("x")
reader.read(1)
end
[reader, writer].each(&:close)
end
end
# One thread, both ends: connect(2) completes against the listen backlog before any accept(2).
def serial_exchange
server = TCPServer.new("127.0.0.1", 0)
socket = TCPSocket.new("127.0.0.1", server.addr.fetch(1))
peer = server.accept
socket.write(raw_request(server.addr.fetch(1)))
read_request(peer)
peer.write(http_response)
check_raw_response(socket)
:ok
ensure
[socket, peer, server].each { |io| io&.close unless io&.closed? }
end
# --- primitives ------------------------------------------------------------------------------
def serve_one(server)
peer = server.accept
read_request(peer)
peer.write(http_response)
peer.flush
:served
ensure
peer&.close
end
def read_request(io)
headers = {}
while (line = io.gets)
line = line.chomp
break if line.empty?
key, value = line.split(":", 2)
headers[key.strip.downcase] = value.to_s.strip if value
end
io.read(Integer(headers.fetch("content-length")))
headers
end
def http_response
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\n" \
"Content-Length: #{RESPONSE_BODY.bytesize}\r\nConnection: close\r\n\r\n#{RESPONSE_BODY}"
end
def raw_request(port)
"POST /graphql HTTP/1.1\r\nHost: 127.0.0.1:#{port}\r\ncontent-type: application/json\r\n" \
"Content-Length: #{REQUEST_BODY.bytesize}\r\nConnection: close\r\n\r\n#{REQUEST_BODY}"
end
def post(port)
http = Net::HTTP.new("127.0.0.1", port)
http.open_timeout = 5
http.read_timeout = 5
response = http.post("/graphql", REQUEST_BODY, "content-type" => "application/json")
raise "status #{response.code}" unless response.code == "200"
raise "body #{response.body.inspect}" unless response.body == RESPONSE_BODY
:ok
ensure
http.finish if http&.started?
end
def raw_post(port)
socket = TCPSocket.new("127.0.0.1", port)
socket.write(raw_request(port))
check_raw_response(socket)
:ok
ensure
socket&.close unless socket&.closed?
end
def check_raw_response(socket)
status = socket.gets
raise "no status line" if status.nil?
raise "status #{status.strip.inspect}" unless status.start_with?("HTTP/1.1 200")
length = nil
while (line = socket.gets)
break if line.chomp.empty?
length = Integer(line.split(":", 2).fetch(1).strip) if line.downcase.start_with?("content-length")
end
raise "no content-length" if length.nil?
body = socket.read(length)
raise "body #{body.inspect}" unless body == RESPONSE_BODY
end
def in_ractor(*args, &block)
ractor_value(Ractor.new(*args, &block))
end
# Re-raise a copy rather than the cause itself: re-raising the cause of the exception being
# handled builds a cause cycle, and Ruby reports that as "ArgumentError: circular causes".
def ractor_value(ractor)
ractor.respond_to?(:value) ? ractor.value : ractor.take
rescue Ractor::RemoteError => error
cause = error.cause
raise error if cause.nil?
raise cause.class, cause.message, cause.backtrace
end
end
end
Socket.tcp_fast_fallback = false if ENV["NO_HEV2"]
requested = ARGV[0] || "all"
iterations = Integer(ARGV[1] || 300)
variants = requested == "all" ? RactorFactors::VARIANTS.keys : [requested]
# Running arrangements in blocks makes their counts incomparable: the same arrangement scored 196/200
# and 0/200 in consecutive builds, so a block's count measures host load during that block as much as
# the arrangement. Interleaving one iteration of each per round spreads every arrangement over the
# same load windows. raw_inner is excluded because its VM abort would end the whole run.
if requested == "interleave"
variants = RactorFactors::VARIANTS.keys - ["raw_inner"]
tally = variants.to_h { |variant| [variant, 0] }
signatures = {}
iterations.times do |round|
variants.rotate(round).each do |variant|
RactorFactors.run(variant)
rescue => error
tally[variant] += 1
key = "#{variant} | #{error.class}: #{error.message.to_s.gsub(/\d+/, "N")[0, 90]}"
signatures[key] = signatures.fetch(key, 0) + 1
end
if (round + 1) % 25 == 0
puts "round #{round + 1}/#{iterations} loadavg=#{File.read("/proc/loadavg").split.first(3).join(" ")} " \
"bad=#{tally.reject { |_, count| count.zero? }.inspect}"
$stdout.flush
end
end
puts
variants.each do |variant|
factors, description = RactorFactors::VARIANTS.fetch(variant)
puts format("%-17s %-6s %4d bad / %-4d %s", variant, factors, tally.fetch(variant), iterations, description)
end
puts
signatures.sort_by { |_, count| -count }.each { |key, count| puts " #{count}x #{key}" }
exit 0
end
puts RUBY_DESCRIPTION
puts "platform=#{RUBY_PLATFORM} nprocessors=#{Etc.nprocessors} " \
"RUBY_MAX_CPU=#{ENV.fetch("RUBY_MAX_CPU", "(unset)")} " \
"tcp_fast_fallback=#{Socket.respond_to?(:tcp_fast_fallback) ? Socket.tcp_fast_fallback : "n/a"}"
puts "iterations=#{iterations}"
puts
puts "factors: T=2nd thread in Ractor S=2nd thread does socket IO B=2nd thread does blocking IO"
puts " L=accept(2) inside Ractor E=both connection ends in Ractor"
puts " M=CPU-burning thread in the MAIN Ractor instead of in the non-main one"
puts
variants.each do |variant|
factors, description = RactorFactors::VARIANTS.fetch(variant)
failures = {}
bad = 0
iterations.times do
RactorFactors.run(variant)
rescue => error
bad += 1
signature = "#{error.class}: #{error.message.to_s.gsub(/\d+/, "N")[0, 110]}"
(failures[signature] ||= []) << error
end
puts format("%-17s %-6s %4d bad / %-4d %s", variant, factors, bad, iterations, description)
failures.each do |signature, errors|
puts " #{errors.length}x #{signature}"
puts " #{errors.first.backtrace&.first(4)&.join("\n ")}"
end
$stdout.flush
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment