RubyLLM: repeated TLS handshakes across standalone judge calls

README.md

RubyLLM connection reuse probe

Two sequential RubyLLM.judge calls using the async_http adapter. The script records completed TLS handshakes and prints them after each call. It uses real requests to the TypeSafe Jev API, without HTTP mocks or an application-managed Async reactor.

Run

Save Gemfile and probe.rb in one directory. With TYPESAFE_API_KEY set in your environment, run:

bundle install
bundle exec ruby probe.rb

The script sends two small, billable API requests. It reads the key from the environment and does not print it. RubyLLM retries are disabled.

What to look for

The second call should be able to reuse the connection from the first call. In the tested version, each call completes a new TLS handshake on a different socket. The summary is:

Total: 2 calls, 2 TLS handshakes

ALPN="h2" means HTTP/2 was negotiated. session_reused=false means the TLS session was not resumed. TLS session resumption, if reported, would still be a handshake on a new connection, not reuse of the existing connection.

The model's answer to the keep-alive question is just an Easter egg. The handshake records are the evidence of connection setup. The probe does not measure a latency improvement.

Tested environment

• Ruby 4.0.6
• RubyLLM commit f7fd97d14b65c546d49606f91f97c3fdcf82f64b
• Faraday 2.14.4
• async-http-faraday 0.24.0
• async-http 0.105.0
• Async 2.46.0
• OpenSSL gem 4.0.2

The Gemfile pins RubyLLM and the adapter. Other dependencies may resolve to newer versions.

Sample run

See output.txt for a run on the reporter's computer on 2026-09-27. The Ruby IO::Buffer experimental warning was omitted.

Times use a monotonic clock. Call duration covers the full RubyLLM.judge call. TLS handshake covers elapsed time from the first SSL connect attempt to handshake completion, including waits. It excludes DNS lookup and TCP connection setup. The TLS time is part of the call duration, not an additional duration to add to it.

This sample shows two new TLS handshakes. It does not compare against connection reuse or establish how much faster a pooled implementation would be.

Gemfile

source 'https://rubygems.org'

gem 'ruby_llm', github: 'crmne/ruby_llm', ref: 'f7fd97d14b65c546d49606f91f97c3fdcf82f64b'
gem 'async-http-faraday', '0.24.0'

output.txt

Call 1
  TLS handshake: 11.2 ms, socket=832, version=TLSv1.3, ALPN="h2", session_reused=false
  Call duration: 293.4 ms
  Keep-alive expected: 0.79
  New TLS handshakes: 1
Call 2
  TLS handshake: 10.7 ms, socket=856, version=TLSv1.3, ALPN="h2", session_reused=false
  Call duration: 259.7 ms
  Keep-alive expected: 0.79
  New TLS handshakes: 1
Total: 2 calls, 2 TLS handshakes

probe.rb

require 'bundler/setup'
require 'ruby_llm'
require 'async/http/faraday'
require 'openssl'

$stdout.sync = true

module HandshakeLog
  class << self
    attr_accessor :events
  end
  self.events = []

  def connect(...)
    @probe_handshake_started_at ||= Process.clock_gettime(Process::CLOCK_MONOTONIC)
    result = super
    record_handshake(result)
    result
  end

  def connect_nonblock(...)
    @probe_handshake_started_at ||= Process.clock_gettime(Process::CLOCK_MONOTONIC)
    result = super
    record_handshake(result)
    result
  end

  private

  def record_handshake(result)
    return unless result.equal?(self)
    return if @probe_handshake_recorded

    @probe_handshake_recorded = true
    elapsed_ms = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - @probe_handshake_started_at) * 1000
    HandshakeLog.events << "#{format('%.1f', elapsed_ms)} ms, socket=#{object_id}, version=#{ssl_version}, " \
                           "ALPN=#{alpn_protocol.inspect}, session_reused=#{session_reused?}"
  end
end

OpenSSL::SSL::SSLSocket.prepend(HandshakeLog)

RubyLLM.configure do |config|
  config.typesafe_api_key = ENV.fetch('TYPESAFE_API_KEY')
  config.faraday_adapter = :async_http
  config.max_retries = 0
  config.request_timeout = 20
end

RubyLLM.models.find('jev-latest')

2.times do |index|
  call_number = index + 1
  puts "Call #{call_number}"
  before = HandshakeLog.events.size
  started_at = Process.clock_gettime(Process::CLOCK_MONOTONIC)

  result = RubyLLM.judge(
    'RubyLLM is a Ruby library for calling AI APIs from applications, including Rails web requests and background jobs.',
    model: 'jev-latest',
    questions: {
      keep_alive_expected: {
        type: :probability,
        instructions: 'Would you expect a library like RubyLLM to reuse HTTP connections across API calls when configured with an adapter that supports keep-alive?'
      }
    }
  )

  elapsed_ms = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_at) * 1000
  HandshakeLog.events.drop(before).each { |event| puts "  TLS handshake: #{event}" }
  puts "  Call duration: #{format('%.1f', elapsed_ms)} ms"
  puts "  Keep-alive expected: #{result.keep_alive_expected.probability}"
  puts "  New TLS handshakes: #{HandshakeLog.events.size - before}"
end

puts "Total: 2 calls, #{HandshakeLog.events.size} TLS handshakes"