Skip to content

Instantly share code, notes, and snippets.

@kojix2

kojix2/profiling-crystal-io-pipe-with-strace.ipynb

Last active June 10, 2026 04:16
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save kojix2/5cc67943332cc54266f67abade941aeb to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save kojix2/5cc67943332cc54266f67abade941aeb to your computer and use it in GitHub Desktop.
Download ZIP
Profiling Crystal IO.pipe with strace.ipynb
Display the source blob
Display the rendered blob
Raw
profiling-crystal-io-pipe-with-strace.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"authorship_tag": "ABX9TyPgbKDVNf+CMfcr691GXp6v",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/kojix2/5cc67943332cc54266f67abade941aeb/profiling-crystal-io-pipe-with-strace.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"source": [
"# Profiling Crystal IO.pipe with strace\n",
"\n",
"https://forum.crystal-lang.org/t/user-space-pipe-implementation/8837"
],
"metadata": {
"id": "1fCbgIPHsfCZ"
}
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"collapsed": true,
"id": "VsfyksN9mb0Y",
"outputId": "aa45f1d5-9ed8-4246-d701-8d7f4bbc394e"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"deb http://download.opensuse.org/repositories/devel:/languages:/crystal/xUbuntu_22.04/ /\n",
"Hit:1 https://cli.github.com/packages stable InRelease\n",
"Get:2 https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ InRelease [3,632 B]\n",
"Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB]\n",
"Get:4 https://r2u.stat.illinois.edu/ubuntu jammy InRelease [6,555 B]\n",
"Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease\n",
"Get:6 http://download.opensuse.org/repositories/devel:/languages:/crystal/xUbuntu_22.04 InRelease [1,585 B]\n",
"Get:7 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB]\n",
"Hit:8 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu jammy InRelease\n",
"Hit:9 https://ppa.launchpadcontent.net/ubuntugis/ppa/ubuntu jammy InRelease\n",
"Get:10 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 Packages [3,050 kB]\n",
"Get:11 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB]\n",
"Get:12 https://r2u.stat.illinois.edu/ubuntu jammy/main all Packages [10.3 MB]\n",
"Get:13 http://download.opensuse.org/repositories/devel:/languages:/crystal/xUbuntu_22.04 Packages [8,607 B]\n",
"Get:14 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [4,006 kB]\n",
"Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1,301 kB]\n",
"Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [7,183 kB]\n",
"Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [4,357 kB]\n",
"Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,608 kB]\n",
"Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [7,489 kB]\n",
"Fetched 39.7 MB in 4s (9,994 kB/s)\n",
"Reading package lists... Done\n",
"W: Skipping acquire of configured file 'main/source/Sources' as repository 'https://r2u.stat.illinois.edu/ubuntu jammy InRelease' does not seem to provide it (sources.list entry misspelt?)\n",
"Reading package lists... Done\n",
"Building dependency tree... Done\n",
"Reading state information... Done\n",
"The following additional packages will be installed:\n",
" crystal1.20 libgmp-dev libgmpxx4ldbl libyaml-dev\n",
"Suggested packages:\n",
" gmp-doc libgmp10-doc libmpfr-dev libyaml-doc\n",
"The following NEW packages will be installed:\n",
" crystal crystal1.20 libgmp-dev libgmpxx4ldbl libyaml-dev\n",
"0 upgraded, 5 newly installed, 0 to remove and 27 not upgraded.\n",
"Need to get 43.1 MB of archives.\n",
"After this operation, 163 MB of additional disk space will be used.\n",
"Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgmpxx4ldbl amd64 2:6.2.1+dfsg-3ubuntu1 [9,580 B]\n",
"Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgmp-dev amd64 2:6.2.1+dfsg-3ubuntu1 [337 kB]\n",
"Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 libyaml-dev amd64 0.2.2-1build2 [62.8 kB]\n",
"Get:5 http://download.opensuse.org/repositories/devel:/languages:/crystal/xUbuntu_22.04 crystal 1.20.0-1+1.1 [1,090 B]\n",
"Get:4 http://download.opensuse.org/repositories/devel:/languages:/crystal/xUbuntu_22.04 crystal1.20 1.20.2-1+1.1 [42.7 MB]\n",
"Fetched 43.1 MB in 4s (10.4 MB/s)\n",
"debconf: unable to initialize frontend: Dialog\n",
"debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 78, <> line 5.)\n",
"debconf: falling back to frontend: Readline\n",
"debconf: unable to initialize frontend: Readline\n",
"debconf: (This frontend requires a controlling tty.)\n",
"debconf: falling back to frontend: Teletype\n",
"dpkg-preconfigure: unable to re-open stdin: \n",
"Selecting previously unselected package libgmpxx4ldbl:amd64.\n",
"(Reading database ... 118243 files and directories currently installed.)\n",
"Preparing to unpack .../libgmpxx4ldbl_2%3a6.2.1+dfsg-3ubuntu1_amd64.deb ...\n",
"Unpacking libgmpxx4ldbl:amd64 (2:6.2.1+dfsg-3ubuntu1) ...\n",
"Selecting previously unselected package libgmp-dev:amd64.\n",
"Preparing to unpack .../libgmp-dev_2%3a6.2.1+dfsg-3ubuntu1_amd64.deb ...\n",
"Unpacking libgmp-dev:amd64 (2:6.2.1+dfsg-3ubuntu1) ...\n",
"Selecting previously unselected package crystal1.20.\n",
"Preparing to unpack .../crystal1.20_1.20.2-1+1.1_amd64.deb ...\n",
"Unpacking crystal1.20 (1.20.2-1+1.1) ...\n",
"Selecting previously unselected package crystal.\n",
"Preparing to unpack .../crystal_1.20.0-1+1.1_all.deb ...\n",
"Unpacking crystal (1.20.0-1+1.1) ...\n",
"Selecting previously unselected package libyaml-dev:amd64.\n",
"Preparing to unpack .../libyaml-dev_0.2.2-1build2_amd64.deb ...\n",
"Unpacking libyaml-dev:amd64 (0.2.2-1build2) ...\n",
"Setting up libyaml-dev:amd64 (0.2.2-1build2) ...\n",
"Setting up crystal1.20 (1.20.2-1+1.1) ...\n",
"Setting up libgmpxx4ldbl:amd64 (2:6.2.1+dfsg-3ubuntu1) ...\n",
"Setting up crystal (1.20.0-1+1.1) ...\n",
"Setting up libgmp-dev:amd64 (2:6.2.1+dfsg-3ubuntu1) ...\n",
"Processing triggers for man-db (2.10.2-1) ...\n",
"Processing triggers for libc-bin (2.35-0ubuntu3.13) ...\n",
"/sbin/ldconfig.real: /usr/local/lib/libtbbbind.so.3 is not a symbolic link\n",
"\n",
"/sbin/ldconfig.real: /usr/local/lib/libtbbmalloc_proxy.so.2 is not a symbolic link\n",
"\n",
"/sbin/ldconfig.real: /usr/local/lib/libtbb.so.12 is not a symbolic link\n",
"\n",
"/sbin/ldconfig.real: /usr/local/lib/libur_adapter_opencl.so.0 is not a symbolic link\n",
"\n",
"/sbin/ldconfig.real: /usr/local/lib/libtcm_debug.so.1 is not a symbolic link\n",
"\n",
"/sbin/ldconfig.real: /usr/local/lib/libtbbbind_2_0.so.3 is not a symbolic link\n",
"\n",
"/sbin/ldconfig.real: /usr/local/lib/libtcm.so.1 is not a symbolic link\n",
"\n",
"/sbin/ldconfig.real: /usr/local/lib/libumf.so.1 is not a symbolic link\n",
"\n",
"/sbin/ldconfig.real: /usr/local/lib/libtbbbind_2_5.so.3 is not a symbolic link\n",
"\n",
"/sbin/ldconfig.real: /usr/local/lib/libtbbmalloc.so.2 is not a symbolic link\n",
"\n",
"/sbin/ldconfig.real: /usr/local/lib/libur_loader.so.0 is not a symbolic link\n",
"\n",
"/sbin/ldconfig.real: /usr/local/lib/libur_adapter_level_zero.so.0 is not a symbolic link\n",
"\n",
"/sbin/ldconfig.real: /usr/local/lib/libur_adapter_level_zero_v2.so.0 is not a symbolic link\n",
"\n",
"/sbin/ldconfig.real: /usr/local/lib/libhwloc.so.15 is not a symbolic link\n",
"\n"
]
}
],
"source": [
"!curl -fsSL https://crystal-lang.org/install.sh | sudo bash"
]
},
{
"cell_type": "code",
"source": [
"!apt install -yq strace"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"collapsed": true,
"id": "8FdHaDgznNcD",
"outputId": "a9a7bf14-ef81-40db-9249-5d413c9e38a7"
},
"execution_count": 2,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Reading package lists...\n",
"Building dependency tree...\n",
"Reading state information...\n",
"The following NEW packages will be installed:\n",
" strace\n",
"0 upgraded, 1 newly installed, 0 to remove and 27 not upgraded.\n",
"Need to get 567 kB of archives.\n",
"After this operation, 2,048 kB of additional disk space will be used.\n",
"Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 strace amd64 5.16-0ubuntu3 [567 kB]\n",
"Fetched 567 kB in 0s (6,029 kB/s)\n",
"Selecting previously unselected package strace.\n",
"(Reading database ... 120212 files and directories currently installed.)\n",
"Preparing to unpack .../strace_5.16-0ubuntu3_amd64.deb ...\n",
"Unpacking strace (5.16-0ubuntu3) ...\n",
"Setting up strace (5.16-0ubuntu3) ...\n",
"Processing triggers for man-db (2.10.2-1) ...\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"!git clone https://github.com/jgaskins/pipe\n",
"# Reduce transfer size for faster benchmarking (1GB -> 16MB)\n",
"!cd pipe && sed -i 's/1<<30/1<<24/g' examples/benchmark.cr\n",
"!cd pipe && crystal build --release examples/benchmark.cr"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"collapsed": true,
"id": "Ou1Vhys7mdbQ",
"outputId": "f929d09d-be2b-45c8-fb38-8fe69b48346c"
},
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Cloning into 'pipe'...\n",
"remote: Enumerating objects: 79, done.\u001b[K\n",
"remote: Counting objects: 100% (79/79), done.\u001b[K\n",
"remote: Compressing objects: 100% (46/46), done.\u001b[K\n",
"remote: Total 79 (delta 29), reused 70 (delta 20), pack-reused 0 (from 0)\u001b[K\n",
"Receiving objects: 100% (79/79), 14.38 KiB | 14.38 MiB/s, done.\n",
"Resolving deltas: 100% (29/29), done.\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"!pipe/benchmark"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "R3o9Xjt4mmOe",
"outputId": "ca0d3faa-e8a6-4b89-ef81-b702707e5b41"
},
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" user system total real\n",
"IO.pipe 0.052165 0.178608 0.230773 ( 0.233273)\n",
"Pipe.create 0.019918 0.008012 0.027930 ( 0.028080)\n",
"[\"16.0MiB\", \"16.0MiB\"]\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"!strace -c pipe/benchmark"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "gvCswW9bnOQM",
"outputId": "13cb58da-0db2-4845-b59c-5250d1c39852"
},
"execution_count": 5,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" user system total real\n",
"IO.pipe 1.143811 8.739151 9.882962 ( 110.515096)\n",
"Pipe.create 0.019484 0.016442 0.035926 ( 0.037564)\n",
"[\"16.0MiB\", \"16.0MiB\"]\n",
"% time seconds usecs/call calls errors syscall\n",
"------ ----------- ----------- --------- --------- ------------------\n",
" 99.66 4.301954 4 1048881 255 write\n",
" 0.24 0.010452 13 782 257 read\n",
" 0.06 0.002509 4 511 epoll_wait\n",
" 0.01 0.000444 13 33 mmap\n",
" 0.01 0.000221 221 1 execve\n",
" 0.01 0.000220 5 43 timerfd_settime\n",
" 0.00 0.000156 17 9 openat\n",
" 0.00 0.000087 9 9 close\n",
" 0.00 0.000081 8 10 newfstatat\n",
" 0.00 0.000071 7 9 mprotect\n",
" 0.00 0.000042 6 7 epoll_ctl\n",
" 0.00 0.000035 17 2 readlink\n",
" 0.00 0.000033 4 8 getrusage\n",
" 0.00 0.000025 3 7 rt_sigprocmask\n",
" 0.00 0.000022 5 4 pread64\n",
" 0.00 0.000021 7 3 brk\n",
" 0.00 0.000019 9 2 stat\n",
" 0.00 0.000018 18 1 munmap\n",
" 0.00 0.000016 1 16 fcntl\n",
" 0.00 0.000016 4 4 prlimit64\n",
" 0.00 0.000013 6 2 ioctl\n",
" 0.00 0.000010 3 3 getrandom\n",
" 0.00 0.000008 4 2 1 arch_prctl\n",
" 0.00 0.000008 4 2 sched_getaffinity\n",
" 0.00 0.000007 1 6 rt_sigaction\n",
" 0.00 0.000007 7 1 1 access\n",
" 0.00 0.000005 2 2 fstat\n",
" 0.00 0.000004 4 1 set_tid_address\n",
" 0.00 0.000004 4 1 rseq\n",
" 0.00 0.000003 3 1 set_robust_list\n",
" 0.00 0.000000 0 3 madvise\n",
" 0.00 0.000000 0 1 sigaltstack\n",
" 0.00 0.000000 0 1 timerfd_create\n",
" 0.00 0.000000 0 1 eventfd2\n",
" 0.00 0.000000 0 1 epoll_create1\n",
" 0.00 0.000000 0 2 pipe2\n",
"------ ----------- ----------- --------- --------- ------------------\n",
"100.00 4.316511 4 1050372 514 total\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"!crystal eval \"p Random::Secure.random_bytes.bytesize\""
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "MmRg5UjAuEP8",
"outputId": "eedae429-e528-4d7b-9198-edbf373f8d47"
},
"execution_count": 6,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"16\n"
]
}
]
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment