processx Non-Blocking I/O Failure on Windows Named Pipes

processx_failure.md

Summary The R processx package is unable to perform non-blocking reads or polling on Windows Named Pipes when they are opened as external files. Functions that are intended to be asynchronous or non-blocking—specifically poll(), conn_read_chars(), and conn_read_lines()—behave as blocking calls that freeze the R session until data is available.

Environment

• Operating System: Windows (win32)
• Library: processx (R package)
• Transport: Windows Named Pipes (\.\pipe\pipe_name)
• Scenario: R connecting to a pipe created by an independent external process (e.g., a Node.js server).

Detailed Problem Description When a connection to a Windows Named Pipe is established via processx::conn_create_file() or processx::conn_connect_fifo(), the following behaviors occur:

1. Indefinite Hanging on poll(): Calling poll(list(con), timeout=0) does not return immediately. Instead, it hangs the R console.
2. Blocking Reads: Calling conn_read_chars(con, n) or conn_read_lines(con, n=1) on an empty pipe freezes the R thread. It does not return an empty result; it waits until the server writes data to the pipe.
3. Unstoppable Execution: Because the hang occurs within a low-level Windows C-system call, R's setTimeLimit() and user interrupts (Ctrl+C) are often ignored, requiring the process to be killed via Task Manager or taskkill.

Technical Root Cause The failure is rooted in the difference between Synchronous and Asynchronous (Overlapped) I/O on Windows:

• Synchronous Mode: By default, Windows opens files and pipes in synchronous mode. In this mode, a ReadFile operation suspends the calling thread until the requested data is available.
• Asynchronous Mode: To perform non-blocking operations, a pipe must be opened with the FILE_FLAG_OVERLAPPED flag.
• The processx Limitation: processx::conn_create_file() opens external paths using standard synchronous flags. Furthermore, the Windows implementation of poll() in processx relies on WaitForMultipleObjects, which does not support Named Pipes unless they are in Overlapped mode.

Workarounds

1. Use Base R Connections (Recommended for Pipes) For communication with external Windows Named Pipes, use R's native connection system. It contains specific Windows-only logic to handle non-blocking pipe I/O that processx currently lacks.

1 con <- file("\\.\pipe\my_pipe", open = "r+b", blocking = FALSE) 2 # Use readLines(con, n = 1) or readBin()

2. Standard Streams (stdin/stdout) If you must use processx functions, avoid Named Pipes. Launch the external process as a child of R using processx::process$new(stdout = "|", stdin = "|"). processx automatically configures these internal pipes for asynchronous I/O, allowing poll() and non-blocking reads to work as intended.

3. Native poll Alternative Avoid processx::poll() on Windows Named Pipes. If you use processx for the connection, only call conn_read_chars() and expect it to block, or use a separate "watchdog" process to monitor the pipe.