Strace 6.1 handy help

README.md

strace - System Call Tracer

Synopsis

strace [OPTIONS] { -p PID | PROG [ARGS] }
strace -c [OPTIONS] { -p PID | PROG [ARGS] }

---

Quick Start Examples

# Trace a running process
strace -p 1234

# Trace a new command
strace ls -la

# Follow forks and save to file
strace -ff -o trace.log myprogram

# Summary statistics only
strace -c ./myapp

# Trace specific syscalls
strace -e trace=open,read,write ./myapp

---

General Options

Expression Filtering

-e EXPR        Qualifying expression for filtering traces

Available options:

• trace - Filter which syscalls to trace
• abbrev - Abbreviate output for syscalls
• verbose - Show detailed structure information
• raw - Print undecoded syscall arguments
• signal - Filter signals to trace
• read - Dump data read from file descriptors
• write - Dump data written to file descriptors
• fault - Inject faults into syscalls
• inject - Tamper with syscall behavior
• status - Filter by return status
• quiet - Suppress informational messages
• kvm - Print KVM vcpu exit reasons
• decode-fds - Show file descriptor details

Format: OPTION=[!]all or OPTION=[!]VAL1[,VAL2]...

---

Startup Options

Environment & Process Control

Option	Description
-E VAR=VAL --env=VAR=VAL	Set environment variable for command
-E VAR --env=VAR	Remove environment variable for command
-p PID --attach=PID	Trace existing process (can be repeated)
-u USERNAME --user=USERNAME	Run command as specified user

---

Tracing Options

Process Following & Detachment

Option	Description
-b execve --detach-on=execve	Detach from process after execve
-f --follow-forks	Follow child processes created by fork
-ff --follow-forks --output-separately	Follow forks with separate output files

Daemon Modes

Option	Description
-D --daemonize[=grandchild]	Run tracer as grandchild process
-DD --daemonize=pgroup	Run tracer in separate process group
-DDD --daemonize=session	Run tracer in separate session

Signal Handling

-I N, --interruptible=N

Levels:

1. anywhere - No signals blocked
2. waiting - Fatal signals blocked during syscall decode (default)
3. never - Fatal signals always blocked (default with -o FILE)
4. never_tstp - Fatal signals and SIGTSTP (Ctrl+Z) always blocked

---

Filtering Options

Syscall Filtering

-e trace=[!][?]{{SYSCALL|GROUP|all|/REGEX}[@64|@32|@x32]|none}
--trace=[!][?]{{SYSCALL|GROUP|all|/REGEX}[@64|@32|@x32]|none}

Syscall Groups:

• %clock - Clock and time syscalls
• %creds - Credential syscalls
• %desc - Descriptor operations
• %file - File operations
• %fstat - File status (fstat family)
• %fstatfs - Filesystem status (fstatfs family)
• %ipc - Inter-process communication
• %lstat - Symbolic link status
• %memory - Memory management
• %net - Network operations
• %process - Process management
• %pure - Pure syscalls (no side effects)
• %signal - Signal handling
• %stat - File status (stat family)
• %%stat - Alternative stat group
• %statfs - Filesystem status (statfs family)
• %%statfs - Alternative statfs group

Examples:

# Trace only open/read/write
strace -e trace=open,read,write ./app

# Trace all network calls
strace -e trace=%net ./app

# Exclude certain syscalls
strace -e trace=\!open ./app

# Trace syscalls matching regex
strace -e trace=/^read.* ./app

Signal Filtering

-e signal=SET, --signal=SET

Trace only specified signals (e.g., SIGTERM,SIGKILL)

Status Filtering

-e status=SET, --status=SET

Available statuses:

• successful - Syscalls that succeeded
• failed - Syscalls that failed
• unfinished - Syscalls not yet completed
• unavailable - Status unavailable
• detached - Process was detached

Path Filtering

-P PATH, --trace-path=PATH

Trace only syscalls accessing the specified path

Quick Filters

Option	Description
-z --successful-only	Show only successful syscalls
-Z --failed-only	Show only failed syscalls

---

Output Format Options

Basic Formatting

Option	Description
-a COLUMN --columns=COLUMN	Alignment column for results (default: 40)
-o FILE --output=FILE	Write output to file instead of stderr
-A --output-append-mode	Append to output file
--output-separately	Create separate files per process (append PID)
-v --no-abbrev	Verbose mode - no abbreviations

String Handling

Option	Description
-s STRSIZE --string-limit=STRSIZE	Limit string length (default: 32)
-x --strings-in-hex=non-ascii	Print non-ASCII strings in hex
-xx --strings-in-hex[=all]	Print all strings in hex
--strings-in-hex=non-ascii-chars	Use hex in escape sequences

Syscall Details

-e abbrev=SET, --abbrev=SET

Abbreviate output for specified syscalls

-e verbose=SET, --verbose=SET

Show detailed structure information for specified syscalls

-e raw=SET, --raw=SET

Print undecoded (raw) arguments for specified syscalls

Data Dumping

-e read=SET, --read=SET

Dump data read from specified file descriptors

-e write=SET, --write=SET

Dump data written to specified file descriptors

File Descriptor Decoding

-y, --decode-fds[=path]
-yy, --decode-fds=all
-e decode-fds=SET, --decode-fds=SET

Detail types:

• path - Show file paths
• dev - Show device major/minor for block/char devices
• pidfd - Show associated PID for pidfds
• socket - Show protocol info for socket descriptors

PID Information

Option	Description
--decode-pids=pidns	Print PIDs in strace's namespace
-Y --decode-pids=comm	Print command names with PIDs

Quietness Levels

-q, --quiet=attach,personality
-qq, --quiet=attach,personality,exit
-qqq, --quiet=all
-e quiet=SET, --quiet=SET

Suppressible messages:

• attach - Attaching/detaching messages
• exit - Process exit status
• path-resolution - Path resolution details
• personality - Personality changes
• thread-execve - Thread execve messages

Additional Information

Option	Description
-i --instruction-pointer	Print instruction pointer at syscall time
-k --stack-traces	Show stack trace for each syscall
-n --syscall-number	Print syscall numbers
-X FORMAT --const-print-style=FORMAT	Format for constants (raw/abbrev/verbose)

---

Timestamp Options

Absolute Timestamps

-t, --absolute-timestamps[=time]

Print absolute timestamp (HH:MM:SS)

-tt, --absolute-timestamps=[time,]us

Print absolute timestamp with microseconds (HH:MM:SS.microseconds)

-ttt, --absolute-timestamps=unix,us

Print Unix epoch time with microseconds

--absolute-timestamps=[[format:]FORMAT[,[precision:]PRECISION]]

Formats:

• none - No timestamps
• time - Wall clock time (default)
• unix - Unix epoch

Precision: s, ms, us, ns (default: seconds)

Relative Timestamps

-r, --relative-timestamps[=PRECISION]

Print time since previous syscall

Precision: s, ms, us, ns (default: microseconds)

Syscall Duration

-T, --syscall-times[=PRECISION]

Print time spent in each syscall

Precision: s, ms, us, ns (default: microseconds)

---

Statistics Options

Summary Reports

Option	Description
-c --summary-only	Show only statistics summary
-C --summary	Show both trace and statistics

Summary Configuration

-O OVERHEAD[UNIT], --summary-syscall-overhead=OVERHEAD[UNIT]

Set syscall tracing overhead (for accurate time calculations)

Units: s, ms, us, ns (default: microseconds)

-S SORTBY, --summary-sort-by=SORTBY

Sort options:

• time - Total time (default)
• min-time - Minimum time
• max-time - Maximum time
• avg-time - Average time
• calls - Number of calls
• errors - Number of errors
• name - Syscall name
• nothing - No sorting

-U COLUMNS, --summary-columns=COLUMNS

Available columns:

• time-percent - Percentage of total time
• total-time - Total time spent
• min-time - Minimum call time
• max-time - Maximum call time
• avg-time - Average call time
• calls - Number of calls
• errors - Number of errors
• name - Syscall name

Default: time-percent,total-time,avg-time,calls,errors,name

Time Measurement

-w, --summary-wall-clock

Use wall clock time instead of system time

---

Syscall Tampering

Fault Injection

-e inject=SET[:error=ERRNO|:retval=VALUE][:signal=SIG][:syscall=SYSCALL]
              [:delay_enter=DELAY][:delay_exit=DELAY]
              [:poke_enter=@argN=DATAN,@argM=DATAM...]
              [:poke_exit=@argN=DATAN,@argM=DATAM...]
              [:when=WHEN]

--inject=SET[:error=ERRNO|:retval=VALUE][:signal=SIG][:syscall=SYSCALL]
         [:delay_enter=DELAY][:delay_exit=DELAY]
         [:poke_enter=@argN=DATAN,@argM=DATAM...]
         [:poke_exit=@argN=DATAN,@argM=DATAM...]
         [:when=WHEN]

Parameters:

• error=ERRNO - Make syscall fail with specified errno
• retval=VALUE - Set specific return value
• signal=SIG - Deliver signal to process
• syscall=SYSCALL - Inject different syscall
• delay_enter=DELAY - Delay before syscall
• delay_exit=DELAY - Delay after syscall
• poke_enter - Modify arguments on entry
• poke_exit - Modify arguments on exit
• when=WHEN - When to inject (see below)

Delay format: Microseconds or NUMBER{s|ms|us|ns}

When format: FIRST[..LAST][+[STEP]]

• 1 - First call only
• 1..5 - Calls 1 through 5
• 1..5+2 - Calls 1, 3, 5

Fault Shorthand

-e fault=SET[:error=ERRNO][:when=WHEN]
--fault=SET[:error=ERRNO][:when=WHEN]

Synonym for -e inject with default error ENOSYS

Examples:

# Make all open calls fail with ENOENT
strace -e inject=open:error=ENOENT ./app

# Delay read syscalls by 100ms
strace -e inject=read:delay_exit=100ms ./app

# Fail first 3 write calls
strace -e inject=write:error=EIO:when=1..3 ./app

---

Miscellaneous Options

Option	Description
-d --debug	Enable debug output to stderr
-h --help	Print help message
-V --version	Print version information
--seccomp-bpf	Enable seccomp-bpf filtering

Tips & Tricks

--tips[=[[id:]ID][,[format:]FORMAT]]

Show helpful strace tips on exit

ID: Non-negative integer or random (default: random)

Format:

• none - No tips
• compact - Brief format (default)
• full - Detailed format

---

Common Use Cases

Debugging

# Find which files a program accesses
strace -e trace=open,openat,stat ./myprogram

# Debug network issues
strace -e trace=%net -s 1000 ./myapp

# See what's blocking a process
strace -T -tt -p 1234

# Follow a daemon and all its children
strace -ff -o /tmp/daemon.log -p 1234

Performance Analysis

# Get syscall statistics
strace -c ./myprogram

# Find slow syscalls (>100ms)
strace -T ./myprogram 2>&1 | grep -E '<0\.[1-9]'

# Profile with wall-clock time
strace -c -w ./myprogram

# Sort by average time
strace -c -S avg-time ./myprogram

Testing & Fault Injection

# Simulate disk full
strace -e inject=write:error=ENOSPC ./myapp

# Add network latency
strace -e inject=recv:delay_exit=100ms ./myapp

# Test error handling
strace -e fault=open:error=EACCES:when=5+ ./myapp

---

Tips

1. Use -o to avoid mixing output: Send trace to a file to keep stdout/stderr clean
2. Combine with grep: strace -e trace=open ./app 2>&1 | grep ENOENT
3. Watch live: strace -p $(pidof myapp) to attach to running process
4. Limit string output: Use -s 0 for no string output, or increase for more detail
5. Focus on errors: Use -Z to see only failed syscalls
6. Follow forks properly: Always use -ff when tracing daemons that fork
7. Time analysis: Combine -T and -tt for detailed timing information

---

Exit Status

strace exits with:

• 0 if successful
• 1 if there was an error
• The exit status of the traced command if different