Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save loren-osborn/81eb75e4f9be38c35240ba541a55198b to your computer and use it in GitHub Desktop.
Save loren-osborn/81eb75e4f9be38c35240ba541a55198b to your computer and use it in GitHub Desktop.
I believe this is mostly misguided, but was hoping for some feedback
Please write a GNU autotools detection mechanism to test for `grep` on the target system, and if grep is present, detect it's capabilities to support EREs, PCREs, and the correct command line switches to put grep into that mode.
As autotools traditionally tests systems and/or tools for their support for individual features, to see if they are properly supported. My suggestion would be to test the grep command in four phase:
1. Including any possible shell builtins, find all grep variations (including`grep`, `egrep`, `fgrep`, etc.) that are defined or installed in the current shell path.
2. Enumerate all modal "production features" (not features like `--help` as it is intended as a user/developer aid, and not intended to be used within a script) and any command line flags/switches that have been used to enable them in any popular versions of grep, then determine which switches each version of grep variation recognizes and/or allows without generating an error. Ideally I would prefer to identify a long and short version of each flag/switch/option.
3. Test each command option on each grep variation to verify it performs the expected behavior. be aware that the same option letter may enable different features in different versions of grep. You can use different regexs and different inputs to distinguish a regex interpreted as a BRE, ERE or PCRE, as well as variations in the output to distinguish different modeal behaviours like context inclusion.
4. Prioritize non-depricated option names first, shell builtins second, and earlier in the PATH variables over later ones in the path. (steps 2, 3 and 4 can be round-robined, so if all recognizable features are supported, in their non-depricated form in the shell's builtin grep implementation, subsequent grep binaries in the PATH don't need to be tested.) These variations should be presented to autotools in a way that simpler invocations (simply using a BRE match, with non-default options) can expand to the first implementation that supports it, but a call to grep requiring PCRE support may expand to a grep binary later in the path that may include PCRE support.
The grep option you should test for are summarized in the following YAML data structure:
```
static_detection_data:
command_options:
- anticipated_names:
- anticipated_name: "G"
deprecated: false
- anticipated_name: "basic-regexp"
deprecated: false
parameter_type: null
modal_group: "syntax"
default_when_name_eq: "grep"
verification_tests:
- option_enabled: true
input_data: "q Fr[aeiou]d\\(ae\\|IO\\|u\\)ZZ z\nr FredIOZZ y\ns Fred(ae|IO|u)ZZ x\n"
pattern: "Fr[aeiou]d\\(ae\\|IO\\|u\\)ZZ"
expected_output: "r FredIOZZ y\n"
expected_error: ""
expected_status: 0
- option_enabled: true
input_data: "q Fr[aeiou]d\\(ae\\|io\\|u\\)ZZ z\nr FredioZZ y\ns Fred(ae|io|u)ZZ x\n"
pattern: "Fr[aeiou]d\\(ae\\|IO\\|u\\)ZZ"
expected_output: ""
expected_error: ""
expected_status: 1
- anticipated_names:
- anticipated_name: "E"
deprecated: false
- anticipated_name: "extended-regexp"
deprecated: false
parameter_type: null
modal_group: "syntax"
default_when_name_eq: "egrep"
verification_tests:
- option_enabled: true
input_data: "q Fr[aeiou]d\\(ae\\|IO\\|u\\)ZZ z\nr FredIOZZ y\ns Fred(ae|IO|u)ZZ x\n"
pattern: "Fr[aeiou]d\\(ae\\|IO\\|u\\)ZZ"
expected_output: "s Fred(ae|IO|u)ZZ x\n"
expected_error: ""
expected_status: 0
- option_enabled: true
input_data: "q Fr[aeiou]d\\(ae\\|io\\|u\\)ZZ z\nr FredioZZ y\ns Fred(ae|io|u)ZZ x\n"
pattern: "Fr[aeiou]d\\(ae\\|IO\\|u\\)ZZ"
expected_output: ""
expected_error: ""
expected_status: 1
- anticipated_names:
- anticipated_name: "P"
deprecated: false
- anticipated_name: "perl-regexp"
deprecated: false
parameter_type: null
modal_group: "syntax"
default_when_name_eq: "pcregrep"
verification_tests:
- option_enabled: true
input_data: "q FRr[aeiou]d\\(ae\\|IO\\|u\\)ZZ z\nr FRredIOZZ y\ns FRred(ae|IO|u)ZZ x\n"
pattern: "F(?i)rR(?-i)[aeiou]d\\(ae\\|IO\\|u\\)ZZ"
expected_output: "q Fr[aeiou]d\\(ae\\|IO\\|u\\)ZZ z\n"
expected_error: ""
expected_status: 0
- option_enabled: true
input_data: "q fRr[aeiou]d\\(ae\\|io\\|u\\)ZZ z\nr fRredioZZ y\ns fRred(ae|io|u)ZZ x\n"
pattern: "F(?i)rR(?-i)[aeiou]d\\(ae\\|IO\\|u\\)ZZ"
expected_output: ""
expected_error: ""
expected_status: 1
- anticipated_names:
- anticipated_name: "F"
deprecated: false
- anticipated_name: "fixed-strings"
deprecated: false
parameter_type: null
modal_group: "syntax"
default_when_name_eq: "fgrep"
verification_tests:
- option_enabled: true
input_data: "q Fr[aeiou]d\\(ae\\|IO\\|u\\)ZZ z\nr FredIOZZ y\ns Fred(ae|IO|u)ZZ x\n"
pattern: "Fr[aeiou]d\\(ae\\|IO\\|u\\)ZZ"
expected_output: "q Fr[aeiou]d\\(ae\\|IO\\|u\\)ZZ z\n"
expected_error: ""
expected_status: 0
- option_enabled: true
input_data: "q Fr[aeiou]d\\(ae\\|io\\|u\\)ZZ z\nr FredioZZ y\ns Fred(ae|io|u)ZZ x\n"
pattern: "Fr[aeiou]d\\(ae\\|IO\\|u\\)ZZ"
expected_output: ""
expected_error: ""
expected_status: 1
- anticipated_names:
- anticipated_name: "A"
deprecated: false
- anticipated_name: "after-context"
deprecated: false
parameter_type: "integer"
verification_tests:
- option_enabled: true
parameter: 2
input_data: "apple\nbanana\ncherry\ndate\nelderberry\nfig\ngrape\nhoneydew\nkiwi\n"
pattern: "elderberry"
expected_output: "elderberry\nfig\ngrape\n"
expected_error: ""
expected_status: 0
- option_enabled: false
input_data: "apple\nbanana\ncherry\ndate\nelderberry\nfig\ngrape\nhoneydew\nkiwi\n"
pattern: "elderberry"
expected_output: "elderberry\n"
expected_error: ""
expected_status: 0
- anticipated_names:
- anticipated_name: "B"
deprecated: false
- anticipated_name: "before-context"
deprecated: false
parameter_type: "integer"
verification_tests:
- option_enabled: true
parameter: 2
input_data: "apple\nbanana\ncherry\ndate\nelderberry\nfig\ngrape\nhoneydew\nkiwi\n"
pattern: "elderberry"
expected_output: "cherry\ndate\nelderberry\n"
expected_error: ""
expected_status: 0
- option_enabled: false
input_data: "apple\nbanana\ncherry\ndate\nelderberry\nfig\ngrape\nhoneydew\nkiwi\n"
pattern: "elderberry"
expected_output: "elderberry\n"
expected_error: ""
expected_status: 0
- anticipated_names:
- anticipated_name: "C"
deprecated: false
- anticipated_name: "context"
deprecated: false
parameter_type: "integer"
verification_tests:
- option_enabled: true
parameter: 2
input_data: "apple\nbanana\ncherry\ndate\nelderberry\nfig\ngrape\nhoneydew\nkiwi\n"
pattern: "elderberry"
expected_output: "cherry\ndate\nelderberry\nfig\ngrape\n"
expected_error: ""
expected_status: 0
- option_enabled: false
input_data: "apple\nbanana\ncherry\ndate\nelderberry\nfig\ngrape\nhoneydew\nkiwi\n"
pattern: "elderberry"
expected_output: "elderberry\n"
expected_error: ""
expected_status: 0
- anticipated_names:
- anticipated_name: "c"
deprecated: false
- anticipated_name: "count"
deprecated: false
parameter_type: null
verification_tests:
- option_enabled: true
input_data: "abc\n123def\ndefXYZ\n"
pattern: "def"
expected_output: "2\n"
expected_error: ""
expected_status: 0
- option_enabled: true
input_data: "abc\n123\ndef\n"
pattern: "xyz"
expected_output: "0\n"
expected_error: ""
expected_status: 1
- option_enabled: false
input_data: "abc\n123def\ndefXYZ\n"
pattern: "def"
expected_output: "123def\ndefXYZ\n"
expected_error: ""
expected_status: 0
- anticipated_names:
- anticipated_name: "e"
deprecated: false
- anticipated_name: "regexp"
deprecated: false
parameter_type: "string"
verification_tests:
- option_enabled: true
parameter: "john"
input_data: "alice\njohn\n"
pattern: ""
expected_output: "john\n"
expected_error: ""
expected_status: 0
- option_enabled: false
input_data: "alice\njohn\n"
pattern: "alice"
expected_output: "alice\n"
expected_error: ""
expected_status: 0
- anticipated_names:
- anticipated_name: "i"
deprecated: false
- anticipated_name: "ignore-case"
deprecated: false
parameter_type: null
verification_tests:
- option_enabled: true
input_data: "fooBAR\nFOObar\n"
pattern: "ooBA"
expected_output: "fooBAR\nFOObar\n"
expected_error: ""
expected_status: 0
- option_enabled: false
input_data: "fooBAR\nFOObar\n"
pattern: "ooBA"
expected_output: "fooBAR\n"
expected_error: ""
expected_status: 0
- option_enabled: true
input_data: "FOObar\n"
pattern: "ooBA"
expected_output: "FOObar\n"
expected_error: ""
expected_status: 0
- option_enabled: false
input_data: "FOObar\n"
pattern: "ooBA"
expected_output: ""
expected_error: ""
expected_status: 1
- anticipated_names:
- anticipated_name: "n"
deprecated: false
- anticipated_name: "line-number"
deprecated: false
parameter_type: null
verification_tests:
- option_enabled: true
input_data: "apple\nbanana\ncherry\ndate\nelderberry\nfig\ngrape\nhoneydew\nkiwi\n"
pattern: "a"
expected_output: "1:apple\n2:banana\n4:date\n7:grape\n"
expected_error: ""
expected_status: 0
- option_enabled: false
input_data: "apple\nbanana\ncherry\ndate\nelderberry\nfig\ngrape\nhoneydew\nkiwi\n"
pattern: "a"
expected_output: "apple\nbanana\ndate\ngrape\n"
expected_error: ""
expected_status: 0
- anticipated_names:
- anticipated_name: "o"
deprecated: false
- anticipated_name: "only-matching"
deprecated: false
parameter_type: null
verification_tests:
- option_enabled: true
input_data: "appleBananaCherry\n"
pattern: "Banana"
expected_output: "Banana\n"
expected_error: ""
expected_status: 0
- option_enabled: false
input_data: "appleBananaCherry\n"
pattern: "Banana"
expected_output: "appleBananaCherry\n"
expected_error: ""
expected_status: 0
- anticipated_names:
- anticipated_name: "q"
deprecated: false
- anticipated_name: "quiet"
deprecated: false
- anticipated_name: "silent"
deprecated: false
- anticipated_name: "s"
deprecated: true
parameter_type: null
verification_tests:
- option_enabled: true
input_data: "fred\n"
pattern: "george"
expected_output: ""
expected_error: ""
expected_status: 1
- option_enabled: true
input_data: "fred\n"
pattern: "fred"
expected_output: ""
expected_error: ""
expected_status: 0
- option_enabled: false
input_data: "fred\n"
pattern: "george"
expected_output: ""
expected_error: ""
expected_status: 1
- option_enabled: false
input_data: "fred\n"
pattern: "fred"
expected_output: "fred\n"
expected_error: ""
expected_status: 0
- anticipated_names:
- anticipated_name: "v"
deprecated: false
- anticipated_name: "invert-match"
deprecated: false
parameter_type: null
verification_tests:
- option_enabled: true
input_data: "abc123\nDEF456\n"
pattern: "c12"
expected_output: "abc123\n"
expected_error: ""
expected_status: 0
- option_enabled: false
input_data: "abc123\nDEF456\n"
pattern: "c12"
expected_output: "DEF456\n"
expected_error: ""
expected_status: 0
- option_enabled: true
input_data: "abc123\nDEFc124\n"
pattern: "c12"
expected_output: ""
expected_error: ""
expected_status: 1
- option_enabled: false
input_data: "abc123\nDEF456\n"
pattern: "xyz"
expected_output: ""
expected_error: ""
expected_status: 1
known_command_names:
- name: "grep"
deprecated: false
- name: "egrep"
deprecated: true
- name: "fgrep"
deprecated: true
- name: "pcregrep"
deprecated: false
runtime_detection_data:
```
@nobitakuroku
Copy link

test for grep on the target system

you can only test for things on the build system

grep should have had detection routines long ago, but I haven’t found anything

perhaps you did not search well enough.
google "autoconf grep", and there'll be a link to the autoconf manual subsection titled "Particular Programs". (alternatively, use info autoconf on a linux shell to the same effect)

find all grep variations

that's not what autoconf is for. it find one variation, as documentated. The point is to find some reasonably-well-behaving grep from a fixed list of commonly used names so that a Makefile or something can do text filtering later on when you run make.

But these days, no one practically bothers with AC_PROG_GREP; I would argue most people just expect grep to be there, and to be POSIX-conforming.

@loren-osborn
Copy link
Author

test for grep on the target system

you can only test for things on the build system

yes, exactly. My goal was to test for things for the ShellSpec test suite can use to test the build system (which has its own test suite included)

grep should have had detection routines long ago, but I haven’t found anything

perhaps you did not search well enough. google "autoconf grep", and there'll be a link to the autoconf manual subsection titled "Particular Programs". (alternatively, use info autoconf on a linux shell to the same effect)

this looks very helpful. I don’t always search effectively for what I’m looking for… I think I was searching for “autotools grep” and it returned a list of auto mechanic videos… lol

find all grep variations

that's not what autoconf is for. it find one variation, as documentated. The point is to find some reasonably-well-behaving grep from a fixed list of commonly used names so that a Makefile or something can do text filtering later on when you run make.

I think my intent was a bit unclear here. My goal was for the developer to specify that they needed ERE support, or PCRE support, or something like that, the goal was for configure would search for the first non-deprecated command with the requested functionality, (or a deprecated one if that’s the only one available)… My only thoughts on detecting more than one version was different parts of the code requested different capabilities, it seemed like it might have been less complicated that the different requested capabilities ended up with different detected binaries… not really a requirement… just more my guess at the “simplest” implementation

But these days, no one practically bothers with AC_PROG_GREP; I would argue most people just expect grep to be there, and to be POSIX-conforming.

it wasn’t so much detecting grep itself, and more: does grep support xyz… did you look at any of the sample detection tests I spec’s out in the YAML file?

Thank you very much. Your response has been very helpful!

@nobitakuroku
Copy link

My goal was for the developer to specify that they needed ERE support

Same page, AC_CHECK_EGREP.

did you look at any of the sample detection tests I spec’s out in the YAML file?

yaml only declares a data structure, but you still need something to interpret that and run grep in various ways. that is overly complicated. nor is there a need to run so many patterns, let alone such overly complicated patterns. AC_PROG_EGREP 's pattern is in essence just

echo a | grep -E "(a|b)"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment