Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clap derive #1067

Merged
merged 28 commits into from
Nov 1, 2022
Merged

Clap derive #1067

merged 28 commits into from
Nov 1, 2022

Conversation

tmccombs
Copy link
Collaborator

Switch to using clap-derive for parsing command line arguments. For most options, this removes some boilerplate.

Open questions:

  • does this hurt performance?
  • does this hurt executable size?
  • Should the completions generation feature be turned off by default, and we just build with that feature for generating the completions?

@sharkdp
Copy link
Owner

sharkdp commented Jul 25, 2022

I've been using this branch locally and just found this by chance:

fd -l
ls: invalid argument 'Auto' for '--color'
Valid arguments are:
  - 'always', 'yes', 'force'
  - 'never', 'no', 'none'
  - 'auto', 'tty', 'if-tty'
Try 'ls --help' for more information.

Debugging with strace shows me that fd executes:

execve("/usr/bin/ls", ["ls", "-l", "-h", "-d", "--color=Auto", …

where the capitalization of Auto is wrong.

src/cli.rs Outdated Show resolved Hide resolved
src/cli.rs Outdated Show resolved Hide resolved
src/cli.rs Outdated Show resolved Hide resolved
src/cli.rs Outdated Show resolved Hide resolved
src/cli.rs Outdated Show resolved Hide resolved
src/cli.rs Outdated Show resolved Hide resolved
src/cli.rs Outdated
Comment on lines 676 to 652
.grouped_values_of("exec")
.map(CommandSet::new)
.or_else(|| {
matches
.grouped_values_of("exec-batch")
.map(CommandSet::new_batch)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

side note: Oh, didn't know you use this feature. No one has really been stepping up to advocate for it and finish it up

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there anything I could do to help finish it?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The linked tracking issue mentions the remaining work.

src/cli.rs Outdated Show resolved Hide resolved
Because that is what is needed by clap 2
Copy link

@epage epage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, I'd say the derive code looks idiomatic at this point. I have not done any comparison with the builder code to ensure consistency or done any manual testing to verify it. I have also not examined any of the changes unrelated to switching to derive (like completions) or refactors of the business logic in support of the derive.

src/cli.rs Outdated Show resolved Hide resolved
src/cli.rs Outdated Show resolved Hide resolved
src/cli.rs Outdated Show resolved Hide resolved
src/cli.rs Outdated
Comment on lines 583 to 597
// TODO: windows?
#[cfg(feature = "completions")]
fn guess_shell() -> anyhow::Result<Shell> {
let env_shell = std::env::var_os("SHELL").map(PathBuf::from);
let shell = env_shell
.as_ref()
.and_then(|s| s.file_name())
.and_then(|s| s.to_str())
.ok_or_else(|| anyhow!("Unable to get shell from environment"))?;
shell
.parse::<Shell>()
.map_err(|_| anyhow!("Unknown shell {}", shell))
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

side note: Huh, this might be neat to have in clap_complete

src/cli.rs Outdated Show resolved Hide resolved
Comment on lines +615 to +618
#[derive(Copy, Clone, PartialEq, Eq, Debug, ValueEnum)]
pub enum ColorWhen {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

side note: we should implement ValueEnum for clap's version of this enum

@sharkdp sharkdp mentioned this pull request Sep 11, 2022
This makes the definition of arguments to fd a little more ergonomic,
and makes it easier to insure the types for the arguments are consitent.
Make it more like it used to be.
@@ -0,0 +1,37 @@
PROFILE=release
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I liked that we could previously generate everything with a cargo build. Is there a specific reason for switching to a Makefile?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See discussion here about generating completions in the build.rs file.

Basically, getting this to work with generating the completions from build.rs was getting complicated. So I needed a way to do something after the build finished. There's probably a cargo plugin that could do this (for example https://github.com/phil-opp/cargo-post), but a makefile seemed like the most straighforward way to do it, and probably wouldn't require installing anything new.

src/walk.rs Outdated Show resolved Hide resolved
@sharkdp
Copy link
Owner

sharkdp commented Oct 17, 2022

Awesome work, @tmccombs. Thank you very much! Do we have any before and after comparisons of the -h and --help texts? It would be great to verify that nothing has been lost in the process.

@tmccombs
Copy link
Collaborator Author

I did look at a diff of the before and after for -h and --help, and there were no significant changes, but that was before upgrading to clap 4.0 , and making some other changes, so it's probably worth doing again.

@tmccombs
Copy link
Collaborator Author

tmccombs commented Oct 20, 2022

Diffs for help outputs:

NOTE: the identation appears to be less. in the new output. so I ignore whitespace in the diffs to avoid every line showing up as different

-h:

❯ diff -uw short-help-old.txt short-help.txt                                                                                                                                                            at 00:09:00 
--- short-help-old.txt	2022-10-20 00:04:54.634971667 -0600
+++ short-help.txt	2022-10-20 00:09:00.182465491 -0600
@@ -1,13 +1,10 @@
-fd 8.4.0
+Usage: fd [OPTIONS] [PATTERN] [PATH]...
 
-USAGE:
-    fd [OPTIONS] [--] [pattern] [path]...
+Arguments:
+  [PATTERN]  the search pattern (a regular expression, unless '--glob' is used; optional)
+  [PATH]...  the root directories for the filesystem search (optional)
 
-ARGS:
-    <pattern>    the search pattern (a regular expression, unless '--glob' is used; optional)
-    <path>...    the root directory for the filesystem search (optional)
-
-OPTIONS:
+Options:
     -H, --hidden                       Search hidden files and directories
     -I, --no-ignore                    Do not respect .(git|fd)ignore files
     -s, --case-sensitive               Case-sensitive search (default: smart case)
@@ -24,12 +21,13 @@
     -x, --exec <cmd>...                Execute a command for each search result
     -X, --exec-batch <cmd>...          Execute a command with all search results at once
     -E, --exclude <pattern>            Exclude entries that match the given glob pattern
-    -c, --color <when>                 When to use colors: never, *auto*, always
+  -c, --color <when>               When to use colors [default: auto] [possible values: auto,
+                                   always, never]
     -S, --size <size>                  Limit results based on the size of files
         --changed-within <date|dur>    Filter by file modification time (newer than)
         --changed-before <date|dur>    Filter by file modification time (older than)
     -o, --owner <user:group>           Filter by owning user and/or group
-    -h, --help                         Print help information
+  -h, --help                       Print help information (use `--help` for more detail)
     -V, --version                      Print version information
 
 Note: `fd -h` prints a short and concise overview while `fd --help` gives all details.

--help:

--- long-help-old.txt	2022-10-20 00:04:44.594938312 -0600
+++ long-help.txt	2022-10-20 00:30:52.887061119 -0600
@@ -1,85 +1,111 @@
-fd 8.4.0
+Usage: fd [OPTIONS] [PATTERN] [PATH]...
 
-USAGE:
-    fd [OPTIONS] [--] [pattern] [path]...
+Arguments:
+  [PATTERN]
+          the search pattern (a regular expression, unless '--glob' is used; optional)
 
-ARGS:
-    <pattern>
             the search pattern which is either a regular expression (default) or a glob pattern (if
             --glob is used). If no pattern has been specified, every entry is considered a match. If
             your pattern starts with a dash (-), make sure to pass '--' first, or it will be
             considered as a flag (fd -- '-foo').
 
-    <path>...
-            The directory where the filesystem search is rooted (optional). If omitted, search the
-            current working directory.
+  [PATH]...
+          the root directories for the filesystem search (optional)
 
-OPTIONS:
+          The directories where the filesystem search is rooted. If omitted, search the current
+          working directory.
+
+Options:
     -H, --hidden
+          Search hidden files and directories
+          
             Include hidden directories and files in the search results (default: hidden files and
-            directories are skipped). Files and directories are considered to be hidden if their
-            name starts with a `.` sign (dot). The flag can be overridden with --no-hidden.
+          directories are skipped). Files and directories are considered to be hidden if their name
+          starts with a `.` sign (dot). The flag can be overridden with --no-hidden.
 
     -I, --no-ignore
+          Do not respect .(git|fd)ignore files
+          
             Show search results from files and directories that would otherwise be ignored by
             '.gitignore', '.ignore', '.fdignore', or the global ignore file. The flag can be
             overridden with --ignore.
 
         --no-ignore-vcs
+          Do not respect .gitignore files
+          
             Show search results from files and directories that would otherwise be ignored by
             '.gitignore' files. The flag can be overridden with --ignore-vcs.
 
         --no-ignore-parent
+          Do not respect .(git|fd)ignore files in parent directories
+          
             Show search results from files and directories that would otherwise be ignored by
             '.gitignore', '.ignore', or '.fdignore' files in parent directories.
 
-    -u, --unrestricted
+  -u, --unrestricted...
             Perform an unrestricted search, including ignored and hidden files. This is an alias for
-            '--no-ignore --hidden'.
+          '--no-ignore --hidden'
 
     -s, --case-sensitive
-            Perform a case-sensitive search. By default, fd uses case-insensitive searches, unless
-            the pattern contains an uppercase character (smart case).
+          Case-sensitive search (default: smart case)
+          
+          Perform a case-sensitive search. By default, fd uses case-insensitive searches, unless the
+          pattern contains an uppercase character (smart case).
 
     -i, --ignore-case
+          Case-insensitive search (default: smart case)
+          
             Perform a case-insensitive search. By default, fd uses case-insensitive searches, unless
             the pattern contains an uppercase character (smart case).
 
     -g, --glob
+          Glob-based search (default: regular expression)
+          
             Perform a glob-based search instead of a regular expression search.
 
         --regex
-            Perform a regular-expression based search (default). This can be used to override
-            --glob.
+          Perform a regular-expression based search (default). This can be used to override --glob
 
     -F, --fixed-strings
-            Treat the pattern as a literal string instead of a regular expression. Note that this
-            also performs substring comparison. If you want to match on an exact filename, consider
-            using '--glob'.
+          Treat the pattern as a literal string instead of a regular expression. Note that this also
+          performs substring comparison. If you want to match on an exact filename, consider using
+          '--glob'
 
     -a, --absolute-path
-            Shows the full path starting from the root as opposed to relative paths. The flag can be
+          Show absolute instead of relative paths
+          
+          Shows the full path starting with the root as opposed to relative paths. The flag can be
             overridden with --relative-path.
 
     -l, --list-details
+          Use a long listing format with file metadata
+          
             Use a detailed listing format like 'ls -l'. This is basically an alias for '--exec-batch
             ls -l' with some additional 'ls' options. This can be used to see more metadata, to show
             symlink targets and to achieve a deterministic sort order.
 
     -L, --follow
+          Follow symbolic links
+          
             By default, fd does not descend into symlinked directories. Using this flag, symbolic
-            links are also traversed. Flag can be overriden with --no-follow.
+          links are also traversed. Flag can be overridden with --no-follow.
 
     -p, --full-path
-            By default, the search pattern is only matched against the filename (or directory name).
-            Using this flag, the pattern is matched against the full (absolute) path. Example:
+          Search full abs. path (default: filename only)
+          
+          By default, the search pattern is only matched against the filename (or
+          directory name). Using this flag, the pattern is matched against the full
+          (absolute) path.
+          Example:
               fd --glob -p '**/.git/config'
 
     -0, --print0
             Separate search results by the null character (instead of newlines). Useful for piping
-            results to 'xargs'.
+          results to 'xargs'
 
     -d, --max-depth <depth>
+          Set maximum search depth (default: none)
+          
             Limit the directory traversal to a given depth. By default, there is no limit on the
             search depth.
 
@@ -89,14 +115,18 @@
 
         --exact-depth <depth>
             Only show search results at the exact given depth. This is an alias for '--min-depth
-            <depth> --max-depth <depth>'.
+          <depth> --max-depth <depth>'
 
         --prune
             Do not traverse into directories that match the search criteria. If you want to exclude
-            specific directories, use the '--exclude=…' option.
+          specific directories, use the '--exclude=…' option
 
     -t, --type <filetype>
+          Filter by type: file (f), directory (d), symlink (l),
+          executable (x), empty (e), socket (s), pipe (p)
+          
             Filter the search by type:
+          
               'f' or 'file':         regular files
               'd' or 'directory':    directories
               'l' or 'symlink':      symbolic links
@@ -106,13 +136,15 @@
               'x' or 'executable':   executables
               'e' or 'empty':        empty files or directories
             
-            This option can be specified more than once to include multiple file types. Searching
-            for '--type file --type symlink' will show both regular files as well as symlinks. Note
-            that the 'executable' and 'empty' filters work differently: '--type executable' implies
-            '--type file' by default. And '--type empty' searches for empty files and directories,
-            unless either '--type file' or '--type directory' is specified in addition.
+          This option can be specified more than once to include multiple file types.
+          Searching for '--type file --type symlink' will show both regular files as
+          well as symlinks. Note that the 'executable' and 'empty' filters work differently:
+          '--type executable' implies '--type file' by default. And '--type empty' searches
+          for empty files and directories, unless either '--type file' or '--type directory'
+          is specified in addition.
             
             Examples:
+          
               - Only search for files:
                   fd --type file …
                   fd -tf …
@@ -127,13 +159,14 @@
                   fd -te -tf
               - Find empty directories:
                   fd --type empty --type directory
-                  fd -te -td
+                fd -te -td"
 
     -e, --extension <ext>
+          Filter by file extension
+          
             (Additionally) filter search results by their file extension. Multiple allowable file
-            extensions can be specified.
-            If you want to search for files without extension, you can use the regex '^[^.]+$' as a
-            normal search pattern.
+          extensions can be specified. If you want to search for files without extension, you can
+          use the regex '^[^.]+$' as a normal search pattern.
 
     -x, --exec <cmd>...
             Execute a command for each search result in parallel (use --threads=1 for sequential
@@ -185,13 +218,16 @@
                   fd -e rs -X wc -l
 
         --batch-size <size>
-            Maximum number of arguments to pass to the command given with -X. If the number of
-            results is greater than the given size, the command given with -X is run again with
-            remaining arguments. A batch size of zero means there is no limit (default), but note
-            that batching might still happen due to OS restrictions on the maximum length of command
-            lines.
+          Maximum number of arguments to pass to the command given with -X. If the number of results
+          is greater than the given size, the command given with -X is run again with remaining
+          arguments. A batch size of zero means there is no limit (default), but note that batching
+          might still happen due to OS restrictions on the maximum length of command lines
+          
+          [default: 0]
 
     -E, --exclude <pattern>
+          Exclude entries that match the given glob pattern
+          
             Exclude files/directories that match the given glob pattern. This overrides any other
             ignore logic. Multiple exclude patterns can be specified.
             
@@ -200,19 +236,29 @@
               --exclude node_modules
 
         --ignore-file <path>
-            Add a custom ignore-file in '.gitignore' format. These files have a low precedence.
+          Add a custom ignore-file in '.gitignore' format. These files have a low precedence
 
     -c, --color <when>
-            Declare when to use color for the pattern match output:
+          When to use colors
+          
               'auto':      show colors if the output goes to an interactive console (default)
               'never':     do not use colorized output
               'always':    always use colorized output
 
+          [default: auto]
+
+          Possible values:
+          - auto:   show colors if the output goes to an interactive console (default)
+          - always: always use colorized output
+          - never:  do not use colorized output
+
     -j, --threads <num>
             Set number of threads to use for searching & executing (default: number of available CPU
             cores)
 
     -S, --size <size>
+          Limit results based on the size of files
+          
             Limit results based on the size of files using the format <+-><NUM><UNIT>.
                '+': file size must be greater than or equal to this
                '-': file size must be less than or equal to this
@@ -231,74 +277,95 @@
                 'ti': tebibytes
 
         --changed-within <date|dur>
-            Filter results based on the file modification time. The argument can be provided as a
-            specific point in time (YYYY-MM-DD HH:MM:SS) or as a duration (10h, 1d, 35min). If the
-            time is not specified, it defaults to 00:00:00. '--change-newer-than' or '--newer' can
-            be used as aliases.
+          Filter by file modification time (newer than)
+          
+          The argument can be provided as a specific point in time (YYYY-MM-DD HH:MM:SS)
+          or as a duration (10h, 1d, 35min).
+          If the time is not specified, it defaults to 00:00:00.
+          '--change-newer-than' or '--newer' can be used as aliases.
             Examples:
                 --changed-within 2weeks
                 --change-newer-than '2018-10-27 10:00:00'
                 --newer 2018-10-27
 
         --changed-before <date|dur>
-            Filter results based on the file modification time. The argument can be provided as a
-            specific point in time (YYYY-MM-DD HH:MM:SS) or as a duration (10h, 1d, 35min).
+          Filter by file modification time (older than)
+          
+          The argument can be provided as a specific point in time (YYYY-MM-DD HH:MM:SS)
+          or as a duration (10h, 1d, 35min).
             '--change-older-than' or '--older' can be used as aliases.
+          
             Examples:
                 --changed-before '2018-10-27 10:00:00'
                 --change-older-than 2weeks
                 --older 2018-10-27
 
         --max-results <count>
+          Limit number of search results
+          
             Limit the number of search results to 'count' and quit immediately.
 
     -1
-            Limit the search to a single result and quit immediately. This is an alias for
-            '--max-results=1'.
+          Limit search to a single result and quit immediately
+          
+          This is an alias for '--max-results=1'.
 
     -q, --quiet
-            When the flag is present, the program does not print anything and will return with an
-            exit code of 0 if there is at least one match. Otherwise, the exit code will be 1.
+          Print nothing, exit code 0 if match found, 1 otherwise
+          
+          When the flag is present, the program does not print anything and will return with an exit
+          code of 0 if there is at least one match. Otherwise, the exit code will be 1.
+          
             '--has-results' can be used as an alias.
 
         --show-errors
-            Enable the display of filesystem errors for situations such as insufficient permissions
-            or dead symlinks.
+          Show filesystem errors
+          
+          Enable the display of filesystem errors for situations such as insufficient permissions or
+          dead symlinks.
 
         --base-directory <path>
             Change the current working directory of fd to the provided path. This means that search
-            results will be shown with respect to the given base path. Note that relative paths
-            which are passed to fd via the positional <path> argument or the '--search-path' option
-            will also be resolved relative to this directory.
+          results will be shown with respect to the given base path. Note that relative paths which
+          are passed to fd via the positional <path> argument or the '--search-path' option will
+          also be resolved relative to this directory
 
         --path-separator <separator>
-            Set the path separator to use when printing file paths. The default is the OS-specific
-            separator ('/' on Unix, '\' on Windows).
+          Set path separator when printing file paths Set the path separator to use when printing
+          file paths. The default is the OS-specific separator ('/' on Unix, '\\' on Windows)
+
+      --search-path <SEARCH_PATH>
+          Provides paths to search as an alternative to the positional <path> argument
 
-        --search-path <search-path>
-            Provide paths to search as an alternative to the positional <path> argument. Changes the
-            usage to `fd [OPTIONS] --search-path <path> --search-path <path2> [<pattern>]`
+          Changes the usage to `fd [OPTIONS] --search-path <path> --search-path <path2> [<pattern>]`
 
         --strip-cwd-prefix
+          strip './' prefix from -0/--print-0 output
+          
             By default, relative paths are prefixed with './' when -x/--exec, -X/--exec-batch, or
             -0/--print0 are given, to reduce the risk of a path starting with '-' being treated as a
             command line option. Use this flag to disable this behaviour.
 
     -o, --owner <user:group>
-            Filter files by their user and/or group. Format: [(user|uid)][:(group|gid)]. Either side
-            is optional. Precede either side with a '!' to exclude files instead.
+          Filter by owning user and/or group
+          
+          Format: [(user|uid)][:(group|gid)].
+          Either side is optional. Precede either side with a '!' to exclude files instead.
+          
             Examples:
                 --owner john
                 --owner :students
                 --owner '!john:students'
 
         --one-file-system
+          Do not descend into a different file system
+          
             By default, fd will traverse the file system tree as far as other options dictate. With
             this flag, fd ensures that it does not descend into a different file system than the one
             it started in. Comparable to the -mount or -xdev filters of find(1).
 
     -h, --help
-            Print help information
+          Print help information (use `-h` for a summary)
 
     -V, --version
             Print version information
             ```
             
             This is more significantly different. The biggest change is that with doc comments, the entire doc comment is included in the long help, while before, we sometimes had significantly different text in the short help than the long help. In some cases, it is reasonable to adjust the message to be less repetitive with both, or to remove the short help if we hide the option from the short help anyway. I've tried to fix most of these. For other's it is less clear how to make it less awkward. 

@epage would it be worth having an option in the derive api to exclude the first line of the doc comment from the long_help message?

@sharkdp
Copy link
Owner

sharkdp commented Oct 31, 2022

@tmccombs I would really like to merge this, but the additional inclusion of the short help text in the long help text descriptions is a bit unfortunate. Is there any way we can work around this?

src/cli.rs Outdated
Comment on lines 87 to 88
after_help = "Note: `fd -h` prints a short and concise overview while `fd --help` gives all \
details.",
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: clap has a changed description for -h and --help (see your diff output) which makes this after_help message somewhat superfluous.

@tmccombs
Copy link
Collaborator Author

Is there any way we can work around this?

Yeah. I think we can still pass in the short and long descriptions directly as attributes. It's a little less readable, but is probably ok for now. I'll put together a commit for that.

@sharkdp
Copy link
Owner

sharkdp commented Oct 31, 2022

Yeah. I think we can still pass in the short and long descriptions directly as attributes. It's a little less readable, but is probably ok for now. I'll put together a commit for that.

Maybe you can still leave the long help text as doc-comment and pass in the short one as an attribute? Or the other way around?

tmccombs and others added 2 commits November 1, 2022 02:30
In particular, we specifically use `long_help` instead of doc comments
because using  doc comments will always trim the "." off the end of the
first paragraph, and will include the short help as the first paragraph
of the full help.
@sharkdp sharkdp merged commit 13a47c3 into sharkdp:master Nov 1, 2022
@sharkdp
Copy link
Owner

sharkdp commented Nov 1, 2022

Thank you very much for your work and patience!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants