gh-90300: Reformat the Python CLI help output

[serhiy-storchaka]

• All lines are now shorter than 80 columns
• All wrapped lines use the same identation
• Remove empty lines between -X options
• Output only one empty line between sections in --help-all

#90300

• Issue: python --help output is too long #90300

[merwok]

Would you mind showing us the new output of python --help-all for comparison?

[merwok]

The previous spacing was following the style of spacing for short options (see lines just above)

[serhiy-storchaka]

No, it was not. All colons were at the same column, except for long option --check-hash-based-pycs, in which case the colon was in the first available column. And the same pattern was used for earlier environment variables.

[merwok]

Well it was my intent when I added them! Maybe you disagree on the result 🙂
When I say «following», I meant that all short options have colons at the same column, check-hash-based-pycs is special and uses a new line (with the help text aligned to the help texts for short options), and this new group of long options has colons at the same column (but not the same at the short options, that doesn’t work).

[serhiy-storchaka]

Old output:

usage: (null) [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options (and corresponding environment variables):
-b     : issue warnings about str(bytes_instance), str(bytearray_instance)
         and comparing bytes/bytearray with str. (-bb: issue errors)
-B     : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d     : turn on parser debugging output (for experts only, only works on
         debug builds); also PYTHONDEBUG=x
-E     : ignore PYTHON* environment variables (such as PYTHONPATH)
-h     : print this help message and exit (also -? or --help)
-i     : inspect interactively after running script; forces a prompt even
         if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-I     : isolate Python from the user's environment (implies -E and -s)
-m mod : run library module as a script (terminates option list)
-O     : remove assert and __debug__-dependent statements; add .opt-1 before
         .pyc extension; also PYTHONOPTIMIZE=x
-OO    : do -O changes and also discard docstrings; add .opt-2 before
         .pyc extension
-P     : don't prepend a potentially unsafe path to sys.path
-q     : don't print version and copyright messages on interactive startup
-s     : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S     : don't imply 'import site' on initialization
-u     : force the stdout and stderr streams to be unbuffered;
         this option has no effect on stdin; also PYTHONUNBUFFERED=x
-v     : verbose (trace import statements); also PYTHONVERBOSE=x
         can be supplied multiple times to increase verbosity
-V     : print the Python version number and exit (also --version)
         when given twice, print more information about the build
-W arg : warning control; arg is action:message:category:module:lineno
         also PYTHONWARNINGS=arg
-x     : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option
--check-hash-based-pycs always|default|never:
         control how Python invalidates hash-based .pyc files
--help-env      : print help about Python environment variables and exit
--help-xoptions : print help about implementation-specific -X options and exit
--help-all      : print complete help information and exit
Arguments:
file   : program read from script file
-      : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]


Environment variables that change behavior:
PYTHONSTARTUP: file executed on interactive startup (no default)
PYTHONPATH   : ':'-separated list of directories prefixed to the
               default module search path.  The result is sys.path.
PYTHONSAFEPATH: don't prepend a potentially unsafe path to sys.path.
PYTHONHOME   : alternate <prefix> directory (or <prefix>:<exec_prefix>).
               The default module search path uses <prefix>/lib/pythonX.X.
PYTHONPLATLIBDIR : override sys.platlibdir.
PYTHONCASEOK : ignore case in 'import' statements (Windows).
PYTHONUTF8: if set to 1, enable the UTF-8 mode.
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.
PYTHONFAULTHANDLER: dump the Python traceback on fatal errors.
PYTHONHASHSEED: if this variable is set to 'random', a random value is used
   to seed the hashes of str and bytes objects.  It can also be set to an
   integer in the range [0,4294967295] to get hash values with a
   predictable seed.
PYTHONMALLOC: set the Python memory allocators and/or install debug hooks
   on Python memory allocators. Use PYTHONMALLOC=debug to install debug
   hooks.
PYTHONCOERCECLOCALE: if this variable is set to 0, it disables the locale
   coercion behavior. Use PYTHONCOERCECLOCALE=warn to request display of
   locale coercion and locale compatibility warnings on stderr.
PYTHONBREAKPOINT: if this variable is set to 0, it disables the default
   debugger. It can be set to the callable of your debugger of choice.
PYTHONDEVMODE: enable the development mode.
PYTHONPYCACHEPREFIX: root directory for bytecode cache (pyc) files.
PYTHONWARNDEFAULTENCODING: enable opt-in EncodingWarning for 'encoding=None'.
PYTHONNODEBUGRANGES: If this variable is set, it disables the inclusion of the 
   tables mapping extra location information (end line, start column offset 
   and end column offset) to every instruction in code objects. This is useful 
   when smaller code objects and pyc files are desired as well as suppressing the 
   extra visual location indicators when the interpreter displays tracebacks.
These variables have equivalent command-line parameters (see --help for details):
PYTHONDEBUG             : enable parser debug mode (-d)
PYTHONDONTWRITEBYTECODE : don't write .pyc files (-B)
PYTHONINSPECT           : inspect interactively after running script (-i)
PYTHONNOUSERSITE        : disable user site directory (-s)
PYTHONOPTIMIZE          : enable level 1 optimizations (-O)
PYTHONUNBUFFERED        : disable stdout/stderr buffering (-u)
PYTHONVERBOSE           : trace import statements (-v)
PYTHONWARNINGS=arg      : warning control (-W arg)


The following implementation-specific options are available:

-X faulthandler: enable faulthandler

-X showrefcount: output the total reference count and number of used
    memory blocks when the program finishes or after each statement in the
    interactive interpreter. This only works on debug builds

-X tracemalloc: start tracing Python memory allocations using the
    tracemalloc module. By default, only the most recent frame is stored in a
    traceback of a trace. Use -X tracemalloc=NFRAME to start tracing with a
    traceback limit of NFRAME frames

-X importtime: show how long each import takes. It shows module name,
    cumulative time (including nested imports) and self time (excluding
    nested imports). Note that its output may be broken in multi-threaded
    application. Typical usage is python3 -X importtime -c 'import asyncio'

-X dev: enable CPython's "development mode", introducing additional runtime
    checks which are too expensive to be enabled by default. Effect of the
    developer mode:
       * Add default warning filter, as -W default
       * Install debug hooks on memory allocators: see the PyMem_SetupDebugHooks()
         C function
       * Enable the faulthandler module to dump the Python traceback on a crash
       * Enable asyncio debug mode
       * Set the dev_mode attribute of sys.flags to True
       * io.IOBase destructor logs close() exceptions

-X utf8: enable UTF-8 mode for operating system interfaces, overriding the default
    locale-aware mode. -X utf8=0 explicitly disables UTF-8 mode (even when it would
    otherwise activate automatically)

-X pycache_prefix=PATH: enable writing .pyc files to a parallel tree rooted at the
    given directory instead of to the code tree

-X warn_default_encoding: enable opt-in EncodingWarning for 'encoding=None'

-X no_debug_ranges: disable the inclusion of the tables mapping extra location 
   information (end line, start column offset and end column offset) to every 
   instruction in code objects. This is useful when smaller code objects and pyc 
   files are desired as well as suppressing the extra visual location indicators 
   when the interpreter displays tracebacks.

-X frozen_modules=[on|off]: whether or not frozen modules should be used.
   The default is "on" (or "off" if you are running a local build).

New output:

usage: (null) [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options (and corresponding environment variables):
-b     : issue warnings about str(bytes_instance), str(bytearray_instance)
         and comparing bytes/bytearray with str. (-bb: issue errors)
-B     : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d     : turn on parser debugging output (for experts only, only works on
         debug builds); also PYTHONDEBUG=x
-E     : ignore PYTHON* environment variables (such as PYTHONPATH)
-h     : print this help message and exit (also -? or --help)
-i     : inspect interactively after running script; forces a prompt even
         if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-I     : isolate Python from the user's environment (implies -E and -s)
-m mod : run library module as a script (terminates option list)
-O     : remove assert and __debug__-dependent statements; add .opt-1 before
         .pyc extension; also PYTHONOPTIMIZE=x
-OO    : do -O changes and also discard docstrings; add .opt-2 before
         .pyc extension
-P     : don't prepend a potentially unsafe path to sys.path
-q     : don't print version and copyright messages on interactive startup
-s     : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S     : don't imply 'import site' on initialization
-u     : force the stdout and stderr streams to be unbuffered;
         this option has no effect on stdin; also PYTHONUNBUFFERED=x
-v     : verbose (trace import statements); also PYTHONVERBOSE=x
         can be supplied multiple times to increase verbosity
-V     : print the Python version number and exit (also --version)
         when given twice, print more information about the build
-W arg : warning control; arg is action:message:category:module:lineno
         also PYTHONWARNINGS=arg
-x     : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option
--check-hash-based-pycs always|default|never:
         control how Python invalidates hash-based .pyc files
--help-env: print help about Python environment variables and exit
--help-xoptions: print help about implementation-specific -X options and exit
--help-all: print complete help information and exit
Arguments:
file   : program read from script file
-      : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]

Environment variables that change behavior:
PYTHONSTARTUP: file executed on interactive startup (no default)
PYTHONPATH: ':'-separated list of directories prefixed to the
         default module search path.  The result is sys.path.
PYTHONSAFEPATH: don't prepend a potentially unsafe path to sys.path.
PYTHONHOME: alternate <prefix> directory (or <prefix>:<exec_prefix>).
         The default module search path uses <prefix>/lib/pythonX.X.
PYTHONPLATLIBDIR: override sys.platlibdir.
PYTHONCASEOK: ignore case in 'import' statements (Windows).
PYTHONUTF8: if set to 1, enable the UTF-8 mode.
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.
PYTHONFAULTHANDLER: dump the Python traceback on fatal errors.
PYTHONHASHSEED: if this variable is set to 'random', a random value is used
         to seed the hashes of str and bytes objects.  It can also be set to
         an integer in the range [0,4294967295] to get hash values with a
         predictable seed.
PYTHONMALLOC: set the Python memory allocators and/or install debug hooks
         on Python memory allocators. Use PYTHONMALLOC=debug to install debug
         hooks.
PYTHONCOERCECLOCALE: if this variable is set to 0, it disables the locale
         coercion behavior. Use PYTHONCOERCECLOCALE=warn to request display
         of locale coercion and locale compatibility warnings on stderr.
PYTHONBREAKPOINT: if this variable is set to 0, it disables the default
         debugger. It can be set to the callable of your debugger of choice.
PYTHONDEVMODE: enable the development mode.
PYTHONPYCACHEPREFIX: root directory for bytecode cache (pyc) files.
PYTHONWARNDEFAULTENCODING: enable opt-in EncodingWarning for 'encoding=None'.
PYTHONNODEBUGRANGES: If this variable is set, it disables the inclusion of
         the tables mapping extra location information (end line, start column
         offset and end column offset) to every instruction in code objects.
         This is useful when smaller code objects and pyc files are desired as
         well as suppressing the extra visual location indicators when the
         interpreter displays tracebacks.
These variables have equivalent command-line options (see --help for details):
PYTHONDEBUG: enable parser debug mode (-d)
PYTHONDONTWRITEBYTECODE: don't write .pyc files (-B)
PYTHONINSPECT: inspect interactively after running script (-i)
PYTHONNOUSERSITE: disable user site directory (-s)
PYTHONOPTIMIZE: enable level 1 optimizations (-O)
PYTHONUNBUFFERED: disable stdout/stderr buffering (-u)
PYTHONVERBOSE: trace import statements (-v)
PYTHONWARNINGS=arg: warning control (-W arg)

The following implementation-specific options are available:
-X faulthandler: enable faulthandler
-X showrefcount: output the total reference count and number of used
         memory blocks when the program finishes or after each statement in
         the interactive interpreter. This only works on debug builds
-X tracemalloc: start tracing Python memory allocations using the
         tracemalloc module. By default, only the most recent frame is stored
         in a traceback of a trace. Use -X tracemalloc=NFRAME to start tracing
         with a traceback limit of NFRAME frames
-X importtime: show how long each import takes. It shows module name,
         cumulative time (including nested imports) and self time (excluding
         nested imports). Note that its output may be broken in multi-threaded
         application.
         Typical usage is python3 -X importtime -c 'import asyncio'
-X dev : enable CPython's "development mode", introducing additional runtime
         checks which are too expensive to be enabled by default. Effect of
         the developer mode:
            * Add default warning filter, as -W default
            * Install debug hooks on memory allocators: see the
              PyMem_SetupDebugHooks() C function
            * Enable the faulthandler module to dump the Python traceback on
              a crash
            * Enable asyncio debug mode
            * Set the dev_mode attribute of sys.flags to True
            * io.IOBase destructor logs close() exceptions
-X utf8: enable UTF-8 mode for operating system interfaces, overriding the
         default locale-aware mode. -X utf8=0 explicitly disables UTF-8 mode
         (even when it would otherwise activate automatically)
-X pycache_prefix=PATH: enable writing .pyc files to a parallel tree rooted
         at the given directory instead of to the code tree
-X warn_default_encoding: enable opt-in EncodingWarning for 'encoding=None'
-X no_debug_ranges: disable the inclusion of the tables mapping extra location
         information (end line, start column offset and end column offset) to
         every instruction in code objects. This is useful when smaller code
         objects and pyc files are desired as well as suppressing the extra
         visual location indicators when the interpreter displays tracebacks.
-X frozen_modules=[on|off]: whether or not frozen modules should be used.
         The default is "on" (or "off" if you are running a local build).

[serhiy-storchaka]

Would you prefer the following output?

usage: (null) [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options (and corresponding environment variables):
-b             : issue warnings about str(bytes_instance),
                 str(bytearray_instance) and comparing bytes/bytearray with
                 str. (-bb: issue errors)
-B             : don't write .pyc files on import; also
                 PYTHONDONTWRITEBYTECODE=x
-c cmd         : program passed in as string (terminates option list)
-d             : turn on parser debugging output (for experts only, only works
                 on debug builds); also PYTHONDEBUG=x
-E             : ignore PYTHON* environment variables (such as PYTHONPATH)
-h             : print this help message and exit (also -? or --help)
-i             : inspect interactively after running script; forces a prompt
                 even if stdin does not appear to be a terminal; also
                 PYTHONINSPECT=x
-I             : isolate Python from the user's environment (implies -E and -s)
-m mod         : run library module as a script (terminates option list)
-O             : remove assert and __debug__-dependent statements; add .opt-1
                 before .pyc extension; also PYTHONOPTIMIZE=x
-OO            : do -O changes and also discard docstrings; add .opt-2 before
                 .pyc extension
-P             : don't prepend a potentially unsafe path to sys.path
-q             : don't print version and copyright messages on interactive
                 startup
-s             : don't add user site directory to sys.path; also
                 PYTHONNOUSERSITE
-S             : don't imply 'import site' on initialization
-u             : force the stdout and stderr streams to be unbuffered;
                 this option has no effect on stdin; also PYTHONUNBUFFERED=x
-v             : verbose (trace import statements); also PYTHONVERBOSE=x
                 can be supplied multiple times to increase verbosity
-V             : print the Python version number and exit (also --version)
                 when given twice, print more information about the build
-W arg         : warning control; arg is action:message:category:module:lineno
                 also PYTHONWARNINGS=arg
-x             : skip first line of source, allowing use of non-Unix forms of
                 #!cmd
-X opt         : set implementation-specific option
--check-hash-based-pycs always|default|never:
                 control how Python invalidates hash-based .pyc files
--help-env     : print help about Python environment variables and exit
--help-xoptions: print help about implementation-specific -X options and exit
--help-all     : print complete help information and exit
Arguments:
file           : program read from script file
-              : program read from stdin (default; interactive mode if a tty)
arg ...        : arguments passed to program in sys.argv[1:]

Environment variables that change behavior:
PYTHONSTARTUP  : file executed on interactive startup (no default)
PYTHONPATH     : ':'-separated list of directories prefixed to the
                 default module search path.  The result is sys.path.
PYTHONSAFEPATH : don't prepend a potentially unsafe path to sys.path.
PYTHONHOME     : alternate <prefix> directory (or <prefix>:<exec_prefix>).
                 The default module search path uses <prefix>/lib/pythonX.X.
PYTHONPLATLIBDIR: override sys.platlibdir.
PYTHONCASEOK   : ignore case in 'import' statements (Windows).
PYTHONUTF8     : if set to 1, enable the UTF-8 mode.
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.
PYTHONFAULTHANDLER: dump the Python traceback on fatal errors.
PYTHONHASHSEED : if this variable is set to 'random', a random value is used
                 to seed the hashes of str and bytes objects.  It can also be
                 set to an integer in the range [0,4294967295] to get hash
                 values with a predictable seed.
PYTHONMALLOC   : set the Python memory allocators and/or install debug hooks
                 on Python memory allocators. Use PYTHONMALLOC=debug to
                 install debug hooks.
PYTHONCOERCECLOCALE: if this variable is set to 0, it disables the locale
                 coercion behavior. Use PYTHONCOERCECLOCALE=warn to request
                 display of locale coercion and locale compatibility warnings
                 on stderr.
PYTHONBREAKPOINT: if this variable is set to 0, it disables the default
                 debugger. It can be set to the callable of your debugger of
                 choice.
PYTHONDEVMODE  : enable the development mode.
PYTHONPYCACHEPREFIX: root directory for bytecode cache (pyc) files.
PYTHONWARNDEFAULTENCODING: enable opt-in EncodingWarning for 'encoding=None'.
PYTHONNODEBUGRANGES: If this variable is set, it disables the inclusion of
                 the tables mapping extra location information (end line,
                 start column offset and end column offset) to every
                 instruction in code objects. This is useful when smaller code
                 objects and pyc files are desired as well as suppressing the
                 extra visual location indicators when the interpreter
                 displays tracebacks.
These variables have equivalent command-line options (see --help for details):
PYTHONDEBUG    : enable parser debug mode (-d)
PYTHONDONTWRITEBYTECODE: don't write .pyc files (-B)
PYTHONINSPECT  : inspect interactively after running script (-i)
PYTHONNOUSERSITE: disable user site directory (-s)
PYTHONOPTIMIZE : enable level 1 optimizations (-O)
PYTHONUNBUFFERED: disable stdout/stderr buffering (-u)
PYTHONVERBOSE  : trace import statements (-v)
PYTHONWARNINGS=arg: warning control (-W arg)

The following implementation-specific options are available:
-X faulthandler: enable faulthandler
-X showrefcount: output the total reference count and number of used
                 memory blocks when the program finishes or after each
                 statement in the interactive interpreter. This only works on
                 debug builds
-X tracemalloc : start tracing Python memory allocations using the
                 tracemalloc module. By default, only the most recent frame is
                 stored in a traceback of a trace. Use -X tracemalloc=NFRAME
                 to start tracing with a traceback limit of NFRAME frames
-X importtime  : show how long each import takes. It shows module name,
                 cumulative time (including nested imports) and self time
                 (excluding nested imports). Note that its output may be
                 broken in multi-threaded application.
                 Typical usage is python3 -X importtime -c 'import asyncio'
-X dev         : enable CPython's "development mode", introducing additional
                 runtime checks which are too expensive to be enabled by
                 default. Effect of the developer mode:
                    * Add default warning filter, as -W default
                    * Install debug hooks on memory allocators: see the
                      PyMem_SetupDebugHooks() C function
                    * Enable the faulthandler module to dump the Python
                      traceback on a crash
                    * Enable asyncio debug mode
                    * Set the dev_mode attribute of sys.flags to True
                    * io.IOBase destructor logs close() exceptions
-X utf8        : enable UTF-8 mode for operating system interfaces, overriding
                 the default locale-aware mode. -X utf8=0 explicitly disables
                 UTF-8 mode (even when it would otherwise activate
                 automatically)
-X pycache_prefix=PATH: enable writing .pyc files to a parallel tree rooted
                 at the given directory instead of to the code tree
-X warn_default_encoding: enable opt-in EncodingWarning for 'encoding=None'
-X no_debug_ranges: disable the inclusion of the tables mapping extra location
                 information (end line, start column offset and end column
                 offset) to every instruction in code objects. This is useful
                 when smaller code objects and pyc files are desired as well
                 as suppressing the extra visual location indicators when the
                 interpreter displays tracebacks.
-X frozen_modules=[on|off]: whether or not frozen modules should be used.
                 The default is "on" (or "off" if you are running a local
                 build).

[serhiy-storchaka]

Added the example of the old output and collapse output in the comments above.

[merwok]

Alternate proposal with fewer changes to the -X options and envvars sections:

usage: (null) [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options (and corresponding environment variables):
-b     : issue warnings about str(bytes_instance), str(bytearray_instance)
         and comparing bytes/bytearray with str. (-bb: issue errors)
-B     : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d     : turn on parser debugging output (for experts only, only works on
         debug builds); also PYTHONDEBUG=x
-E     : ignore PYTHON* environment variables (such as PYTHONPATH)
-h     : print this help message and exit (also -? or --help)
-i     : inspect interactively after running script; forces a prompt even
         if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-I     : isolate Python from the user's environment (implies -E and -s)
-m mod : run library module as a script (terminates option list)
-O     : remove assert and __debug__-dependent statements; add .opt-1 before
         .pyc extension; also PYTHONOPTIMIZE=x
-OO    : do -O changes and also discard docstrings; add .opt-2 before
         .pyc extension
-P     : don't prepend a potentially unsafe path to sys.path
-q     : don't print version and copyright messages on interactive startup
-s     : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S     : don't imply 'import site' on initialization
-u     : force the stdout and stderr streams to be unbuffered;
         this option has no effect on stdin; also PYTHONUNBUFFERED=x
-v     : verbose (trace import statements); also PYTHONVERBOSE=x
         can be supplied multiple times to increase verbosity
-V     : print the Python version number and exit (also --version)
         when given twice, print more information about the build
-W arg : warning control; arg is action:message:category:module:lineno
         also PYTHONWARNINGS=arg
-x     : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option
--check-hash-based-pycs always|default|never:
         control how Python invalidates hash-based .pyc files
--help-env      : print help about Python environment variables and exit
--help-xoptions : print help about implementation-specific -X options and exit
--help-all      : print complete help information and exit
Arguments:
file   : program read from script file
-      : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]

Environment variables that change behavior:
PYTHONSTARTUP: file executed on interactive startup (no default).
PYTHONPATH   : ':'-separated list of directories prefixed to the
               default module search path (sys.path).
PYTHONSAFEPATH: don't prepend a potentially unsafe path to sys.path.
PYTHONHOME   : alternate <prefix> directory (or <prefix>:<exec_prefix>).
               The default module search path uses <prefix>/lib/pythonX.X.
PYTHONPLATLIBDIR : override sys.platlibdir.
PYTHONCASEOK : ignore case in 'import' statements (Windows).
PYTHONUTF8   : if set to 1, enable the UTF-8 mode.
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.
PYTHONFAULTHANDLER: dump the Python traceback on fatal errors.
PYTHONHASHSEED: if this variable is set to 'random', a random value is used
   to seed the hashes of str and bytes objects.  It can also be set to an
   integer in the range [0,4294967295] to get hash values with a
   predictable seed.
PYTHONMALLOC : set the Python memory allocators and/or install debug hooks
   on Python memory allocators. Use PYTHONMALLOC=debug to install debug
   hooks.
PYTHONCOERCECLOCALE: if this variable is set to 0, it disables the locale
   coercion behavior. Use PYTHONCOERCECLOCALE=warn to request display of
   locale coercion and locale compatibility warnings on stderr.
PYTHONBREAKPOINT: if this variable is set to 0, it disables the default
   debugger. It can be set to the callable of your debugger of choice.
PYTHONDEVMODE: enable the development mode.
PYTHONPYCACHEPREFIX: root directory for bytecode cache (pyc) files.
PYTHONWARNDEFAULTENCODING: enable EncodingWarning for 'encoding=None'.
PYTHONNODEBUGRANGES: If this variable is set, it disables the inclusion of
   the tables mapping extra location information (end line, start column
   offset and end column offset) to every instruction in code objects. This
   is useful when smaller code objects and pyc files are desired as well as
   suppressing the extra visual location indicators when the interpreter
   displays tracebacks.
These variables have equivalent command-line options (see --help):
PYTHONDEBUG        : enable parser debug mode (-d)
PYTHONDONTWRITEBYTECODE : don't write .pyc files (-B)
PYTHONINSPECT      : inspect interactively after running script (-i)
PYTHONNOUSERSITE   : disable user site directory (-s)
PYTHONOPTIMIZE     : enable level 1 optimizations (-O)
PYTHONUNBUFFERED   : disable stdout/stderr buffering (-u)
PYTHONVERBOSE      : trace import statements (-v)
PYTHONWARNINGS=arg : warning control (-W arg)

The following implementation-specific options are available:
-X faulthandler: enable faulthandler
-X showrefcount: output the total reference count and number of used
    memory blocks when the program finishes or after each statement in the
    interactive interpreter. This only works on debug builds
-X tracemalloc: start tracing Python memory allocations using the
    tracemalloc module. By default, only the most recent frame is stored in a
    traceback of a trace. Use -X tracemalloc=NFRAME to start tracing with a
    traceback limit of NFRAME frames
-X importtime: show how long each import takes. It shows module name,
    cumulative time (including nested imports) and self time (excluding
    nested imports). Note that its output may be broken in multi-threaded
    application. Typical usage is python3 -X importtime -c 'import asyncio'
-X dev: enable CPython's "development mode", introducing additional runtime
    checks which are too expensive to be enabled by default. Effect of the
    developer mode:
       * Add default warning filter, as -W default
       * Install debug hooks on memory allocators: see the
         PyMem_SetupDebugHooks() C function
       * Enable the faulthandler module to dump the Python traceback on a
         crash
       * Enable asyncio debug mode
       * Set the dev_mode attribute of sys.flags to True
       * io.IOBase destructor logs close() exceptions
-X utf8: enable UTF-8 mode for operating system interfaces, overriding the
    default locale-aware mode. -X utf8=0 explicitly disables UTF-8 mode (even
    when it would otherwise activate automatically)
-X pycache_prefix=PATH: enable writing .pyc files to a parallel tree rooted at
    the given directory instead of to the code tree
-X warn_default_encoding: enable opt-in EncodingWarning for 'encoding=None'
-X no_debug_ranges: disable the inclusion of the tables mapping extra location
    information (end line, start column offset and end column offset) to every
    instruction in code objects. This is useful when smaller code objects and
    pyc files are desired as well as suppressing the extra visual location
    indicators when the interpreter displays tracebacks.
-X frozen_modules=[on|off]: whether or not frozen modules should be used.
    The default is "on" (or "off" if you are running a local build).

[ambv]

I like the output @serhiy-storchaka suggests in #93415 (comment) the most.

[serhiy-storchaka]

Alternate proposal with fewer changes to the -X options and envvars sections:

• The indentation of descriptions for --help-xxx options is inconsistent with indentations for other options (including the long one) and positional arguments.
• The indentation of descriptions for some (!) of envvars and -X options is even less than indentations for options. And it is inconsistent inside a group -- there is three different values for indentations for envvars.
• There is a space before colon for some long envvar names, and there is no space before colon for shorter envvar names.

I thing that indentations and positions of columns should be consistent at least inside every page. If not use the same position globally, we can increase it for envvars only:

usage: (null) [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options (and corresponding environment variables):
-b     : issue warnings about str(bytes_instance), str(bytearray_instance)
         and comparing bytes/bytearray with str. (-bb: issue errors)
-B     : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d     : turn on parser debugging output (for experts only, only works on
         debug builds); also PYTHONDEBUG=x
-E     : ignore PYTHON* environment variables (such as PYTHONPATH)
-h     : print this help message and exit (also -? or --help)
-i     : inspect interactively after running script; forces a prompt even
         if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-I     : isolate Python from the user's environment (implies -E and -s)
-m mod : run library module as a script (terminates option list)
-O     : remove assert and __debug__-dependent statements; add .opt-1 before
         .pyc extension; also PYTHONOPTIMIZE=x
-OO    : do -O changes and also discard docstrings; add .opt-2 before
         .pyc extension
-P     : don't prepend a potentially unsafe path to sys.path
-q     : don't print version and copyright messages on interactive startup
-s     : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S     : don't imply 'import site' on initialization
-u     : force the stdout and stderr streams to be unbuffered;
         this option has no effect on stdin; also PYTHONUNBUFFERED=x
-v     : verbose (trace import statements); also PYTHONVERBOSE=x
         can be supplied multiple times to increase verbosity
-V     : print the Python version number and exit (also --version)
         when given twice, print more information about the build
-W arg : warning control; arg is action:message:category:module:lineno
         also PYTHONWARNINGS=arg
-x     : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option
--check-hash-based-pycs always|default|never:
         control how Python invalidates hash-based .pyc files
--help-env: print help about Python environment variables and exit
--help-xoptions: print help about implementation-specific -X options and exit
--help-all: print complete help information and exit
Arguments:
file   : program read from script file
-      : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]

Environment variables that change behavior:
PYTHONSTARTUP   : file executed on interactive startup (no default)
PYTHONPATH      : ':'-separated list of directories prefixed to the
                  default module search path.  The result is sys.path.
PYTHONSAFEPATH  : don't prepend a potentially unsafe path to sys.path.
PYTHONHOME      : alternate <prefix> directory (or <prefix>:<exec_prefix>).
                  The default module search path uses <prefix>/lib/pythonX.X.
PYTHONPLATLIBDIR: override sys.platlibdir.
PYTHONCASEOK    : ignore case in 'import' statements (Windows).
PYTHONUTF8      : if set to 1, enable the UTF-8 mode.
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.
PYTHONFAULTHANDLER: dump the Python traceback on fatal errors.
PYTHONHASHSEED  : if this variable is set to 'random', a random value is used
                  to seed the hashes of str and bytes objects.  It can also
                  be set to an integer in the range [0,4294967295] to get
                  hash values with a predictable seed.
PYTHONMALLOC    : set the Python memory allocators and/or install debug hooks
                  on Python memory allocators.  Use PYTHONMALLOC=debug to
                  install debug hooks.
PYTHONCOERCECLOCALE: if this variable is set to 0, it disables the locale
                  coercion behavior.  Use PYTHONCOERCECLOCALE=warn to request
                  display of locale coercion and locale compatibility
                  warnings on stderr.
PYTHONBREAKPOINT: if this variable is set to 0, it disables the default
                  debugger.  It can be set to the callable of your debugger
                  of choice.
PYTHONDEVMODE   : enable the development mode.
PYTHONPYCACHEPREFIX: root directory for bytecode cache (pyc) files.
PYTHONWARNDEFAULTENCODING: enable opt-in EncodingWarning for 'encoding=None'.
PYTHONNODEBUGRANGES: If this variable is set, it disables the inclusion of
                  the tables mapping extra location information (end line,
                  start column offset and end column offset) to every
                  instruction in code objects.  This is useful when smaller
                  code objects and pyc files are desired as well as
                  suppressing the extra visual location indicators when the
                  interpreter displays tracebacks.
These variables have equivalent command-line options (see --help for details):
PYTHONDEBUG     : enable parser debug mode (-d)
PYTHONDONTWRITEBYTECODE: don't write .pyc files (-B)
PYTHONINSPECT   : inspect interactively after running script (-i)
PYTHONNOUSERSITE: disable user site directory (-s)
PYTHONOPTIMIZE  : enable level 1 optimizations (-O)
PYTHONUNBUFFERED: disable stdout/stderr buffering (-u)
PYTHONVERBOSE   : trace import statements (-v)
PYTHONWARNINGS=arg: warning control (-W arg)

The following implementation-specific options are available:
-X faulthandler: enable faulthandler
-X showrefcount: output the total reference count and number of used
         memory blocks when the program finishes or after each statement in
         the interactive interpreter.  This only works on debug builds
-X tracemalloc: start tracing Python memory allocations using the
         tracemalloc module.  By default, only the most recent frame is
         stored in a traceback of a trace.  Use -X tracemalloc=NFRAME to
         start tracing with a traceback limit of NFRAME frames
-X importtime: show how long each import takes.  It shows module name,
         cumulative time (including nested imports) and self time (excluding
         nested imports).  Note that its output may be broken in
         multi-threaded application.
         Typical usage is python3 -X importtime -c 'import asyncio'
-X dev : enable CPython's "development mode", introducing additional runtime
         checks which are too expensive to be enabled by default.  Effect of
         the developer mode:
          * Add default warning filter, as -W default
          * Install debug hooks on memory allocators: see the
            PyMem_SetupDebugHooks() C function
          * Enable the faulthandler module to dump the Python traceback on
            a crash
          * Enable asyncio debug mode
          * Set the dev_mode attribute of sys.flags to True
          * io.IOBase destructor logs close() exceptions
-X utf8: enable UTF-8 mode for operating system interfaces, overriding the
         default locale-aware mode.  -X utf8=0 explicitly disables UTF-8 mode
         (even when it would otherwise activate automatically)
-X pycache_prefix=PATH: enable writing .pyc files to a parallel tree rooted
         at the given directory instead of to the code tree
-X warn_default_encoding: enable opt-in EncodingWarning for 'encoding=None'
-X no_debug_ranges: disable the inclusion of the tables mapping extra location
         information (end line, start column offset and end column offset) to
         every instruction in code objects.  This is useful when smaller code
         objects and pyc files are desired as well as suppressing the extra
         visual location indicators when the interpreter displays tracebacks.
-X frozen_modules=[on|off]: whether or not frozen modules should be used.
         The default is "on" (or "off" if you are running a local build).

I also doubled spaces between sentences for readability.

[serhiy-storchaka]

We can also follow the existing precedence for long option and always start the description from a new line:

-W arg : warning control; arg is action:message:category:module:lineno
         also PYTHONWARNINGS=arg
-x     : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option
--check-hash-based-pycs always|default|never:
         control how Python invalidates hash-based .pyc files
--help-env:
         print help about Python environment variables and exit
--help-xoptions:
         print help about implementation-specific -X options and exit
--help-all:
         print complete help information and exit
Arguments:
file   : program read from script file
-      : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]

[serhiy-storchaka]

Current output with this PR:

usage: (null) [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options (and corresponding environment variables):
-b     : issue warnings about str(bytes_instance), str(bytearray_instance)
         and comparing bytes/bytearray with str. (-bb: issue errors)
-B     : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d     : turn on parser debugging output (for experts only, only works on
         debug builds); also PYTHONDEBUG=x
-E     : ignore PYTHON* environment variables (such as PYTHONPATH)
-h     : print this help message and exit (also -? or --help)
-i     : inspect interactively after running script; forces a prompt even
         if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-I     : isolate Python from the user's environment (implies -E and -s)
-m mod : run library module as a script (terminates option list)
-O     : remove assert and __debug__-dependent statements; add .opt-1 before
         .pyc extension; also PYTHONOPTIMIZE=x
-OO    : do -O changes and also discard docstrings; add .opt-2 before
         .pyc extension
-P     : don't prepend a potentially unsafe path to sys.path
-q     : don't print version and copyright messages on interactive startup
-s     : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S     : don't imply 'import site' on initialization
-u     : force the stdout and stderr streams to be unbuffered;
         this option has no effect on stdin; also PYTHONUNBUFFERED=x
-v     : verbose (trace import statements); also PYTHONVERBOSE=x
         can be supplied multiple times to increase verbosity
-V     : print the Python version number and exit (also --version)
         when given twice, print more information about the build
-W arg : warning control; arg is action:message:category:module:lineno
         also PYTHONWARNINGS=arg
-x     : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option
--check-hash-based-pycs always|default|never:
         control how Python invalidates hash-based .pyc files
--help-env: print help about Python environment variables and exit
--help-xoptions: print help about implementation-specific -X options and exit
--help-all: print complete help information and exit
Arguments:
file   : program read from script file
-      : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]

Environment variables that change behavior:
PYTHONSTARTUP   : file executed on interactive startup (no default)
PYTHONPATH      : ':'-separated list of directories prefixed to the
                  default module search path.  The result is sys.path.
PYTHONSAFEPATH  : don't prepend a potentially unsafe path to sys.path.
PYTHONHOME      : alternate <prefix> directory (or <prefix>:<exec_prefix>).
                  The default module search path uses <prefix>/lib/pythonX.X.
PYTHONPLATLIBDIR: override sys.platlibdir.
PYTHONCASEOK    : ignore case in 'import' statements (Windows).
PYTHONUTF8      : if set to 1, enable the UTF-8 mode.
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.
PYTHONFAULTHANDLER: dump the Python traceback on fatal errors.
PYTHONHASHSEED  : if this variable is set to 'random', a random value is used
                  to seed the hashes of str and bytes objects.  It can also be
                  set to an integer in the range [0,4294967295] to get hash
                  values with a predictable seed.
PYTHONMALLOC    : set the Python memory allocators and/or install debug hooks
                  on Python memory allocators.  Use PYTHONMALLOC=debug to
                  install debug hooks.
PYTHONCOERCECLOCALE: if this variable is set to 0, it disables the locale
                  coercion behavior.  Use PYTHONCOERCECLOCALE=warn to request
                  display of locale coercion and locale compatibility warnings
                  on stderr.
PYTHONBREAKPOINT: if this variable is set to 0, it disables the default
                  debugger.  It can be set to the callable of your debugger of
                  choice.
PYTHONDEVMODE   : enable the development mode.
PYTHONPYCACHEPREFIX: root directory for bytecode cache (pyc) files.
PYTHONWARNDEFAULTENCODING: enable opt-in EncodingWarning for 'encoding=None'.
PYTHONNODEBUGRANGES: If this variable is set, it disables the inclusion of
                  the tables mapping extra location information (end line,
                  start column offset and end column offset) to every
                  instruction in code objects.  This is useful when smaller
                  code objects and pyc files are desired as well as
                  suppressing the extra visual location indicators when the
                  interpreter displays tracebacks.
These variables have equivalent command-line options (see --help for details):
PYTHONDEBUG     : enable parser debug mode (-d)
PYTHONDONTWRITEBYTECODE: don't write .pyc files (-B)
PYTHONINSPECT   : inspect interactively after running script (-i)
PYTHONNOUSERSITE: disable user site directory (-s)
PYTHONOPTIMIZE  : enable level 1 optimizations (-O)
PYTHONUNBUFFERED: disable stdout/stderr buffering (-u)
PYTHONVERBOSE   : trace import statements (-v)
PYTHONWARNINGS=arg: warning control (-W arg)

The following implementation-specific options are available:
-X faulthandler: enable faulthandler
-X showrefcount: output the total reference count and number of used
         memory blocks when the program finishes or after each statement in
         the interactive interpreter.  This only works on debug builds
-X tracemalloc: start tracing Python memory allocations using the
         tracemalloc module.  By default, only the most recent frame is stored
         in a traceback of a trace.  Use -X tracemalloc=NFRAME to start
         tracing with a traceback limit of NFRAME frames
-X importtime: show how long each import takes.  It shows module name,
         cumulative time (including nested imports) and self time (excluding
         nested imports).  Note that its output may be broken in
         multi-threaded application.
         Typical usage is python3 -X importtime -c 'import asyncio'
-X dev : enable CPython's "development mode", introducing additional runtime
         checks which are too expensive to be enabled by default.  Effect of
         the developer mode:
          * Add default warning filter, as -W default
          * Install debug hooks on memory allocators: see the
            PyMem_SetupDebugHooks() C function
          * Enable the faulthandler module to dump the Python traceback on
            a crash
          * Enable asyncio debug mode
          * Set the dev_mode attribute of sys.flags to True
          * io.IOBase destructor logs close() exceptions
-X utf8: enable UTF-8 mode for operating system interfaces, overriding the
         default locale-aware mode.  -X utf8=0 explicitly disables UTF-8 mode
         (even when it would otherwise activate automatically)
-X pycache_prefix=PATH: enable writing .pyc files to a parallel tree rooted
         at the given directory instead of to the code tree
-X warn_default_encoding: enable opt-in EncodingWarning for 'encoding=None'
-X no_debug_ranges: disable the inclusion of the tables mapping extra location
         information (end line, start column offset and end column offset) to
         every instruction in code objects.  This is useful when smaller code
         objects and pyc files are desired as well as suppressing the extra
         visual location indicators when the interpreter displays tracebacks.
-X frozen_modules=[on|off]: whether or not frozen modules should be used.
         The default is "on" (or "off" if you are running a local build).

It uses the same indentation for descriptions of all options (including -X) and larger indentation for descriptions of envvars.

[merwok]

Latest comment looks good!
I can resolve the merge conflicts (and check if new options/envvars need to be formatted) if you want, as I’ve been the blocker here.

[serhiy-storchaka]

I resolved the merge conflicts.

The new output:

usage: ./python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options (and corresponding environment variables):
-b     : issue warnings about str(bytes_instance), str(bytearray_instance)
         and comparing bytes/bytearray with str. (-bb: issue errors)
-B     : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d     : turn on parser debugging output (for experts only, only works on
         debug builds); also PYTHONDEBUG=x
-E     : ignore PYTHON* environment variables (such as PYTHONPATH)
-h     : print this help message and exit (also -? or --help)
-i     : inspect interactively after running script; forces a prompt even
         if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-I     : isolate Python from the user's environment (implies -E and -s)
-m mod : run library module as a script (terminates option list)
-O     : remove assert and __debug__-dependent statements; add .opt-1 before
         .pyc extension; also PYTHONOPTIMIZE=x
-OO    : do -O changes and also discard docstrings; add .opt-2 before
         .pyc extension
-P     : don't prepend a potentially unsafe path to sys.path; also PYTHONSAFEPATH
-q     : don't print version and copyright messages on interactive startup
-s     : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S     : don't imply 'import site' on initialization
-u     : force the stdout and stderr streams to be unbuffered;
         this option has no effect on stdin; also PYTHONUNBUFFERED=x
-v     : verbose (trace import statements); also PYTHONVERBOSE=x
         can be supplied multiple times to increase verbosity
-V     : print the Python version number and exit (also --version)
         when given twice, print more information about the build
-W arg : warning control; arg is action:message:category:module:lineno
         also PYTHONWARNINGS=arg
-x     : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option
--check-hash-based-pycs always|default|never:
         control how Python invalidates hash-based .pyc files
--help-env: print help about Python environment variables and exit
--help-xoptions: print help about implementation-specific -X options and exit
--help-all: print complete help information and exit
Arguments:
file   : program read from script file
-      : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]

Environment variables that change behavior:
PYTHONSTARTUP   : file executed on interactive startup (no default)
PYTHONPATH      : ':'-separated list of directories prefixed to the
                  default module search path.  The result is sys.path.
PYTHONHOME      : alternate <prefix> directory (or <prefix>:<exec_prefix>).
                  The default module search path uses <prefix>/lib/pythonX.X.
PYTHONPLATLIBDIR: override sys.platlibdir.
PYTHONCASEOK    : ignore case in 'import' statements (Windows).
PYTHONUTF8      : if set to 1, enable the UTF-8 mode.
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.
PYTHONFAULTHANDLER: dump the Python traceback on fatal errors.
PYTHONHASHSEED  : if this variable is set to 'random', a random value is used
                  to seed the hashes of str and bytes objects.  It can also be
                  set to an integer in the range [0,4294967295] to get hash
                  values with a predictable seed.
PYTHONINTMAXSTRDIGITS: limits the maximum digit characters in an int value
                  when converting from a string and when converting an int
                  back to a str.  A value of 0 disables the limit.
                  Conversions to or from bases 2, 4, 8, 16, and 32 are never
                  limited.
PYTHONMALLOC    : set the Python memory allocators and/or install debug hooks
                  on Python memory allocators.  Use PYTHONMALLOC=debug to
                  install debug hooks.
PYTHONCOERCECLOCALE: if this variable is set to 0, it disables the locale
                  coercion behavior.  Use PYTHONCOERCECLOCALE=warn to request
                  display of locale coercion and locale compatibility warnings
                  on stderr.
PYTHONBREAKPOINT: if this variable is set to 0, it disables the default
                  debugger.  It can be set to the callable of your debugger of
                  choice.
PYTHON_CPU_COUNT: Overrides the return value of os.process_cpu_count(),
                  os.cpu_count(), and multiprocessing.cpu_count() if set to
                  a positive integer.
PYTHONDEVMODE   : enable the development mode.
PYTHONPYCACHEPREFIX: root directory for bytecode cache (pyc) files.
PYTHONWARNDEFAULTENCODING: enable opt-in EncodingWarning for 'encoding=None'.
PYTHONNODEBUGRANGES: if this variable is set, it disables the inclusion of
                  the tables mapping extra location information (end line,
                  start column offset and end column offset) to every
                  instruction in code objects.  This is useful when smaller
                  code objects and pyc files are desired as well as
                  suppressing the extra visual location indicators when the
                  interpreter displays tracebacks.
PYTHON_FROZEN_MODULES: if this variable is set, it determines whether or not
                  frozen modules should be used.  The default is "on" (or
                  "off" if you are running a local build).
PYTHON_COLORS   : If this variable is set to 1, the interpreter will colorize
                  various kinds of output.  Setting it to 0 deactivates
                  this behavior.
These variables have equivalent command-line options (see --help for details):
PYTHONDEBUG     : enable parser debug mode (-d)
PYTHONDONTWRITEBYTECODE: don't write .pyc files (-B)
PYTHONINSPECT   : inspect interactively after running script (-i)
PYTHONINTMAXSTRDIGITS: limit max digit characters in an int value
                  (-X int_max_str_digits=number)
PYTHONNOUSERSITE: disable user site directory (-s)
PYTHONOPTIMIZE  : enable level 1 optimizations (-O)
PYTHONSAFEPATH  : don't prepend a potentially unsafe path to sys.path.
PYTHONUNBUFFERED: disable stdout/stderr buffering (-u)
PYTHONVERBOSE   : trace import statements (-v)
PYTHONWARNINGS=arg: warning control (-W arg)
PYTHON_PRESITE=pkg.mod: import this module before site.py is run

The following implementation-specific options are available:
-X faulthandler: enable faulthandler
-X showrefcount: output the total reference count and number of used
         memory blocks when the program finishes or after each statement in
         the interactive interpreter.  This only works on debug builds
-X tracemalloc: start tracing Python memory allocations using the
         tracemalloc module.  By default, only the most recent frame is stored
         in a traceback of a trace.  Use -X tracemalloc=NFRAME to start
         tracing with a traceback limit of NFRAME frames
-X importtime: show how long each import takes.  It shows module name,
         cumulative time (including nested imports) and self time (excluding
         nested imports).  Note that its output may be broken in
         multi-threaded application.
         Typical usage is python3 -X importtime -c 'import asyncio'
-X dev : enable CPython's "development mode", introducing additional runtime
         checks which are too expensive to be enabled by default.  Effect of
         the developer mode:
          * Add default warning filter, as -W default
          * Install debug hooks on memory allocators: see the
            PyMem_SetupDebugHooks() C function
          * Enable the faulthandler module to dump the Python traceback on
            a crash
          * Enable asyncio debug mode
          * Set the dev_mode attribute of sys.flags to True
          * io.IOBase destructor logs close() exceptions
-X utf8: enable UTF-8 mode for operating system interfaces, overriding the
         default locale-aware mode.  -X utf8=0 explicitly disables UTF-8 mode
         (even when it would otherwise activate automatically)
-X pycache_prefix=PATH: enable writing .pyc files to a parallel tree rooted
         at the given directory instead of to the code tree
-X warn_default_encoding: enable opt-in EncodingWarning for 'encoding=None'
-X no_debug_ranges: disable the inclusion of the tables mapping extra location
         information (end line, start column offset and end column offset) to
         every instruction in code objects.  This is useful when smaller code
         objects and pyc files are desired as well as suppressing the extra
         visual location indicators when the interpreter displays tracebacks.
-X perf: activate support for the Linux "perf" profiler by activating the
         "perf" trampoline.  When this option is activated, the Linux "perf"
         profiler will be able to report Python calls.  This option is only
         available on some platforms and will do nothing if is not supported
         on the current system.  The default value is "off".
-X frozen_modules=[on|off]: whether or not frozen modules should be used.
         The default is "on" (or "off" if you are running a local build).
-X int_max_str_digits=number: limit the size of int<->str conversions.
         This helps avoid denial of service attacks when parsing untrusted
         data.  The default is sys.int_info.default_max_str_digits.
         0 disables.
-X cpu_count=[n|default]: Override the return value of os.cpu_count(),
         os.process_cpu_count(), and multiprocessing.cpu_count().  This can
         help users who need to limit resources in a container.
-X presite=package.module: import this module before site.py is run.

[hugovk]

Looks good, although it's quite a big wall of text. I think some blank newlines between chunks would help with readability.

At least before "Arguments:" and "These variables have equivalent command-line options (see --help for details):", for example:

Details

usage: ./python.exe [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options (and corresponding environment variables):
-b     : issue warnings about str(bytes_instance), str(bytearray_instance)
         and comparing bytes/bytearray with str. (-bb: issue errors)
-B     : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d     : turn on parser debugging output (for experts only, only works on
         debug builds); also PYTHONDEBUG=x
-E     : ignore PYTHON* environment variables (such as PYTHONPATH)
-h     : print this help message and exit (also -? or --help)
-i     : inspect interactively after running script; forces a prompt even
         if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-I     : isolate Python from the user's environment (implies -E and -s)
-m mod : run library module as a script (terminates option list)
-O     : remove assert and __debug__-dependent statements; add .opt-1 before
         .pyc extension; also PYTHONOPTIMIZE=x
-OO    : do -O changes and also discard docstrings; add .opt-2 before
         .pyc extension
-P     : don't prepend a potentially unsafe path to sys.path; also PYTHONSAFEPATH
-q     : don't print version and copyright messages on interactive startup
-s     : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S     : don't imply 'import site' on initialization
-u     : force the stdout and stderr streams to be unbuffered;
         this option has no effect on stdin; also PYTHONUNBUFFERED=x
-v     : verbose (trace import statements); also PYTHONVERBOSE=x
         can be supplied multiple times to increase verbosity
-V     : print the Python version number and exit (also --version)
         when given twice, print more information about the build
-W arg : warning control; arg is action:message:category:module:lineno
         also PYTHONWARNINGS=arg
-x     : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option
--check-hash-based-pycs always|default|never:
         control how Python invalidates hash-based .pyc files
--help-env: print help about Python environment variables and exit
--help-xoptions: print help about implementation-specific -X options and exit
--help-all: print complete help information and exit

Arguments:
file   : program read from script file
-      : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]

Environment variables that change behavior:
PYTHONSTARTUP   : file executed on interactive startup (no default)
PYTHONPATH      : ':'-separated list of directories prefixed to the
                  default module search path.  The result is sys.path.
PYTHONHOME      : alternate <prefix> directory (or <prefix>:<exec_prefix>).
                  The default module search path uses <prefix>/lib/pythonX.X.
PYTHONPLATLIBDIR: override sys.platlibdir.
PYTHONCASEOK    : ignore case in 'import' statements (Windows).
PYTHONUTF8      : if set to 1, enable the UTF-8 mode.
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.
PYTHONFAULTHANDLER: dump the Python traceback on fatal errors.
PYTHONHASHSEED  : if this variable is set to 'random', a random value is used
                  to seed the hashes of str and bytes objects.  It can also be
                  set to an integer in the range [0,4294967295] to get hash
                  values with a predictable seed.
PYTHONINTMAXSTRDIGITS: limits the maximum digit characters in an int value
                  when converting from a string and when converting an int
                  back to a str.  A value of 0 disables the limit.
                  Conversions to or from bases 2, 4, 8, 16, and 32 are never
                  limited.
PYTHONMALLOC    : set the Python memory allocators and/or install debug hooks
                  on Python memory allocators.  Use PYTHONMALLOC=debug to
                  install debug hooks.
PYTHONCOERCECLOCALE: if this variable is set to 0, it disables the locale
                  coercion behavior.  Use PYTHONCOERCECLOCALE=warn to request
                  display of locale coercion and locale compatibility warnings
                  on stderr.
PYTHONBREAKPOINT: if this variable is set to 0, it disables the default
                  debugger.  It can be set to the callable of your debugger of
                  choice.
PYTHON_CPU_COUNT: Overrides the return value of os.process_cpu_count(),
                  os.cpu_count(), and multiprocessing.cpu_count() if set to
                  a positive integer.
PYTHONDEVMODE   : enable the development mode.
PYTHONPYCACHEPREFIX: root directory for bytecode cache (pyc) files.
PYTHONWARNDEFAULTENCODING: enable opt-in EncodingWarning for 'encoding=None'.
PYTHONNODEBUGRANGES: if this variable is set, it disables the inclusion of
                  the tables mapping extra location information (end line,
                  start column offset and end column offset) to every
                  instruction in code objects.  This is useful when smaller
                  code objects and pyc files are desired as well as
                  suppressing the extra visual location indicators when the
                  interpreter displays tracebacks.
PYTHON_FROZEN_MODULES: if this variable is set, it determines whether or not
                  frozen modules should be used.  The default is "on" (or
                  "off" if you are running a local build).
PYTHON_COLORS   : If this variable is set to 1, the interpreter will colorize
                  various kinds of output.  Setting it to 0 deactivates
                  this behavior.
PYTHON_HISTORY  : the location of a .python_history file.

These variables have equivalent command-line options (see --help for details):
PYTHONDEBUG     : enable parser debug mode (-d)
PYTHONDONTWRITEBYTECODE: don't write .pyc files (-B)
PYTHONINSPECT   : inspect interactively after running script (-i)
PYTHONINTMAXSTRDIGITS: limit max digit characters in an int value
                  (-X int_max_str_digits=number)
PYTHONNOUSERSITE: disable user site directory (-s)
PYTHONOPTIMIZE  : enable level 1 optimizations (-O)
PYTHONSAFEPATH  : don't prepend a potentially unsafe path to sys.path.
PYTHONUNBUFFERED: disable stdout/stderr buffering (-u)
PYTHONVERBOSE   : trace import statements (-v)
PYTHONWARNINGS=arg: warning control (-W arg)
PYTHON_PRESITE=pkg.mod: import this module before site.py is run

The following implementation-specific options are available:
-X faulthandler: enable faulthandler
-X showrefcount: output the total reference count and number of used
         memory blocks when the program finishes or after each statement in
         the interactive interpreter.  This only works on debug builds
-X tracemalloc: start tracing Python memory allocations using the
         tracemalloc module.  By default, only the most recent frame is stored
         in a traceback of a trace.  Use -X tracemalloc=NFRAME to start
         tracing with a traceback limit of NFRAME frames
-X importtime: show how long each import takes.  It shows module name,
         cumulative time (including nested imports) and self time (excluding
         nested imports).  Note that its output may be broken in
         multi-threaded application.
         Typical usage is python3 -X importtime -c 'import asyncio'
-X dev : enable CPython's "development mode", introducing additional runtime
         checks which are too expensive to be enabled by default.  Effect of
         the developer mode:
          * Add default warning filter, as -W default
          * Install debug hooks on memory allocators: see the
            PyMem_SetupDebugHooks() C function
          * Enable the faulthandler module to dump the Python traceback on
            a crash
          * Enable asyncio debug mode
          * Set the dev_mode attribute of sys.flags to True
          * io.IOBase destructor logs close() exceptions
-X utf8: enable UTF-8 mode for operating system interfaces, overriding the
         default locale-aware mode.  -X utf8=0 explicitly disables UTF-8 mode
         (even when it would otherwise activate automatically)
-X pycache_prefix=PATH: enable writing .pyc files to a parallel tree rooted
         at the given directory instead of to the code tree
-X warn_default_encoding: enable opt-in EncodingWarning for 'encoding=None'
-X no_debug_ranges: disable the inclusion of the tables mapping extra location
         information (end line, start column offset and end column offset) to
         every instruction in code objects.  This is useful when smaller code
         objects and pyc files are desired as well as suppressing the extra
         visual location indicators when the interpreter displays tracebacks.
-X perf: activate support for the Linux "perf" profiler by activating the
         "perf" trampoline.  When this option is activated, the Linux "perf"
         profiler will be able to report Python calls.  This option is only
         available on some platforms and will do nothing if is not supported
         on the current system.  The default value is "off".
-X frozen_modules=[on|off]: whether or not frozen modules should be used.
         The default is "on" (or "off" if you are running a local build).
-X int_max_str_digits=number: limit the size of int<->str conversions.
         This helps avoid denial of service attacks when parsing untrusted
         data.  The default is sys.int_info.default_max_str_digits.
         0 disables.
-X cpu_count=[n|default]: Override the return value of os.cpu_count(),
         os.process_cpu_count(), and multiprocessing.cpu_count().  This can
         help users who need to limit resources in a container.
-X presite=package.module: import this module before site.py is run.

[hugovk]

Also the first two sections are concatenated this with backslashes:

static const char usage_help[] = "\
Options (and corresponding environment variables):\n\
-b     : issue warnings about str(bytes_instance), str(bytearray_instance)\n\
         and comparing bytes/bytearray with str. (-bb: issue errors)\n\
-B     : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x\n\
...
";

static const char usage_xoptions[] = "\
The following implementation-specific options are available:\n\
-X faulthandler: enable faulthandler\n\
-X showrefcount: output the total reference count and number of used\n\
...
;

And the last with double quotes:

static const char usage_envvars[] =
"Environment variables that change behavior:\n"
"PYTHONSTARTUP   : file executed on interactive startup (no default)\n"
"PYTHONPATH      : '%lc'-separated list of directories prefixed to the\n"
...
;

Is it worth defining them all the same way?

(They're also formatted differently, at least in PyCharm.)

[serhiy-storchaka]

I am going to rewrite also help messages. For most options and variables they should be only 1-2 lines long. More details can be read in the documentation. But this is a different issue.

[hugovk]

Looks good, the extra space really helps readability.

Left is 3.13.0a3, right is this PR:

Can we have some extra padding/alignment for --help-env, --help-xoptions and --help-all, similar to before?

[serhiy-storchaka]

It would be not consistent with the way how other longer-than-normal options and envvars are formatted. Padding is only added if the name is shorter-than-normal. If it is longer-than-normal, the colon is added immediately after the name.

[serhiy-storchaka]

We can also write:

-X opt : set implementation-specific option\n\
--check-hash-based-pycs always|default|never:\n\
         control how Python invalidates hash-based .pyc files\n\
--help-env:\n\
         print help about Python environment variables and exit\n\
--help-xoptions:\n\
         print help about implementation-specific -X options and exit\n\
--help-all:\n\
         print complete help information and exit\n\

But it would be a waste of precious vertical space.

[hugovk]

I'm not too worried about using extra vertical or horizontal space when it adds room to breathe/see.

If we have to stick to that consistency rule (do we?), can we increase the base indent for this section? For example, the PYTHON* env vars section are further out to better accommodate longer text.

[merwok]

The added newlines (before arguments and envvars with equivalents) are a nice addition, but I do agree with Serhiy’s comment that taking too much vertical space would not be good.

[serhiy-storchaka]

If we have to stick to that consistency rule (do we?), can we increase the base indent for this section? For example, the PYTHON* env vars section are further out to better accommodate longer text.

It will be not good for almost all of options. Too large distance between the option and its description will harm readability. It will also require more line wraps.

[ambv]

This PR is in bikeshedding hell since mid-2022 and I think we're all agreeing it's improving the status quo, there's just disagreement on possible further improvements. I feel like this is ready to go. Unless somebody stops me, I will be merging this tomorrow.

[hugovk]

This definitely improves things, so 👍 from me. Thanks @serhiy-storchaka!

[merwok]

Bit harsh to call all the reviews here bikeshedding, but regardless this is ready 👍🏽

[miss-islington-app]

Thanks @serhiy-storchaka for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

[miss-islington-app]

Sorry, @serhiy-storchaka and @ambv, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 2e92ffd7fa89e3bd33ee2f31541d3dc53aaa2d12 3.12

[miss-islington-app]

Sorry, @serhiy-storchaka and @ambv, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 2e92ffd7fa89e3bd33ee2f31541d3dc53aaa2d12 3.11

[ambv]

I'll deal with the backports.

[serhiy-storchaka]

Thank you @ambv.

[bedevere-app]

GH-115847 is a backport of this pull request to the 3.12 branch.

[ambv]

I'll base the 3.11 backport on the 3.12 backport.