Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync latest distutils #4103

Merged
merged 34 commits into from
Nov 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
5a8ca1b
Rely on pytest as found in pytest-dev/pytest#11155. Fixes pypa/distut…
jaraco Jul 1, 2023
6e19647
Merge https://github.com/jaraco/skeleton into feature/skeleton
jaraco Jul 4, 2023
4a70331
Clean up docstrings and remove crufty comments. Replace integer liter…
jaraco Nov 4, 2023
c4e27db
"Refactor to newer_group to utilize higher level constructs ("any"), …
jaraco Nov 4, 2023
d7aa188
Prefer os.path.getmtime
jaraco Nov 4, 2023
dfc8e60
Inline check for target presence.
jaraco Nov 4, 2023
bdffb48
Add test for newer_pairwise, bringing coverage in dep_util to 100%.
jaraco Nov 4, 2023
0720b98
Replace for/append loop with a filter function (newer_pair).
jaraco Nov 4, 2023
131eff7
Replace explicit list check with zip(strict=True). Allows inputs to b…
jaraco Nov 4, 2023
4d82dc4
Extract a 'starfilter', similar to itertools.starmap, to generalize t…
jaraco Nov 4, 2023
5deb5ac
Replace '_starfilter' with 'jaraco.functools.splat'.
jaraco Nov 5, 2023
9494203
Move dep_util to _modified and mark dep_util as deprecated.
jaraco Nov 5, 2023
ce9efc4
Extend tests for newer_pairwise and fix failed expectation when no fi…
jaraco Nov 5, 2023
2972e29
Add newer_pairwise_group (inspired by setuptools.dep_util).
jaraco Nov 5, 2023
378d0d5
Added tests for newer_pairwise_group().
Jan 15, 2017
501b753
Modernize test_newer_pairwise_group by using tmp_path and a SimpleNam…
jaraco Nov 5, 2023
fe9018c
Merge https://github.com/jaraco/skeleton into feature/skeleton
jaraco Nov 5, 2023
6039322
Remove latent references in docs.
jaraco Nov 5, 2023
cf6fb0c
Merge branch 'main' into feature/skeleton
jaraco Nov 5, 2023
03f03e7
Limit sphinxlint jobs to 1. Workaround for sphinx-contrib/sphinx-lint…
jaraco Nov 5, 2023
4673d63
Merge https://github.com/jaraco/skeleton into feature/skeleton
jaraco Nov 5, 2023
41b9cce
Replace git version with released version. Ref #186.
jaraco Nov 5, 2023
d3e5de0
Disable cygwin tests for now. Ref pypa/setuptools#3921
jaraco May 11, 2023
6ec501a
Merge branch 'main' into feature/skeleton
jaraco Nov 5, 2023
d23e28a
Disable integration test due to known breakage from deprecation warni…
jaraco Nov 5, 2023
a49aff3
Merge branch 'main' into feature/skeleton
jaraco Nov 5, 2023
03c6392
Allow diffcov to fail also, as it requires the tests to pass on the l…
jaraco Nov 5, 2023
6e6ee97
Remove newsfragment
jaraco Nov 5, 2023
64eb463
Merge pull request pypa/distutils#217 from pypa/feature/skeleton
jaraco Nov 5, 2023
7a04cbd
Copy concurrency setting from setuptools
jaraco Nov 5, 2023
f6f5563
Merge https://github.com/pypa/distutils into feature/distutils-sync
jaraco Nov 5, 2023
0296279
Rely on distutils._modified and deprecated setuptools.dep_util.
jaraco Nov 5, 2023
987bc92
Add news fragment.
jaraco Nov 8, 2023
c614ef5
Fallback when SETUPTOOLS_USE_DISTUTILS=stdlib
jaraco Nov 8, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions docs/deprecated/distutils/configfile.rst
Original file line number Diff line number Diff line change
Expand Up @@ -131,13 +131,6 @@ Note that the ``doc_files`` option is simply a whitespace-separated string
split across multiple lines for readability.


.. seealso::

:ref:`inst-config-syntax` in "Installing Python Modules"
More information on the configuration files is available in the manual for
system administrators.


.. rubric:: Footnotes

.. [#] This ideal probably won't be achieved until auto-configuration is fully
Expand Down
13 changes: 6 additions & 7 deletions docs/deprecated/distutils/packageindex.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,10 @@
The Python Package Index (PyPI)
*******************************

The `Python Package Index (PyPI)`_ stores metadata describing distributions
packaged with distutils and other publishing tools, as well the distribution
archives themselves.
The `Python Package Index (PyPI) <https://pypi.org>`_ stores
metadata describing distributions packaged with distutils and
other publishing tools, as well the distribution archives
themselves.

References to up to date PyPI documentation can be found at
:ref:`publishing-python-packages`.

.. _Python Package Index (PyPI): https://pypi.org
The best resource for working with PyPI is the
`Python Packaging User Guide <https://packaging.python.org>`_.
5 changes: 3 additions & 2 deletions docs/deprecated/distutils/uploading.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,6 @@
Uploading Packages to the Package Index
***************************************

References to up to date PyPI documentation can be found at
:ref:`publishing-python-packages`.
See the
`Python Packaging User Guide <https://packaging.python.org>`_
for the best guidance on uploading packages.
1 change: 1 addition & 0 deletions newsfragments/+f8383dcd.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Merged with pypa/distutils@7a04cbda0fc714.
53 changes: 53 additions & 0 deletions setuptools/_distutils/_functools.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import collections.abc
import functools


Expand All @@ -18,3 +19,55 @@ def wrapper(param, *args, **kwargs):
return func(param, *args, **kwargs)

return wrapper


# from jaraco.functools 4.0
@functools.singledispatch
def _splat_inner(args, func):
"""Splat args to func."""
return func(*args)


@_splat_inner.register
def _(args: collections.abc.Mapping, func):
"""Splat kargs to func as kwargs."""
return func(**args)


def splat(func):
"""
Wrap func to expect its parameters to be passed positionally in a tuple.

Has a similar effect to that of ``itertools.starmap`` over
simple ``map``.

>>> import itertools, operator
>>> pairs = [(-1, 1), (0, 2)]
>>> _ = tuple(itertools.starmap(print, pairs))
-1 1
0 2
>>> _ = tuple(map(splat(print), pairs))
-1 1
0 2

The approach generalizes to other iterators that don't have a "star"
equivalent, such as a "starfilter".

>>> list(filter(splat(operator.add), pairs))
[(0, 2)]

Splat also accepts a mapping argument.

>>> def is_nice(msg, code):
... return "smile" in msg or code == 0
>>> msgs = [
... dict(msg='smile!', code=20),
... dict(msg='error :(', code=1),
... dict(msg='unknown', code=0),
... ]
>>> for msg in filter(splat(is_nice), msgs):
... print(msg)
{'msg': 'smile!', 'code': 20}
{'msg': 'unknown', 'code': 0}
"""
return functools.wraps(func)(functools.partial(_splat_inner, func=func))
72 changes: 72 additions & 0 deletions setuptools/_distutils/_modified.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
"""Timestamp comparison of files and groups of files."""

import functools
import os.path

from .errors import DistutilsFileError
from .py39compat import zip_strict
from ._functools import splat


def _newer(source, target):
return not os.path.exists(target) or (
os.path.getmtime(source) > os.path.getmtime(target)
)


def newer(source, target):
"""
Is source modified more recently than target.

Returns True if 'source' is modified more recently than
'target' or if 'target' does not exist.

Raises DistutilsFileError if 'source' does not exist.
"""
if not os.path.exists(source):
raise DistutilsFileError("file '%s' does not exist" % os.path.abspath(source))

return _newer(source, target)


def newer_pairwise(sources, targets, newer=newer):
"""
Filter filenames where sources are newer than targets.

Walk two filename iterables in parallel, testing if each source is newer
than its corresponding target. Returns a pair of lists (sources,
targets) where source is newer than target, according to the semantics
of 'newer()'.
"""
newer_pairs = filter(splat(newer), zip_strict(sources, targets))
return tuple(map(list, zip(*newer_pairs))) or ([], [])


def newer_group(sources, target, missing='error'):
"""
Is target out-of-date with respect to any file in sources.

Return True if 'target' is out-of-date with respect to any file
listed in 'sources'. In other words, if 'target' exists and is newer
than every file in 'sources', return False; otherwise return True.
``missing`` controls how to handle a missing source file:

- error (default): allow the ``stat()`` call to fail.
- ignore: silently disregard any missing source files.
- newer: treat missing source files as "target out of date". This
mode is handy in "dry-run" mode: it will pretend to carry out
commands that wouldn't work because inputs are missing, but
that doesn't matter because dry-run won't run the commands.
"""

def missing_as_newer(source):
return missing == 'newer' and not os.path.exists(source)

ignored = os.path.exists if missing == 'ignore' else None
return any(
missing_as_newer(source) or _newer(source, target)
for source in filter(ignored, sources)
)


newer_pairwise_group = functools.partial(newer_pairwise, newer=newer_group)
2 changes: 1 addition & 1 deletion setuptools/_distutils/bcppcompiler.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
)
from .ccompiler import CCompiler, gen_preprocess_options
from .file_util import write_file
from .dep_util import newer
from ._modified import newer
from ._log import log


Expand Down
2 changes: 1 addition & 1 deletion setuptools/_distutils/ccompiler.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
from .spawn import spawn
from .file_util import move_file
from .dir_util import mkpath
from .dep_util import newer_group
from ._modified import newer_group
from .util import split_quoted, execute
from ._log import log

Expand Down
4 changes: 2 additions & 2 deletions setuptools/_distutils/cmd.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import logging

from .errors import DistutilsOptionError
from . import util, dir_util, file_util, archive_util, dep_util
from . import util, dir_util, file_util, archive_util, _modified
from ._log import log


Expand Down Expand Up @@ -428,7 +428,7 @@ def make_file(
# If 'outfile' must be regenerated (either because it doesn't
# exist, is out-of-date, or the 'force' flag is true) then
# perform the action that presumably regenerates it
if self.force or dep_util.newer_group(infiles, outfile):
if self.force or _modified.newer_group(infiles, outfile):
self.execute(func, args, exec_msg, level)
# Otherwise, print the "skip" message
else:
Expand Down
2 changes: 1 addition & 1 deletion setuptools/_distutils/command/build_ext.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
)
from ..sysconfig import customize_compiler, get_python_version
from ..sysconfig import get_config_h_filename
from ..dep_util import newer_group
from .._modified import newer_group
from ..extension import Extension
from ..util import get_platform
from distutils._log import log
Expand Down
2 changes: 1 addition & 1 deletion setuptools/_distutils/command/build_scripts.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from stat import ST_MODE
from distutils import sysconfig
from ..core import Command
from ..dep_util import newer
from .._modified import newer
from ..util import convert_path
from distutils._log import log
import tokenize
Expand Down
104 changes: 11 additions & 93 deletions setuptools/_distutils/dep_util.py
Original file line number Diff line number Diff line change
@@ -1,96 +1,14 @@
"""distutils.dep_util
import warnings

Utility functions for simple, timestamp-based dependency of files
and groups of files; also, function based entirely on such
timestamp dependency analysis."""
from . import _modified

import os
from .errors import DistutilsFileError


def newer(source, target):
"""Return true if 'source' exists and is more recently modified than
'target', or if 'source' exists and 'target' doesn't. Return false if
both exist and 'target' is the same age or younger than 'source'.
Raise DistutilsFileError if 'source' does not exist.
"""
if not os.path.exists(source):
raise DistutilsFileError("file '%s' does not exist" % os.path.abspath(source))
if not os.path.exists(target):
return 1

from stat import ST_MTIME

mtime1 = os.stat(source)[ST_MTIME]
mtime2 = os.stat(target)[ST_MTIME]

return mtime1 > mtime2


# newer ()


def newer_pairwise(sources, targets):
"""Walk two filename lists in parallel, testing if each source is newer
than its corresponding target. Return a pair of lists (sources,
targets) where source is newer than target, according to the semantics
of 'newer()'.
"""
if len(sources) != len(targets):
raise ValueError("'sources' and 'targets' must be same length")

# build a pair of lists (sources, targets) where source is newer
n_sources = []
n_targets = []
for i in range(len(sources)):
if newer(sources[i], targets[i]):
n_sources.append(sources[i])
n_targets.append(targets[i])

return (n_sources, n_targets)


# newer_pairwise ()


def newer_group(sources, target, missing='error'):
"""Return true if 'target' is out-of-date with respect to any file
listed in 'sources'. In other words, if 'target' exists and is newer
than every file in 'sources', return false; otherwise return true.
'missing' controls what we do when a source file is missing; the
default ("error") is to blow up with an OSError from inside 'stat()';
if it is "ignore", we silently drop any missing source files; if it is
"newer", any missing source files make us assume that 'target' is
out-of-date (this is handy in "dry-run" mode: it'll make you pretend to
carry out commands that wouldn't work because inputs are missing, but
that doesn't matter because you're not actually going to run the
commands).
"""
# If the target doesn't even exist, then it's definitely out-of-date.
if not os.path.exists(target):
return 1

# Otherwise we have to find out the hard way: if *any* source file
# is more recent than 'target', then 'target' is out-of-date and
# we can immediately return true. If we fall through to the end
# of the loop, then 'target' is up-to-date and we return false.
from stat import ST_MTIME

target_mtime = os.stat(target)[ST_MTIME]
for source in sources:
if not os.path.exists(source):
if missing == 'error': # blow up when we stat() the file
pass
elif missing == 'ignore': # missing source dropped from
continue # target's dependency list
elif missing == 'newer': # missing source means target is
return 1 # out-of-date

source_mtime = os.stat(source)[ST_MTIME]
if source_mtime > target_mtime:
return 1
else:
return 0


# newer_group ()
def __getattr__(name):
if name not in ['newer', 'newer_group', 'newer_pairwise']:
raise AttributeError(name)
warnings.warn(
"dep_util is Deprecated. Use functions from setuptools instead.",
DeprecationWarning,
stacklevel=2,
)
return getattr(_modified, name)
2 changes: 1 addition & 1 deletion setuptools/_distutils/file_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ def copy_file( # noqa: C901
# changing it (ie. it's not already a hard/soft link to src OR
# (not update) and (src newer than dst).

from distutils.dep_util import newer
from distutils._modified import newer
from stat import ST_ATIME, ST_MTIME, ST_MODE, S_IMODE

if not os.path.isfile(src):
Expand Down
Loading
Loading