Build failure: python312Packages.scrapy

[D3vil0p3r]

Steps To Reproduce

Steps to reproduce the behavior:

1. build python312Packages.scrapy

Build log

Fails during the test phase:

=================================== FAILURES ===================================
____________________ UtilsPythonTestCase.test_get_func_args ____________________

self = <tests.test_utils_python.UtilsPythonTestCase testMethod=test_get_func_args>

    def test_get_func_args(self):
        def f1(a, b, c):
            pass
    
        def f2(a, b=None, c=None):
            pass
    
        def f3(a, b=None, *, c=None):
            pass
    
        class A:
            def __init__(self, a, b, c):
                pass
    
            def method(self, a, b, c):
                pass
    
        class Callable:
            def __call__(self, a, b, c):
                pass
    
        a = A(1, 2, 3)
        cal = Callable()
        partial_f1 = functools.partial(f1, None)
        partial_f2 = functools.partial(f1, b=None)
        partial_f3 = functools.partial(partial_f2, None)
    
        self.assertEqual(get_func_args(f1), ["a", "b", "c"])
        self.assertEqual(get_func_args(f2), ["a", "b", "c"])
        self.assertEqual(get_func_args(f3), ["a", "b", "c"])
        self.assertEqual(get_func_args(A), ["a", "b", "c"])
        self.assertEqual(get_func_args(a.method), ["a", "b", "c"])
        self.assertEqual(get_func_args(partial_f1), ["b", "c"])
        self.assertEqual(get_func_args(partial_f2), ["a", "c"])
        self.assertEqual(get_func_args(partial_f3), ["c"])
        self.assertEqual(get_func_args(cal), ["a", "b", "c"])
        self.assertEqual(get_func_args(object), [])
        self.assertEqual(get_func_args(str.split, stripself=True), ["sep", "maxsplit"])
        self.assertEqual(get_func_args(" ".join, stripself=True), ["iterable"])
    
        if platform.python_implementation() == "CPython":
            # doesn't work on CPython: https://bugs.python.org/issue42785
>           self.assertEqual(get_func_args(operator.itemgetter(2)), [])

/build/Scrapy-2.11.1/tests/test_utils_python.py:243: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/nix/store/gvl7vqff3ircmxnbzdaxad8gadjy2pjk-python3.12-twisted-24.3.0/lib/python3.12/site-packages/twisted/trial/_synctest.py:444: in assertEqual
    super().assertEqual(first, second, msg)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tests.test_utils_python.UtilsPythonTestCase testMethod=test_get_func_args>
msg = "Lists differ: ['args', 'kwargs'] != []\n\nFirst list contains 2 additional elements.\nFirst extra element 0:\n'args'\n\n- ['args', 'kwargs']\n+ []"

    def fail(self, msg: Optional[object] = None) -> NoReturn:
        """
        Absolutely fail the test.  Do not pass go, do not collect $200.
    
        @param msg: the message that will be displayed as the reason for the
        failure
        """
>       raise self.failureException(msg)
E       twisted.trial.unittest.FailTest: Lists differ: ['args', 'kwargs'] != []
E       
E       First list contains 2 additional elements.
E       First extra element 0:
E       'args'
E       
E       - ['args', 'kwargs']
E       + []

/nix/store/gvl7vqff3ircmxnbzdaxad8gadjy2pjk-python3.12-twisted-24.3.0/lib/python3.12/site-packages/twisted/trial/_synctest.py:381: FailTest
=============================== warnings summary ===============================
tests/test_addons.py: 2 warnings
tests/test_crawler.py: 2 warnings
tests/test_downloaderslotssettings.py: 1 warning
tests/test_extension_periodic_log.py: 16 warnings
tests/test_spider.py: 6 warnings
  /build/Scrapy-2.11.1/scrapy/utils/request.py:254: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting.
  
  It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy.
  
  See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation.
    return cls(crawler)

tests/test_contracts.py::ContractsManagerTest::test_returns_async
  /build/Scrapy-2.11.1/scrapy/contracts/__init__.py:170: RuntimeWarning: coroutine 'TestSpider.returns_request_async' was never awaited
    results.addError(case, sys.exc_info())

tests/test_crawl.py: 2 warnings
tests/test_downloader_handlers.py: 149 warnings
tests/test_downloader_handlers_http2.py: 32 warnings
tests/test_webclient.py: 4 warnings
  /build/Scrapy-2.11.1/scrapy/core/downloader/contextfactory.py:90: DeprecationWarning: Passing method to twisted.internet.ssl.CertificateOptions was deprecated in Twisted 17.1.0. Please use a combination of insecurelyLowerMinimumTo, raiseMinimumTo, and lowerMaximumSecurityTo instead, as Twisted will correctly configure the method.
    return CertificateOptions(

tests/test_feedexport.py: 31 warnings
  /build/Scrapy-2.11.1/scrapy/extensions/feedexport.py:203: ScrapyDeprecationWarning: `botocore` usage has been deprecated for S3 feed export, please use `boto3` to avoid problems
    warnings.warn(

tests/test_feedexport.py::StdoutFeedStoragePreFeedOptionsTest::test_init
tests/test_feedexport.py::StdoutFeedStoragePreFeedOptionsTest::test_init
  /build/Scrapy-2.11.1/scrapy/extensions/feedexport.py:49: ScrapyDeprecationWarning: StdoutFeedStorageWithoutFeedOptions does not support the 'feed_options' keyword argument. Add a 'feed_options' parameter to its signature to remove this warning. This parameter will become mandatory in a future version of Scrapy.
    warnings.warn(

tests/test_feedexport.py::S3FeedStoragePreFeedOptionsTest::test_from_crawler
tests/test_feedexport.py::S3FeedStoragePreFeedOptionsTest::test_from_crawler
  /build/Scrapy-2.11.1/scrapy/extensions/feedexport.py:49: ScrapyDeprecationWarning: S3FeedStorageWithoutFeedOptionsWithFromCrawler.from_crawler does not support the 'feed_options' keyword argument. Add a 'feed_options' parameter to its signature to remove this warning. This parameter will become mandatory in a future version of Scrapy.
    warnings.warn(

tests/test_feedexport.py::S3FeedStoragePreFeedOptionsTest::test_init
tests/test_feedexport.py::S3FeedStoragePreFeedOptionsTest::test_init
  /build/Scrapy-2.11.1/scrapy/extensions/feedexport.py:49: ScrapyDeprecationWarning: S3FeedStorageWithoutFeedOptions does not support the 'feed_options' keyword argument. Add a 'feed_options' parameter to its signature to remove this warning. This parameter will become mandatory in a future version of Scrapy.
    warnings.warn(

tests/test_feedexport.py::FTPFeedStoragePreFeedOptionsTest::test_from_crawler
tests/test_feedexport.py::FTPFeedStoragePreFeedOptionsTest::test_from_crawler
  /build/Scrapy-2.11.1/scrapy/extensions/feedexport.py:49: ScrapyDeprecationWarning: FTPFeedStorageWithoutFeedOptionsWithFromCrawler.from_crawler does not support the 'feed_options' keyword argument. Add a 'feed_options' parameter to its signature to remove this warning. This parameter will become mandatory in a future version of Scrapy.
    warnings.warn(

tests/test_feedexport.py::FTPFeedStoragePreFeedOptionsTest::test_init
tests/test_feedexport.py::FTPFeedStoragePreFeedOptionsTest::test_init
  /build/Scrapy-2.11.1/scrapy/extensions/feedexport.py:49: ScrapyDeprecationWarning: FTPFeedStorageWithoutFeedOptions does not support the 'feed_options' keyword argument. Add a 'feed_options' parameter to its signature to remove this warning. This parameter will become mandatory in a future version of Scrapy.
    warnings.warn(

tests/test_utils_datatypes.py::CaseInsensitiveDictTest::test_getdefault
tests/test_utils_datatypes.py::CaselessDictTest::test_getdefault
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:95: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = CaselessDict()

tests/test_utils_datatypes.py::CaseInsensitiveDictTest::test_setdefault
tests/test_utils_datatypes.py::CaselessDictTest::test_setdefault
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:101: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = CaselessDict({"a": 1, "b": 2})

tests/test_utils_datatypes.py::CaselessDictTest::test_caseless
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:79: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = self.dict_class()

tests/test_utils_datatypes.py::CaselessDictTest::test_contains
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:132: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = self.dict_class()

tests/test_utils_datatypes.py::CaselessDictTest::test_copy
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:185: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    h1 = self.dict_class({"header1": "value"})

tests/test_utils_datatypes.py::CaselessDictTest::test_copy
tests/test_utils_datatypes.py::CaselessDictTest::test_copy
  /build/Scrapy-2.11.1/scrapy/utils/datatypes.py:55: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    return self.__class__(self)

tests/test_utils_datatypes.py::CaselessDictTest::test_delete
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:89: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = self.dict_class({"key_lower": 1})

tests/test_utils_datatypes.py::CaselessDictTest::test_fromkeys
tests/test_utils_datatypes.py::CaselessDictTest::test_fromkeys
tests/test_utils_datatypes.py::CaselessDictTest::test_fromkeys
tests/test_utils_datatypes.py::CaselessDictTest::test_fromkeys
tests/test_utils_datatypes.py::CaselessDictTest::test_normvalue
  /build/Scrapy-2.11.1/scrapy/utils/datatypes.py:80: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    return cls((k, value) for k in keys)

tests/test_utils_datatypes.py::CaselessDictTest::test_fromkeys
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:122: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    instance = self.dict_class()

tests/test_utils_datatypes.py::CaselessDictTest::test_init_dict
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:24: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = self.dict_class(seq)

tests/test_utils_datatypes.py::CaselessDictTest::test_init_mapping
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:49: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = self.dict_class(seq)

tests/test_utils_datatypes.py::CaselessDictTest::test_init_mutable_mapping
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:74: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = self.dict_class(seq)

tests/test_utils_datatypes.py::CaselessDictTest::test_init_pair_sequence
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:30: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = self.dict_class(seq)

tests/test_utils_datatypes.py::CaselessDictTest::test_normkey
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:149: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = MyDict()

tests/test_utils_datatypes.py::CaselessDictTest::test_normvalue
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:161: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = MyDict({"key": 1})

tests/test_utils_datatypes.py::CaselessDictTest::test_normvalue
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:165: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = MyDict()

tests/test_utils_datatypes.py::CaselessDictTest::test_normvalue
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:170: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = MyDict()

tests/test_utils_datatypes.py::CaselessDictTest::test_normvalue
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:175: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = MyDict()

tests/test_utils_datatypes.py::CaselessDictTest::test_pop
  /build/Scrapy-2.11.1/tests/test_utils_datatypes.py:137: ScrapyDeprecationWarning: scrapy.utils.datatypes.CaselessDict is deprecated, please use scrapy.utils.datatypes.CaseInsensitiveDict instead
    d = self.dict_class()

tests/test_utils_misc/test_return_with_argument_inside_generator.py::UtilsMiscPy3TestCase::test_generators_return_none
tests/test_utils_misc/test_return_with_argument_inside_generator.py::UtilsMiscPy3TestCase::test_generators_return_none
tests/test_utils_misc/test_return_with_argument_inside_generator.py::UtilsMiscPy3TestCase::test_generators_return_something
tests/test_utils_misc/test_return_with_argument_inside_generator.py::UtilsMiscPy3TestCase::test_generators_return_something
tests/test_utils_misc/test_return_with_argument_inside_generator.py::UtilsMiscPy3TestCase::test_generators_return_something
tests/test_utils_misc/test_return_with_argument_inside_generator.py::UtilsMiscPy3TestCase::test_generators_return_something
tests/test_utils_misc/test_return_with_argument_inside_generator.py::UtilsMiscPy3TestCase::test_generators_return_something
  /build/Scrapy-2.11.1/scrapy/utils/misc.py:249: DeprecationWarning: ast.NameConstant is deprecated and will be removed in Python 3.14; use ast.Constant instead
    value is None or isinstance(value, ast.NameConstant) and value.value is None

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/test_utils_python.py::UtilsPythonTestCase::test_get_func_args - twisted.trial.unittest.FailTest: Lists differ: ['args', 'kwargs'] != []
= 1 failed, 2598 passed, 323 skipped, 159 deselected, 18 xfailed, 289 warnings in 326.74s (0:05:26) =
/nix/store/9wnvhjyxjykwn5y06xc9a2h8rs5fbfia-stdenv-linux/setup: line 1579: pop_var_context: head of shell_variables not a function context
error: builder for '/nix/store/rzksfsqch017mc6km2s79naqqd2405ss-python3.12-scrapy-2.11.1.drv' failed with exit code 1
error: 1 dependencies of derivation '/nix/store/293xk1knwvkfsg3drczbb6vzh2nvgib4-man-paths.drv' failed to build
error: 1 dependencies of derivation '/nix/store/gsb5ybmyrs0766dmgp3qmlwfy56ajnr0-python3.12-scrapy-2.11.1_fish-completions.drv' failed to build
error: 1 dependencies of derivation '/nix/store/96i4p8wwsnwj8dqgpjzblb4crrv2hrid-python3.12-scrapy-fake-useragent-1.4.4.drv' failed to build
error: 1 dependencies of derivation '/nix/store/v03yh8fw9bc2c2b0785x95gbsxf6axnw-python3.12-scrapy-splash-0.9.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/y5a3zqc43bd7pmz3hfd577bgjz2pqvcj-system-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/czw1ij9p1l73zb8h1nrn9w7b1m1h426z-nixos-system-athenaos-24.05pre617818.58a1abdbae32.drv' failed to build

Additional context

Add any other context about the problem here.

Notify maintainers

@vinnymeller

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.6.28, NixOS, 24.05 (Uakari), 24.05pre615148.6143fc5eeb9c`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.2`
 - channels(root): `"home-manager, nixos, nixpkgs"`
 - nixpkgs: `/home/athena/.nix-defexpr/channels/nixpkgs`

---

Add a 👍 reaction to issues you find important.

[vinnymeller]

This is fixed in upstream here

Only 1 part of 1 test is failing. Could just ignore the test for now as is proposed here or could bump to an unstable version since scrapy's releases seem fairly irregular / sometimes far apart

[D3vil0p3r]

What about to create a patch with those changes?

[vinnymeller]

is there any general preference between an unstable version & patch? i checked & confirmed that both updating to unstable @ the commit of the linked fix passes all tests, and also just patching the one test at the current release passes all tests

[D3vil0p3r]

Patching could prevent us to push the package to an unstable version of the tool and still keep a stable version.

[FliegendeWurst]

Fixed in #309729