Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade third_party/pyyaml to 03c67afd452cdff45b41bfe65e19a2fb5b80a0e8 #54001

Merged
merged 2 commits into from
Jul 19, 2024

Conversation

cbracken
Copy link
Member

@cbracken cbracken commented Jul 19, 2024

We were using a very old version of this library that broke with Python3.12. This updates to a Python 3.12 compatible SHA.

The specific breakage at the former commit was that collections.Hashable was deprecated and moved to collections.abc.Hashable in Python 3.7. It was removed in Python 3.8. For anyone on newer versions of Python3, such as Python 3.9.6 which ships with macOS 15 Sequoia, this is broken.

The error on versions of Python newer than 3.8 is:

________ running 'python3 src/flutter/tools/pub_get_offline.py' in '/Users/chris/Developer/flutter/engine'
Traceback (most recent call last):
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/pub_get_offline.py", line 167, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/pub_get_offline.py", line 157, in main
    if not package_uses_workspace_resolution(package):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/pub_get_offline.py", line 76, in package_uses_workspace_resolution
    return yaml.safe_load(pubspec_file).get('resolution') == 'workspace'
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/__init__.py", line 75, in load
    return loader.get_single_data()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 37, in get_single_data
    return self.construct_document(node)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 46, in construct_document
    for dummy in generator:
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 398, in construct_yaml_map
    value = self.construct_mapping(node)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 204, in construct_mapping
    return super().construct_mapping(node, deep=deep)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 126, in construct_mapping
    if not isinstance(key, collections.Hashable):
                           ^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'collections' has no attribute 'Hashable'
Error: Command 'python3 src/flutter/tools/pub_get_offline.py' returned non-zero exit status 1 in /Users/chris/Developer/flutter/engine

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
  • I listed at least one issue that this PR fixes in the description above.
  • I added new tests to check the change I am making or feature I am adding, or the PR is test-exempt. See testing the engine for instructions on writing and running engine tests.
  • I updated/added relevant documentation (doc comments with ///).
  • I signed the CLA.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

We were using a very old version of this library that broke with
Python3.12. This updates to a Python 3.12 compatible SHA.

The specific breakage at the former commit was that
`collections.Hashable` was deprecated and moved to
`collections.abc.Hashable` in Python 3.7. It was removed in Python 3.8.
For anyone on newer versions of Python3, such as Python 3.9.6 which
ships with macOS 15 Sequoia, this is broken.

The error on versions of Python newer than 3.8 is:
```
________ running 'python3 src/flutter/tools/pub_get_offline.py' in '/Users/chris/Developer/flutter/engine'
Traceback (most recent call last):
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/pub_get_offline.py", line 167, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/pub_get_offline.py", line 157, in main
    if not package_uses_workspace_resolution(package):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/pub_get_offline.py", line 76, in package_uses_workspace_resolution
    return yaml.safe_load(pubspec_file).get('resolution') == 'workspace'
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/__init__.py", line 75, in load
    return loader.get_single_data()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 37, in get_single_data
    return self.construct_document(node)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 46, in construct_document
    for dummy in generator:
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 398, in construct_yaml_map
    value = self.construct_mapping(node)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 204, in construct_mapping
    return super().construct_mapping(node, deep=deep)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 126, in construct_mapping
    if not isinstance(key, collections.Hashable):
                           ^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'collections' has no attribute 'Hashable'
Error: Command 'python3 src/flutter/tools/pub_get_offline.py' returned non-zero exit status 1 in /Users/chris/Developer/flutter/engine
```
@cbracken cbracken requested a review from matanlurey July 19, 2024 01:35
@flutter-dashboard
Copy link

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact "@test-exemption-reviewer" in the #hackers channel in Chat (don't just cc them here, they won't see it! Use Discord!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

@cbracken cbracken added the autosubmit Merge PR when tree becomes green via auto submit App label Jul 19, 2024
@matanlurey
Copy link
Contributor

test-exempt: 💩

Copy link
Contributor

auto-submit bot commented Jul 19, 2024

auto label is removed for flutter/engine/54001, due to - The status or check suite Linux linux_fuchsia has failed. Please fix the issues identified (or deflake) before re-applying this label.

@auto-submit auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jul 19, 2024
@cbracken cbracken added the autosubmit Merge PR when tree becomes green via auto submit App label Jul 19, 2024
@cbracken cbracken merged commit c2190d9 into flutter:main Jul 19, 2024
33 checks passed
@cbracken cbracken deleted the fml branch July 19, 2024 03:17
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jul 19, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jul 19, 2024
auto-submit bot pushed a commit to flutter/flutter that referenced this pull request Jul 19, 2024
…152018)

flutter/engine@ea1e53a...969fb7a

2024-07-19 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Drop deprecated hash_code functions (#54000)" (flutter/engine#54002)
2024-07-19 [email protected] Upgrade third_party/pyyaml to 03c67afd452cdff45b41bfe65e19a2fb5b80a0e8 (flutter/engine#54001)
2024-07-19 [email protected] Don't leak binary messenger in tests (flutter/engine#53985)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC [email protected],[email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
cbracken added a commit to cbracken/flutter_engine that referenced this pull request Jul 22, 2024
Previously we were appending Flutter's fork of third_party/pyyaml to the
Python sys.path. In the case where a bot image or local install already
has pyyaml installed elsewhere on the path, ours will fail to be picked
up. Instead prepend to the path.

Issue spotted by jsimmons@.

Related: flutter#54001
auto-submit bot pushed a commit that referenced this pull request Jul 22, 2024
Previously we were appending Flutter's fork of third_party/pyyaml to the Python sys.path. In the case where a bot image or local install already has pyyaml installed elsewhere on the path, ours will fail to be picked up. Instead prepend to the path.

Issue spotted by jsimmons@.

Related: #54001

[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
TytaniumDev pushed a commit to TytaniumDev/flutter that referenced this pull request Aug 7, 2024
…lutter#152018)

flutter/engine@ea1e53a...969fb7a

2024-07-19 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Drop deprecated hash_code functions (flutter#54000)" (flutter/engine#54002)
2024-07-19 [email protected] Upgrade third_party/pyyaml to 03c67afd452cdff45b41bfe65e19a2fb5b80a0e8 (flutter/engine#54001)
2024-07-19 [email protected] Don't leak binary messenger in tests (flutter/engine#53985)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC [email protected],[email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Buchimi pushed a commit to Buchimi/flutter that referenced this pull request Sep 2, 2024
…lutter#152018)

flutter/engine@ea1e53a...969fb7a

2024-07-19 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Drop deprecated hash_code functions (flutter#54000)" (flutter/engine#54002)
2024-07-19 [email protected] Upgrade third_party/pyyaml to 03c67afd452cdff45b41bfe65e19a2fb5b80a0e8 (flutter/engine#54001)
2024-07-19 [email protected] Don't leak binary messenger in tests (flutter/engine#53985)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC [email protected],[email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
autosubmit Merge PR when tree becomes green via auto submit App
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants