-
Notifications
You must be signed in to change notification settings - Fork 763
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor OkWrap
to not call .into_py(py)
#3595
Conversation
I didn't dare to do this refactor as part of the async work, but I fully agree with this simplification |
pub fn map_result_into_ptr<T: IntoPy<PyObject>>( | ||
py: Python<'_>, | ||
result: PyResult<T>, | ||
) -> PyResult<*mut ffi::PyObject> { | ||
result.map(|obj| obj.into_py(py).into_ptr()) | ||
} | ||
|
||
/// This is a follow-up function to `OkWrap::wrap` that converts the result into | ||
/// a safe wrapper. | ||
pub fn map_result_into_py<T: IntoPy<PyObject>>( | ||
py: Python<'_>, | ||
result: PyResult<T>, | ||
) -> PyResult<PyObject> { | ||
result.map(|err| err.into_py(py)) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pub fn map_result_into_ptr<T: IntoPy<PyObject>>( | |
py: Python<'_>, | |
result: PyResult<T>, | |
) -> PyResult<*mut ffi::PyObject> { | |
result.map(|obj| obj.into_py(py).into_ptr()) | |
} | |
/// This is a follow-up function to `OkWrap::wrap` that converts the result into | |
/// a safe wrapper. | |
pub fn map_result_into_py<T: IntoPy<PyObject>>( | |
py: Python<'_>, | |
result: PyResult<T>, | |
) -> PyResult<PyObject> { | |
result.map(|err| err.into_py(py)) | |
} | |
pub fn map_result_into_ptr<T, E>( | |
py: Python<'_>, | |
result: Result<T, E>, | |
) -> PyResult<*mut ffi::PyObject> | |
where | |
T: IntoPy<PyObject>, | |
PyErr: From<E>, | |
{ | |
Ok(result?.into_py(py).into_ptr()) | |
} | |
/// This is a follow-up function to `OkWrap::wrap` that converts the result into | |
/// a safe wrapper. | |
pub fn map_result_into_py<T, E>( | |
py: Python<'_>, | |
result: Result<T, E>, | |
) -> PyResult<PyObject> | |
where | |
T: IntoPy<PyObject>, | |
PyErr: From<E>, | |
{ | |
Ok(result?.into_py(py)) | |
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess it's a bit of a style thing, I slightly prefer the combinators over ?
in these one-line implementations.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The suggestion is not about the ?
but using Result<T, E>
instead of PyResult<T>
. If you use map_err
instead of ?
, you can in fact relax the bound to E: Into<PyErr>
instead of PyErr: From<E>
, and that's maybe better (I should also relax the bound in async support implementation by the way)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah I see. I take your point (and I did initially accept Result<T, E>
) but having the E: Into<PyErr>
bound introduced on this function and doing .map_err
in here instead of directly in the macro generated code means this internal function map_result_into_ptr
appears in the compiler error.
I didn't think that was useful (that was the UI test regression noted in the comment below) so I kept with just PyResult
for now as all we need.
I nearly changed the async support but actually I think it's better if I don't touch that in this PR so that it conflicts less with what you've got stacked up.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[...] means this internal function
map_result_into_ptr
appears in the compiler error.
Makes sense indeed.
@@ -33,7 +33,7 @@ fn test_compile_errors() { | |||
t.compile_fail("tests/ui/invalid_pymethod_receiver.rs"); | |||
t.compile_fail("tests/ui/missing_intopy.rs"); | |||
// adding extra error conversion impls changes the output | |||
#[cfg(all(target_os = "linux", not(any(feature = "eyre", feature = "anyhow"))))] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why this modification?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a bit of a drive-by but I was working on macOS today and I broke CI in the first attempt because this UI test regressed. Turns out the macos and linux outputs are the same.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(I think) I added this because the output is different on Windows. The error message for this lists all the variants, and windows has additional error classes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep that's what I recall too, which is the why I left windows in the not(any())
list
👍 for me |
Thanks again for review, as this touches more code which other contributors initially helped design I'll wait to merge a little bit in case of other opinions. But not too long, as you'll need to rebase off this 🙂 |
Summary: NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is! In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place: ## [PyO3] * Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>` * Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review. * Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already. ## [PyPi + Rust] * Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1) * Added `README.md` notes for future upgrades * Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22) * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344) * Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2) * Removed old `third-party/pypi/pydantic-core` versions * Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3) * Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865) * Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1) * Removed old `third-party/pypi/tokenizers` versions * Fixed `third-party/pypi/tokenizers/BUCK` * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc. * Removed Windows support from `third-party/pypi/tokenizers` ## [Rust] * Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6) * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed: ```bash error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior. --> fbcode/hphp/hack/src/package/types.rs:76:16 | 76 | self.0.take(value) | ^^^^ | = note: `-D deprecated` implied by `-D warnings` = help: to override `-D warnings` add `#[allow(deprecated)]` ``` Reviewed By: capickett Differential Revision: D56671179 fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
Summary: NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is! In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place: ## [PyO3] * Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>` * Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review. * Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already. ## [PyPi + Rust] * Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1) * Added `README.md` notes for future upgrades * Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22) * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344) * Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2) * Removed old `third-party/pypi/pydantic-core` versions * Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3) * Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865) * Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1) * Removed old `third-party/pypi/tokenizers` versions * Fixed `third-party/pypi/tokenizers/BUCK` * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc. * Removed Windows support from `third-party/pypi/tokenizers` ## [Rust] * Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6) * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed: ```bash error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior. --> fbcode/hphp/hack/src/package/types.rs:76:16 | 76 | self.0.take(value) | ^^^^ | = note: `-D deprecated` implied by `-D warnings` = help: to override `-D warnings` add `#[allow(deprecated)]` ``` Reviewed By: capickett Differential Revision: D56671179 fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
Summary: NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is! In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place: ## [PyO3] * Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>` * Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review. * Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already. ## [PyPi + Rust] * Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1) * Added `README.md` notes for future upgrades * Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22) * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344) * Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2) * Removed old `third-party/pypi/pydantic-core` versions * Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3) * Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865) * Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1) * Removed old `third-party/pypi/tokenizers` versions * Fixed `third-party/pypi/tokenizers/BUCK` * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc. * Removed Windows support from `third-party/pypi/tokenizers` ## [Rust] * Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6) * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed: ```bash error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior. --> fbcode/hphp/hack/src/package/types.rs:76:16 | 76 | self.0.take(value) | ^^^^ | = note: `-D deprecated` implied by `-D warnings` = help: to override `-D warnings` add `#[allow(deprecated)]` ``` Reviewed By: capickett Differential Revision: D56671179 fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
Summary: NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is! In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place: ## [PyO3] * Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>` * Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review. * Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already. ## [PyPi + Rust] * Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1) * Added `README.md` notes for future upgrades * Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22) * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344) * Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2) * Removed old `third-party/pypi/pydantic-core` versions * Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3) * Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865) * Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1) * Removed old `third-party/pypi/tokenizers` versions * Fixed `third-party/pypi/tokenizers/BUCK` * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc. * Removed Windows support from `third-party/pypi/tokenizers` ## [Rust] * Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6) * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed: ```bash error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior. --> fbcode/hphp/hack/src/package/types.rs:76:16 | 76 | self.0.take(value) | ^^^^ | = note: `-D deprecated` implied by `-D warnings` = help: to override `-D warnings` add `#[allow(deprecated)]` ``` Reviewed By: capickett Differential Revision: D56671179 fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
Summary: NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is! In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place: ## [PyO3] * Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>` * Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review. * Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already. ## [PyPi + Rust] * Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1) * Added `README.md` notes for future upgrades * Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22) * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344) * Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2) * Removed old `third-party/pypi/pydantic-core` versions * Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3) * Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865) * Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1) * Removed old `third-party/pypi/tokenizers` versions * Fixed `third-party/pypi/tokenizers/BUCK` * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc. * Removed Windows support from `third-party/pypi/tokenizers` ## [Rust] * Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6) * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed: ```bash error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior. --> fbcode/hphp/hack/src/package/types.rs:76:16 | 76 | self.0.take(value) | ^^^^ | = note: `-D deprecated` implied by `-D warnings` = help: to override `-D warnings` add `#[allow(deprecated)]` ``` Reviewed By: capickett Differential Revision: D56671179 fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
Summary: NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is! In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place: ## [PyO3] * Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>` * Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review. * Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already. ## [PyPi + Rust] * Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1) * Added `README.md` notes for future upgrades * Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22) * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344) * Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2) * Removed old `third-party/pypi/pydantic-core` versions * Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3) * Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865) * Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1) * Removed old `third-party/pypi/tokenizers` versions * Fixed `third-party/pypi/tokenizers/BUCK` * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc. * Removed Windows support from `third-party/pypi/tokenizers` ## [Rust] * Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6) * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed: ```bash error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior. --> fbcode/hphp/hack/src/package/types.rs:76:16 | 76 | self.0.take(value) | ^^^^ | = note: `-D deprecated` implied by `-D warnings` = help: to override `-D warnings` add `#[allow(deprecated)]` ``` Reviewed By: capickett Differential Revision: D56671179 fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
Summary: NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is! In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place: ## [PyO3] * Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>` * Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review. * Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already. ## [PyPi + Rust] * Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1) * Added `README.md` notes for future upgrades * Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22) * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344) * Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2) * Removed old `third-party/pypi/pydantic-core` versions * Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3) * Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865) * Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1) * Removed old `third-party/pypi/tokenizers` versions * Fixed `third-party/pypi/tokenizers/BUCK` * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc. * Removed Windows support from `third-party/pypi/tokenizers` ## [Rust] * Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6) * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed: ```bash error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior. --> fbcode/hphp/hack/src/package/types.rs:76:16 | 76 | self.0.take(value) | ^^^^ | = note: `-D deprecated` implied by `-D warnings` = help: to override `-D warnings` add `#[allow(deprecated)]` ``` Reviewed By: capickett Differential Revision: D56671179 fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
Summary: NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is! In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place: ## [PyO3] * Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>` * Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review. * Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already. ## [PyPi + Rust] * Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1) * Added `README.md` notes for future upgrades * Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22) * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344) * Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2) * Removed old `third-party/pypi/pydantic-core` versions * Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3) * Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865) * Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1) * Removed old `third-party/pypi/tokenizers` versions * Fixed `third-party/pypi/tokenizers/BUCK` * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc. * Removed Windows support from `third-party/pypi/tokenizers` ## [Rust] * Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6) * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed: ```bash error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior. --> fbcode/hphp/hack/src/package/types.rs:76:16 | 76 | self.0.take(value) | ^^^^ | = note: `-D deprecated` implied by `-D warnings` = help: to override `-D warnings` add `#[allow(deprecated)]` ``` Reviewed By: capickett Differential Revision: D56671179 fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
Summary: NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is! In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place: ## [PyO3] * Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>` * Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review. * Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already. ## [PyPi + Rust] * Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1) * Added `README.md` notes for future upgrades * Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22) * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344) * Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2) * Removed old `third-party/pypi/pydantic-core` versions * Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3) * Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865) * Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1) * Removed old `third-party/pypi/tokenizers` versions * Fixed `third-party/pypi/tokenizers/BUCK` * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc. * Removed Windows support from `third-party/pypi/tokenizers` ## [Rust] * Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6) * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed: ```bash error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior. --> fbcode/hphp/hack/src/package/types.rs:76:16 | 76 | self.0.take(value) | ^^^^ | = note: `-D deprecated` implied by `-D warnings` = help: to override `-D warnings` add `#[allow(deprecated)]` ``` Reviewed By: capickett Differential Revision: D56671179 fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
Motivated by https://github.com/PyO3/pyo3/pull/3588/files#r1403933922
The goal here is to keep the
OkWrap
trait as simple as it can possibly be.@wyfo what do you think? It conflicts a little bit with #3588, so that'd need rebasing if this merged, but I like how this keeps
OkWrap
simple.