Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compile with stable rust #5

Closed
6 tasks done
fafhrd91 opened this issue May 19, 2017 · 38 comments · Fixed by #969
Closed
6 tasks done

Compile with stable rust #5

fafhrd91 opened this issue May 19, 2017 · 38 comments · Fixed by #969

Comments

@fafhrd91
Copy link
Contributor

fafhrd91 commented May 19, 2017

PyO3 requires following unstable

@fafhrd91 fafhrd91 changed the title Stable rust Compile with stable rust Jun 12, 2017
@tailhook
Copy link

Doesn't proc macro work on stable rust? (or just partially) And there is proc-macro2

@messense
Copy link
Member

I think currently only proc macro for custom derive works on stable Rust, proc_macro_attribute don't.

@fafhrd91
Copy link
Contributor Author

proc_macro_attribute is unstable, info is in linked tracking issue.

@tailhook
Copy link

tailhook commented Dec 2, 2017

Can this trick be used instead of proc_macro_attribute ?

@fafhrd91
Copy link
Contributor Author

fafhrd91 commented Dec 2, 2017

no, it is not. pyo3 strictly requires proc_macro_attribute, because it requires access to token stream of impl sections.

@tailhook
Copy link

tailhook commented Dec 2, 2017

no, it is not. pyo3 strictly requires proc_macro_attribute, because it requires access to token stream of impl sections.

I think this is exactly what it does, i.e. given the example:

#[macro_use] extern crate procedural_masquerade;
extern crate proc_macro;

define_proc_macros! {
    #[allow(non_snake_case)]
    pub fn foo_internal__stringify_const(input: &str) -> String {
        format!("const STRINGIFIED: &'static str = {:?};", input)
    }
}

The foo_internal__stringify_const function receives original code and returns generated code. You get a token stream using syn crate as with macros 1.1.

And all this works by wrapping a body of the macro into a type definition to leverage macros 1.1, to get a token stream.

The only difference is that:

#[mod2init]
fn init_function() {
}

Would be written as:

mod2init! {
  fn init_function() {
  }
}

@fafhrd91
Copy link
Contributor Author

fafhrd91 commented Dec 2, 2017

I see. But that would be quite large task, I don’t think I can spend any time on it. Also there is specialization, I am not sure if it possible to avoid it

If you want to work on this task I can give you permission to repo

@fafhrd91
Copy link
Contributor Author

this features are stable now, const_unsafe_cell_new, const_size_of, const_ptr_null, const_ptr_null_mut

@termoshtt
Copy link

Stabilization of TryFrom has been reverted rust-lang/rust#50121

@fafhrd91
Copy link
Contributor Author

fafhrd91 commented May 3, 2018

pyo3 does not use TryFrom

@termoshtt
Copy link

impl<'source> $crate::std::convert::TryFrom<&'source $crate::PyObjectRef> for $t

The macro pyobject_extract! uses TryFrom. To use PyO3 in stable rust, this library itself must be compiled with stable rustc.
IMHO, there is no merit to use this unstable feature, and should be dropped.

konstin added a commit to konstin/pyo3 that referenced this issue May 5, 2018
@konstin
Copy link
Member

konstin commented May 5, 2018

@termoshtt Good catch! pyo3 implements TryFrom, even though it currently uses PyTryFrom instead. I put a feature gate onto the implementation of TryFrom and removed the try_from feature. Once TryFrom becomes stable, we can than replace PyTryFrom.

@termoshtt
Copy link

Drop of fn_must_use in afcc87e causes many warnings

warning: `#[must_use]` on methods is experimental (see issue #43302)
   --> src/typeob.rs:154:5
    |
154 |     #[must_use]
    |     ^^^^^^^^^^^
    |
    = help: add #![feature(fn_must_use)] to the crate attributes to enable

@konstin
Copy link
Member

konstin commented Dec 15, 2018

Unfortunately not, specialization is still not stable

@pohutukawa
Copy link

I'm trying to compile a Python manylinux wheel using PyO3. I know that it requires a nightly build, but isn't that (by now) a thing of history? It states the requirement of 1.34.0-nightly, however Rust stable is now at 1.35.0, but I'm still seeing the build fail (see below), though in my perception 1.35.0 (stable) is >= 1.34.0-nightly.

Any insights?

Error: pyo3 requires a nightly or dev version of Rust.
Installed version is: 1.35.0 (2019-05-20). Minimum required: 1.34.0-nightly (2019-02-06).
thread 'main' panicked at 'Aborting compilation due to incompatible compiler.', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/pyo3-0.6.0/build.rs:540:17

@bgw
Copy link

bgw commented May 30, 2019

@pohutukawa

my perception 1.35.0 (stable) is >= 1.34.0-nightly.

Nope. Nightly builds have additional feature flags turned on, most of which won't make it to the next stable release, so 1.34.0-nightly has a lot more features than 1.35.0 (including specialization).

@ikamensh
Copy link

ikamensh commented Jul 3, 2019

I find the message confusing due to the second line:

Error: pyo3 requires a nightly or dev version of Rust.
Installed version is: 1.35.0 (2019-05-20). Minimum required: 1.34.0-nightly (2019-02-06).

I wanted to create an issue about this, but luckily found this. Also, how about telling Python users who are newbies to rust how to get the nightly version right in the error message?

I did it using
rustup toolchain install nightly
rustup default nightly

@nickray
Copy link

nickray commented Dec 19, 2019

@konstin any thoughts on whether dtolnay's trick https://github.com/dtolnay/case-studies/blob/master/autoref-specialization/README.md linked to by @io12 above would be a feasible/pragmatic way to for this library to compile on stable?

@TheButlah
Copy link

TheButlah commented Dec 21, 2019

Having to use not only a new programming language, but the unstable version of that language, is not something I feel comfortable pitching to my coworkers (no matter how stable nightly actually may be). Because easy python interoperability is a necessity for any new language that my work would use, and because it seems like there aren't other options with the same level of python interoperability with Rust that Py03 provides, I cannot in good faith pitch using rust at work until this is using the stable version of rust 😢

@jcdyer
Copy link

jcdyer commented Mar 26, 2020

@TheButlah: You can create libraries to interoperate with python without using PyO3. Don't confuse the library with the language. You can currently either create native library bindings for your rust code, and interface with them using cffi, or use https://github.com/dgrunwald/rust-cpython. PyO3 provides a nicer interface, it's true, but if your requirement is stable rust, that interface isn't available to you.

@davidclevenger
Copy link

Bump: Is there any ETA / update on stable rust toolchain?

@davidhewitt
Copy link
Member

davidhewitt commented Apr 22, 2020

Progress slowly continues; we want to get there but need more volunteer time to get this over the finish line yet. (pyo3 maintainer time in the last few months has been spent mostly on soundness problems as well as bugfixes, which are also very important.)

Last items to resolve are in #697. I can offer mentoring advice to anyone interested in helping out with pieces of the solution in there.

@TheButlah
Copy link

TheButlah commented Apr 23, 2020

@jcdyer The emphasis is on easy interoperability, not just any interoperability, which is the purpose of PyO3. That being said, I did pitch rust to my coworkers, and got enough leeway to experiment with it in a side project - because I realized that nightly wasn't as bad as it sounded.

That being said, I work in a research lab not an enterprise company and our risk tolerance for these sorts of things is significantly higher, so continuing to focus on using stable would be a big selling point for the language overall. Glad to see how active this community is - it says a lot about the language and ecosystem :)

@kngwyu
Copy link
Member

kngwyu commented Apr 24, 2020

we want to get there but need more volunteer time to get this over the finish line yet

Yeah, I also think it's a matter of time.
Now I have a serious NeurIPS deadline. Please wait for a bit.

@nacaclanga
Copy link

Hello. It appears to me, from reading #210, that specialisisation is mostly used to implement class declaration. So how hard would it be to introducing a feature no-unstable-class-support to pyo3 that simply switches off class declaration, but allows code to be compiled on stable rust? People that want to use pyo3 instead of rust-cpython and can live without classes could then do so without having to use unstable.

Once specialisation is stable or pyo3 managed to get rid of it, class declaration would be allowed in no-unstable-class-support and the feature would become identical to the default configuration again. I think this would also be very much in line on how rustc itself adds new features.

An alternative would be the other way around: Make no-unstable-class-support the default and introduce a feature allow-unstable-class-support.

@programmerjake
Copy link
Contributor

if we're going to use a cargo feature, we should use the enable unstable code feature rather than the disable version, since cargo features should be additive since that's how cargo's designed.

as an example, if one crate uses the unstable code and another one doesn't, then using the enable version of the feature would correctly deduce that unstable code should be enabled for the crate that requires it. if it was the disable version then cargo would incorrectly disable the unstable code even though a crate requires it, leading to compile failure.

If anything requires unstable code, then you need to use the unstable compiler on all the crates anyway in order to have the compiler versions match, so that's not a limitation.

@nacaclanga
Copy link

Make sense. The reason I thought it up the other way round is, that it might make things appear from the documentation, but one should be able to work around this.

@philmb3487
Copy link

philmb3487 commented May 15, 2020

Hi. I don't know a lot about Rust, but trying to run an application I use often that recently introduced bits of rust.
Why can't PyO3 work with Rust 1.43.1, released last week? My OS ships stable rust. This ticket is 3 years old now, this is annoying.

@davidhewitt
Copy link
Member

Read comments above, work is ongoing to fix this but it takes time...

@jcdyer
Copy link

jcdyer commented May 15, 2020

@sinetek The answer to the question you asked is in the thread above.

If you would like to get it working, the recommended way to work with rust as a developer is to use rustup to manage your rust toolchains, so you can have nightly and stable running side by side. You can get rustup from https://rustup.rs.

Then run the following:

rustup toolchain install nightly
rustup default nightly

Then when you want do run on stable, you can switch back with rustup default stable. You can also set per-directory overrides with rustup override. Run rustup override --help for details.

You can find more detail here: https://github.com/rust-lang/rustup#working-with-nightly-rust

@konstin konstin mentioned this issue Jun 18, 2020
@davidhewitt
Copy link
Member

For all those following this thread... the current PyO3 master branch just had a PR merged which added support for stable Rust.

Extra special thanks to @kngwyu and @konstin who have been working to remove specialization from PyO3 for a long time - and as of this morning, we finally got there! 🎉 🚀 🐍 🦀

@ikamensh
Copy link

ikamensh commented Jun 21, 2020 via email

@gilescope
Copy link
Contributor

That's awesome progress you've been making this past 6 months! Huge congratulations on all your efforts.

@TheButlah
Copy link

excitement intensifies

@fafhrd91
Copy link
Contributor Author

congrats!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.