Uplift to the nursery?

[aturon]

I've long wanted to kick off this conversation, and this seems as good a place to do it as any.

The winapi crates have become a pillar of the Rust ecosystem, with a massive number of crates (likely the majority of the ecosystem) transitively depending on them. They play a role somewhat similar to the libc crate, which we also see as providing foundational system bindings that show up transitively in most projects.

Being such a core part of the ecosystem, I want to propose that the crate be given more "official" status by migration into rust-lang-nursery, with the ultimate being inclusion in rust-lang itself. This move would:

• Immediately increase the pool of maintainers, with all of the libs team providing help with maintenance.
• Increase mutual visibility between winapi and the various Rust teams.
• Start us down the road to deeper CI support on the Windows side of the ecosystem.

Moving to rust-lang itself requires an RFC, but moving to the nursery only requires lib team approval.

Is this something you would be interested in and open to?

[retep998]

I actually want to move away from the multiple crate model, especially since I abandoned the links = "foo" thing after that turned out to cause problems. I have a plan for how I want to reorganize things for 0.3 taking advantage of cargo features and such. After that's done then I'll consider migrating this to the nursery.

[alexcrichton]

@retep998 could you elaborate on your plans for reorganization? Compile times are already a problem and I'd just want to make sure that's taken into account.

[retep998]

Well there are several things I'll do to improve compile times like making enums simple integers, removing Debug, not emitting the COM shims by default, but also having opt-in cargo features for entire swaths of APIs so they don't get compiled by default.

[retep998]

For import libraries, currently I bundle import libraries with each -sys crate for windows-gnu since I can't assume the presence of a full mingw installation. What I'll do instead is have a single crate with all the import libraries, but it'll be a target specific dependency so only people using windows-gnu will have to download it, and for normal users it'll be downloaded just once and cached anyways.

[alexcrichton]

Interesting! Requiring a Cargo opt-in to get a bunch of the API sounds like a good path forward, although I may also recommend benchmarking that just to be sure that parsing or expanding isn't too onerous. I suspect though that avoiding codegen (with the impl shim) will shave a huge amount of time.

I'm a little unfamiliar with the import library story, though, so could you elaborate a bit? Do some MinGW installations come with "all the libraries" and others only half "half the libraries"? Do you generate these import libraries or are they copied from a known location? It'd be kinda cool if the import libraries could be generated on the fly (not sure how complicated they are), but that may be farther out.

I'd be somewhat wary of sticking the entire windows ecosystem into a crate, is there a downside of shipping the import libraries with each crate today? Are there size or download concerns?

[retep998]

To avoid needing import libraries we'd need rust-lang/rust#30027. Once that lands I'd be able to modify everything to use that and ditch import libraries. Rust too would be able to avoid having to vendor mingw import libraries that way. Right now I get my import libraries from the mingw-w64 installed by msys2. Basically since windows-gnu is intended to be a standalone toolchain that can work without the user installing their own Mingw, the only import libraries I can rely on to exist are the ones that Rust itself vendors, everything else I currently have to supply otherwise windows-gnu users will have to install Mingw just to get those import libraries. There is the annoyance that basically everyone who ends up depending on a -sys crate has the import library included in the crate, which is a bit of extra bandwidth taken up for what is mostly people who don't even use Windows. By moving them to a dedicated target specific crate, they will only be downloaded for windows-gnu users, benefiting everyone else.

All the -sys crates currently define just the functions, with all the types and constants already defined in winapi, and there are far more types and constants than there are functions, so having it all together shouldn't impact compile times too much, and judicious use of cargo features to selectively enable/disable API sets will eliminate that impact entirely.

[alexcrichton]

That all sounds like a good plan to me, thanks for the explanation!

To clarify on the Cargo side of things, with nicer syntax for target-specific dependencies along with lazily downloading packages, it may end up being the case that winapi related crates aren't download on non-Windows systems most of the time anyway. In that sense the import libraries would only be "unnecessarily downloaded" on MSVC compiles, right?

If the route of "one crate to hold all import libraries" is taken, is the set of import libraries to be included in that crate essentially fixed? The crate is cached on each host, but it'd still have to be downloaded on all hosts (think CI builds) and also whenever the crate changes (e.g. a new import library was added). I guess to me the import libraries seems small enough (we're talking kilobytes, not megabytes, right?) and when compressed they're even smaller, so downloading a few extra as part of a compile once (they're cached as well) doesn't seem so bad?

The tradeoff seems to be either:

• MinGW builds download everything
• MSVC builds download a little bit too extra

This definitely depend on how large "everything" is and how large "too extra" is, but I'd guess at least that the "everything" crate would actually be in the megabytes?

[retep998]

If I shove all the x86_64 import libraries winapi currently has in a .tar.gz it takes up 124KB.

[alexcrichton]

Wow, that's tiny! Sounds like the point is moot after all :)

Thanks again for the updates! Sounds exciting to move to the nursery!

[retep998]

As an update, I've switched to rolling my own import libraries instead of using the ones provided by MinGW-w64, which enables access to functions from newer versions of Windows. Unfortunately the import libraries provided by Rust itself conflict with some of them. I've also added a bunch more import libraries so the current package size is around 600KB. If there was some way to reliably have access to either dlltool or a cross platform Rust library that does the equivalent, I'd be able to bring down the package size by an order of magnitude.

[crlf0710]

Now that 0.3 is released, what's the plan next? Ping @retep998 @aturon @alexcrichton

[pravic]

Ping, too.