WIP: Initial prototype of how to exclude code from Base

[quinnj]

An idea of getting started on #5155.

Currently implemented:

• ability to do make NODATES=1 when building julia
• creation of an Excluded module for pre-definitions of modules/types to potentially exclude
• With a regular make, everything works as before, with make NODATES=1, Date and DateTime types are defined, but any usage immediately errors

Additional things to think about:

• This is obviously one of the simplest cases of code that is almost completely stand-alone
• What about "more entangled" code like BigInt/BigFloat where usage is more wide-spread?
• Do we actually need something like the Excluded module? I found it at least allowed the type definitions so that any downstream code was still able to be defined, even if any downstream usage would immediately error.
• Where should these theoretical "default packages" live? In the base/ directory? In a stand-alone github repo? A mix of both? (if the latter, we probably need some more make magic to first fetch a repo (if the code is to be included) so that it can be included in sysimg.jl)

[JeffBezanson]

Great to see some thinking and progress on this.

I suspect it will be too complicated to have a mechanism specifically for things that are either part of Base or excluded. I think extern (#15705) is the way to get the equivalent of the Excluded module.

I'm envisioning something like this: (1) Many pieces of Base become packages, (2) There is an stdlib directory that gets populated at build time by git cloning a certain set of packages, (3) Pkg somehow finds these packages so that you only need to do using in a default build to get them.

[tkelman]

I'm with Jeff, I don't think the mechanism here is quite the right one but it's useful to brainstorm.

Pkg will have to get a little bit smarter to handle everything properly, but we need to revise the way Pkg works anyway. There are questions about whether "default packages" should be version-pinned or allowed to update, and how you handle precedence of different copies of a package installed by default and also installed elsewhere (perhaps Pkg is allowed to update the second copy to use newer versions).

Even for things that stay in Base in the short term, it would be useful to try to modularize them and more clearly express their interdependencies so you could more feasibly do an incremental modification and rebuild of different pieces as opposed to the monolithic single list of parts of Base we have now.

[quinnj]

Yes, this is definitely just in brainstorm-mode and trying some things out.

A couple of thoughts/questions:

• Jeff, in your setup, are you imagining these stdlib packages not being included at all in the sysimg.jl? They'd literally be external up until the user did using StdLibPackage in a script/repl? I'm wondering how we'd "deprecate" the current setup where everything's included; even with extern, we'd probably have to have a stdlib.h type file included in Base in order to get the types into the system
• I would imagine we'd want to have stdlib packages pinned by default and officially bump the pins with overall Julia releases. You could still possibly do a Pkg.clone on a stdlib package to get the "dev" version. We could architect it so that using StdLibPackage looked first for a dev version, falling back to the default pinned version in the stdlib directory

[JeffBezanson]

Yes, they should not be included in the sysimg. We should start with bits that can be separated very cleanly. Hopefully there aren't that many cases where Base really needs to refer to a type in one of the stdlib packages. If it does, we can either (1) use extern or (2) define the type and minimal methods in Base, with most functions in a package (that's probably how we'd handle e.g. BigInt).

[quinnj]

Yeah, another case I ran into was deprecated.jl, where it refers to all sorts of types in Base (including DateTime). I guess we could move the deprecations to their respective modules.