Base logging todo

[c42f]

Now that #24490 is merged containing the core functionality, there's still a lot of loose ends to tidy up, but at least they can be merged incrementally. Here's a list [edit - organized and sorted by priority]:

Important user-visible changes for 1.0

Core log message generation

• Dispatch flisp syntax/lowering depwarns to logging system (PR: Send all parser and lowering depwarns to the logging system #25257)
• Dispatch C-level runtime warnings to logging system, arguably most importantly jl_binding_deprecation_warning (should fix any final parts of depwarn warnings should show location of the problem #19686 and logging() in .juliarc.jl does not work #24028)
• Remove --depwarn=error and replace with something finer grained. Arguably blocked until the previous two points are resolved.

User experience / logger backends

• More capable Logging.ConsoleLogger, with markdown-based pretty printing, and other formatting niceties which will allow SimpleLogger to be further simplified. See MicroLogging.InteractiveLogger for a start at this. See also Option to disable the location info of a logging message #25106, and RFC: Base logging #24490 (comment) at least. This is less API changes, and more setting a strong precedent for the data people include in their log events when they use the system.
• Standardize progress API or simply remove all reference to it from the documentation for now (see Generalized progress API (for download etc.) #8064, and comments starting at [WIP] Flexible and convenient logging Juleps#30 (comment), also Progress monitoring fallback to the console JunoLab/Juno.jl#29)
• Refine conventions for harvesting of backtraces (see ConsoleLogger for more fully featured log printing #25370 (comment) )
• Simple logger configuration ( Logging configuration and installation #25404 )

Non-API breaking tweaks and bug fixes

• depwarn frame sanitization (see Depwarns at global scope have incorrect caller #25130 and RFC: Base logging #24490 (comment))
• User documentation (document new logging API #25154)
• Resolve naming conflict with @kmsquire's package ( See also Build is flaky due to ~/.julia/ being used #25182 and PR: Blacklist the original Logging module for julia-0.7 METADATA.jl#12653 )
• Resolve issues with deprecated tests ( See 5d77d1b#r26334167 and PR Logging depwarn cleanup #25239 / rm tests of deprecated logging functions #25229 )

Non critical breaking changes

• Make @show use the logging system so that shown variables can be captured as values rather than text when desired. (Main issue - which log level should this be? debug kinda makes sense, but info is arguably more useful for quick hacking.) Didn't have time to do this before 1.0.
• Improve @test_logs usage for matching multiple log messages - nicer syntax + be slightly more conservative about features.
• Clean up @test_logs failure Test.Result type - improve the way test sets capture results.

Wildcards - advice needed

• Log forwarding from remote workers. In particular people using parallel workloads probably care about this. Pkg2 also uses separate processes extensively, and the tests clearly ended up a bit of a mess, with some logs being captured on stderr from a separate process.

[StefanKarpinski]

How much of this is user-visible? We should heavily prioritize user-visible changes since we're trying to do a feature freeze here and get an alpha out. Non-user-visible changes can continue to be made during the alpha period, although ideally those should be minimal, this seems important enough that we can allow it.

[c42f]

I've organized the list here into a rough priority. I think I've got a fairly solid handle on how to do most of these things because I've already done the bulk of the investigation work and a fair chunk of the code. As to priority, I've laid them out as I see them, roughly, but happy to rearrange a bit.

The main thing I feel quite unsure about at this point is the remote workers issue as I really haven't done any investigation here.

[StefanKarpinski]

Remove --depwarn=error and replace with something finer grained.

Note that we can always keep this and add a more fine-grained mechanism on top of it in the future and just redefine the behavior of this option in terms of the more fine-grained mechanism.

[c42f]

That's a good point, it's not strictly required that this be deprecated. My concern is that when implementing this with a logger, --depwarn=error is essentially a logger configuration mechanism and I suspect a command line option will compose badly with whatever logging configuration system we end up with. It can probably be hacked together in some way.

[StefanKarpinski]

I guess we can always deprecate --depwarn and codename the release "Inception".

[samoconnor]

@c42f Does the new logging infrastructure have support for multiple simultaneous log streams?
e.g. could I have a seperate log stream per task? or collect all the logs from a function call into a seperate log stream?

[c42f]

Yes, this is exactly the way it's designed: with_logger(f, logger) calls f with all logs for the current task directed to logger. In addition, any child tasks which are spawned by f (or any deeper part of the call stack) inherit logger as the task local logger.

[c42f]

This list is kinda out of date at this point. The items which haven't been dealt with are either low priority or are tracked in their own separate issues. So I think I'll close this.