Design tracker

[c42f]

Here's a todo list of design issues.

Logging macros

• Lightweight @show like syntax (Added as @info "msg" expr for any expr, to give a keyword named Symbol(expr))
• Keyword namespacing: how do we distinguish keywords which have a standard meaning (id, file, line, progress) vs user-defined keys? (Added using underscore prefixes for _file, _line etc)
• Use logger as transport for @show macro, integrate with a collect_shown() function to get a list of shown values, not just printing them as text.

AbstractLogger conventions

• API naming review handle_message, shouldlog, min_enabled_level
• Decide on a default log level (should it be different for interactive vs production use? Eg, Verbose/MoreInfo vs Info levels? cf. isinteractive() based level switching in loading.jl)

Log record structure

• Clear semantics for the log message. The message is the stable part of a log record, and is good for identifying where it came from. Message templates with interpolated variables are somewhat conceptually at odds with the backend doing the formatting. Perhaps the message template is just a formatting hint? (Answer: the message is a human readable summary string in markdown format by convention. Structured data is passed in keywords)
• Systematic convention for logging exceptions in the common case of catch, log and continue. Should the exception be given as a keyword or as a positional parameter? (Answer: use the exception keyword. Add a backtrace as the second element of a tuple here if required.)
• What is the convention for inferring that a backtrace should or could be added to the log record? (Answer: Formatting keywords with type Exception will attempt to add a backtrace using catch_backtrace(). A type of Tuple{Exception,Any} implies a backtrace in the second argument).
• Are module, group, id, file, line required positional arguments in the backend? Or just additional key value pairs? (cf, handle_message argument list)
• Grouping of log records in addition to using the module (Added a group keyword)
• Consider another log level (verbose? notice?) between debug and warn, which would be enabled globally but disabled in the Logger by default.
• Variables which are captured into a message could be stored (?)
• Can we achieve a stable but unique message id? (Answer: hash the message AST passed to the macro, combined with module name and uniquified with a Set)
• Revisit id AST hashing - should it contain the keywords?

Filtering

• Clear semantics for early log filtering (what's a log level for? where is it looked up? Answer: log levels are for early filtering. They're looked up via the current logger in the TLS.)
• shouldlog() could allow pattern matching on the message template (before formatting)
• shouldlog() API stability: remove progress, max_log? (Done)

Formatting

• Uniform interface for message formatting; code sharing between AbstractLoggers
• Extensible interface for passing systematic formatting hints (Deferred for now.)
• For readability, markdown formatted log messages would be really cool (Done: all string messages are considered to be markdown by default.)
• Sort out the mess with display vs string etc (cf uniform message formatting) (Done: message text is markdown. Keywords are just formatted with string for now.)

Configuration

• Logger configuration system like python's logging.config
  • Rethink configure_logging() - it's not general enough.
  • Replacement for existing environment variables (things like JL_DEBUG_LOADING, DOCUMENTER_DEBUG etc)
  • Replacement for julia command line options like --depwarn={no|yes}?

Default InteractiveLogger backend

• Neat tabular formatting of variables in messages

Concurrency

• Log forwarding and aggregation from remote nodes
• Async logging ?

Performance

• Lookup speed for logger in task local storage suffers a 4x performance hit due to the TLS hash table (see early-filter-performance branch)

Case studies

• Try replacing Base.depwarn
• Try replacing debug logging in base/loading.jl
• Use for log messages arising from the julia parser and lowering code
• Integrate with Memento.jl as a backend
• Use as a transport mechanism for the Juno progress API?
• Integrate with ProgressMeter
• Integrate with Documenter

Base integration

• Use as a transport mechanism for @show
• Use as a backend for julia runtime log messages (ie, arising from the core C code)
• Have a logger that can be used in context where GC or STDERR is not available - see, eg, Base.showerror_nostdio

[c42f]

Eek. It turns out that depwarn is interesting - it's a special kind of nonlocal warning - a warning which should probably be attributed to a different stack frame (and hence file, line number, module, id?) than where the actual warning generating code exists.