airframe-log: possible merge into Scribe?

[darkfrog26]

I really like a lot of the features related to coloring in your logging library. Would you be interested in a possible merging of your project with Scribe (https://github.com/outr/scribe)?

[xerial]

@darkfrog26 Hi. Nice to e-meet you!

I'm aware of your project, scribe, especially because I also want to support Scala, Scala.js, and Scala.native (and upcoming new Scala bindings).

Could you explain more about how do you want to merge?

A current concern is:

• The use of java.util.logging in airframe-log is a requirement because using airframe-log + java.util.logging is convenient for us to merge logs from Facebook Presto, Twitter Finagle, etc. Can we redirect the log from libraries that use java.util.logging to scribe? I think it's technically not difficult, but I'm not sure whether it makes sense to have such a wrapper in scribe.

[darkfrog26]

Sorry for the very late response. I could have sworn I responded to this.

Scribe has no references to java.util.loggin in the core library, but the scribe-slf4j module can be used to log from anything that supports SLF4J (including JUL).

As for reasons of a merge, I think there's so much duplication of effort in the logging space that I'd really like to see turn into a collaboration to push the ball forward instead of everyone creating their own copy of the same thing. You have some color support that I think could benefit Scribe, although in 2.5 we recently added some basic ANSI support.

[xerial]

@darkfrog26 Thank you for the explanation.

Although creating a sort of logging standard (like Scribe) sound good, the real world is not so simple. I don't mean any offense, but I do have very pragmatic opinions on using the shared libraries.

For example,

• A) Scala projects need to address binary compatibility between versions, so in order to try one of the latest versions of Scala (e.g., Scala 2.13.0-M4, Scala.js 1.0.0-M3, etc.), I always need to ask you to publish the new logger version.
  • This is already happening even for more famous and widely used projects like ScalaTest, ScalaCheck, Scala Prallele collection, etc. Scala 2.13.0-M4 #175
• B) Producing a colored log format is really a tiny piece of the project. Contributing a part of log formatter to your project is totally OK, but the benefit for us is quite small because A) can be a potential blocker for future releases.
• C) Pure Scala approach is good, but not good for using JVM specific feature like JMX (e.g., you can configure log levels outside JVM by using JMX interface). If we implement JVM specific features to Scribe, I believe it will lose the value of Scribe.
• D) java.util.logging to SLF4j conversion has high overhead as described here https://www.slf4j.org/legacy.html#jul-to-slf4j

[darkfrog26]

A.) I agree that's a big frustration point for me as well. However, I'm typically very responsive.
B.) I'd like to know more about the features and functionality you as integral to airframe's logging lib?
C.) JVM-specific features don't detract from Scribe's benefits. Look at FileWriter. It obviously has nothing to do with Scala.js, but is a necessary platform-specific feature. The same can be accomplished for something like JMX. This would actually be a really nice feature for Scribe I believe.
D.) Ah, I wasn't aware. I'd be happy to create a scribe-jul implementation (can't imagine it would be more than a few lines of code) if that's a point of concern.

[xerial]

OK. Let's ignore A) part for a while because you and I will be active Scala developers, at least, for several years.
D) also sounds good.

For B) and C), airframe-log also include:

• Log file rotation and .gz compression. Currently it's based on logback-core.
• Asynchronous logger using a background thread, which is used when writing logs is a slow process.
• Runtime log-level configuration file scanner. This is super-convenient while debugging the code using ScalaTest + sbt.
• JMX-based configuration (as we already talked in the above)

These features are mentioned in this doc:
http://wvlet.org/airframe/docs/airframe-log.html

[xerial]

@darkfrog26
Supporting all of these features in scribe-side might be ok, but airframe-log is used across the entire airframe modules, so I hope you understand my reluctance in adding any external dependencies. I've already paid an effort to reduce external dependencies to a maintainable level (my blog post on this issue: https://medium.com/@taroleo/3-tips-for-maintaining-your-scala-projects-e54a2feea9c4)

Some facts:

• I've already finished implementing necessary features for human readable logging in airframe-log side, and airframe-log is already a stable project (I haven't touched the basic functionality of airframe-log for a year).
• Supporting Scala Native is a missing part in airframe-log, and that is a reason I considered using scribe in the past, but Scala Native is far from supporting Scala 2.12 and 2.13 for now. So anyway Scala Native will not be a big factor for several years.

What I need right now:

• Pure-scala implementation of java.util.logging that works all Scala environments including Scala.js, Scala Native.
• log rotation, coloring, asynchronous logging, JMX support can be implemented on top of java.util.logging as I do in airframe-log, so if scribe's core module can be used as a portable java.util.logging, I will use it.

[darkfrog26]

1. Apart from ScalaTest, Scribe core has only one third-party dependency, and it actually used to be integrated into Scribe and migrated into its own project (https://github.com/outr/scribe/blob/master/build.sbt#L29). I'm a firm believer in minimizing dependencies, especially on core libraries like logging.
2. I understand if it's not practical to migrate to Scribe in the near future. I'm just hoping at such point as you need new functionality in your logging library, you'll be willing to consider collaborating with Scribe to help make it happen instead of continuing your own.
3. I agree about Scala Native. It's a "great to have" bullet point, but practically speaking, very few people or projects really use it.
4. I'd be happy to collaborate on a pure Scala JUL implementation. I think it should be very trivial to support. If it's a few minor features like this keeping you from Scribe, I could have those added this week.
5. Log rotation is fully and powerfully supported in Scribe 2.5
6. Coloring is basically supported, though it could probably use some additional TLC and perhaps even migration into Perfolation as I'm not sure it technically belongs in a logging library directly.
7. Asynchronous logging in Scribe is better than ANY other logger (http://jmh.morethan.io/?gist=https://gist.githubusercontent.com/darkfrog26/2adabe432f47aa97c7188bc62422fc45/raw/e1d14f1e65fefe22ea646408569a5462d7f924bd/benchmark.json). Scribe's asynchronous logger is more than ten times faster than Scribe synchronous, which is three times faster than the next fastest logging library.
8. I honestly have never even looked into JMX in this regard (I've only ever used it for introspecting running JVMs) so I'd need to do a little research to better understand, but it sounds like a very beneficial feature for Scribe.
9. Scribe is so flexible about how it can be used, I would bet we could swap it into all of your projects with just a stub class in airframe-logging.

[xerial]

FYI, TwitterServer is providing a UI for log level configuration through JMX https://twitter.github.io/twitter-server/Admin.html#admin-logging

[darkfrog26]

Very cool. The same could be done with Scribe since it's runtime configurable, but providing a hook via JMX would be very useful for integration with such services.

[xerial]

@darkfrog26 Let me close this ticket because airframe is designed to be self-dependent as much as possible, and creating an external dependency is not an ideal for us.