Add custom log rotator option and hourly log rotation

[windkit]

I am seeking to use lager as the logging back-end for LeoFS, so would like to open up the possibility of custom log rotator.

This should not change the current behaviour, only when user opt-in with custom log rotator

[jadeallenx]

Thanks for the PR. We will review this during our normal triage session, Oct 19.

[jadeallenx]

Thanks for your patience! This is great work. Thank you for the PR!

We like this patch and would like to merge it, but first we would like to see more documentation for the features being added here:

1. There should be a section in the README describing how to write a custom log rotation module that conforms to the behavior included in this PR.
2. The section that discusses file rotation ought to be extended to include the hourly rotation feature.

Thanks again. Lookjng forward to getting this into the project.

[windkit]

@mrallen1 Thanks for taking a look on this

[jadeallenx]

Sorry to do this, but would you mind rebasing this PR? If you can get to it, I'll merge it. Thank you.

[windkit]

Sure I will do this later this week. 2018年1月1日(月) 12:21 Mark Allen <[email protected]>:

Sorry to do this, but would you mind rebasing this PR? If you can get to it, I'll merge it. Thank you. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#420 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAWBtKgA09EHWuqNmLZCI28uLi6PsMo9ks5tGE8-gaJpZM4P2axS> .

-- Wishes, Wilson

[Vagabond]

If you're willing to refactor the rotator into its own module (get it out of util now it is pluggable), that'd be great too. Otherwise we can do it after this merges.

[windkit]

No problem I will do it tomorrow 2018年1月2日(火) 6:22 Andrew Thompson <[email protected]>:

If you're willing to refactor the rotator into its own module (get it out of util now it is pluggable), that'd be great too. Otherwise we can do it after this merges. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#420 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAWBtGbTIbt0pcpZ7qZywdXywnz7Js6Fks5tGUyDgaJpZM4P2axS> .

-- Wishes, Wilson

[windkit]

@mrallen1 @Vagabond I have rebased the PR and refactored the rotator into a separate module.
Could you help to review this again?

While the code passed the unit test from my environment with OTP 18.3, it fails in Travis with

  lager_test_backend: async_threshold_test_ (async threshold works)...*failed*
in function lager_test_backend:'-async_threshold_test_/0-fun-6-'/0 (test/lager_test_backend.erl, line 1768)
in call from lager_test_backend:'-async_threshold_test_/0-fun-10-'/0 (test/lager_test_backend.erl, line 1768)
**error:{assertMatch,[{module,lager_test_backend},
              {line,1768},
              {expression,"ets : lookup ( async_threshold_test , sync_toggled )"},
              {pattern,"[ { sync_toggled , N } ] when N > 0"},
              {value,[{sync_toggled,0}]}]}
  output:<<"">>
  lager_test_backend: high_watermark_test_ (Nothing dropped when error_logger high watermark is undefined)...[0.602 s] ok
  lager_test_backend: high_watermark_test_ (Mostly dropped according to error_logger high watermark)...[1.102 s] ok
  lager_test_backend: high_watermark_test_ (Non-notifications are not dropped)...*failed*
in function lager_test_backend:'-high_watermark_test_/0-fun-8-'/0 (test/lager_test_backend.erl, line 1867)
**error:{assert,[{module,lager_test_backend},
         {line,1867},
         {expression,"count ( ) < 10"},
         {expected,true},
         {value,false}]}
  output:<<"">>

Do you have any clues what do these errors mean?

[Vagabond]

It's a race condition, not sure why it hits 18.3 so consistently, it is not a concern.

[Vagabond]

This needs to change, right?

[windkit]

oops, sorry for that, let me fix it. Rebase later

[Vagabond]

Looking at this now, what would a custom rotator look like, given it seems to mainly just handle file operations, not the decision around when to rotate.

[jadeallenx]

Just want to say "thank you!" for rebasing and pulling all of the changes into their own module.

[windkit]

@Vagabond To us, we want to change how the log files are named.
https://github.com/leo-project/leo_logger/blob/develop/src/leo_logger_rotator.erl

[jadeallenx]

OK, this is a good start. Ultimately, I think we want all the rotation logic to be encapsulated inside a module/behaviour. So I am going to merge this and open an issue to migrate the rotation decision making inside the same module/behaviour.