Follow with filter

[faiq]

Okay so this is a combination of my last 2 pull requests: #149 and #151 and I think this provides really what an end user would want.

So let me describe the changes here:

1. take in a ctx.Context rather than a time.Time

This allows more flexibility in use cases as described here:

For example, I am working on a daemon that forwards journald logs to an external logging service over http. The only reason I would ever want it killed is if a signal interrupt were given, so to do this I send a time.Now() from my signal handler into this channel to stop the program gracefully.

If I use a context, its more clear that I want to cancel processing of the logs rather than sending a time. Alternatively if someone wants to finish processing at a certain time, they can still do so. All they would have to do is send a ctx.Done() whenever the time hts

2. Use a filter function to figure out what part of the journald log you want

It's very reasonable for someone to want the entire journal entry, rather than just the message field. In fact, we use the Send() method from the journal library https://github.com/coreos/go-systemd/blob/master/journal/journal.go#L75 to write custom log fields for our programs into journald. It's only suitable that we should be able to pull them out for later processing

3. Send the journal entries with the filtered fields to msgs chan<- map[string]interface{}

This allows us to read all fields from the jorunal without worrying about the type and send them asynchronously without having to marshal and unmarshal the entries

Willing to close #151 in favor of this.

Thanks for all the help

[yifan-gu]

Sorry for not being responsive, (was not cc'd, so I lost this PR in notification).
Will take a review.

[yifan-gu]

Maybe change the size to 0, and close the channel instead of sending a bool?

[yifan-gu]

Or better, can we reuse these channels ?

[yifan-gu]

Ref rkt/rkt#2041 as this is an API break change.

cc @derekparker

[lucab]

@faiq ping :)

This stale PR has been pending for some time, do you still want to proceed on this? If so, can you please address @yifan-gu comments?

[faiq]

yikes sorry about being so non-responsive and thanks for the ping. it turns out we did not end up needing this functionality. I am more than happy to finish work on this or close it, depending on if you think this patch would be helpful or not.

[lucab]

Why value type is interface{}? Aren't strings enough?

[lucab]

@faiq I think it would be nice to have some portions of this.

Point (1) looks good and point (3) makes the Follow() API more useful for the general case (even though I'm not sure why values are interfaces). We now have a GetEntry() which should help reducing the size of this PR.

Regarding point (2) I think the library shouldn't be concerned about this. And once the other two points are in place, it should be easy for consumers to just provide their own filtering wrapper.

[faiq]

Sorry I'm so slow at responding to this 😞 lots of work makes it hard to contribute to open source.

regarding why we're sending interface{} through the channel, this is because the values on the fields can be json by something like https://github.com/wercker/journalhook and logrus.

i can probably take out the enumerating logic for a call to GetEntry instead, and filter in go-land.

how does this all sound?

[lucab]

regarding why we're sending interface{} through the channel, this is because the values on the fields can be json by something like https://github.com/wercker/journalhook and logrus.

I'm not completely sure I'm understanding your proposal. Current API will return you either a string or a []byte, so even if the inner content is a JSON the receiver still need to unmarshal it. E.g. in the case of GetEntry below, this is already a map[string]string.

i can probably take out the enumerating logic for a call to GetEntry instead, and filter in go-land.

Just to be clear, who will be performing the filtering? Do you need go-systemd to filter fields, or will the receiver take care of that? I was suggesting the latter, but I may understand in some performance-related scenarios the former may be interesting too...