Simplify Fetcher interface

[AndrewSisley]

Currently we have:

FetchNext(ctx context.Context) (EncodedDocument, error)
FetchNextDecoded(ctx context.Context) (*client.Document, error)
FetchNextDoc(ctx context.Context, mapping *core.DocumentMapping) ([]byte, core.Doc, error)

• FetchNext is never used.
• FetchNextDecoded cannot return deleted items.
• Nowhere in the codebase mix and matches (which would be really odd to do anyway), i.e. calls FetchNextDecoded and FetchNextDoc - is always one or the other.
• FetchNextDecoded and FetchNextDoc are really just use-case transformations of fetcher output, not fetcher output itself.

This all makes it harder to maintain and use, particularly when layering stuff on top of it, like the Lens wrapper as it has to deal with 3 funcs that nearly do the same thing but not quite.

We can remove FetchNextDecoded and FetchNextDoc, they can sit as a layer on top (is just a stateless func call transforming the output of FetchNext). I am also in favour of changing it so that the Fetcher interface just extends the Enumerable interface. The transforms then just become enumerable.Selects, and stuff like Lens becomes super easy (todo in codebase, in the Lens fetcher).

Be careful with the time of this however. Many people are currently working in the same location.

[jsimnz]

Overall, im a big fan of this, less is more.

As for the enumerable interface implementation on the Fetcher side, what did you have in mind for the value of the type parameter, ie, what should the base Value method on the Fetcher return. Would EncodedDocument be too internal / diffucult to use in the context of Lens. My initial intuition would be that EncodedDocument should work well for the existin use cases that the fetcher is consumed by, but I dont yet have any intuition on the lens use cases regarding the fetcher.

As you mentioned, I think having the "decode" functions of FetchNextDecoded nd FetchNextDoc be moved off the interface into some utility functions in the fetcher package works great, as it allows whatever downstream consumer to use the representation they care about, without further complicating the fetcher.

[AndrewSisley]

Noted in standup, but adding here for reference. I had EncodedDocument in mind (possible because that is what is already there), but have not given it too much thought, and currently have no strong feelings.