[PROPOSAL] Introduce Bloom Filter Index

[sezruby]

What is the context for this pull request?

• Tracking Issue: [PROPOSAL]: Introduce BloomFilter index #161
• Parent Issue: N/A
• Dependencies: N/A
  • Issue 1
  • Issue 2

What changes were proposed in this pull request?

A draft design document for #161

Does this PR introduce any user-facing change?

No

How was this patch tested?

Intellij markdown util

[sezruby]

@imback82 Can we merge this PR? So that @thugsatbay could update the document.

[imback82]

@rapoth What's the plan for proposals moving forward? Are we checking in as a doc or have it as an issue similar to #263?

[imback82]

Another way is to create a "Proposal" category under Discussions.

[imback82]

In summary, we can pick one of the three to write a proposal:

1. Check in as a doc: current PR
2. Create as an issue and label is as "proposal": [PROPOSAL]: Support variable schema for included columns #263
3. Create a proposal under Discussions.

@rapoth @sezruby @pirz @apoorvedave1 @thugsatbay WDYT?

[imback82]

My vote is 2) since 1) doesn't seem to be going well: check https://github.com/microsoft/hyperspace/tree/master/docs/design. Since design docs can/need to be iterated quite a few times, requiring a check in to the repo doesn't seem like a good idea for productivity.

[rapoth]

I agree. It looks like we're having more luck with writing proposals as Issues. The second form was only meant for very large items but we might as well evolve the template inside Issues itself. The only rationale behind a check-in was to encourage PR reviews (a.k.a. Design reviews) but let's do whatever scales for the community.

I am neutral as far as using Disussions is concerned. There is a category Idea that @thugsatbay pointed me to yesterday that we could use for initial discussions (before proposals are written). It looks like we can create a new category as well that we could leverage for Proposals. However, do note that since discussions are not conventional issues, they do not show up in our planning system so it might be hard to incorporate planning. Therefore, unless someone strongly objects, we could have initial ideas as discussions but Proposals should remain as issues.

[sezruby]

Yep 2) would be good as it's easy to edit & track.

[pirz]

Using (2) is Ok as long as people do check them there. The main issue is that adding an item under issues which is really a design doc and needs interactive comment/reply does not fit with the "issues" category. What if we have separate category just for design proposals? Is that doable in Github?

[thugsatbay]

Even if we end up discussing on Discussion platform on Github we would end up creating an # Issue for tracking for planning which will just make the flow ugly. So I guess I feel option 2 is good enough. By adding correct/differentiating labels while creating # Issues for proposal and design docs we can do a quick filter in our issues that allows us to categorize our content and should work for what @pirz asked in his last question ?!.