Skip to content
This repository has been archived by the owner on Feb 8, 2023. It is now read-only.

Create 2018-10-29--go-core-dev-team-weekly.md #734

Merged
merged 2 commits into from
Nov 12, 2018
Merged

Create 2018-10-29--go-core-dev-team-weekly.md #734

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
267a90f
Create 2018-10-29--go-core-dev-team-weekly.md
Oct 29, 2018
798007c
link to bug fix in 2018-10-29 go core dev meeting notes
Stebalien Nov 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
189 changes: 189 additions & 0 deletions meeting-notes/2018/2018-10-29--go-core-dev-team-weekly.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
# 💫Golang Core Dev Team Weekly Sync 🙌🏽 October 29, 2018

- **Meeting Lead:** @eingenito
- **Notetaker:**
- **Attendees:**
- bigs
- erikingenito
- hannah howard
- warpfork
- djdv
- Adin Schmahmann
- Molly Mackinlay
- David Dias
- Kevin Atkinson
- Hector Sanjuan
- Travis Person
- **Recording:** _add link to recording once it's online_

## Agenda

- Ask everyone to put their name into the list of attendees
- Round of updates
- What have you accomplished since the last Weekly?
- Were there any blockers? If so, which ones? Is it still blocked? Why?
- What is the next important thing you should focus on?
- Ask for general questions. Could be things like:
- I'm stuck with something, I don't know who to ask. Who knows who to ask?
- Who can help me with xyz?
- Plan this week
- Select issues to work on
- Review remaining issues if there is time left


## Notes

### Week Update

@name
- Done:
- Blocked:
- Next:


@marten-seemann

- Done: completed the switch to tls-tris for QUIC (still a bunch of issues to fix though)
- Blocked: nothing
- Next: preparing for the IETF meeting next week
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anything that the group can do in order to prepare for the meeting? Does the IETF QUIC group know about libp2p? Are they excited about it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm preparing for the meeting by following the discussions of multiple WGs and reading drafts that might be interesting for us at some point in the future.
Of course, I'm bringing up p2p in general, and libp2p and IPFS from time to time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be valuable to point out that there is a draft spec for HTTP2-P2P ipfs/specs#21


@schomatis (not attending)
- Done:
- Solved some issues and reviewed PRs on UnixFS and related components.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which ones exactly? Any new development?

- Reviewed more UnixFS/MFS code.
- Next: Coordinate with Rob to start producing some IPFS Files documentation.
Copy link
Member

@daviddias daviddias Nov 2, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@schomatis, Rob is no longer available. Check in with the merged Community/Docs group (context here ipfs/ipfs#349) for IPFS Docs.

Ping @mikeal, @pkafei and @terichadbourne


@warpfork
- Done:
- a few hours on refmt obj traversal improvements.
- researching prior art for IPLD schemas (protobuf, graphql, fuchsia FIDL) and gathering notes
- did you know? basically nobody allows non-string map keys
- did you know? graphql actually doesn't support maps at all (just structs)
- some experimental code for internal reprs of a type system. just exploring.
- reading up on alternative chunkers. need to digest more. (ex; adler32 vs buzhash; casync switched from former to latter: why?)
- Blocked:
- more concrete IPLD data model specs is still on the wishlist.
- Next:
- need to do some PR reviewing (ex: ipld dagwalker)
- figure out where to land some of these notes and design sketches for IPLD

@travisperson
- Done:
- first pass at updating iptb
Copy link
Member

@daviddias daviddias Nov 2, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@travisperson Updating what part of iptb? Could you record a demo showing how to create a simple testbed for go-ipfs, go-libp2p, js-ipfs and js-libp2p?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Travis updated the version of iptb that was being used in the go-ipfs project.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

understood. Thanks for the clarification :)

- Blocked: N / A
- Next:
- Mauppdate iptb with a few fixes
- update iptb to match ipfs gx deps
- release iptb 2.0

@aschmahmann
- Done:
- Basic ID-based message passing/protocol structure
- Research on GraphSync and planning next steps for graph synchronization related to multi-writer files
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aschmahmann Are you aware of the new GraphSync specs? Read through https://github.com/ipld/specs/pulls

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read about GraphSync a week or so ago, but I'll check out the latest.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@diasdavid It's hard for me to tell how similar my "graph synchronization" issue is to GraphSync. My understanding of GraphSync is that it's designed to answer questions (or give the data corresponding to) the nodes descending from a Merkle root. The question I'm trying to answer is that if user A has a graph of changes (i.e. file versions where branching is allowed) and user B as another graph of changes, how can they best communicate so that they both get the graph Union(A,B).

These problems are certainly close together, but I'm not sure the same implementation will address both issues. Any thoughts you or anyone else has are welcome though. Who working on GraphSync should I be speaking with?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that what you are looking for is something very close to the coSelectors thinking -- ipfs/notes#154 (if Selectors are recursive, merkle root to leafs, coSelectors are merkleRoot coRecursive, pointing at future notes that point to our node).

Today, the way you do that is by having an append-only log that projects any other data structure that is the list of changes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This definitely shares similarities with the coSelectors you linked to. Just like you would have to do with synchronizing two append-only logs v1 of graph synchronization sends the entire graph between the two parties. The hope is that a later version of this type of graph synchronization requires less bandwidth per sync.

Should I try and leave this area mostly alone and focus on the interfaces and tooling that utilizes the simpler v1 implementation?

- Blocked:
- Next:
- Implement basic graph synchronization
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aschmahmann in what level? Over IPFS or by implementing a GraphSync prototype?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A graph synchronization prototype over libp2p where the graph nodes correspond to version changes (where the versions are stored in IPFS).

- Work towards testing framework
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aschmahmann do you plan on using IPTB?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would love to, need to learn more about it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aschmahmann the sharness tests in go-ipfs are good examples of how to use iptb in a straight forward way - at least I found them to be so. https://github.com/ipfs/go-ipfs/blob/master/test/sharness/t0130-multinode.sh is one that is pretty clear.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx for the link, it seems like IPTB should be helpful.


@djdv
- Done:
- email/response catch up
- various FUSE improvements
- reviews
- reading up on components /subsystems
- Blocked:
- None this week, some next week
- Next:
- Handle some outlying PRs (my own)
- more reviews
- focus on bitswap effort

@kevina
- Done:
- First attempt at converting Provider/FindProvider to use multihashes (https://github.com/libp2p/go-libp2p-routing/issues/32)
- Blocked:
- `--cid-base` option p.r. needs review (https://github.com/ipfs/go-ipfs/pull/5464)
right now I just need feedback on the general approach
- Need feedback and guidance on converting Provider/FindProvider to use multihashes
- This Week:
- Continue work on Provider/FindProvider
- Try to get `--cid-base` option p.r. in
- Help out with code review

@stebalien
- Done:
- Fixed race in query events.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Stebalien url to PR?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libp2p/go-libp2p-routing#31 and in go-ipfs: ipfs/kubo@e41ac96

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wanna add it to the notes too?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels a bit overkill but I've added it anyways.

- Cut 0.4.18-rc1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Congrats on then releasing 0.4.18 :D 👏🏽

- Added a delay to calling FindProviders in bitswap.
- Lots of testing of ipfs/go-bitswap#8
- Fixed another yamux race (libp2p, technically)
- Got everyone to agree on making ds.Delete idempotent for now :)
- Fixed a ancient race in the mock network.
- Buffered bitswap writes (don't send two packets per wantlist!).
- Got pubsub signing support into go-ipfs
- Fixed encoding of errors WRT MakeTypedEncoder.
- Fixed pin-lock bug in `ipfs dag put`
- Asks:
- Need review of ipfs/go-ipfs-config#16
- Please review commands-lib update PRs.
- This week:
- Cut a release.
- Spend a bit more time on libp2p things (reviewing auto-relay stuff, dialer
stuff, etc.). Preparing for a go-ipfs release took over my week last week.

@eingenito
- Done:
- working on bitswap benchmarking sharness tests
- Understanding bitswap code/protocol
- Blocked:
- Next:
- Get a rudimentary benchmark checked in and start using it to test bitswap changes.
- Iterop testing of the new RC

@hannahhoward
- Done:

- RevReviewed and discussed Bitswap PR, decisions is no merge, but extract parts
- Discovered issue with bitswap sessions not running due to code changes
- Worked on Bitsw testing with @eingenito
- Blocked:
- n/a
- Next:
- Fix Bitswap sessions using extracted file from PR
- Extract testing from PR

@hsanjuan
- Done:
- Un-force Batching in impoters. Enable explicitally on go-ipfs: https://github.com/ipfs/go-ipfs/pull/5626
- Needs merging in go-ipfs
- Everything else (dep-tree) is merged

@bigs
- Done:
- Daemon DHT protocol spec
- Testbed coordination with Jacob
- Workplan for new libp2p contractor
- Blocked: n/a
- Next:
- Second round review on dialer
- Kickoff call for testbed work with Jacob
- Finish DHT protocol spec
- Code review autorelay/autonat once again
- Services design disiscussion

@momack2
- Done:
- IPFS Project roadmap moved one step forward! All go-ipfs core should an email in their inbox to schedule a time to dive in deeper!
- Blocked: n/a
- Next:
- Itterate on Roadmap resources (for WGs and the wider org)
- KPIs and other project WG planning


### Questions
- @aschmahmann: Are there examples or "standard" ways that we implement message passing between peers? I recently built a basic message passing structure for synchronizing graphs using protobuf, but it feels like these structures must already exist elsewhere.
- Answer: no. We usually open a stream, write a message, read a response (and use protobufs). However, I would *like* a more standardized request framework...

### Other notes

- @aschmahmann's question precipitated a conversation about how to best tter coordinate between the Go and JS initiatives. @stebalien said that there has historically been poor communication between the JS and Golang efforts. @daviddias suggested perhaps joining the Go and JS weekly calls every other week (on week together next week separate) so that there's a time when everyone can discuss work underway and what cross project decisions need to be made. @eingenito suggested perhaps a brief weekly meeting where specific interop or group decisions are considered. Meeting wrapped up without conclusion but we'll create an issue to figure out exactly how to proceed.