Blockchain Engineering - class of 2024 - Team Democracy-1: blockchain networking (b)

[synctext]

Project assignment for Delft University of Technology master course called "blockchain engineering".
Democracy-1: Blockchain networking

• Replay historical voting rounds on the live network. Emulate 10..50 identities from a single smartphone.
• Understand and re-use existing ready-to-go datasets with DAO votes.
• Create transaction blocks within Trustchain in a custom IPv8 community using Kotlin superapp
• Share votes using a ledger-based gossip protocol
• Binary transfer of bulk votes using QUIC or uTP.
• The outcome of your entire project is a single number. Remember, running code is required for a passing grade 💥. Your focus is the amounts of blockchain-data bits you can transfer between two phones.
• BONUS, add carrier-grade NAT puncturing

Background reading:

• The SIM-card experiment with carrier-grade NAT puncturing. Real details: msc placeholder: 5G overlay infrastructure for decentralised learning ??? #7258
• Read about EVA protocol!
• TrustVault also uses binary transfer

Upcoming sprint assignment

• fetch major code update for superapp modules and IPv8 sub-module
• understand existing code
• NAT puncturing testing, using the app feature. Try to get a receiving UDP packet!
• get a cmdline Java transfer going between 2 peers, with uTP.

[synctext]

• working emulator
• got uTP running and data packets flowing
• self-initiative on Wireshark and seeing packet 🚀
• looked into NAT puncturing
• Note, 1 master students in now in Paris trying to get French 4G NAT puncturing going
• investigating Bulgarian 4G SIM card carrier-grade NATs
• 10 MByte data transfer test done with uTP.
• fault-tollerant: near zero mistakes, networking faults, or errors

Stack	performance
Local 1 laptop, 2 Java cmdline processes	100 MBbyte/sec
2 laptops, 2 Java cmdline processes Wifi	20 MBbyte/sec
2 laptops superapp emulators	5 MByte/sec
2 phones 5G Superapp	4 MBbyte/sec

Epic sprint goal:

• superapp .APK with network test ability. test transfer of 5, 50,200,500 MByte file (send/receive) {part of voting sub-app?}
• pull request on Superapp of this code enhancement (for other teams possibly)

[DeaLlazo]

• The first testing results for the first 2 lines of the table are below. The testing is done through the Wi-Fi at school with 1GB of data transfer:

1. Local 1 laptop, 2 Java cmdline processes | 65 MBbyte/sec

2. 2 laptops, 2 Java cmdline processes Wifi | 1 MBbyte/sec ()

[synctext]

• organised to work in 2 parts: puncturing and uTP transfer
• peer discovery then you can do uTP test transfer
• Lua script for uTP packet visibility in Wireshark 😮 😮 😮
• Expose complete puncture infrastructure?
  • You heard about an IPv8 peer (pub-key, last-seen IPv4, port)
  • Maybe never connected to this peer (see status for each peer)
  • Other peers in this magic community will always help you as coordinating 'servers'? Puncture-Request
    • peers that have send you a UDP packet within past 30 seconds
    • Known peers to be responsive
    • They can help you setup a connection
  • Exposure all these details for a single manual UDP puncture. Manual puncture button.
  • Success of puncture enable test transfer!
    • either re-use the IPv8 socket
    • or more simple is to create and bind() to you own manual puncture and measure UDP socket
    • debug screen can test transfer 2 hard-coded files: votes.csv (3.3. MBytes) and votes.csv (13.7 MBytes) in the DAO votes datasets
  • Discovery of any firewall in between two laptops or 2 phones
  • Any connected peer can be selected for uTP test transfer 🥇
  • debug info for carrier-grade NAT puncturing
    • goal: see if people are firewalled or behind NAT
    • peers see each other on different IPv4 and ports for symetric NATs
    • share the external IP of connected peers

ToDo: sprint goal is to have 2 laptops with Android emulator, 2 Android smartphones, and maximum SIM cards doing test punctures and test transfers. Upload the Lua scripts and public repo of changes to Github please. issue update for next meeting. (Bulgarian, Albanian, Dutch, etc. provider roaming tests)

[KolevVelyan]

Made debug window for uTP.

Made .lua script for Wireshark to be able to recognise ipv8 packets for easier debugging

Source code: https://github.com/Elkozel/IPv8-Wireshark-Support

Made .lua script for Wireshark to be able to recognise utp4j packets for easier debugging

Source code: https://github.com/KolevVelyan/utp4j-Wireshark-Support

[KolevVelyan]

Link to forked repository where our changes can be seen: https://github.com/KolevVelyan/trustchain-superapp

[grimadas]

This weeks progress:

• Fix puncturing on specific port on the person
• UTP implemented between two devices (screenshot in previous comment)
• Client/Server investable: the sequence number is fixed, switched the server/client but the group.
• Which port established the socket

For the next week:

• Solving the problem with the firewall (any tips to do that? )
• Working on DAO votes and showing in the app

[KolevVelyan]

This week's progress:

• finished puncturing a specific port (that the sender will need to open)
• updated uTP repo to not use random sequence numbers
• successfully established uTP transfer b/w 2 emulators (but with disabled windows firewalls for specific IPs)
• updated debug screen to use the transfer and show new outputs
  

[KolevVelyan]

progress this week:

• make it so you can select a file as input (any .csv file works as long as it is in the assets folder)
• add way to see how much and how fast data was transmitted during transfer
• make it so the receiver does not need to know the data size before transfer (as this is unrealistic). now the data size is exchanged by the sender when the puncturing happens
• so the flow is now (sender choses data that he wants to send and peer, receiver chooses other peer and punctures, then sender sends, finally receiver receives
• question: should we try to combine the sending and receiving (sounds good at first hand but then it might couple them too much so its harder to understand for others how it works)
• fix small problem with how data was passed in puncture packets
• data is now displayed on the screen as human readable characters (up to the first 1000 characters)
• imporving what we print during communication (tried to make it so the text updates live but couldn't)
• working on firewall bypass

New menu look:

apk link: https://drive.google.com/file/d/1UabiNfEJ873x0gp0LJlvYkhra-y2h29M/view?usp=sharing

[synctext]

• making solid progress on understanding our whole technology stack 🎉
• Create an example IPv8 community with utp4j and use endpoint-send-ipv8 to send a single UDP via endpoint override
  • advanced architecture, keep IPv8 very simple
  • do binary transfer in the IPv8 community space
  • re-usage of the existing IPv8 port
  • what layer to do serialisation? (IPv8 header 100-ish Bytes, uTP 20-ish Bytes)
  • Can peers be always in the ready-to-receive test file mode?
    • "incoming test file" from one side at GUI somehow. plus "transfer speed"
    • send test file button, show "transfer speed", packet loss, and real-time update
• make an .APK for testing?
• Read about IPv8 peer discovery, see basics
  • active demo community walking every 5 seconds
  • use 3 colors please plus incoming and outgoing connections.
  • orange, just heard recently about a peer, no connection.
  • red, heard about a peer over 60 seconds ago, never connected.
  • green, got an incoming packet in last 60 seconds
  • yellow, got an incoming packet in more than 60 seconds ago
  • grey, got an incoming packet in more than 120 seconds ago
  • remove dead peers after 300 seconds
• expand your puncture, send, and receive into full network debugger
  • see design and running code from 8 years ago 😲
  • app-to-app communicator on Android #2131 (comment)
  • 
• Robustness: packet loss can happen to the initial send_binary(). Make sure to re-try the initial request for a uTP transmission, if nobody is answering you.
• awesome bonus, birthday attack details
• robust networking API, superapp should react in real-time to switches between 5G and wifi
• Background reading on LEDBAT. This was new in 2010 and we implemented it in Python, wrote this article: Performance evaluation of a Python implementation of the new LEDBAT congestion control algorithm

[KolevVelyan]

What we did this week:

• send and receiver are not blocking the UI thread no more :)
• fixed dropdown styling so now you can always see it
• implemented basic version of full network debugger
  -- page displays details about the user's peer
  -- and a list [everyone is in "incoming" for now] for everyone in the community with status that changes (green->yellow->red) based on the last time we received a packet from them
  -- also, added indicators for when receiving from or sending to another peer (see screenshot below)
  -- peers get removed from list automatically when they disappear in the demo community
  -- based the design and implementation on the old implementation (https://github.com/Tribler/app-to-app-communicator)
• going back to uTP debug screen:
  -- removed "receive" button by attaching listeners that are triggered every time a send is made to a peer (still need to puncture initially but this will be removed later on as well)
  -- made sender run on another thread so that UI is non-blocking
  -- made it so you cannot send while receiving or sending (since socket is used)
  -- moved send and receive logic to a separate class so that it does not depend on the fragment it is used in
  -- moved methods used by sender/receiver in superclass [code cleanup]
  -- made peer list dropdown in a separate class so it can be reused easily [code cleanup]
  -- in general code in the uTP debug fragment is much less as it is split in the corresponding files making it easy to understand

We are currently working on removing the puncture by going through ipv8s socket. We have made the serializer and deserializer for uTP packets on top of ipv8. Currently, we are looking into setting up the socket correctly.

Full network debugger screen:

[grimadas]

• Great progress: many nice QoL improvements 🎉

For the next week:

• Clarify "S", "R" in UI, or think of better status indicator
• Ipv8 keep-alive status and UTP alive status
• Demo with apk and improved UI
• Big goal: bypass the firewall by integrating with ipv8 socket.
• Remove puncture button
• Start working on documentation: Readme, modularising, improved readability of the code.

[KolevVelyan]

Progress this week:

• integrated UTP with ipv8.
• No need to use firewalls anymore.
• Works with 4G.
• custom ipv8 socket on top of current socket.
• integrated network debugger with UTP
• now peer is in incoming if we have received UTP data from them (green if < 10s; yellow b/w 10 and 20s; red >20s)
• same for outgoing but for when we have sent them UTP data
• added info button about what the colors mean on the debugger
• some code clean up again
• NEW APK LINK: https://drive.google.com/file/d/1IxbA5m1eXBHUk-JXQfCKA1hofxzjtB1_/view?usp=sharing

Info screen:

[synctext]

• Sent all 1023 Kb of data in 0:3.810 (268.504 Kb/s) 🎉 🎉 🎉
• Binary transfer operational! (yet not fully stable)
• Note the difference between bits and bytes is 8x 😨
• Great GUI
• Lets also merge into a single list (forget the incoming/outgoing of who sent the first packet)
• The IPv8 packet types
• Indicate for the whole session which IPv8 packets types you observed P Puncture, D DATA etc. (no blinking, just permanent)
  • how to visualise an IPv4 you only heard about in an Introduction-Response
  • So it's a IPv8 network view
  • Popup for the uTP test started remotely or by user
  • Expose uTP status updates, now it blocks 🚫
• Could you inform the user of:
  1. network and the health of the overlay assuming you are with thousands of users
  2. enable sending test data to each user: new screen shows up, select amount of data, see real-time progress of transfer
• 5G KPN to 5G Vodafone test, no luck. Wifi to 5G Vodafone works. LAN 0.0.0.0

[KolevVelyan]

Progress this week:

• main screen:
  • removed the outgoing and incoming lists and merged them into one
  • reverted to the screen we had 2 weeks ago (with one list and peer details on top)
  • removed the blinking "S" and "R" indicators
• sending/receiving UTP:
  • made it so that when you click a peer the old screen pops up where you can send UTP with the same settings as before
  • when somebody sends you something popup automatically opens on top of any existing pop-ups (and closes any previously receiving popups that were not closed)
  • on both sending and receiving popups added a way to see the speed at which you receive/send with information such as total data received (which can be more than the actual file you are sending as packets get retransmitted if not acknowledged in time), speed of sending/receiving, the hight acknowledgement sent/received, the highest sequence number sent/received, total packets sent/received.
  • this updates during the transfer. you can also see how many packets (max SEQ number expected) are expected to be sent
  • caught an error where you could input numbers larger than an integer resulting in a crash
• changed custom IPV8 socket
  • fix some bugs including sending a big file, closing the connection sometimes resulted in a crash, and sometimes peer was not in list resulting in crash
  • made it so you can track each packet sent through it for the progress of a scan
• improved comment quality on all files

Still left to do:

• test thoroughly custom socket and UTP send/receive for crashes
• finalize code including comments and README
• add unit tests
• deal with premature finish of transfer
• some places we consider KB to be 1024 bytes but it should be 1000 (quick fix)
• test performance of new screens for many peers on screen
• test cancelling of scans of popups closing
• decide (status indicator) coloring of peers

Screenshots from this week:

1. Network Debugger screen
   

2. Sending UTP popup
   

3. Receiving UTP popup
   

[grimadas]

For the final week:

• Prepare the demo, make sure it goes smoothly.
• Hand in repository and application
• What are the limitations of your protocol? How big of the file it can handle? What are the assumptions on the network (latency/bandwidth)
• Going gray first when the peers are introduced and then it can
• What are letters S, A, TP stand for?
• Focus on tests, unit test would be nice, at least make a test cases (integration) for the corner cases that you mentioned

[KolevVelyan]

Links

• Our code is on a separate fork of trustchain-superapp with a release for the final version: https://github.com/KolevVelyan/trustchain-superapp/releases/tag/utp-debug-v1
• final apk: Can be found in the release of our trustchain-superapp repo
• README: https://github.com/KolevVelyan/trustchain-superapp/blob/master/debug/src/main/java/nl/tudelft/trustchain/debug/README.md
• our fork of utp4j with minor changes: https://github.com/KolevVelyan/utp4j

Final week progress

• added info button to explain S, A and TP flags
• popups now properly cancel transfer when closed
• added color grey for peers that are walkable but not in peer list , i.e., peers you know about but have not connected
• changed the MAX_CWND_INCREASE_PACKETS_PER_RTT to 30 000 as suggested 3 weeks ago
• added comments everywhere where missing
• made extensive unit tests to test the UTPCommunication (Sender and Receiver) and the custom IPv8 socket (can be found in common and debug)
• made README which can be found in debug
• did some tests for transfer speed from which we found that on Wi-Fi we get around 1MB/s whereas on 4G we get around 0.1-0.2MB/s (they can be found in README) and max data that can be sent using UTP
• fixed several bugs which crashed the phone
• changed how the popups can be scrolled to be more intuitive
• updated printed statements to more properly reflect what is happening during transfer
• fixed bug where we did not account for the 31-byte IPv8 header so not the whole data was being sent
• handle premature finish of transfer (if not all expected data is sent/received then transfer is considered unsuccessful)

Note

• UTP transfer still does not work great when on 5G because of the way puncturing works so we have not tested with doing transfers on 5G
• The maximum data that can currently be sent is 65535 packets using UTP as that is a limit of UTP (this is around 113MB). So if you try to send more than that only the first 113MB will be sent and transfer will count as successful. This is a limitation of UTP. Trying to send more than this can result in a crash on the receiver side.

[synctext]

• You got binary transfer to work at 1 MByte/sec 🐎 🚀 🎉
• Very responsive to our feedback and solid coding skills
• Amazing wireshark additions, starting working hard early on 💪
  • https://github.com/Elkozel/IPv8-Wireshark-Support
  • https://github.com/KolevVelyan/utp4j-Wireshark-Support
• Bit of lag peer list and scroll problem
• With all these unique 4G/5G SIM cards in the team, solid benchmarking could have been done (that's a lot for 5 ECTS course)
• documentation is solid, minor improvement point is deep code linkage and .GIF

[KolevVelyan]

Gif of working debugger (2x speed)