Collaborative Editing using WebRTC ( Only 2 peers support ) #1877

abhishekgahlot · 2017-07-13T05:37:59Z

Work continues in #3741!!!

Please note this is incomplete and is currently being updated.

Detailed Description:

We create a grtc constructor inside middleware.
Middleware is used to listen to all the actions being dispatched so that we can send those to P2P data channel. Few states are filtered in middleware that we don't send.
For peer colors, we use a different action which is always dispatched when you select a block shown on the basis of peerID which is unique to each peer.
Locking also happens on the basis of borders. If border for collaboration is visible on one peer side it is locked.

The module used is: https://github.com/abhishekgahlot/gutenberg-rtc
How GRTC works: https://github.com/abhishekgahlot/gutenberg-rtc/blob/master/DESIGN.md

TODO tasks

Blockers

Editable: Treat value as HTML string, eliminate children source

Improvements

Editable: Invoke onChange on every input

jasmussen · 2017-07-24T08:14:51Z

Nice! Let's tweak the boundaries of a locked block so they look and behave like other blocks. Like this:

Let's make sure all floats and alignments work, with the tweaked markup. @youknowriad not urgent but if at some point you have time can you look at this PR? Thanks!

youknowriad · 2017-07-25T09:45:36Z

blocks/editable/index.js

@@ -387,6 +387,7 @@ export default class Editable extends Component {
 	}

 	getContent() {
+		return this.editor.getContent();


Is this intentional ?

@youknowriad No it's being worked at, I added this because redux states had react elements which I was not able to serialize. This works fine and output string but we are working on this for proper structure.

youknowriad · 2017-07-25T09:47:15Z

editor/modes/visual-editor/block.js

@@ -345,66 +348,74 @@ class VisualEditorBlock extends Component {
 		/* eslint-disable jsx-a11y/no-static-element-interactions, jsx-a11y/onclick-has-role, jsx-a11y/click-events-have-key-events */
 		return (
 			<div


Is it possible to avoid creating a new div here, and reuse the inner div?

youknowriad · 2017-07-25T09:49:49Z

editor/selectors.js

+export function getPeerData( state, uid ) {
+	const { peerID, peerName, peerColor, blocksByUid } = state.collaborationMode;
+
+	if ( peerColor && peerID && peerName && peerID !== window.grtcProps.peerID && blocksByUid === uid ) {


It's a bit weird to use global variables in a selector? What's the purpose of the window.grtcProps ? Can't we move this to the store maybe?

youknowriad · 2017-07-25T09:53:11Z

editor/state.js

@@ -3,6 +3,7 @@
 */
 import optimist from 'redux-optimist';
 import { combineReducers, applyMiddleware, createStore } from 'redux';
+import ReduxThunk from 'redux-thunk';


Why do we need redux thunk? We're using an alternative in Gutenberg https://github.com/aduth/refx

Also, where are we using thunks in this code?

youknowriad · 2017-07-25T09:57:26Z

This PR is not easy to review since I'm lacking a lot of information and knowledge using P2P collaboration.

Some questions:

Is the gutenberg-rtc lib a generic rtc communication lib or does it have some specificites related to Gutenberg?
Should we move it to the Gutenberg repository too?

abhishekgahlot · 2017-07-25T10:01:20Z

@youknowriad Thanks for the comments, I will check and fix them. Gutenberg RTC is not a generic library but wrapper written over simple-peer library. I didn't add that to Gutenberg because it does one thing which is maintain P2P data channel (encrypted) with the help of serverside key value store.

youknowriad · 2017-07-25T10:09:42Z

@abhishekgahlot What about moving the library to a root folder in the gutenberg repository? we have several separate modules already, element, utils, components, date...

abhishekgahlot · 2017-07-25T10:11:00Z

@youknowriad Sure will do that too. Thanks for notifying.

aduth · 2017-07-25T20:24:35Z

editor/state.js

 	} ) );

-	const enhancers = [ applyMiddleware( refx( effects ) ) ];
+	const enhancers = [ applyMiddleware( refx( effects ) ), applyMiddleware( ReduxThunk, grtcMiddleware ) ];


This can be merged into a single applyMiddleware: applyMiddleware( refx( effects ), grtcMiddleware )

aduth · 2017-07-25T20:27:40Z

editor/state.js

@@ -3,6 +3,7 @@
 */
 import optimist from 'redux-optimist';
 import { combineReducers, applyMiddleware, createStore } from 'redux';
+import ReduxThunk from 'redux-thunk';


Also, where are we using thunks in this code?

abhishekgahlot · 2017-07-27T23:40:37Z

editor/selectors.js

+				'border-bottom': '2px solid ' + peerColor,
+			},
+		};
+		grtcProps.lastPeerData = {


Directly mutating the state, Fix this

aduth · 2017-08-03T13:15:02Z

editor/sidebar/collaboration/index.js

+/**
+ * WordPress dependencies
+ */
+import { __ } from 'i18n';


Note with #2172, these need to be updated to prefix the dependencies with @wordpress/. You will need to perform a rebase against the latest version of master and apply your changes:

git fetch origin git rebase origin/master

abhishekgahlot · 2017-09-26T10:23:18Z

@gziolo These are the few things that have been bottleneck till now.

Serialization of data so it can be transferred to other peers.
Fix the edge case for 2 peers if one takes a longer time to connect can result in both peers in the unstable state.
Ability to queue data so that when another peer/user comes he can see the changes already done by another peer/user.
Optimization for XHR Polling.
Better UI/UX for users when they are connected.
Locking of blocks is done by CSS so navigating by keyboard events is still possible.
When collaborator leaves, block remains locked.

gziolo · 2017-09-26T10:31:27Z

editor/reducer.js

+			grtc.on( 'peerData', onReceivedAction );
+			// Temporary change for testing, Not using alert. Need UI.
+			grtc.on( 'peerConnected', function() {
+				alert( 'Peer connected' );


We should display global notice instead.

Updated, but it still needs some work. Moved to todo list.

mtias · 2017-09-26T10:51:49Z

editor/modes/visual-editor/block.js

@@ -320,6 +327,11 @@ class VisualEditorBlock extends Component {
 		// Generate a class name for the block's editable form
 		let { className = getBlockDefaultClassname( block.name ) } = blockType;
 		className = classnames( className, block.attributes.className );
+		// Don't show controls when collaboration is enabled.
+		if ( peerShowStyle ) {


@gziolo @abhishekgahlot wonder if the changes to the visual-block component could be abstracted in a HoC withCollabEditing to try to clean this file cleaner.

That would be nice, I will explore this idea. Moved to todo list :)

mtias · 2017-09-26T10:52:24Z

editor/modes/visual-editor/style.scss

@@ -83,6 +82,76 @@
 		transition: 0.2s outline;
 	}

+	&.is-collaboration {


This class should be more semantic: is-being-edited perhaps.

mtias · 2017-09-26T10:52:50Z

editor/modes/visual-editor/style.scss

+			opacity: 1;
+		}
+
+		&.collab-red {


These classes should be is-red etc

mtias · 2017-09-26T10:53:12Z

editor/modes/visual-editor/style.scss

+			opacity: 0.8;
+		}
+
+		&.collab-null {


Why do we need to support a null class?

mtias · 2017-09-26T10:53:28Z

editor/reducer.js

@@ -16,6 +16,14 @@ import { getBlockTypes } from '@wordpress/blocks';
 import { combineUndoableReducers } from './utils/undoable-reducer';

 /**
+* Collaboration Dependencies


Missing spaces here it seems

mtias · 2017-09-26T10:53:42Z

editor/reducer.js

+*/
+import GRTC from '../grtc/app';
+
+const isMobile = window.innerWidth < 782;


This doesn't seem a good place for this.

Both constants were never used. I removed them.

mtias · 2017-09-26T10:54:25Z

editor/reducer.js

+	}
+
+	/**
+	 * @param {Object} action is redux action


Let's use // comments for these.

mtias · 2017-09-26T10:55:11Z

editor/sidebar/collaboration/index.js

+ */
+import { collaborationMode } from '../../actions';
+
+function CollaborationPanel( { active = false, ...props } ) {


@jasmussen had a nicer idea for placing this in an ellipsis menu. (It's ok for an initial pass, though.)

Added to the todo list.

mtias · 2017-09-26T10:55:36Z

editor/sidebar/collaboration/index.js

+import { collaborationMode } from '../../actions';
+
+function CollaborationPanel( { active = false, ...props } ) {
+	const changeMode = () => props.collaborationMode( ! active );


Maybe "enableCollaboration" or "toggleCollaborationMode"?

I would like to unify language we use for this feature. If we would go for collaborative editing then I would go for toggleCollaborativeEditing.

mtias · 2017-09-26T10:56:37Z

grtc/DESIGN.md

@@ -0,0 +1,193 @@
+# Design


Can we call this "architecture" to avoid clashes with design docs on searches, etc?

Updated. Should we keep grtc in its own package?

It's much smaller after encryption got removed. If we would keep it, where should we put it in the repository? Can it remain in the top level or rather should be moved to editor top level folder?

mtias · 2017-09-26T10:57:14Z

grtc/DESIGN.md

@@ -0,0 +1,193 @@
+# Design


Also @gziolo maybe some of these notes can be added as comments in the code, which is always better.

I updated the docs, they may require further iterations. Let's wait until we stabilize API.

mtias · 2017-09-26T10:58:04Z

lib/collaboration.php

+ *
+ * @param object $data as request data along with args.
+ * @return bool else username if logged in.
+ */


These functions need the version since info.

Moved to todo list.

mtias · 2017-09-26T10:59:20Z

lib/collaboration.php

+ * @param string base64
+ * @return bool if its valid
+ */
+function base64_check( $string ) {


This is a rather generic function that might need consideration as part of WP utility functions. cc @pento @mdawaffe

Why are the extra sanity checks needed over the strict flag in base64_decode()?

aduth · 2017-09-26T20:51:18Z

grtc/crypto.js

@@ -0,0 +1,32 @@
+const forge = require( 'node-forge' );


Crypto packages tend to be extremely large for browser consumption. This one is approximately 266kb minified and gzipped. I'd sought similar in Automattic/wp-calypso#17356, though I'm unsure of a good alternative for RSA key generation as you need here. Do you know of other options we could evaluate? I think the size as-is is quite excessive.

We can remove the support of encryption and trust the turn servers in the relay. It can be removed completely.

Removed completely with 4ebad12.

aduth · 2017-09-26T20:51:42Z

grtc/app.js

@@ -0,0 +1,471 @@
+'use strict';


We shouldn't need 'use strict'; with Babel (it is added automatically)

Would it be possible to split this file up, perhaps into one-file-per-class?

use strict removed and classes moved to their own files.

aduth · 2017-09-26T20:52:40Z

grtc/app.js

+		const self = this;
+		return new Promise( ( resolve, reject ) => {
+			const data = { peerID: self.peerID, peerName: self.peerName, type: 'initial', signal: self.signalID };
+			jQuery.post( self.url + '/remove', { [ self.grtcID ]: window.btoa( JSON.stringify( data ) ) }, ( resp ) => {


jQuery is not defined as a dependency for this script (i.e. incidental that another plugin depends on jQuery, but cannot rely on this). In any case, we might want to consider if we can do without it (maybe fetch, which is already promise based?)

My initial iteration uses window.fetch. We can update later id we decide on some different approach.

aduth · 2017-09-26T20:53:13Z

grtc/app.js

+	clearSignal() {
+		const self = this;
+		return new Promise( ( resolve, reject ) => {
+			const data = { peerID: self.peerID, peerName: self.peerName, type: 'initial', signal: self.signalID };


This and the following line could be made more readable by splitting into separate lines.

Makes sense. Done 👍

aduth · 2017-09-26T20:54:00Z

grtc/app.js

+	 * @return {promise} promise object
+	 */
+	clearSignal() {
+		const self = this;


Arrow functions preserve this reference, you don't need to create a separate variable for tracking this.

Yes, removed all self occurrences.

aduth · 2017-09-26T20:54:27Z

grtc/app.js

+	 * signalID is peer signal used to traverse and connect P2P.
+	 */
+	constructor( url, grtcID, peerID, peerName, signalID ) {
+		const self = this;


With arrow functions, I think this pattern of assigning a self variable should be largely avoidable and arguably discouraged.

gziolo · 2017-10-06T09:12:10Z

@aduth @mtias I addressed almost all your comments. Those remaining are now on the todo list. I did quite a heavy refactoring, but there are a few things that can be further improved. In particular I would like to change the way we shaped Redux state. If I understood properly it can't properly handle more than 2 peers at the moment. Middleware is still a bit hard to read, so I plan to tackle this one, too.

@abhishekgahlot what did you mean by "Optimization for XHR Polling". Can you elaborate on this one?

abhishekgahlot · 2017-10-06T20:18:31Z

@gziolo Right now the number of requests that goes to the server in order to check for new peers is done every 5 seconds this can become huge very soon. Maybe something can be done to optimize stop polling when both peers connect?

codecov · 2017-10-18T08:30:46Z

Codecov Report

Merging #1877 into master will increase coverage by 1.35%.
The diff coverage is 40.65%.

@@            Coverage Diff            @@
##           master   #1877      +/-   ##
=========================================
+ Coverage   31.64%     33%   +1.35%     
=========================================
  Files         235     208      -27     
  Lines        6658    6039     -619     
  Branches     1196    1074     -122     
=========================================
- Hits         2107    1993     -114     
+ Misses       3824    3409     -415     
+ Partials      727     637      -90

Impacted Files	Coverage Δ
editor/selectors.js	`95.4% <ø> (+1.01%)`	⬆️
editor/reducer.js	`89.03% <ø> (-1.03%)`	⬇️
editor/modes/visual-editor/block.js	`0% <ø> (ø)`	⬆️
editor/sidebar/coediting-panel/index.js	`0% <0%> (ø)`
editor/actions.js	`33.33% <0%> (-5.8%)`	⬇️
blocks/editable/index.js	`10.5% <0%> (-0.53%)`	⬇️
editor/sidebar/post-settings/index.js	`0% <0%> (ø)`	⬆️
editor/store.js	`80% <100%> (-3.34%)`	⬇️
utils/lang.js	`100% <100%> (ø)`
editor/middlewares.js	`14.89% <14.89%> (ø)`
... and 138 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e5e6027...84a1680. Read the comment docs.

… Gutenberg

gziolo · 2017-11-30T14:37:08Z

Closing in favor of #3741.

Merge v1.22.0 to develop

abhishekgahlot changed the title ~~Collaborative Editing using WebRTC ( With 2 peers )~~ Collaborative Editing using WebRTC ( Only 2 peers support ) Jul 13, 2017

ehg mentioned this pull request Jul 17, 2017

Collaborative editing using WebRTC #1930

Closed

youknowriad reviewed Jul 25, 2017

View reviewed changes

aduth reviewed Jul 25, 2017

View reviewed changes

abhishekgahlot commented Jul 27, 2017

View reviewed changes

aduth reviewed Aug 3, 2017

View reviewed changes

jasmussen mentioned this pull request Aug 15, 2017

Turning a static block into a dynamic reusable block #1516

Closed

aduth mentioned this pull request Aug 18, 2017

Try: Framework-agnostic block interoperability (Vanilla, Vue) #2463

Closed

gziolo self-assigned this Sep 26, 2017

gziolo reviewed Sep 26, 2017

View reviewed changes

mtias reviewed Sep 26, 2017

View reviewed changes

aduth reviewed Sep 26, 2017

View reviewed changes

gziolo added [Feature] Blocks Overall functionality of blocks [Status] In Progress Tracking issues with work in progress [Type] Task Issues or PRs that have been broken down into an individual action to take labels Oct 2, 2017

abhishekgahlot self-assigned this Nov 2, 2017

abhishekgahlot added this to the Future milestone Nov 2, 2017

abhishekgahlot and others added 3 commits November 30, 2017 15:05

Peer to peer collaboration

fd1a75e

Coediting: Refactor Collaborative Editing to be more independent from…

1ed68d0

… Gutenberg

initial buffer for 2 peers

da83dfc

gziolo mentioned this pull request Nov 30, 2017

Collaborative Editing using WebRTC (only 2 peers supported) #3741

Closed

18 tasks

gziolo closed this Nov 30, 2017

gziolo mentioned this pull request Nov 7, 2019

[WIP] Draft: Sync Editor React state with Yjs #18357

Closed

4 tasks

Tug pushed a commit that referenced this pull request Feb 28, 2020

Merge pull request #1877 from wordpress-mobile/merge-v1.22.0-to-develop

03b2179

Merge v1.22.0 to develop

Collaborative Editing using WebRTC ( Only 2 peers support ) #1877

Collaborative Editing using WebRTC ( Only 2 peers support ) #1877

Conversation

abhishekgahlot commented Jul 13, 2017 • edited Loading

Work continues in #3741!!!

TODO tasks

Blockers

Improvements

jasmussen commented Jul 24, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

youknowriad commented Jul 25, 2017

abhishekgahlot commented Jul 25, 2017

youknowriad commented Jul 25, 2017

abhishekgahlot commented Jul 25, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abhishekgahlot commented Sep 26, 2017 • edited by gziolo Loading

Choose a reason for hiding this comment

gziolo Oct 4, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gziolo Oct 4, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gziolo Sep 29, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtias Sep 26, 2017 • edited Loading

Choose a reason for hiding this comment

gziolo Oct 2, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gziolo Sep 26, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gziolo Oct 4, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gziolo Oct 5, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gziolo Oct 5, 2017 • edited Loading

Choose a reason for hiding this comment

gziolo commented Oct 6, 2017 • edited Loading

abhishekgahlot commented Oct 6, 2017

abhishekgahlot commented Jul 13, 2017 •

edited

Loading

abhishekgahlot commented Sep 26, 2017 •

edited by gziolo

Loading

gziolo Oct 4, 2017 •

edited

Loading

gziolo Oct 4, 2017 •

edited

Loading

gziolo Sep 29, 2017 •

edited

Loading

mtias Sep 26, 2017 •

edited

Loading

gziolo Oct 2, 2017 •

edited

Loading

gziolo Sep 26, 2017 •

edited

Loading

gziolo Oct 4, 2017 •

edited

Loading

gziolo Oct 5, 2017 •

edited

Loading

gziolo Oct 5, 2017 •

edited

Loading

gziolo commented Oct 6, 2017 •

edited

Loading

codecov bot commented Oct 18, 2017 •

edited

Loading