Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sdf #2

Merged
merged 3,683 commits into from
Mar 15, 2020
Merged

sdf #2

merged 3,683 commits into from
Mar 15, 2020

Conversation

JohnMcLear
Copy link
Owner

sdf

muxator and others added 30 commits December 9, 2018 15:56
The .tex source was updated to fix some typos, but the corresponding pdf was not
regenerated.

Command used to generate the pdf:
  pdflatex easysync-full-description.tex

The incorporated changes are:
- 49114d2 (2014-06-01)
- c754845 (2017-09-14)
I've tried to install `node-inspector` using Node 8 and it looks like it is not supported. 

According to the [documentation of that tool](https://www.npmjs.com/package/node-inspector):

> Since version 6.3, Node.js provides a built-in DevTools-based debugger which mostly deprecates Node Inspector (...). The built-in debugger (...) provides certain advanced features (...) that are too difficult to implement in Node Inspector.

As [we require nodejs >= 6.9.0](https://github.com/ether/etherpad-lite#requirements), and as [`node-inspector` only works on Chrome and Opera](https://www.npmjs.com/package/node-inspector#debug), it looks like a good approach to remove the dependency of that tool and use Chrome DevTools directly. Besides, [there are other tools available](https://nodejs.org/en/docs/guides/debugging-getting-started/#inspector-clients) for debugging, if Chrome is not an option.

This PR also allow external connections to the inspector, so Etherpad instances running on containers can also be debugged. [There are obviously some risks to leave that opened on public IPs](https://nodejs.org/en/docs/guides/debugging-getting-started/#exposing-the-debug-port-publicly-is-unsafe), but I assumed no instance would run on debug mode for the final user.
Since this code can end up loaded in browsers when using client side plugins,
avoid use of ES6 syntax features such as arrow functions until MSIE support is
finally dropped.
…tall(), uninstall()

We cannot use arrow functions in this file, because code in /src/static can end
up being loaded in browsers, and we still support IE11.
This commit vastly shortens (and simplifies) the version table within
handler/APIHandler.js by building each version's entry incrementally based off
the previous version.

The resulting table has been validated by comparing the "before" and "after"
output of the following loop on both versions of the code (albeit with an
intermediate "sort" step to account for the different insertion order)

  for (let v in version) {
    let m = version[v];
    for (let [k, a] of Object.entries(m)) {
      console.log(v, k, a);
    }
  }

The patch also fixes a few typos, and removes a duplicate definition of
getChatHistory which in each applicable version was defined with two different
parameter lists, but where only the second would be used.
Also fixed a bug where the system would make a request to the central server for
the plugin list for every search even if the list was already cached.
Use real `async` instead of async.js where applicable.
The `getPluginTests()` function was never truly async anyway because it only
contains calls to synchronous `fs` modules.
`getPadAccess()` (src/node/padaccess.js) is now "promise only", resolving to
`true` or `false` as appropriate, and throwing an exception if there's an
error.

The two call sites (padreadonly.js and importexport.js) updated to match.
Promisified methods:
  - get()
  - set()
  - findKeys()
  - getSub()
  - setSub()
  - remove()
  - doShutdown()
Removed the 's' for consistency with the other `doesFooExist()` manager calls.
Retained an alias for plugins that might be using it.
Removed the 's' for consistency with the other `doesFooExist()` manager calls.
Retained an alias for plugins that might be using it.
The function is now iterative rather than recursive.
Converted those functions that API.js still depends on, and others that at this
point are never called via the nodeback mechanism.
muxator and others added 28 commits December 1, 2019 01:52
In shell scripts an unquoted $* is rarely useful, for example because it breaks
in presence of file names with spaces.

References:
- https://google.github.io/styleguide/shell.xml
  Use "$@" unless you have a specific reason to use $*.

- https://unix.stackexchange.com/questions/41571/what-is-the-difference-between-and#94200
  Short answer: use "$@" (note the double quotes). The other forms are very
  rarely useful.
Before this change, the docker user had home in a directory it had no
permissions on. The inability of creating a cache directory in `$HOME`
prevented npm to work properly.

Additionally, the `node_modules` in the base working directory had its
owner set to root, preventing further changes.

With this change, the `etherpad` user has a home directory.
Additionally, `npm i` is now run by `etherpad` rather than the root
user; this way, it is possible to dynamically change the `node_modules`
content in day 2 operations.

Note that while switching to the `useradd` builtin, a conflict was
discovered with the GID 65534 that was previously used. This change is
changing the `etherpad` user's UID to 5001 to avoid said conflict. As a
consequence, a `chmod -R 5001:5001` must be run prior to attaching
volumes created from previous Etherpad versions.
The "io" cookie is created by socket.io, and its purpose is to offer an handle
to perform load balancing with session stickiness when the library falls back to
long polling or below.

In Etherpad's case, if an operator needs to load balance, he can use the
"express_sid" cookie, and thus "io" is of no use.

Moreover, socket.io API does not offer a way of setting the "secure" flag on it,
and thus is a liability.

Let's simply nuke it.

References:
  https://socket.io/docs/using-multiple-nodes/#Sticky-load-balancing
  socketio/socket.io#2276 (comment) (not totally true, actually, see above)
…sid" and "language" cookie

The mechanism used for determining if the application is being served over SSL
is wrapped by the "express-session" library for "express_sid", and manual for
the "language" cookie, but it's very similar in both cases.

The "secure" flag is set if one of these is true:

1. we are directly serving Etherpad over SSL using the native nodejs
   functionality, via the "ssl" options in settings.json

2. Etherpad is being served in plaintext by nodejs, but we are using a reverse
   proxy for terminating the SSL for us;
   In this case, the user has to be instructed to properly set trustProxy: true
   in settings.json, and the information wheter the application is over SSL or
   not will be extracted from the X-Forwarded-Proto HTTP header.

Please note that this will not be compatible with applications being served over
http and https at the same time.

The change on webaccess.js amends 009b61b, which did not work when the SSL
termination was performed by a reverse proxy.

Reference for automatic "express_sid" configuration:
https://github.com/expressjs/session/blob/v1.17.0/README.md#cookiesecure

Closes #3561.
Colibris skin was first introduced in 1.7.5 and received some bugfixes in 1.8.0.
It is now time to make it the default for new installs.
This fixes some security vulnerabilites, among them an arbitrary file overwrite.


The output of `npm audit` goes from this:
  found 17 vulnerabilities (15 low, 2 high) in 13344 scanned packages
    run `npm audit fix` to fix 6 of them.
    1 vulnerability requires semver-major dependency updates.
    10 vulnerabilities require manual review. See the full report for details.

To this:
  found 5 vulnerabilities (3 low, 2 high) in 13370 scanned packages
    1 vulnerability requires semver-major dependency updates.
    4 vulnerabilities require manual review. See the full report for details.


Changelog:
- https://github.com/npm/cli/releases


6.13.4 (2019-12-11)
    BUGFIXES
    320ac9aee npm/bin-links#12 npm/gentle-fs#7 Do not remove global bin/man links inappropriately (@isaacs)

    DEPENDENCIES
    52fd21061 [email protected] (@isaacs)
    d06f5c0b0 [email protected] (@isaacs)

6.13.3 (2019-12-09)
    DEPENDENCIES
    19ce061a2 [email protected] Properly normalize, sanitize, and verify bin entries in package.json.
    59c836aae [email protected]
    fb4ecd7d2 [email protected]
        5f33040 #476 npm/pacote#22 npm/pacote#14 fix: Do not drop perms in git when not root (isaacs, @darcyclarke)
        6f229f7 sanitize and normalize package bin field (isaacs)
    1743cb339 [email protected]

6.13.2 (2019-12-03)
    BUG FIXES
    4429645b3 #546 fix docs target typo (@richardlau)
    867642942 #142 fix(packageRelativePath): fix 'where' for file deps (@larsgw)
    d480f2c17 #527 Revert "windows: Add preliminary WSL support for npm and npx" (@craigloewen-msft)
    e4b97962e #504 remove unnecessary package.json read when reading shrinkwrap (@Lighting-Jack)
    1c65d26ac #501 fix(fund): open url for string shorthand (@ruyadorno)
    ae7afe565 #263 Don't log error message if git tagging is disabled (@woppa684)
    4c1b16f6a #182 Warn the user that it is uninstalling npm-install (@Hoidberg)
…6.3 -> 10.18.0

This is the latest version as of today.
In the following commits Pierre is going to copy & modify some files.
This commit prepares the source files in order to minimize those differences,
so we can re-unify them as soon as possible.

No functional changes.
This is just needed to slim up the diffs for the next commits.
Non functional changes.
The dependency on java was introduced in 2012 (c021cf5) to start
Sauce-Connect from sauce labs.

Probably at the time it was a runtime dependency, but it is no longer the case
today. It is possible that java was already not needed when db003a1 changed
from downloading Sauce-Connect-latest.zip to sc-latest-linux.tar.gz.

Moreover, I am quite sure tests/frontend/travis/sauce_tunnel.sh no longer works
today, because tests/frontend/travis/sauce_tunnel.sh downloads from an url that
gives HTTP/404 now: sc-latest-linux.tar.gz if no longer a valid file name, we
would need to explicitly download a specific version.
… later

In the next commit Pierre will start adding tests for the docker build, and this
lays out the structure for doing that.
No functional changes.

The relevant TravisCI docs that motivates moving under a jobs section is
https://docs.travis-ci.com/user/build-matrix/

> There are two ways to specify multiple parallel jobs (what we call the build
> matrix) with a single .travis.yml configuration file:
>
> * combine a language-and-environment dependent set of configuration options to
>   automatically create a matrix of all possible combinations. This is called
>   matrix expansion. For example, the following configuration produces a build
>   matrix that expands to 8 individual (2 * 2 * 2) jobs
>   [...]
>
> * specify the exact combination of configurations you want in jobs.include.
>   For example, if not all of those combinations are interesting, you can
>   specify just the combinations you want
Add a `Test Dockerfile` job to Travis that checks the `docker build` exit
code.
More useful tests can be added later.
Note by muxator:
This commit introduced a copied & modified version of the testing files
loadSettings.js and pad.js.

It's Christmas night, and we want to shipt this feature, so I merged it anyway,
adding a note in both the original and copied files so that hopefully someone
in the distant future is going to merge them back again.
Supersedes #3631

Co-authored-by: RaymondCavallaro <[email protected]>
…o longer causes a crash

Steps to reproduce (via HTTP API):
1. create a group via createGroup()
2. create a group pad inside that group via createGroupPad()
3. make that pad public calling setPublicStatus(true)
4. access the pad via a clean web browser (with no sessions)
5. UnhandledPromiseRejectionWarning: apierror: sessionID does not exist

This was due to an overlook in 7699337: "apierror: sessionID does not
exist" may be a legal condition if we are also visiting a public pad. The
function that could throw that error was sessionManager.getSessionInfo(), and
thus it needed to be inside the try...catch block.

Please note that calling getText() on the pad always return the pad contents,
*even for non-public pads*, because the API bypasses the security checks and
directly talks to the DB layer.

Fixes #3600.
@JohnMcLear JohnMcLear merged commit 9ea5ad9 into JohnMcLear:develop Mar 15, 2020
JohnMcLear pushed a commit that referenced this pull request Mar 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.