From 877d39acdb03c9f952e8aafce15a34275bbcd5ca Mon Sep 17 00:00:00 2001 From: Florent Date: Fri, 8 Mar 2024 15:18:02 +0100 Subject: [PATCH] updates readme doc --- CLIENT.md | 6 +- INSPECTOR.md | 71 +++++ SERVER.md | 664 ++++++++++++++++++++++++++++++++++++++++++++ inspector/README.md | 69 ----- server/README.md | 663 ------------------------------------------- 5 files changed, 739 insertions(+), 734 deletions(-) create mode 100644 INSPECTOR.md create mode 100644 SERVER.md diff --git a/CLIENT.md b/CLIENT.md index fc0b9dd..aaedbc5 100644 --- a/CLIENT.md +++ b/CLIENT.md @@ -1,3 +1,5 @@ +# RxPaired-client + ## How to build RxPaired-client and what to do with it To build it, you first need to install its dependencies. @@ -9,7 +11,7 @@ Then in the root directory, open a terminal and type: npm install ``` -You have to create a `.npmrc` file in the root directory. +If not already done, you have to create a `.npmrc` file in the root directory. You can base yourself on the `.npmrc.sample` file: ```sh @@ -53,7 +55,7 @@ At last, you have two choices in how to deploy that script: - Either you store it on an HTTP(S) server, in which case you'll have to indicate its URI to the `RxPaired-inspector` (more information on that - in its directory: `./inspector`). + in `INSPECTOR.md`). This is the recommended choice. If you choose to go this way, the `RxPaired-inspector` will conveniently diff --git a/INSPECTOR.md b/INSPECTOR.md new file mode 100644 index 0000000..f834b8d --- /dev/null +++ b/INSPECTOR.md @@ -0,0 +1,71 @@ +# RxPaired-inspector + +## How to build and run it + +To build it, you first need to install its dependencies. + +To do that, make sure that you have `npm` installed and this repository cloned. +Then go to this directory on a terminal, and type: + +```sh +npm install +``` + +If not already done, you have to create a `.npmrc` file in the root directory. +You can base yourself on the `.npmrc.sample` file: + +```sh +# in root directory +cp .npmrc.sample .npmrc +``` + +In that new `.npmrc` file, you'll need to set two URLs: + +1. `inspector_debugger_url`: + + This will be the WebSocket address `RxPaired-server` is listening to for + `RxPaired-inspector` connections. + + If you didn't start the `RxPaired-server` yet, you should first start doing this by + using `npm run serve --workspace=server`. + + Note that the URL already present uses the default port used by the server. If your + server runs locally in the default config, you might not actually have to update it. + + In most cases however, you might want to use HTTPS/WSS instead of the default HTTP/WS + (to make the `RxPaired-Client` runnable in HTTPS pages). + There, you might need to update this value to the actual HTTPS URL used. + +2. `device_script_url`: + + This is the URL the `RxPaired-Client` (the script that will be deployed to devices) + is available at. + + You can leave this URL empty if you do not wish to serve that script though HTTP(S) + means. + In that case, you will have to copy/paste the content of that script into the HTML + page running on the device. + +Once this is done, you can start building the `RxPaired-inspector`. + +In your terminal, type: + +```sh +npm run build:min --workspace="inspector" +``` + +You can now start using the `RxPaired-inspector`, you just need to serve both the +`inspector/index.html` and the newly generated `inspector/inspector.js` files. + +You can do so easily by typing in a terminal: + +```sh +npm run serve --workspace="inspector" +``` + +Multiple persons can even build and serve their own `RxPaired-inspector` while +debugging the same devices as long as the same `RxPaired-server` is used. + +Though you might prefer the simplicity of just letting a "real" HTTP(S) server serve both +of those files instead. This way this build step does not need to be repeated for each +user. diff --git a/SERVER.md b/SERVER.md new file mode 100644 index 0000000..fe8ed96 --- /dev/null +++ b/SERVER.md @@ -0,0 +1,664 @@ +# RxPaired-server + +## Note about the default server behavior + +This server was initially written mostly for short-scale usage, to be exposed to +the internet on a relatively small server and to require as few maintenance as +possible. + +As such, it will by default generate a password and will shutdown automatically +when any "abnormal" but impactful behavior is detected (all of this is +configurable): + +1. when more than 50 passwords have been received the last 24h +2. when more than 500 client connections happen in the last 24h +3. when more than 500 device connections happen in the last 24h +4. When more than a million WebSocket messages coming from all devices is + received in a group of 24h (also, client messages that are too long are + ignored) +5. When more than 1000 message coming from all inspectors is received in a + group of 24h. + +The server can in any case be restarted and those limitations be disabled at any +time. + +This was mostly done as a prevention measure from our own mistakes (we could for +example have accidently written an infinite loop blasting logs!) but also as a +signal for when curious traffic is coming from elsewhere. + +## How to run it + +Before being able to start this server, you first need to install the few node +dependencies it needs and build it: + +```sh +npm install +npm run build --workspace="server" +``` + +This server can then be run simply by calling the `./server/RxPaired-server` executable: + +```sh +node ./server/RxPaired-server +``` + +It is recommended that you look at this server options first to see if there's +things you want to tweak. You can look at all available options by calling: + +```sh +node ./server/RxPaired-server --help +``` + +## About HTTPS/WSS + +The RxPaired-server uses WebSocket connections relying on HTTP, and not HTTPS. + +This may be problematic when testing applications on devices running as an HTTPS +page, as browsers will generally block such "unsecure" requests when coming from +a "secure" web page. + +To work-around this issue, it is for now recommended to set up an HTTPS proxy, +which will redirect to the right RxPaired port(s). +This can be done by using server softwares like [Apache](https://httpd.apache.org/) +or [Nginx](https://www.nginx.com/) on servers with a valid certificate, **OR**, +if you don't have such server at hand, solutions like [ngrok](https://ngrok.com/). + +Note that technically, if you prefer to only proxify one WebSocket connection +between the two listened to, the one listening for devices is usually the most +important one. +This is because the other one will be linked to the `RxPaired-inspector`, which +you're hosting yourself. As such, you may just decide to run it from an HTTP URL +which itself has no issue relyng on another HTTP server. +And even when launching the RxPaired-inspector through HTTPS, it should +generally be easy enough to disable the corresponding "mixed contents" security +checks on your personal browsers. + +## How to update it and build it + +The RxPaired-server code is written in TypeScript and can be found in the `server/src` +directory, the main file (the first file being called) being `server/src/main.ts`. + +Before building it, you should install all dependencies: + +```sh +npm install +``` + +And you can then call the `build` script to update the code in `./server/build` (on +which `./server/RxPaired-server` points to): + +```sh +npm run build --workspace="server" +``` + +You can look at the other available npm scripts by peeking inside +`server/package.json`. + +## Server API + +This server exposes multiple routes by WebSocket to perform various actions. + +Note that calling the following routes should already be implemented by the +RxPaired-Inspector and RxPaired-Client implementations. As such you don't need +to read this chapter unless either: + +- You want to update either the RxPaired-Inspector or RxPaired-Client and that + implies understanding the current API. + +- You want to update the RxPaired-server's code. + +- You want to re-implement any of the bricks of RxPaired, yet not all as them, + and as such you want to understand its API. + +- You're curious about how it works. + +### Through the inspector port + +Messages communicated through the inspector port (see server flags) are intended +for exchanges with the RxPaired-inspector web inspector. + +If the server is configured with a password, like it is by default, **all** URLs +reached through the inspector port should first begin by the `/` +path, where `` is the server's password, then concatenating on +top of it the following paths. + +If the server is configured with no password, the following paths should be used +directly. + +#### `/!list` + +List the active tokens and gives some information about the RxPaired-server. + +"Active tokens" are tokens that can currently be used by a device (running +RxPaired-client) and that can be inspected through an RxPaired-inspector (see +corresponding routes). + +The server will regularly send (every ~3 seconds) that data back to the +inspector through that WebSocket connection starting immediately when the +request has been performed. + +The data sent is an UTF-8-encoded JSON with the following keys: + +- `isNoTokenEnabled` (boolean): If `true`, the `/!notoken` route can be used + by devices (see devices API) as this feature has been enabled on the server. + +- `tokenList` (Array.Object): Array of objects each having the following + properties: + + - `tokenId` (string): The token identifier itself, which can be used on + the inspector (and which can be used in stored logs) to inspect the + corresponding logs. + + - `date` (number): Unix timestamp on the server at the time the token has + been created. + + - `isPersistent` (boolean): If `true` this token will only expire once + its expiration date (see `msUntilExpiration` property) has been + reached. + + If `false`, the token will expire either when its expiration date has + been reached **OR** when both no inspector and no device are connected, + through this RxPaired-server, to that token anymore. + + - `msUntilExpiration` (number): amount of milliseconds until the token + expires and becomes thus invalid. + After that delay, the token has to be either re-created, or a new manual + expiration has to be set on it through the right API. + +#### `//` + +Where: + +- `` is a string composed only of alphanumeric characters. + +- `` is a time in milliseconds. + + Note that that part is completely optional (description of the semantics + and syntax of not setting it in the next route's description). + **Not setting an expiration time is probably what you want to do in most + cases**. This API is only listed first as it is a more general case. + +##### Behavior + +If no token currently exists with that ``, creates a regular +(non-persistent) token which will allow a unique device to send logs to +the server. + +Non-persistent tokens will remain available until either (whichever comes first): + +- its expiration time is reached, which is provided by `` + (which is the remaining availability time of that token from the point this + request is received, in milliseconds). + +- all WebSocket connections linked to that token (including this one, the + device's and maybe other RxPaired-inspectors listening to it) are all + disconnected from the server. + +If you want a token to stay alive until its expiration date, even when no +WebSocket connection linked to it are alive, you may want to create a +"persistent" token instead (see corresponding route). + +If a new request is done through that route while a token with that +`` was already created, the already-created token will keep its +original persistence status but the expiration date will be updated with +the new set ``. + +This WebSocket connection is then kept alive and different kind of messages +(device logs, instruction results etc. see below) will be sent through it. +The inspector may also send information through this WebSocket (see below). + +##### Messages sent by the server + +Various messages may be sent by the server through that WebSocket connection: + +- **ack**: ack messages are just an UTF-8-encoded `ack` string sent by the + server as soon as it receives the connection and considers it valid (for + example: the right password has been set). In cases where the request is + invalid, the server will close the WebSocket instead. + + It is always the first message sent, and is sent only once. + + There is no response to this message, it is just here to indicate that + the connection has been well received. + +- **Initial Device Message**: Message sent by the server which describes the + first log(s) sent by the device. It should be sent before any logs messages + originating from the device. + + The content of this message is an UTF-8 encoded JSON, with the following + keys: + + - `type` (string): Always set to `"Init"`. This allows you to identify + that this is an `Init` message. + + - `value` (object): Object with the following keys: + + - `timestamp` (number): The monotically-increasing timestamp in + milliseconds choosen by the device when its first message was sent. + + It is intended to be compared with `dateMs` so other messages + originating from the device may be easily dated. + + - `dateMs` (number): Unix timestamp in milliseconds generated by the + device at the same moment that the `timestamp` property was produced. + + - `history` (Array.string): The last `N` "logs" sent by the device in + chronological order, with `N` ranging from `0` to the `maxHistorySize` + value communicated in this same object. + + History is only enabled if you explicitely enabled it through the + right server flag. In other cases, this array is empty. + + Note that only device messages that are actually passed through to + inspectors (as opposed to messages only meant for the server) are + stored in this history (more details below). + + - `maxHistorySize` (number): The maximum number of elements the + `history` array (communicated through this same object) can contain. + + If `history` has the same length, then it may be that older logs + have been removed from it to respect that limit. + +- **Evaluation results**: An inspector can send JavaScript code to be executed + on the device, those are called "evaluations" by the RxPaired-server. + + When and if a device executes such instructions and if it executes with + success, it will return the result of executing it in an "Evaluation result" + message which is described here. + + Note that ALL inspector listening to a given device (through its token), + even those that did not send the original JavaScript code, will receive the + corresponding evaluation result if sent by the device. + + Evaluation results messages are UTF-8 encoded JSON with the following + keys: + + - `type` (string): Always set to `"eval-result"`. This allows you to + identify that this is an Evaluation result message. + + - `value` (object): Object with the following keys: + + - `id` (string): The same identifier that was sent alongside the + corresponding original evaluation message sent by the inspector. + + Can be used if multiple evaluation messages were sent to know + which result is about which evaluation. + + - `data` (string|undefined): The optional result itself, transformed + to a string format by the RxPaired-client. + +- **Evaluation errors**: An inspector can send JavaScript code to be executed + on the device, those are called "evaluations" by the RxPaired-server. + + When and if a device executes such instructions and if the corresponding + code throws on the device, the RxPaired-client will return the error + obtained when executing it in an "Evaluation error" message which is + described here. + + Note that ALL inspector listening to a given device (through its token), + even those that did not send the original JavaScript code, will receive the + corresponding evaluation error if sent by the device. + + Evaluation errors messages are UTF-8 encoded JSON with the following + keys: + + - `type` (string): Always set to `"eval-error"`. This allows you to + identify that this is an Evaluation error message. + + - `value` (object): Object with the following keys: + + - `id` (string): The same identifier that was sent alongside the + corresponding original evaluation message sent by the inspector. + + Can be used if multiple evaluation messages were sent to know + which error is about which evaluation message. + + - `error` (object): Object with the following keys: + + - `name` (string | undefined): Optional `name` property on the + corresponding error JavaScript object thrown. + + - `message` (string | undefined): Optional `message` property on + the corresponding error JavaScript object thrown. + +- **ping**: Ping messages are just an UTF-8-encoded `ping` string, which the + other side of the connection (the inspector) is supposed to respond + by `pong`. + + The idea behind this message is to ensure that the connection stays open + even when no message has been received in a long time, as some applications + (such as servers) might automatically decide to close the connection + otherwise. + +- **logs**: Other messages that do not correspond to either of the preceding + conditions are "logs" directly originating from the device, the server just + passing them through through the inspector(s)'s WebSocket directly as they + are received. + + They should be encoded as an UTF-8 string with the following format: + ` [] ` + + Where: + + - `` is the monotically-increasing timestamp in milliseconds + produced by the same method than the "Initial Device Message" sent by + the device. + + - `` is an identifier for the type of log. It is generally + correlated to the level of the logs (e.g. `error`, `debug` etc.) + but may also be used to identify another category of log (e.g. `Network` + for Network-related log). + + - `` is the logged message itself. + + The message may be truncated (compared to what has been sent by the + device) if it was originally longer than the maximum length configured + by the RxPaired-server (see server flags), which is expressed in UTF-16 + code units (JavaScript!). + +##### Messages sent TO the server + +As a websocket connection, exchanges are bi-directional. An inspector can send +through that route the following types of messages: + +- **pong**: Pong messages are just an UTF-8-encoded `pong` string, which is + the message an inspector should send after receiving a `ping` message + through that same connection. + +- **Evaluation**: Those messages allow an inspector to ask the device to + execute some JavaScript code present in this message. + + After executing the corresponding code, the device will either send an + "Evaluation result" message if it executed with success or an + "Evaluation error" message if it threw. Both of those messages are + communicated by the server and are documented in this API documentation. + + Evaluation messages are UTF-8 encoded JSON with the following keys: + + - `type` (string): Always set to `"eval"`. This allows the server and + device to identify that this is an Evaluation message. + + - `value` (object): Object with the following keys: + + - `id` (string): Identifier that will be repeated by the device on + the corresponding result (either an "Evaluation result" if it + executed with success or an "Evaluation error" if it threw). + + - `instruction` (string): The JavaScript code to execute on the + device. Note that the `return` JS keywork can be used to return + data alongside the corresponding "Evaluation result" message, just + as if returning that data as the last instruction of a JS function. + +#### `/` + +Like for `//`, but: + +- if no token with that name was created, we will rely on the default + expiration time configured on the server instead (see `-h` server flag). + +- if a token with that name was already created, its original expiration date + will remain unchanged. + +#### `/!persist//` + +Where: + +- `` is a string composed only of alphanumeric characters. + +- `` is a time in milliseconds. + + Note that that part is completely optional (description of the semantics + and syntax of not setting it in the next route's description). + +##### Behavior + +Creates a new "persistent" token - which allows device to send logs to the +server - with the identifier indicated by ``. + +The WebSocket connection will then remain open, and communicate the same +messages than the equivalent `//` route (see its +documentation). + +**Non**-persistent tokens are removed as soon as its linked device and all +inspector pages stopped listening to it. +Persistent tokens however are kept alive by the server until their expiration +date is reached, which is provided by `` (which is the +remaining availability time of that token from the point that request is +received, in milliseconds). + +Unlike regular tokens, persistent tokens thus allows the device to send logs to +the server whenever it wants, even when no inspector is listening to that token, +as long as the expiration date is not reached. + +This is useful either when needing to debug on another person's device without +needing to synchronize the setup of an RxPaired-inspector page or just to speed +up some quick tests when you don't want to open an RxPaired-inspector. +The flip side however is that any device knowing that token's value may be able +to send logs to the server during the lifetime of this token, whereas +non-persistent tokens offer more security by generally being very ephemeral. + +Note that you can update a persistent token's expiration, by re-calling the same +API for the same `` but with a different ``, which +will be the new one considered. + +To remove a persisted token, you can set its `` to either +`0` or to a negative value. + +If a new request is done through that route while a token with that +`` was already created, the already-created token will become +persistent if it's wasn't already and the expiration date will be updated with +the new set ``. + +#### `/!persist/` + +Like for `/!persist//`, but: + +- if no token with that name was created, we will rely on the default + expiration time configured on the server instead (see `-h`server flag). + +- if a token with that name was already created, it will be made persistent if + it wasn't already but its original expiration date will remain unchanged. + +### Through the device port + +Messages communicated through the device port (see server flags) are intended +for exchanges with the RxPaired-client running on the device. + +Unlike messages communicated through this inspector, those do not need to have +the server's password (if set) prepended to its routes, unless explicitely +specified. + +The following subchapters will list the various routes exposed by the +RxPaired-server through the device port. + +#### `/` + +Where: + +- `` is a string composed only of alphanumeric characters. + + That has been previously explicitely created by an RxPaired-inspector + **AND** that is still considered "active" (see routes exposed by the + inspector ports for more information on what this implies). + +##### Behavior + +Allow the device to beginning sending logs linked to that token, so the server +may store them (if the right server flag is set) and communicate it to the +RxPaired-inspectors currently listening to that token. + +Note that **only one device at a time** can maintain a connection on that +specific token. If multiple WebSocket connections for the same token are done +at the same time on the device port of the RxPaired-server, the token will be +automatically and directly invalidated and all connections linked to it, both +through the inspector port and through the device ports, will be closed. + +##### Messages sent by the server + +Messages may be sent by the server through that WebSocket connection: + +- **ack**: ack messages are just an UTF-8-encoded `ack` string sent by the + server as soon as it receives the connection and considers it valid (for + example: the wanted token exists). In cases where the request is invalid, + the server will close the WebSocket instead. + + It is always the first message sent, and is sent only once. + + There is no response to this message, it is just here to indicate that + the connection has been well received. + +- **ping**: Ping messages are just an UTF-8-encoded `ping` string, which the + other side of the connection (the RxPaired-client) is supposed to respond + by `pong`. + + The idea behind this message is to ensure that the connection stays open + even when no message has been received in a long time, as some applications + (such as servers) might automatically decide to close the connection + otherwise. + +- **Evaluation**: Those messages originates from an RxPaired-inspector and + allows it to execute some JavaScript code present in this message. + + After executing the corresponding code, the device should either send an + "Evaluation result" message if it executed with success or an + "Evaluation error" message if it threw. Both of those messages are at + destinations of RxPaired-inspectors. + + Evaluation messages are UTF-8 encoded JSON with the following keys: + + - `type` (string): Always set to `"eval"`. This allows you to identify + that this is an Evaluation message. + + - `value` (object): Object with the following keys: + + - `id` (string): Identifier that should be repeated by on the + corresponding result (either an "Evaluation result" if it + executed with success or an "Evaluation error" if it threw). + + - `instruction` (string): The JavaScript code to execute. + Note that the `return` JS keywork indicates that the returned + data should be sent alongside the corresponding "Evaluation result" + message, just as if returning that data as the last instruction of a + JS function. + +##### Messages sent TO the server + +- **Initial Message**: This should be the initial message send by the + device, describing information on the device. + + The content of this message is an UTF-8 encoded string, with the following + format: + `Init v1 ` + + Where: + + - ` is a monotically-increasing timestamp in milliseconds + choosen by the device when its first message was sent. + + It is intended to be compared with `` so other messages + originating from the device may be easily dated. + + - is a unix timestamp in milliseconds generated by the + device at the same moment that the `` property was produced. + +- **pong**: Pong messages are just an UTF-8-encoded `pong` string, which is + the message an inspector should send after receiving a `ping` message + through that same connection. + +- **Evaluation results**: When an instruction, through an Evaluation message + of that same WebSocket connection, executes with success, the device should + send back a corresponding "Evaluation result" message described here. + + Evaluation results messages should be UTF-8 encoded JSON with the following + keys: + + - `type` (string): Always set to `"eval-result"`. This allows the server + and inspectors to identify that this is an Evaluation result message. + + - `value` (object): Object with the following keys: + + - `id` (string): The same identifier that was sent alongside the + corresponding original Evaluation message. + + Can be used if multiple evaluation messages were sent to know + which result is about which evaluation. + + - `data` (string|undefined): The optional result itself, transformed + to a string format by a method of the RxPaired-client choosing. + +- **Evaluation errors**: When an instruction, through an Evaluation message + of that same WebSocket connection, throws while executing, the device should + send back a corresponding "Evaluation error" message described here. + + Evaluation errors messages should be UTF-8 encoded JSON with the following + keys: + + - `type` (string): Always set to `"eval-error"`. This allows the server + and inspectors to identify that this is an Evaluation error message. + + - `value` (object): Object with the following keys: + + - `id` (string): The same identifier that was sent alongside the + corresponding original Evaluation message. + + Can be used if multiple evaluation messages were sent to know + which error is about which evaluation message. + + - `error` (object): Object with the following keys: + + - `name` (string | undefined): Optional `name` property on the + corresponding error JavaScript object thrown. + + - `message` (string | undefined): Optional `message` property on + the corresponding error JavaScript object thrown. + +- **logs**: Other messages that do not correspond to either of the preceding + conditions are all considered "logs". + + They should be encoded as an UTF-8 string with the following format: + ` [] ` + + Where: + + - `` is the monotically-increasing timestamp in milliseconds + produced by the same method than the "Initial Message" sent by the + device. + + - `` is an identifier for the type of log. It is generally + correlated to the level of the logs (e.g. `error`, `debug` etc.) + but may also be used to identify another category of log (e.g. `Network` + for Network-related log). + + - `` is the logged message itself. + + The message should not be longer than the maximum length configured by + the RxPaired-server (see server flags), which is expressed in UTF-16 + code units (JavaScript!). If the message is actually longer, it will be + truncated when received by the server and inspectors - which may render + it unusable. + +#### `/!notoken/` + +**OR** just `/!notoken` if there is no password. + +Where: + +- `` is the server's password if configured (see server + flags). + +**NOTE**: This route could have been disabled depending on the flags associated +to the RxPaired-server. + +Same behavior and messages than for the `/` route on the device port +excepted that a token will be automatically generated by the server. This token +will have the default token expiration (see server flags) and be removed based +on the same rules that non-persistent tokens generated through the inspector +port (see corresponding route). + +This token wont be communicated to the device, but: + +- If the right flags have been set, it will be present in the name of the + corresponding log file generated on disk by the server. + +- The `/!list` route, exposed on the inspector port, will list that token, + allowing RxPaired-inspectors to listen to it. diff --git a/inspector/README.md b/inspector/README.md index d16f456..868d51a 100644 --- a/inspector/README.md +++ b/inspector/README.md @@ -5,72 +5,3 @@ RxPaired tool. This is the code of the web interface allowing to inspect what's going on on the inspected device. - -## How to build and run it - -To build it, you first need to install its dependencies. - -To do that, make sure that you have `npm` installed and this repository cloned. -Then go to this directory on a terminal, and type: - -```sh -npm install -``` - -You then have to create a `.npmrc` file in this directory. -You can base yourself on the `.npmrc.sample` file: - -```sh -cp .npmrc.sample .npmrc -``` - -In that new `.npmrc` file, you'll need to set two URLs: - -1. `inspector_debugger_url`: - - This will be the WebSocket address `RxPaired-server` is listening to for - `RxPaired-inspector` connections. - - If you didn't start the `RxPaired-server` yet, you should first start doing this by - going to the `../server` directory. - - Note that the URL already present uses the default port used by the server. If your - server runs locally in the default config, you might not actually have to update it. - - In most cases however, you might want to use HTTPS/WSS instead of the default HTTP/WS - (to make the `RxPaired-Client` runnable in HTTPS pages). - There, you might need to update this value to the actual HTTPS URL used. - -2. `device_script_url`: - - This is the URL the `RxPaired-Client` (the script that will be deployed to devices) - is available at. - - You can leave this URL empty if you do not wish to serve that script though HTTP(S) - means. - In that case, you will have to copy/paste the content of that script into the HTML - page running on the device. - -Once this is done, you can start building the `RxPaired-inspector`. - -In your terminal, type: - -```sh -npm run build:min -``` - -You can now start using the `RxPaired-inspector`, you just need to serve both the -`index.html` and the newly generated `inspector.js` files in the current directory. - -You can do so easily by typing in a terminal: - -```js -npm run serve -``` - -Multiple persons can even build and serve their own `RxPaired-inspector` while -debugging the same devices as long as the same `RxPaired-server` is used. - -Though you might prefer the simplicity of just letting a "real" HTTP(S) server serve both -of those files instead. This way this build step does not need to be repeated for each -user. diff --git a/server/README.md b/server/README.md index 825dd5d..d5ca0eb 100644 --- a/server/README.md +++ b/server/README.md @@ -19,666 +19,3 @@ Usually, tokens need first to be declared by web inspectors before a device can contact the server with it, yet other methods exist to let the device declare new tokens itself instead. This is all indicated in the web inspector's pages and by the API documentation below. - -## Note about the default server behavior - -This server was initially written mostly for short-scale usage, to be exposed to -the internet on a relatively small server and to require as few maintenance as -possible. - -As such, it will by default generate a password and will shutdown automatically -when any "abnormal" but impactful behavior is detected (all of this is -configurable): - -1. when more than 50 passwords have been received the last 24h -2. when more than 500 client connections happen in the last 24h -3. when more than 500 device connections happen in the last 24h -4. When more than a million WebSocket messages coming from all devices is - received in a group of 24h (also, client messages that are too long are - ignored) -5. When more than 1000 message coming from all inspectors is received in a - group of 24h. - -The server can in any case be restarted and those limitations be disabled at any -time. - -This was mostly done as a prevention measure from our own mistakes (we could for -example have accidently written an infinite loop blasting logs!) but also as a -signal for when curious traffic is coming from elsewhere. - -## How to run it - -Before being able to start this server, you first need to install the few node -dependencies it needs and build it: - -```sh -npm install -npm run build -``` - -This server can then be run simply by calling the `./RxPaired-server` executable: - -```sh -node ./RxPaired-server -``` - -It is recommended that you look at this server options first to see if there's -things you want to tweak. You can look at all available options by calling: - -```sh -node ./RxPaired-server --help -``` - -## About HTTPS/WSS - -The RxPaired-server uses WebSocket connections relying on HTTP, and not HTTPS. - -This may be problematic when testing applications on devices running as an HTTPS -page, as browsers will generally block such "unsecure" requests when coming from -a "secure" web page. - -To work-around this issue, it is for now recommended to set up an HTTPS proxy, -which will redirect to the right RxPaired port(s). -This can be done by using server softwares like [Apache](https://httpd.apache.org/) -or [Nginx](https://www.nginx.com/) on servers with a valid certificate, **OR**, -if you don't have such server at hand, solutions like [ngrok](https://ngrok.com/). - -Note that technically, if you prefer to only proxify one WebSocket connection -between the two listened to, the one listening for devices is usually the most -important one. -This is because the other one will be linked to the `RxPaired-inspector`, which -you're hosting yourself. As such, you may just decide to run it from an HTTP URL -which itself has no issue relyng on another HTTP server. -And even when launching the RxPaired-inspector through HTTPS, it should -generally be easy enough to disable the corresponding "mixed contents" security -checks on your personal browsers. - -## How to update it and build it - -The RxPaired-server code is written in TypeScript and can be found in the `src` -directory, the main file (the first file being called) being `src/main.ts`. - -Before building it, you should install all dependencies: - -```sh -npm install -``` - -And you can then call the `build` script to update the code in `./build` (on -which `./RxPaired-server` points to): - -```sh -npm run build -``` - -You can look at the other available npm scripts by peeking inside -`package.json`. - -## Server API - -This server exposes multiple routes by WebSocket to perform various actions. - -Note that calling the following routes should already be implemented by the -RxPaired-Inspector and RxPaired-Client implementations. As such you don't need -to read this chapter unless either: - -- You want to update either the RxPaired-Inspector or RxPaired-Client and that - implies understanding the current API. - -- You want to update the RxPaired-server's code. - -- You want to re-implement any of the bricks of RxPaired, yet not all as them, - and as such you want to understand its API. - -- You're curious about how it works. - -### Through the inspector port - -Messages communicated through the inspector port (see server flags) are intended -for exchanges with the RxPaired-inspector web inspector. - -If the server is configured with a password, like it is by default, **all** URLs -reached through the inspector port should first begin by the `/` -path, where `` is the server's password, then concatenating on -top of it the following paths. - -If the server is configured with no password, the following paths should be used -directly. - -#### `/!list` - -List the active tokens and gives some information about the RxPaired-server. - -"Active tokens" are tokens that can currently be used by a device (running -RxPaired-client) and that can be inspected through an RxPaired-inspector (see -corresponding routes). - -The server will regularly send (every ~3 seconds) that data back to the -inspector through that WebSocket connection starting immediately when the -request has been performed. - -The data sent is an UTF-8-encoded JSON with the following keys: - -- `isNoTokenEnabled` (boolean): If `true`, the `/!notoken` route can be used - by devices (see devices API) as this feature has been enabled on the server. - -- `tokenList` (Array.Object): Array of objects each having the following - properties: - - - `tokenId` (string): The token identifier itself, which can be used on - the inspector (and which can be used in stored logs) to inspect the - corresponding logs. - - - `date` (number): Unix timestamp on the server at the time the token has - been created. - - - `isPersistent` (boolean): If `true` this token will only expire once - its expiration date (see `msUntilExpiration` property) has been - reached. - - If `false`, the token will expire either when its expiration date has - been reached **OR** when both no inspector and no device are connected, - through this RxPaired-server, to that token anymore. - - - `msUntilExpiration` (number): amount of milliseconds until the token - expires and becomes thus invalid. - After that delay, the token has to be either re-created, or a new manual - expiration has to be set on it through the right API. - -#### `//` - -Where: - -- `` is a string composed only of alphanumeric characters. - -- `` is a time in milliseconds. - - Note that that part is completely optional (description of the semantics - and syntax of not setting it in the next route's description). - **Not setting an expiration time is probably what you want to do in most - cases**. This API is only listed first as it is a more general case. - -##### Behavior - -If no token currently exists with that ``, creates a regular -(non-persistent) token which will allow a unique device to send logs to -the server. - -Non-persistent tokens will remain available until either (whichever comes first): - -- its expiration time is reached, which is provided by `` - (which is the remaining availability time of that token from the point this - request is received, in milliseconds). - -- all WebSocket connections linked to that token (including this one, the - device's and maybe other RxPaired-inspectors listening to it) are all - disconnected from the server. - -If you want a token to stay alive until its expiration date, even when no -WebSocket connection linked to it are alive, you may want to create a -"persistent" token instead (see corresponding route). - -If a new request is done through that route while a token with that -`` was already created, the already-created token will keep its -original persistence status but the expiration date will be updated with -the new set ``. - -This WebSocket connection is then kept alive and different kind of messages -(device logs, instruction results etc. see below) will be sent through it. -The inspector may also send information through this WebSocket (see below). - -##### Messages sent by the server - -Various messages may be sent by the server through that WebSocket connection: - -- **ack**: ack messages are just an UTF-8-encoded `ack` string sent by the - server as soon as it receives the connection and considers it valid (for - example: the right password has been set). In cases where the request is - invalid, the server will close the WebSocket instead. - - It is always the first message sent, and is sent only once. - - There is no response to this message, it is just here to indicate that - the connection has been well received. - -- **Initial Device Message**: Message sent by the server which describes the - first log(s) sent by the device. It should be sent before any logs messages - originating from the device. - - The content of this message is an UTF-8 encoded JSON, with the following - keys: - - - `type` (string): Always set to `"Init"`. This allows you to identify - that this is an `Init` message. - - - `value` (object): Object with the following keys: - - - `timestamp` (number): The monotically-increasing timestamp in - milliseconds choosen by the device when its first message was sent. - - It is intended to be compared with `dateMs` so other messages - originating from the device may be easily dated. - - - `dateMs` (number): Unix timestamp in milliseconds generated by the - device at the same moment that the `timestamp` property was produced. - - - `history` (Array.string): The last `N` "logs" sent by the device in - chronological order, with `N` ranging from `0` to the `maxHistorySize` - value communicated in this same object. - - History is only enabled if you explicitely enabled it through the - right server flag. In other cases, this array is empty. - - Note that only device messages that are actually passed through to - inspectors (as opposed to messages only meant for the server) are - stored in this history (more details below). - - - `maxHistorySize` (number): The maximum number of elements the - `history` array (communicated through this same object) can contain. - - If `history` has the same length, then it may be that older logs - have been removed from it to respect that limit. - -- **Evaluation results**: An inspector can send JavaScript code to be executed - on the device, those are called "evaluations" by the RxPaired-server. - - When and if a device executes such instructions and if it executes with - success, it will return the result of executing it in an "Evaluation result" - message which is described here. - - Note that ALL inspector listening to a given device (through its token), - even those that did not send the original JavaScript code, will receive the - corresponding evaluation result if sent by the device. - - Evaluation results messages are UTF-8 encoded JSON with the following - keys: - - - `type` (string): Always set to `"eval-result"`. This allows you to - identify that this is an Evaluation result message. - - - `value` (object): Object with the following keys: - - - `id` (string): The same identifier that was sent alongside the - corresponding original evaluation message sent by the inspector. - - Can be used if multiple evaluation messages were sent to know - which result is about which evaluation. - - - `data` (string|undefined): The optional result itself, transformed - to a string format by the RxPaired-client. - -- **Evaluation errors**: An inspector can send JavaScript code to be executed - on the device, those are called "evaluations" by the RxPaired-server. - - When and if a device executes such instructions and if the corresponding - code throws on the device, the RxPaired-client will return the error - obtained when executing it in an "Evaluation error" message which is - described here. - - Note that ALL inspector listening to a given device (through its token), - even those that did not send the original JavaScript code, will receive the - corresponding evaluation error if sent by the device. - - Evaluation errors messages are UTF-8 encoded JSON with the following - keys: - - - `type` (string): Always set to `"eval-error"`. This allows you to - identify that this is an Evaluation error message. - - - `value` (object): Object with the following keys: - - - `id` (string): The same identifier that was sent alongside the - corresponding original evaluation message sent by the inspector. - - Can be used if multiple evaluation messages were sent to know - which error is about which evaluation message. - - - `error` (object): Object with the following keys: - - - `name` (string | undefined): Optional `name` property on the - corresponding error JavaScript object thrown. - - - `message` (string | undefined): Optional `message` property on - the corresponding error JavaScript object thrown. - -- **ping**: Ping messages are just an UTF-8-encoded `ping` string, which the - other side of the connection (the inspector) is supposed to respond - by `pong`. - - The idea behind this message is to ensure that the connection stays open - even when no message has been received in a long time, as some applications - (such as servers) might automatically decide to close the connection - otherwise. - -- **logs**: Other messages that do not correspond to either of the preceding - conditions are "logs" directly originating from the device, the server just - passing them through through the inspector(s)'s WebSocket directly as they - are received. - - They should be encoded as an UTF-8 string with the following format: - ` [] ` - - Where: - - - `` is the monotically-increasing timestamp in milliseconds - produced by the same method than the "Initial Device Message" sent by - the device. - - - `` is an identifier for the type of log. It is generally - correlated to the level of the logs (e.g. `error`, `debug` etc.) - but may also be used to identify another category of log (e.g. `Network` - for Network-related log). - - - `` is the logged message itself. - - The message may be truncated (compared to what has been sent by the - device) if it was originally longer than the maximum length configured - by the RxPaired-server (see server flags), which is expressed in UTF-16 - code units (JavaScript!). - -##### Messages sent TO the server - -As a websocket connection, exchanges are bi-directional. An inspector can send -through that route the following types of messages: - -- **pong**: Pong messages are just an UTF-8-encoded `pong` string, which is - the message an inspector should send after receiving a `ping` message - through that same connection. - -- **Evaluation**: Those messages allow an inspector to ask the device to - execute some JavaScript code present in this message. - - After executing the corresponding code, the device will either send an - "Evaluation result" message if it executed with success or an - "Evaluation error" message if it threw. Both of those messages are - communicated by the server and are documented in this API documentation. - - Evaluation messages are UTF-8 encoded JSON with the following keys: - - - `type` (string): Always set to `"eval"`. This allows the server and - device to identify that this is an Evaluation message. - - - `value` (object): Object with the following keys: - - - `id` (string): Identifier that will be repeated by the device on - the corresponding result (either an "Evaluation result" if it - executed with success or an "Evaluation error" if it threw). - - - `instruction` (string): The JavaScript code to execute on the - device. Note that the `return` JS keywork can be used to return - data alongside the corresponding "Evaluation result" message, just - as if returning that data as the last instruction of a JS function. - -#### `/` - -Like for `//`, but: - -- if no token with that name was created, we will rely on the default - expiration time configured on the server instead (see `-h` server flag). - -- if a token with that name was already created, its original expiration date - will remain unchanged. - -#### `/!persist//` - -Where: - -- `` is a string composed only of alphanumeric characters. - -- `` is a time in milliseconds. - - Note that that part is completely optional (description of the semantics - and syntax of not setting it in the next route's description). - -##### Behavior - -Creates a new "persistent" token - which allows device to send logs to the -server - with the identifier indicated by ``. - -The WebSocket connection will then remain open, and communicate the same -messages than the equivalent `//` route (see its -documentation). - -**Non**-persistent tokens are removed as soon as its linked device and all -inspector pages stopped listening to it. -Persistent tokens however are kept alive by the server until their expiration -date is reached, which is provided by `` (which is the -remaining availability time of that token from the point that request is -received, in milliseconds). - -Unlike regular tokens, persistent tokens thus allows the device to send logs to -the server whenever it wants, even when no inspector is listening to that token, -as long as the expiration date is not reached. - -This is useful either when needing to debug on another person's device without -needing to synchronize the setup of an RxPaired-inspector page or just to speed -up some quick tests when you don't want to open an RxPaired-inspector. -The flip side however is that any device knowing that token's value may be able -to send logs to the server during the lifetime of this token, whereas -non-persistent tokens offer more security by generally being very ephemeral. - -Note that you can update a persistent token's expiration, by re-calling the same -API for the same `` but with a different ``, which -will be the new one considered. - -To remove a persisted token, you can set its `` to either -`0` or to a negative value. - -If a new request is done through that route while a token with that -`` was already created, the already-created token will become -persistent if it's wasn't already and the expiration date will be updated with -the new set ``. - -#### `/!persist/` - -Like for `/!persist//`, but: - -- if no token with that name was created, we will rely on the default - expiration time configured on the server instead (see `-h`server flag). - -- if a token with that name was already created, it will be made persistent if - it wasn't already but its original expiration date will remain unchanged. - -### Through the device port - -Messages communicated through the device port (see server flags) are intended -for exchanges with the RxPaired-client running on the device. - -Unlike messages communicated through this inspector, those do not need to have -the server's password (if set) prepended to its routes, unless explicitely -specified. - -The following subchapters will list the various routes exposed by the -RxPaired-server through the device port. - -#### `/` - -Where: - -- `` is a string composed only of alphanumeric characters. - - That has been previously explicitely created by an RxPaired-inspector - **AND** that is still considered "active" (see routes exposed by the - inspector ports for more information on what this implies). - -##### Behavior - -Allow the device to beginning sending logs linked to that token, so the server -may store them (if the right server flag is set) and communicate it to the -RxPaired-inspectors currently listening to that token. - -Note that **only one device at a time** can maintain a connection on that -specific token. If multiple WebSocket connections for the same token are done -at the same time on the device port of the RxPaired-server, the token will be -automatically and directly invalidated and all connections linked to it, both -through the inspector port and through the device ports, will be closed. - -##### Messages sent by the server - -Messages may be sent by the server through that WebSocket connection: - -- **ack**: ack messages are just an UTF-8-encoded `ack` string sent by the - server as soon as it receives the connection and considers it valid (for - example: the wanted token exists). In cases where the request is invalid, - the server will close the WebSocket instead. - - It is always the first message sent, and is sent only once. - - There is no response to this message, it is just here to indicate that - the connection has been well received. - -- **ping**: Ping messages are just an UTF-8-encoded `ping` string, which the - other side of the connection (the RxPaired-client) is supposed to respond - by `pong`. - - The idea behind this message is to ensure that the connection stays open - even when no message has been received in a long time, as some applications - (such as servers) might automatically decide to close the connection - otherwise. - -- **Evaluation**: Those messages originates from an RxPaired-inspector and - allows it to execute some JavaScript code present in this message. - - After executing the corresponding code, the device should either send an - "Evaluation result" message if it executed with success or an - "Evaluation error" message if it threw. Both of those messages are at - destinations of RxPaired-inspectors. - - Evaluation messages are UTF-8 encoded JSON with the following keys: - - - `type` (string): Always set to `"eval"`. This allows you to identify - that this is an Evaluation message. - - - `value` (object): Object with the following keys: - - - `id` (string): Identifier that should be repeated by on the - corresponding result (either an "Evaluation result" if it - executed with success or an "Evaluation error" if it threw). - - - `instruction` (string): The JavaScript code to execute. - Note that the `return` JS keywork indicates that the returned - data should be sent alongside the corresponding "Evaluation result" - message, just as if returning that data as the last instruction of a - JS function. - -##### Messages sent TO the server - -- **Initial Message**: This should be the initial message send by the - device, describing information on the device. - - The content of this message is an UTF-8 encoded string, with the following - format: - `Init v1 ` - - Where: - - - ` is a monotically-increasing timestamp in milliseconds - choosen by the device when its first message was sent. - - It is intended to be compared with `` so other messages - originating from the device may be easily dated. - - - is a unix timestamp in milliseconds generated by the - device at the same moment that the `` property was produced. - -- **pong**: Pong messages are just an UTF-8-encoded `pong` string, which is - the message an inspector should send after receiving a `ping` message - through that same connection. - -- **Evaluation results**: When an instruction, through an Evaluation message - of that same WebSocket connection, executes with success, the device should - send back a corresponding "Evaluation result" message described here. - - Evaluation results messages should be UTF-8 encoded JSON with the following - keys: - - - `type` (string): Always set to `"eval-result"`. This allows the server - and inspectors to identify that this is an Evaluation result message. - - - `value` (object): Object with the following keys: - - - `id` (string): The same identifier that was sent alongside the - corresponding original Evaluation message. - - Can be used if multiple evaluation messages were sent to know - which result is about which evaluation. - - - `data` (string|undefined): The optional result itself, transformed - to a string format by a method of the RxPaired-client choosing. - -- **Evaluation errors**: When an instruction, through an Evaluation message - of that same WebSocket connection, throws while executing, the device should - send back a corresponding "Evaluation error" message described here. - - Evaluation errors messages should be UTF-8 encoded JSON with the following - keys: - - - `type` (string): Always set to `"eval-error"`. This allows the server - and inspectors to identify that this is an Evaluation error message. - - - `value` (object): Object with the following keys: - - - `id` (string): The same identifier that was sent alongside the - corresponding original Evaluation message. - - Can be used if multiple evaluation messages were sent to know - which error is about which evaluation message. - - - `error` (object): Object with the following keys: - - - `name` (string | undefined): Optional `name` property on the - corresponding error JavaScript object thrown. - - - `message` (string | undefined): Optional `message` property on - the corresponding error JavaScript object thrown. - -- **logs**: Other messages that do not correspond to either of the preceding - conditions are all considered "logs". - - They should be encoded as an UTF-8 string with the following format: - ` [] ` - - Where: - - - `` is the monotically-increasing timestamp in milliseconds - produced by the same method than the "Initial Message" sent by the - device. - - - `` is an identifier for the type of log. It is generally - correlated to the level of the logs (e.g. `error`, `debug` etc.) - but may also be used to identify another category of log (e.g. `Network` - for Network-related log). - - - `` is the logged message itself. - - The message should not be longer than the maximum length configured by - the RxPaired-server (see server flags), which is expressed in UTF-16 - code units (JavaScript!). If the message is actually longer, it will be - truncated when received by the server and inspectors - which may render - it unusable. - -#### `/!notoken/` - -**OR** just `/!notoken` if there is no password. - -Where: - -- `` is the server's password if configured (see server - flags). - -**NOTE**: This route could have been disabled depending on the flags associated -to the RxPaired-server. - -Same behavior and messages than for the `/` route on the device port -excepted that a token will be automatically generated by the server. This token -will have the default token expiration (see server flags) and be removed based -on the same rules that non-persistent tokens generated through the inspector -port (see corresponding route). - -This token wont be communicated to the device, but: - -- If the right flags have been set, it will be present in the name of the - corresponding log file generated on disk by the server. - -- The `/!list` route, exposed on the inspector port, will list that token, - allowing RxPaired-inspectors to listen to it.