Skip to content

Commit

Permalink
chore(docs): fix docs and mark some as deprecated (#1754)
Browse files Browse the repository at this point in the history
* chore(docs): fix docs and mark some as deprecated

* Update docs/contributors/continuous-integration.md

Co-authored-by: Ivan Folgueira Bande <[email protected]>

* Update examples/v2/README.md

Co-authored-by: Ivan Folgueira Bande <[email protected]>

* mark TODO with date

* replace nim-waku with nwaku in docs

---------

Co-authored-by: Ivan Folgueira Bande <[email protected]>
  • Loading branch information
vpavlin and Ivansete-status authored May 25, 2023
1 parent 0fce3d8 commit b51fb61
Show file tree
Hide file tree
Showing 20 changed files with 87 additions and 75 deletions.
15 changes: 8 additions & 7 deletions docs/contributors/continuous-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,25 +6,26 @@ This document describes the continuous integration setup for `nim-waku`.

The CI setup exists on the Status.im Jenkins instance:

https://ci.status.im/job/nim-waku/
https://ci.infra.status.im/job/nim-waku/

It currently consists four jobs:

* [manual](https://ci.status.im/job/nim-waku/job/manual/) - For manually executing builds using parameters.
* [deploy-wakuv1-test](https://ci.status.im/job/nim-waku/job/deploy-wakuv1-test/) - Builds every new commit in `master` and deploys to `wakuv1.test` fleet.
* [deploy-wakuv2-test](https://ci.status.im/job/nim-waku/job/deploy-wakuv2-test/) - Builds every new commit in `master` and deploys to `wakuv2.test` fleet.
* [deploy-wakuv2-prod](https://ci.status.im/job/nim-waku/job/deploy-wakuv2-prod/) - Currently has no automatic trigger, and deploys to `wakuv2.prod` fleet.
* [manual](https://ci.infra.status.im/job/nim-waku/job/manual/) - For manually executing builds using parameters.
* [deploy-wakuv1-test](https://ci.infra.status.im/job/nim-waku/job/deploy-wakuv1-test/) - Builds every new commit in `master` and deploys to `wakuv1.test` fleet.
* [deploy-wakuv2-test](https://ci.infra.status.im/job/nim-waku/job/deploy-wakuv2-test/) - Builds every new commit in `master` and deploys to `wakuv2.test` fleet.
* [deploy-wakuv2-prod](https://ci.infra.status.im/job/nim-waku/job/deploy-wakuv2-prod/) - Currently has no automatic trigger, and deploys to `wakuv2.prod` fleet.

# Configuration

The main configuration file is [`Jenkinsfile`](../../Jenkinsfile) at the root of this repo.
The main configuration file is [`Jenkinsfile.release`](../../ci/Jenkinsfile.release) in the `ci` folder.

Key part is the definition of four `parameters`:
Key part is the definition of five `parameters`:

* `MAKE_TARGET` - Which `Makefile` target is built.
* `IMAGE_TAG` - Tag of the Docker image to push.
* `IMAGE_NAME` - Name of the Docker image to push.
* `NIMFLAGS` - Nim compilation parameters.
* `GIT_REF` - Git reference to build from (branch, tag, commit...)

The use of `?:` [Elvis operator](http://groovy-lang.org/operators.html#_elvis_operator) plays a key role in allowing parameters to be changed for each defined job in Jenkins without it being overridden by the `Jenkinsfile` defaults after every job run.
```groovy
Expand Down
8 changes: 4 additions & 4 deletions docs/contributors/release-process.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ For more context, see https://trunkbaseddevelopment.com/branch-for-release/
### Before release

Ensure all items in this list are ticked:
- [ ] All issues under the corresponding release [milestone](https://github.com/status-im/nwaku/milestones) has been closed or, after consultation, deferred to a next release.
- [ ] All issues under the corresponding release [milestone](https://github.com/waku-org/nwaku/milestones) has been closed or, after consultation, deferred to a next release.
- [ ] All submodules are up to date.
> **IMPORTANT:** Updating submodules requires a PR (and very often several "fixes" to maintain compatibility with the changes in submodules). That PR process must be done and merged a couple of days before the release.
> In case the submodules update has a low effort and/or risk for the release, follow the ["Update submodules"](./git-submodules.md) instructions.
Expand All @@ -36,8 +36,8 @@ git push origin v0.1
4. Open a PR

5. Harden release in release branch
- Create a [Github release](https://github.com/status-im/nwaku/releases) on the release tag.
- Add binaries for `macos` and `ubuntu` as release assets. Binaries can be compiled by triggering the ["Upload Release Asset"](https://github.com/status-im/nwaku/actions/workflows/release-assets.yml) workflow. Where possible, test the binaries before uploading to the release.
- Create a [Github release](https://github.com/waku-org/nwaku/releases) on the release tag.
- Add binaries for `macos` and `ubuntu` as release assets. Binaries can be compiled by triggering the ["Upload Release Asset"](https://github.com/waku-org/nwaku/actions/workflows/release-assets.yml) workflow. Where possible, test the binaries before uploading to the release.

6. Modify tag

Expand Down Expand Up @@ -72,6 +72,6 @@ git push origin v0.1
> Clients are reachable via the corresponding channels on the Vac Discord server.
> It should be enough to inform clients on the `#nwaku` and `#announce` channels on Discord.
> Informal conversations with specific repo maintainers are often part of this process.
- Deploy release to the `wakuv2.prod` fleet from [Jenkins](https://ci.status.im/job/nim-waku/job/deploy-wakuv2-prod/).
- Deploy release to the `wakuv2.prod` fleet from [Jenkins](https://ci.infra.status.im/job/nim-waku/job/deploy-wakuv2-prod/).
- Ensure that nodes successfully start up and monitor health using [Grafana](https://grafana.infra.status.im/d/qrp_ZCTGz/nim-waku-v2?orgId=1) and [Kibana](https://kibana.infra.status.im/goto/a7728e70-eb26-11ec-81d1-210eb3022c76).
- If necessary, revert by deploying the previous release. Download logs and open a bug report issue.
33 changes: 16 additions & 17 deletions docs/contributors/waku-fleets.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Background

Status currently maintains two fleets for `nim-waku` v2 nodes,
Status currently maintains two fleets for `nwaku` v2 nodes,
the `wakuv2.test` fleet and the `wakuv2.prod` (production) fleet.
They'll be referred to as `test` and `prod` in this document.
Status fleet nodes and addresses can be viewed [here](https://fleets.status.im/).
Expand All @@ -13,18 +13,17 @@ At the time of writing this, each fleet consists of three waku v2 nodes,
with a [websockify](https://github.com/novnc/websockify) WebSocket-to-TCP bridge for each node.
Waku v2 peers can choose to connect either directly to a node's TCP endpoint
or the bridged WebSocket depending on their own supported transports.
The `prod` fleet also has a deployed [`chat2bridge`](https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md#bridge-messages-between-chat2-and-matterbridge),
The `prod` fleet also has a deployed [`chat2bridge`](https://github.com/waku-org/nwaku/blob/master/docs/tutorial/chat2.md#bridge-messages-between-chat2-and-matterbridge),
which serves as a bridge between the [Waku v2 toy-chat](https://rfc.vac.dev/spec/22/) and Matterbridge.
The `chat2bridge` is currently deployed to the `node-01.do-ams3` datacentre
and configured to bridge toy-chat messages to the `#waku channel` on the Vac Discord Server.

### Fleet deployment rationale

The `test` fleet is automatically updated after every commit to the `nim-waku` `master` branch
and is therefore the most up to date representation of Waku v2 development.
The `test` fleet is automatically updated after every commit to the `nwaku` repository `master` branch and is therefore the most up to date representation of Waku v2 development.
It is suitable for testing new features before they're rolled out to the (more) stable `prod` fleet.

In general only the latest release of `nim-waku` is deployed to the `prod` fleet.
In general only the latest release of `nwaku` is deployed to the `prod` fleet.
It requires manual updating and should therefore be more stable than `test`.
See the [section on Jenkins](#jenkins-for-deployment) below for more on the deployment process.

Expand All @@ -43,11 +42,11 @@ The rest of this document highlights some infra services of specific interest to
1. [Consul](https://consul.infra.status.im/ui/do-ams3/services?filter=nim-waku) to view the health status of Waku nodes.
2. [Kibana](https://kibana.infra.status.im/app/discover#/) to view and filter logs.
3. [Grafana](https://grafana.infra.status.im/d/qrp_ZCTGz/nim-waku-v2) to view and filter metrics.
4. [Jenkins](https://ci.status.im/job/nim-waku/) to configure and deploy new builds to the fleets.
4. [Jenkins](https://ci.infra.status.im/job/nim-waku/) to configure and deploy new builds to the fleets.

### 1. [Consul](https://consul.infra.status.im/ui/do-ams3/services?filter=nim-waku) for health checks

Consul provides a useful high-level view of the health of the `nim-waku` fleets.
Consul provides a useful high-level view of the health of the `nwaku` fleets.
It aggregates the result of various monitoring checks
and shows the health status for the node itself, the RPC API, exposed WebSocket and metrics.
The datacentre can be changed in the upper left-hand corner.
Expand All @@ -72,35 +71,35 @@ with an overview of the latest connected peers, total messages, CPU usage, repor
The _"General"_ collection contains a more in-depth look at node, libp2p and performance-related metrics.
This is followed by separate panel collections showing _per-protocol_ metrics.

A copy of the `Nim-Waku V2` fleets dashboard is maintained in the [`nim-waku` repo](https://github.com/status-im/nim-waku/blob/master/metrics/waku_fleet_dashboard.json).
A copy of the `Nim-Waku V2` fleets dashboard is maintained in the [`nwaku` repo](https://github.com/waku-org/nwaku/blob/master/metrics/waku-fleet-dashboard.json).
From time to time certain Prometheus queries may fail,
often when the underlying metrics are renamed.
Please report any broken panels via our Discord channels or by [creating an issue in `nim-waku`](https://github.com/status-im/nim-waku/issues/new).
Please report any broken panels via our Discord channels or by [creating an issue in `nwaku`](https://github.com/waku-org/nwaku/issues/new).

### 4. [Jenkins](https://ci.status.im/job/nim-waku/) for deployment

The [`nim-waku` jobs](https://ci.status.im/job/nim-waku/) on Jenkins are configured to deploy `nim-waku` builds to the fleets.
1. [`deploy-wakuv2-test`](https://ci.status.im/job/nim-waku/job/deploy-wakuv2-test/) is triggered automatically after every commit to the `nim-waku` `master` branch.
2. [`deploy-wakuv2-prod`](https://ci.status.im/job/nim-waku/job/deploy-wakuv2-prod/) must be triggered manually. Usually this job is only built after a tagged release in `nim-waku`.
The [`nim-waku` jobs](https://ci.infra.status.im/job/nim-waku/) on Jenkins are configured to deploy `nwaku` builds to the fleets.
1. [`deploy-wakuv2-test`](https://ci.infra.status.im/job/nim-waku/job/deploy-wakuv2-test/) is triggered automatically after every commit to the `nwaku` `master` branch.
2. [`deploy-wakuv2-prod`](https://ci.infra.status.im/job/nim-waku/job/deploy-wakuv2-prod/) must be triggered manually. Usually this job is only built after a tagged release in `nwaku`.

Each job can be manually triggered using the _"Build with Parameters"_ option.
Options under _"Configure"_ include the build triggers, build target and branches to build.
These should only be changed with care.

See [Continuous Integration docs](https://github.com/status-im/nim-waku/blob/master/docs/contributors/continuous-integration.md) for more.
See [Continuous Integration docs](https://github.com/waku-org/nwaku/blob/master/docs/contributors/continuous-integration.md) for more.

## Quick links

1. [`chat2bridge`](https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md#bridge-messages-between-chat2-and-matterbridge)
1. [`chat2bridge`](https://github.com/waku-org/nwaku/blob/master/docs/tutorial/chat2.md#bridge-messages-between-chat2-and-matterbridge)
2. [Consul for do-ams3](https://consul.infra.status.im/ui/do-ams3/services?filter=nim-waku)
3. [Consul for ac-cn-hongkong-c](https://consul.infra.status.im/ui/ac-cn-hongkong-c/services?filter=nim-waku)
4. [Consul for gc-us-central1-a](https://consul.infra.status.im/ui/gc-us-central1-a/services?filter=nim-waku)
5. [Grafana Nim-Waku V2 dashboard](https://grafana.infra.status.im/d/qrp_ZCTGz/nim-waku-v2?orgId=1&refresh=5m)
6. [`infra-docs` repo](https://github.com/status-im/infra-docs)
7. [`infra-nim-waku` repo](https://github.com/status-im/infra-nim-waku)
8. [Jenkins jobs for `nim-waku`](https://ci.status.im/job/nim-waku/)
9. [Jenkins deploy-wakuv2-prod manual trigger](https://ci.status.im/job/nim-waku/job/deploy-wakuv2-prod/build)
10. [Jenkins deploy-wakuv2-test manual trigger](https://ci.status.im/job/nim-waku/job/deploy-wakuv2-test/build)
8. [Jenkins jobs for `nim-waku`](https://ci.infra.status.im/job/nim-waku/)
9. [Jenkins deploy-wakuv2-prod manual trigger](https://ci.infra.status.im/job/nim-waku/job/deploy-wakuv2-prod/build)
10. [Jenkins deploy-wakuv2-test manual trigger](https://ci.infra.status.im/job/nim-waku/job/deploy-wakuv2-test/build)
11. [Kibana logs for `prod`](https://kibana.infra.status.im/goto/87fde8e4bba7246ce3780a0c8344f4f0)
12. [Kibana logs for `test`](https://kibana.infra.status.im/goto/fc23759670fd08e9d32e81bb4e58733d)
13. [Status fleets](https://fleets.status.im/)
Expand Down
2 changes: 2 additions & 0 deletions docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ Grep for "Listening on". It should be printed at INFO level at the beginning. E.

`Oct 7, 2020 @ 23:17:00.383INF 2020-10-07 23:17:00.375+00:00 Listening on topics="wakunode" tid=1 file=wakunode2.nim:140 full=/ip4/0.0.0.0/tcp/60000/p2p/16Uiu2HAmJb2e28qLXxT5kZxVUUoJt72EMzNGXB47Rxx5hw3q4YjS`

Or use the [JSON-RPC API](https://github.com/waku-org/nwaku/blob/master/docs/tutorial/jsonrpc-api.md#perform-a-health-check).

## How do I find out node addresses at the test cluster?

The easiest way is to use `jq` and query the fleets registry that Status operates:
Expand Down
6 changes: 3 additions & 3 deletions docs/operators/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,10 @@ and use existing tools for monitoring and maintaining a running node.

## Getting in touch or reporting an issue

For an inquiry, or if you would like to propose new features, feel free to [open a general issue](https://github.com/status-im/nwaku/issues/new/).
For an inquiry, or if you would like to propose new features, feel free to [open a general issue](https://github.com/waku-org/nwaku/issues/new/).

For bug reports, please [tag your issue with the `bug` label](https://github.com/status-im/nwaku/issues/new/).
For bug reports, please [tag your issue with the `bug` label](https://github.com/waku-org/nwaku/issues/new/).

If you believe the reported issue requires critical attention, please [use the `critical` label](https://github.com/status-im/nwaku/issues/new?labels=critical,bug) to assist with triaging.
If you believe the reported issue requires critical attention, please [use the `critical` label](https://github.com/waku-org/nwaku/issues/new?labels=critical,bug) to assist with triaging.

To get help, or participate in the conversation, join the [Vac Discord](https://discord.gg/KNj3ctuZvZ) server.
2 changes: 1 addition & 1 deletion docs/operators/docker-quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ docker pull statusteam/nim-waku:v0.16.0 # or, whichever tag you prefer in the fo
You can also build the Docker image locally using

```bash
git clone --recurse-submodules https://github.com/status-im/nwaku
git clone --recurse-submodules https://github.com/waku-org/nwaku
cd nwaku
docker build -t statusteam/nim-waku:latest .
```
Expand Down
2 changes: 1 addition & 1 deletion docs/operators/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ see our [Docker guide](./docker-quickstart.md).
## 1. Build

[Build the nwaku node](./how-to/build.md)
or download a precompiled binary from our [releases page](https://github.com/status-im/nwaku/releases).
or download a precompiled binary from our [releases page](https://github.com/waku-org/nwaku/releases).
Docker images are published to [statusteam/nim-waku](https://hub.docker.com/r/statusteam/nim-waku/tags) on Docker Hub.
See our [Docker quickstart guide](./docker-quickstart.md) to run nwaku in a Docker container.

Expand Down
2 changes: 1 addition & 1 deletion docs/operators/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ see our [step-by-step guide](./overview.md).
such as a C compiler, Make, Bash and Git.*

```bash
git clone --recurse-submodules https://github.com/status-im/nwaku
git clone --recurse-submodules https://github.com/waku-org/nwaku
cd nwaku
make wakunode2
./build/wakunode2 \
Expand Down
33 changes: 16 additions & 17 deletions docs/tutorial/db-migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,35 +4,34 @@ This tutorial explains the database migration process in nim-waku.
# Contributors Guide
## Database Migration Flow
Nim-waku utilizes the built-in `user_version` variable that Sqlite provides for tracking the database versions.
The [user_version](https://github.com/status-im/nim-waku/blob/master/waku/v2/node/storage/migration/migration_types.nim) MUST be bumped up for every update on the database e.g, table schema/title change.
The [user_version](https://github.com/waku-org/nwaku/blob/master/waku/v2/waku_archive/driver/sqlite_driver/migrations.nim) MUST be bumped up for every update on the database e.g, table schema/title change.
Each update should be accompanied by a migration script to move the content of the old version of the database to the new version.
The script MUST be added to the respective folder as explained in [Migration Folder Structure](#migration-folder-structure) with the proper naming as given in [ Migration Script Naming](#migration-file-naming).

The migration is invoked whenever the database `user_version` is behind the target [user_version](https://github.com/status-im/nim-waku/blob/master/waku/v2/node/storage/migration/migration_types.nim) indicated in the nim-waku application.
The respective migration scripts located in the [migrations folder](https://github.com/status-im/nim-waku/tree/master/waku/v2/node/storage/migration) will be executed to upgrade the database from its old version to the target version.
The migration is invoked whenever the database `user_version` is behind the target [user_version](https://github.com/waku-org/nwaku/blob/master/waku/v2/waku_archive/driver/sqlite_driver/migrations.nim) indicated in the nim-waku application.
The respective migration scripts located in the [migrations folder](https://github.com/waku-org/nwaku/tree/master/migrations) will be executed to upgrade the database from its old version to the target version.

## Migration Folder Structure
The [migrations folder](https://github.com/status-im/nim-waku/tree/master/waku/v2/node/storage/migration) is structured as below.
The [migrations folder](https://github.com/waku-org/nwaku/tree/master/migrations) is structured as below.

```
|-- migration
| |--migration_scripts
| | |--message
| | | |--00001_basicMessageTable.up.sql
| | | |--00002_addSenderTimeStamp.up
| | | |-- ...
| | |--peer
| | | |--00001_basicPeerTable.up.sql
| | | |-- ...
migrations/
├── message_store
│   ├── 00001_addMessageTable.up.sql
│   ├── 00002_addSenderTimeStamp.up.sql
│   ├── ...
└── peer_store
└── 00001_addPeerTable.up.sql
```

The migration scripts are managed in two separate folders `message` and `peer`.
The `message` folder contains the migration scripts related to the message store tables.
Similarly, the `peer` folder contains the scripts relevant to the peer store tables.


The migration scripts are managed in two separate folders `message_store` and `peer_store`.
The `message_store` folder contains the migration scripts related to the message store tables. Similarly, the `peer_store` folder contains the scripts relevant to the peer store tables.


## Migration File Naming
The files in [migrations folder](https://github.com/status-im/nim-waku/tree/master/waku/v2/node/storage/migration) MUST follow the following naming style in order to be properly included in the migration process.
The files in [migrations folder](https://github.com/waku-org/nwaku/tree/master/migrations) MUST follow the following naming style in order to be properly included in the migration process.
Files with invalid naming will be eliminated from the migration process.

`<version number>_<migration script description>.<up|down>.sql`
Expand Down
4 changes: 3 additions & 1 deletion docs/tutorial/dingpu.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Dingpu testnet

> TODO (2023-05-24): Deprecate or fix
*NOTE: Some of these addresses might change. To get the latest, please see `curl -s https://fleets.status.im | jq '.fleets["wakuv2.test"]'`*

## Basic chat usage
Expand All @@ -19,7 +21,7 @@ Then type messages to publish.

## Interactively add a node

There is also an interactive mode. Type `/connect` then paste address of other node. However, this currently has some timing issues with mesh not being updated, so it is adviced not to use this until this has been addressed. See https://github.com/status-im/nim-waku/issues/231
There is also an interactive mode. Type `/connect` then paste address of other node. However, this currently has some timing issues with mesh not being updated, so it is adviced not to use this until this has been addressed. See https://github.com/waku-org/nwaku/issues/231

## Dingpu cluster node

Expand Down
2 changes: 2 additions & 0 deletions docs/tutorial/filter.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Running Filter Protocol

> TODO (2023-05-24): Deprecate or fix
## How to

Build:
Expand Down
Loading

0 comments on commit b51fb61

Please sign in to comment.