Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Toml/doc Operation and Maintenance update #2899

Merged
merged 7 commits into from
Oct 12, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions doc/modules/mod_muc_light.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,10 +115,10 @@ When enabled, rooms the user occupies are included in their roster.
Allowed `config_schema` items are (may be mixed):

* Field name and a default value. The value has to be a string. An example:
```
field = "field_name"
value = "default_value"
```

field = "field_name"
value = "default_value"

* Field name, a default value, an internal key representation string and a type.
Valid config field types are:

Expand All @@ -127,18 +127,18 @@ Valid config field types are:
* `float`

Useful only for debugging or custom applications. An example:
```
field = "display-lines"
value = 30
internal_key = "display_lines"
type = "integer"
```

field = "display-lines"
value = 30
internal_key = "display_lines"
type = "integer"

**WARNING!** Lack of the `roomname` field will cause room names in Disco results and Roster items be set to the room username.


### Example Configuration

```
```toml
[modules.mod_muc_light]
host = "muclight.example.com"
equal_occupants = true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,18 +23,20 @@ If you are using **Ubuntu**, all `/etc/pam.d/common-session*` files should inclu

### `vm.args` file

This file contains erlang options used when starting the VM.
It is located in `REL_ROOT/etc/vm.args` where `REL_ROOT` is the path to a MonoogseIM release
chrzaszcz marked this conversation as resolved.
Show resolved Hide resolved
This file contains Erlang options used when starting the VM.
It is located in `REL_ROOT/etc/vm.args` where `REL_ROOT` is the path to a MongooseIM release
(ie. `_build/prod/rel/mongooseim` if you build MongooseIM from source).

When using an SSL/TLS connection we advise to increase `ERL_MAX_PORTS` to `350000`.
This value specifies how many ports (files, drivers, sockets etc) can be used by Erlang VM.
This value specifies how many ports (files, drivers, sockets etc) can be used by the Erlang VM.
Be cautious - it preallocates some structures inside the VM and will have impact on the memory usage.
We suggest 350000 for 100 k users when using an SSL/TLS connection or 250000 in other cases.

To check how memory consumption changes depending on `ERL_MAX_PORTS`, use the following command:

`env ERL_MAX_PORTS=[given value] erl -noinput -eval 'io:format("~p~n",[erlang:memory(system)]).' -s erlang halt`
```bash
env ERL_MAX_PORTS=[given value] erl -noinput -eval 'io:format("~p~n",[erlang:memory(system)]).' -s erlang halt
```

Another change you need to make when building a MongooseIM cluster is setting the `-sname`.
To do it, just set the `-sname` option in `vm.args` with node's hostname,
Expand All @@ -46,12 +48,14 @@ To connect to other nodes, a freshly started node uses a port from the range `in

To enable this, add the following line to the `vm.args` file:

`-kernel inet_dist_listen_min 50000 inet_dist_listen_max 50010`
```
-kernel inet_dist_listen_min 50000 inet_dist_listen_max 50010
```

Make sure that the range you set provides enough ports for all the nodes in the cluster.

Remember to keep an epmd port open (port 4369) if any firewall restrictions are required.
Epmd keeps track of which erlang node is using which ports on the local machine.
Epmd keeps track of which Erlang node is using which ports on the local machine.

## Connecting nodes

Expand Down Expand Up @@ -80,16 +84,16 @@ mongooseimctl join_cluster ClusterMember
`ClusterMember` is the name of a running node set in `vm.args` file, for example `mongooseim@localhost`.
This node has to be part of the cluster we'd like to join.

First MongooseIM will display a warning and a question if the operation should proceed:
First, MongooseIM will display a warning and a question if the operation should proceed:

```
```text
Warning. This will drop all current connections and will discard all persistent data from Mnesia. Do you want to continue? (yes/no)
```

If you type `yes` MongooseIM will start joining the cluster.
Successful output may look like the following:

```
```text
You have successfully joined the node mongooseim2@localhost to the cluster with node member mongooseim@localhost
```

Expand All @@ -110,7 +114,9 @@ Similarly to `join_cluster` a warning and a question will be displayed unless th

The successful output from the above command may look like the following:

`You have successfully left the node mongooseim2@localhost from the cluster`.
```text
The node mongooseim2@localhost has successfully left the cluster
```

### Removing a node from the cluster

Expand All @@ -124,7 +130,9 @@ where `RemoteNodeName` is a name of the node that we'd like to remove from our c
This command could be useful when the node is dead and not responding and we'd like to remove it remotely.
The successful output from the above command may look like the following:

`The node mongooseim2@localhost has been removed from the cluster`
```text
The node mongooseim2@localhost has been removed from the cluster
```

### Cluster status

Expand Down Expand Up @@ -167,11 +175,8 @@ Load balancing can be performed on a DNS level.
A DNS response can have a number of IP addresses that can be returned to the client side in a random order.

On the AWS stack this type of balancing is provided by Route53.
The description of their service can be found at:

http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/WeightedResourceRecordSets.html
The description of their service can be found in the [Route53 Developer's Guide](http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/WeightedResourceRecordSets.html).

### Other

The approaches described above can be mixed - we can use DNS load balancing to pick a software load balancer which will select one of the nodes.

Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
This applies to bare metal, virtualization, hypervisor, containers and other technologies.
These apply to bare metal, virtualization, hypervisor, containers and other technologies.

## Single-node MongooseIM

With a single-node MongooseIM, one can set up a vertically scalable system, that is a function of the server resources. MongooseIM can scale from hundreds to tens of thousands of concurrent users.

Note that in a single-node MongooseIM, there is no load distribution, and no fallback or failover in case of failure.
Note that in a single-node MongooseIM, there is no load distribution, and no fallback or failover in case of a failure.

This architecture is suitable for low-scale deployments, such as testing and development environments on embedded devices, personal computers, or servers.

Expand Down Expand Up @@ -38,14 +38,14 @@ With a multi-datacenter MongooseIM, one can set up a system that is highly scala

This applies to large and very large-scale deployments.

Contact us.
We advise [contacting us](https://www.erlang-solutions.com/contact.html) in case of such a big deployment.

## Summary table

**Setup**: reflects the number of nodes in your cluster. <br/>
**Purpose**: is the goal and use of this cluster. <br/>
**Low-end**: number of concurent users on low-power machines, such as laptops, embedded devides, entry-level cloud or bare metal.<br/>
**High-end**: number of concurent users on powerful machines, with lots of memory, multi-core CPU, whether they or cloud or bare metal.
**Low-end**: number of concurrent users on low-power machines, such as laptops, embedded devices, entry-level cloud or bare metal.<br/>
**High-end**: number of concurrent users on powerful machines, with lots of memory, multi-core CPU, whether in cloud or bare metal.

Setup | Purpose | Low-end | High-end
------|---------|---------:|---------:
Expand All @@ -70,7 +70,7 @@ Scalability highly depends on variables such as:
* MUC light
* PubSub
* Presences
* HTTP notifications (with queuing systems such as RabbitMQ or Kafka)
* HTTP notifications (may include queuing systems such as RabbitMQ or Kafka)
* latency of messaging, both real-time and archived messages

#### OS configuration
Expand Down
16 changes: 8 additions & 8 deletions doc/operation-and-maintenance/Logging-&-monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ To activate it you have to add `syslogger` to the applications section in `src/m

%% syslogger, % uncomment to enable a logger handler for syslog

You also need to edit `rel/files/app.config` and uncomment the line:
You also need to edit `rel/files/app.config` and uncomment the lines:

% Uncomment these lines to enable logging to syslog.
% Remember to add syslogger as a dependency in mongooseim.app.src.
Expand All @@ -30,7 +30,7 @@ in the `syslogger's` [GitHub page](https://github.com/NelsonVides/syslogger/):
* `facility` - the facility to log to (see the syslog documentation).
* `log_opts` - see the syslog documentation for the description.

Depending on the system platform you use, remember also to add the appropriate line in the syslog config file.
Depending on the system you use, remember to also add the appropriate line in the syslog config file.
For example, if the facility `local0` is set:

local0.info /var/log/mongooseim.log
Expand All @@ -45,7 +45,7 @@ Example log (e.g `tail -f /var/log/mongooseim.log`):

For more advanced processing and analysis of logs, including gathering logs from multiple machines,
you can use one of the many available systems (e.g. logstash/elasticsearch/kibana, graylog, splunk),
by redirecting mongoose logs to such service with an appropriate [Logger](Logger)'s handler.
by redirecting mongoose logs to such service with an appropriate [Logger][Logger]'s handler.

Check [Logging](Logging.md) for more information.

Expand All @@ -71,8 +71,8 @@ To monitor MongooseIM during load testing, we recommend the following open sourc

MongooseIM uses [a fork of Exometer library](https://github.com/esl/exometer_core) for collecting metrics.
Exometer has many plug-in reporters that can send metrics to external services. We maintain [exometer_report_graphite](https://github.com/esl/exometer_report_graphite) and [exometer_report_statsd](https://github.com/esl/exometer_report_statsd) for Graphite and StatsD respectively.
It is possible to enable them in MoongooseIM via the `app.config` file.
The file sits next to the `mongooseim.cfg` file in the `rel/files` and `_REL_DIR_/etc` directories.
It is possible to enable them in MongooseIM via the `app.config` file.
The file sits next to the `mongooseim.toml` file in the `rel/files` and `_REL_DIR_/etc` directories.

Below you can find a sample configuration.
It shows setting up a reporter connecting to graphite running on localhost.
Expand Down Expand Up @@ -101,16 +101,16 @@ By default, the resolution is set to 60 seconds.

### Run Graphite & Grafana in Docker - quick start

The following commands will download the latest version of `kamon/grafana_graphite` docker image that contains both Grafana and Graphite, and start them while mounting local directory `./docker-grafana-graphite-master/data` for metrics persistance:
The following commands will download the latest version of `kamon/grafana_graphite` docker image that contains both Grafana and Graphite, and start them while mounting the local directory `./docker-grafana-graphite-master/data` for metric persistence:

$ curl -SL https://github.com/kamon-io/docker-grafana-graphite/archive/master.tar.gz | tar -xzf -
$ make -C docker-grafana-graphite-master up

Go to http://localhost to view Grafana dashboard that's already set up to use metrics from Graphite.
Go to http://localhost:80 to view the Grafana dashboard that's already set up to use metrics from Graphite.

### Add metrics to Grafana dashboard

We recommend the following metrics as a baseling for tracking your MongooseIM installation.
We recommend the following metrics as a baseline for tracking your MongooseIM installation.
For time-based metrics, you can choose to display multiple calculated values for a reporting period - we recommend tracking at least `max`, `median` and `mean`.

```
Expand Down
12 changes: 2 additions & 10 deletions doc/operation-and-maintenance/Logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
The main configuration for logging is in the Application Config file.
You can find it in `mongooseim/etc/app.config` in the release directory.


# Primary log level

Primary log level sets maximum log level in the system.
Expand All @@ -20,8 +19,7 @@ Primary log level, that is used before MongooseIM config is loaded:
].
```

Once MongooseIM config is loaded, `loglevel` option is used instead.

Once MongooseIM config is loaded, the [`loglevel`](../advanced-configuration/general.md#generalloglevel) option from `mongooseim.toml` is used instead.

# Primary filters

Expand All @@ -48,8 +46,7 @@ unless you are planning to extend the filtering logic.
```

`preserve_acc_filter` filter is disabled by default, but could be enabled,
if you are interested in debugging the accumulator logic (`mongoose_acc` module).

if you are interested in debugging the accumulator logic (see the `mongoose_acc` module).

# Shell log handler

Expand All @@ -68,7 +65,6 @@ if you are interested in debugging the accumulator logic (`mongoose_acc` module)
}},
```


# File log handler

- Controls what and how MongooseIM prints into files.
Expand Down Expand Up @@ -102,7 +98,6 @@ if you are interested in debugging the accumulator logic (`mongoose_acc` module)
}},
```


# Logfmt file log handler

Wrapper around the [flatlog](https://github.com/ferd/flatlog) library with
Expand All @@ -125,7 +120,6 @@ Options:
}}
```


# JSON file log handler

JSON formatted file. It could be used to store messages in ELK or in Splunk.
Expand All @@ -152,7 +146,6 @@ Options:
}}
```


# Different log level for a specific module

Motivation:
Expand All @@ -174,7 +167,6 @@ Changes:
{module_level, debug, [ejabberd_c2s]},
```


# Separate log for module debugging

Motivation:
Expand Down
4 changes: 2 additions & 2 deletions doc/operation-and-maintenance/Mongoose-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ It is actually a one-element proplist: `[{value, N}]`.

### `gauge`

It is similiar to a `value` type but consists of two properties:
It is similar to a `value` type but consists of two properties:

* `value`
* `ms_since_reset` - Time in milliseconds elapsed from the last metric update.
Expand Down Expand Up @@ -195,7 +195,7 @@ These are **total** times of respective operations.
One operation usually requires only a single call to an auth backend but sometimes with e.g. 3 backends configured, the operation may fail for first 2 backends.
In such case, these metrics will be updated with combined time of 2 failed and 1 successful request.

Additionaly, RDBMS layer in MongooseIM exposes two more metrics, if RDBMS is configured:
Additionally, the RDBMS layer in MongooseIM exposes two more metrics, if RDBMS is configured:

* `[global, backends, mongoose_rdbms, query]` - Execution time of a "simple"" (not prepared) query by a DB driver.
* `[global, backends, mongoose_rdbms, execute]` - Execution time of a prepared query by a DB driver.
Loading