v3.5 Concurrency Model

Contents

1. Approach
    1.1. Service Tasks Execution
    1.2. I/O Tasks Execution
     1.2.1. NIO storage drivers
     1.2.2. Netty-based storage drivers
   
2. Actual Concurrency Measurement
    2.1. Requirements
    2.2. Limitations
    2.3. Configuration
    2.4. Reporting

1. Approach

Mongoose is required to execute some types of the tasks concurrently:

1. Calculate the values defined by a pattern in the configuration
2. Generate the I/O tasks
3. Distribute the generated I/O tasks among the storage drivers uniformly
4. Storage drivers dispatch the incoming I/O tasks
5. Storage drivers perform the file/socket I/O
6. Load Controller collects the actual concurrency measurements from the storage drivers
7. Load Controller collects the completed I/O task results from the storage drivers
8. Refresh the registered metrics and their snapshots

The count of the tasks which are required to be executed concurrently may be large (for example in a case of distributed mode with many remote storage drivers). The traditional thread-per-task approach is inefficient if the count of the threads is much higher than the count of the available CPU cores. The coroutines are used in Mongoose to execute the required work efficiently.

1.1. Service Tasks Execution

Each Mongoose process shares the global service tasks executor (CoroutineProcessor instance). By default the count of the service tasks executor's threads is equal to the count of the available CPU cores. An user may set the different count of these threads using the configuration option load-service-threads.

1.2. I/O Tasks Execution

1.2.1. NIO storage drivers

The NIO storage driver derivatives (FS storage driver, for example) are using the service tasks executor to perform the I/O on the storage. The count of the concurrent I/O tasks at is limited also by the storage-driver-threads configuration option which is CPU core count by default. This means that I/O tasks will occupy no more load service tasks executor threads than the configured. However the I/O tasks are reentrant and the count of the active (started but not completed yet) I/O tasks may be much more (may be limited by load-limit-concurrency) configuration option).

1.2.2. Netty-based storage drivers

Netty-based storage driver derivatives are using separate worker threads pool which is reused by all netty-based storage driver instances in a process. The count of these threads is controlled by the storage-driver-threads configuration option.

2. Actual Concurrency Measurement

Usually Mongoose is used to measure the rates/timings with given concurrency limit value. By default Mongoose's concurrency limit is 1. But it may be very useful to allow Mongoose to work with unlimited (as high as possible) concurrency level (concurrent connections, open files). Unlimited here doesn't mean infinite, because there are OS limits and the internal number representation size limit (32-bit signed integer).

2.1. Requirements

1. Allow to disable the concurrency limit.
2. Measure and report the actual concurrency level.

2.2. Limitations

1. ulimit -n system setting (usually set to 1024). The OS will not allow to use more simultaneous connections than count. Set that system setting value to higher value if a higher concurrency level is required.

2. It's not possible to set ulimit -n value to more than 1048576 on the most Linux systems. So single process can't use higher count of the simultaneous connections even if the ulimit -n is set to maximum possible value (1048576). Use distributed mode if a higher concurrency level is required.

3. 32-bit signed integer limit. It's not possible to reach concurrency level more than 2,147,483,647. In practice, such high concurrency level is considered as unreachable and not useful for the performance measurement purposes (this will require also the distributed mode using at least 2048 separate storage drivers).

2.3. Configuration

To disable the concurrency limit it's enough to set the corresponding configuration parameter to 0:

java -jar mongoose-<VER>/mongoose.jar --load-limit-concurrency=0

Also, it should be useful to limit the load rate:

java -jar mongoose-<VER>/mongoose.jar \
    --load-limit-concurrency=0 \
    --load-limit-rate=1000

The last example will show how many simultaneous clients/users/connections can handle the service/storage under the test.

2.4. Reporting

The actual concurrency is reported as metric. See Metrics Output documentation for details.