Skip to content

Commit

Permalink
chore: updates after transfer (#11)
Browse files Browse the repository at this point in the history
* chore: updates after transfer

* docs: update README
  • Loading branch information
aceforeverd authored Aug 9, 2023
1 parent ba43a8d commit 952fce8
Show file tree
Hide file tree
Showing 5 changed files with 100 additions and 93 deletions.
2 changes: 1 addition & 1 deletion .env
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
OPENMLDB_VERSION=0.8.1
OPENMLDB_VERSION=0.8.2
ENV_TYPE=src
9 changes: 9 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,14 @@ updates:
# Maintain dependencies for GitHub Actions
- package-ecosystem: "github-actions"
directory: "/"
reviewers:
- "aceforeverd"
schedule:
interval: "daily"

- package-ecosystem: "pip"
directory: "/"
reviewers:
- "aceforeverd"
schedule:
interval: "daily"
178 changes: 88 additions & 90 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
# OpenMLDB Prometheus Exporter
# OpenMLDB Exporter

[![PyPI](https://img.shields.io/pypi/v/openmldb-exporter?label=openmldb-exporter)](https://pypi.org/project/openmldb-exporter/)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/openmldb-exporter)

## Intro
## Features

This directory contains

1. OpenMLDB Exporter exposing prometheus metrics
2. OpenMLDB mixin provides well-configured examples for prometheus server and grafana dashboard
- [OpenMLDB](https://github.com/4paradigm/OpenMLDB) Prometheus Exporter exposing metrics
- [OpenMLDB](https://github.com/4paradigm/OpenMLDB) mixin provides well-configured examples for [Prometheus](https://prometheus.io/) server and [Grafana](https://grafana.com/) dashboard

## Requirements

Expand All @@ -17,128 +15,128 @@ This directory contains
- Python >= 3.8


## Setup
## Quick Start

For those want to try, simply install from pip, it will install the latest release:
**You can run openmdlb-exporter from docker, or install and run directly from PyPI.**

```bash
pip install openmldb-exporter
```

Then type
<details open=true><summary>Use docker</summary>

```sh
openmldb-exporter -h
docker run ghcr.io/4paradigm/openmldb-exporter \
--config.zk_root=<openmldb_zk_addr> \
--config.zk_path=<openmldb_zk_path>
```

To see which flags should provided.
</details>

Developers may refer [Development](#development) for how to setup openmldb exporter from source code.
<details open=true><summary>Install and Run from PyPI</summary>

```sh
pip install openmldb-exporter

## Development
# start
openmldb-exporter \
--config.zk_root=<openmldb_zk_addr> \
--config.zk_path=<openmldb_zk_path>
```
</details></br>

### Extra Requirements
And replace `<openmdlb_zk_addr>` and `<openmldb_zk_path>` to correct value. Afterwards, you can check metrics with curl:

- Same in [Requirements](#requirements)
- [poetry](https://github.com/python-poetry/poetry) as build tool
- cmake (optional if want to test latest commit)
```sh
curl http://<IP>:8000/metrics
```
`<IP>` is docker container IP, or `127.0.0.1` if installing from PyPI.

<details><summary>Example output</summary>

```sh
# HELP openmldb_connected_seconds_total duration for a component conncted time in seconds
# TYPE openmldb_connected_seconds_total counter
openmldb_connected_seconds_total{endpoint="172.17.0.15:9520",role="tablet"} 208834.70900011063
openmldb_connected_seconds_total{endpoint="172.17.0.15:9521",role="tablet"} 208834.70700001717
openmldb_connected_seconds_total{endpoint="172.17.0.15:9522",role="tablet"} 208834.71399998665
openmldb_connected_seconds_total{endpoint="172.17.0.15:9622",role="nameserver"} 208833.70000004768
openmldb_connected_seconds_total{endpoint="172.17.0.15:9623",role="nameserver"} 208831.70900011063
openmldb_connected_seconds_total{endpoint="172.17.0.15:9624",role="nameserver"} 208829.7230000496
# HELP openmldb_connected_seconds_created duration for a component conncted time in seconds
# TYPE openmldb_connected_seconds_created gauge
openmldb_connected_seconds_created{endpoint="172.17.0.15:9520",role="tablet"} 1.6501813860467942e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9521",role="tablet"} 1.6501813860495396e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9522",role="tablet"} 1.650181386050323e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9622",role="nameserver"} 1.6501813860512116e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9623",role="nameserver"} 1.650181386051238e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9624",role="nameserver"} 1.6501813860512598e+09
```

### Build python SDK (Optional)
</details>

openmldb exporter depends on [openmldb python SDK](https://pypi.org/project/openmldb/). For those introducing changes in python SDK or native code as well, this steps is required in order to take the latest Python SDK locally instead of the published one.
## Configuration

#### 1. Build native code
You can view the help from:
```sh
openmldb-exporter -h
```
`--config.zk_root` and `--config.zk_path` are mandatory.

`cd` to root directory of OpenMLDB, and run:
<details><summary>Available options</summary>

```bash
make SQL_PYSDK_ENABLE=ON
```
usage: openmldb-exporter [-h] [--log.level LOG.LEVEL] [--web.listen-address WEB.LISTEN_ADDRESS]
[--web.telemetry-path WEB.TELEMETRY_PATH] [--config.zk_root CONFIG.ZK_ROOT]
[--config.zk_path CONFIG.ZK_PATH] [--config.interval CONFIG.INTERVAL]
OpenMLDB exporter
optional arguments:
-h, --help show this help message and exit
--log.level LOG.LEVEL
config log level, default WARN
--web.listen-address WEB.LISTEN_ADDRESS
process listen port, default 8000
--web.telemetry-path WEB.TELEMETRY_PATH
Path under which to expose metrics, default metrics
--config.zk_root CONFIG.ZK_ROOT
endpoint to zookeeper, default 127.0.0.1:6181
--config.zk_path CONFIG.ZK_PATH
root path in zookeeper for OpenMLDB, default /
--config.interval CONFIG.INTERVAL
interval in seconds to pull metrics periodically, default 30.0
```

This will generate compiled shared library in `python` source directory.
</details>

#### 2. Fix dependency list
## Development

Back to openmldb exporter directory, modify `pyproject.toml` and make content like below:
### Extra Requirements

```toml
[tool.poetry.dependencies]
# ...
# openmldb = "^0.6.0"
# uncomment below to use openmldb sdk built from source
# set develop = true so changes in python will take effect immediately
openmldb = { path = "../python/", develop = true }
```
- Same in [Requirements](#requirements)
- [poetry](https://github.com/python-poetry/poetry) as build tool

### Run

1. Setup python dependencies:

```bash
```sh
poetry install
```

2. Start openmldb exporter

```bash
```sh
poetry run openmldb-exporter
```

You need pass necessary flags after `openmldb-exporter`. Run `poetry run openmldb-exporter --help` to get the help info

```bash
usage: openmldb-exporter [-h] [--log.level LOG.LEVEL] [--web.listen-address WEB.LISTEN_ADDRESS]
[--web.telemetry-path WEB.TELEMETRY_PATH] [--config.zk_root CONFIG.ZK_ROOT]
[--config.zk_path CONFIG.ZK_PATH] [--config.interval CONFIG.INTERVAL]
OpenMLDB exporter
optional arguments:
-h, --help show this help message and exit
--log.level LOG.LEVEL
config log level, default WARN
--web.listen-address WEB.LISTEN_ADDRESS
process listen port, default 8000
--web.telemetry-path WEB.TELEMETRY_PATH
Path under which to expose metrics, default metrics
--config.zk_root CONFIG.ZK_ROOT
endpoint to zookeeper, default 127.0.0.1:6181
--config.zk_path CONFIG.ZK_PATH
root path in zookeeper for OpenMLDB, default /
--config.interval CONFIG.INTERVAL
interval in seconds to pull metrics periodically, default 30.0
```

3. View the available metrics, you can pull through `curl`

```bash
curl http://127.0.0.1:8000/metrics
```
Pass in necessary flags after `openmldb-exporter`. Run `poetry run openmldb-exporter --help` to get the help info.

A example output:

```bash
# HELP openmldb_connected_seconds_total duration for a component conncted time in seconds
# TYPE openmldb_connected_seconds_total counter
openmldb_connected_seconds_total{endpoint="172.17.0.15:9520",role="tablet"} 208834.70900011063
openmldb_connected_seconds_total{endpoint="172.17.0.15:9521",role="tablet"} 208834.70700001717
openmldb_connected_seconds_total{endpoint="172.17.0.15:9522",role="tablet"} 208834.71399998665
openmldb_connected_seconds_total{endpoint="172.17.0.15:9622",role="nameserver"} 208833.70000004768
openmldb_connected_seconds_total{endpoint="172.17.0.15:9623",role="nameserver"} 208831.70900011063
openmldb_connected_seconds_total{endpoint="172.17.0.15:9624",role="nameserver"} 208829.7230000496
# HELP openmldb_connected_seconds_created duration for a component conncted time in seconds
# TYPE openmldb_connected_seconds_created gauge
openmldb_connected_seconds_created{endpoint="172.17.0.15:9520",role="tablet"} 1.6501813860467942e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9521",role="tablet"} 1.6501813860495396e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9522",role="tablet"} 1.650181386050323e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9622",role="nameserver"} 1.6501813860512116e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9623",role="nameserver"} 1.650181386051238e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9624",role="nameserver"} 1.6501813860512598e+09
```

## Release History

- 0.8.0
* Features
- Upgrade OpenMLDB SDK to v0.8
- improve test code

- 0.7.1
* Features
- Upgrade OpenMLDB SDK to v0.7
Expand Down
2 changes: 1 addition & 1 deletion prod.env
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
OPENMLDB_VERSION=0.8.1
OPENMLDB_VERSION=0.8.2
ENV_TYPE=prod
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ authors = ["aceforeverd <[email protected]>"]
license = "Apache-2.0"
readme = "README.md"
homepage = "https://openmldb.ai"
repository = "https://github.com/4paradigm/OpenMLDB"
repository = "https://github.com/4paradigm/openmldb-exporter"
documentation = "https://openmldb.ai/docs/zh/main/maintain/monitoring.html"
keywords = ["openmldb", "prometheus"]
classifiers = [
Expand Down

0 comments on commit 952fce8

Please sign in to comment.