Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: updates after transfer #11

Merged
merged 2 commits into from
Aug 9, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
OPENMLDB_VERSION=0.8.1
OPENMLDB_VERSION=0.8.2
ENV_TYPE=src
9 changes: 9 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,14 @@ updates:
# Maintain dependencies for GitHub Actions
- package-ecosystem: "github-actions"
directory: "/"
reviewers:
- "aceforeverd"
schedule:
interval: "daily"

- package-ecosystem: "pip"
directory: "/"
reviewers:
- "aceforeverd"
schedule:
interval: "daily"
178 changes: 88 additions & 90 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
# OpenMLDB Prometheus Exporter
# OpenMLDB Exporter

[![PyPI](https://img.shields.io/pypi/v/openmldb-exporter?label=openmldb-exporter)](https://pypi.org/project/openmldb-exporter/)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/openmldb-exporter)

## Intro
## Features

This directory contains

1. OpenMLDB Exporter exposing prometheus metrics
2. OpenMLDB mixin provides well-configured examples for prometheus server and grafana dashboard
- [OpenMLDB](https://github.com/4paradigm/OpenMLDB) Prometheus Exporter exposing metrics
- [OpenMLDB](https://github.com/4paradigm/OpenMLDB) mixin provides well-configured examples for [Prometheus](https://prometheus.io/) server and [Grafana](https://grafana.com/) dashboard

## Requirements

Expand All @@ -17,128 +15,128 @@ This directory contains
- Python >= 3.8


## Setup
## Quick Start

For those want to try, simply install from pip, it will install the latest release:
**You can run openmdlb-exporter from docker, or install and run directly from PyPI.**

```bash
pip install openmldb-exporter
```

Then type
<details open=true><summary>Use docker</summary>

```sh
openmldb-exporter -h
docker run ghcr.io/4paradigm/openmldb-exporter \
--config.zk_root=<openmldb_zk_addr> \
--config.zk_path=<openmldb_zk_path>
```

To see which flags should provided.
</details>

Developers may refer [Development](#development) for how to setup openmldb exporter from source code.
<details open=true><summary>Install and Run from PyPI</summary>

```sh
pip install openmldb-exporter

## Development
# start
openmldb-exporter \
--config.zk_root=<openmldb_zk_addr> \
--config.zk_path=<openmldb_zk_path>
```
</details></br>

### Extra Requirements
And replace `<openmdlb_zk_addr>` and `<openmldb_zk_path>` to correct value. Afterwards, you can check metrics with curl:

- Same in [Requirements](#requirements)
- [poetry](https://github.com/python-poetry/poetry) as build tool
- cmake (optional if want to test latest commit)
```sh
curl http://<IP>:8000/metrics
```
`<IP>` is docker container IP, or `127.0.0.1` if installing from PyPI.

<details><summary>Example output</summary>

```sh
# HELP openmldb_connected_seconds_total duration for a component conncted time in seconds
# TYPE openmldb_connected_seconds_total counter
openmldb_connected_seconds_total{endpoint="172.17.0.15:9520",role="tablet"} 208834.70900011063
openmldb_connected_seconds_total{endpoint="172.17.0.15:9521",role="tablet"} 208834.70700001717
openmldb_connected_seconds_total{endpoint="172.17.0.15:9522",role="tablet"} 208834.71399998665
openmldb_connected_seconds_total{endpoint="172.17.0.15:9622",role="nameserver"} 208833.70000004768
openmldb_connected_seconds_total{endpoint="172.17.0.15:9623",role="nameserver"} 208831.70900011063
openmldb_connected_seconds_total{endpoint="172.17.0.15:9624",role="nameserver"} 208829.7230000496
# HELP openmldb_connected_seconds_created duration for a component conncted time in seconds
# TYPE openmldb_connected_seconds_created gauge
openmldb_connected_seconds_created{endpoint="172.17.0.15:9520",role="tablet"} 1.6501813860467942e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9521",role="tablet"} 1.6501813860495396e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9522",role="tablet"} 1.650181386050323e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9622",role="nameserver"} 1.6501813860512116e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9623",role="nameserver"} 1.650181386051238e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9624",role="nameserver"} 1.6501813860512598e+09
```

### Build python SDK (Optional)
</details>

openmldb exporter depends on [openmldb python SDK](https://pypi.org/project/openmldb/). For those introducing changes in python SDK or native code as well, this steps is required in order to take the latest Python SDK locally instead of the published one.
## Configuration

#### 1. Build native code
You can view the help from:
```sh
openmldb-exporter -h
```
`--config.zk_root` and `--config.zk_path` are mandatory.

`cd` to root directory of OpenMLDB, and run:
<details><summary>Available options</summary>

```bash
make SQL_PYSDK_ENABLE=ON
```
usage: openmldb-exporter [-h] [--log.level LOG.LEVEL] [--web.listen-address WEB.LISTEN_ADDRESS]
[--web.telemetry-path WEB.TELEMETRY_PATH] [--config.zk_root CONFIG.ZK_ROOT]
[--config.zk_path CONFIG.ZK_PATH] [--config.interval CONFIG.INTERVAL]

OpenMLDB exporter

optional arguments:
-h, --help show this help message and exit
--log.level LOG.LEVEL
config log level, default WARN
--web.listen-address WEB.LISTEN_ADDRESS
process listen port, default 8000
--web.telemetry-path WEB.TELEMETRY_PATH
Path under which to expose metrics, default metrics
--config.zk_root CONFIG.ZK_ROOT
endpoint to zookeeper, default 127.0.0.1:6181
--config.zk_path CONFIG.ZK_PATH
root path in zookeeper for OpenMLDB, default /
--config.interval CONFIG.INTERVAL
interval in seconds to pull metrics periodically, default 30.0
```

This will generate compiled shared library in `python` source directory.
</details>

#### 2. Fix dependency list
## Development

Back to openmldb exporter directory, modify `pyproject.toml` and make content like below:
### Extra Requirements

```toml
[tool.poetry.dependencies]
# ...
# openmldb = "^0.6.0"
# uncomment below to use openmldb sdk built from source
# set develop = true so changes in python will take effect immediately
openmldb = { path = "../python/", develop = true }
```
- Same in [Requirements](#requirements)
- [poetry](https://github.com/python-poetry/poetry) as build tool

### Run

1. Setup python dependencies:

```bash
```sh
poetry install
```

2. Start openmldb exporter

```bash
```sh
poetry run openmldb-exporter
```

You need pass necessary flags after `openmldb-exporter`. Run `poetry run openmldb-exporter --help` to get the help info

```bash
usage: openmldb-exporter [-h] [--log.level LOG.LEVEL] [--web.listen-address WEB.LISTEN_ADDRESS]
[--web.telemetry-path WEB.TELEMETRY_PATH] [--config.zk_root CONFIG.ZK_ROOT]
[--config.zk_path CONFIG.ZK_PATH] [--config.interval CONFIG.INTERVAL]

OpenMLDB exporter

optional arguments:
-h, --help show this help message and exit
--log.level LOG.LEVEL
config log level, default WARN
--web.listen-address WEB.LISTEN_ADDRESS
process listen port, default 8000
--web.telemetry-path WEB.TELEMETRY_PATH
Path under which to expose metrics, default metrics
--config.zk_root CONFIG.ZK_ROOT
endpoint to zookeeper, default 127.0.0.1:6181
--config.zk_path CONFIG.ZK_PATH
root path in zookeeper for OpenMLDB, default /
--config.interval CONFIG.INTERVAL
interval in seconds to pull metrics periodically, default 30.0
```

3. View the available metrics, you can pull through `curl`

```bash
curl http://127.0.0.1:8000/metrics
```
Pass in necessary flags after `openmldb-exporter`. Run `poetry run openmldb-exporter --help` to get the help info.

A example output:

```bash
# HELP openmldb_connected_seconds_total duration for a component conncted time in seconds
# TYPE openmldb_connected_seconds_total counter
openmldb_connected_seconds_total{endpoint="172.17.0.15:9520",role="tablet"} 208834.70900011063
openmldb_connected_seconds_total{endpoint="172.17.0.15:9521",role="tablet"} 208834.70700001717
openmldb_connected_seconds_total{endpoint="172.17.0.15:9522",role="tablet"} 208834.71399998665
openmldb_connected_seconds_total{endpoint="172.17.0.15:9622",role="nameserver"} 208833.70000004768
openmldb_connected_seconds_total{endpoint="172.17.0.15:9623",role="nameserver"} 208831.70900011063
openmldb_connected_seconds_total{endpoint="172.17.0.15:9624",role="nameserver"} 208829.7230000496
# HELP openmldb_connected_seconds_created duration for a component conncted time in seconds
# TYPE openmldb_connected_seconds_created gauge
openmldb_connected_seconds_created{endpoint="172.17.0.15:9520",role="tablet"} 1.6501813860467942e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9521",role="tablet"} 1.6501813860495396e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9522",role="tablet"} 1.650181386050323e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9622",role="nameserver"} 1.6501813860512116e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9623",role="nameserver"} 1.650181386051238e+09
openmldb_connected_seconds_created{endpoint="172.17.0.15:9624",role="nameserver"} 1.6501813860512598e+09
```

## Release History

- 0.8.0
* Features
- Upgrade OpenMLDB SDK to v0.8
- improve test code

- 0.7.1
* Features
- Upgrade OpenMLDB SDK to v0.7
Expand Down
2 changes: 1 addition & 1 deletion prod.env
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
OPENMLDB_VERSION=0.8.1
OPENMLDB_VERSION=0.8.2
ENV_TYPE=prod
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ authors = ["aceforeverd <[email protected]>"]
license = "Apache-2.0"
readme = "README.md"
homepage = "https://openmldb.ai"
repository = "https://github.com/4paradigm/OpenMLDB"
repository = "https://github.com/4paradigm/openmldb-exporter"
documentation = "https://openmldb.ai/docs/zh/main/maintain/monitoring.html"
keywords = ["openmldb", "prometheus"]
classifiers = [
Expand Down