Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added rethinkDB #68

Merged
merged 2 commits into from
Dec 15, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ A list of supported targets is below.
| | [Percona](https://www.percona.com/) |stores.percona | [Usage](targets/stores/percona) | [Example](examples/stores/percona) |
| | [Cockroachdb](https://www.cockroachlabs.com/) |stores.cockroachdb | [Usage](targets/stores/cockroachdb) | [Example](examples/stores/cockroachdb) |
| | [Aerospike](https://www.aerospike.com/) |stores.aerospike | [Usage](targets/stores/aerospike) | [Example](examples/stores/aerospike) |
| | [RethinkDB](https://rethinkdb.com/) |stores.rethinkdb | [Usage](targets/stores/rethinkdb) | [Example](examples/stores/rethinkdb) |
| Messaging | | | | |
| | [Kafka](https://kafka.apache.org/) |messaging.kafka | [Usage](targets/messaging/kafka) | [Example](examples/messaging/kafka) |
| | [Nats](https://nats.io/) |messaging.nats | [Usage](targets/messaging/nats) | [Example](examples/messaging/nats) |
Expand Down
14 changes: 14 additions & 0 deletions examples/stores/rethinkdb/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
bindings:
- name: rethinkdb
source:
kind: kubemq.query
properties:
address: localhost:50000
channel: query.rethinkdb
target:
kind: stores.rethinkdb
properties:
host: localhost:28015
password: root
username: root
properties: {}
66 changes: 66 additions & 0 deletions examples/stores/rethinkdb/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
package main

import (
"context"
"encoding/json"
"fmt"
"github.com/kubemq-hub/kubemq-targets/types"
"github.com/kubemq-io/kubemq-go"
"github.com/nats-io/nuid"
"log"
"time"
)

func main() {
args := make(map[string]interface{})
args["id"] = "test_user"
args["password"] = 1
insert, err := json.Marshal(args)
if err != nil {
log.Fatal(err)
}
client, err := kubemq.NewClient(context.Background(),
kubemq.WithAddress("localhost", 50000),
kubemq.WithClientId(nuid.Next()),
kubemq.WithTransportType(kubemq.TransportTypeGRPC))
if err != nil {
log.Fatal(err)
}
randomKey := nuid.Next()
// set request
setRequest := types.NewRequest().
SetMetadataKeyValue("method", "insert").
SetMetadataKeyValue("db_name", "test").
SetMetadataKeyValue("table", "test").
SetData(insert)
querySetResponse, err := client.SetQuery(setRequest.ToQuery()).
SetChannel("query.rethinkdb").
SetTimeout(10 * time.Second).Send(context.Background())
if err != nil {
log.Fatal(err)
}
setResponse, err := types.ParseResponse(querySetResponse.Body)
if err != nil {
log.Fatal(err)
}
log.Println(fmt.Sprintf("set request for key: %s executed, response: %s", randomKey, setResponse.Metadata.String()))

// get request
getRequest := types.NewRequest().
SetMetadataKeyValue("method", "get").
SetMetadataKeyValue("db_name", "test").
SetMetadataKeyValue("key", "test_user").
SetMetadataKeyValue("table", "test")

queryGetResponse, err := client.SetQuery(getRequest.ToQuery()).
SetChannel("query.rethinkdb").
SetTimeout(10 * time.Second).Send(context.Background())
if err != nil {
log.Fatal(err)
}
getResponse, err := types.ParseResponse(queryGetResponse.Body)
if err != nil {
log.Fatal(err)
}
log.Println(fmt.Sprintf("get request for key: %s executed, response: %s, data: %s", randomKey, getResponse.Metadata.String(), string(getResponse.Data)))
}
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ require (
google.golang.org/api v0.32.0
google.golang.org/genproto v0.0.0-20200925023002-c2d885f95484
google.golang.org/grpc v1.32.0
gopkg.in/rethinkdb/rethinkdb-go.v6 v6.2.1
)

replace github.com/Azure/azure-service-bus-go => github.com/Azure/azure-service-bus-go v0.10.3
Expand Down
3 changes: 2 additions & 1 deletion targets-manifest.json

Large diffs are not rendered by default.

180 changes: 180 additions & 0 deletions targets/stores/rethinkdb/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# Kubemq Rethinkdb Target Connector

Kubemq Rethinkdb target connector allows services using kubemq server to access Rethinkdb database services.

## Prerequisites
The following are required to run the Rethinkdb target connector:

- kubemq cluster
- Rethinkdb server
- kubemq-targets deployment

## Configuration

Rethinkdb target connector configuration properties:

| Properties Key | Required | Description | Example |
|:--------------------------|:---------|:-------------------------------------|:--------------------------|
| host | yes | Rethinkdb host address | "localhost:27017" |
| username | no | Rethinkdb username(if user exists) | "admin" |
| password | no | Rethinkdb password | "password" |
| timeout | no | timeout in seconds | "5" |
| keep_alive_period | no | keep alive period in seconds | "5" |
| auth_key | no | auth key if needed for connection | "" |
| ssl | no | set if ssl is needed | "false","true" |
| cert_file | no | ssl certificate file in string format| "my_file" |
| cert_key | no | ssl certificate key in string format | "my_key" |
| handShakeVersion | no | server hand shake version | "1" |
| number_of_retries | no | number of retries for each request | "1" |
| initial_cap | no | server initial cap | "0" |
| max_open | no | max open for server | "0" |



Example:

```yaml
bindings:
- name: kubemq-query-Rethinkdb
source:
kind: kubemq.query
name: kubemq-query
properties:
address: "kubemq-cluster:50000"
client_id: "kubemq-query-Rethinkdb-connector"
auth_token: ""
channel: "query.Rethinkdb"
group: ""
concurrency: "1"
auto_reconnect: "true"
reconnect_interval_seconds: "1"
max_reconnects: "0"
target:
kind: stores.Rethinkdb
name: target-Rethinkdb
properties:
host: "localhost:27017"
username: "admin"
password: "password"
database: "admin"
collection: "test"
write_concurrency: "majority"
read_concurrency: ""
params: ""
operation_timeout_seconds: "2"
```

## Usage

### Get Request

Get request metadata setting:

| Metadata Key | Required | Description | Possible values |
|:-------------|:---------|:-----------------|:----------------|
| key | no | key name | any string |
| method | yes | get | "get" |
| db_name | yes | db name | "my_db" |
| table | yes | table name | "my_table" |

Example:

```json
{
"metadata": {
"key": "your-Rethinkdb-key",
"db_name": "test",
"table": "users",
"method": "get"
},
"data": null
}
```

### Update Request

Update request metadata setting:

| Metadata Key | Required | Description | Possible values |
|:-------------|:---------|:-----------------|:----------------|
| key | yes | key name | any string |
| method | yes | get | "update" |
| db_name | yes | db name | "my_db" |
| table | yes | table name | "my_table" |

Update request data setting:

| Data Key | Required | Description | Possible values |
|:---------|:---------|:------------------------------|:--------------------|
| data | yes | map of string interface | base64 bytes array |

Example:

```json
{
"metadata": {
"key": "your-Rethinkdb-key",
"db_name": "test",
"table": "users",
"method": "update"
},
"data": "ICAibWV0YWRhdGEiOiB7CiAgICAia2V5IjogInlvdXItUmV0aGlua2RiLWtleSIsCiAgICAiZGJfbmFtZSI6ICJ0ZXN0IiwKICAgICJ0YWJsZSI6ICJ1c2VycyIsCiAgICAibWV0aG9kIjogImdldCIKICB9LA=="
}
```
### Delete Request

Delete request metadata setting:

| Metadata Key | Required | Description | Possible values |
|:-------------|:---------|:-----------------|:----------------|
| key | yes | key name | any string |
| method | yes | get | "delete" |
| db_name | yes | db name | "my_db" |
| table | yes | table name | "my_table" |


Example:

```json
{
"metadata": {
"key": "your-Rethinkdb-key",
"db_name": "test",
"table": "users",
"method": "delete"
},
"data": null
}
```


### Insert Request

insert request metadata setting:

| Metadata Key | Required | Description | Possible values |
|:-------------|:---------|:-----------------|:----------------|
| key | yes | key name | any string |
| method | yes | get | "insert" |
| db_name | yes | db name | "my_db" |
| table | yes | table name | "my_table" |

Insert request data setting:

| Data Key | Required | Description | Possible values |
|:---------|:---------|:------------------------------|:--------------------|
| data | yes | map of string interface | base64 bytes array |

Example:

```json
{
"metadata": {
"key": "your-Rethinkdb-key",
"db_name": "test",
"table": "users",
"method": "insert"
},
"data": "ICAibWV0YWRhdGEiOiB7CiAgICAia2V5IjogInlvdXItUmV0aGlua2RiLWtleSIsCiAgICAiZGJfbmFtZSI6ICJ0ZXN0IiwKICAgICJ0YWJsZSI6ICJ1c2VycyIsCiAgICAibWV0aG9kIjogImdldCIKICB9LA=="
}
```
Loading