Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added cockroachDB #64

Merged
merged 3 commits into from
Dec 9, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions config.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
bindings:
- name: hazelcast
- name: cockroachdb
source:
kind: kubemq.query
properties:
address: localhost:50000
channel: query.hazelcast
channel: query.cockroachdb
target:
kind: cache.hazelcast
kind: stores.cockroachdb
properties:
address: localhost:5701
server name: test
connection: postgres://root:postgres@localhost:26257/postgres?sslmode=disable
properties: {}
2 changes: 1 addition & 1 deletion config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import (
"strings"
)

const defaultApiPort = 8081
const defaultApiPort = 8080

var configFile string
var logr = logger.NewLogger("config")
Expand Down
12 changes: 12 additions & 0 deletions examples/stores/cockroachdb/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
bindings:
- name: cockroachdb
source:
kind: kubemq.query
properties:
address: localhost:50000
channel: query.cockroachdb
target:
kind: stores.cockroachdb
properties:
connection: postgres://root:postgres@localhost:26257/postgres?sslmode=disable
properties: {}
66 changes: 66 additions & 0 deletions examples/stores/cockroachdb/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
package main

import (
"context"
"fmt"
"github.com/kubemq-hub/kubemq-targets/types"
"github.com/kubemq-io/kubemq-go"
"github.com/nats-io/nuid"
"log"
"time"
)

var (
transactionString = ` DROP TABLE IF EXISTS post;
CREATE TABLE post (
ID serial,
TITLE varchar(40),
CONTENT varchar(255),
CONSTRAINT pk_post PRIMARY KEY(ID)
);
INSERT INTO post(ID,TITLE,CONTENT) VALUES
(1,NULL,'Content One'),
(2,'Title Two','Content Two');`
queryString = `SELECT id,title,content FROM post;`
)

func main() {
client, err := kubemq.NewClient(context.Background(),
kubemq.WithAddress("localhost", 50000),
kubemq.WithClientId(nuid.Next()),
kubemq.WithTransportType(kubemq.TransportTypeGRPC))
if err != nil {
log.Fatal(err)
}

transactionRequest := types.NewRequest().
SetMetadataKeyValue("method", "transaction").
SetData([]byte(transactionString))
queryTransactionResponse, err := client.SetQuery(transactionRequest.ToQuery()).
SetChannel("query.cockroachdb").
SetTimeout(10 * time.Second).Send(context.Background())
if err != nil {
log.Fatal(err)
}
transactionResponse, err := types.ParseResponse(queryTransactionResponse.Body)
if err != nil {
log.Fatal(err)
}
log.Println(fmt.Sprintf("transaction request result: %s ", transactionResponse.Metadata.String()))

queryRequest := types.NewRequest().
SetMetadataKeyValue("method", "query").
SetData([]byte(queryString))

queryResponse, err := client.SetQuery(queryRequest.ToQuery()).
SetChannel("query.cockroachdb").
SetTimeout(10 * time.Second).Send(context.Background())
if err != nil {
log.Fatal(err)
}
response, err := types.ParseResponse(queryResponse.Body)
if err != nil {
log.Fatal(err)
}
log.Println(fmt.Sprintf("query request results: %s ", string(response.Data)))
}
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ require (
github.com/Shopify/sarama v1.27.0
github.com/aws/aws-sdk-go v1.34.31
github.com/bradfitz/gomemcache v0.0.0-20190913173617-a41fca850d0b
github.com/cockroachdb/cockroach-go v2.0.1+incompatible
github.com/couchbase/gocb/v2 v2.1.6
github.com/denisenkom/go-mssqldb v0.0.0-20200910202707-1e08a3fab204
github.com/eclipse/paho.mqtt.golang v1.2.0
Expand Down
2 changes: 1 addition & 1 deletion targets-manifest-hash.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
c6a0ef67d4d192f0cd37c6755efe732d90cf2213d369a7af4cf87742673bd351
91f3917cb3424a0e03a68b4d766eb0ffd1da787c509f00641665aee5ff81ece7
2 changes: 1 addition & 1 deletion targets-manifest.json

Large diffs are not rendered by default.

152 changes: 152 additions & 0 deletions targets/stores/cockroachdb/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
# Kubemq Cockroach Target Connector

Kubemq Cockroach target connector allows services using kubemq server to access Cockroach database services.

## Prerequisites
The following are required to run the Cockroach target connector:

- kubemq cluster
- Cockroach server
- kubemq-targets deployment

## Configuration

Cockroach target connector configuration properties:

| Properties Key | Required | Description | Example |
|:--------------------------------|:---------|:--------------------------------------------|:-----------------------------------------------------------------------|
| connection | yes | Cockroach connection string address | "postgres://root:postgres@localhost:26257/Cockroach?sslmode=disable" |
| max_idle_connections | no | set max idle connections | "10" |
| max_open_connections | no | set max open connections | "100" |
| connection_max_lifetime_seconds | no | set max lifetime for connections in seconds | "3600" |


Example:

```yaml
bindings:
- name: cockroachdb
source:
kind: kubemq.query
properties:
address: localhost:50000
channel: query.cockroachdb
target:
kind: stores.cockroachdb
properties:
connection: postgres://root:postgres@localhost:26257/postgres?sslmode=disable
properties: {}

```

## Usage

### Query Request

Query request metadata setting:

| Metadata Key | Required | Description | Possible values |
|:-------------|:---------|:-----------------|:----------------|
| method | yes | set type of request | "query" |

Query request data setting:

| Data Key | Required | Description | Possible values |
|:---------|:---------|:-------------|:-------------------|
| data | yes | query string | base64 bytes array |

Example:

Query string: `SELECT id,title,content FROM post;`

```json
{
"metadata": {
"method": "query"
},
"data": "U0VMRUNUIGlkLHRpdGxlLGNvbnRlbnQgRlJPTSBwb3N0Ow=="
}
```

### Exec Request

Exec request metadata setting:

| Metadata Key | Required | Description | Possible values |
|:----------------|:---------|:---------------------------------------|:-------------------|
| method | yes | set type of request | "exec" |
| isolation_level | no | set isolation level for exec operation | "" |
| | | | "read_uncommitted" |
| | | | "read_committed" |
| | | | "repeatable_read" |
| | | | "serializable" |
| | | | |


Exec request data setting:

| Data Key | Required | Description | Possible values |
|:---------|:---------|:------------------------------|:--------------------|
| data | yes | exec string | base64 bytes array |

Example:

Exec string:
```sql
INSERT INTO post(ID,TITLE,CONTENT) VALUES (1,NULL,'Content One'),(2,'Title Two','Content Two');
```

```json
{
"metadata": {
"method": "exec",
"isolation_level": "read_uncommitted"
},
"data": "SU5TRVJUIElOVE8gcG9zdChJRCxUSVRMRSxDT05URU5UKSBWQUxVRVMKCSAgICAgICAgICAgICAgICAgICAgICAgKDEsTlVMTCwnQ29udGVudCBPbmUnKSwKCSAgICAgICAgICAgICAgICAgICAgICAgKDIsJ1RpdGxlIFR3bycsJ0NvbnRlbnQgVHdvJyk7"
}
```

### Transaction Request

Transaction request metadata setting:

| Metadata Key | Required | Description | Possible values |
|:----------------|:---------|:---------------------------------------|:-------------------|
| method | yes | set type of request | "transaction" |
| isolation_level | no | set isolation level for exec operation | "" |
| | | | "read_uncommitted" |
| | | | "read_committed" |
| | | | "repeatable_read" |
| | | | "serializable" |


Transaction request data setting:

| Data Key | Required | Description | Possible values |
|:---------|:---------|:------------------------------|:--------------------|
| data | yes | string string | base64 bytes array |

Example:

Transaction string:
```sql
DROP TABLE IF EXISTS post;
CREATE TABLE post (
ID serial,
TITLE varchar(40),
CONTENT varchar(255),
CONSTRAINT pk_post PRIMARY KEY(ID)
);
INSERT INTO post(ID,TITLE,CONTENT) VALUES
(1,NULL,'Content One'),
(2,'Title Two','Content Two');
```
```json
{
"metadata": {
"key": "your-Cockroach-key",
"method": "delete"
},
"data": "CURST1AgVEFCTEUgSUYgRVhJU1RTIHBvc3Q7CiAgICBDUkVBVEUgVEFCTEUgcG9zdCAoCgkgICAgICAgICBJRCBzZXJpYWwsCgkgICAgICAgICBUSVRMRSB2YXJjaGFyKDQwKSwKCSAgICAgICAgIENPTlRFTlQgdmFyY2hhcigyNTUpLAoJICAgICAgICAgQ09OU1RSQUlOVCBwa19wb3N0IFBSSU1BUlkgS0VZKElEKQoJICAgICAgICk7CiAgICBJTlNFUlQgSU5UTyBwb3N0KElELFRJVExFLENPTlRFTlQpIFZBTFVFUwoJICAgICAgICAgICAgICAgICAgICAgICAoMSxOVUxMLCdDb250ZW50IE9uZScpLAoJICAgICAgICAgICAgICAgICAgICAgICAoMiwnVGl0bGUgVHdvJywnQ29udGVudCBUd28nKTs="
}
```
Loading