Skip to content

Commit

Permalink
Add Configure KSQL to Syntax Guide, fix/clariy config examples
Browse files Browse the repository at this point in the history
  • Loading branch information
Michael G. Noll committed Sep 12, 2017
1 parent 3af28e7 commit a037280
Show file tree
Hide file tree
Showing 2 changed files with 55 additions and 16 deletions.
31 changes: 15 additions & 16 deletions docs/examples.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Examples

| [Overview](/docs#ksql-documentation) |[Quick Start](/docs/quickstart#quick-start) | [Concepts](/docs/concepts.md#concepts) | [Syntax Reference](/docs/syntax-reference.md#syntax-reference) |[Demo](/ksql-clickstream-demo#clickstream-analysis) | Examples | [FAQ](/docs/faq.md#frequently-asked-questions) | [Roadmap](/docs/roadmap.md#roadmap) |
| [Overview](/docs#ksql-documentation) |[Quick Start](/docs/quickstart#quick-start) | [Concepts](/docs/concepts.md#concepts) | [Syntax Reference](/docs/syntax-reference.md#syntax-reference) |[Demo](/ksql-clickstream-demo#clickstream-analysis) | Examples | [FAQ](/docs/faq.md#frequently-asked-questions) | [Roadmap](/docs/roadmap.md#roadmap) |
|---|----|-----|----|----|----|----|----|

**Table of Contents**
Expand Down Expand Up @@ -252,30 +252,29 @@ CREATE STREAM pageviews_interest_contact AS \

## Configuring KSQL

You can set config properties for KSQL and your queries with the SET statement. This includes configuring settings
relating to Kafka's Streams API as well as settings for Kafka's producer and consumer clients.
Common configuration properties that you might want to change from their default values include:

```sql
SET '<property-name>'='<property-value>';
```
- [auto.offset.reset](https://kafka.apache.org/documentation/#newconsumerconfigs):
The default value in KSQL is `latest` meaning all the Kafka topics will be read from the current offset (aka latest
available data). You can change it using the following statement:

Both property name and property value should be enclosed in single quotes.
A property that is set using the SET statement will remain in effect for the remainder of the KSQL CLI session
until you issue another SET statement to change it.

Here are some of the common config properties that you might want to change from their default values:
```sql
SET 'auto.offset.reset'='earliest';
```

- `auto.offset.reset`: The default value in KSQL is `latest` meaning all the Kafka topics will be read
from the current offset (aka latest available data). You can change it using the following statement:
- [bootstrap.servers](https://kafka.apache.org/documentation/#newconsumerconfigs):
Where to find the Kafka cluster that KSQL should connect to. KSQL defaults to `localhost:9092`.

```sql
SET 'auto.offset.reset'='earliest';
SET 'bootstrap.servers'='kafka-broker1:9092,kafka-broker2:9092'
```

- `commit.interval.ms`: The default value is `2000`. Here is an example to change the value to `5000`:
- [commit.interval.ms](https://kafka.apache.org/documentation/#streamsconfigs):
The default value in KSQL is `2000`. Here is an example to change the value to `5000`:

```sql
SET 'commit.interval.ms'='5000';
```

- `cache.max.bytes.buffering`: The default value is `10485760` (10 MB);
- [cache.max.bytes.buffering](https://kafka.apache.org/documentation/#streamsconfigs):
The default value in KSQL is `10000000` (~ 10 MB);
40 changes: 40 additions & 0 deletions docs/syntax-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ The KSQL CLI provides a terminal-based interactive shell for running queries.
- [KSQL statements](#ksql-statements)
- [Scalar functions](#scalar-functions)
- [Aggregate functions](#aggregate-functions)
- [Configuring KSQL](#configuring-ksql)


# CLI-specific commands
Expand Down Expand Up @@ -477,3 +478,42 @@ Terminate a persistent query. Persistent queries run continuously until they are
| MAX | `MAX(col1)` | Return the maximum value for a given column and window |
| MIN | `MIN(col1)` | Return the minimum value for a given column and window |
| SUM | `SUM(col1)` | Sums the column values |


# Configuring KSQL

You can set configuration properties for KSQL and your queries with the SET statement. This includes
[settings for Kafka's Streams API](https://kafka.apache.org/documentation/#streamsconfigs)
(e.g., `cache.max.bytes.buffering`) as well as
settings for Kafka's [producer client](https://kafka.apache.org/documentation/#producerconfigs) and
[consumer client](https://kafka.apache.org/documentation/#newconsumerconfigs) (e.g., `auto.offset.reset`).

```sql
SET '<property-name>'='<property-value>';
```

Examples:

```
ksql> SET 'bootstrap.servers'='kafka-broker1:9092,kafka-broker2:9092';
ksql> SET 'commit.interval.ms'='5000';
```
Both property name and property value should be enclosed in single quotes.
Once a property has been set, it will remain in effect for the remainder of the KSQL CLI session until you issue another
SET statement to change it.
You can also use a *properties file* instead of the SET statement:
```bash
# Show the example contents of a properties file
$ cat ksql.properties
bootstrap.servers=kafka-broker1:9092,kafka-broker2:9092
# Start KSQL in standalone mode with the custom properties above
$ ksql-cli local --properties-file ./ksql.properties
# Start a KSQL server node (for client-server mode) with the custom properties above
$ ksql-server-start --properties-file ./ksql.properties
```

0 comments on commit a037280

Please sign in to comment.