Skip to content

Commit

Permalink
Copy readme in index
Browse files Browse the repository at this point in the history
  • Loading branch information
aandres committed Jul 11, 2023
1 parent 252b7be commit e451352
Show file tree
Hide file tree
Showing 3 changed files with 23 additions and 17 deletions.
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,15 @@ in both realtime and batch jobs.
## Key Features


* Works in **real time** (eg: reading from kafka) and **replay mode** (eg: reading from parquet)
* Optimized for analytics, it uses micro-batching (instead of processing records one by one)
* Similar to [incremental][3], it updates nodes in a dag incrementally
* Taking inspiration from [kafka streams][4], there are two types of nodes in the dag:
* **Stream:** ephemeral micro-batches of events (cleared after every cycle)
* **State:** durable state derived from streams
* Clear separation between the business logic and the IO.
- Works in **real time** (eg: reading from kafka) and **replay mode** (eg: reading from parquet)
- Optimized for analytics, it uses micro-batching (instead of processing records one by one)
- Similar to [incremental][3], it updates nodes in a dag incrementally
- Taking inspiration from [kafka streams][4], there are two types of nodes in the dag:
* **Stream**: ephemeral micro-batches of events (cleared after every cycle)
* **State**: durable state derived from streams
- Clear separation between the business logic and the IO.
So the same dag can be used in real time mode, replay mode or can be easily tested.
* Functional interface: no inheritance or decorator required
- Functional interface: no inheritance or decorator required


[1]: https://github.com/tradewelltech/beavers
Expand Down
23 changes: 14 additions & 9 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

# Beavers

[Beavers][1] is a python library for stream processing, optimized for analytics.
Expand All @@ -10,15 +9,21 @@ in both realtime and batch jobs.
## Key Features


* Works in **real time** (eg: reading from kafka) and **replay mode** (eg: reading from parquet)
* Optimized for analytics, it uses micro-batching (instead of processing records one by one)
* Similar to [incremental][3], it updates nodes in a dag incrementally
* Taking inspiration from [kafka streams][4], there are two types of nodes in the dag:
* **Stream:** ephemeral micro-batches of events (cleared after every cycle)
* **State:** durable state derived from streams
* Clear separation between the business logic and the IO.
- Works in **real time** (eg: reading from kafka) and **replay mode** (eg: reading from parquet)
- Optimized for analytics, it uses micro-batching (instead of processing records one by one)
- Similar to [incremental][3], it updates nodes in a dag incrementally
- Taking inspiration from [kafka streams][4], there are two types of nodes in the dag:
* **Stream**: ephemeral micro-batches of events (cleared after every cycle)
* **State**: durable state derived from streams
- Clear separation between the business logic and the IO.
So the same dag can be used in real time mode, replay mode or can be easily tested.
* Functional interface: no inheritance or decorator required
- Functional interface: no inheritance or decorator required


[1]: https://github.com/tradewelltech/beavers
[2]: https://www.tradewelltech.co/
[3]: https://github.com/janestreet/incremental
[4]: https://www.confluent.io/blog/kafka-streams-tables-part-1-event-streaming/


[1]: https://github.com/tradewelltech/beavers
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ plugins:
show_source: false

markdown_extensions:
- def_list
- pymdownx.inlinehilite
- pymdownx.superfences
- pymdownx.snippets:
Expand Down

0 comments on commit e451352

Please sign in to comment.