Skip to content

Commit

Permalink
Fix up examples documentation and add the earthquake example (#346)
Browse files Browse the repository at this point in the history
Surface end-to-end examples in Home Page and add a detailed table of examples

---------

Signed-off-by: Pavithra Eswaramoorthy <[email protected]>
Signed-off-by: Adam Glustein <[email protected]>
Co-authored-by: Pavithra Eswaramoorthy <[email protected]>
  • Loading branch information
AdamGlustein and pavithraes committed Jul 18, 2024
1 parent 65207a4 commit 4a0a80c
Show file tree
Hide file tree
Showing 10 changed files with 235 additions and 28 deletions.
41 changes: 36 additions & 5 deletions docs/wiki/Home.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,24 +16,55 @@ CSP (Composable Stream Processing) is a library for high-performance real-time e
- **PyData interoperability:** Use your favorite libraries from the Scientific Python Ecosystem for numerical and statistical computations
- **Functional/declarative style:** Write concise and composable code for stream processing by building graphs in Python

<!-- ## Applications -->

## Get Started

- [Install CSP](Installation) and [write your first CSP program](First-Steps)
- Learn more about [nodes](CSP-Node), [graphs](CSP-Graph), and [execution modes](Execution-Modes)
- Learn to extend CSP with [adapters](Adapters)

<!-- - Check out the [examples](Examples) for various CSP features and use cases -->
## Applications

> \[!TIP\]
> Find relevant docs with GitHub’s search function, use `repo:Point72/csp type:wiki <search terms>` to search the documentation Wiki Pages.
<table>
<tr>
<td>Real-time public transportation</td>
<td>Tracking Wikipedia page edits</td>
<td>Processing seismic waveforms</td>
<td>Visualizing global earthquakes</td>
</tr>
<tr>
<td>
<a href="https://github.com/Point72/csp/blob/main/examples/07_end_to_end/mta.ipynb">
<img src="images/mta-example.png" width="300px">
</a>
</td>
<td>
<a href="https://github.com/Point72/csp/blob/main/examples/07_end_to_end/wikimedia.ipynb">
<img src="images/wikimedia-example.png" width="300px">
</a>
</td>
<td>
<a href="https://github.com/Point72/csp/blob/main/examples/07_end_to_end/seismic_waveform.ipynb">
<img src="images/seismic-waveform-example.png" width="300px">
</a>
</td>
<td>
<a href="https://github.com/Point72/csp/blob/main/examples/07_end_to_end/earthquake.ipynb">
<img src="images/earthquake-example.png" width="300px">
</a>
</td>
</tr>
</table>

Check out the [examples directory](https://github.com/Point72/csp/tree/main/examples) for a comprehensive collection of CSP applications.

## Community

- [Contribute](Contribute) to CSP and help improve the project
- Read about future plans in the [project roadmap](Roadmap)

> \[!TIP\]
> Find relevant docs with GitHub’s search function, use `repo:Point72/csp type:wiki <search terms>` to search the documentation Wiki Pages.
## License

CSP is licensed under the Apache 2.0 license. See the [LICENSE](https://github.com/Point72/csp/blob/main/LICENSE) file for details.
2 changes: 1 addition & 1 deletion docs/wiki/_Sidebar.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@ Notes for editors:
- [`csp.Struct` API](csp.Struct-API)
- [`csp.dynamic` API](csp.dynamic-API)
- [`csp.profiler` API](csp.profiler-API)
- [Examples](Examples)
- [Glossary of Terms](Glossary)
- [Examples](https://github.com/Point72/csp/tree/main/examples)

**Developer Guide**

Expand Down
Binary file added docs/wiki/images/earthquake-example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/wiki/images/mta-example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/wiki/images/seismic-waveform-example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/wiki/images/wikimedia-example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
11 changes: 0 additions & 11 deletions docs/wiki/references/Examples.md

This file was deleted.

4 changes: 2 additions & 2 deletions examples/01_basics/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@

- [Simplest Possible Graph](./e1_basic.py)
- [Ticking Graphs](./e2_ticking.py)
- [Complete Example (Trading)](./e3_trade_pnl.py)
- [Visualizing a Graph](./e4_show_graph.py)
- [Visualizing a Graph](./e3_show_graph.py)
- [Complete Example (Trading)](./e4_trade_pnl.py)
1 change: 1 addition & 0 deletions examples/03_using_adapters/websocket/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# Websocket Adapter

- [Websocket Client](./e1_websocket_client.py)
- [Websocket Output](./e2_websocket_output.py)
204 changes: 195 additions & 9 deletions examples/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,197 @@
# `csp` examples

- [Basics](./01_basics/)
- [Intermediate](./02_intermediate/)
- [Adapters](./03_using_adapters/)
- [Writing Adapters](./04_writing_adapters/)
- [Writing C++ Nodes and Adapters](./05_cpp/)
- [Advanced](./06_advanced/)
- [End to End Examples](./07_end_to_end/)
- [Just for fun!](./98_just_for_fun/)
- [Developer Tools](./99_developer_tools/)
<table>
<thead>
<tr>
<th>Type</th>
<th>Example</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<!-- Basic -->
<tr>
<td rowspan=4><a href="./01_basics/">Basics</a></td>
<td><a href="./01_basics/e1_basic.py">Basic Graph</a></td>
<td>Basic sum of constant integers</td>
</tr>
<tr>
<td><a href="./01_basics/e2_ticking.py">Ticking Graphs</a></td>
<td>Cumulative sum of streaming integers</td>
</tr>
<tr>
<td><a href="./01_basics/e3_show_graph.py">Visualizing a Graph</a></td>
<td>Bid-ask spread of a trade with graph visualization</td>
</tr>
<tr>
<td><a href="./01_basics/e4_trade_pnl.py">Complete Example (Trading)</a></td>
<td>Volume weighted average price (VWAP) and profit and loss (PnL)</td>
</tr>
<!-- Intermediate -->
<tr>
<td rowspan=4><a href="./02_intermediate/">Intermediate</a></td>
<td><a href="./02_intermediate/e1_feedback.py">Feedback Connections</a></td>
<td>Add a feedback edge between nodes in a graph</td>
</tr>
<tr>
<td><a href="./e02_intermediate/2_stats.py">Statistics Nodes</a></td>
<td>Use the CSP statistics library on simulated trading data</td>
</tr>
<tr>
<td><a href="./02_intermediate/e3_numpy_stats.py">Statistics Nodes with Numpy</a></td>
<td>Rolling window statistics on a set of three symbols using NumPy arrays</td>
</tr>
<tr>
<td><a href="./02_intermediate/e4_exprtk.py">Expression Nodes with <code>exprtk</code></a></td>
<td>Calculate mathematical expressions with streaming data</td>
</tr>
<!-- Using Adapters -->
<tr>
<td rowspan=4><a href="./03_using_adapters/">Using Adapters</a></td>
<td><a href="./03_using_adapters/kafka/e1_kafka.py">Kafka Adapter Example</a></td>
<td>
Stream data from a Kafka bus using KafkaAdapterManager and MessageMapper
</td>
</tr>
<tr>
<td><a href="./03_using_adapters/parquet/e1_parquet_write_read.py">Parquet Adapter Example</a></td>
<td>
Read from and write CSP data to Parquet files
</td>
</tr>
<tr>
<td><a href="./03_using_adapters/websocket/e1_websocket_client.py">Websocket Client</a></td>
<td>
Send and receive messages over a websocket connection
</td>
</tr>
<tr>
<td><a href="./03_using_adapters/websocket/e2_websocket_output.py">Websocket Output</a></td>
<td>
Send data over a websocket connection and view HTML output
</td>
</tr>
<!-- Writing Adapters -->
<tr>
<td rowspan=8><a href="./04_writing_adapters/">Writing Adapters</a></td>
<td><a href="./04_writing_adapters/e1_generic_push_adapter.py">Generic Push Adapter</a></td>
<td>
Push real-time data into a CSP graph
</td>
</tr>
<tr>
<td><a href="./04_writing_adapters/e2_pullinput.py">Pull Input Adapter</a></td>
<td>
Replay historical data using a pull adapter
</td>
</tr>
<tr>
<td><a href="./04_writing_adapters/e3_adaptermanager_pullinput.py">Pull Input Adapter with Adapter
Manager</a></td>
<td>
Provide data to multiple input adapters from a single source
</td>
</tr>
<tr>
<td><a href="./04_writing_adapters/e4_pushinput.py">Push Input Adapter</a></td>
<td>
Write your own push adapter for real-time data
</td>
</tr>
<tr>
<td><a href="./04_writing_adapters/e5_adaptermanager_pushinput.py">Push Input Adapter with Adapter
Manager</a></td>
<td>
Use an adapter manager with real-time data sources
</td>
</tr>
<tr>
<td><a href="./04_writing_adapters/e6_outputadapter.py">Output Adapter</a></td>
<td>
Create a custom writer for CSP output data
</td>
</tr>
<tr>
<td><a href="./04_writing_adapters/e7_adaptermanager_inputoutput.py">Complete Input/Output Adapter with
Adapter Manager</a></td>
<td>
Manage input and output adapters with a single adapter manager
</td>
</tr>
<tr>
<td><a href="./07_end_to_end/earthquake.ipynb">Push-Pull Input Adapter for Earthquake Data</a></td>
<td>
Create a push-pull adapter which transitions from replay to live execution
</td>
</tr>
<!-- Writing C++ Nodes and Adapters -->
<tr>
<td rowspan=2><a href="./05_cpp/">Writing C++ Nodes and Adapters</a></td>
<td><a href="./05_cpp/1_cpp_node/">C++ Node</a></td>
<td>
Extend CSP with a pig latin C++ node
</td>
</tr>
<tr>
<td><a href="./05_cpp/2_cpp_node_with_struct/">C++ Node with <code>csp.Struct</code></a></td>
<td>
Write a C++ node with a csp.Struct input
</td>
</tr>
<!-- Advanced -->
<tr>
<td rowspan=2><a href="./06_advanced/">Advanced</a></td>
<td><a href="./06_advanced/e1_dynamic.py">Dynamic Graphs</a></td>
<td>
Update the shape of a graph at runtime
</td>
</tr>
<tr>
<td><a href="./06_advanced/e2_pandas_extension.py">Pandas Extension</a></td>
<td>
Use CSP within a pandas DataFrame
</td>
</tr>
<!-- End-to-end examples -->
<tr>
<td rowspan=4><a href="./07_end_to_end/">End-to-end examples</a></td>
<td><a href="./07_end_to_end/mta.ipynb">MTA Subway Data</a></td>
<td>
Access real-time New York City transit data
</td>
</tr>
<tr>
<td><a href="./07_end_to_end/seismic_waveform.ipynb">Seismic Data with obspy</a></td>
<td>
Analyze seismic waveforms and compare with batch processing methods
</td>
</tr>
<tr>
<td><a href="./07_end_to_end/wikimedia.ipynb">Wikipedia Updates and Edits</a></td>
<td>
Monitor live updates to all Wikimedia sites
</td>
</tr>
<tr>
<td><a href="./07_end_to_end/earthquake.ipynb">World Earthquake Dashboard</a></td>
<td>
Display recent earthquakes on a live-updating world map
</td>
</tr>
<!-- Others -->
<tr>
<td><a href="./98_just_for_fun/">Just for fun!</a></td>
<td><a href="./98_just_for_fun/e1_csp_nand_computer.py">NAND Computer</a></td>
<td>
Understand <code>csp.node</code> & <code>csp.graph</code> by connecting NAND logic gates
</td>
</tr>
<tr>
<td><a href="./99_developer_tools/">Developer Tools</a></td>
<td><a href="./99_developer_tools/e1_profiling.py">Profiling <code>csp</code> code</a></td>
<td>
Profile a CSP Graph, view static attributes and runtime performance
</td>
</tr>
</tbody>
</table>

0 comments on commit 4a0a80c

Please sign in to comment.