Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update README and workflow to use new doc location #126

Merged
merged 4 commits into from
Feb 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/deploy_gh_pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ on:
branches:
- main
paths:
- 'docs/gh_pages_v2/**'
- 'docs/gh_pages/**'

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
Expand All @@ -32,7 +32,7 @@ jobs:
# Install and build Docusaurus website
- name: Build Docusaurus website
run: |
cd docs/gh_pages_v2
cd docs/gh_pages
npm install
npm run build
- name: Deploy to GitHub Pages
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
<p align="center">
<img src="https://github.com/daxa-ai/pebblo/blob/main/docs/gh_pages/assets/img/pebblo-logo.png?raw=true" />
<img src="https://github.com/daxa-ai/pebblo/blob/main/docs/gh_pages/static/img/pebblo-logo-name.png?raw=true" />
</p>

---
[![GitHub](https://img.shields.io/badge/GitHub-pebblo-blue?logo=github)](https://github.com/daxa-ai/pebblo)
[![MIT license](https://img.shields.io/badge/license-MIT-brightgreen.svg)](http://opensource.org/licenses/MIT)
[![Documentation](https://img.shields.io/badge/Documentation-pebblo-blue?logo=read-the-docs)](https://daxa-ai.github.io/pebblo-docs/)
[![Documentation](https://img.shields.io/badge/Documentation-pebblo-blue?logo=read-the-docs)](https://daxa-ai.github.io/pebblo/)

[![PyPI](https://img.shields.io/pypi/v/pebblo?logo=pypi)](https://pypi.org/project/pebblo/)
![PyPI - Downloads](https://img.shields.io/pypi/dm/pebblo)
Expand Down Expand Up @@ -97,7 +97,7 @@ The Pebblo SafeLoader can be enabled with few lines of code change to the above
vectordb = Chroma.from_documents(documents, OpenAIEmbeddings())
```

See [here](https://github.com/srics/pebblo/tree/main/samples) for samples with Pebblo enabled RAG applications and [this](https://daxa-ai.github.io/pebblo-docs/rag.html) document for more details.
See [here](https://github.com/srics/pebblo/tree/main/samples) for samples with Pebblo enabled RAG applications and [this](https://daxa-ai.github.io/pebblo/rag) document for more details.

# Contribution

Expand Down
File renamed without changes.
File renamed without changes.
3 changes: 0 additions & 3 deletions docs/gh_pages/_config.yml

This file was deleted.

File renamed without changes.
File renamed without changes.
8 changes: 3 additions & 5 deletions docs/gh_pages/daemon.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
# Pebblo Daemon


`Pebblo Daemon` is a REST API application that exposes API endpoints for Pebblo Safe DataLoader to connect. This component provides deep data visibility on the types of Topics and Entities ingested into the Gen-AI application. It uses the snippets received from the `Pebblo Safe DataLoader` to run through both a Topic Classifier and Entity Classifier to produce the insights and reporting. For more details on how to Pebblo enable your Langchain application see this [Pebblo Safe DataLoader for Langchain](/pebblo-docs/rag.html) document.

`Pebblo Daemon` is a REST API application that exposes API endpoints for Pebblo Safe DataLoader to connect. This component provides deep data visibility on the types of Topics and Entities ingested into the Gen-AI application. It uses the snippets received from the `Pebblo Safe DataLoader` to run through both a Topic Classifier and Entity Classifier to produce the insights and reporting. For more details on how to Pebblo enable your Langchain application see this [Pebblo Safe DataLoader for Langchain](rag.md) document.

By default `Pebblo Daemon` runs at `localhost:8000`. The `Pebblo Safe DataLoader` by default connects to this hostname and port. If the daemon is running in a different port or a different hostname, the `Pebblo Safe DataLoader` env variable `PEBBLI_CLASSIFIER_URL` need to set to the correct URL.

Expand All @@ -12,7 +10,7 @@ A separate `Data Report` will be generated for every complete document load oper

## Report Location

By default all the reports will be stored in a `.pebblo` in the home directory of the system running `Pebblo Daemon`. Separate subdirectories named with the RAG application name is used when multiple RAG applications uses the same `Pebblo Daemon`.
By default all the reports will be stored in a `.pebblo` in the home directory of the system running `Pebblo Daemon`. Separate subdirectories named with the RAG application name is used when multiple RAG applications uses the same `Pebblo Daemon`.

```bash

Expand All @@ -24,4 +22,4 @@ $ tree
│   │   ├── metadata
│   │   │   └── metadata.json
│   │   └── report.json
```
```
1 change: 0 additions & 1 deletion docs/gh_pages/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,6 @@ pebblo

Pebblo daemon now listens to `localhost:8000` to accept Gen-AI application document snippets for inspection and reporting.


## Creating a pull request

See [these instructions](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork)
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
31 changes: 0 additions & 31 deletions docs/gh_pages/index.md

This file was deleted.

14 changes: 7 additions & 7 deletions docs/gh_pages/installation.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,29 @@
## Installation
# Installation

### Pre-requisites

#### Mac OSX

```sh
```
brew install pango
```

#### Linux (debian/ubuntu)

```sh
```
sudo apt-get install libpango-1.0-0 libpangoft2-1.0-0
```

### Pebblo Daemon

```sh
```
pip install pebblo
```

## Run Pebblo daemon
# Run Pebblo daemon

```sh
```
pebblo
```

Pebblo daemon now listens to `localhost:8000` to accept Gen-AI application data snippets for inspection and reporting.
Pebblo daemon now listens to `localhost:8000` to accept Gen-AI application data snippets for inspection and reporting.
File renamed without changes.
File renamed without changes.
5 changes: 2 additions & 3 deletions docs/gh_pages/rag.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Pebblo Safe DataLoader for Langchain

This document describes how to augment your existing Langchain DocumentLoader with Pebblo Safe DataLoader to get deep data visibility on the types of Topics and Entities ingested into the Gen-AI Langchain application. For details on `Pebblo Daemon` see this [pebblo daemon](/pebblo-docs/daemon.html) document.
This document describes how to augment your existing Langchain DocumentLoader with Pebblo Safe DataLoader to get deep data visibility on the types of Topics and Entities ingested into the Gen-AI Langchain application. For details on `Pebblo Daemon` see this [pebblo daemon](daemon.md) document.

Pebblo Safeloader enables safe data ingestion for Langchain document loader<sup>1</sup>. This is done by wrapping the document loader call with `Pebblo Safe DataLoader`.

Expand All @@ -10,7 +10,6 @@ Assume a Langchain RAG application snippet using `CSVLoader` to read a CSV docum

Here is the snippet of Lanchain RAG application using `CSVLoader`.


```python
from langchain.document_loaders.csv_loader import CSVLoader

Expand All @@ -35,7 +34,7 @@ The Pebblo SafeLoader can be enabled with few lines of code change to the above
vectordb = Chroma.from_documents(documents, OpenAIEmbeddings())
```

A data report with all the findings, both Topics and Entities, will be generated and available for inspection in the `Pebblo Daemon`. See this [pebblo daemon](/pebblo-docs/daemon.html) for further details.
A data report with all the findings, both Topics and Entities, will be generated and available for inspection in the `Pebblo Daemon`. See this [pebblo daemon](daemon.md) for further details.

## Supported Document Loaders

Expand Down
33 changes: 0 additions & 33 deletions docs/gh_pages/reporting.md

This file was deleted.

File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
25 changes: 0 additions & 25 deletions docs/gh_pages_v2/docs/daemon.md

This file was deleted.

71 changes: 0 additions & 71 deletions docs/gh_pages_v2/docs/development.md

This file was deleted.

29 changes: 0 additions & 29 deletions docs/gh_pages_v2/installation.md

This file was deleted.

59 changes: 0 additions & 59 deletions docs/gh_pages_v2/rag.md

This file was deleted.

Binary file removed docs/gh_pages_v2/static/img/pebblo-overview.jpg
Binary file not shown.