GraphRAG QA Example

[akollegger]

Description

An example of GraphRAG using the SEC Edgar dataset with tool-based access over unstructured, structured, and semistructured data.

Issues

n/a

Type of change

List the type of change like below. Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds new functionality)
• Breaking change (fix or feature that would break existing design and interface)
• Others (enhancement, documentation, validation, etc.)

Dependencies

• custom Streamlit UI
• calls out to OpenAI for embedding + chat completion

Tests

Describe the tests that you ran to verify your changes.

[rbrugaro]

@akollegger great to see you contribution! looking forward to trying it out.

I have been working on the below PRs to integrate GraphRAG (microsoft pipeline: graph extraction from text, clustering, community summaries, partial answers, final answer…) using llama-index and neo4j. Please check it out!
#1007
opea-project/GenAIComps#793

What do you think about these two approaches used jointly with an agent in real deployments? I am thinking: agent based on query type can decide if this is better answered by KG query or "microsoft graphRAG" for broader corpus theme/multihop queries. This can also be an efficient cost optimization strategy since "microsoft graphRAG" is more expensive with many more LLM calls.

@arun-gupta

[chensuyue]

We can't distribute a dataset in this repo. Better to ask user download from public. If it's a new dataset, we need to distribute somewhere else after the legal process done.

[akollegger]

We can't distribute a dataset in this repo. Better to ask user download from public. If it's a new dataset, we need to distribute somewhere else after the legal process done.

Understood. This data is the last stage of processing public data set from https://www.sec.gov/search-filings/edgar-search-assistance/accessing-edgar-data . I will look for a reasonable public host for the files.