Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add deepseek-chat #222

Closed
zx9597446 opened this issue May 11, 2024 · 6 comments
Closed

feat: add deepseek-chat #222

zx9597446 opened this issue May 11, 2024 · 6 comments
Assignees
@f-aguzzi @VinciGit00
Labels
released on @dev released on @stable

Comments

@zx9597446
Copy link

please add deepseek-chat from deepseek.com

expose openai_base_url should work.

model name: deepseek-chat

they provide 5M token for free. thanks.
@VinciGit00 VinciGit00 self-assigned this May 11, 2024
@f-aguzzi
Copy link
Member

I'm working on this with @VinciGit00 's supervision. We'll figure this out soon.

@f-aguzzi f-aguzzi mentioned this issue May 11, 2024
Merged
@VinciGit00
Copy link
Collaborator

hi, it's in the pre/beta branch. install from there

github-actions bot pushed a commit that referenced this issue May 12, 2024
@semantic-release-bot
ci(release): 0.11.0-beta.3 [skip ci]
106fb12 
## [0.11.0-beta.3](v0.11.0-beta.2...v0.11.0-beta.3) (2024-05-12)

### Features

* add support for deepseek-chat ([156b67b](156b67b)), closes [#222](#222)

### Docs

* add diagram showing general structure/flow of the library ([13ae918](13ae918))
* update overview diagram with more models ([b441b30](b441b30))
@github-actions GitHub Actions
Copy link

🎉 This issue has been resolved in version 0.11.0-beta.3 🎉

The release is available on:

Your semantic-release bot 📦🚀

@VinciGit00
Copy link
Collaborator

check it out please https://github.com/VinciGit00/Scrapegraph-ai/blob/pre/beta/examples/deepseek/smart_scarper_deepseek.py

github-actions bot pushed a commit that referenced this issue May 14, 2024
@semantic-release-bot
ci(release): 0.11.0 [skip ci]
c55a3b1 
## [0.11.0](v0.10.1...v0.11.0) (2024-05-14)

### Features

* **parallel-exeuction:** add asyncio event loop dispatcher with semaphore for parallel graph instances ([627cbee](627cbee))
* **webdriver-backend:** add dynamic import scripts from module and file ([db2234b](db2234b))
* add gpt-4o ([52a4a3b](52a4a3b)), closes [#232](#232)
* add new prompt info ([e2350ed](e2350ed))
* **proxy-rotation:** add parse (IP address) or search (from broker) functionality for proxy rotation ([2170131](2170131))
* add support for deepseek-chat ([156b67b](156b67b)), closes [#222](#222)
* Add support for passing pdf path as source ([f10f3b1](f10f3b1))
* **omni-search:** added omni search graph and updated docs ([fcb3abb](fcb3abb))
* added proxy rotation ([0c36a7e](0c36a7e))
* **safe-web-driver:** enchanced the original `AsyncChromiumLoader` web driver with proxy protection and flexible kwargs and backend ([768719c](768719c))
* **gpt-4o:** image to text single node test ([90955ca](90955ca))
* revert fetch_node ([864aa91](864aa91))
* **batchsize:** tested different batch sizes and systems ([a8d5e7d](a8d5e7d))
* update info ([4ed0fb8](4ed0fb8))
* **omni-scraper:** working OmniScraperGraph with images ([a296927](a296927))

### Bug Fixes

* **pytest:** add dependency for mocking testing functions ([2f4fd45](2f4fd45))
* add json integration ([0ab31c3](0ab31c3))
* Augment the information getting fetched from a webpage ([f8ce3d5](f8ce3d5))
* bug for claude ([d0167de](d0167de))
* **fetch_node:** bug in handling local files ([a6e1813](a6e1813))
* **chromium-loader:** ensure it subclasses langchain's base loader ([b54d984](b54d984))
* fixed bugs for csv and xml ([324e977](324e977))
* limit python version to < 3.12 ([a37fbbc](a37fbbc))
* **proxy-rotation:** removed duplicated arg and passed the loader_kwarhs correctly to the node ([1e9a564](1e9a564))
* **fetch-node:** removed isSoup from default ([0c15947](0c15947))
* **proxy-rotation:** removed max_shape duplicate ([5d6d996](5d6d996))
* **asyncio:** replaced deepcopy with copy due to serialization problems ([dedc733](dedc733))

### chore

* update models_tokens.py with new model configurations ([d9752b1](d9752b1))

### Docs

* add diagram showing general structure/flow of the library ([13ae918](13ae918))
* **refactor:** added proxy-rotation usage and refactor readthedocs ([e256b75](e256b75))
* **refactor:** changed example ([c7ec114](c7ec114))
* **concurrent:** refactor theme and added benchmarck searchgraph ([ced2bbc](ced2bbc))
* update overview diagram with more models ([b441b30](b441b30))

### CI

* **release:** 0.10.0-beta.3 [skip ci] ([ad32298](ad32298))
* **release:** 0.10.0-beta.4 [skip ci] ([548bff9](548bff9))
* **release:** 0.10.0-beta.5 [skip ci] ([28c9dce](28c9dce))
* **release:** 0.10.0-beta.6 [skip ci] ([460d292](460d292))
* **release:** 0.11.0-beta.1 [skip ci] ([63c0dd9](63c0dd9))
* **release:** 0.11.0-beta.10 [skip ci] ([218b8ed](218b8ed)), closes [#232](#232)
* **release:** 0.11.0-beta.11 [skip ci] ([8727d03](8727d03))
* **release:** 0.11.0-beta.2 [skip ci] ([7ae50c0](7ae50c0))
* **release:** 0.11.0-beta.3 [skip ci] ([106fb12](106fb12)), closes [#222](#222)
* **release:** 0.11.0-beta.4 [skip ci] ([4ccddda](4ccddda))
* **release:** 0.11.0-beta.5 [skip ci] ([353382b](353382b))
* **release:** 0.11.0-beta.6 [skip ci] ([2724d3d](2724d3d))
* **release:** 0.11.0-beta.7 [skip ci] ([f0f7373](f0f7373))
* **release:** 0.11.0-beta.8 [skip ci] ([fa4edb4](fa4edb4))
* **release:** 0.11.0-beta.9 [skip ci] ([d2877d8](d2877d8))
@github-actions GitHub Actions
Copy link

🎉 This issue has been resolved in version 0.11.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@zx9597446
Copy link
Author

there are 2 issues here:

  1. abstract_graph.py needs to import DeepSeek:
    from scrapegraphai.models.deepseek import DeepSeek

  2. Deepseek no embedding provided, so examples needs local embedding instead.

@zx9597446 zx9597446 mentioned this issue May 21, 2024
Closed
rav-d added a commit to rav-d/scrapygrapy that referenced this issue Jul 30, 2024
@rav-d
ci(release): 0.11.0-beta.3 [skip ci]
cc7c659 
## [0.11.0-beta.3](ScrapeGraphAI/Scrapegraph-ai@v0.11.0-beta.2...v0.11.0-beta.3) (2024-05-12)

### Features

* add support for deepseek-chat ([156b67b](ScrapeGraphAI/Scrapegraph-ai@156b67b)), closes [#222](ScrapeGraphAI/Scrapegraph-ai#222)

### Docs

* add diagram showing general structure/flow of the library ([13ae918](ScrapeGraphAI/Scrapegraph-ai@13ae918))
* update overview diagram with more models ([b441b30](ScrapeGraphAI/Scrapegraph-ai@b441b30))
rav-d added a commit to rav-d/scrapygrapy that referenced this issue Jul 30, 2024
@rav-d
ci(release): 0.11.0 [skip ci]
dee65c4 
## [0.11.0](ScrapeGraphAI/Scrapegraph-ai@v0.10.1...v0.11.0) (2024-05-14)

### Features

* **parallel-exeuction:** add asyncio event loop dispatcher with semaphore for parallel graph instances ([627cbee](ScrapeGraphAI/Scrapegraph-ai@627cbee))
* **webdriver-backend:** add dynamic import scripts from module and file ([db2234b](ScrapeGraphAI/Scrapegraph-ai@db2234b))
* add gpt-4o ([52a4a3b](ScrapeGraphAI/Scrapegraph-ai@52a4a3b)), closes [#232](ScrapeGraphAI/Scrapegraph-ai#232)
* add new prompt info ([e2350ed](ScrapeGraphAI/Scrapegraph-ai@e2350ed))
* **proxy-rotation:** add parse (IP address) or search (from broker) functionality for proxy rotation ([2170131](ScrapeGraphAI/Scrapegraph-ai@2170131))
* add support for deepseek-chat ([156b67b](ScrapeGraphAI/Scrapegraph-ai@156b67b)), closes [#222](ScrapeGraphAI/Scrapegraph-ai#222)
* Add support for passing pdf path as source ([f10f3b1](ScrapeGraphAI/Scrapegraph-ai@f10f3b1))
* **omni-search:** added omni search graph and updated docs ([fcb3abb](ScrapeGraphAI/Scrapegraph-ai@fcb3abb))
* added proxy rotation ([0c36a7e](ScrapeGraphAI/Scrapegraph-ai@0c36a7e))
* **safe-web-driver:** enchanced the original `AsyncChromiumLoader` web driver with proxy protection and flexible kwargs and backend ([768719c](ScrapeGraphAI/Scrapegraph-ai@768719c))
* **gpt-4o:** image to text single node test ([90955ca](ScrapeGraphAI/Scrapegraph-ai@90955ca))
* revert fetch_node ([864aa91](ScrapeGraphAI/Scrapegraph-ai@864aa91))
* **batchsize:** tested different batch sizes and systems ([a8d5e7d](ScrapeGraphAI/Scrapegraph-ai@a8d5e7d))
* update info ([4ed0fb8](ScrapeGraphAI/Scrapegraph-ai@4ed0fb8))
* **omni-scraper:** working OmniScraperGraph with images ([a296927](ScrapeGraphAI/Scrapegraph-ai@a296927))

### Bug Fixes

* **pytest:** add dependency for mocking testing functions ([2f4fd45](ScrapeGraphAI/Scrapegraph-ai@2f4fd45))
* add json integration ([0ab31c3](ScrapeGraphAI/Scrapegraph-ai@0ab31c3))
* Augment the information getting fetched from a webpage ([f8ce3d5](ScrapeGraphAI/Scrapegraph-ai@f8ce3d5))
* bug for claude ([d0167de](ScrapeGraphAI/Scrapegraph-ai@d0167de))
* **fetch_node:** bug in handling local files ([a6e1813](ScrapeGraphAI/Scrapegraph-ai@a6e1813))
* **chromium-loader:** ensure it subclasses langchain's base loader ([b54d984](ScrapeGraphAI/Scrapegraph-ai@b54d984))
* fixed bugs for csv and xml ([324e977](ScrapeGraphAI/Scrapegraph-ai@324e977))
* limit python version to < 3.12 ([a37fbbc](ScrapeGraphAI/Scrapegraph-ai@a37fbbc))
* **proxy-rotation:** removed duplicated arg and passed the loader_kwarhs correctly to the node ([1e9a564](ScrapeGraphAI/Scrapegraph-ai@1e9a564))
* **fetch-node:** removed isSoup from default ([0c15947](ScrapeGraphAI/Scrapegraph-ai@0c15947))
* **proxy-rotation:** removed max_shape duplicate ([5d6d996](ScrapeGraphAI/Scrapegraph-ai@5d6d996))
* **asyncio:** replaced deepcopy with copy due to serialization problems ([dedc733](ScrapeGraphAI/Scrapegraph-ai@dedc733))

### chore

* update models_tokens.py with new model configurations ([d9752b1](ScrapeGraphAI/Scrapegraph-ai@d9752b1))

### Docs

* add diagram showing general structure/flow of the library ([13ae918](ScrapeGraphAI/Scrapegraph-ai@13ae918))
* **refactor:** added proxy-rotation usage and refactor readthedocs ([e256b75](ScrapeGraphAI/Scrapegraph-ai@e256b75))
* **refactor:** changed example ([c7ec114](ScrapeGraphAI/Scrapegraph-ai@c7ec114))
* **concurrent:** refactor theme and added benchmarck searchgraph ([ced2bbc](ScrapeGraphAI/Scrapegraph-ai@ced2bbc))
* update overview diagram with more models ([b441b30](ScrapeGraphAI/Scrapegraph-ai@b441b30))

### CI

* **release:** 0.10.0-beta.3 [skip ci] ([ad32298](ScrapeGraphAI/Scrapegraph-ai@ad32298))
* **release:** 0.10.0-beta.4 [skip ci] ([548bff9](ScrapeGraphAI/Scrapegraph-ai@548bff9))
* **release:** 0.10.0-beta.5 [skip ci] ([28c9dce](ScrapeGraphAI/Scrapegraph-ai@28c9dce))
* **release:** 0.10.0-beta.6 [skip ci] ([460d292](ScrapeGraphAI/Scrapegraph-ai@460d292))
* **release:** 0.11.0-beta.1 [skip ci] ([63c0dd9](ScrapeGraphAI/Scrapegraph-ai@63c0dd9))
* **release:** 0.11.0-beta.10 [skip ci] ([218b8ed](ScrapeGraphAI/Scrapegraph-ai@218b8ed)), closes [#232](ScrapeGraphAI/Scrapegraph-ai#232)
* **release:** 0.11.0-beta.11 [skip ci] ([8727d03](ScrapeGraphAI/Scrapegraph-ai@8727d03))
* **release:** 0.11.0-beta.2 [skip ci] ([7ae50c0](ScrapeGraphAI/Scrapegraph-ai@7ae50c0))
* **release:** 0.11.0-beta.3 [skip ci] ([106fb12](ScrapeGraphAI/Scrapegraph-ai@106fb12)), closes [#222](ScrapeGraphAI/Scrapegraph-ai#222)
* **release:** 0.11.0-beta.4 [skip ci] ([4ccddda](ScrapeGraphAI/Scrapegraph-ai@4ccddda))
* **release:** 0.11.0-beta.5 [skip ci] ([353382b](ScrapeGraphAI/Scrapegraph-ai@353382b))
* **release:** 0.11.0-beta.6 [skip ci] ([2724d3d](ScrapeGraphAI/Scrapegraph-ai@2724d3d))
* **release:** 0.11.0-beta.7 [skip ci] ([f0f7373](ScrapeGraphAI/Scrapegraph-ai@f0f7373))
* **release:** 0.11.0-beta.8 [skip ci] ([fa4edb4](ScrapeGraphAI/Scrapegraph-ai@fa4edb4))
* **release:** 0.11.0-beta.9 [skip ci] ([d2877d8](ScrapeGraphAI/Scrapegraph-ai@d2877d8))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@f-aguzzi f-aguzzi

@VinciGit00 VinciGit00

Labels
released on @dev released on @stable
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@zx9597446 @f-aguzzi @VinciGit00