Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use datadeps for AvgPerceptronTagger, add pos tagging over document types #166

Merged
merged 10 commits into from
Aug 12, 2019

Conversation

Ayushk4
Copy link
Member

@Ayushk4 Ayushk4 commented Aug 1, 2019

I have switched to using datadeps instead of storing weights locally, similar to the NER API.
As of now, I am directly taking it from version control history, but perhaps it could be released as an asset.

@Ayushk4 Ayushk4 changed the title Use datadeps for Avg Perceptron Tagger instead storing inside package Use datadeps for AvgPerceptronTagger instead storing weights inside package Aug 1, 2019
src/TextAnalysis.jl Outdated Show resolved Hide resolved
@aviks aviks mentioned this pull request Aug 10, 2019
3 tasks
@Ayushk4
Copy link
Member Author

Ayushk4 commented Aug 10, 2019

Thanks for the review, I will make the changes ASAP.

@Ayushk4
Copy link
Member Author

Ayushk4 commented Aug 10, 2019

@aviks I have made the suggested changes.

I also provided POS Tagger support over various Document types and String types, updated tests, docstrings and Documentation for the same.

Please review this (since there will be some merge conflicts with #167_Comment )

@Ayushk4
Copy link
Member Author

Ayushk4 commented Aug 10, 2019

As of now, the AvgPerceptronTagger is taking the weights from the git version control history (link). It may be neater to release the file as an asset on GitHub (similar to MetalHead.jl).

Also, various other NLP libraries like SpaCy use avg perceptron tagger from POS. Maybe the model weights from these libraries along with one we currently provide could be tested, and then the best one could be made available from this package. Once this is done, we can release the weights.

@aviks
Copy link
Member

aviks commented Aug 10, 2019

release the file as an asset on GitHub

I have added the file (zipped) into a release on this package.

https://github.com/JuliaText/TextAnalysis.jl/releases/download/v0.6.0/pretrainedMod.bson.zip

@Ayushk4
Copy link
Member Author

Ayushk4 commented Aug 11, 2019

I have changed the link to the Perceptrony Tagger weights.

@aviks aviks merged commit 284f11a into JuliaText:master Aug 12, 2019
@Ayushk4 Ayushk4 changed the title Use datadeps for AvgPerceptronTagger instead storing weights inside package Use datadeps for AvgPerceptronTagger, add pos tagging over document types Aug 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants