V2.2.0

Compatibility update for torch 2.1. Now supports newer versions (for neural models for Hebrew):

scikit-learn==1.3.2
joblib==1.3.2
pandas==2.1.2
xgboost==2.0.3
flair==0.13.0

V2.0.0 - add transformer based features

• add transformer based features

V1.1.0 - sklearn 0.23 and joblib

Compatibility update for sklearn 0.23 + joblib

V1.0.1 - Arabic and setup for PyPI

• Added Arabic model
• Re-arranged folder setup for PyPI deployment

V1.0.0 - new and improved classifier

• Single file implementation, no classes for multilabel encoder etc.
• Switch classifier to xgboost
• Add interface for getting probabilities instead of predictions
• Add hyperparameter optimization code
• Various minor improvements

V0.9.0 - first release of RFTokenizer

For more details see:

Zeldes, Amir (2018) A Characterwise Windowed Approach to Hebrew Morphological Segmentation. In: Proceedings of the 15th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology. Brussels, Belgium.

https://arxiv.org/abs/1808.07214