Fix #29

[DoodleBears]

Summary

• Close wrong split result when punctuations and digits are near each other #29

Now, when

merge_across_punctuation = True and merge_across_digit= False

merge_across_punctuation will [A, B, C]
if A is punctuation, it connect to B
if B is punctuation, it connect to A
if C is punctuation, it connect to B

text = "(2.2)术语和定义 (2.2)Terms and Definitions"

0|digit:(2.2)
1|zh:术语和定义 (
2|digit:2.2)
3|en:Terms and Definitions

merge_across_punctuation = False and merge_across_digit= True

merge_across_digit will [A, B, C]
if one of A or B is digit, it connect together

text = "(2.2)术语和定义 (2.2)Terms and Definitions"

0|zh:(2.2)术语和定义
1|en:(2.2)Terms and Definitions

merge_across_punctuation = True and merge_across_digit= True

text = "(2.2)术语和定义 (2.2)Terms and Definitions"

0|zh:(2.2)术语和定义
1|en:(2.2)Terms and Definitions