CLUE NER

Here is a short summary of our solution on CLUE NER benchmark.

CLUENER2020

The example of fine-tuning and doing inference on CLUENER2020 dataset with google_zh_model.bin:

python3 finetune/run_ner.py --pretrained_model_path models/google_zh_model.bin \
                            --vocab_path models/google_zh_vocab.txt \
                            --config_path models/bert/base_config.json \
                            --train_path datasets/cluener2020/train.tsv \
                            --dev_path datasets/cluener2020/dev.tsv \
                            --label2id_path datasets/cluener2020/label2id.json \
                            --output_model_path models/ner_model.bin \
                            --epochs_num 5 --batch_size 16

python3 inference/run_ner_infer.py --load_model_path models/ner_model.bin \
                                   --vocab_path models/google_zh_vocab.txt \
                                   --config_path models/bert/base_config.json \
                                   --test_path datasets/cluener2020/test_nolabel.tsv \
                                   --prediction_path datasets/cluener2020/prediction.tsv \
                                   --label2id_path datasets/cluener2020/label2id.json

The example of fine-tuning and doing inference on CLUENER2020 dataset with cluecorpussmall_roberta_wwm_large_seq512_model.bin:

python3 finetune/run_ner.py --pretrained_model_path models/cluecorpussmall_roberta_wwm_large_seq512_model.bin \
                            --vocab_path models/google_zh_vocab.txt \
                            --config_path models/bert/large_config.json \
                            --train_path datasets/cluener2020/train.tsv \
                            --dev_path datasets/cluener2020/dev.tsv \
                            --output_model_path models/ner_model.bin \
                            --label2id_path datasets/cluener2020/label2id.json \
                            --epochs_num 5 --batch_size 16

python3 inference/run_ner_infer.py --load_model_path models/ner_model.bin \
                                   --vocab_path models/google_zh_vocab.txt \
                                   --config_path models/bert/large_config.json \
                                   --test_path datasets/cluener2020/test_nolabel.tsv \
                                   --prediction_path datasets/cluener2020/prediction.tsv \
                                   --label2id_path datasets/cluener2020/label2id.json