Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KRL的数据读入报错 examples = [make_example(line, fields) for line in reader]报错,请问怎么解决 #5

Open
XuanYang1991 opened this issue Nov 13, 2019 · 4 comments

Comments

@XuanYang1991
Copy link

XuanYang1991 commented Nov 13, 2019

_____________________ ERROR collecting test_krl_transe.py _____________________
test_krl_transe.py:26: in
krl.train(train_path, model_type=model_type, dev_path=train_path, save_path='./krl_{}_saves'.format(model_type))
..........\Anaconda3\envs\python367\lib\site-packages\lightkg\krl\module.py:29: in train
train_dataset = rl_tool.get_dataset(train_path)
..........\Anaconda3\envs\python367\lib\site-packages\lightkg\krl\tool.py:31: in get_dataset
rl_dataset = TabularDataset(path, format=file_type, fields=fields, skip_header=skip_header)
..........\Anaconda3\envs\python367\lib\site-packages\torchtext\data\dataset.py:271: in init
examples = [make_example(line, fields) for line in reader]
..........\Anaconda3\envs\python367\lib\site-packages\torchtext\data\dataset.py:271: in
examples = [make_example(line, fields) for line in reader]
..........\Anaconda3\envs\python367\lib\site-packages\torchtext\utils.py:130: in unicode_csv_reader
csv.field_size_limit(sys.maxsize)
E OverflowError: Python int too large to convert to C long
------------------------------- Captured stderr -------------------------------
[2019-11-13 16:58:36] [INFO] [MainThread] [tool.py:30] loading dataset from C:\Users\Administrator\work\jupyter\study\Untitled Folder\kre\train.sample.csv
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!
=========================== 1 error in 3.36 seconds ===========================
Process finished with exit code 0
我的数据和你的一样

科学,包涵,自然、社会、思维等领域
科学,外文名,science
科学,拼音,kē xué
科学,中文名,科学
科学,解释,发现、积累的真理的运用与实践
语法学,外文名,syntactics
语法学,中文名,语法学
物理宇宙学,对象,大尺度结构和宇宙形成
物理宇宙学,时间,二十世纪
物理宇宙学,所属,天体物理学
物理宇宙学,中文名,物理宇宙学
曹禺,出生地,天津
曹禺,毕业院校,清华大学
曹禺,逝世日期,1996年(丙子年)12月13日
曹禺,中文名,万家宝
曹禺,国籍,中国

@XuanYang1991 XuanYang1991 changed the title KRL的数据读入报错,能够发点原始数据,谢谢 KRL的数据读入报错 examples = [make_example(line, fields) for line in reader]报错,请问怎么解决 Nov 13, 2019
@smilelight
Copy link
Owner

你的文件是什么格式的,是utf-8吗

@XuanYang1991
Copy link
Author

是的,utf-8,gbk2312等尝试过

@May-Sunshine
Copy link

请问问题解决了吗?我跟你一样的错误

@smilelight
Copy link
Owner

@May-Sunshine 看这报的错误,应该不是框架的问题吧?你可以网上搜一搜,看看能不能找到类似的解决方案。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants