too slow if training custom corps

[dxteam2024]

i have corps with 6000000 lines data , and i try to train the corps but it's too slow ,is there any ways to improve the performance in ChatterBot ?

[gunthercox]

Right now, no. Luckily training only has to be done once for a chat bot. You may encounter further issues once you finish training with that much data. I'm currently working on changes to solve performance issues for large data sets.

[neutralboy]

@aliray , if you don't mind can you please state where you acquired the data from?

[laonan]

Corps with 900000 lines data, use mongodb, super slow.

[jxfruit]

@laonan hello, have u solved the problem of efficiency. I meet the same problem

[laonan]

not yet, if u have any good news, would pls let me know? thx.

…

在 2017年9月30日，下午4:43，Lee Tao ***@***.***> 写道： @laonan <https://github.com/laonan> hello, have u solved the problem of efficiency. I meet the same problem — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#908 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABYyVo7otDYXF_A6xGfCTk_Ht8Tfp13Yks5snf9OgaJpZM4OwNqn>.

[jxfruit]

@laonan I have read the doc and asked the author. There is no good approach to efficiency and the author is working on it. The later version will get some changes and I may do some preprocessing before training to improve retrieve speed of response

[calmzealA]

Corps with 10000 lines data, use mongodb,
trainning once is acceptable.
But getresponse more than 2 second.

BestMatch get all 10000 lines and compute confidence each

[gunthercox]

I'm going to close this ticket off because as far as I can see there isn't any actionable items that can be done for it. For details on the planned performance improvements, please refer to #925.

[zxsimple]

@calmzealA has you installed python-Levenshtein?

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.