We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
mongodb4.0 --> shake 2.4.18 --> mongodb 4.2
增量同步一开始没有问题,之后出现io/timeout问题. [2021/03/04 15:47:18 CST] [INFO] [name=mgset-16700847, stage=full, get=296786, write_success=296786, tps=0] [2021/03/04 15:47:23 CST] [INFO] [name=mgset-16700847, stage=full, get=296786, write_success=296786, tps=0] [2021/03/04 15:47:28 CST] [INFO] [name=mgset-16700847, stage=full, get=296786, write_success=296786, tps=0] [2021/03/04 15:47:33 CST] [WARN] insert docs with length[128] into ns[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] of dest mongo failed[index[0], msg[read tcp 192.168.0.40:56852->192.168.0.3:27017: i/o timeout], dup[false]] [2021/03/04 15:47:33 CST] [CRIT] DocExecutor[1632] collectionExecutor[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] sync failed: bulk run message[[{_id 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {records [[{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail [email protected]} {chapterId 034cd33d-4984-414d-ab0f-89ea1575346f} {chapterName C42 Venomous Spider of the Western Regions} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 39bc4aee-292e-4d8f-b92c-5f295a663c2b} {expireDate 2020-01-09 17:15:38.639 +0000 UTC}] [{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail [email protected]} {chapterId 129c9967-a35d-4b43-9923-8d71b1d0a084} {chapterName C43 Life-Suspended Line} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id b42c6f3b-3937-4783-91cf-780d693cf2e4} {expireDate 2020-01-09 17:20:02.512 +0000 UTC}]]}]], failed[read tcp 192.168.0.40:56852->192.168.0.3:27017: i/o timeout], index[0] dup[false] [15:47:33 CST 2021/03/04] [CRIT] (mongoshake/collector/docsyncer.(*DocExecutor).start:157) DocExecutor[1632] collectionExecutor[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] sync failed: bulk run message[[{_id 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {records [[{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail [email protected]} {chapterId 034cd33d-4984-414d-ab0f-89ea1575346f} {chapterName C42 Venomous Spider of the Western Regions} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 39bc4aee-292e-4d8f-b92c-5f295a663c2b} {expireDate 2020-01-09 17:15:38.639 +0000 UTC}] [{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail [email protected]} {chapterId 129c9967-a35d-4b43-9923-8d71b1d0a084} {chapterName C43 Life-Suspended Line} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id b42c6f3b-3937-4783-91cf-780d693cf2e4} {expireDate 2020-01-09 17:20:02.512 +0000 UTC}]]}]], failed[read tcp 192.168.0.40:56852->192.168.0.3:27017: i/o timeout], index[0] dup[false] [2021/03/04 15:47:33 CST] [WARN] insert docs with length[83] into ns[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] of dest mongo failed[index[0], msg[read tcp 192.168.0.40:56926->192.168.0.3:27017: i/o timeout], dup[false]] [2021/03/04 15:47:33 CST] [CRIT] DocExecutor[1633] collectionExecutor[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] sync failed: bulk run message[[{_id a1378d95-7117-475a-be94-160d1dd4d5ef} {records [[{userId a1378d95-7117-475a-be94-160d1dd4d5ef} {userName topaksina33} {userEmail [email protected]} {chapterId 1ae05879-3724-4e68-a7e4-41a5b70e5832} {chapterName C73 Crown Prince's birthday party} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 18929b3c-7618-476c-8539-705ebe70e9c8} {expireDate 2020-01-10 01:08:05.626 +0000 UTC}] [{userId a1378d95-7117-475a-be94-160d1dd4d5ef} {userName topaksina33} {userEmail [email protected]} {chapterId 89e1a3bc-ef3f-42c6-8cc8-fc7e36c18909} {chapterName C74 Bai Jiajia} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 150b6a19-24d0-48e0-a6b0-78a61612e209} {expireDate 2020-01-10 02:33:53.985 +0000 UTC}]]}]], failed[read tcp 192.168.0.40:56926->192.168.0.3:27017: i/o timeout], index[0] dup[false] panic: send on closed channel
[2021/03/04 15:47:18 CST] [INFO] [name=mgset-16700847, stage=full, get=296786, write_success=296786, tps=0] [2021/03/04 15:47:23 CST] [INFO] [name=mgset-16700847, stage=full, get=296786, write_success=296786, tps=0] [2021/03/04 15:47:28 CST] [INFO] [name=mgset-16700847, stage=full, get=296786, write_success=296786, tps=0] [2021/03/04 15:47:33 CST] [WARN] insert docs with length[128] into ns[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] of dest mongo failed[index[0], msg[read tcp 192.168.0.40:56852->192.168.0.3:27017: i/o timeout], dup[false]] [2021/03/04 15:47:33 CST] [CRIT] DocExecutor[1632] collectionExecutor[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] sync failed: bulk run message[[{_id 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {records [[{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail [email protected]} {chapterId 034cd33d-4984-414d-ab0f-89ea1575346f} {chapterName C42 Venomous Spider of the Western Regions} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 39bc4aee-292e-4d8f-b92c-5f295a663c2b} {expireDate 2020-01-09 17:15:38.639 +0000 UTC}] [{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail [email protected]} {chapterId 129c9967-a35d-4b43-9923-8d71b1d0a084} {chapterName C43 Life-Suspended Line} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id b42c6f3b-3937-4783-91cf-780d693cf2e4} {expireDate 2020-01-09 17:20:02.512 +0000 UTC}]]}]], failed[read tcp 192.168.0.40:56852->192.168.0.3:27017: i/o timeout], index[0] dup[false] [15:47:33 CST 2021/03/04] [CRIT] (mongoshake/collector/docsyncer.(*DocExecutor).start:157) DocExecutor[1632] collectionExecutor[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] sync failed: bulk run message[[{_id 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {records [[{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail [email protected]} {chapterId 034cd33d-4984-414d-ab0f-89ea1575346f} {chapterName C42 Venomous Spider of the Western Regions} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 39bc4aee-292e-4d8f-b92c-5f295a663c2b} {expireDate 2020-01-09 17:15:38.639 +0000 UTC}] [{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail [email protected]} {chapterId 129c9967-a35d-4b43-9923-8d71b1d0a084} {chapterName C43 Life-Suspended Line} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id b42c6f3b-3937-4783-91cf-780d693cf2e4} {expireDate 2020-01-09 17:20:02.512 +0000 UTC}]]}]], failed[read tcp 192.168.0.40:56852->192.168.0.3:27017: i/o timeout], index[0] dup[false] [2021/03/04 15:47:33 CST] [WARN] insert docs with length[83] into ns[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] of dest mongo failed[index[0], msg[read tcp 192.168.0.40:56926->192.168.0.3:27017: i/o timeout], dup[false]] [2021/03/04 15:47:33 CST] [CRIT] DocExecutor[1633] collectionExecutor[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] sync failed: bulk run message[[{_id a1378d95-7117-475a-be94-160d1dd4d5ef} {records [[{userId a1378d95-7117-475a-be94-160d1dd4d5ef} {userName topaksina33} {userEmail [email protected]} {chapterId 1ae05879-3724-4e68-a7e4-41a5b70e5832} {chapterName C73 Crown Prince's birthday party} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 18929b3c-7618-476c-8539-705ebe70e9c8} {expireDate 2020-01-10 01:08:05.626 +0000 UTC}] [{userId a1378d95-7117-475a-be94-160d1dd4d5ef} {userName topaksina33} {userEmail [email protected]} {chapterId 89e1a3bc-ef3f-42c6-8cc8-fc7e36c18909} {chapterName C74 Bai Jiajia} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 150b6a19-24d0-48e0-a6b0-78a61612e209} {expireDate 2020-01-10 02:33:53.985 +0000 UTC}]]}]], failed[read tcp 192.168.0.40:56926->192.168.0.3:27017: i/o timeout], index[0] dup[false] panic: send on closed channel
该场景下shake无法正常工作,一直开着这个页面。
The text was updated successfully, but these errors were encountered:
已经定位是网络问题,shake到目的端跨全球同步,延迟过大丢包率高导致
Sorry, something went wrong.
No branches or pull requests
同步架构
mongodb4.0 --> shake 2.4.18 --> mongodb 4.2
场景
目标端自建docker MongoDB4.2
增量同步一开始没有问题,之后出现io/timeout问题.
[2021/03/04 15:47:18 CST] [INFO] [name=mgset-16700847, stage=full, get=296786, write_success=296786, tps=0] [2021/03/04 15:47:23 CST] [INFO] [name=mgset-16700847, stage=full, get=296786, write_success=296786, tps=0] [2021/03/04 15:47:28 CST] [INFO] [name=mgset-16700847, stage=full, get=296786, write_success=296786, tps=0] [2021/03/04 15:47:33 CST] [WARN] insert docs with length[128] into ns[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] of dest mongo failed[index[0], msg[read tcp 192.168.0.40:56852->192.168.0.3:27017: i/o timeout], dup[false]] [2021/03/04 15:47:33 CST] [CRIT] DocExecutor[1632] collectionExecutor[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] sync failed: bulk run message[[{_id 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {records [[{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail [email protected]} {chapterId 034cd33d-4984-414d-ab0f-89ea1575346f} {chapterName C42 Venomous Spider of the Western Regions} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 39bc4aee-292e-4d8f-b92c-5f295a663c2b} {expireDate 2020-01-09 17:15:38.639 +0000 UTC}] [{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail [email protected]} {chapterId 129c9967-a35d-4b43-9923-8d71b1d0a084} {chapterName C43 Life-Suspended Line} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id b42c6f3b-3937-4783-91cf-780d693cf2e4} {expireDate 2020-01-09 17:20:02.512 +0000 UTC}]]}]], failed[read tcp 192.168.0.40:56852->192.168.0.3:27017: i/o timeout], index[0] dup[false] [15:47:33 CST 2021/03/04] [CRIT] (mongoshake/collector/docsyncer.(*DocExecutor).start:157) DocExecutor[1632] collectionExecutor[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] sync failed: bulk run message[[{_id 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {records [[{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail [email protected]} {chapterId 034cd33d-4984-414d-ab0f-89ea1575346f} {chapterName C42 Venomous Spider of the Western Regions} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 39bc4aee-292e-4d8f-b92c-5f295a663c2b} {expireDate 2020-01-09 17:15:38.639 +0000 UTC}] [{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail [email protected]} {chapterId 129c9967-a35d-4b43-9923-8d71b1d0a084} {chapterName C43 Life-Suspended Line} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id b42c6f3b-3937-4783-91cf-780d693cf2e4} {expireDate 2020-01-09 17:20:02.512 +0000 UTC}]]}]], failed[read tcp 192.168.0.40:56852->192.168.0.3:27017: i/o timeout], index[0] dup[false] [2021/03/04 15:47:33 CST] [WARN] insert docs with length[83] into ns[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] of dest mongo failed[index[0], msg[read tcp 192.168.0.40:56926->192.168.0.3:27017: i/o timeout], dup[false]] [2021/03/04 15:47:33 CST] [CRIT] DocExecutor[1633] collectionExecutor[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] sync failed: bulk run message[[{_id a1378d95-7117-475a-be94-160d1dd4d5ef} {records [[{userId a1378d95-7117-475a-be94-160d1dd4d5ef} {userName topaksina33} {userEmail [email protected]} {chapterId 1ae05879-3724-4e68-a7e4-41a5b70e5832} {chapterName C73 Crown Prince's birthday party} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 18929b3c-7618-476c-8539-705ebe70e9c8} {expireDate 2020-01-10 01:08:05.626 +0000 UTC}] [{userId a1378d95-7117-475a-be94-160d1dd4d5ef} {userName topaksina33} {userEmail [email protected]} {chapterId 89e1a3bc-ef3f-42c6-8cc8-fc7e36c18909} {chapterName C74 Bai Jiajia} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 150b6a19-24d0-48e0-a6b0-78a61612e209} {expireDate 2020-01-10 02:33:53.985 +0000 UTC}]]}]], failed[read tcp 192.168.0.40:56926->192.168.0.3:27017: i/o timeout], index[0] dup[false] panic: send on closed channel
目标端为阿里云MongoDB4.2
该场景下shake无法正常工作,一直开着这个页面。
The text was updated successfully, but these errors were encountered: