Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

目标端MongoDB4.2 无法同步 #552

Closed
misstick86 opened this issue Mar 4, 2021 · 1 comment
Closed

目标端MongoDB4.2 无法同步 #552

misstick86 opened this issue Mar 4, 2021 · 1 comment

Comments

@misstick86
Copy link

同步架构

mongodb4.0 --> shake 2.4.18 --> mongodb 4.2

场景

目标端自建docker MongoDB4.2

增量同步一开始没有问题,之后出现io/timeout问题.
[2021/03/04 15:47:18 CST] [INFO] [name=mgset-16700847, stage=full, get=296786, write_success=296786, tps=0] [2021/03/04 15:47:23 CST] [INFO] [name=mgset-16700847, stage=full, get=296786, write_success=296786, tps=0] [2021/03/04 15:47:28 CST] [INFO] [name=mgset-16700847, stage=full, get=296786, write_success=296786, tps=0] [2021/03/04 15:47:33 CST] [WARN] insert docs with length[128] into ns[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] of dest mongo failed[index[0], msg[read tcp 192.168.0.40:56852->192.168.0.3:27017: i/o timeout], dup[false]] [2021/03/04 15:47:33 CST] [CRIT] DocExecutor[1632] collectionExecutor[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] sync failed: bulk run message[[{_id 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {records [[{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail dongchengwei18@gmail.com} {chapterId 034cd33d-4984-414d-ab0f-89ea1575346f} {chapterName C42 Venomous Spider of the Western Regions} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 39bc4aee-292e-4d8f-b92c-5f295a663c2b} {expireDate 2020-01-09 17:15:38.639 +0000 UTC}] [{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail dongchengwei18@gmail.com} {chapterId 129c9967-a35d-4b43-9923-8d71b1d0a084} {chapterName C43 Life-Suspended Line} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id b42c6f3b-3937-4783-91cf-780d693cf2e4} {expireDate 2020-01-09 17:20:02.512 +0000 UTC}]]}]], failed[read tcp 192.168.0.40:56852->192.168.0.3:27017: i/o timeout], index[0] dup[false] [15:47:33 CST 2021/03/04] [CRIT] (mongoshake/collector/docsyncer.(*DocExecutor).start:157) DocExecutor[1632] collectionExecutor[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] sync failed: bulk run message[[{_id 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {records [[{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail dongchengwei18@gmail.com} {chapterId 034cd33d-4984-414d-ab0f-89ea1575346f} {chapterName C42 Venomous Spider of the Western Regions} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 39bc4aee-292e-4d8f-b92c-5f295a663c2b} {expireDate 2020-01-09 17:15:38.639 +0000 UTC}] [{userId 0123bebb-6a58-4c2b-ada3-a3a5126ca818} {userName dongchengwei18} {userEmail dongchengwei18@gmail.com} {chapterId 129c9967-a35d-4b43-9923-8d71b1d0a084} {chapterName C43 Life-Suspended Line} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id b42c6f3b-3937-4783-91cf-780d693cf2e4} {expireDate 2020-01-09 17:20:02.512 +0000 UTC}]]}]], failed[read tcp 192.168.0.40:56852->192.168.0.3:27017: i/o timeout], index[0] dup[false] [2021/03/04 15:47:33 CST] [WARN] insert docs with length[83] into ns[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] of dest mongo failed[index[0], msg[read tcp 192.168.0.40:56926->192.168.0.3:27017: i/o timeout], dup[false]] [2021/03/04 15:47:33 CST] [CRIT] DocExecutor[1633] collectionExecutor[{lemon_novel borrowed_chapter_0e80210d-3c5c-41e2-b859-20f30aa5db51}] sync failed: bulk run message[[{_id a1378d95-7117-475a-be94-160d1dd4d5ef} {records [[{userId a1378d95-7117-475a-be94-160d1dd4d5ef} {userName topaksina33} {userEmail topaksina33@gmail.com} {chapterId 1ae05879-3724-4e68-a7e4-41a5b70e5832} {chapterName C73 Crown Prince's birthday party} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 18929b3c-7618-476c-8539-705ebe70e9c8} {expireDate 2020-01-10 01:08:05.626 +0000 UTC}] [{userId a1378d95-7117-475a-be94-160d1dd4d5ef} {userName topaksina33} {userEmail topaksina33@gmail.com} {chapterId 89e1a3bc-ef3f-42c6-8cc8-fc7e36c18909} {chapterName C74 Bai Jiajia} {bookId 0e80210d-3c5c-41e2-b859-20f30aa5db51} {bookName Reborn Empress vs. Devil Crown Prince} {_id 150b6a19-24d0-48e0-a6b0-78a61612e209} {expireDate 2020-01-10 02:33:53.985 +0000 UTC}]]}]], failed[read tcp 192.168.0.40:56926->192.168.0.3:27017: i/o timeout], index[0] dup[false] panic: send on closed channel

目标端为阿里云MongoDB4.2

该场景下shake无法正常工作,一直开着这个页面。
image

@vinllen
Copy link
Collaborator

vinllen commented Mar 5, 2021

已经定位是网络问题,shake到目的端跨全球同步,延迟过大丢包率高导致

@vinllen vinllen closed this as completed Mar 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants