Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server timeout when prefetching a large amount of refs #1127

Open
martiert opened this issue Apr 26, 2024 · 0 comments
Open

Server timeout when prefetching a large amount of refs #1127

martiert opened this issue Apr 26, 2024 · 0 comments

Comments

@martiert
Copy link

client: vdirsyncer-0.19.2, also reproduced with HEAD of main branch.
server: davmail-6.2.0
python: 3.11.9
OS: Linux (NixOS)

debug: <CIMultiDictProxy('Server': 'DavMail Gateway 6.2.0-3464', 'DAV': '1, calendar-access, calendar-auto-schedule, calendarserver-private-events, addressbook', 'Date': 'Fri, 26 Apr 2024 07:13:21 +0000
', 'Expires': 'Fri, 26 Apr 2024 07:13:21 +0000', 'Cache-Control': 'private, max-age=0', 'Transfer-Encoding': 'chunked', 'Content-Type': 'text/xml;charset=UTF-8', 'Connection': 'keep-alive')>
debug: <StreamReader>
error: Unknown error occurred for calendar_cisco/calendar:
error: Use `-vdebug` to see the full traceback.
debug:   File "/nix/store/91zpapc0v9sy12f0im5kck4w7m3585q9-python3.11-vdirsyncer-0.19.2/lib/python3.11/site-packages/vdirsyncer/cli/tasks.py", line 72, in sync_collection
debug:     await sync.sync(
debug:   File "/nix/store/91zpapc0v9sy12f0im5kck4w7m3585q9-python3.11-vdirsyncer-0.19.2/lib/python3.11/site-packages/vdirsyncer/sync/__init__.py", line 146, in sync
debug:     a_nonempty = await a_info.prepare_new_status()
debug:                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
debug:   File "/nix/store/91zpapc0v9sy12f0im5kck4w7m3585q9-python3.11-vdirsyncer-0.19.2/lib/python3.11/site-packages/vdirsyncer/sync/__init__.py", line 66, in prepare_new_status
debug:     async for href, item, etag in self.storage.get_multi(prefetch):
debug:   File "/nix/store/91zpapc0v9sy12f0im5kck4w7m3585q9-python3.11-vdirsyncer-0.19.2/lib/python3.11/site-packages/vdirsyncer/storage/dav.py", line 529, in get_multi
debug:     await response.content.read()
debug:   File "/nix/store/2y4s33nck0v5x3grszrcb7c1vmbgg3gm-python3.11-aiohttp-3.9.3/lib/python3.11/site-packages/aiohttp/streams.py", line 373, in read
debug:     block = await self.readany()
debug:             ^^^^^^^^^^^^^^^^^^^^
debug:   File "/nix/store/2y4s33nck0v5x3grszrcb7c1vmbgg3gm-python3.11-aiohttp-3.9.3/lib/python3.11/site-packages/aiohttp/streams.py", line 395, in readany
debug:     await self._wait("readany")
debug:   File "/nix/store/2y4s33nck0v5x3grszrcb7c1vmbgg3gm-python3.11-aiohttp-3.9.3/lib/python3.11/site-packages/aiohttp/streams.py", line 301, in _wait
debug:     with self._timer:
debug:   File "/nix/store/2y4s33nck0v5x3grszrcb7c1vmbgg3gm-python3.11-aiohttp-3.9.3/lib/python3.11/site-packages/aiohttp/helpers.py", line 735, in __exit__
debug:     raise asyncio.TimeoutError from None

A potential fix for the issue can be found in https://github.com/martiert/vdirsyncer/tree/batch_prefetch which batches the requests into chunks of 20

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant