Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Errors by downloading files from Wikimedia #252

Closed
kelson42 opened this issue May 4, 2022 · 0 comments
Closed

Errors by downloading files from Wikimedia #252

kelson42 opened this issue May 4, 2022 · 0 comments
Assignees
Labels
Milestone

Comments

@kelson42
Copy link
Contributor

kelson42 commented May 4, 2022

[IMG-T-93::2022-05-02 08:18:34,806] DEBUG:Uploading https://i.stack.imgur.com/zob6i.png to S3::https/i.stack.imgur.com/zob6i.png with {'ident': '"0ebeff5f25a567ae84ea87723a2cac49"', 'encoder_version': '1'}
[IMG-T-50::2022-05-02 08:18:34,837] DEBUG:Uploading https://i.stack.imgur.com/L1RaK.gif to S3::https/i.stack.imgur.com/L1RaK.gif with {'ident': '"4bcef343431e3db5986288b41cddf714"', 'encoder_version': '1'}
[IMG-T-79::2022-05-02 08:18:35,009] ERROR:Failed to download/convert/optim source at https://upload.wikimedia.org/wikipedia/commons/thumb/0/0d/Hubble_ultra_deep_field_high_rez_edit1.jpg/600px-Hubble_ultra_deep_field_high_rez_edit1.jpg
[IMG-T-79::2022-05-02 08:18:35,009] ERROR:403 Client Error: Forbidden. Please comply with the User-Agent policy: https://meta.wikimedia.org/wiki/User-Agent_policy for url: https://upload.wikimedia.org/wikipedia/commons/thumb/0/0d/Hubble_ultra_deep_field_high_rez_edit1.jpg/600px-Hubble_ultra_deep_field_high_rez_edit1.jpg
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/sotoki-2.0.0-py3.8.egg/sotoki/utils/imager.py", line 296, in process_image
fileobj = self.get_image_data(url.geturl(), **resize_args)
File "/usr/local/lib/python3.8/site-packages/sotoki-2.0.0-py3.8.egg/sotoki/utils/imager.py", line 129, in get_image_data
stream_file(url=url, byte_stream=src)
File "/usr/local/lib/python3.8/site-packages/zimscraperlib/download.py", line 192, in stream_file
resp.raise_for_status()
File "/usr/local/lib/python3.8/site-packages/requests/models.py", line 960, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden. Please comply with the User-Agent policy: https://meta.wikimedia.org/wiki/User-Agent_policy for url: https://upload.wikimedia.org/wikipedia/commons/thumb/0/0d/Hubble_ultra_deep_field_high_rez_edit1.jpg/600px-Hubble_ultra_deep_field_high_rez_edit1.jpg
[IMG-T-88::2022-05-02 08:18:35,035] DEBUG:Uploading https://i.stack.imgur.com/hYkwm.png to S3::https/i.stack.imgur.com/hYkwm.png with {'ident': '"2d89fe432fd258cbea8ca6d7e68cc874"', 'encoder_version': '1'}
[IMG-T-40::2022-05-02 08:18:35,206] DEBUG:Resize Error for https://i.stack.imgur.com/AmUzD.jpg: 'Image is too small, Image size : 500, Required size : 540'

From https://farm.openzim.org/pipeline/21b0f5ee4634b387c9f8f626/debug

@kelson42 kelson42 added the bug label May 17, 2022
@kelson42 kelson42 added this to the 2.1.0 milestone May 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants