support_description_long_description #183
Conversation
There was a problem hiding this comment.
Thank you for this first version. Some modifications are necessary.
Please use the "Fix ####" format in your first comment to link this PR to the original issue (see https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests)
Please also not that it might not be that clear in the original issue but we need to check for validity of provided description and long description as soon as possible. Same for other ZIM parameters (e.g. illustation). Here we check after get_content has been called, which is an issue because it means the scraper can spend hours to retrieve the whole content and only then fail because the description is too long (for instance). You need to find a way to change this and check that ZIM metadata (illustration, description, title, long_description, ...) are valid. Basically a call to validate_metadata must be made asap to check everything is ok before downloading all content.
openedx2zim/scraper.py
Outdated
| @@ -831,6 +842,24 @@ def render(self): | |||
| self.build_dir.joinpath("assets"), | |||
| ) | |||
|
|
|||
|
|
|||
| def handle_descriptions(self, default_description, description=None, long_description=None): | |||
There was a problem hiding this comment.
Instead of this custom code, please use the one from https://github.com/openzim/python-scraperlib/blob/4dc30126a54040b4383ffed3617a1394a51b5a78/src/zimscraperlib/inputs.py#L56
|
okay sure will make the necessary changes :) |
|
@benoit74 ,Regarding the check, before downloading the zim file may i add the check before |
This check must be done ASAP. To be honest, I think you shouldn't spend too much time on this issue, the scraper is not working anymore and complex, you won't be able to really test your change. Taking an issue from youtube / ted / freecodecamp / kolibri / zimfarm would be more appropriate. |
|
Closing this for now, feel free to reopen |
Fix #181
I have added a function called
handle_descriptionsthat takes in three inputs:default_description,description, andlong_description. It returns the description either aslong_descriptionordescription. I have also made changes in theget_zim_infofunction.Additionally, the newer version of
make_zim_filedoes not takefaviconas an input parameter, so I changed it toillustrations.