Merge pull request #170 from openzim/upgrade_deps

Upgrade dependencies especially zimscraperlib 3.x

.gitignore

@@ -375,8 +375,9 @@ src/ted2zim/templates/assets/videojs-ogvjs.js
 src/ted2zim/templates/assets/polyfills.js
 src/ted2zim/templates/assets/webp-hero.bundle.js
 
-# output dir
+# output and tmp dir
 output
+tmp
 
 # ignore all vscode, this is not standard configuration in this place
-.vscode
+.vscode

.pre-commit-config.yaml

@@ -7,15 +7,15 @@ repos:
   -   id: trailing-whitespace
   -   id: end-of-file-fixer
 - repo: https://github.com/psf/black
-  rev: "23.12.0"
+  rev: "24.3.0"
   hooks:
   -   id: black
 - repo: https://github.com/astral-sh/ruff-pre-commit
-  rev: v0.1.8
+  rev: v0.3.3
   hooks:
   - id: ruff
 - repo: https://github.com/RobertCraigie/pyright-python
-  rev: v1.1.341
+  rev: v1.1.354
   hooks:
   - id: pyright
     name: pyright (system)

CHANGELOG.md

@@ -7,9 +7,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased
 
+### Added
+
+- New `long_description` CLI argument to set the ZIM long description
+- New `disable_metadata_check` CLI argument to disable the metadata checks which are automated since zimscraperlib 3.x
+
 ### Changed
 
 - Changed default publisher metadata from 'Kiwix' to 'openZIM'
+- Validate ZIM metadata as early as possible
+- Migrate to zimscraperlib 3.3.2 (including **new VideoLowWebm encoder preset version 2**)
+- Upgrade Python dependencies, including migration to Python 3.12
+
+## Fixed
+
+- Fix language metadata computation (list, but not yet fully properly ordered)
+- Fix computation of automatic description and long description
 
 ## [2.1.0] - 2024-01-08
 

Dockerfile

@@ -1,4 +1,4 @@
-FROM python:3.11-slim-bookworm
+FROM python:3.12-slim-bookworm
 LABEL org.opencontainers.image.source https://github.com/openzim/ted
 
 # Install necessary packages
@@ -14,7 +14,7 @@ COPY entrypoint.sh /usr/local/bin/entrypoint.sh
 ENTRYPOINT ["entrypoint.sh"]
 
 # Copy pyproject.toml and its dependencies
-COPY pyproject.toml README.md hatch_build.py get_js_deps.sh /src/
+COPY pyproject.toml openzim.toml README.md /src/
 COPY src/ted2zim/__about__.py /src/src/ted2zim/__about__.py
 
 # Install Python dependencies

README.md

@@ -13,14 +13,17 @@ TED (Technology, Entertainment, Design) is a global set of conferences under the
 
 This project is aimed at creating a sustainable solution to make TED accessible offline by creating ZIM files providing these videos in a similar manner like online.
 
+`ted2zim` adheres to openZIM's [Contribution Guidelines](https://github.com/openzim/overview/wiki/Contributing).
+
+`ted2zim` has implemented openZIM's [Python bootstrap, conventions and policies](https://github.com/openzim/_python-bootstrap/docs/Policy.md) **v1.0.0**.
 
 ## Getting started :rocket:
 
 #### Install the dependencies
 Make sure that you have `python3`, `unzip`, `ffmpeg`, `wget` and `curl` installed on your system before running the scraper (otherwise you'll get a warning to install them).
 
 #### Setup the package
-One can easily install the PyPI version but let's setup the source version. 
+One can easily install the PyPI version but let's setup the source version.
 
 First, clone this repository.
 

openzim.toml

@@ -0,0 +1,45 @@
+[files.assets.config]
+target_dir="src/ted2zim/templates/assets"
+execute_after=[
+    "fix_ogvjs_dist .",
+]
+
+[files.assets.actions."video.js"]
+action="extract_all"
+source="https://github.com/videojs/video.js/releases/download/v7.8.1/video-js-7.8.1.zip"
+target_dir="videojs"
+remove = ["alt","examples",]
+
+[files.assets.actions."chosen.jquery.js"]
+action="extract_all"
+source="https://github.com/harvesthq/chosen/releases/download/v1.8.7/chosen_v1.8.7.zip"
+target_dir="chosen"
+remove = ["docsupport","chosen.proto.*","*.html","*.md"]
+
+[files.assets.actions."jquery.min.js"]
+action="get_file"
+source="https://code.jquery.com/jquery-3.5.1.min.js"
+target_file="jquery.min.js"
+
+[files.assets.actions."ogv.js"]
+action="extract_items"
+source="https://github.com/brion/ogv.js/releases/download/1.8.9/ogvjs-1.8.9.zip"
+zip_paths=["ogvjs-1.8.9"]
+target_paths=["ogvjs"]
+remove = ["ogvjs/COPYING","ogvjs/*.txt","ogvjs/*.md",]
+
+[files.assets.actions."videojs-ogvjs.js"]
+action="extract_items"
+source="https://github.com/hartman/videojs-ogvjs/archive/v1.3.1.zip"
+zip_paths=["videojs-ogvjs-1.3.1/dist/videojs-ogvjs.js"]
+target_paths=["videojs-ogvjs.js"]
+
+[files.assets.actions."webp-hero.polyfills.js"]
+action="get_file"
+source="https://unpkg.com/[email protected]/dist-cjs/polyfills.js"
+target_file="polyfills.js"
+
+[files.assets.actions."webp-hero.bundle.js"]
+action="get_file"
+source="https://unpkg.com/[email protected]/dist-cjs/webp-hero.bundle.js"
+target_file="webp-hero.bundle.js"