Storyboard fixes

[ftk]

Hopefully fixes issues mentioned in #23 and #24, as well as adds support for twitch VoDs storyboards yt-dlp/yt-dlp#4342 (newest yt-dlp version).

Limit thumbnails count for storyboards (separate option since it doesn't make thumbnail generation any faster)
Use XDG cache dir on unix for cache

[marzzzello]

Thanks a lot for this. The issues are fixed. However I can't get thumbnails for twitch videos
Any ideas? Here is the full log:

❯ mpv --msg-level=mpv_thumbnail_script_server=debug,mpv_thumbnail_script_client_osc=debug,ytdl_hook=debug https://www.twitch.tv/videos/1538353627
[ytdl_hook] Loading lua script @ytdl_hook.lua...
[ytdl_hook] loading mp.defaults
[mpv_thumbnail_script_client_osc] Loading lua script /home/marzzzello/.config/mpv/scripts/mpv_thumbnail_script_client_osc.lua...
[mpv_thumbnail_script_server] Loading lua script /home/marzzzello/.config/mpv/scripts/mpv_thumbnail_script_server.lua...
[ytdl_hook] loading @ytdl_hook.lua
[ytdl_hook] reading options for ytdl_hook 
[ytdl_hook] script-opts/ytdl_hook.conf not found. 
[ytdl_hook] lua-settings/ytdl_hook.conf not found. 
[mpv_thumbnail_script_client_osc] loading mp.defaults
[mpv_thumbnail_script_server] loading mp.defaults
[mpv_thumbnail_script_client_osc] loading file /home/marzzzello/.config/mpv/scripts/mpv_thumbnail_script_client_osc.lua
[mpv_thumbnail_script_server] loading file /home/marzzzello/.config/mpv/scripts/mpv_thumbnail_script_server.lua
[mpv_thumbnail_script_server] reading options for mpv_thumbnail_script 
[mpv_thumbnail_script_server] Opened config file script-opts/mpv_thumbnail_script.conf. 
[mpv_thumbnail_script_client_osc] reading options for mpv_thumbnail_script 
[mpv_thumbnail_script_client_osc] Opened config file script-opts/mpv_thumbnail_script.conf. 
[mpv_thumbnail_script_client_osc] reading options for osc 
[mpv_thumbnail_script_client_osc] script-opts/osc.conf not found. 
[mpv_thumbnail_script_client_osc] lua-settings/osc.conf not found. 
[ytdl_hook] ytdl:// hook 
[auto_profiles] Applying auto profile: twitch 
[ytdl_hook] not a ytdl:// url 
[mpv_thumbnail_script_client_osc] osc_init 
[mpv_thumbnail_script_client_osc] osc_init 
[mpv_thumbnail_script_server] Announcing self to master... 
[mpv_thumbnail_script_client_osc] Registered worker mpv_thumbnail_script_server 
[mpv_thumbnail_script_server] Successfully registered with master 
[ytdl_hook] full hook 
[ytdl_hook] No youtube-dl found with path yt-dlp in config directories 
[ytdl_hook] Running: yt-dlp --no-warnings -J --flat-playlist --sub-format ass/srt/best --format bestvideo+bestaudio/best --all-subs --no-playlist -- https://www.twitch.tv/videos/1538353627 
[ytdl_hook] Found youtube-dl with path yt-dlp in PATH 
[ytdl_hook] youtube-dl succeeded! 
[ytdl_hook] ytdl parsing took 0.002468 seconds 
[ytdl_hook] No fragments to join into EDL 
[ytdl_hook] format selection: youtube-dl (single) 
[ytdl_hook] streamurl: https://d1ymi26ma8va5x.cloudfront.net/0f7f63d452b7ee373b8a_therealknossi_39647150072_1658409956/chunked/index-dvr.m3u8 
[ytdl_hook] adding subtitle [rechat] 
[ytdl_hook] Adding pre-parsed chapters 
[ytdl_hook] script running time: 0.003748 seconds 
[mpv_thumbnail_script_client_osc] osc_init 
[mpv_thumbnail_script_client_osc] osc_init 
[ytdl_hook] Setting chapters 
 (+) Video --vid=1 (h264 1920x1080 59.940fps)
 (+) Audio --aid=1 (aac 2ch 48000Hz)
     Subs  --sid=1 --slang=rechat 'json' (null) (external)
[ffmpeg] AVHWDeviceContext: Failed to query surface attributes: 20 (the requested function is not implemented).
[vo/gpu/vaapi-egl] failed to retrieve libavutil frame constraints
Using hardware decoding (vaapi).
[mpv_thumbnail_script_client_osc] osc_init 
AO: [pulse] 48000Hz stereo 2ch float
VO: [gpu] 1920x1080 vaapi[nv12]
[mpv_thumbnail_script_client_osc] Gathering video/thumbnail state 
AV: 00:00:00 / 02:03:41 (0%) A-V:  0.000 Cache: 0.5s/438KB
[mpv_thumbnail_script_client_osc] Thumbnailer.state: {"thumbnails" = {-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1}, "worker_extra" = {}, "is_remote" = true, "thumbnail_count" = 62, "ready" = true, "available" = true, "thumbnail_directory" = "/tmp/mpv_thumbs_cache/1538353627-0", "thumbnail_template" = "/tmp/mpv_thumbs_cache/1538353627-0/%06d.bgra", "thumbnail_delta" = 120, "thumbnail_size" = {"w" = 400, "h" = 225}, "finished_thumbnails" = 0} 
[mpv_thumbnail_script_client_osc] Trying to get storyboard info... 
[mpv_thumbnail_script_client_osc] osc_init 
AV: 00:00:01 / 02:03:41 (0%) A-V:  0.000 Cache: 2.5s/2MB
[mpv_thumbnail_script_client_osc] Storyboard info acquired! 200 
[mpv_thumbnail_script_client_osc] Creating thumbnail directory /tmp/mpv_thumbs_cache/1538353627-0 
[mpv_thumbnail_script_client_osc] Splitting 200 thumbnails amongst 1 worker(s) 
[mpv_thumbnail_script_client_osc] Giving workers state: {"thumbnails":[-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1],"worker_input_path":"https://d1ymi26ma8va5x.cloudfront.net/0f7f63d452b7ee373b8a_therealknossi_39647150072_1658409956/chunked/index-dvr.m3u8","worker_extra":[],"storyboard":{"cols":5,"fragments":[{"duration":1855.250000,"url":"https://dgeft87wbj63p.cloudfront.net/0f7f63d452b7ee373b8a_therealknossi_39647150072_1658409956/storyboards/1538353627-high-0.jpg"},{"duration":1855.250000,"url":"https://dgeft87wbj63p.cloudfront.net/0f7f63d452b7ee373b8a_therealknossi_39647150072_1658409956/storyboards/1538353627-high-1.jpg"},{"duration":1855.250000,"url":"https://dgeft87wbj63p.cloudfront.net/0f7f63d452b7ee373b8a_therealknossi_39647150072_1658409956/storyboards/1538353627-high-2.jpg"},{"duration":1855.250000,"url":"https://dgeft87wbj63p.cloudfront.net/0f7f63d452b7ee373b8a_therealknossi_39647150072_1658409956/storyboards/1538353627-high-3.jpg"}],"divisor":1,"rows":10},"thumbnail_directory":"/tmp/mpv_thumbs_cache/1538353627-0","is_remote":true,"thumbnail_count":200,"ready":true,"available":true,"enabled":true,"thumbnail_template":"/tmp/mpv_thumbs_cache/1538353627-0/%06d.bgra","thumbnail_delta":37.105000,"thumbnail_size":{"w":220,"h":124},"finished_thumbnails":0} 
[mpv_thumbnail_script_client_osc] Assigning job to mpv_thumbnail_script_server [1,65,129,193,33,97,161,17,49,81,113,145,177,9,25,41,57,73,89,105,121,137,153,169,185,5,13,21,29,37,45,53,61,69,77,85,93,101,109,117,125,133,141,149,157,165,173,181,189,197,3,7,11,15,19,23,27,31,35,39,43,47,51,55,59,63,67,71,75,79,83,87,91,95,99,103,107,111,115,119,123,127,131,135,139,143,147,151,155,159,163,167,171,175,179,183,187,191,195,199,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78,80,82,84,86,88,90,92,94,96,98,100,102,104,106,108,110,112,114,116,118,120,122,124,126,128,130,132,134,136,138,140,142,144,146,148,150,152,154,156,158,160,162,164,166,168,170,172,174,176,178,180,182,184,186,188,190,192,194,196,198,200] 
[mpv_thumbnail_script_server] Handling given job 
[mpv_thumbnail_script_server] Generating 200 thumbnails @ 220x124 for "https://d1ymi26ma8va5x.cloudfront.net/0f7f63d452b7ee373b8a_therealknossi_39647150072_1658409956/chunked/index-dvr.m3u8" 
[mpv_thumbnail_script_server] Starting work on thumbnail 0 
AV: 00:00:01 / 02:03:41 (0%) A-V:  0.000 Cache: 3.0s/2MB
[mpv_thumbnail_script_server] Thumbnail suddenly disappeared! 
AV: 00:00:06 / 02:03:41 (0%) A-V:  0.000 Dropped: 1 Cache: 9.4s/7MB

Exiting... (Quit)
[ytdl_hook] Exiting...
[mpv_thumbnail_script_server] Exiting...
[mpv_thumbnail_script_client_osc] Exiting...

yt-dlp version is 2022.07.18

[ftk]

Strange, can't reproduce. Can you remove os.remove(atlas_path) line and check if 0.bgra.atlas is being downloaded and has correct size (5*10*220*124*4 bytes)?

[marzzzello]

The file 000000.bgra.atlas is empty (0 bytes)

[ftk]

Does the same happen with prefer_mpv=true, false? What does the log for atlas say? ( mpv_keep_logs=true is maybe needed)

[marzzzello]

prefer_mpv=no was the problem. I still had it in my config to reproduce another issue

[NicolaSmaniotto]

I've been using these commits (applying patches on top of 0.4.9) and everything seems to work correctly.

I'd say it is a good idea to merge this PR and release a new tagged version, since the code in the master branch currently has issues.

[marzzzello]

Sorry for the long wait >.<