Failing to fetch VOD ID after livestream ends causes queue to fail

[russelg]

While testing the fix for #354, my internet went out :/. That caused the live video downloads to end, which is expected. But even though the video download task in the queue is marked complete, it's not moving on to video convert/move. And chat download is stuck in progress still.

Looks like the video download workflow failed due to the fetch VOD ID failing. Perhaps we can just ignore that failure? There's already handling for when the VOD ID isn't found, we can probably use the same logic in the case the fetch fails.

Most relevant line:

{"level":"error","ActivityType":"DownloadTwitchLiveVideo","Attempt":1,"Error":"failed to get twitch videos: Get \"https://api.twitch.tv/helix/videos?user_id=90937535&type=archive&first=100\": dial tcp: lookup api.twitch.tv on 127.0.0.11:53: read udp 127.0.0.1:41249->127.0.0.11:53: i/o timeout","Namespace":"default","RunID":"d9e1e6ad-08aa-4efe-b883-e76a30fa4d86","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"6a89acb8-12eb-4b80-a595-8155f9fcde77_5","time":"2024-01-15T09:45:09+08:00","message":"Activity error."}

Full log:

{"level":"debug","time":"2024-01-15T09:37:55+08:00","message":"workflow id b45d1f90-b346-11ee-a650-0242ac12000f started for live stream 43418657787"}
{"level":"debug","Attempt":1,"ChildWorkflowID":"148902df-f5e8-4456-bcdd-3e78cf9936dc_5","Namespace":"default","RunID":"148902df-f5e8-4456-bcdd-3e78cf9936dc","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"b45d1f90-b346-11ee-a650-0242ac12000f","WorkflowType":"CreateDirectoryWorkflow","time":"2024-01-15T09:37:55+08:00","message":"ExecuteChildWorkflow"}
{"level":"debug","ActivityID":"5","ActivityType":"CreateDirectory","Attempt":1,"Namespace":"default","RunID":"99d578de-03b1-41e4-8830-5a6d05017d66","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"148902df-f5e8-4456-bcdd-3e78cf9936dc_5","WorkflowType":"CreateDirectoryWorkflow","time":"2024-01-15T09:37:55+08:00","message":"ExecuteActivity"}
{"level":"debug","time":"2024-01-15T09:37:55+08:00","message":"creating folder: lyricaldota/2024-01-15 - (LIVE)_Nouns_vs_BammySoy [43418657787][live][b45d1f90-b346-11ee-a650-0242ac12000f]"}
{"level":"debug","Attempt":1,"ChildWorkflowID":"148902df-f5e8-4456-bcdd-3e78cf9936dc_14","Namespace":"default","RunID":"148902df-f5e8-4456-bcdd-3e78cf9936dc","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"b45d1f90-b346-11ee-a650-0242ac12000f","WorkflowType":"DownloadTwitchLiveThumbnailsWorkflow","time":"2024-01-15T09:37:55+08:00","message":"ExecuteChildWorkflow"}
{"level":"debug","ActivityID":"5","ActivityType":"DownloadTwitchLiveThumbnails","Attempt":1,"Namespace":"default","RunID":"3386fbbf-4186-42ec-8fd8-135da3bacf5e","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"148902df-f5e8-4456-bcdd-3e78cf9936dc_14","WorkflowType":"DownloadTwitchLiveThumbnailsWorkflow","time":"2024-01-15T09:37:55+08:00","message":"ExecuteActivity"}
{"level":"debug","time":"2024-01-15T09:37:57+08:00","message":"checking if tasks are done for video 43418657787"}
{"level":"debug","Attempt":1,"ChildWorkflowID":"148902df-f5e8-4456-bcdd-3e78cf9936dc_23","Namespace":"default","RunID":"148902df-f5e8-4456-bcdd-3e78cf9936dc","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"b45d1f90-b346-11ee-a650-0242ac12000f","WorkflowType":"SaveTwitchLiveVideoInfoWorkflow","time":"2024-01-15T09:37:57+08:00","message":"ExecuteChildWorkflow"}
{"level":"debug","ActivityID":"5","ActivityType":"SaveTwitchLiveVideoInfo","Attempt":1,"Namespace":"default","RunID":"b9b520dd-703d-40ab-9697-3751e87de7f2","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"148902df-f5e8-4456-bcdd-3e78cf9936dc_23","WorkflowType":"SaveTwitchLiveVideoInfoWorkflow","time":"2024-01-15T09:37:57+08:00","message":"ExecuteActivity"}
{"level":"debug","time":"2024-01-15T09:37:57+08:00","message":"checking if tasks are done for video 43418657787"}
{"level":"debug","Attempt":1,"ChildWorkflowID":"148902df-f5e8-4456-bcdd-3e78cf9936dc_32","Namespace":"default","RunID":"148902df-f5e8-4456-bcdd-3e78cf9936dc","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"b45d1f90-b346-11ee-a650-0242ac12000f","WorkflowType":"ArchiveTwitchLiveChatWorkflow","time":"2024-01-15T09:37:57+08:00","message":"ExecuteChildWorkflow"}
{"level":"debug","time":"2024-01-15T09:37:57+08:00","message":"Live chat archive workflow ID: 148902df-f5e8-4456-bcdd-3e78cf9936dc_32"}
{"level":"debug","Attempt":1,"ChildWorkflowID":"148902df-f5e8-4456-bcdd-3e78cf9936dc_37","Namespace":"default","RunID":"148902df-f5e8-4456-bcdd-3e78cf9936dc","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"b45d1f90-b346-11ee-a650-0242ac12000f","WorkflowType":"DownloadTwitchLiveChatWorkflow","time":"2024-01-15T09:37:57+08:00","message":"ExecuteChildWorkflow"}
{"level":"debug","time":"2024-01-15T09:37:57+08:00","message":"Live chat workflow ID: 148902df-f5e8-4456-bcdd-3e78cf9936dc_37"}
{"level":"debug","Attempt":1,"ChildWorkflowID":"148902df-f5e8-4456-bcdd-3e78cf9936dc_42","Namespace":"default","RunID":"148902df-f5e8-4456-bcdd-3e78cf9936dc","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"b45d1f90-b346-11ee-a650-0242ac12000f","WorkflowType":"ArchiveTwitchLiveVideoWorkflow","time":"2024-01-15T09:37:57+08:00","message":"ExecuteChildWorkflow"}
{"level":"debug","Attempt":1,"ChildWorkflowID":"6a89acb8-12eb-4b80-a595-8155f9fcde77_5","Namespace":"default","RunID":"6a89acb8-12eb-4b80-a595-8155f9fcde77","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"148902df-f5e8-4456-bcdd-3e78cf9936dc_42","WorkflowType":"DownloadTwitchLiveVideoWorkflow","time":"2024-01-15T09:37:57+08:00","message":"ExecuteChildWorkflow"}
{"level":"debug","time":"2024-01-15T09:37:57+08:00","message":"starting heartbeat download-livevideo-43418657787"}
{"level":"debug","time":"2024-01-15T09:37:58+08:00","message":"streamlink live args: [--progress=force --force https://twitch.tv/lyricaldota best,best --http-header Authorization=OAuth ... --twitch-low-latency --twitch-disable-hosting -o /tmp/43418657787_b45d1f90-b346-11ee-a650-0242ac12000f-video.mp4]"}
{"level":"debug","time":"2024-01-15T09:37:58+08:00","message":"running: streamlink --progress=force --force https://twitch.tv/lyricaldota best,best --http-header Authorization=OAuth ... --twitch-low-latency --twitch-disable-hosting -o /tmp/43418657787_b45d1f90-b346-11ee-a650-0242ac12000f-video.mp4"}
{"level":"debug","ActivityID":"9","ActivityType":"DownloadTwitchLiveChat","Attempt":1,"Namespace":"default","RunID":"560112a8-0ea4-4357-9e0f-82c74d712328","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"148902df-f5e8-4456-bcdd-3e78cf9936dc_37","WorkflowType":"DownloadTwitchLiveChatWorkflow","time":"2024-01-15T09:37:58+08:00","message":"ExecuteActivity"}
{"level":"debug","time":"2024-01-15T09:37:58+08:00","message":"starting heartbeat download-livechat-43418657787"}
{"level":"debug","time":"2024-01-15T09:44:59+08:00","message":"finished downloading live video for 43418657787"}
{"level":"error","ActivityType":"DownloadTwitchLiveVideo","Attempt":1,"Error":"failed to get twitch videos: Get \"https://api.twitch.tv/helix/videos?user_id=90937535&type=archive&first=100\": dial tcp: lookup api.twitch.tv on 127.0.0.11:53: read udp 127.0.0.1:41249->127.0.0.11:53: i/o timeout","Namespace":"default","RunID":"d9e1e6ad-08aa-4efe-b883-e76a30fa4d86","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"6a89acb8-12eb-4b80-a595-8155f9fcde77_5","time":"2024-01-15T09:45:09+08:00","message":"Activity error."}
{"level":"debug","time":"2024-01-15T09:45:09+08:00","message":"stopping heartbeat download-livevideo-43418657787"}
{"level":"warn","ActivityID":"9","ActivityType":"DownloadTwitchLiveChat","Attempt":1,"Error":"workflow execution already completed","Namespace":"default","RunID":"560112a8-0ea4-4357-9e0f-82c74d712328","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"148902df-f5e8-4456-bcdd-3e78cf9936dc_37","WorkflowType":"DownloadTwitchLiveChatWorkflow","time":"2024-01-15T09:45:19+08:00","message":"RecordActivityHeartbeat with error"}
{"level":"warn","ActivityID":"9","ActivityType":"DownloadTwitchLiveChat","Attempt":1,"Error":"context canceled","Namespace":"default","RunID":"560112a8-0ea4-4357-9e0f-82c74d712328","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"148902df-f5e8-4456-bcdd-3e78cf9936dc_37","WorkflowType":"DownloadTwitchLiveChatWorkflow","time":"2024-01-15T09:46:19+08:00","message":"RecordActivityHeartbeat with error"}
{"level":"warn","ActivityID":"9","ActivityType":"DownloadTwitchLiveChat","Attempt":1,"Error":"context canceled","Namespace":"default","RunID":"560112a8-0ea4-4357-9e0f-82c74d712328","TaskQueue":"archive","WorkerID":"b183487d2851","WorkflowID":"148902df-f5e8-4456-bcdd-3e78cf9936dc_37","WorkflowType":"DownloadTwitchLiveChatWorkflow","time":"2024-01-15T09:47:19+08:00","message":"RecordActivityHeartbeat with error"}
<above repeated every minute>

Originally posted by @russelg in #354 (comment)

[Zibbp]

Ah the recently added feature to update the stream ID with the VOD ID is causing this. Updated the functionality to log an error instead of returning an error as this isn't critical to the archive flow afea0f5 in #357.