chat_convert issues after latest update

[NothingTV]

After updating to the most recent version, I got few chat_convert issues. The Tasklog shows the following error:

[LOG] - Input bit emote count: 0. Output count: 0

[LOG] - Input badge count: 0. Output count: 16

[LOG] - Input 1st party emote count: 0. Output count: 390
Unhandled exception. System.AggregateException: One or more errors occurred. (Object reference not set to an instance of an object.)
 ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at TwitchDownloaderCore.TwitchHelper.GetStvEmoteData(Int32 streamerId, List`1 stvResponse, Boolean allowUnlistedEmotes)
   at TwitchDownloaderCore.TwitchHelper.GetThirdPartyEmoteData(Int32 streamerId, Boolean getBttv, Boolean getFfz, Boolean getStv, Boolean allowUnlistedEmotes, CancellationToken cancellationToken)
   at TwitchDownloaderCore.TwitchHelper.GetThirdPartyEmotes(List`1 comments, Int32 streamerId, String cacheFolder, EmbeddedData embeddedData, Boolean bttv, Boolean ffz, Boolean stv, Boolean allowUnlistedEmotes, Boolean offline, CancellationToken cancellationToken)
   at TwitchDownloaderCore.ChatUpdater.ThirdPartyEmoteTask(IProgress`1 progress, CancellationToken cancellationToken)
   at TwitchDownloaderCore.ChatUpdater.UpdateEmbeds(Int32 currentStep, Int32 totalSteps, IProgress`1 progress, CancellationToken cancellationToken)
   at TwitchDownloaderCore.ChatUpdater.UpdateAsync(IProgress`1 progress, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean )
   at System.Threading.Tasks.Task.Wait(Int32 , CancellationToken )
   at System.Threading.Tasks.Task.Wait()
   at TwitchDownloaderCLI.Modes.UpdateChat.Update(ChatUpdateArgs inputOptions)
   at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult`1 result, Action`1 action)
   at TwitchDownloaderCLI.Program.Main(String[] args)

This error occured with these channels: Staiy and MauriceWeber.

[Zibbp]

This is an upstream issue with TwitchDownloader. Please check there if this has been reported or create a new issue if it hasn't. Providing the chat file in the issue helps them troubleshoot the issue.

[NothingTV]

Someone already submitted it here: lay295/TwitchDownloader#630
So it's "known" already, I'll let this issue open until this is fixed, so you don't have duplicates of it.

[c-hri-s]

It would be nice if there was a way to handle the error better so that we could still watch the streams, albeit without chat instead of leaving them hanging on 'archiving'

[Zibbp]

If you want to still watch the stream while the chat is rendering or is in a failed state, you can set "is processing" to disabled when editing a VOD in the admin panel. How do you propose handling the error better?

[c-hri-s]

If there's an error converting the chat, just give up and place the stream into the archive as normal?

[Zibbp]

For livestreams specifically, I could mark the VOD as "processed" (allowing users to watch the stream) if any chat related queue task fails but keep the queue item around if the user wants to manually try a restart.
For VOD archives I would not have this as chat render errors are less common and can usually be fixed.

[c-hri-s]

Sounds good to me. I only really record livestreams, so the recent blip has caused quite a few to get stuck processing

[Zibbp]

Made some changes to the chat tasks in for livestreams in the v1.2.8 release. Let me know what you think about them and if there are any issues.

[nguyend24]

Having the following errors in the logs with the latest update.

at TwitchDownloaderCore.ChatRenderer.DrawNonAccentedMessage(Comment comment, List1 sectionImages, List1 emotePositionList, Boolean highlightWords, Point& drawPos, Point& defaultPos)
at TwitchDownloaderCore.ChatRenderer.GenerateCommentSection(Int32 commentIndex, Int32 sectionDefaultYPos, HighlightMessage highlightIcons)
at TwitchDownloaderCore.ChatRenderer.GenerateUpdateFrame(Int32 currentTick, Int32 sectionDefaultYPos, HighlightMessage highlightIcons, UpdateFrame lastUpdate)
at TwitchDownloaderCore.ChatRenderer.RenderVideoSection(Int32 startTick, Int32 endTick, FfmpegProcess ffmpegProcess, FfmpegProcess maskProcess, CancellationToken cancellationToken)
at TwitchDownloaderCore.ChatRenderer.<>c__DisplayClass25_0.b__1()
at System.Threading.Tasks.Task.InnerInvoke()
at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object )
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread , ExecutionContext , ContextCallback , Object )
--- End of stack trace from previous location ---
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread , ExecutionContext , ContextCallback , Object )
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& , Thread )
--- End of stack trace from previous location ---
at TwitchDownloaderCore.ChatRenderer.RenderVideoAsync(CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean )
at System.Threading.Tasks.Task.Wait(Int32 , CancellationToken )
at System.Threading.Tasks.Task.Wait()
at TwitchDownloaderCLI.Modes.RenderChat.Render(ChatRenderArgs inputOptions)
at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult1 result, Action1 action)
at TwitchDownloaderCLI.Program.Main(String[] args)

Fails very quickly, doesn't get to 1%. Downloaded the v1.52.3 of TwitchDownloader and running the chatrender outside ganymede works.

Happens to live streams and vods. Here's one example.

https://www.twitch.tv/videos/1648686324

[Zibbp]

Looks like an upstream issue with TwitchDownloader. I would check if there is an issue already created there. If not, create an issue with the error and chat json file.

Edit: may also be be a font issue if it renders outside of Ganymede. Did you use the GUI when testing outside of Ganymede or the CLI?

[nguyend24]

TwitchDownloaderCLI

The command I ran

./TwitchDownloaderCLI chatrender -i ganymede/tmp/1652395872_fb1e4c92-bb1a-11ed-b463-0242ac130002-ch at.json -h 1440 -w 340 --framerate 30 --font Inter --font-size 13 -o out.mp4

[nguyend24]

Sounds like it's a font issue from this:
lay295/TwitchDownloader#663 (comment)

[nguyend24]

Did some more digging. Attached to the container and noticed that fc-list only showed Terminus fonts.

Ran fc-cache -f and extra fonts show up(noto, dejavu, etc.). chatrender isn't crashing anymore.

Going to add fc-cache -f to the Dockerfile and monitor it over the next few days.

[ScrubN]

I would also like to mention that chat_convert will work now as of TwitchDownloader 1.52.3