settings not saved after update #477

Serph91P · 2023-11-15T03:59:58Z

Seems like the date settings have changed and need to be updated before you can change the settings again:

04:48:54
Error: Invalid value for setting 'date_format': dd-mm-yyyy
04:48:54
    at _Config.setConfig (/usr/local/share/twitchautomator/server/src/Core/Config.ts:360:27)
04:48:54
    at SaveSettings (/usr/local/share/twitchautomator/server/src/Controllers/Settings.ts:126:38)
04:48:54
    at Layer.handle3 (/usr/local/share/twitchautomator/server/.yarn/cache/express-npm-4.18.2-bb15ff679a-869ae89ed6.zip/node_modules/express/lib/router/layer.js:95:5)
04:48:54
    at next (/usr/local/share/twitchautomator/server/.yarn/cache/express-npm-4.18.2-bb15ff679a-869ae89ed6.zip/node_modules/express/lib/router/route.js:144:13)
04:48:54
    at AuthAdmin (/usr/local/share/twitchautomator/server/src/Extend/express-auth.ts:96:9)
04:48:54
 - - [15/Nov/2023:03:48:54 +0000] "PUT /api/v0/settings HTTP/1.1" 500 1761 "-" "Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Mobile Safari/537.36"
04:48:54
    at Layer.handle3 (/usr/local/share/twitchautomator/server/.yarn/cache/express-npm-4.18.2-bb15ff679a-869ae89ed6.zip/node_modules/express/lib/router/layer.js:95:5)
04:48:54
    at next (/usr/local/share/twitchautomator/server/.yarn/cache/express-npm-4.18.2-bb15ff679a-869ae89ed6.zip/node_modules/express/lib/router/route.js:144:13)
04:48:54
    at Route2.dispatch (/usr/local/share/twitchautomator/server/.yarn/cache/express-npm-4.18.2-bb15ff679a-869ae89ed6.zip/node_modules/express/lib/router/route.js:114:3)
04:48:54
    at Layer.handle3 (/usr/local/share/twitchautomator/server/.yarn/cache/express-npm-4.18.2-bb15ff679a-869ae89ed6.zip/node_modules/express/lib/router/layer.js:95:5)
04:48:54
    at <anonymous> (/usr/local/share/twitchautomator/server/.yarn/cache/express-npm-4.18.2-bb15ff679a-869ae89ed6.zip/node_modules/express/lib/router/index.js:284:15)
04:49:10

The text was updated successfully, but these errors were encountered:

MrBrax · 2023-11-15T10:04:24Z

is this perhaps an old value? the choices don't have that in the list

Serph91P · 2023-11-15T10:26:43Z

is this perhaps an old value? the choices don't have that in the list

I cah chose the same settings here?

MrBrax · 2023-11-15T10:43:35Z

maybe you have to re-select it and save? weird how it would use the old format if not

Serph91P · 2023-11-15T10:46:41Z

maybe you have to re-select it and save? weird how it would use the old format if not

yeah that is what I did.

MrBrax · 2023-11-15T12:32:37Z

so is it still broken after selecting a new one?

Serph91P · 2023-11-15T13:08:39Z

so is it still broken after selecting a new one?

nono just informing you that I had that selected before updating and after updating this error pop up and I could not save any other settings until I fixed this one first.

Serph91P · 2023-11-20T14:44:39Z

so is it still broken after selecting a new one?

sorry have to reopen. Since the last update we had the problem with the config not beeing the same anymore with the date which was a easy fix to just set the date again. But since then I got alot of Capture failed. The recording is still in the cache folder, and the json has no chapters. Only seem to be some streamer/folder but the all have the same permissions and before the recording worked fine.

here are the logs I could find:

Log.txt

MrBrax · 2023-11-20T14:51:17Z

which version of streamlink is packaged into the docker container now?

Serph91P · 2023-11-20T16:13:35Z

which version of streamlink is packaged into the docker container now?

Serph91P · 2023-11-20T16:50:48Z

which version of streamlink is packaged into the docker container now?

Seems to be the latest version.

MrBrax · 2023-11-20T18:21:30Z

i'll check my recordings if it happens, that log is very confusing

Serph91P · 2023-11-21T08:46:57Z

i'll check my recordings if it happens, that log is very confusing

I did not copy everything but those parts where errors happened.

MrBrax · 2023-11-21T08:48:45Z

multiple streams recorded now and it seems fine, maybe it's the emoji and other stuff in the filename?

Serph91P · 2023-11-21T08:56:14Z

multiple streams recorded now and it seems fine, maybe it's the emoji and other stuff in the filename?

This I changed after I noticed that they did not get recorded. Because the filenames did change in the update aswell. And I added the title just for fun I can remove it and see.

MrBrax · 2023-11-21T09:02:40Z

are the captured mp4 files actually playable?

Serph91P · 2023-11-21T11:16:20Z

are the captured mp4 files actually playable?

Yeah only the filenames was the default.

Serph91P · 2023-11-21T16:14:54Z

are the captured mp4 files actually playable?

fount this:

Myun2023 - S202311 E09 - (❁´◡`❁) hi. anime waifu drinks - !uwu.json
and is playable so dunno what is happening. Cleaned up the cache directory with all the old files. Space cannot be the issue (16TB) Internet should have been up the whole time (hetzner)

MrBrax · 2023-11-21T17:20:53Z

maybe something happened while it was converting or capturing? i'm at a loss here at the moment. unless someone else also has this problem it might be difficult to reproduce. i'll add more debugging info in mediainfo but that's about it

MrBrax · 2023-11-22T09:25:21Z

could you check if there are more verbose errors with mediainfo on the dev branch? no need to capture anything new, just using one of the broken vods

Serph91P · 2023-11-22T16:41:54Z

could you check if there are more verbose errors with mediainfo on the dev branch? no need to capture anything new, just using one of the broken vods

Can I change back after? I deleted the old ones because they where just taking away space.

MrBrax · 2023-11-22T16:59:55Z

i would probably not recommend changing back in the same instance

Serph91P · 2023-11-22T17:05:32Z

i would probably not recommend changing back in the same instance

Then how would I see the old files if I don't use thos instance. Would have to setup a new one with dev and new config and cache ECT. So dunno how I should go about this.

MrBrax · 2023-11-22T20:30:01Z

i don't know how those docker hosts work, but copying the data directory and pointing a new instance to it

Serph91P · 2023-11-23T12:08:26Z

i don't know how those docker hosts work, but copying the data directory and pointing a new instance to it

could I use the same storage directory because this is quite big.

MrBrax · 2023-11-23T12:09:30Z

it might overwrite old versions with new stuff, unsure if i've added something new though. you could just copy the failing videos

Serph91P · 2023-11-23T12:10:39Z

it might overwrite old versions with new stuff, unsure if i've added something new though. you could just copy the failing videos

ok will try thanks for the fast responses.

Serph91P · 2023-11-23T12:28:41Z

it might overwrite old versions with new stuff, unsure if i've added something new though. you could just copy the failing videos

I hope I got everyything you wanted.
dev_log.txt

MrBrax · 2023-11-23T12:32:58Z

i feel it must be the filename characters, unless the debug mode shows more

MrBrax · 2023-11-24T10:30:42Z

hmm, try checking the capture logs for the failing streamers, both stdout and stderr. are you using an adblock proxy?

Serph91P · 2023-11-24T10:32:02Z

hmm, try checking the capture logs for the failing streamers, both stdout and stderr. are you using an adblock proxy?

No not using the AdBlock Proxy. Will check the logs later.

Serph91P · 2023-11-24T14:43:06Z

hmm, try checking the capture logs for the failing streamers, both stdout and stderr. are you using an adblock proxy?

the only things I could find in the logs are this:

[warnings][streamlinkdeprecation] `hls-segment-threads` has been deprecated in favor of the `stream-segment-threads` option
[warnings][streamlinkdeprecation] `hls-segment-timeout` has been deprecated in favor of the `stream-segment-timeout` option
[warnings][streamlinkdeprecation] `hls-timeout` has been deprecated in favor of the `stream-timeout` option

/usr/local/share/twitchautomator/data/cache/capture/Bonjwa2023 - s202311e23.ts
[plugins.twitch][info] Will skip ad segments
[plugins.twitch][info] Waiting for pre-roll ads to finish, be patient
[stream.hls][info] Filtering out segments and pausing stream output
[stream.hls][warning] Encountered a stream discontinuity. This is unsupported and will result in incoherent output data.
[stream.hls][info] Resuming stream output
[cli][info] Stream ended
[cli][info] Closing currently open stream...

/home/node/.local/share/virtualenvs/twitchautomator-rNbiFSV2/bin/streamlink --hls-live-edge 99999 --hls-timeout 200 --hls-segment-timeout 200 --hls-segment-threads 5 --ffmpeg-fout mpegts --twitch-disable-hosting --twitch-disable-ads --twitch-disable-reruns --retry-streams 10 --retry-max 5 -o /usr/local/share/twitchautomator/data/cache/capture/Bonjwa2023 - s202311e23.ts --url https://twitch.tv/bonjwa --default-stream best

its almost like a virus it keeps going this was ok before so def not a recording thing my guess, this streamer is not online atm:

MrBrax · 2023-11-24T15:54:27Z

wondering if it's some kind of network issue on your end, or something twitch is doing to block streaming... so it outputs to the ts files and fails, and the ts files are just partial streams, right?

Serph91P · 2023-11-24T15:59:38Z

wondering if it's some kind of network issue on your end, or something twitch is doing to block streaming... so it outputs to the ts files and fails, and the ts files are just partial streams, right?

I doubt that its a network issue kinda. Everything else is working like intended.
Cache directory:

storage:

Serph91P · 2023-11-24T16:06:30Z

wondering if it's some kind of network issue on your end, or something twitch is doing to block streaming... so it outputs to the ts files and fails, and the ts files are just partial streams, right?

happening now to all channels:

MrBrax · 2023-11-24T19:55:04Z

just so we're clear about this;

there's one main captured vod that's the entire thing
there's one captured vod part for every chapter

or

there's one captured vod part for every chapter and it quits prematurely

or what else is going on? still very confusing.

Serph91P · 2023-11-24T20:55:35Z

just so we're clear about this;

* there's one main captured vod that's the entire thing

* there's one captured vod part for every chapter

or

* there's one captured vod part for every chapter and it quits prematurely

or what else is going on? still very confusing.

It's neither.

There are multiple streams being registered or somehow split I think. They are the same stream but are registered somehow. It looks like the first Json is being updated but the last is the one recording somehow. But the crazy part is that in my opinion it happens somewhere not directly when the stream starts. I checked the last days and it at first glance seems to be working fine, but when I check the next morning or something the got all fucked up

MrBrax · 2023-11-24T21:34:40Z

cosmic rays? yeah this just makes no sense. would more lean on docker or the environment. i see nothing of this and there are no functions to save multiples of one vod

MrBrax · 2023-11-24T21:48:46Z

and just as i said that i notice that two of my capturing streams stopped half an hour ago, this must be a streamlink issue

Serph91P · 2023-11-25T06:32:49Z

and just as i said that i notice that two of my capturing streams stopped half an hour ago, this must be a streamlink issue

Oh wow well maybe progress.

MrBrax · 2023-11-25T10:04:25Z

it might have been a stream pause, not sure yet. still no broken streams anywhere

Serph91P · 2023-11-25T10:46:42Z

it might have been a stream pause, not sure yet. still no broken streams anywhere

Will report back after the update and a stream breaking again.

Serph91P · 2023-11-27T07:15:28Z

it might have been a stream pause, not sure yet. still no broken streams anywhere

I don't get it. Was working for one stream and now it's happening again. Internet was up the whole time have uptime Kuma running now. Space was sufficient the whole time and ram with 64gb cannot be the problem. Look at this specific stream yesterday an everything was working fine. And now it's worked again.

MrBrax · 2023-11-27T08:38:45Z

is the channel header/menu item for the streamer red/flashing/live?

Serph91P · 2023-11-27T09:04:20Z

is the channel header/menu item for the streamer red/flashing/live?

Nope

Serph91P · 2023-11-27T09:32:40Z

is the channel header/menu item for the streamer red/flashing/live?

checked for another channel where the same problem occured yesterday, this is what I found in the log there seems to be some kind of pause and then it changes to no chapters found:

26.11.2023
23:06:59
2023-11-26T22:06:59.360Z | hook.HookTwitch <info> Hook called with message ID v8x-GcgomtBa4_5fmqcwmc38CZGViLjRvvQ0kGjIlag=, s-version 1, s-type channel.update, b false (m-retry 0, m-type notification, m-date 2023-11-26T22:06:58.09615899Z)
26.11.2023
23:06:59
2023-11-26T22:06:59.375Z | automator.handle <info> Automator channel.update event for bonjwa
26.11.2023
23:06:59
redacted - - [26/Nov/2023:22:06:59 +0000] "POST /api/v0/hook/twitch HTTP/1.1" 200 0 "-" "Go-http-client/1.1"
26.11.2023
23:06:59
2023-11-26T22:06:59.567Z | tw.channel.getStreams <info> Querying streams for streamer id 73437396 returned 1 streams
26.11.2023
23:06:59
2023-11-26T22:06:59.570Z | automator.updateGame <success> Channel data for bonjwa fetched from notification.
26.11.2023
23:06:59
2023-11-26T22:06:59.570Z | TwitchVODChapter.fromJSON <warn> No game_id for chapter: Quests und Bosse grinden mit Matteo, @reismitmais, @suun
26.11.2023
23:06:59
2023-11-26T22:06:59.578Z | vod.addChapter <info> Adding chapter Quests und Bosse grinden mit Matteo, @reismitmais, @suun () to Bonjwa2023 - s202311e25
26.11.2023
23:06:59
2023-11-26T22:06:59.579Z | chapter.calculateDurationAndOffset <warn> No next chapter or vod end time for chapter Quests und Bosse grinden mit Matteo, @reismitmais, @suun (2023-11-26T22:06:59.375Z), duration will probably be 0.
26.11.2023
23:06:59
2023-11-26T22:06:59.579Z | vod.saveJSON <success> Saving JSON of Bonjwa2023 - s202311e25  (game update)
26.11.2023
23:06:59
2023-11-26T22:06:59.585Z | automator.updateGame <success> Stream updated on 'bonjwa' to '' (Quests und Bosse grinden mit Matteo, @reismitmais, @suun) using eventsub.
26.11.2023
23:07:00
2023-11-26T22:07:00.064Z | automator.captureTicker <info> Streamlink resuming stream output.
26.11.2023
23:07:00
2023-11-26T22:07:00.065Z | automator.captureTicker <info> Pause detected for Bonjwa2023 - s202311e26, 14s long.
26.11.2023
23:07:14
68.169.129.134 - - [26/Nov/2023:22:07:14 +0000] "HEAD / HTTP/1.1" 200 1208 "-" "Notifiarr"
26.11.2023
23:08:44
🎥 Bonjwa2023 - s202311e26 1080p60 104.58 MB / 7311 kbps
26.11.2023
23:10:44
🎥 Bonjwa2023 - s202311e26 1080p60 227.52 MB / 8594 kbps
26.11.2023
23:11:44
2023-11-26T22:11:44.227Z | tw.channel.getStreams <info> Querying streams for streamer id 73437396 returned 1 streams
26.11.2023
23:11:44
2023-11-26T22:11:44.228Z | vod.addViewerCount <info> Added viewer count for Bonjwa2023 - s202311e26 (4486)
26.11.2023
23:11:44
2023-11-26T22:11:44.228Z | vod.saveJSON <warn> Saving JSON of Bonjwa2023 - s202311e26 with no chapters!!
26.11.2023
23:11:44
2023-11-26T22:11:44.228Z | vod.saveJSON <success> Saving JSON of Bonjwa2023 - s202311e26  (add viewer count)
26.11.2023
23:11:52
2023-11-26T22:11:52.327Z | hook.HookTwitch <info> Hook called with message ID COwAH7aET0jwzvLfm56hcy9Ar5KzIO6SgKkhwc-kUW4=, s-version 1, s-type channel.update, b false (m-retry 0, m-type notification, m-date 2023-11-26T22:11:51.059483229Z)
26.11.2023
23:11:52
2023-11-26T22:11:52.348Z | automator.handle <info> Automator channel.update event for bonjwa

and after that I looked even further back and found this?

22:17:10
2023-11-26T21:17:10.797Z | helper.execSimple <info> Process 60 for 'mediainfo' exited with code 1
26.11.2023
22:17:10
2023-11-26T21:17:10.797Z | helper.videometadata <error> Trying to get mediainfo of /usr/local/share/twitchautomator/data/storage/vods/myun/Season 202311/Myun2023 - S202311 E11 - (❁´◡`❁) cozy   - !uwu.mp4 returned: undefined
26.11.2023
22:17:10
2023-11-26T21:17:10.797Z | vod.getMediainfo <error> Could not get mediainfo of Myun2023 - S202311 E11 - (❁´◡`❁) cozy   - !uwu (/usr/local/share/twitchautomator/data/storage/vods/myun/Season 202311/Myun2023 - S202311 E11 - (❁´◡`❁) cozy   - !uwu.mp4 @ /usr/local/share/twitchautomator/data/storage/vods/myun/Season 202311): undefined
26.11.2023
22:17:10
2023-11-26T21:17:10.797Z | vod.getDuration <error> Could not find duration of Myun2023 - S202311 E11 - (❁´◡`❁) cozy   - !uwu
26.11.2023
22:17:10
🛠️ [channel parseVODs] Myun2023 - S202311 E11 - (❁´◡`❁) cozy   - !uwu duration: null
26.11.2023
22:17:10
🛠️ [channel parseVODs] Myun2023 - S202311 E11 - (❁´◡`❁) cozy   - !uwu has no duration
26.11.2023
22:17:10
[TwitchVOD] Returning cached vod Myun2023 - s202311e15.json
26.11.2023
22:17:10
Watching channel myun folders...
26.11.2023
22:17:10
2023-11-26T21:17:10.818Z | dvr.load.tw <success> Loaded channel myun with 13 vods
26.11.2023
22:17:10
2023-11-26T21:17:10.818Z | dvr.loadChannels <success> Loaded 17 channels!
26.11.2023
22:17:10
2023-11-26T21:17:10.827Z | job.loadJobsFromCache <info> Loaded 3 jobs from cache
26.11.2023
22:17:10
2023-11-26T21:17:10.834Z | scheduler.defaultJobs <info> Set up default jobs
26.11.2023
22:17:10
2023-11-26T21:17:10.836Z | scheduler.schedule <info> Scheduled job 'check_muted_vods' with cronTime '0 */12 * * *'
26.11.2023
22:17:10
2023-11-26T21:17:10.836Z | scheduler.schedule <info> Scheduled job 'check_deleted_vods' with cronTime '10 */12 * * *'
26.11.2023
22:17:10
2023-11-26T21:17:10.837Z | scheduler.schedule <info> Scheduled job 'match_vods' with cronTime '30 */12 * * *'
26.11.2023
22:17:10
2023-11-26T21:17:10.837Z | scheduler.schedule <info> Scheduled job 'clipdownload' with cronTime '0 0 * * *'
26.11.2023
22:17:10
2023-11-26T21:17:10.838Z | scheduler.schedule <info> Scheduled job 'validate_oauth' with cronTime '0 */1 * * *'
26.11.2023
22:17:10
2023-11-26T21:17:10.838Z | scheduler.schedule <info> Scheduled job 'export_vods' with cronTime '0 1 * * *'
26.11.2023
22:17:10
2023-11-26T21:17:10.838Z | scheduler.defaultJobs <info> Default job 'check_muted_vods' enabled
26.11.2023
22:17:10
2023-11-26T21:17:10.838Z | scheduler.defaultJobs <info> Default job 'check_deleted_vods' enabled
26.11.2023
22:17:10
2023-11-26T21:17:10.838Z | scheduler.defaultJobs <info> Default job 'match_vods' disabled
26.11.2023
22:17:10
2023-11-26T21:17:10.838Z | scheduler.defaultJobs <info> Default job 'clipdownload' disabled
26.11.2023
22:17:10
2023-11-26T21:17:10.838Z | scheduler.defaultJobs <info> Default job 'export_vods' disabled
26.11.2023
22:17:10
2023-11-26T21:17:10.838Z | tw.helper <info> Eventsub is not using websocket
26.11.2023
22:17:10

22:17:10
2023-11-26T21:17:10.839Z | helper.execSimple <error> Process undefined for 'linux process status (capture_bonjwa_41458113464)' error: Error: spawn ps ENOENT
26.11.2023
22:17:10
2023-11-26T21:17:10.839Z | job.getStatus <error> Error checking status for linux job capture_bonjwa_41458113464 (false)
26.11.2023
22:17:10
2023-11-26T21:17:10.839Z | job.checkStaleJobs <warn> Job capture_bonjwa_41458113464 is stale, no process found. Clearing.
26.11.2023
22:17:10
2023-11-26T21:17:10.839Z | job.clear <info> Clear job capture_bonjwa_41458113464 with PID 767
26.11.2023
22:17:10
No WebSocket server attached to broker for data: {"action":"job_clear","data":{"job_name":"capture_bonjwa_4145811...
26.11.2023
22:17:10
2023-11-26T21:17:10.840Z | job.clear <success> Job capture_bonjwa_41458113464 removed from jobs list
26.11.2023
22:17:10
2023-11-26T21:17:10.845Z | helper.execSimple <error> Process undefined for 'linux process status (capture_gronkh_41457342888)' error: Error: spawn ps ENOENT
26.11.2023
22:17:10
2023-11-26T21:17:10.845Z | job.getStatus <error> Error checking status for linux job capture_gronkh_41457342888 (false)
26.11.2023
22:17:10
2023-11-26T21:17:10.845Z | job.checkStaleJobs <warn> Job capture_gronkh_41457342888 is stale, no process found. Clearing.
26.11.2023
22:17:10
2023-11-26T21:17:10.845Z | job.clear <info> Clear job capture_gronkh_41457342888 with PID 705
26.11.2023
22:17:10
No WebSocket server attached to broker for data: {"action":"job_clear","data":{"job_name":"capture_gronkh_4145734...
26.11.2023
22:17:10
2023-11-26T21:17:10.845Z | job.clear <success> Job capture_gronkh_41457342888 removed from jobs list
26.11.2023
22:17:10
2023-11-26T21:17:10.850Z | helper.execSimple <error> Process undefined for 'linux process status (remux_PietSmiet2023 - s202311e25.ts)' error: Error: spawn ps ENOENT
26.11.2023
22:17:10
2023-11-26T21:17:10.850Z | job.getStatus <error> Error checking status for linux job remux_PietSmiet2023 - s202311e25.ts (false)
26.11.2023
22:17:10
2023-11-26T21:17:10.850Z | job.checkStaleJobs <warn> Job remux_PietSmiet2023 - s202311e25.ts is stale, no process found. Clearing.
26.11.2023
22:17:10
2023-11-26T21:17:10.850Z | job.clear <info> Clear job remux_PietSmiet2023 - s202311e25.ts with PID 810
26.11.2023
22:17:10
No WebSocket server attached to broker for data: {"action":"job_clear","data":{"job_name":"remux_PietSmiet2023 - ...
26.11.2023
22:17:10
2023-11-26T21:17:10.851Z | job.clear <success> Job remux_PietSmiet2023 - s202311e25.ts removed from jobs list

After that I thought this almost looks like it is rebooting somehow and found this:

26.11.2023
22:15:59
2023-11-26T21:15:59.266Z | vod.saveJSON <success> Saving JSON of PietSmiet2023 - s202311e25  (remove short chapters)
26.11.2023
22:15:59
😴 Sleeping for 30000 ms
26.11.2023
22:16:08
2023-11-26T21:16:08.211Z | hook.HookTwitch <info> Hook called with message ID zyeFMELmy25yXqsQxFmq4dtROqxUFU4xc79KxxKWEo0=, s-version 1, s-type stream.offline, b false (m-retry 0, m-type notification, m-date 2023-11-26T21:16:07.141566116Z)
26.11.2023
22:16:08
2023-11-26T21:16:08.239Z | automator.end <info> Stream end
26.11.2023
22:16:08
2023-11-26T21:16:08.240Z | automator.end <info> Stream offline for pietsmiet
26.11.2023
22:16:08
2023-11-26T21:16:08.241Z | clientBroker.notify <info> (streamOffline) pietsmiet has gone offline!: Was streaming for about 4 hours.
26.11.2023
22:16:08
2023-11-26T21:16:08.241Z | automator.end <warn> Stream offline notification for pietsmiet but channel is not capturing.
26.11.2023
22:16:08
52.10.250.8 - - [26/Nov/2023:21:16:08 +0000] "POST /api/v0/hook/twitch HTTP/1.1" 200 0 "-" "Go-http-client/1.1"
26.11.2023
22:16:29
2023-11-26T21:16:29.273Z | vod.saveJSON <success> Saving JSON of PietSmiet2023 - s202311e25  (is_converting set)
26.11.2023
22:16:29
2023-11-26T21:16:29.275Z | vod.saveFFMPEGChapters <info> Saving FFMPEG chapters file for PietSmiet2023 - s202311e25 to /usr/local/share/twitchautomator/data/storage/vods/pietsmiet/Season 202311/PietSmiet2023 - s202311e25-ffmpeg-chapters.txt
26.11.2023
22:16:29
2023-11-26T21:16:29.275Z | video.remux <info> Remuxing /usr/local/share/twitchautomator/data/cache/capture/PietSmiet2023 - s202311e25.ts to /usr/local/share/twitchautomator/data/storage/vods/pietsmiet/Season 202311/PietSmiet2023 - s202311e25.mp4
26.11.2023
22:16:29
startJob process /usr/bin/ffmpeg [
26.11.2023
22:16:29
  '/usr/bin/ffmpeg',
26.11.2023
22:16:29
  '-i',
26.11.2023
22:16:29
  '/usr/local/share/twitchautomator/data/cache/capture/PietSmiet2023 - s202311e25.ts',
26.11.2023
22:16:29
  '-i',
26.11.2023
22:16:29
  '/usr/local/share/twitchautomator/data/storage/vods/pietsmiet/Season 202311/PietSmiet2023 - s202311e25-ffmpeg-chapters.txt',
26.11.2023
22:16:29
  '-map_metadata',
26.11.2023
22:16:29
  '1',
26.11.2023
22:16:29
  '-c',
26.11.2023
22:16:29
  'copy',
26.11.2023
22:16:29
  '-bsf:a',
26.11.2023
22:16:29
  'aac_adtstoasc',
26.11.2023
22:16:29
  '-movflags',
26.11.2023
22:16:29
  'faststart',
26.11.2023
22:16:29
  '/usr/local/share/twitchautomator/data/storage/vods/pietsmiet/Season 202311/PietSmiet2023 - s202311e25.mp4'
26.11.2023
22:16:29
]
26.11.2023
22:16:29
2023-11-26T21:16:29.280Z | exec.startJob <info> Executing /usr/bin/ffmpeg -i /usr/local/share/twitchautomator/data/cache/capture/PietSmiet2023 - s202311e25.ts -i /usr/local/share/twitchautomator/data/storage/vods/pietsmiet/Season 202311/PietSmiet2023 - s202311e25-ffmpeg-chapters.txt -map_metadata 1 -c copy -bsf:a aac_adtstoasc -movflags faststart /usr/local/share/twitchautomator/data/storage/vods/pietsmiet/Season 202311/PietSmiet2023 - s202311e25.mp4
26.11.2023
22:16:29
2023-11-26T21:16:29.280Z | exec.startJob <success> Spawned process 810 for remux_PietSmiet2023 - s202311e25.ts
26.11.2023
22:16:29
2023-11-26T21:16:29.281Z | job.save <info> Save job remux_PietSmiet2023 - s202311e25.ts with PID 810 to /usr/local/share/twitchautomator/data/cache/pids/remux_PietSmiet2023 - s202311e25.ts.json
26.11.2023
22:16:29
2023-11-26T21:16:29.281Z | exec.startJob <info> Attached to all streams for process 810 for remux_PietSmiet2023 - s202311e25.ts
26.11.2023
22:16:29
2023-11-26T21:16:29.300Z | vod.watch <info> VOD file /usr/local/share/twitchautomator/data/storage/vods/pietsmiet/Season 202311/PietSmiet2023 - s202311e25-ffmpeg-chapters.txt changed (change)!
26.11.2023
22:16:29
```
Why is it warning about this??? 

```
2023-11-26T21:16:29.300Z | clientBroker.notify <info> (system) VOD file changed externally: PietSmiet2023 - s202311e25-ffmpeg-chapters.txt
26.11.2023
22:16:29
Remux total duration for PietSmiet2023 - s202311e25.ts: 12688
26.11.2023
22:16:34
🎞 Remuxing PietSmiet2023 - s202311e25.ts - 568/12688 seconds (4%)
26.11.2023
22:16:53
26.11.2023
22:16:53
<--- Last few GCs --->
26.11.2023
22:16:53
26.11.2023
22:16:53
[34:0x5f10ae0] 129112834 ms: Mark-Compact 4043.8 (4131.4) -> 4034.5 (4138.7) MB, 726.1 / 0.0 ms  (average mu = 0.777, current mu = 0.192) allocation failure; scavenge might not succeed
26.11.2023
22:16:53
[34:0x5f10ae0] 129114162 ms: Mark-Compact 4050.3 (4138.7) -> 4040.7 (4144.4) MB, 1316.0 / 0.0 ms  (average mu = 0.511, current mu = 0.010) allocation failure; scavenge might not succeed
26.11.2023
22:16:53
26.11.2023
22:16:53
26.11.2023
22:16:53
<--- JS stacktrace --->
26.11.2023
22:16:53
26.11.2023
22:16:53
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
26.11.2023
22:16:53
 1: 0xbf9890 node::Abort() [/usr/local/bin/node]
26.11.2023
22:16:53
 2: 0xaf6859  [/usr/local/bin/node]
26.11.2023
22:16:53
 3: 0xddaf80 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/usr/local/bin/node]
26.11.2023
22:16:53
 4: 0xddb336 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/usr/local/bin/node]
26.11.2023
22:16:53
 5: 0xfd9db5  [/usr/local/bin/node]
26.11.2023
22:16:53
 6: 0xfed4e5 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
26.11.2023
22:16:53
 7: 0xfc924f v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node]
26.11.2023
22:16:53
 8: 0xfca297 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node]
26.11.2023
22:16:53
 9: 0xfa99fa v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/usr/local/bin/node]
26.11.2023
22:16:53
10: 0x13bd76f v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/usr/local/bin/node]
26.11.2023
22:16:53
11: 0x18443f9  [/usr/local/bin/node]
26.11.2023
22:16:55
Initialising...
26.11.2023
22:16:55
Environment: undefined
26.11.2023
22:16:55
Running as user undefined
26.11.2023
22:16:55
AppRoot: /usr/local/share/twitchautomator
26.11.2023
22:16:55
DataRoot: /usr/local/share/twitchautomator/data
26.11.2023
22:16:55
Setting debug to false
26.11.2023
22:16:55
Client is built: /usr/local/share/twitchautomator/client-vue/dist/index.html
26.11.2023
22:16:55
Chat dumper is built: /usr/local/share/twitchautomator/twitch-chat-dumper/build/index.js
26.11.2023
22:16:55
Application did not exit cleanly, please check logs for more information. Will continue to run.
26.11.2023
22:16:55
Loading key-value pairs...
26.11.2023
22:16:55
Loaded 308 key-value pairs
26.11.2023
22:16:55
Loading config...
26.11.2023
22:16:55
2023-11-26T21:16:55.090Z | config.loadConfig <info> Migrated setting 'locale.date-format' to 'date_format'.
26.11.2023
22:16:55
Saved setting 'date_format' does not exist, deprecated? Discarding.
26.11.2023
22:16:55
Setting 'locale.date-format' not configured, using default value 'yyyy-MM-dd'.
26.11.2023
22:16:55
2023-11-26T21:16:55.093Z | config.saveConfig <success> Saved config from missing default values
26.11.2023
22:16:55
Setting debug to false
26.11.2023
22:16:55
✔ 106 settings loaded.
26.11.2023
22:16:55
2023-11-26T21:16:55.093Z | config.cfg <warn> Setting 'bin_path_python' does not exist.
26.11.2023
22:16:55
Setting 'bin_path_python' does not exist.
26.11.2023
22:16:55
Overriding setting 'BIN_PATH_PYTHON' with environment variable: '/usr/bin/python'
26.11.2023
22:16:55
Overriding setting 'NODE_PATH' with environment variable: '/usr/local/bin/node'
26.11.2023
22:16:55
Overriding setting 'BIN_DIR' with environment variable: '/usr/local/bin'
26.11.2023
22:16:55
2023-11-26T21:16:55.094Z | config.cfg <warn> Setting 'bin_path_python3' does not exist.
26.11.2023
22:16:55
Setting 'bin_path_python3' does not exist.
26.11.2023
22:16:55
Overriding setting 'BIN_PATH_PYTHON3' with environment variable: '/usr/bin/python3'
26.11.2023
22:16:55
Overriding setting 'SERVER_PORT' with environment variable: '8080'
26.11.2023
22:16:55
Overriding setting 'MEDIAINFO_PATH' with environment variable: '/usr/bin/mediainfo'
26.11.2023
22:16:55
2023-11-26T21:16:55.094Z | config.cfg <warn> Setting 'python_enable_pipenv' does not exist.
26.11.2023
22:16:55
Setting 'python_enable_pipenv' does not exist.
26.11.2023
22:16:55
Overriding setting 'PYTHON_ENABLE_PIPENV' with environment variable: '1'
26.11.2023
22:16:55
2023-11-26T21:16:55.094Z | config.cfg <warn> Setting 'docker' does not exist.
26.11.2023
22:16:55
Setting 'docker' does not exist.
26.11.2023
22:16:55
Overriding setting 'DOCKER' with environment variable: '1'
26.11.2023
22:16:55
Overriding setting 'FFMPEG_PATH' with environment variable: '/usr/bin/ffmpeg'
26.11.2023
22:16:55
Overriding setting 'TWITCHDOWNLOADER_PATH' with environment variable: '/usr/local/bin/TwitchDownloaderCLI'
26.11.2023
22:16:55
Overriding setting 'WEBSOCKET_ENABLED' with environment variable: 'true'
26.11.2023
22:16:55
Language changed to en
26.11.2023
22:16:55
2023-11-26T21:16:55.094Z | YouTubeHelper.setupClient <warn> No client_id or client_secret set up. YouTube uploads will not work.
26.11.2023
22:16:55
2023-11-26T21:16:55.095Z | tw.helper.setupAxios <info> Setting up axios...
26.11.2023
22:16:55
2023-11-26T21:16:55.108Z | tw.helper.getAccessTokenApp <info> Stored app access token expires at 12/26/2023, 9:43:16 PM
26.11.2023
22:16:55
2023-11-26T21:16:55.108Z | tw.helper.setupAxios <success> Axios setup with app token.
26.11.2023
22:16:55
2023-11-26T21:16:55.110Z | dvr.init <success> The time is 2023-11-26T21:16:55.109Z. Current topside temperature is 93 degrees, with an estimated high of one hundred and five. The Black Mesa compound is maintained at a pleasant 68 degrees at all times.
26.11.2023
22:16:55
2023-11-26T21:16:55.118Z | helper.execSimple <info> Process 45 for 'git hash check' exited with code 128
26.11.2023
22:16:55
2023-11-26T21:16:55.118Z | config.getGitHash <warn> Could not fetch git hash
26.11.2023
22:16:55
2023-11-26T21:16:55.121Z | helper.execSimple <info> Process 46 for 'git branch check' exited with code 128
26.11.2023
22:16:55
2023-11-26T21:16:55.121Z | config.getGitBranch <warn> Could not fetch git branch
26.11.2023
22:16:55
2023-11-26T21:16:55.122Z | game.populateGameDatabase <info> Populating game database...
26.11.2023
22:16:55
2023-11-26T21:16:55.122Z | game.populateGameDatabase <info> Game database populated with 130 games.
26.11.2023
22:16:55
2023-11-26T21:16:55.122Z | game.populateFavouriteGames <info> Populating favourite games...
26.11.2023
22:16:55
2023-11-26T21:16:55.123Z | game.populateFavouriteGames <info> Favourite games populated with 90 games.
26.11.2023
22:16:55
2023-11-26T21:16:55.123Z | dvr.loadChannelsConfig <info> Loading channel configs...
26.11.2023
22:16:55
2023-11-26T21:16:55.123Z | dvr.loadChannelsConfig <success> Loaded 17 channel configs!
26.11.2023
22:16:55
2023-11-26T21:16:55.124Z | dvr.loadChannelsConfig <warn> Channel folder sukidingels is not in channel config, left over?
26.11.2023
22:16:55
2023-11-26T21:16:55.124Z | tw.channel.loadChannelsCache <success> Loaded 19 channels from cache.
26.11.2023
22:16:55
2023-11-26T21:16:55.124Z | dvr.loadChannels <info> Loading channels...
26.11.2023
22:16:55
2023-11-26T21:16:55.124Z | dvr.loadChannels <info> Loading channel 322a1dc8-f09a-459b-8652-e4337a0859cc, provider twitch...
26.11.2023
22:16:55
2023-11-26T21:16:55.125Z | tw.channel.loadAbstract <warn> Channel gronkh is online, stale?
26.11.2023
22:16:55
2023-11-26T21:16:55.749Z | tw.channel.kodi <info> Wrote nfo file for gronkh to /usr/local/share/twitchautomator/data/storage/vods/gronkh/tvshow.nfo
26.11.2023
22:16:55
2023-11-26T21:16:55.786Z | vod.parseChapters <error> Time error for chapter parsing found for GRONKH2023 - s202311e09 (started_at: Sun Nov 26 2023 16:59:20 GMT+0000 (Coordinated Universal Time), ended_at: undefined)
26.11.2023
22:16:55
2023-11-26T21:16:55.786Z | vod.load <warn> Loaded VOD GRONKH2023 - s202311e09 is not finalized!
26.11.2023
22:16:55
🛠️ [channel parseVODs] GRONKH2023 - s202311e09 is capturing but process not running. Setting to false for fixing.
26.11.2023
22:16:55
2023-11-26T21:16:55.787Z | vod.saveJSON <warn> Saving JSON of GRONKH2023 - s202311e09 with no chapters!!
26.11.2023
22:16:55

```

After I read about the out of memory error I thouth wtf how, and checked on my system.
![grafik](https://github.com/MrBrax/LiveStreamDVR/assets/8904984/fec7a36d-206d-4eb8-8dd9-9805c5a6c88d)
18Gig it is using

Checked again with htop and this reported this:
![grafik](https://github.com/MrBrax/LiveStreamDVR/assets/8904984/2ebae9eb-fd8a-4b4c-b72b-7fb9092df405)

But still wtf is going on.

MrBrax · 2023-11-27T09:38:23Z

well shit, it's #444 i guess? hmm, wonder how i'd debug this

Serph91P · 2023-11-27T10:32:57Z

well shit, it's #444 i guess? hmm, wonder how i'd debug this

Add one of the streamers mentioned and add more debugging logs?

Serph91P · 2023-11-27T10:38:33Z

well shit, it's #444 i guess? hmm, wonder how i'd debug this

Dunno if this will help?

https://stackoverflow.com/questions/38558989/node-js-heap-out-of-memory

MrBrax · 2023-11-27T10:49:33Z

i've never gotten a memory leak on my server, so it would be difficult to create one

MrBrax · 2023-11-27T11:00:12Z

i suppose it could be worth disabling watching storage in config

Serph91P · 2023-11-27T11:40:21Z

i suppose it could be worth disabling watching storage in config

you mean this one?

MrBrax · 2023-11-27T12:16:32Z

yeah, it's the one thing that i know about that might affect memory. your logs say it's enabled so

Serph91P · 2023-11-27T12:19:41Z

yeah, it's the one thing that i know about that might affect memory. your logs say it's enabled so

I enabled it just now as I posted the screen. So maybe a bit confusing.

MrBrax · 2023-11-27T12:46:03Z

so it was disabled from the start? i do see entries in the logs saying it isn't...

Serph91P · 2023-11-27T12:47:00Z

so it was disabled from the start? i do see entries in the logs saying it isn't...

Like I said I enabled it before taking the screen.

related to #444 and #477

MrBrax · 2023-11-27T13:26:38Z

i noticed that the option did not disable all the watchers, the upcoming build should hopefully fix that

Serph91P · 2023-11-30T20:23:41Z

i noticed that the option did not disable all the watchers, the upcoming build should hopefully fix that

For the last couple of days everything was working better than before. But now no stream is recording. All seem to have failed because something related to this:

2023-11-30T20:13:22.120Z | automator.fallbackCapture Spawned process 115 for fbcapture_bonjwa_41473701928
21:13:22
2023-11-30T20:13:22.121Z | job.save Save job fbcapture_bonjwa_41473701928 with PID 115 to /usr/local/share/twitchautomator/data/cache/pids/fbcapture_bonjwa_41473701928.json
21:13:22
Redacted - - [30/Nov/2023:20:13:22 +0000] "POST /api/v0/channels/0e4ad0df-c0c2-4bbd-9cf1-1ca25d9b2b3a/force_record HTTP/1.1" 200 63 "-" "Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Mobile Safari/537.36"
21:13:22
2023-11-30T20:13:22.280Z | automator.captureTicker Invalid data received from streamlink
21:13:22
2023-11-30T20:13:22.280Z | automator.captureTicker Invalid data received from streamlink
21:13:22
2023-11-30T20:13:22.281Z | automator.captureTicker Invalid data received from streamlink
21:13:22
2023-11-30T20:13:22.281Z | automator.captureTicker Invalid data received from streamlink
21:13:22
2023-11-30T20:13:22.882Z | automator.captureTicker Invalid data received from streamlink
21:13:22
2023-11-30T20:13:22.882Z | automator.captureTicker Invalid data received from streamlink
21:13:22
2023-11-30T20:13:22.909Z | automator.fallbackCapture Job fbcapture_bonjwa_41473701928 exited with code 1, signal null
21:13:22
2023-11-30T20:13:22.909Z | job.clear Clear job fbcapture_bonjwa_41473701928 with PID 115
21:13:22
2023-11-30T20:13:22.909Z | job.clear Job fbcapture_bonjwa_41473701928 removed from jobs list
21:13:22
2023-11-30T20:13:22.927Z | automator.fallbackCapture Capture 41473701928_2023-11-30_20-13-22 failed
21:13:22
2023-11-30T20:13:22.927Z | automator.download Fallback capture failed for bonjwa: undefined
21:13:22
false
21:13:41
Redacted - - [30/Nov/2023:20:13:41 +0000] "GET /cache/covers/509658.jpg HTTP/1.1" 304 - "-" "Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Mobile Safari/537.36"
21:13:53
🎥 Myun2023 - s202311e19 1080p60 90.74 MB / 6343 kbps
21:14:06
2023-11-30T20:14:06.880Z | tw.channel.getStreams <i

Serph91P closed this as completed Nov 20, 2023

Serph91P reopened this Nov 20, 2023

MrBrax added a commit that referenced this issue Nov 27, 2023

more watchers disabled with setting

9f120ae

related to #444 and #477

settings not saved after update #477

settings not saved after update #477

Comments

Serph91P commented Nov 15, 2023

MrBrax commented Nov 15, 2023

Serph91P commented Nov 15, 2023

MrBrax commented Nov 15, 2023

Serph91P commented Nov 15, 2023

MrBrax commented Nov 15, 2023

Serph91P commented Nov 15, 2023

Serph91P commented Nov 20, 2023

MrBrax commented Nov 20, 2023

Serph91P commented Nov 20, 2023

Serph91P commented Nov 20, 2023

MrBrax commented Nov 20, 2023

Serph91P commented Nov 21, 2023

MrBrax commented Nov 21, 2023

Serph91P commented Nov 21, 2023

MrBrax commented Nov 21, 2023

Serph91P commented Nov 21, 2023

Serph91P commented Nov 21, 2023

MrBrax commented Nov 21, 2023

MrBrax commented Nov 22, 2023

Serph91P commented Nov 22, 2023

MrBrax commented Nov 22, 2023

Serph91P commented Nov 22, 2023

MrBrax commented Nov 22, 2023

Serph91P commented Nov 23, 2023

MrBrax commented Nov 23, 2023

Serph91P commented Nov 23, 2023

Serph91P commented Nov 23, 2023

MrBrax commented Nov 23, 2023

MrBrax commented Nov 24, 2023

Serph91P commented Nov 24, 2023

Serph91P commented Nov 24, 2023 • edited Loading

MrBrax commented Nov 24, 2023

Serph91P commented Nov 24, 2023

Serph91P commented Nov 24, 2023

MrBrax commented Nov 24, 2023

Serph91P commented Nov 24, 2023

MrBrax commented Nov 24, 2023 • edited Loading

MrBrax commented Nov 24, 2023

Serph91P commented Nov 25, 2023

MrBrax commented Nov 25, 2023

Serph91P commented Nov 25, 2023

Serph91P commented Nov 27, 2023

MrBrax commented Nov 27, 2023

Serph91P commented Nov 27, 2023 • edited Loading

Serph91P commented Nov 27, 2023 • edited Loading

MrBrax commented Nov 27, 2023

Serph91P commented Nov 27, 2023

Serph91P commented Nov 27, 2023

MrBrax commented Nov 27, 2023

MrBrax commented Nov 27, 2023

Serph91P commented Nov 27, 2023

MrBrax commented Nov 27, 2023

Serph91P commented Nov 27, 2023

MrBrax commented Nov 27, 2023

Serph91P commented Nov 27, 2023

MrBrax commented Nov 27, 2023

Serph91P commented Nov 30, 2023

Serph91P commented Nov 24, 2023 •

edited

Loading

MrBrax commented Nov 24, 2023 •

edited

Loading

Serph91P commented Nov 27, 2023 •

edited

Loading

Serph91P commented Nov 27, 2023 •

edited

Loading