Process Upload VOD #20
Conversation
| TriggerCallback: fmt.Sprintf("http://localhost:%d/api/mist/trigger", *port), | ||
| } | ||
|
|
||
| listen := fmt.Sprintf("localhost:%d", *port) |
There was a problem hiding this comment.
Ah! This change reverted the listen address to localhost 😢
There was a problem hiding this comment.
Fixed everywhere else in #22
| @@ -209,7 +217,7 @@ func commandUpdateTrigger(streamName, triggerName string, currentTriggers Trigge | |||
| return MistConfig{Config{Triggers: triggersMap}} | |||
There was a problem hiding this comment.
FYI: I'm testing the integration between task-runner and catalyst-api, and I'm getting a panic on the line above (:216) of assignment to entry in nil map:
[2022-08-12 22:34:52] livepeer-catalyst-api (205) INFO: ts=2022-08-12T22:34:52.472726115Z err="assignment to entry in nil map" trace="goroutine 25 [running]:\nruntime/debug.Stack()\n\t/opt/hostedtoolcache/go/1.19.0/x64/src/runtime/debug/stack.go:24 +0x65\ngithub.com/livepeer/catalyst-api/middleware.LogRequest.func1.1.1()\n\t/home/runner/work/catalyst-api/catalyst-api/middleware/logging.go:44 +0x8f\npanic({0x7b1320, 0x8b4260})\n\t/opt/hostedtoolcache/go/1.19.0/x64/src/runtime/panic.go:884 +0x212\ngithub.com/livepeer/catalyst-api/handlers.commandUpdateTrigger(...)\n\t/home/runner/work/catalyst-api/catalyst-api/handlers/client.go:216\ngithub.com/livepeer/catalyst-api/handlers.commandAddTrigger({0xc0002464c8, 0x15}, {0x80e24c, 0x8}, {0xc000236330?, 0x0?}, 0x0?)\n\t/home/runner/work/catalyst-api/catalyst-api/handlers/client.go:200 +0x277\ngithub.com/livepeer/catalyst-api/handlers.(*MistClient).AddTrigger(0xc000215140, {0xc0002464c8, 0x15}, {0x80e24c, 0x8})\n\t/home/runner/work/catalyst-api/catalyst-api/handlers/client.go:58 +0x125\ngithub.com/livepeer/catalyst-api/handlers.(*CatalystAPIHandlersCollection).processUploadVOD(0xc00021c150, {0xc0002464c8, 0x15}, {0xc0002024e0?, 0xc00025cec0?}, {0xc0002444d0, 0xa4})\n\t/home/runner/work/catalyst-api/catalyst-api/handlers/handlers.go:164 +0x74\ngithub.com/livepeer/catalyst-api/handlers.(*CatalystAPIHandlersCollection).UploadVOD.func1({0x8b7638?, 0xc000242a40}, 0xc00028e400, {0x0?, 0xc00029e7e0?, 0xc0000e48e0?})\n\t/home/runner/work/catalyst-api/catalyst-api/handlers/handlers.go:129 +0x3a6\ngithub.com/livepeer/catalyst-api/middleware.IsAuthorized.func1({0x8b7638, 0xc000242a40}, 0xc00028e400, {0x0, 0x0, 0x0})\n\t/home/runner/work/catalyst-api/catalyst-api/middleware/auth.go:29 +0x144\ngithub.com/livepeer/catalyst-api/middleware.LogRequest.func1.1({0x8b78a8?, 0xc000294540}, 0xc00028e400, {0x0, 0x0, 0x0})\n\t/home/runner/work/catalyst-api/catalyst-api/middleware/logging.go:48 +0x183\ngithub.com/julienschmidt/httprouter.(*Router).ServeHTTP(0xc0002222a0, {0x8b78a8, 0xc000294540}, 0xc00028e400)\n\t/home/runner/go/pkg/mod/github.com/julienschmidt/[email protected]/router.go:387 +0x81c\nnet/http.serverHandler.ServeHTTP({0x8b69d0?}, {0x8b78a8, 0xc000294540}, 0xc00028e400)\n\t/opt/hostedtoolcache/go/1.19.0/x64/src/net/http/server.go:2947 +0x30c\nnet/http.(*conn).serve(0xc0002272c0, {0x8b7d60, 0xc0002156b0})\n\t/opt/hostedtoolcache/go/1.19.0/x64/src/net/http/server.go:1991 +0x607\ncreated by net/http.(*Server).Serve\n\t/opt/hostedtoolcache/go/1.19.0/x64/src/net/http/server.go:3102 +0x4db\n"
There was a problem hiding this comment.
Reproduced locally by running catalyst-api and POSTing:
curl --location --request POST 'http://localhost:4949/api/vod' \
--header 'Authorization: Bearer IAmAuthorized' \
--header 'Content-Type: application/json' \
--data-raw '{
"url": "http://example.com",
"callback_url": "http://callback-lol.com",
"output_locations": [
{
"type": "object_store",
"url": "memory://localhost/output",
"outputs": {
"source_segments": true
}
}
]
}'
| defer mc.DeleteTrigger(streamName, "PUSH_END") | ||
|
|
||
| if err := mc.AddTrigger(streamName, "RECORDING_END"); err != nil { | ||
| if err := d.MistClient.PushStart(streamName, targetURL); err != nil { |
There was a problem hiding this comment.
So will mist call the catalyst-uploader in order to save to that targetURL? Cause from what I understood, this will come from the URL inside the first output_location to have a source_segments output set to true, right?
The URLs in the output_location are actually Object Store URLs compatible with the drivers lib moved to go-tools from go-livepeer. This means that Mist won't be able to o a single PUT to that URL. The catalyst-uploader does have support for it tho, but from what I understood it is receiving not only the Object Store URL but the path of the file on the same URL. So someone would have to append the file paths to this Object Store URL for each segmented file that is saved. Do you know if mist is doing that for calling the catalyst-api?
There was a problem hiding this comment.
Good, question, calling catalyst-uploader from Mist is being implemented by @stronk-dev , so Marco may have some insights here.
* aqueduct: Use prebuilt binaries for docker image * aqueduct: Compile everything from scratch for dockerised env * catalyst.go * workflow: Create github action to publish image to dockerhub * catalyst.yaml: Fix typo in secret token The custom checkout action is not needed. * Dockerfile.catalyst: Set catalyst binary as the entrypoint for docker container * catalyst: add working config file generation * catalyst.yaml: Use github caching to reduce docker image build time * Dockerfile: Add `EXPOSE`d port information to final image * Dockerfile: Switch go-livepeer to master branch. Slightly less verbose output during builds for go-livepeer * catalyst: add working --mode=api * catalyst: add ca-certificates * catalyst: fix typo in directory * catalyst: add mainnet support * catalyst: working -orchAddr support * catalyst: bump to catalyst-ui * bump catalyst-ui * Dockerfile.catalyst: switch back to go-livepeer master * Makefile: add make livepeer-catalyst-node Co-authored-by: hjpotter92 <[email protected]>
WIP