Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plugins.goodgame: rewrite using API #5586

Merged

Conversation

bastimeyer
Copy link
Member

@bastimeyer bastimeyer commented Oct 4, 2023

Fixes #5581
Fixes #5585
Fixes #5584
Fixes #5583

@IosifShmidt
Please check and verify.
https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#pull-request-feedback


  1. This now queries their API instead of finding the data in the page's embedded JSON.

    However, URLs using the plugin's default matcher without the /channel/ prefix appear to not have an API endpoint for querying the channel key, so it still needs to be acquired from the embedded JSON.
    https://goodgame.ru/html/api4docs/index.html

    That should work fine though:

    $ streamlink -l debug goodgame.ru/Penta | grep plugins.goodgame
    [plugins.goodgame][debug] channel='Mashinasmert'
    [plugins.goodgame][debug] Channel appears to be offline
    
    $ streamlink goodgame.ru/doesnotexist
    [cli][info] Found matching plugin goodgame for URL goodgame.ru/doesnotexist
    error: Unable to open URL: https://goodgame.ru/doesnotexist (404 Client Error: not found for url: https://goodgame.ru/doesnotexist)
    
  2. Game metadata can either be a string or null. This wasn't obvious in the recent plugin changes. Both the game metadata and stream title metadata are now treated as optional.

    The game title metadata now however is nested in another game object, so it's possible that the entire game object can become null instead of game.title. This needs to be verified first, otherwise we'll have another ValidationError.

  3. The player URL is now supported.

    $ streamlink 'goodgame.ru/player?46071'
    [cli][info] Found matching plugin goodgame for URL goodgame.ru/player?46071
    Available streams: 360p (worst), 540p, 720p, 1080p (best)
    
    $ streamlink --json 'goodgame.ru/player?46071' | jq .metadata
    {
      "id": "46071",
      "author": "AlMahom",
      "category": "Crusader Kings III",
      "title": "Мультиплеерные Династии"
    }
    
  4. Twitch redirection should now be supported unless I'm mistaken about the status and online flags in the API response. Both need to be true for the plugin to redirect to Twitch instead of trying to access the goodgame HLS URL. The online field is self-explanatory and to my understanding, status is true if the user has activated the Twitch player on their goodgame channel. Is that right?

    I couldn't find any live channels with a Twitch player.

  5. The old stream quality weights have been removed, since the player page revealed the URL of the HLS multivariant playlist. The quality weights were needed because of the individual media playlist URLs which were embedded in the JSON data previously.

@bastimeyer bastimeyer added the plugin issue A Plugin does not work correctly label Oct 4, 2023
Copy link
Member

@gravyboat gravyboat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved pending verification by issue reporter.

@IosifShmidt
Copy link

Hi, everything looks great, most streams are working fine.

However, I've found some unexpected problems.
Could you please add some logging function calls?
For example, it would be useful to see http requests at debug level and http responses at trace level.

As for the errors:

  1. https://goodgame.ru/channel/osbvmt%7CVitaTV
    1. curl https://goodgame.ru/api/4/streams/2/channel/osbvmt%7CVitaTV
      {"error":"Not found"}
    2. curl https://goodgame.ru/api/4/streams/2/channel/osbvmt|VitaTV
      {
        "id": 201098,
        "key": "osbvmt|VitaTV",
        "preview": "https://hls.goodgame.ru/previews/201098_240.jpg",
        "title": "OSBVMT|VitaTV",
        "viewers": 1,
        "streamer": {
          "id": 1580220,
          "obj_key": "9:1580220",
          "nickname": "osbvmt1VitaTV",
          "username": "osbvmt1VitaTV",
          "avatar": "https://goodgame.ru//files/avatars/av_1580220_kjzI.jpg",
          "premium": false,
          "banned": false
        },
        "streamKey": "201098",
        "online": true,
        "game": {
          "id": 0,
          "title": null,
          "url": "/games/"
        },
        "poster": "https://goodgame.ru/files/logotypes/ch_201098_7hl4_logo.png",
        "hidden": false,
        "adult": false,
        "link": "https://goodgame.ru/channel/osbvmt%7CVitaTV/",
        "premium": false,
        "players": [
          {
            "title": "GoodGame",
            "status": true,
            "gg": true,
            "online": true,
            "content": "<iframe frameborder=\"0\" width=\"100%\" height=\"100%\" src=\"https://goodgame.ru/player?201098\"></iframe>"
          }
        ],
        "watchTogether": false,
        "premiums": 0,
        "followers": 0,
        "rating": {
          "value": 2406,
          "place": 108
        }
      }
    3. curl https://hls.goodgame.ru/manifest/201098_master.m3u8
      #EXTM3U
      #EXT-X-VERSION:3
      #GG-Balancer:1.0
      #EXT-X-STREAM-INF:BANDWIDTH=3149240,RESOLUTION=1280x720
      https://hls.goodgame.ru/hls/201098.m3u8
      
  2. https://goodgame.ru/channel/sunishere/
    1. curl https://goodgame.ru/api/4/streams/2/channel/sunishere
      {
        "id": 121276,
        "key": "sunishere",
        "preview": "https://hls.goodgame.ru/previews/121276_240.jpg",
        "title": "Привет, это мой первый стрим на Goodgame.ru",
        "viewers": 0,
        "streamer": {
          "id": 629604,
          "obj_key": "9:629604",
          "nickname": "sunishere",
          "username": "sunishere",
          "avatar": "https://goodgame.ru//files/avatars/av_avatar_elf.png",
          "premium": false,
          "banned": false
        },
        "streamKey": "121276",
        "online": true,
        "game": {
          "id": 34,
          "title": "Другое",
          "url": "/games/others"
        },
        "poster": "https://goodgame.ru/images/ico_tv.png",
        "hidden": true,
        "adult": true,
        "link": "https://goodgame.ru/channel/sunishere/",
        "premium": false,
        "players": [
          {
            "title": "Twitch",
            "status": false,
            "gg": false,
            "online": true,
            "content": "<iframe src=\"https://player.twitch.tv/?parent=goodgame.ru&channel=mintymouse\" frameborder=\"0\" allowfullscreen=\"true\" webkitallowfullscreen=\"true\" mozallowfullscreen=\"true\" scrolling=\"no\"  width=\"100%\" height=\"100%\"></iframe>"
          }
        ],
        "watchTogether": false,
        "premiums": 0,
        "followers": 44,
        "rating": {
          "value": 0,
          "place": 0
        }
      }
    2. curl -v https://hls.goodgame.ru/manifest/121276_master.m3u8
      stream is offline or not found

@bastimeyer
Copy link
Member Author

goodgame.ru/channel/osbvmt%7CVitaTV
goodgame.ru/channel/osbvmt|VitaTV

| is not a valid character for URL paths and must be percent-encoded to %7C according to RFC 3986 (section 2).

This is done via requests and urllib3 when making the HTTP request here:

While curl ignores the percent-encoding (check with --verbose)

$ curl -Is --http1.1 'https://goodgame.ru/channel/osbvmt%7CVitaTV' | head -n1
HTTP/1.1 404 not found
$ curl -Is --http1.1 'https://goodgame.ru/channel/osbvmt|VitaTV' | head -n1
HTTP/1.1 200 OK

Web browsers like Chromium for example follow RFC 3986 correctly, whereas Firefox does not:

>>> (await fetch("https://goodgame.ru/channel/osbvmt%7CVitaTV")).status
404
>>> (await fetch("https://goodgame.ru/channel/osbvmt|VitaTV")).status
404

Since there is no control over how urllib3 encodes the URL path, this channel is simply not accessible. They need to fix their server/API and compare the channel name from the URL's path after percent-decoding it.

Could you please add some logging function calls?
For example, it would be useful to see http requests at debug level and http responses at trace level.

Use a Python debugger. Streamlink CLI does only handle output streams of the streamlink root logger. Loggers of other modules like urllib3 need to be set up via the Python API. This is intentional.

goodgame.ru/channel/sunishere

That channel is offline. You must've checked this while it was going offline. I don't see any issues here. The plugin simply checks the online status of the API response, as you can see in the plugin code.

@IosifShmidt
Copy link

IosifShmidt commented Oct 5, 2023

They need to fix their server/API and compare the channel name from the URL's path after percent-decoding it.

I've sent an email to support, they will probably fix this problem because it's not just the API problem. The website has the same problem.

That channel is offline. You must've checked this while it was going offline.

"online": true,

It was online at that moment, but it only has a twitch player.

Other twitch players
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/9743339/
error: Unable to open URL: https://hls.goodgame.ru/manifest/22640_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/22640_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/dokenz/
error: Unable to open URL: https://hls.goodgame.ru/manifest/141117_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/141117_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/resiagoodboy/
error: Unable to open URL: https://hls.goodgame.ru/manifest/17587_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/17587_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/LimGlHf/
error: Unable to open URL: https://hls.goodgame.ru/manifest/25214_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/25214_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/xLife/
error: Unable to open URL: https://hls.goodgame.ru/manifest/19743_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/19743_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/trancezistar/
error: Unable to open URL: https://hls.goodgame.ru/manifest/20206_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/20206_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/woops/
error: Unable to open URL: https://hls.goodgame.ru/manifest/25192_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/25192_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/hstoxicboy/
error: Unable to open URL: https://hls.goodgame.ru/manifest/19620_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/19620_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/Kalem81127/
error: Unable to open URL: https://hls.goodgame.ru/manifest/16238_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/16238_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/CorbenDallas/
error: Unable to open URL: https://hls.goodgame.ru/manifest/7395_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/7395_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/Artosis/
error: Unable to open URL: https://hls.goodgame.ru/manifest/1055_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/1055_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/TorNis/
error: Unable to open URL: https://hls.goodgame.ru/manifest/1152_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/1152_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/Zerocul/
error: Unable to open URL: https://hls.goodgame.ru/manifest/2327_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/2327_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/GGRivolF/
error: Unable to open URL: https://hls.goodgame.ru/manifest/5846_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/5846_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/Daopa/
error: Unable to open URL: https://hls.goodgame.ru/manifest/7854_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/7854_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/vip_koss/
error: Unable to open URL: https://hls.goodgame.ru/manifest/10457_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/10457_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/Axon228/
error: Unable to open URL: https://hls.goodgame.ru/manifest/11257_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/11257_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/streambot/
error: Unable to open URL: https://hls.goodgame.ru/manifest/12654_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/12654_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/Drimkast/
error: Unable to open URL: https://hls.goodgame.ru/manifest/197231_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/197231_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/Hance/
error: Unable to open URL: https://hls.goodgame.ru/manifest/13111_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/13111_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/itsHafu/
error: Unable to open URL: https://hls.goodgame.ru/manifest/14076_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/14076_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/Genrix_161rus/
error: Unable to open URL: https://hls.goodgame.ru/manifest/128013_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/128013_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/welderb/
error: Unable to open URL: https://hls.goodgame.ru/manifest/26421_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/26421_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/deadscum/
error: Unable to open URL: https://hls.goodgame.ru/manifest/181478_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/181478_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/cuteavalanche/
error: Unable to open URL: https://hls.goodgame.ru/manifest/154280_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/154280_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/_ForGG_/
error: Unable to open URL: https://hls.goodgame.ru/manifest/142736_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/142736_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/ripnat/
error: Unable to open URL: https://hls.goodgame.ru/manifest/193046_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/193046_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/VooDooSh/
error: Unable to open URL: https://hls.goodgame.ru/manifest/146613_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/146613_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/BarrinMW/
error: Unable to open URL: https://hls.goodgame.ru/manifest/189092_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/189092_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/bzzzz/
error: Unable to open URL: https://hls.goodgame.ru/manifest/155582_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/155582_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/Xan_x/
error: Unable to open URL: https://hls.goodgame.ru/manifest/162710_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/162710_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/Persona404/
error: Unable to open URL: https://hls.goodgame.ru/manifest/163903_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/163903_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/shroud/
error: Unable to open URL: https://hls.goodgame.ru/manifest/121106_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/121106_master.m3u8)
[cli][info] Found matching plugin goodgame for URL https://goodgame.ru/channel/Tiger_rus/
error: Unable to open URL: https://hls.goodgame.ru/manifest/109653_master.m3u8 (404 Client Error: Not Found for url: https://hls.goodgame.ru/manifest/109653_master.m3u8)

curl https://goodgame.ru/api/4/streams/2/channel/Verginiab | jq '. | .link, .players[]'

https://goodgame.ru/channel/Verginiab/
{
  "title": "GoodGame",
  "status": true,
  "gg": true,
  "online": true,
  "content": "<iframe frameborder=\"0\" width=\"100%\" height=\"100%\" src=\"https://goodgame.ru/player?178133\"></iframe>"
}
{
  "title": "Twitch",
  "status": false,
  "gg": false,
  "online": true,
  "content": "<iframe src=\"https://player.twitch.tv/?parent=goodgame.ru&channel=verginiab\" frameborder=\"0\" allowfullscreen=\"true\" webkitallowfullscreen=\"true\" mozallowfullscreen=\"true\" scrolling=\"no\"
  width=\"100%\" height=\"100%\"></iframe>"
}

It seems that status has the same value as gg.

Sometimes this can be a bit weird. The stream is online, but there are no players.

https://goodgame.ru/channel/Starsky_TV/
{
  "id": 76832,
  "key": "Starsky_TV",
  "preview": "https://hls.goodgame.ru/previews/76832_240.jpg",
  "title": "Комментатор в деле (Starsky stream)",
  "viewers": 0,
  "streamer": {
    "id": 740896,
    "obj_key": "9:740896",
    "nickname": "Starsky_TV",
    "username": "Starsky_TV",
    "avatar": "https://goodgame.ru//files/avatars/av_avatar_hum.png",
    "premium": false,
    "banned": false
  },
  "streamKey": "76832",
  "online": true,
  "game": {
    "id": 55788,
    "title": "Playerunknown's Battlegrounds",
    "url": "/games/playerunknowns-battlegrounds"
  },
  "poster": "https://goodgame.ru/files/logotypes/ch_76832_ODnM_logo.png",
  "hidden": false,
  "adult": false,
  "link": "https://goodgame.ru/channel/Starsky_TV/",
  "premium": false,
  "players": [],
  "watchTogether": false,
  "premiums": 0,
  "followers": 19,
  "rating": {
    "value": 0,
    "place": 0
  }
}

I wonder if the right solution would be to iterate over the players of the channel and select the first one that is online.

@bastimeyer
Copy link
Member Author

I've removed the status check of the embedded players. It now only checks the embedded player's online flag.

Embedded players also get checked when the channel's online flag is false or if the HLS multivariant playlist returns 404.

This should cover all the cases. Online channels, offline channels and offline channels with embedded players.

@IosifShmidt
Copy link

LGTM!

@bastimeyer bastimeyer merged commit 8fbd795 into streamlink:master Oct 6, 2023
23 checks passed
@bastimeyer bastimeyer deleted the plugins/goodgame/api-rewrite branch October 6, 2023 01:16
@bastimeyer
Copy link
Member Author

Thanks for checking!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
plugin issue A Plugin does not work correctly
Projects
None yet
3 participants