Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bugfix] Parse video metadata more accurately; allow Range in fileserver #1342

Merged
merged 4 commits into from
Jan 16, 2023

Conversation

tsmethurst
Copy link
Contributor

@tsmethurst tsmethurst commented Jan 14, 2023

Description

If this is a code change, please include a summary of what you've coded, and link to the issue(s) it closes/implements.

If this is a documentation change, please briefly describe what you've changed and why.

This pull request implements a few fixes to issues that were preventing videos from being played properly in Tusky.

  • For certain videos, Tusky makes requests using the Range HTTP header to fetch the mp4 video and mp4 audio components separately. This failed before; now, by using a temporary file to get a readseeker, Range requests can be respected + videos that were broken before work properly. When neither Range nor If-Range headers are set, GtS will use the previous behavior of just streaming the file directly from storage.
  • Bitrate and duration were not being parsed correctly when decoding video metadata. This is fixed now: duration is the longest of either audio duration / video duration, and bitrate is audio bitrate + video bitrate.
  • We were serving unnecessary fields when serving media attachment json of videos (focus, size, etc). This PR fixes this by only populating those fields for images.

Checklist

Please put an x inside each checkbox to indicate that you've read and followed it: [ ] -> [x]

If this is a documentation change, only the first checkbox must be filled (you can delete the others if you want).

  • I/we have read the GoToSocial contribution guidelines.
  • I/we have discussed the proposed changes already, either in an issue on the repository, or in the Matrix chat.
  • I/we have performed a self-review of added code.
  • I/we have written code that is legible and maintainable by others.
  • I/we have commented the added code, particularly in hard-to-understand areas.
  • I/we have made any necessary changes to documentation.
  • I/we have added tests that cover new code.
  • I/we have run tests and they pass locally with the changes.
  • I/we have run go fmt ./... and golangci-lint run.

notes from matrix:

i figured out why video playback sometimes fails in Tusky, and it's silly

for videos that have metadata encoded at the start of the video, tusky seems to make two requests: one for the whole video file, and then once just for the audio component of the video

in terms of request headers, it looks like this:

Jan 14 15:14:38 goblin.technology gotosocial[474117]: map[Accept-Encoding:[gzip] Connection:[upgrade] User-Agent:[stagefright/1.2 (Linux;Android 12)] X-Forwarded-For:[redacted] X-Forwarded-Proto:[https]]
Jan 14 15:14:38 goblin.technology gotosocial[474117]: timestamp="14/01/2023 15:14:38.939" func=middleware.Logger.func1.1 level=INFO latency=6.112578ms clientIP=redacted userAgent="stagefright/1.2 (Linux;Android 12)" method=GET statusCode=200 path=/fileserver/019H1N6E7XN43Z0PBQWG8WM8T6/attachment/original/01C26XG2XNYHYFJ7AV6BDXEX3C.mp4 msg="OK: wrote 1.34MiB"
Jan 14 15:14:39 goblin.technology gotosocial[474117]: map[Accept-Encoding:[gzip] Connection:[upgrade] Range [bytes=1137860-] User-Agent:[stagefright/1.2 (Linux;Android 12)] X-Forwarded-For:[redacted] X-Forwarded-Proto:[https]]
Jan 14 15:14:39 goblin.technology gotosocial[474117]: timestamp="14/01/2023 15:14:39.648" func=middleware.Logger.func1.1 level=INFO latency=3.636182ms clientIP=redacted userAgent="stagefright/1.2 (Linux;Android 12)" method=GET statusCode=200 path=/fileserver/019H1N6E7XN43Z0PBQWG8WM8T6/attachment/original/01C26XG2XNYHYFJ7AV6BDXEX3C.mp4 msg="OK: wrote 1.34MiB"

so in the first pair of lines, you can see the request headers and then the normal log of the response

in the second pair of lines, you can see that the Tusky user agent is using the Range header to specify that it only wants the last snippet of the file, which is the audio stream

GoToSocial just returns the whole file there, because it doesn't support parsing Range headers

so, i believe we can implement serving Range requests properly using https://pkg.go.dev/net/http#ServeContent

but unfortunately that requires that we wrap the file content in a ReadSeeker, whereas now it's just a ReadCloser iirc

anyway that's also the reason videos that don't work in tusky will work in a web client, because web clients don't bother with range stuff

@tsmethurst tsmethurst merged commit d4cddf4 into main Jan 16, 2023
@tsmethurst tsmethurst deleted the video_serving_updates branch January 16, 2023 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant