Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add media duration to lpms_get_codec_info for GetCodecInfo #407

Merged
merged 27 commits into from
Jul 11, 2024
Merged

Add media duration to lpms_get_codec_info for GetCodecInfo #407

Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
eef214e
add fps and duration to GetCodecInfo
ad-astra-video Oct 31, 2023
1fcc47e
update fps and dur calc
ad-astra-video Nov 3, 2023
904f309
add duration if video or audio present
ad-astra-video Jul 3, 2024
6a2b014
add cmd script to test fps and duration
ad-astra-video Jul 3, 2024
0e84f60
revert to using AV_TIME_BASE and add notes for future improvements
ad-astra-video Jul 3, 2024
2b723a4
Fix duration for audio only file
eliteprox Jul 5, 2024
b601475
Code cleanup
eliteprox Jul 5, 2024
bad8498
Fix duration calculation
eliteprox Jul 5, 2024
146d718
Cleanup test
eliteprox Jul 5, 2024
75bd67c
Merge branch 'add-codec-info' of github.com:eliteprox/lpms into add-c…
eliteprox Jul 8, 2024
e6389f2
Move fps duration test to ffmpeg test, add descriptive comment
eliteprox Jul 8, 2024
f05997e
Add tests, cleanup from review
eliteprox Jul 9, 2024
f0eda1a
Generate test files with ffmpeg, check expected duration and fps, rem…
eliteprox Jul 9, 2024
f3c8917
Remove test file
eliteprox Jul 9, 2024
d025d0a
Fix tests and review comments
eliteprox Jul 9, 2024
6579e6e
Save indention
eliteprox Jul 9, 2024
67db771
Remove seek
eliteprox Jul 9, 2024
cc58010
simplify test
eliteprox Jul 10, 2024
8cdef44
Add webm type to test
eliteprox Jul 10, 2024
a8f19c4
add max_analyze_duration and remove strays
eliteprox Jul 10, 2024
d99d32b
Optimize duration calculation
eliteprox Jul 10, 2024
f2edd37
Use the largest pts for last_pts and smallest pts for first
eliteprox Jul 10, 2024
1fd0f5c
Remove calculate_stream_duration, add ffprobe to tests
eliteprox Jul 11, 2024
afaf25d
remove max_analyze_duration
eliteprox Jul 11, 2024
56c2328
Update ffmpeg/ffmpeg_test.go
eliteprox Jul 11, 2024
0e12477
Update ffmpeg/extras.c
eliteprox Jul 11, 2024
c85f817
add ffprobe expected duration and fps values to test
eliteprox Jul 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions ffmpeg/extras.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,29 @@ int lpms_rtmp2hls(char *listen, char *outf, char *ts_tmpl, char* seg_time, char
return ret == AVERROR_EOF ? 0 : ret;
}

//Calculates the duration of audio stream by counting the number of audio packets
double calculate_stream_duration(AVFormatContext *ic, int astream) {
AVPacket pkt;
av_init_packet(&pkt);
double duration = 0;
int64_t last_pts = AV_NOPTS_VALUE;
// Seek to the beginning of the audio stream
while (av_read_frame(ic, &pkt) >= 0) {
eliteprox marked this conversation as resolved.
Show resolved Hide resolved
if (pkt.stream_index != astream) continue;
if (pkt.pts != AV_NOPTS_VALUE) {
if (last_pts != AV_NOPTS_VALUE) {
// Calculate the difference between the current and last PTS
int64_t pts_diff = pkt.pts - last_pts;
// Convert the PTS difference to seconds and add to duration
duration += pts_diff * av_q2d(ic->streams[astream]->time_base);
}
last_pts = pkt.pts;
}
av_packet_unref(&pkt);
}
return duration;
}

#define GET_CODEC_INTERNAL_ERROR -1
#define GET_CODEC_OK 0
#define GET_CODEC_NEEDS_BYPASS 1
Expand Down Expand Up @@ -164,6 +187,12 @@ int lpms_get_codec_info(char *fname, pcodec_info out)
// instead of returning -1
ret = GET_CODEC_STREAMS_MISSING;
}

if (ic->duration != AV_NOPTS_VALUE) {
out->dur = ic->duration / AV_TIME_BASE;
eliteprox marked this conversation as resolved.
Show resolved Hide resolved
} else {
out->dur = calculate_stream_duration(ic, audio_present ? astream : vstream);
}
// Return
if (video_present && vc->name) {
strncpy(out->video_codec, vc->name, MIN(strlen(out->video_codec), strlen(vc->name))+1);
Expand All @@ -176,6 +205,7 @@ int lpms_get_codec_info(char *fname, pcodec_info out)
}
out->width = ic->streams[vstream]->codecpar->width;
out->height = ic->streams[vstream]->codecpar->height;
out->fps = av_q2d(ic->streams[vstream]->avg_frame_rate);
eliteprox marked this conversation as resolved.
Show resolved Hide resolved
} else {
// Indicate failure to extract video codec from given container
out->video_codec[0] = 0;
Expand Down
2 changes: 2 additions & 0 deletions ffmpeg/extras.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ typedef struct s_codec_info {
int pixel_format;
int width;
int height;
double fps;
double dur;
} codec_info, *pcodec_info;

int lpms_rtmp2hls(char *listen, char *outf, char *ts_tmpl, char *seg_time, char *seg_start);
Expand Down
5 changes: 5 additions & 0 deletions ffmpeg/ffmpeg.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,8 @@ type MediaFormatInfo struct {
Acodec, Vcodec string
PixFormat PixelFormat
Width, Height int
FPS float32
DurSecs float32
}

func (f *MediaFormatInfo) ScaledHeight(width int) int {
Expand Down Expand Up @@ -277,6 +279,9 @@ func GetCodecInfo(fname string) (CodecStatus, MediaFormatInfo, error) {
format.PixFormat = PixelFormat{int(params_c.pixel_format)}
format.Width = int(params_c.width)
format.Height = int(params_c.height)
format.FPS = float32(params_c.fps)
format.DurSecs = float32(params_c.dur)

return status, format, nil
}

Expand Down
42 changes: 42 additions & 0 deletions ffmpeg/ffmpeg_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1869,3 +1869,45 @@ func TestResolution_Clamp(t *testing.T) {
checkError = require.Error
test(l, Size{300, 600}, portrait, Size{300, 600})
}

func TestDurationFPS_GetCodecInfo(t *testing.T) {
run, dir := setupTest(t)
defer os.RemoveAll(dir)

//Generate test files
cmd := `
cp "$1/../data/bunny.mp4" test.mp4
cp "$1/../data/duplicate-audio-dts.ts" test.ts
ffmpeg -loglevel warning -i test.mp4 -vn -c:a flac test.flac
Copy link
Collaborator

@j0sh j0sh Jul 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is good, it would be nice to also add checks for expected fps/duration values from ffprobe - helps ensure that our own calculations are correct

also just checking, which one of these triggers the no-duration case that requires manual calculations? (that should also surface via ffprobe, eg a duration of "N/A")

Copy link
Contributor Author

@eliteprox eliteprox Jul 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be with webm and mp3 files. ffprobe -show_streams test.webm | grep 'duration=' gives N/A for duration. I've added webm to the test, it is working with the latest changes.

Copy link
Collaborator

@j0sh j0sh Jul 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a look and the duration is there for the top-level container (ic->duration) for both webm and mp3 files. You can also see this in the top-level ffprobe output, or ffprobe -show_format. Stream duration is indeed empty (which is the equivalent with ffprobe -show_streams) but we aren't using that here.

If we need to introduce a code path to manually calculate the duration then I would prefer to find a test case which can exercise that.

Also please add the ffprobe outputs to the test case as well, because this validates the assumptions we are testing against, and allows us to spot areas where we diverge from ffmpeg's own behavior. For example:

ffmpeg -loglevel warning -i test.mp4 -c:v libvpx -c:a vorbis -strict -2 -t 1 test.webm
ffprobe -loglevel warning -show_format test.webm | grep duration=1.047000
ffprobe -loglevel warning -show_streams -select_streams v test.webm | grep r_frame_rate=24/1

this example is shortened to 1 second; it doesn't really need to be very long for the purpose of this test case and speeds up test execution since its encoding video which is expensive

BTW the durations you have in these test cases seem very round - do you care about the fractional part of the second at all? If not, then maybe make it an integer type instead of floating point. Otherwise you'll need to cast to double when calculating the duration, eg out->dur = (double)ic->duration / AV_TIME_BASE;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the ffprobe commands, shortened all the videos to 2 seconds. The int64 change is working well so I removed calculate_stream_duration. I am happy with rounding numbers for now

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am happy with rounding numbers for now

Double checking (no pun intended) ... anything less than 1 (eg, 0.99) will be rounded down to 0, 1.99 becomes 1, etc. That seems pretty significant (eg, how do you distinguish between "very short" and "zero duration") but if is still OK for your purposes, then 👍

Otherwise, I believe we can still match the values calculated by ffprobe - our equivalent would be out->dur = ic->duration * av_q2d(AV_TIME_BASE_Q) ... mathematically the same as the old code, but I don't know if the computed precision would differ (and it would only come into play for very large durations, anyhow)

`
run(cmd)

files := []struct {
Filename string
Duration float32
FPS float32
}{
{Filename: "test.mp4", Duration: 60.0, FPS: 23.96206},
{Filename: "test.ts", Duration: 1.9737, FPS: 30.0},
{Filename: "test.flac", Duration: 60.0, FPS: 0.0},
}
for _, file := range files {
start := time.Now()
fname := path.Join(dir, file.Filename)
_, format, err := GetCodecInfo(fname)
data, err := ioutil.ReadFile(fname)
if err != nil {
t.Error(err)
}

_, format, err = GetCodecInfoBytes(data)
if err != nil {
t.Error(err)
}

took := time.Since(start).Milliseconds()
fmt.Printf("%v\ttook=%v\tdur=%v\fps=%v\tvcodec=%v\tacodec=%v\n", fname, took, format.DurSecs, format.FPS, format.Vcodec, format.Acodec)
assert.True(t, format.DurSecs == file.Duration, "Duration should match expected value=%f, actual=%f", file.Duration, format.DurSecs)
assert.True(t, format.FPS == file.FPS, "FPS should match expected value=%f, actual=%f", file.FPS, format.FPS)
eliteprox marked this conversation as resolved.
Show resolved Hide resolved
}
}