Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

concatenate segments #491

Merged
merged 1 commit into from
Mar 2, 2023
Merged

concatenate segments #491

merged 1 commit into from
Mar 2, 2023

Conversation

emranemran
Copy link
Collaborator

@emranemran emranemran commented Feb 27, 2023

The segments returned from the T are first stored in the order they are
received, and then sorted according to the segment index, and then
concatenated into a single .ts file. A nested map with an outer and
inner table is used to temporarily store the segment data streams.

TODOs (will use follow-up PR for these):

  • write concatenated ts segment to disk
  • fix/add tests

@emranemran emranemran force-pushed the emran/concat-ts-segments branch 2 times, most recently from d84c111 to 3761489 Compare February 27, 2023 22:44
@emranemran emranemran marked this pull request as ready for review February 27, 2023 22:44
@emranemran emranemran changed the title Emran/concat ts segments concatenate segments Feb 27, 2023
func (s *TSegmentList) AddSegmentData(segIdx int, data []byte) {
s.mu.Lock()
s.SegmentDataTable[segIdx] = data
s.mu.Unlock()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could do this with a defer like your other examples

Copy link
Contributor

@mjh1 mjh1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall lgtm 👍

@emranemran
Copy link
Collaborator Author

emranemran commented Mar 1, 2023

Updates:

  • gated mp4 generation on GenerateMP4 flag

Also, looks like lint caught some issues that I'll need to address as well.

@emranemran emranemran changed the base branch from main to emran/refactor-mp4-flag March 1, 2023 06:00
@codecov
Copy link

codecov bot commented Mar 1, 2023

Codecov Report

Merging #491 (b165e0f) into main (78e67c4) will decrease coverage by 0.63037%.
The diff coverage is 12.50000%.

Impacted file tree graph

@@                 Coverage Diff                 @@
##                main        #491         +/-   ##
===================================================
- Coverage   55.04615%   54.41578%   -0.63037%     
===================================================
  Files             40          41          +1     
  Lines           3250        3295         +45     
===================================================
+ Hits            1789        1793          +4     
- Misses          1324        1362         +38     
- Partials         137         140          +3     
Impacted Files Coverage Δ
clients/transcode_provider.go 91.89189% <ø> (ø)
pipeline/external.go 0.00000% <0.00000%> (ø)
pipeline/mist.go 10.74766% <0.00000%> (-0.05046%) ⬇️
video/media.go 0.00000% <0.00000%> (ø)
transcode/transcode.go 58.91892% <18.75000%> (-4.02226%) ⬇️
pipeline/coordinator.go 78.73016% <40.00000%> (-0.62468%) ⬇️
clients/mediaconvert.go 70.88949% <100.00000%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c95dc31...b165e0f. Read the comment docs.

Impacted Files Coverage Δ
clients/transcode_provider.go 91.89189% <ø> (ø)
pipeline/external.go 0.00000% <0.00000%> (ø)
pipeline/mist.go 10.74766% <0.00000%> (-0.05046%) ⬇️
video/media.go 0.00000% <0.00000%> (ø)
transcode/transcode.go 58.91892% <18.75000%> (-4.02226%) ⬇️
pipeline/coordinator.go 78.73016% <40.00000%> (-0.62468%) ⬇️
clients/mediaconvert.go 70.88949% <100.00000%> (ø)

Comment on lines 115 to 121
if transcodeRequest.GenerateMP4 {
for _, profile := range transcodeProfiles {
renditionList.AddRenditionSegment(profile.Name,
video.TSegmentList{
SegmentDataTable: make(map[int][]byte),
})
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it could be nice to just late initialise these inside video/media.go when calls are made to GetSegmentList further down

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the rest of the usage of the table happens inside concurrent routines, I wanted to keep the logic simplified and easier to debug by pre-creating the empty nested maps (tables) so that we're just getting/inserting in the go-routines.

Base automatically changed from emran/refactor-mp4-flag to main March 1, 2023 14:50
@emranemran emranemran force-pushed the emran/concat-ts-segments branch 3 times, most recently from e0bfd82 to 5629834 Compare March 1, 2023 22:23
… file

The segments returned from the T are first stored in the order they are
received, and then sorted according to the segment index, and then
concatenated into a single .ts file. A nested map with an outer and
inner table is used to temporarily store the segment data streams.
@emranemran
Copy link
Collaborator Author

Updates:

  • Addressed some good lint errors where I was copying a mutex around. All fixed now.
  • Unit tests will be added in a follow-up PR.

@emranemran emranemran merged commit b003b72 into main Mar 2, 2023
@emranemran emranemran deleted the emran/concat-ts-segments branch March 2, 2023 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants