Low latency DASH support

docs/source/options/dash_options.rst

@@ -95,3 +95,8 @@ DASH options
 
     If enabled, allow adaptive switching between different codecs, if they have 
     the same language, media type (audio, video etc) and container type.
+
+--low_latency_dash_mode
+
+    If enabled, LL-DASH streaming will be used,
+    reducing overall latency by decoupling latency from segment duration.

docs/source/tutorials/low_latency.rst

@@ -0,0 +1,103 @@
+####################################
+Low Latency DASH (LL-DASH) Streaming
+####################################
+
+************
+Introduction
+************
+
+If ``--low_latency_dash_mode`` is enabled, low latency DASH (LL-DASH) packaging will be used.
+
+This will reduce overall latency by ensuring that the media segments are chunk encoded and delivered via an aggregating response.
+The combination of these features will ensure that overall latency can be decoupled from the segment duration.
+For low latency to be achieved, the output of Shaka Packager must be combined with a delivery system which can chain together a set of aggregating responses, such as chunked transfer encoding under HTTP/1.1 or a HTTP/2 or HTTP/3 connection.
+The output of Shaka Packager must be played with a DASH client that understands the availabilityTimeOffset MPD value.
+Furthermore, the player should also understand the throughput estimation and ABR challenges that arise when operating in the low latency regime.
+
+This tutorial covers LL-DASH packaging and uses features from the DASH, HTTP upload, and FFmpeg piping tutorials.
+For more information on DASH, see :doc:`dash`; for HTTP upload, see :doc:`http_upload`;
+for FFmpeg piping, see :doc:`ffmpeg_piping`;
+for full documentation, see :doc:`/documentation`.
+
+*************
+Documentation
+*************
+
+Getting started
+===============
+
+To enable LL-DASH mode, set the ``--low_latency_dash_mode`` flag to ``true``. 
+
+All HTTP requests will use chunked transfer encoding:
+``Transfer-Encoding: chunked``.
+
+.. note::
+
+    Only LL-DASH is supported. LL-HLS support is yet to come.
+
+Synopsis
+========
+
+Here is a basic example of the LL-DASH support. 
+The LL-DASH setup borrows features from "FFmpeg piping" and "HTTP upload",
+see :doc:`ffmpeg_piping` and :doc:`http_upload`.
+
+Define UNIX pipe to connect ffmpeg with packager::
+
+    export PIPE=/tmp/bigbuckbunny.fifo
+    mkfifo ${PIPE}
+
+Acquire and transcode RTMP stream::
+
+    ffmpeg -fflags nobuffer -threads 0 -y \
+        -i rtmp://184.72.239.149/vod/mp4:bigbuckbunny_450.mp4 \
+        -pix_fmt yuv420p -vcodec libx264 -preset:v superfast -acodec aac \
+        -f mpegts pipe: > ${PIPE}
+
+Configure and run packager::
+
+    # Define upload URL
+    export UPLOAD_URL=http://localhost:6767/ll-dash
+
+    # Go
+    packager \
+        "input=${PIPE},stream=audio,init_segment=${UPLOAD_URL}_init.m4s,segment_template=${UPLOAD_URL}/bigbuckbunny-audio-aac-\$Number%04d\$.m4s" \
+        "input=${PIPE},stream=video,init_segment=${UPLOAD_URL}_init.m4s,segment_template=${UPLOAD_URL}/bigbuckbunny-video-h264-450-\$Number%04d\$.m4s" \
+        --io_block_size 65536 \
+        --segment_duration 2 \
+        --low_latency_dash_mode=true \
+        --utc_timings "urn:mpeg:dash:utc:http-xsdate:2014"="https://time.akamai.com/?iso" \
+        --mpd_output "${UPLOAD_URL}/bigbuckbunny.mpd" \
+
+
+*************************
+Low Latency Compatibility
+*************************
+
+For low latency to be achieved, the processes handling Shaka Packager's output, such as the server and player,
+must support LL-DASH streaming.
+
+Delivery Pipeline
+=================
+Shaka Packager will upload the LL-DASH content to the specified output via HTTP chunked transfer encoding.
+The server must have the ability to handle this type of request. If using a proxy or shim for cloud authentication,
+these services must also support HTTP chunked transfer encoding.
+
+Examples of supporting content delivery systems:
+
+* `AWS MediaStore <https://aws.amazon.com/mediastore/>`_
+* `s3-upload-proxy <https://github.com/fsouza/s3-upload-proxy>`_
+* `Streamline Low Latency DASH preview <https://github.com/streamlinevideo/low-latency-preview>`_
+* `go-chunked-streaming-server <https://github.com/mjneil/go-chunked-streaming-server>`_
+
+Player
+======
+The player must support LL-DASH playout.
+LL-DASH requires the player to be able to interpret ``availabilityTimeOffset`` values from the DASH MPD.
+The player should also recognize the the throughput estimation and ABR challenges that arise with low latency streaming.
+
+Examples of supporting players:
+
+* `Shaka Player <https://github.com/google/shaka-player>`_
+* `dash.js <https://github.com/Dash-Industry-Forum/dash.js>`_
+* `Streamline Low Latency DASH preview <https://github.com/streamlinevideo/low-latency-preview>`_

docs/source/tutorials/tutorials.rst

@@ -13,3 +13,4 @@ Tutorials
    ads.rst
    ffmpeg_piping.rst
    http_upload.rst
+   low_latency.rst

packager/app/mpd_flags.cc

@@ -75,3 +75,11 @@ DEFINE_bool(dash_force_segment_list,
             "content is huge and the total number of (sub)segment references "
             "is greater than what the sidx atom allows (65535). Currently "
             "this flag is only supported in DASH ondemand profile.");
+DEFINE_bool(
+    low_latency_dash_mode,
+    false,
+    "If enabled, LL-DASH streaming will be used, "
+    "reducing overall latency by decoupling latency from segment duration. "
+    "Please see "
+    "https://google.github.io/shaka-packager/html/tutorials/low_latency.html "
+    "for more information.");

packager/app/mpd_flags.h

@@ -24,5 +24,6 @@ DECLARE_bool(allow_approximate_segment_timeline);
 DECLARE_bool(allow_codec_switching);
 DECLARE_bool(include_mspr_pro_for_playready);
 DECLARE_bool(dash_force_segment_list);
+DECLARE_bool(low_latency_dash_mode);
 
 #endif  // APP_MPD_FLAGS_H_

packager/app/packager_main.cc

@@ -325,6 +325,7 @@ base::Optional<PackagingParams> GetPackagingParams() {
   ChunkingParams& chunking_params = packaging_params.chunking_params;
   chunking_params.segment_duration_in_seconds = FLAGS_segment_duration;
   chunking_params.subsegment_duration_in_seconds = FLAGS_fragment_duration;
+  chunking_params.low_latency_dash_mode = FLAGS_low_latency_dash_mode;
   chunking_params.segment_sap_aligned = FLAGS_segment_sap_aligned;
   chunking_params.subsegment_sap_aligned = FLAGS_fragment_sap_aligned;
 
@@ -435,6 +436,7 @@ base::Optional<PackagingParams> GetPackagingParams() {
   mp4_params.generate_sidx_in_media_segments =
       FLAGS_generate_sidx_in_media_segments;
   mp4_params.include_pssh_in_stream = FLAGS_mp4_include_pssh_in_stream;
+  mp4_params.low_latency_dash_mode = FLAGS_low_latency_dash_mode;
 
   packaging_params.transport_stream_timestamp_offset_ms =
       FLAGS_transport_stream_timestamp_offset_ms;
@@ -474,6 +476,7 @@ base::Optional<PackagingParams> GetPackagingParams() {
       FLAGS_allow_approximate_segment_timeline;
   mpd_params.allow_codec_switching = FLAGS_allow_codec_switching;
   mpd_params.include_mspr_pro = FLAGS_include_mspr_pro_for_playready;
+  mpd_params.low_latency_dash_mode = FLAGS_low_latency_dash_mode;
 
   HlsParams& hls_params = packaging_params.hls_params;
   if (!GetHlsPlaylistType(FLAGS_hls_playlist_type, &hls_params.playlist_type)) {

packager/file/file.cc

@@ -184,6 +184,7 @@ File* File::CreateInternalFile(const char* file_name, const char* mode) {
   base::StringPiece real_file_name;
   const FileTypeInfo* file_type = GetFileTypeInfo(file_name, &real_file_name);
   DCHECK(file_type);
+  // Calls constructor for the derived File class.
   return file_type->factory_function(real_file_name.data(), mode);
 }
 

packager/file/http_file.cc

@@ -297,7 +297,7 @@ void HttpFile::SetupRequest() {
   curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
   curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, &CurlWriteCallback);
   curl_easy_setopt(curl, CURLOPT_WRITEDATA,
-                   method_ == HttpMethod::kPut ? nullptr : &download_cache_);
+                   method_ == HttpMethod::kGet ? &download_cache_ : nullptr);
   if (method_ != HttpMethod::kGet) {
     curl_easy_setopt(curl, CURLOPT_READFUNCTION, &CurlReadCallback);
     curl_easy_setopt(curl, CURLOPT_READDATA, &upload_cache_);

packager/media/base/media_handler.h

@@ -54,6 +54,8 @@ struct CueEvent {
 
 struct SegmentInfo {
   bool is_subsegment = false;
+  bool is_chunk = false;
+  bool is_final_chunk_in_seg = false;
   bool is_encrypted = false;
   int64_t start_timestamp = -1;
   int64_t duration = 0;

packager/media/chunking/chunking_handler.cc

@@ -112,7 +112,23 @@ Status ChunkingHandler::OnMediaSample(
       started_new_segment = true;
     }
   }
-  if (!started_new_segment && IsSubsegmentEnabled()) {
+
+  // This handles the LL-DASH case.
+  // On each media sample, which is the basis for a chunk,
+  // we must increment the current_subsegment_index_
+  // in order to hit FinalizeSegment() within Segmenter.
+  if (!started_new_segment && chunking_params_.low_latency_dash_mode) {
+    current_subsegment_index_++;
+
+    RETURN_IF_ERROR(EndSubsegmentIfStarted());
+    subsegment_start_time_ = timestamp;
+  }
+
+  // Here, a subsegment refers to a fragment that is within a segment.
+  // This fragment size can be set with the 'fragment_duration' cmd arg.
+  // This is NOT for the LL-DASH case.
+  if (!started_new_segment && IsSubsegmentEnabled() &&
+      !chunking_params_.low_latency_dash_mode) {
     const bool can_start_new_subsegment =
         sample->is_key_frame() || !chunking_params_.subsegment_sap_aligned;
     if (can_start_new_subsegment) {
@@ -151,6 +167,10 @@ Status ChunkingHandler::EndSegmentIfStarted() const {
   auto segment_info = std::make_shared<SegmentInfo>();
   segment_info->start_timestamp = segment_start_time_.value();
   segment_info->duration = max_segment_time_ - segment_start_time_.value();
+  if (chunking_params_.low_latency_dash_mode) {
+    segment_info->is_chunk = true;
+    segment_info->is_final_chunk_in_seg = true;
+  }
   return DispatchSegmentInfo(kStreamIndex, std::move(segment_info));
 }
 
@@ -163,6 +183,8 @@ Status ChunkingHandler::EndSubsegmentIfStarted() const {
   subsegment_info->duration =
       max_segment_time_ - subsegment_start_time_.value();
   subsegment_info->is_subsegment = true;
+  if (chunking_params_.low_latency_dash_mode)
+    subsegment_info->is_chunk = true;
   return DispatchSegmentInfo(kStreamIndex, std::move(subsegment_info));
 }
 

packager/media/chunking/chunking_handler_unittest.cc

@@ -207,5 +207,48 @@ TEST_F(ChunkingHandlerTest, CueEvent) {
                         kDuration, !kEncrypted, _)));
 }
 
+TEST_F(ChunkingHandlerTest, LowLatencyDash) {
+  ChunkingParams chunking_params;
+  chunking_params.low_latency_dash_mode = true;
+  chunking_params.segment_duration_in_seconds = 1;
+  SetUpChunkingHandler(1, chunking_params);
+
+  // Each completed segment will contain 2 chunks
+  const int64_t kChunkDurationInMs = 500;
+  const int64_t kSegmentDurationInMs = 1000;
+
+  ASSERT_OK(Process(StreamData::FromStreamInfo(
+      kStreamIndex, GetVideoStreamInfo(kTimeScale1))));
+
+  for (int i = 0; i < 4; ++i) {
+    ASSERT_OK(Process(StreamData::FromMediaSample(
+        kStreamIndex, GetMediaSample(i * kChunkDurationInMs, kChunkDurationInMs,
+                                     kKeyFrame))));
+  }
+
+  // NOTE: Each MediaSample will create a chunk, dispatching SegmentInfo
+  EXPECT_THAT(
+      GetOutputStreamDataVector(),
+      ElementsAre(
+          IsStreamInfo(kStreamIndex, kTimeScale1, !kEncrypted, _),
+          // Chunk 1 for segment 1
+          IsMediaSample(kStreamIndex, 0, kChunkDurationInMs, !kEncrypted, _),
+          IsSegmentInfo(kStreamIndex, 0, kChunkDurationInMs, kIsSubsegment,
+                        !kEncrypted),
+          // Chunk 2 for segment 1
+          IsMediaSample(kStreamIndex, kChunkDurationInMs, kChunkDurationInMs,
+                        !kEncrypted, _),
+          IsSegmentInfo(kStreamIndex, 0, 2 * kChunkDurationInMs, !kIsSubsegment,
+                        !kEncrypted),
+          // Chunk 1 for segment 2
+          IsMediaSample(kStreamIndex, kSegmentDurationInMs, kChunkDurationInMs,
+                        !kEncrypted, _),
+          IsSegmentInfo(kStreamIndex, kSegmentDurationInMs, kChunkDurationInMs,
+                        kIsSubsegment, !kEncrypted),
+          // Chunk 2 for segment 2
+          IsMediaSample(kStreamIndex, kSegmentDurationInMs + kChunkDurationInMs,
+                        kChunkDurationInMs, !kEncrypted, _)));
+}
+
 }  // namespace media
 }  // namespace shaka

packager/media/event/combined_muxer_listener.cc

@@ -43,12 +43,24 @@ void CombinedMuxerListener::OnMediaStart(const MuxerOptions& muxer_options,
   }
 }
 
+void CombinedMuxerListener::OnAvailabilityOffsetReady() {
+  for (auto& listener : muxer_listeners_) {
+    listener->OnAvailabilityOffsetReady();
+  }
+}
+
 void CombinedMuxerListener::OnSampleDurationReady(int32_t sample_duration) {
   for (auto& listener : muxer_listeners_) {
     listener->OnSampleDurationReady(sample_duration);
   }
 }
 
+void CombinedMuxerListener::OnSegmentDurationReady() {
+  for (auto& listener : muxer_listeners_) {
+    listener->OnSegmentDurationReady();
+  }
+}
+
 void CombinedMuxerListener::OnMediaEnd(const MediaRanges& media_ranges,
                                        float duration_seconds) {
   for (auto& listener : muxer_listeners_) {

packager/media/event/combined_muxer_listener.h

@@ -36,7 +36,9 @@ class CombinedMuxerListener : public MuxerListener {
                     const StreamInfo& stream_info,
                     int32_t time_scale,
                     ContainerType container_type) override;
+  void OnAvailabilityOffsetReady() override;
   void OnSampleDurationReady(int32_t sample_duration) override;
+  void OnSegmentDurationReady() override;
   void OnMediaEnd(const MediaRanges& media_ranges,
                   float duration_seconds) override;
   void OnNewSegment(const std::string& file_name,

packager/media/event/mpd_notify_muxer_listener.cc

@@ -105,6 +105,11 @@ void MpdNotifyMuxerListener::OnMediaStart(const MuxerOptions& muxer_options,
   }
 }
 
+// Record the availability time offset for LL-DASH manifests.
+void MpdNotifyMuxerListener::OnAvailabilityOffsetReady() {
+  mpd_notifier_->NotifyAvailabilityTimeOffset(notification_id_.value());
+}
+
 // Record the sample duration in the media info for VOD so that OnMediaEnd, all
 // the information is in the media info.
 void MpdNotifyMuxerListener::OnSampleDurationReady(int32_t sample_duration) {
@@ -127,6 +132,11 @@ void MpdNotifyMuxerListener::OnSampleDurationReady(int32_t sample_duration) {
   media_info_->mutable_video_info()->set_frame_duration(sample_duration);
 }
 
+// Record the segment duration for LL-DASH manifests.
+void MpdNotifyMuxerListener::OnSegmentDurationReady() {
+  mpd_notifier_->NotifySegmentDuration(notification_id_.value());
+}
+
 void MpdNotifyMuxerListener::OnMediaEnd(const MediaRanges& media_ranges,
                                         float duration_seconds) {
   if (mpd_notifier_->dash_profile() == DashProfile::kLive) {

packager/media/event/mpd_notify_muxer_listener.h

@@ -44,7 +44,9 @@ class MpdNotifyMuxerListener : public MuxerListener {
                     const StreamInfo& stream_info,
                     int32_t time_scale,
                     ContainerType container_type) override;
+  void OnAvailabilityOffsetReady() override;
   void OnSampleDurationReady(int32_t sample_duration) override;
+  void OnSegmentDurationReady() override;
   void OnMediaEnd(const MediaRanges& media_ranges,
                   float duration_seconds) override;
   void OnNewSegment(const std::string& file_name,