Add intent for transcoding

[licaon-kter]

Eg, like SendReduced does.

Workflow:

• gallery app
• open video
• share...to transcoder
• process with last used ( Remember "last used settings" #41 ) or some other profile ( Add "Profiles" #40 )
• show share widget again
• choose some app eg, chat app

[anilgulecha]

Another upvote. This came up on the Newpipe issue-tracker as well, for a method to transcode to MP3.

TeamNewPipe/NewPipe#941 (comment)

[anilgulecha]

Also, ideally, this intent should only ask for a confirmation from the user, and proceed with the transcoding in the background -- so users using the feature from other app (like newpipe) do not get overwhelmed by the UX of another app launching.

So an example integration would look like:

1. 3rdparty app (has a setting to enable transcode).
2. On 3rd party app action, call intent in video-transcoder with data (from location, to location, transcode options, notify_at_end).
3. video-transcoder requests confirmation from user. (This step avoids maliciuous use of the intent).
4. on confirmation, begin transcode (in background, with progress in notification), and on complete, do nothing, or if notify_at_end is set, place a notification item about completed transcode.)

[theScrabi]

I agree.
To point 1. The 3. party app has to be able to tell the transcoder what he wants the transcoder to do, like generate an MP3 out of an MP4, or downscale the MP4

For NewPipe it was also very important to be able to merge an audio only and a video only stream. This way we could enable users to save 1080p videos.

[brarcher]

I like the idea. A few changes will need to be made first, namely a notification during encoding (#43). I'll take a look at that next. After that, the question I'll have is how to get raw file paths for ffmpeg. Intents need to pass content provider URIs instead of file URIs, and I've had difficulty in deriving raw paths from such URIs. Is there a standard way to pass a content URI where the raw file system path can always be derived?

[brarcher]

I've got the notification work done and am looking at accepting an intent to start encoding. I've not done this before, so from reading up on it my impression is that to support an arbitrary app sharing a video to encode the SEND action needs to be supported. As mentioned by @licaon-kter the user will then need to enter the encoding parameters (or select a profile, or something else).

One concern with the SEND intent is that the content URI which is passed can be anything, and ffmpeg needs resolved file system paths. It is not clear to me how to reliably get a file path from a content URI. (I tried to do so previously, but ultimately gave it up as the implementation was brittle and not very portable).

If an app wants to request a file be encoded in a specific way, how best should that be handled? Should it be a separate intent (not SEND) which expects additional parameters in a bundle (such as container format, codecs, etc)? Can someone point me to an example elsewhere I can use for reference?

[anilgulecha]

@brarcher Rather than using a send intent, can you just expose a custom intent, which accepts th filepath as a parameter? yes, not every app will be able to use it, but apps specifically for encoding will know what to send.

Android has no requirement that contentURI be backed by a file, so exposing a SEND intent which tries to look for a file would not succeed anyway?

[brarcher]

I've a prototype of this working, see the intent branch. The intent is supports is called protect.videotranscoder.ENCODE, and takes the following arguments:

String inputVideoFilePath: Path to input video file
String outputFilePath: Path to output video/audio file
String mediaContainer: The output media container. Examples: flv, mkv, mp4, gif, mp3, ogg

If the media container supports video, the following are required:
String videoCodec: Video codec to use, must be supported by media container. Example: h264, mpeg4, mpeg2video, gif
Integer videoBitrateK: Bitrate to use for video stream, in 1000s of bits per second. Example: 2000
String resolution: Screen resolution of video stream. Example: 270x460
String fps: Frame per second. Can be an integer or floating number. Examples: 25, 19.01.

If the media container supports audio, the following are required:
String audioCodec: Audio codec to use, must be supported by media container. Example: aac, vorbis, mp3, none (ignores audio).
Integer audioSampleRate: Samples per second. Example: 22050
Integer audioBitrateK: Bitrate used for audio stream, in 1000s of bits per second. Example: 128.
String audioChannel: Number of channels used for audio. Example: 1, 2.

Example of encoding a video:

adb shell am start -a "protect.videotranscoder.ENCODE" --es inputVideoFilePath "/sdcard/ScreenRecord-2018-01-01-19-07-10.mp4" --es outputFilePath /sdcard/output.flv --es mediaContainer flv --es videoCodec h264 --ei videoBitrateK 2000 --es resolution 270x480 --es fps 19 --es audioCodec aac --ei audioSampleRate 22050 --ei audioBitrateK 100 --es audioChannel 2

This will result in a dialog asking for confirmation:

When encoding is complete a dialog is shown giving a summary and an option to send the file elsewhere:

I was going to attach an APK for anyone to experiment with, but there is a limit of 10MB for uploads to issues.

Questions: @theScrabi what parameters are you expecting to send? Are some fields expected to be not passed and sane defaults picked?

[k3b]

... how to get raw file paths for ffmpeg.

If ffmpeg allows to work with inputstream instead of File your app can use inputstream = getContentResolver().openInputStream(uri) This way your app can also open video/audio content:uri-s

Assuming that an input stream is ok you can also replace your intent proposal "inputVideoFilePath"
with inputstream = extras.get(EXTRA_STREAM) and if not found with uri = intent.getData() so your app can be used from every app that implements intent "action_send" or "action send multible".

If you need help with intents let me know. You can also use https://github.com/k3b/intent-intercept/ to discover/experiment/learn with intents.

[brarcher]

FFmpeg is a command-line utility that operates on files. It might be able to encode from stdin, but to do so I would need to rework the middle layer that handles the ffmpeg binary. Realistically, to get the share intent to work the input stream would need to be captured into a separate file, given to ffmpeg to process, then deleted when done. For small files that is just fine. For large files that may be a problem. But then again, encoding a large file on a phone will take a while and will kill the battery, so maybe that is not the goal use case. What are your thoughts, @k3b?

[k3b]

Thanks for background infos.

This might work to pipe stdin/stdout via a java program.

https://github.com/manishcm/Redirection-JNI

---

this one shows how to feed video through a pipe in a console program

https://superuser.com/questions/585798/ffmpeg-slideshow-piping-input-and-output-for-image-stream

`cat *.jpg | ffmpeg -r 1 -f image2pipe -vcodec mjpeg -i - foo.mp4`

--

I havn-t tried it myself. just read it.

[brarcher]

The issue I'll run into is that I'm using a wrapper for FFmpeg called FFmpeg-Android. This is a compiled version of ffmpeg as well as code to determine the architecture and handing the process lifecycle. It makes the video transcoder application much simpler. To send data to ffmpeg's stdin I would need to override the process lifecycle.

Another approach which would likely work is to us something like libAVCodec. There are other encoding Android applications and at least one I researched is doing this. For me, the goal was to create a simple application to fill a need, namely to have an open-source application that can easily transcode a file. Using libAVCodec directly might be the best solution ultimately, as it would allow the application to be smaller and likely faster. However, it would make the application more complex and would require more knowledge on my side, and I'm not ready to bite that off right now.

[k3b]

I agree that would be a lot of extra work for a small benefit.

And the main goal to let new pipe transcode to mp3 is met with the non standard intent api proposed by you.

Thanks for interesting insights

[brarcher]

@anilgulecha @theScrabi the next release of this app will have a prototype of the intent interface. Please give me any feedback you may have on it. I would not add support in NewPipe just yet, as the interface is undocumented and subject to change. After some feedback we can decide what may need to be changed or improved before solidifying the interface.

[theScrabi]

Nnnice I'll notify my project, so we can take care of our part.

[brarcher]

@theScrabi, the v0.8 release has the ENCODE intent available. When looking into it if you find that the interface is tricky to use, let me know what may work better. I've received a request from another app also interested in using Video Transcoder as a companion app. Any feedback on the interface will help me develop a solution which should be useful for NewPipe and others. Thanks!

[theScrabi]

all right thanks :)
I will not develop such a support personally, at least not soon since I got a lot of other stuff to do at NewPipe, but I'm sure someone will catch up with this and start developing support :) I'll put a notification in our thread.

[brarcher]

The app now supports the SEND intent, as well as the custom ENCODE intent. That should solve most of the use cases. If there is additional feedback on either of these which would require a change, let me know.