Consider increasing length of voice messages

[nadonomy]

Your use case

What would you like to do?

After shipping voice messages, some committed users are frustrated by the 2 minute limit in place on all platforms. We originally placed a 2 minute limit as:

1. We wanted to ship a simple, predictable implementation for high signal spoken communication.
2. Listening to voice messages is arduous. Long voice messages especially so.

@ara4n in particular has a use case of recording himself playing piano to send to others. Some users in Matrix HQ have expressed they like to ramble, although unclear if the receiving users like to listen. 😀

How would you like to achieve it?

We should consider increasing the limit, with the following considerations:

1. What to? 15mins was suggested, but my sense is this would be too long.
2. Some popular implementations let you toggle playback speed (1x, 1.5x, 2x) which is incredibly useful when listening to long messages.
3. Are we confident in how all of web, iOS & Android handle multi tasking when playing back? Does switching between rooms pause? Does switching back reliably resume your playhead? Is the message easy to discover? Is PiP advantageous on web?
4. On iOS, we integrated playback with the OS media controls (control centre, lock screen) which helps listening to long messages. Do we support the Android equivalent?
5. When sending messages, we display a warning when you near the end duration. Should this be more obvious? At one point in exploration we discovered inverting the countdown timer to start at the max length and count down instead. Should we do this?

Have you considered any alternatives?

No response

Additional context

No response

[jakewb-b]

• What to? 15mins was suggested, but my sense is this would be too long.

Out of interest, too long from what point of view? From a purely user benefit perspective, I don't personally see the value in limiting message length to any particular figure, but I realise there will be technical/file size limitations.

• At one point in exploration we discovered inverting the countdown timer to start at the max length and count down instead. Should we do this?

This inverts the question we're answering to 'how much longer can I record for' rather than 'how long is my current recording'. Both are probably questions users have, but I suspect that which one is more important depends a lot on the answer we come up with to question 1. If the limit is quite tight, then the 'how much longer do I have' question will be more relevant for more users. If the limit is longer than most users would ever reach, then the 'how long is my current recording' question will be far more important, and counting down from some arbitrary high number of minutes could just be confusing.

[jakewb-b]

3. Are we confident in how all of web, iOS & Android handle multi tasking when playing back? Does switching between rooms pause? Does switching back reliably resume your playhead? Is the message easy to discover? Is PiP advantageous on web?

In response to this, I've had to look at behaviour on each platform and confirmed the following:

• On all platforms, switching rooms stops (rather than pauses) playback.
• Switching back does not resume playing, and pressing play re-starts it from the beginning.
• On iOS, backgrounding Element or locking the device, does not pause/stop playback.
• On Android, backgrounding Element or locking the device stops playback.
• On Web, locking the screen does not pause/stop playback.

As to how this ought to work, ideally.

• Switching rooms should pause playback, so it can be resumed at the same point.
• I'm not convinced that switching back should automatically resume playing. This could be undesirable, if I switched away from a room with a half-played message some time ago, and then switch back to that room in a location where having audio play would be unwelcome.
• Backgrounding or locking the device should not stop playback.
• I think PiP would be highly advantageous, even expected behaviour, on web.

On iOS, we integrated playback with the OS media controls (control centre, lock screen) which helps listening to long messages. Do we support the Android equivalent?

I think yes, although I wouldn't rank this as a critical feature. Just anecdotally, I have a friend who sends me extremely long voice messages on WhatsApp, and I tend to listen to them through headphones while walking to the shops. Having playback integrated into lock screen controls makes it easier to pause/play the message as needed (e.g. when I get to the checkout). Note that we'd first have to change Android behaviour so playback doesn't stop on locking the phone.

[jakewb-b]

There is a risk with enabling longer voice messages before we have ensured that the playback experience is sufficiently polished to avoid frustrating user experience with longer files.

These issues must be resolved prior to increasing the max length of VMs:

• Enable scrubbing on received voice messages #18713

These issues should ideally be resolved prior to increasing the max length of VMs:

• Pause voice message playback when switching between rooms, and resume at the same point left off #18714
• Allow voice message playback to continue on locked phone, and integrate with media controls element-android#3885

These issues could be resolved alongside or fast-followers to increasing the max length of VMs:

• PiP playback on web
• Toggle playback speed (1x, 1.5x, 2x)

[vmario89]

i think the length of audio message (edit: there is a general file size limit for attachments; maybe could be set individually for s/mime) should be a server wide settings and maybe a client override setting.

an idea: each user could set a maximum time limit per room. If you have a room with 2 persons and one sets to 5 mins, the other one to 1 mins, the longest would be one minute for both participants.

[KeenMaron]

Any updates? Coming from whatsapp and signal, the most users i know are using VM excessive. And i like it too, especially with speed up function.

[jakewb-b]

@KeenMaron - this is absolutely going to happen, however there are two updates we want to make first: implementing scrubbing across all platforms, and maintaining drafts when you switch rooms. I think that if we allow much longer VMs without those two changes, it could be frustrating to use.
As soon as those are done, we'll be back to looking at increasing the length.
(We also have a task on our backlog to add the speed up function, so I am sure that will happen too!)

[KeenMaron]

That sounds awesome :), thank you for the reply!

[ShadowJonathan]

implementing scrubbing across all platforms

@jakewb-b I think you might find matrix-org/synapse#4780 interesting, it makes some/most media handlers automatically able to scrub through data without having to download it all first, this also applies to some other media formats, and can greatly help the user experience.

[ivy-reps]

1. From my perspective if it does controlled by:

const TARGET_MAX_LENGTH = 120; // 2 minutes in seconds. Somewhat arbitrary, though longer == larger files.

which are part of /src/audio/VoiceRecording.ts, then please just change it to 900 seconds which are 15 minutes.

I'm fully agreed with @nadonomy, which have written that offer just a year ago. If it's one line change of feature which already works like for a year???, then it should be done like a year ago. Just number change...

2. Also I'm offering to make TARGET_WARN_TIME_LEFT which by default equals to 10 seconds, which are 8.333333333333332% from 120 seconds, to make dependent from TARGET_MAX_LENGTH and calculate it by percentage.

const TARGET_WARN_TIME_LEFT = TARGET_MAX_LENGTH * 8.3333333333333332 / 100

Actually it will be better to inform person about end coming may be few times before duration will reach the end.

With Respect, it will be great if you guys will focus on the subject of that issue.
Of course, it's your decision what to be focused on.

Of course it's good to have scrubbing (@ShadowJonathan) and rightly working background play functionality, and remembering of place of pause, but these all things are not about the subject of that issue.

I think that if we allow much longer VMs without those two changes, it could be frustrating to use.

@jakewb-b, you are watching on your messenger, and listening messages that have recorded by your coroomer. No need to go to background when you are working for example, you have to get the clue of what it's all about. So it's not 100% true that you will go to other windows of your phone OS. When it's 15 messages by 2 minutes instead of 2 messages by 15 minutes it's really frustrating to use. Chat increasing it's height. My coroomer interrupting on a half of the word, because when he is going on the street, he aren't able to watch the screen all the time, he needs to watch what is happening around, and he wanted to save time by answering or asking me while he moving to some place. Just yesterday he was confused when message recording had stopped, and really it is drags you from thought that you are trying to say.

i think the length of audio message (edit: there is a general file size limit for attachments; maybe could be set individually for s/mime) should be a server wide settings and maybe a client override setting.

@vmario89, totally agreed:
server attachment max size -> server voice message max duration -> room voice message max duration value -> client voice message max duration value (can be limited by desire of user, for example by the request of cooroms or cooroom, it will be like notification for the person to be short but with ability to increase the size in case when longer length should be used)

an idea: each user could set a maximum time limit per room. If you have a room with 2 persons and one sets to 5 mins, the other one to 1 mins, the longest would be one minute for both participants.

@vmario89, that can be implemented as additional feature, but may be it should be requested separately? But with relation to voice messages part of Matrix React JS SDK?

Anyway guys, I really appreciate that someone just made that all. That's incredible to have it all in common. Protocol, client. Feels like peoples doing something together and it is working! Incredible!

[turt2live]

it may be a 1 line change, but it's still important that the user experience be suited for longer messages. Code complexity is not the same as feature complexity.

[ivy-reps]

it may be a 1 line change, but it's still important that the user experience be suited for longer messages. Code complexity is not the same as feature complexity.

@turt2live, the user have to decide what limit he will prefer, then the user is the one who is deciding. at least if we are talking about home servers definitely.

Within your reasoning you are:

• limiting the user because you are thinking that it is better for user
• limiting the user to make the decision
• forcing the user to make many inidentifiable voice messages
• forcing the user to create a huge number of message blocks, making it very difficult for the user to read/scroll/quote the history
• forcing the user to be interrupted/to interrupt record every two minutes
• as I can see while room opens, the history rendering happening much slower, because of the records quantity that should be transferred from all room clients and theirs duplicates (desktop/laptop/phone) and then they should be processed (probably deciphered (?)). It will be slower because of messages/records/blocks processing anyway. But in one hand it's because the the redundant split due to the two minute limit, on the other hand, it's because it's just the latest text/voice messages, not a bunch of two minute ones.

All these issues, all are voting/yelling for changing that line to much bigger value. Moreover, I can't imagine how these can be ignored for a year...

Again, if you prefer so, of course I will respect that. I'm actually nobody to say something about it and I understand that. Also, if you are contributor/author of that/these product(s), you probably understand it much better, and actually you know better why. May be you have other reasons which can't be listed here on public.

From my perspective, anyway, if you are really thinking about the end user experience, then the user have to decide, at least if we are talking about homeservers, not about the matrix.org or some other official servers.

Again, I'm really grateful for the whole product. Can't imagine how hard it is to make it all and how much time needed to make anything even closer to that and to have all these products now. I'm really respectful for what the whole world already have because of your work, the team, and other contributors.

Thank you for responding so soon!

[Karamelmar]

I would find it a good solution if element web/android/... would get the max length limit of a voice message from the matrix server itself, rather then hard coded limits in different element clients... f.e. a variable in the synapse homeserver.yaml.

The users of our matrix synapse instance have adopted to the 2 minutes default limit, which, as mentioned earlier, results in rather huge stacks of 2min messages...