[Bug]: Uses versioned directory for storage

[dingens]

What happened?

The snap stores Signal's data in the version-specific directory. This causes

• a big waste of storage space, and
• the update to take very long because all data is copied

In my case, this looks like this:

$ du -sh snap/signal-desktop/*/
6.7G	snap/signal-desktop/456/
6.7G	snap/signal-desktop/463/
19M	snap/signal-desktop/common/

What should have happened?

The snap/signal-desktop/common/ directory should be used for data.

Output of snap info $snap_name

name:      signal-desktop
summary:   Private messaging from your desktop.
publisher: Snapcrafters
store-url: https://snapcraft.io/signal-desktop
contact:   https://github.com/snapcrafters/signal-desktop/issues
license:   AGPL-3.0-only
description: |
  Private messaging from your desktop.
  
  To use the Signal desktop app, Signal must first be installed on your phone.
  
  **Are you having issues?**
  
  Let us know by creating a new issue here: https://github.com/snapcrafters/signal-desktop/issues
  
  **Authors**
  
  This snap is maintained by the Snapcrafters community, and is not necessarily endorsed or
  officially maintained by the upstream developers.
commands:
  - signal-desktop
snap-id:      r4LxMVp7zWramXsJQAKdamxy6TAWlaDD
tracking:     latest/stable
refresh-date: yesterday at 19:20 CET
channels:
  latest/stable:    6.9.0 2023-03-09 (463) 193MB -
  latest/candidate: 6.9.0 2023-03-09 (463) 193MB -
  latest/beta:      ↑                            
  latest/edge:      ↑                            
installed:          6.9.0            (463) 193MB -

Output of snap connections $snap_name

Interface               Plug                                   Slot                            Notes
audio-playback          signal-desktop:audio-playback          :audio-playback                 -
audio-record            signal-desktop:audio-record            :audio-record                   -
browser-support         signal-desktop:browser-support         :browser-support                -
camera                  signal-desktop:camera                  :camera                         -
content[gnome-42-2204]  signal-desktop:gnome-42-2204           gnome-42-2204:gnome-42-2204     -
content[gtk-3-themes]   signal-desktop:gtk-3-themes            gtk-common-themes:gtk-3-themes  -
content[icon-themes]    signal-desktop:icon-themes             gtk-common-themes:icon-themes   -
content[sound-themes]   signal-desktop:sound-themes            gtk-common-themes:sound-themes  -
desktop                 signal-desktop:desktop                 :desktop                        -
desktop-legacy          signal-desktop:desktop-legacy          :desktop-legacy                 -
gsettings               signal-desktop:gsettings               :gsettings                      -
home                    signal-desktop:home                    :home                           -
network                 signal-desktop:network                 :network                        -
opengl                  signal-desktop:opengl                  :opengl                         -
removable-media         signal-desktop:removable-media         -                               -
screen-inhibit-control  signal-desktop:screen-inhibit-control  :screen-inhibit-control         -
unity7                  signal-desktop:unity7                  :unity7                         -
wayland                 signal-desktop:wayland                 :wayland                        -
x11                     signal-desktop:x11                     :x11                            -

Output of snap version

snap    2.58.2
snapd   2.58.2
series  16
ubuntu  20.04
kernel  5.4.0-144-generic

Relevant log output

No response

Teminal output of app

No response

[merlijn-sebrechts]

@dingens This is currently by design. We want a user to be able to revert to a previous version of both the snap and the database using snap revert signal-desktop.

The signal database is not always backwards-compatible. Signal sometimes performs a database migration after an update. If there is a bug in this migration, or a bug in a new version of Signal Desktop, users should be able to revert back to a working state of both using snap revert signal-desktop.

This post explains the different storage locations for snap more: https://snapcraft.io/blog/hey-snap-wheres-my-data

You can manually remove the backup using snap remove signal-desktop --revision=456.

[lengau]

The specific issue here seems to come down to Signal's attachments. Everything other than the attachments.noindex directory is tiny and a non-issue, but Signal stores all of your attachments in that directory with no clear way to reap old ones.

I posted a workaround on a Signal-Desktop issue a while back that could help reduce the directory size (and thus update time). If the attachments.noindex directory can be safely moved to common, that would separate the large data component from the much more fragile database files

[dingens]

Thanks @merlijn-sebrechts for the clarifications. That is indeed understandable (even though one might argue that that's what backups are there for but in practice how many normal users do backups... :)

I'd second @lengau 's suggestion: move the attachments out. That would solve the problem for >90% of the storage. The rest of my data folder is just a few 100 megabytes, having only these duplicated would hurt way less.

Another solution that would help, at least for people using cow filesystems (btrfs etc -- probably a majority nowadays?) would be that snap made lightweight copies, but that's out of your control right?

You can manually remove the backup using snap remove signal-desktop --revision=456.

Sure but that's neither a permanent nor a user-friendly solution...

By the way - Does anyone of you know whether there is a way I can tell snap "I don't care, just put everything into common/"?

[lucyllewy]

regarding the backups point, the reason this versioned directory exists is exactly for that purpose. Because snaps automatically update in the background there is no guarantee that a user's own backup regime will be able to catch each step, so snapd does that for you. Three copies are retained (two backups for previous versions, and one for the active copy) so that you can roll back to a previously installed version of the snap and the data associated with that version. By relying on the user doing backups themselves users would not have that ease of roll-back capability.

[lengau]

Out of curiosity I moved attachments.noindex to common on my machine and made current use a symlink to it. I'll report back if anything starts to go weird. I don't think it will since this directory appears to just contain the actual attachment files named using a sha256 hash; however, it does mean that if a signal update breaks my signal config it'll probably nuke those attachments.

Not sure whether that's a safe thing to do, but that's what I'm experimenting with :-)

[lucyllewy]

that's a good idea to try. Thanks for giving it a looksee and a test, @lengau . If it is just the cache of downloaded images and files then it should be safe to move out of the versioned directory into common. We will need to see how it reacts if you rollback the revision to one from an earlier point in time.

[lengau]

@lucyllewy will do! I'm also going to do a couple of random revision hops this weekend and see how it reacts. Will report back.

[lengau]

UPDATE: After prudently doing a snap save signal-desktop, I reverted to older revisions that were still on disk and re-applied that symlink. Then I went through the call for testing history and started refreshing to those, each time checking that Signal still had all my media.

At no point have I been able to break any media stuff from signal, going back to 7.1.1, when the app itself broke for database reasons (I didn't have that revision to revert to). Refreshing the latest candidate still has the media working without need for a restore.

This is a sample size of 1, but I think it's worth pursuing. Maybe we could make a branch that does this and publish it to latest/candidate/common-media (or a similar branch name) for some opt-in testing?