Having 5132 deleted files results in a unresponsive webbrowser #2860

a-schild · 2013-04-10T20:37:21Z

Server ownCloud 5.0.3 (and 5.0.4RC1) on Debian 6.0.7 64bit with apache+mysql

Due to some clean-up of files I now have 5132 files in the "deleted files" section of owncloud.
When I browse to the recycle bin, it trys to show all 5132 files in a long list,
which actually freezes firefox 20.0 on Ubuntu 12.10.

There are also about 100-200 deleted folders (if it matters)

Not sure how owncloud is behaving when a not-deleted folder holds several thousand files.

BernhardPosselt · 2013-04-10T21:16:36Z

if youve enabled the trashbin app, those files are copied to the trashbin. Thats why its taking so long. As for the rendering this is a general issue of the files app because it doesnt use autopaging cc @schiesbn @icewind1991

jancborchardt · 2013-08-07T18:28:58Z

I’m closing this issue because it has been inactive for a few months. This probably means that the issue is not reproducible or it has been fixed in a newer version.

Please reopen if you still encounter this issue with the latest stable version (currently ownCloud 5.0.9) and then please use the issue template. You an also contribute directly by providing a patch – see the developer manual. :)

Thank you!

a-schild · 2013-08-07T18:40:59Z

No,

this issue is not resolved in the latest release.
The basic problem is that when you have meny thousand entries in the trash, the server tells the webbrowser to show all filenames, which then takes long and longer.

Is the same a issue #3361

jancborchardt · 2013-08-07T18:53:01Z

Ok, @a-schild thank you for retesting and letting us know.

Now can we get the coding experts on this? @bantu @ringmaster @VicDeo please help.

fekepp · 2013-08-08T15:33:17Z

Same issue here, currently 28247 files with 38GB in my user's trashbin and so far not able to clean them manually. So far OK for me in terms of disk space, but sometimes I like to clean-up :) OC takes currently 500GB on my server's disk.

ls -l files_trashbin/files/ | wc -l
28247

du -hs files_trashbin/
38G files_trashbin/

Not Loading all items is of course required to not crash the browser. I don't know if its possible to keep the old directory structure? But still ppl could have thousands of files in a directory. Also a queued maintenance task would be an option for the deletion of large file collections that asynchronous deletes the files when the user selects files for deletion (if not already the case).

schiessle · 2013-08-08T15:54:57Z

If you delete the files with the web interface the folder structure is preserved. But with the sync client we get one delete request for every single file which results in such large file lists in the first place. See: owncloud/client#612

fekepp · 2013-08-12T13:29:01Z

As a workaround you may temporary (or if you want permanent) set
'trashbin_retention_obligation' => 1,
in your config.php, documented here:
http://doc.owncloud.org/server/5.0/user_manual/files/deletedfiles.html

This will trigger the files_trashbin app to delete files in your trashbin after one day which helped me to cleanup (I didn't test the behaviour if you set it to zero). Default is 180 days/50% of available free disk space.

I am not sure if it is triggered by the regular cron jobs (which it should because many ppl don't use the frontend to handle files, like me) or just when you delete an item in the web frontend. It blocks the user interface while deleting the files which took a long time (~1h/20k files) in this case. Access with another browser at the same time was possible.

It also triggers a lot of
{"app":"PHP","message":"filesize(): stat failed for PATH_TO_FILE_IN_TRASHBIN at /var/www/owncloud/lib/files/storage/local.php#74","level":2,"time":1376314904}
errors logged in at log level 2. maybe worth another bug.

schiessle · 2013-08-12T13:49:22Z

@fekepp The trash bin expire old files every time a file gets deleted and every time we overstep the max size if a new file gets added to your ownCloud. This is completely independent from the interface you use (web, mobile client, desktop client or plain webdav)

fekepp · 2013-08-12T13:54:32Z

Sounds good. My case was also a bit artificial because i triggered the permanent deletion of 20k+ files at the same time with the new configuration.

a-schild · 2013-08-13T06:52:00Z

Setting 'trashbin_retention_obligation' => 30, solved the problem for me. Or better it is a work arround.
The trash is not cleaned up by the cron job, but is triggered when a file is deleted.
I will open a issue to move that trigger out in the cron job.

The basic problem with the UI freezing when too many files/folders are to be displayed still exists.

schiessle · 2013-08-13T08:12:51Z

@a-schild Do you deleted the files with the sync client? Than the main problem needs to be solved here: owncloud/client#612

I'm also a bit confused about all the .part files in your trash. I will check this, .part files shouldn't be copied to the trash at all.

a-schild · 2013-08-13T08:20:26Z

Yes, most deletes have been done with the desktopclient, either from windows or ubuntu.

DeepDiver1975 · 2013-10-02T08:27:32Z

We need endless scrolling in the files app and then we can have this in the trash bin as well

schiessle · 2013-10-02T08:33:35Z

BTW, the problem that the sync client sends a delete request for every single file and not for the complete folder should be fixed in the next version (1.5): owncloud/client#612

Gitju · 2013-10-25T14:52:21Z

The problem still exists in owncloud.

PVince81 · 2013-10-30T19:53:45Z

How about adding an "empty trash" action as a workaround ?

I'll look into infinite scrolling for OC7.

Gitju · 2013-10-30T19:55:37Z

Empty trash would definitely be a nice workaround.

PVince81 · 2013-10-30T20:01:12Z

The catch is that if we put that button in the trash files view, the UI will still need quite some time to just render the list before you can click it...

PVince81 · 2013-10-30T20:35:04Z

@schiesbn I remember you mentioned something about the trashbin auto-deleting files when disk space is low ?

For the "clean trash" another idea would be to put the button on the personal page (instead of or additionally to the trashbin app). Why the personal page ? Because that's where you can see how much of your quota you are using, so it might make sense to put a "Cleanup" button there.

Thoughts ? @owncloud/designers

When people have too many files, it is currently difficult to select them all since the browser is unresponsive when updating so many elements. The empty trash button will send a simple POST request to delete ALL files in the trashbin. Fixes #2860

PVince81 · 2013-10-30T21:42:32Z

Here we go, the mighty "Delete all that stinky trash" button.
I had a test with 12000 files. It took some time for the browser to render them in the trash app, but eventually I was able to click that new button and delete them. (no need to select all)

PR will follow soon.

Gitju · 2013-10-30T23:01:48Z

Nice

neebski · 2013-10-31T11:44:52Z

I vote for having a clear trash button in the personal settings near the
quota usage. Maybe even have in the quota bar segments for "free space"
"deleted files" and "file versions" shown so users understand the
additional space they have used. Similar how Android or iOS display the
different types of files on your device.
On Oct 30, 2013 7:02 PM, "Gitju" [email protected] wrote:

Nice

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2860#issuecomment-27447503
.

jancborchardt · 2013-11-02T13:02:29Z

No. The resolution to this is not introducing more buttons and features,
but rather making it work under the hood.
On Oct 31, 2013 12:45 PM, "neebski" [email protected] wrote:

I vote for having a clear trash button in the personal settings near the
quota usage. Maybe even have in the quota bar segments for "free space"
"deleted files" and "file versions" shown so users understand the
additional space they have used. Similar how Android or iOS display the
different types of files on your device.
On Oct 30, 2013 7:02 PM, "Gitju" [email protected] wrote:

Nice

—
Reply to this email directly or view it on GitHub<
https://github.com/owncloud/core/issues/2860#issuecomment-27447503>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2860#issuecomment-27479351
.

karlitschek · 2014-02-18T15:56:59Z

@PVince81 Didn't we improve this lately?

PVince81 · 2014-02-18T16:00:25Z

A bit. If you delete all files it will be a bit faster.
There were three performance issues

When opening the page and loading the file list
When clicking "select all", it would automatically tick all 5000 files
When clicking "Delete", sending the whole file list to the server for deletion

The only thing that was improved on 6.0.1 is 3), when deleting it only sends a flag "allfiles=true".

and 2) should work better with infinite scrolling Infinite scrolling for files app #7167 which is work in progress.

I intended to close this issue once that PR is merged.

PVince81 · 2014-03-06T14:30:40Z

Closing this in favor of the corresponding task ticket, please follow up there: #7597

jancborchardt closed this as completed Aug 7, 2013

a-schild mentioned this issue Aug 7, 2013

loading trash takes minutes #3031

Closed

jancborchardt reopened this Aug 7, 2013

a-schild mentioned this issue Aug 7, 2013

Too much files and folders in trash bin #3361

Closed

schiessle closed this as completed Aug 8, 2013

schiessle reopened this Aug 8, 2013

VicDeo mentioned this issue Oct 30, 2013

Deleted Files - can't remove a huge amout of deleted Files #5573

Closed

PVince81 mentioned this issue Oct 30, 2013

Deleting all files in trash now only sends a single flag #5641

Merged

PVince81 mentioned this issue Jan 27, 2014

Can't show or delete the files in the trash bin #6948

Closed

PVince81 closed this as completed Mar 6, 2014

lock bot locked as resolved and limited conversation to collaborators Aug 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Having 5132 deleted files results in a unresponsive webbrowser #2860

Having 5132 deleted files results in a unresponsive webbrowser #2860

a-schild commented Apr 10, 2013

BernhardPosselt commented Apr 10, 2013

jancborchardt commented Aug 7, 2013

a-schild commented Aug 7, 2013

jancborchardt commented Aug 7, 2013

fekepp commented Aug 8, 2013

schiessle commented Aug 8, 2013

fekepp commented Aug 12, 2013

schiessle commented Aug 12, 2013

fekepp commented Aug 12, 2013

a-schild commented Aug 13, 2013

schiessle commented Aug 13, 2013

a-schild commented Aug 13, 2013

DeepDiver1975 commented Oct 2, 2013

schiessle commented Oct 2, 2013

Gitju commented Oct 25, 2013

PVince81 commented Oct 30, 2013

Gitju commented Oct 30, 2013

PVince81 commented Oct 30, 2013

PVince81 commented Oct 30, 2013

PVince81 commented Oct 30, 2013

Gitju commented Oct 30, 2013

neebski commented Oct 31, 2013

jancborchardt commented Nov 2, 2013

karlitschek commented Feb 18, 2014

PVince81 commented Feb 18, 2014

PVince81 commented Mar 6, 2014

Having 5132 deleted files results in a unresponsive webbrowser #2860

Having 5132 deleted files results in a unresponsive webbrowser #2860

Comments

a-schild commented Apr 10, 2013

BernhardPosselt commented Apr 10, 2013

jancborchardt commented Aug 7, 2013

a-schild commented Aug 7, 2013

jancborchardt commented Aug 7, 2013

fekepp commented Aug 8, 2013

schiessle commented Aug 8, 2013

fekepp commented Aug 12, 2013

schiessle commented Aug 12, 2013

fekepp commented Aug 12, 2013

a-schild commented Aug 13, 2013

schiessle commented Aug 13, 2013

a-schild commented Aug 13, 2013

DeepDiver1975 commented Oct 2, 2013

schiessle commented Oct 2, 2013

Gitju commented Oct 25, 2013

PVince81 commented Oct 30, 2013

Gitju commented Oct 30, 2013

PVince81 commented Oct 30, 2013

PVince81 commented Oct 30, 2013

PVince81 commented Oct 30, 2013

Gitju commented Oct 30, 2013

neebski commented Oct 31, 2013

jancborchardt commented Nov 2, 2013

karlitschek commented Feb 18, 2014

PVince81 commented Feb 18, 2014

PVince81 commented Mar 6, 2014