15.0.7 large files are not synchronizing from clients or uploading from web. #15095

gessel · 2019-04-14T18:27:57Z

Steps to reproduce

Upload file larger than 1MB via web, or synchronize via desktop client, or Android client

Note, I've followed the advice from similar issues to no avail including:

https://help.nextcloud.com/t/uploading-large-files-fails-even-with-desktop-sync-client/11035
(1.4 MB isn't large, but checked all php.ini and other configs (note this was working fine under a series of previous versions, but stopped working with 15.0.5 & 15.0.07).
https://help.nextcloud.com/t/sabre-dav-exception-badrequest-expected-filesize-10000000-got-2490368/29761/4
(file size problem isn't close to 512MB)
https://central.owncloud.org/t/expected-filesize-xxx-got-yyy-0/816
(I haven't tried all the possible issues listed here as no changes were made to configuration from a working version before it started failing, but I checked them and it /seems/ ok)

Expected behaviour

File should be copied/synchronized to the server

Actual behaviour

Errors are thrown, file is not synchronized

Server configuration

Operating system: FreeBSD 11.2-RELEASE-p6 #0 r341740:

Web server: apache24-2.4.39

Database: mysql56-server-5.6.43

PHP version: php72-7.2.17_2

Nextcloud version: nextcloud-php72-15.0.7

Updated from an older Nextcloud/ownCloud or fresh install: updated from 15.0.2, 15.0.5 and 15.0.7 both gave the problem

Where did you install Nextcloud from: FreeBSD ports collection

Signing status:

Signing status

Login as admin user into your Nextcloud and access 
http://example.com/index.php/settings/integrity/failed 
paste the results here.

Technical information
=====================
The following list covers which files have failed the integrity check. Please read
the previous linked documentation to learn more about the errors and how to fix
them.

Results
=======
- passman
	- EXCEPTION
		- OC\IntegrityCheck\Exceptions\InvalidSignatureException
		- Certificate is not valid.

Raw output
==========
Array
(
    [passman] => Array
        (
            [EXCEPTION] => Array
                (
                    [class] => OC\IntegrityCheck\Exceptions\InvalidSignatureException
                    [message] => Certificate is not valid.
                )

        )

)

List of activated apps:

App list


 # cd /usr/local/www/nextcloud
root@matanawa:/usr/local/www/nextcloud # su -m www -c "php occ app:list"
Enabled:
  - accessibility: 1.1.0
  - activity: 2.8.2
  - admin_audit: 1.5.0
  - bruteforcesettings: 1.3.0
  - calendar: 1.6.4
  - cloud_federation_api: 0.1.0
  - comments: 1.5.0
  - contacts: 3.1.0
  - dav: 1.8.1
  - federatedfilesharing: 1.5.0
  - federation: 1.5.0
  - files: 1.10.0
  - files_pdfviewer: 1.4.0
  - files_sharing: 1.7.0
  - files_texteditor: 2.7.0
  - files_trashbin: 1.5.0
  - files_versions: 1.8.0
  - files_videoplayer: 1.4.0
  - firstrunwizard: 2.4.0
  - gallery: 18.2.0
  - logreader: 2.0.0
  - lookup_server_connector: 1.3.0
  - metadata: 0.9.0
  - music: 0.9.4
  - nextcloud_announcements: 1.4.0
  - notes: 2.5.1
  - notifications: 2.3.0
  - oauth2: 1.3.0
  - passman: 2.2.1
  - password_policy: 1.5.0
  - previewgenerator: 2.1.0
  - provisioning_api: 1.5.0
  - serverinfo: 1.5.0
  - sharebymail: 1.5.0
  - support: 1.0.0
  - survey_client: 1.3.0
  - systemtags: 1.5.0
  - tasks: 0.9.8
  - theming: 1.6.0
  - theming_customcss: 1.3.0
  - twofactor_backupcodes: 1.4.1
  - updatenotification: 1.5.0
  - user_usage_report: 1.1.1
  - workflowengine: 1.5.0
Disabled:
  - checksum
  - dashboard
  - encryption
  - files_accesscontrol
  - files_external
  - files_fulltextsearch
  - files_rightclick
  - fulltextsearch
  - fulltextsearch_elasticsearch
  - groupfolders
  - quota_warning
  - ransomware_protection
  - socialsharing_email
  - spreed
  - user_ldap

Nextcloud configuration:

Config report


# su -m www -c "php occ config:list system"
{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "nas.blackrosetech.com",
            "173.228.36.144",
            "inas.blackrosetech.com"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "https:\/\/nas.blackrosetech.com\/nextcloud",
        "dbtype": "mysql",
        "version": "15.0.7.0",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "memcache.local": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0,
            "dbindex": 0,
            "password": "***REMOVED SENSITIVE VALUE***",
            "timeout": 1.5
        },
        "mail_smtpmode": "smtp",
        "mail_smtpauthtype": "LOGIN",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25",
        "maintenance": false,
        "apps_paths": [
            {
                "path": "\/usr\/local\/www\/nextcloud\/apps",
                "url": "\/apps",
                "writable": true
            },
            {
                "path": "\/usr\/local\/www\/nextcloud\/apps-pkg",
                "url": "\/apps-pkg",
                "writable": false
            }
        ],
        "logfile": "\/var\/log\/nextcloud.log",
        "loglevel": 0,
        "theme": "",
        "session_lifetime": 6220800
    }
}

Are you using external storage, if yes which one: nfs mount point (running in jail, mountpoint set by host, read/write verified, free space verified.

Are you using encryption: Geli (disk level) not application level

Are you using an external user-backend, if yes which one: No

Client configuration

Browser: Chrome Version 73.0.3683.86 (Official Build) Built on Ubuntu , running on LinuxMint 19.1 (64-bit)

Operating system: LinuxMint 19.1 (64-bit)

Logs

Web server error log

[Sun Apr 14 11:00:24.397262 2019] [ssl:info] [pid 18700:tid 35167396608] [client 95.170.219.130:64436] AH01964: Connection to child 151 established (server nas.blackrosetech.com:443)
[Sun Apr 14 11:00:25.694313 2019] [reqtimeout:info] [pid 18697:tid 34898059776] [client 109.127.77.144:15784] AH01382: Request body read timeout
[Sun Apr 14 11:00:26.819125 2019] [ssl:info] [pid 18697:tid 35167392768] [client 109.127.77.144:14456] AH01964: Connection to child 20 established (server nas.blackrosetech.com:443)

Nextcloud log (data/nextcloud.log)

Nextcloud log


HPACK decompression failed

It appears the non-HPACK files are smaller, the server log error for large files (9.1MB) is

Error	no app in context	Sabre\DAV\Exception\BadRequest: expected filesize 10000000 got 1302528
/usr/local/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php - line 156:
OCA\DAV\Connector\Sabre\File->put(null)
/usr/local/www/nextcloud/apps/dav/lib/Upload/UploadFolder.php - line 39:
OCA\DAV\Connector\Sabre\Directory->createFile("00000000", null)
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 1096:
OCA\DAV\Upload\UploadFolder->createFile("00000000", null)
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 525:
Sabre\DAV\Server->createFile("uploads/ges ... 0", null, null)
<<closure>>
Sabre\DAV\CorePlugin->httpPut(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})
/usr/local/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php - line 105:
call_user_func_array([ Sabre\DAV\ ... "], [ Sabre\HTTP ... }])
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 479:
Sabre\Event\EventEmitter->emit("method:PUT", [ Sabre\HTTP ... }])
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 254:
Sabre\DAV\Server->invokeMethod(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})
/usr/local/www/nextcloud/apps/dav/lib/Server.php - line 301:
Sabre\DAV\Server->exec()
/usr/local/www/nextcloud/apps/dav/appinfo/v2/remote.php - line 35:
OCA\DAV\Server->exec()
/usr/local/www/nextcloud/remote.php - line 163:
require_once("/usr/local/ ... p")


while smaller files might trigger an error like

Error	no app in context	Sabre\DAV\Exception\BadRequest: expected filesize 9054450 got 713728
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 1130:
OCA\DAV\Connector\Sabre\File->put("*** sensiti ... *")
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 513:
Sabre\DAV\Server->updateFile("*** sensiti ... *")
<<closure>>
Sabre\DAV\CorePlugin->httpPut(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})
/usr/local/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php - line 105:
call_user_func_array([ Sabre\DAV\ ... "], [ Sabre\HTTP ... }])
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 479:
Sabre\Event\EventEmitter->emit("method:PUT", [ Sabre\HTTP ... }])
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 254:
Sabre\DAV\Server->invokeMethod(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})
/usr/local/www/nextcloud/apps/dav/lib/Server.php - line 301:
Sabre\DAV\Server->exec()
/usr/local/www/nextcloud/apps/dav/appinfo/v2/remote.php - line 35:
OCA\DAV\Server->exec()
/usr/local/www/nextcloud/remote.php - line 163:
require_once("/usr/local/ ... p")


App error

Error	no app in context	Sabre\DAV\Exception\BadRequest: expected filesize 4212384 got 81920
/usr/local/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php - line 156:
OCA\DAV\Connector\Sabre\File->put(null)
/usr/local/www/nextcloud/apps/dav/lib/Upload/UploadFolder.php - line 39:
OCA\DAV\Connector\Sabre\Directory->createFile("00000000000 ... 4", null)
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 1096:
OCA\DAV\Upload\UploadFolder->createFile("00000000000 ... 4", null)
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 525:
Sabre\DAV\Server->createFile("uploads/ges ... 4", null, null)
<<closure>>
Sabre\DAV\CorePlugin->httpPut(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})
/usr/local/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php - line 105:
call_user_func_array([ Sabre\DAV\ ... "], [ Sabre\HTTP ... }])
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 479:
Sabre\Event\EventEmitter->emit("method:PUT", [ Sabre\HTTP ... }])
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 254:
Sabre\DAV\Server->invokeMethod(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})
/usr/local/www/nextcloud/apps/dav/lib/Server.php - line 301:
Sabre\DAV\Server->exec()
/usr/local/www/nextcloud/apps/dav/appinfo/v2/remote.php - line 35:
OCA\DAV\Server->exec()
/usr/local/www/nextcloud/remote.php - line 163:
require_once("/usr/local/ ... p")

after restarting the server and the phone and setting log level to debug

Debug	no app in context	No cache entry found for /gessel/uploads/922f79255724990a1b93889c87acdab5/0000000000000000-0000000004212384 (storage: home::gessel, internalPath: uploads/922f79255724990a1b93889c87acdab5/0000000000000000-0000000004212384) (repeated)

Debug	webdav	Sabre\DAV\Exception\MethodNotAllowed: The resource you tried to create already exists
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 590:
Sabre\DAV\Server->createCollection("uploads/ges ... 5", Sabre\DAV\MkCol {})
<<closure>>
Sabre\DAV\CorePlugin->httpMkcol(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})
/usr/local/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php - line 105:
call_user_func_array([ Sabre\DAV\ ... "], [ Sabre\HTTP ... }])
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 479:
Sabre\Event\EventEmitter->emit("method:MKCOL", [ Sabre\HTTP ... }])
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 254:
Sabre\DAV\Server->invokeMethod(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})
/usr/local/www/nextcloud/apps/dav/lib/Server.php - line 301:
Sabre\DAV\Server->exec()
/usr/local/www/nextcloud/apps/dav/appinfo/v2/remote.php - line 35:
OCA\DAV\Server->exec()
/usr/local/www/nextcloud/remote.php - line 163:
require_once("/usr/local/ ... p")


Debug	webdav	Sabre\DAV\Exception\NotFound: File with name PMC/Gulan Issues/Cemetery Gate Construction/IMG_20190414_111821.jpg could not be located
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 81:
OCA\DAV\Connector\Sabre\ObjectTree->getNodeForPath("PMC/Gulan I ... g")
<<closure>>
Sabre\DAV\CorePlugin->httpGet(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})
/usr/local/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php - line 105:
call_user_func_array([ Sabre\DAV\ ... "], [ Sabre\HTTP ... }])
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 479:
Sabre\Event\EventEmitter->emit("method:GET", [ Sabre\HTTP ... }])
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 253:
Sabre\DAV\Server->invokeMethod(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {}, false)
<<closure>>
Sabre\DAV\CorePlugin->httpHead(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})
/usr/local/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php - line 105:
call_user_func_array([ Sabre\DAV\ ... "], [ Sabre\HTTP ... }])
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 479:
Sabre\Event\EventEmitter->emit("method:HEAD", [ Sabre\HTTP ... }])
/usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 254:
Sabre\DAV\Server->invokeMethod(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})
/usr/local/www/nextcloud/apps/dav/appinfo/v1/webdav.php - line 80:
Sabre\DAV\Server->exec()
/usr/local/www/nextcloud/remote.php - line 163:
require_once("/usr/local/ ... p")

Browser log

core.js?v=1d49041f-6:4 PUT https://nas.blackrosetech.com/remote.php/webdav/barzan/Barzan%20Documentation/Axis%203003-e%20loudspeaker%20installation%20guide.pdf 400
send @ core.js?v=1d49041f-6:4
ajax @ core.js?v=1d49041f-6:4
send @ merged-index.js?v=1d49041f-6:3048
(anonymous) @ core.js?v=1d49041f-6:2
j @ core.js?v=1d49041f-6:2
add @ core.js?v=1d49041f-6:2
(anonymous) @ core.js?v=1d49041f-6:2
each @ core.js?v=1d49041f-6:2
(anonymous) @ core.js?v=1d49041f-6:2
a.Deferred @ core.js?v=1d49041f-6:7
then @ core.js?v=1d49041f-6:2
_onSend @ merged-index.js?v=1d49041f-6:3092
(anonymous) @ core.js?v=1d49041f-6:13
data.submit @ merged-index.js?v=1d49041f-6:2816
(anonymous) @ merged-index.js?v=1d49041f-6:849
(anonymous) @ core.js?v=1d49041f-6:2
j @ core.js?v=1d49041f-6:2
add @ core.js?v=1d49041f-6:2
(anonymous) @ core.js?v=1d49041f-6:2
each @ core.js?v=1d49041f-6:2
(anonymous) @ core.js?v=1d49041f-6:2
a.Deferred @ core.js?v=1d49041f-6:7
then @ core.js?v=1d49041f-6:2
submit @ merged-index.js?v=1d49041f-6:848
(anonymous) @ merged-index.js?v=1d49041f-6:1154
m.each.m.forEach @ core.js?v=1d49041f-6:18
submitUploads @ merged-index.js?v=1d49041f-6:1152
onNoConflicts @ merged-index.js?v=1d49041f-6:1578
checkExistingFiles @ merged-index.js?v=1d49041f-6:1361
add @ merged-index.js?v=1d49041f-6:1596
_trigger @ core.js?v=1d49041f-6:13
(anonymous) @ merged-index.js?v=1d49041f-6:3179
each @ core.js?v=1d49041f-6:2
_onAdd @ merged-index.js?v=1d49041f-6:3172
(anonymous) @ core.js?v=1d49041f-6:13
(anonymous) @ merged-index.js?v=1d49041f-6:3377
j @ core.js?v=1d49041f-6:2
add @ core.js?v=1d49041f-6:2
always @ core.js?v=1d49041f-6:2
_onChange @ merged-index.js?v=1d49041f-6:3367
(anonymous) @ core.js?v=1d49041f-6:13
u @ core.js?v=1d49041f-6:13
dispatch @ core.js?v=1d49041f-6:3
r.handle @ core.js?v=1d49041f-6:3

The text was updated successfully, but these errors were encountered:

kesselb · 2019-04-14T18:39:05Z

AH01382: Request body read timeout

If the connection interrupted upload may not work. Could you try to disable mod reqtimeout?

gessel · 2019-04-14T18:51:30Z

@kesselb dang... good call. disbled, restarted, upload works.

This is my configuration: extra/httpd-default.conf: RequestReadTimeout header=20-40,MinRate=500 body=20,MinRate=500
Any advice on a more compatible configuration or is there a bug in Apache?

kesselb · 2019-04-14T19:00:54Z

owncloud/core#14603
http://sabre.io/dav/webservers/

As reference. I think the limits are to strict. Most people just disable it.

gessel · 2019-04-14T20:57:21Z

The header and body settings are sensible as there are apparently some slow SSL attacks that this is meant to mitigate. There's an extra quirk, I gather, with FreeBSD which uses AcceptFilter: "When an AcceptFilter is in use (usually the case on Linux and FreeBSD), the socket is not sent to the server process before at least one byte (or the whole request for httpready) is received. The handshake and header timeouts configured with RequestReadTimeout are only effective after the server process has received the socket."

So maybe no timeout is best. It does work, thanks!

gessel added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Apr 14, 2019

gessel closed this as completed Apr 14, 2019

This was referenced Apr 16, 2019

Unable to upload files from Android (Error: Sabre\DAV\Exception\BadRequest: expected filesize XXX got 0) #14884

Closed

6GB upload ends with corrupted 4294967295 Bytes file #6338

Closed

Uploading big files > 2gb with php-fpm is buged #7799

Closed

kesselb mentioned this issue May 9, 2019

broken upload on shared folder by URL #15479

Closed

kesselb mentioned this issue Jun 27, 2019

"Expected filesize" errors with HTTP2 enabled #16138

Closed

kesselb mentioned this issue Jul 13, 2019

Sabre-DAV-Exception: expected filesize 10240000 got 7217152 nextcloud/android#4005

Closed

kesselb mentioned this issue Jul 21, 2019

NC16.0.3 Upload fails with files 2GiB in 32bit instances (again) #16431

Closed

jame-mni mentioned this issue Feb 5, 2020

Sabre\DAV\Exception\BadRequest: expected filesize X got Y error nextcloud-snap/nextcloud-snap#1203

Closed

loelkes mentioned this issue Apr 27, 2020

Browserupload fails with net::ERR_NETWORK_CHANGED #20699

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

15.0.7 large files are not synchronizing from clients or uploading from web. #15095

15.0.7 large files are not synchronizing from clients or uploading from web. #15095

gessel commented Apr 14, 2019 •

edited

Loading

kesselb commented Apr 14, 2019

gessel commented Apr 14, 2019

kesselb commented Apr 14, 2019

gessel commented Apr 14, 2019

15.0.7 large files are not synchronizing from clients or uploading from web. #15095

15.0.7 large files are not synchronizing from clients or uploading from web. #15095

Comments

gessel commented Apr 14, 2019 • edited Loading

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

Client configuration

Logs

Web server error log

Nextcloud log (data/nextcloud.log)

Browser log

kesselb commented Apr 14, 2019

gessel commented Apr 14, 2019

kesselb commented Apr 14, 2019

gessel commented Apr 14, 2019

gessel commented Apr 14, 2019 •

edited

Loading