Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve uploading file in chunks #33692

Merged
merged 1 commit into from
Nov 29, 2018
Merged

improve uploading file in chunks #33692

merged 1 commit into from
Nov 29, 2018

Conversation

individual-it
Copy link
Member

@individual-it individual-it commented Nov 29, 2018

Description

improve the step that does split a file and uploads it in chunks

Related Issue

Motivation and Context

be able to do https://github.com/owncloud/files_classifier/issues/114

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Technical debt
  • Tests

Checklist:

  • Code changes
  • Unit tests added
  • Acceptance tests added
  • Documentation ticket raised:

Open tasks:

  • Backport (if applicable set "backport-request" label and remove when the backport was done)

$size = \filesize(
$this->acceptanceTestsDirLocation() . $source
PHPUnit_Framework_Assert::assertGreaterThan(
0, $noOfChunks, "What does it mean to have $noOfChunks chunks?"
Copy link
Contributor

@phil-davis phil-davis Nov 29, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

future data compression algorithm that can fit the whole file content into 0 chunks ;)
research project?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its all there https://github.com/philipl/pifs

πfs is a revolutionary new file system that, instead of wasting space storing your data on your hard drive, stores your data in π! You'll never run out of space again - π holds every file that could possibly exist! They said 100% compression was impossible? You're looking at it!

Copy link
Contributor

@phil-davis phil-davis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@phil-davis
Copy link
Contributor

It broke:

  Scenario: Upload file to received federated share while quota is set on home storage                                                   # /drone/src/tests/acceptance/features/apiFederation/federated.feature:173
    Given user "user0" from server "REMOTE" has shared "/PARENT" with user "user1" from server "LOCAL"                                   # FederationContext::userFromServerHasSharedWithUserFromServer()
    And user "user1" from server "LOCAL" has accepted the last pending share                                                             # FederationContext::userFromServerHasAcceptedLastPendingShare()
    And using server "LOCAL"                                                                                                             # FeatureContext::usingServer()
    When user "user1" uploads file "filesForUpload/textfile.txt" to "/PARENT (2)/testquota.txt" with all mechanisms using the WebDAV API # FeatureContext::userUploadsAFileToWithAllMechanisms()
      What does it mean to have old chunks?
      Failed asserting that 'old' is greater than 0.
    Then the HTTP status code of all upload responses should be "201"                                                                    # FeatureContext::theHTTPStatusCodeOfAllUploadResponsesShouldBe()
    Then as user "user0" on server "REMOTE" the files uploaded to "/PARENT/testquota.txt" with all mechanisms should exist               # FeatureContext::asUserOnServerTheFilesUploadedToWithAllMechanismsShouldExit()

https://drone.owncloud.com/owncloud/core/13014/305

@individual-it
Copy link
Member Author

should be fixed now

@individual-it
Copy link
Member Author

backport in #33694

@codecov
Copy link

codecov bot commented Nov 29, 2018

Codecov Report

Merging #33692 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##             master   #33692   +/-   ##
=========================================
  Coverage     64.34%   64.34%           
  Complexity    18305    18305           
=========================================
  Files          1195     1195           
  Lines         69228    69228           
  Branches       1276     1276           
=========================================
  Hits          44548    44548           
  Misses        24308    24308           
  Partials        372      372
Flag Coverage Δ Complexity Δ
#javascript 52.96% <ø> (ø) 0 <ø> (ø) ⬇️
#phpunit 65.66% <ø> (ø) 18305 <ø> (ø) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 177aa97...f02b841. Read the comment docs.

@phil-davis phil-davis merged commit 74ba2e1 into master Nov 29, 2018
@phil-davis phil-davis deleted the newChunkingUploadSteps branch November 29, 2018 12:18
@lock lock bot locked as resolved and limited conversation to collaborators Nov 29, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make more convenient chunking upload API acceptance test steps
2 participants