Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix incorrect Read calls #4792

Merged
merged 5 commits into from
Mar 23, 2018
Merged

fix incorrect Read calls #4792

merged 5 commits into from
Mar 23, 2018

Conversation

Stebalien
Copy link
Member

@Stebalien Stebalien commented Mar 8, 2018

follow up on #4790

review commit by commit

Note: I ignored all test cases.

@ghost ghost assigned Stebalien Mar 8, 2018
@ghost ghost added the status/in-progress In progress label Mar 8, 2018
@Kubuxu Kubuxu self-requested a review March 8, 2018 03:27
@Stebalien Stebalien mentioned this pull request Mar 8, 2018
9 tasks
cr.reader = nil
err = nil
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure we want to nil out the error in this case and not propagate the io.EOF? This is Read method after all.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to think about that for a moment... yes. We're concatenating output so this is the continue case. The next time the user calls Read, we'll hit the cr.reader == nil case. If there are no more readers, we'll return the EOF.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, right.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this create the case @kevina described in #4793 (comment) (where we return nothing and no error)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. We should probably loop.

if err != io.EOF {
continue
}

Copy link
Member

@Kubuxu Kubuxu Mar 8, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Stebalien what do you think about using io.ReadFull or io.ReadAtLeast here. They will return io.ErrUnexpectedEOF or io.EOF if EOF is hit before the buffer is full and thus telling us to load the next buffer.

This would decrease significantly complexity of this function.

For context: https://golang.org/src/io/io.go?s=10738:10804#L293

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered it but then I got lazy. However, you are right. It'll probably make this simpler.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, this function actually needs to read multiple files so using ReadFull ends up making it more complex.

@ghost ghost assigned Kubuxu Mar 8, 2018
@Kubuxu
Copy link
Member

Kubuxu commented Mar 8, 2018

@Stebalien I've updated it to use io.ReadFull.

Your changes to the dagmodifier seem to cause tests to fail.

@Kubuxu
Copy link
Member

Kubuxu commented Mar 8, 2018

Just confirmed and it is the 1296cda

@Stebalien
Copy link
Member Author

Your changes to the dagmodifier seem to cause tests to fail.

Fixed.

@Kubuxu
Copy link
Member

Kubuxu commented Mar 9, 2018

I will squash it to correct commit, so we won't have broken commit in middle of tree.

Edit: fixed gramma

@Stebalien
Copy link
Member Author

I will squash it to correct commit not to have broken commit in middle of tree.

Fair enough. I just didn't want to hide the change.

@Kubuxu Kubuxu requested a review from magik6k March 9, 2018 11:09
@Kubuxu Kubuxu added need/review Needs a review and removed status/in-progress In progress labels Mar 9, 2018
Copy link
Member

@magik6k magik6k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 comment, rest LGTM

cr.reader = nil
err = nil
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this create the case @kevina described in #4793 (comment) (where we return nothing and no error)?

@ghost ghost added the status/in-progress In progress label Mar 9, 2018
@Kubuxu Kubuxu added RFM and removed status/in-progress In progress need/review Needs a review labels Mar 10, 2018
@Kubuxu Kubuxu added this to the go-ipfs 0.4.15 milestone Mar 13, 2018
@whyrusleeping
Copy link
Member

@Stebalien can i get a rebase here?

The dagmodifier *only* works because we're using a bytes.Buffer.

License: MIT
Signed-off-by: Steven Allen <[email protected]>
Stebalien and others added 3 commits March 23, 2018 13:39
* don't assume that Read fills the buffer.
* don't succeed if the API file is too large.

License: MIT
Signed-off-by: Steven Allen <[email protected]>
License: MIT
Signed-off-by: Steven Allen <[email protected]>
License: MIT
Signed-off-by: Jakub Sztandera <[email protected]>
@Kubuxu
Copy link
Member

Kubuxu commented Mar 23, 2018

Rebased

@Kubuxu Kubuxu removed the status/in-progress In progress label Mar 23, 2018
@whyrusleeping whyrusleeping merged commit 3cbf97b into master Mar 23, 2018
@whyrusleeping whyrusleeping deleted the fix/read-eof branch March 23, 2018 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants