Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce use of tempfile (fix #25) #26

Merged
merged 1 commit into from
Sep 16, 2024
Merged

Reduce use of tempfile (fix #25) #26

merged 1 commit into from
Sep 16, 2024

Conversation

rec
Copy link
Owner

@rec rec commented Sep 8, 2024

Summary by Sourcery

Reduce the use of temporary files by refining the logic for handling temporary file creation and permissions. Fix test assertions to accurately reflect the number of temporary files created. Add a test to ensure consistent permissions for temporary files.

Bug Fixes:

  • Fix the logic in test assertions to correctly account for the number of temporary files created during tests.

Enhancements:

  • Improve the handling of temporary file permissions by ensuring consistent permissions across different temporary file scenarios.

Tests:

  • Add a new test to verify that temporary files have consistent permissions when created with different configurations.

Copy link

sourcery-ai bot commented Sep 8, 2024

Reviewer's Guide by Sourcery

This pull request reduces the use of temporary files and modifies the behavior of file operations in the 'safer' library. The changes primarily affect the test suite and the initialization of the library's main class.

File-Level Changes

Change Details Files
Modify file count assertions in test cases
  • Update assertions to expect one less file after operations
  • Remove redundant assertions
  • Adjust expected file count differences
test/test_open.py
Add new test case for temporary file permissions
  • Create test_tempfile_perms function
  • Test permissions for different temp_file configurations
  • Assert that permissions are consistent across configurations
test/test_open.py
Modify temporary file naming in the main class
  • Add logic to generate a temporary file name when temp_file is True
  • Use parent directory and original filename to create temp file name
safer/__init__.py
Remove explicit chmod operation on temporary files
  • Remove the else clause that set permissions to 0o100644
  • Rely on shutil.copymode for setting permissions when target file exists
safer/__init__.py

Tips
  • Trigger a new Sourcery review by commenting @sourcery-ai review on the pull request.
  • Continue your discussion with Sourcery by replying directly to review comments.
  • You can change your review settings at any time by accessing your dashboard:
    • Enable or disable the Sourcery-generated pull request summary or reviewer's guide;
    • Change the review language;
  • You can always contact us if you have any questions or feedback.

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @rec - I've reviewed your changes - here's some feedback:

Overall Comments:

  • There appears to be a duplication in the test file. The assertions assert len(before) + 1 == len(after) and assert len(after.difference(before)) == 1 are repeated twice in a row. Please remove the duplicate lines.
Here's what I looked at during the review
  • 🟡 General issues: 1 issue found
  • 🟡 Security: 1 issue found
  • 🟡 Testing: 4 issues found
  • 🟢 Complexity: all looks good
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment to tell me if it was helpful.

safer/__init__.py Show resolved Hide resolved
safer/__init__.py Show resolved Hide resolved
test/test_open.py Show resolved Hide resolved
test/test_open.py Show resolved Hide resolved
test/test_open.py Outdated Show resolved Hide resolved
test/test_open.py Show resolved Hide resolved
test/test_open.py Outdated Show resolved Hide resolved
test/test_open.py Show resolved Hide resolved
test/test_open.py Outdated Show resolved Hide resolved
@rec
Copy link
Owner Author

rec commented Sep 8, 2024

The AI schtick wasn't bad and I pushed some consequent changes to clean things up.

@Erotemic
Copy link

Erotemic commented Sep 8, 2024

This looks reasonable to me, but your change on line 660:

        if temp_file is True:
            parent, file = os.path.split(target_file)
            temp_file = os.path.join(parent, f'.{file}.tmp-safer')
        super().__init__(temp_file, delete_failures, parent)
            

Looks like it will mean that 607 will never be called:

        if temp_file is True:
            fd, temp_file = tempfile.mkstemp(dir=parent)
            os.close(fd)

That might be a reasonable thing to do. Also, this isn't necessarily a problem, but it is a behavior change: by using {file}.tmp-safer, you lose the randomized names that could prevent (or perhaps delay) issues with attempts to write to the same file. It looks like mkstemp is doing some complex magic to ensure names are randomized in a thread safe way. I'm not sure if that's worth using here as well or not. However, I do like that the temporary file now gives a much better indication of what the file being written will be called. That is a feature I was interested in.

@rec
Copy link
Owner Author

rec commented Sep 9, 2024

Ah, sorry, I should have written up the change like I said I was going to, but I got distracted by the AI code review! :-D

Two very perceptive comments, thanks!

Line 607 does still get called, exactly in the case where you have a stream which is not a file or a file handle (for example, a socket connection). EDIT: I just checked it by assert False in that clause and 17 tests fail, exactly those cases.


Indeed, I thought for a while about replicating the magic that mkstemp does! But then I realized the following: if you call safer.open on the same file twice at the same time, you're going to get unpredictable results anyway.

I will emphasize this in the documentation (EDIT: I just did) but I'm just going to have a blanket prohibition on trying to write to the same file twice at the same time with safer, because there's no way to make it work.

In that case, the "fixed" temp file name is perfectly OK as there will never be collisions.

@rec rec merged commit 2007c7c into main Sep 16, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants