Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

introduce std.event.fs for async file system functions #1294

Merged
merged 26 commits into from
Aug 10, 2018
Merged

Conversation

andrewrk
Copy link
Member

@andrewrk andrewrk commented Jul 26, 2018

checklist

  • implementation for linux. It uses a separate thread because Linux does not have async I/O for the file system.
  • basic fs ops implementation for macos
  • basic fs ops implementation for windows
  • implement pwritev
  • file watching for linux
  • file watching for macos
  • file watching for windows
  • fix a cleanup race condition in stage2 tests

After this I want to implement fs watching because I want to use it in the self hosted compiler.

@bnoordhuis you may be interested in this change since it's the zig equivalent to libuv :-)

std/event/fs.zig Outdated

/// This abstraction helps to close file handles in defer expressions
/// without suspending. Start a CloseOperation before opening a file.
pub const CloseOperation = struct {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bnoordhuis specific question for you: is this necessary? Is there any reason that we shouldn't invoke the close syscall in a blocking fashion for a file system file descriptor? Note that this is not fclose where it may do some flushing, this is the actual close syscall.

See line 278 for usage.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say so. close() can block if there is still data to write, and may not be immediate on NFS.

 * add std.event.RwLock and std.event.RwLocked
 * std.debug.warn does its printing locked
 * add std.Mutex, however it's currently implemented as a spinlock
 * rename std.event.Group.cancelAll to std.event.Group.deinit and change
   the docs and assumptions.
 * add std.HashMap.clone
@andrewrk
Copy link
Member Author

andrewrk commented Aug 3, 2018

Here's a quick demo of what this branch does: https://youtu.be/b_Pm29crq6Q

The stage2 compiler tests still pass for linux, so my goal is now to get this stuff working on MacOS and Windows (the above checkboxes) and then merge.

@kristate
Copy link
Contributor

kristate commented Aug 4, 2018

Question: is suspend { resume @handle(); } still required?

@andrewrk
Copy link
Member Author

andrewrk commented Aug 4, 2018

Yes. That is #1194. It's an issue with LLVM that we can work around and there are some other considerations.

@kristate
Copy link
Contributor

kristate commented Aug 5, 2018

Working on darwin branch. Using kqueue(2) for files and FSEvents for directories.

@andrewrk andrewrk merged commit c4b9466 into master Aug 10, 2018
@kristate
Copy link
Contributor

Yay! Good work!

@andrewrk andrewrk deleted the async-fs branch April 15, 2020 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants