Skip to content

Commit

Permalink
builtins.readFile: do not truncate content
Browse files Browse the repository at this point in the history
This closes NixOS#3026 by allowing `builtins.readFile` to read a file with a
wrongly reported file size, for example, files in `/proc` may report a
file size of 0. Reading file in `/proc` is not a good enough motivation,
however I do think it just makes nix more robust by allowing more file
to be read.  Especially, I do considerer the previous behavior to be
dangerous because nix was previously reading truncated files. Examples
of file system which incorrectly report file size may be network file
system or dynamic file system (for performance reason, a dynamic file
system such as FUSE may generate the content of the file on demand).

```
nix-repl> builtins.readFile "/proc/version"
""
```

With this commit:

```
nix-repl> builtins.readFile "/proc/version"
"Linux version 5.6.7 (nixbld@localhost) (gcc version 9.3.0 (GCC)) NixOS#1-NixOS SMP Thu Apr 23 08:38:27 UTC 2020\n"
```

Here is a summary of the behavior changes:

- If the reported size is smaller, previous implementation
was silently returning a truncated file content. The new implementation
is returning the correct file content.

- If a file had a bigger reported file size, previous implementation was
failing with an exception, but the new implementation is returning the
correct file content. This change of behavior is coherent with this pull
request.

Open questions

- The behavior is unchanged for correctly reported file size, however
performances may vary because it uses the more complex sink interface.
Considering that sink is used a lot, I don't think this impacts the
performance a lot.
- `builtins.readFile` on an infinite file, such as `/dev/random` may
fill the memory.
- it does not support adding file to store, such as `${/proc/version}`.
  • Loading branch information
guibou committed Apr 29, 2020
1 parent 06849c3 commit 5a34a47
Showing 1 changed file with 1 addition and 8 deletions.
9 changes: 1 addition & 8 deletions src/libutil/util.cc
Original file line number Diff line number Diff line change
Expand Up @@ -312,14 +312,7 @@ unsigned char getFileType(const Path & path)

string readFile(int fd)
{
struct stat st;
if (fstat(fd, &st) == -1)
throw SysError("statting file");

std::vector<unsigned char> buf(st.st_size);
readFull(fd, buf.data(), st.st_size);

return string((char *) buf.data(), st.st_size);
return drainFD(fd, true);
}


Expand Down

0 comments on commit 5a34a47

Please sign in to comment.