Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nuke the C++ implementation of Zig from orbit using WASI #13560

Merged
merged 68 commits into from
Dec 6, 2022

Commits on Dec 6, 2022

  1. remove -fstage1 option

    After this commit, the self-hosted compiler does not offer the option to
    use stage1 as a backend anymore.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    2851447 View commit details
    Browse the repository at this point in the history
  2. build: introduce -Dwasi-bootstrap option

    Also, make -Donly-c prevent Autodoc from being included in the binary.
    
    This produces a 2.6 MiB zig.wasm file. 781 KB if piped through zstd.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    db023b9 View commit details
    Browse the repository at this point in the history
  3. std.fs: fix openDirAbsolute

    These functions had a compile error since the introduction of
    IterableDir.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    4e2a960 View commit details
    Browse the repository at this point in the history
  4. actually remove stage1

    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    3ba9165 View commit details
    Browse the repository at this point in the history
  5. build.zig: add bulk_memory to -Dwasi-bootstrap

    In theory this will enhance the performance of interpreting a WASI build
    of Zig.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    0a2fdfb View commit details
    Browse the repository at this point in the history
  6. WASI: remove absolute path emulation from std lib

    Instead of checking for absolute paths and current working directories
    in various file system operations, there is one simple solution: allow
    overriding `std.fs.cwd` on WASI.
    
    os.realpath is back to causing a compile error when used on WASI. This
    caused a compile error in the Sema handling of `@src()`. The compiler
    should never call realpath, so the commit that made this change is
    reverted (95ab942). If this breaks
    debug info, a different strategy is needed to solve it other than using
    realpath.
    
    I also removed the preopens code and replaced it with something much
    simpler. There is no longer any global state in the standard library.
    
    Additionally-
     * os.openat no longer does an unnecessary fstat on WASI when O.WRONLY
       is not provided.
     * os.chdir is back to causing a compile error on WASI.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    d5312d5 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    3a81174 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    6b0d773 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    55ca43a View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    3dcdd55 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    8c1c67b View commit details
    Browse the repository at this point in the history
  12. redo CMakeLists.txt to use WASI interpreter

    current status is that it hits error: OutOfMemory for unknown reasons,
    probably due to bugs in the C WASI interpreter port.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    ef447c3 View commit details
    Browse the repository at this point in the history
  13. interpret the WASI blob to produce zig2.c and compiler_rt.c

     * synchronize zig1.c from zig-wasi external project
     * change the way argv works to avoid absolute paths
     * autodetect isatty
     * compiler_rt: disable some functions when object format is C
     * add missing flag from config.zig.in
    
    The next problem is that compiling compiler_rt.c with gcc gives
    "conflicting types" errors for `__eqhf2` and friends.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    39fd77b View commit details
    Browse the repository at this point in the history
  14. add zstd v1.5.2

    only the decompression files
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    d1b3409 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    fb9a608 View commit details
    Browse the repository at this point in the history
  16. enable the LLVM backend in stage2

    This takes a bit longer since the interpreted part has to do more work
    but it saves a round trip through the compiler by allowing `zig2 build`
    to be the final step. 1-2-3, done.
    
    For me this is currently failing due to compilation errors generated by
    GCC when compiling zig2.c but in theory if those are fixed, it should
    work!
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    34e9bbb View commit details
    Browse the repository at this point in the history
  17. zig1.c: autodetect host target triple

    instead of assuming x8_64-linux in CMake
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    33e3db1 View commit details
    Browse the repository at this point in the history
  18. CMake: use -O1 instead of -O2 for building generated .c code

    Idea here is that this is a sweet spot of not wasting time waiting for
    optimizations but also getting decent runime performance.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    eef780e View commit details
    Browse the repository at this point in the history
  19. Some fixes for the wasi interpreter for macOS (#13587)

    * Remove usage of O_PATH, ifdef for macos stat struct, and integer formatting
    mattnite authored and andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    e9d9be2 View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    6337808 View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    9cb06f3 View commit details
    Browse the repository at this point in the history
  22. std.fmt: more descriptive names for impl functions

    This is a workaround for a limitation of the C backend.
    jacobly0 authored and andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    906120b View commit details
    Browse the repository at this point in the history
  23. Configuration menu
    Copy the full SHA
    ba8d5fc View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    7ef745d View commit details
    Browse the repository at this point in the history
  25. CI: make Windows build from source like everybody else

    Now that OOM is no longer a problem, the Windows CI can do the same
    process every other system does instead of building from a recent Zig
    binary.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    02456a3 View commit details
    Browse the repository at this point in the history
  26. zig1.c: remove executable bit

    This was set on accident.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    9b8cf13 View commit details
    Browse the repository at this point in the history
  27. Configuration menu
    Copy the full SHA
    fd54a01 View commit details
    Browse the repository at this point in the history
  28. build: remove -Dwasi-bootstrap; add update-zig1 step

    Now it is a single command tdo update zig1.wasm.zst.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    7e151cb View commit details
    Browse the repository at this point in the history
  29. CMake: use ReleaseSmall instead of ReleaseFast

    When producing C source code. This enables strip, which should avoid
    bloat and save compilation time.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    a63305b View commit details
    Browse the repository at this point in the history
  30. use zig-wasm2c for bootstrapping

    jacobly0 authored and andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    1263346 View commit details
    Browse the repository at this point in the history
  31. wasi: implement file truncation

    The way this is implemented destroys the contents of the file, so
    revisit if this causes issues in the future.
    jacobly0 authored and andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    e000db2 View commit details
    Browse the repository at this point in the history
  32. CMake adjustments

    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    1e638cb View commit details
    Browse the repository at this point in the history
  33. Configuration menu
    Copy the full SHA
    54b960a View commit details
    Browse the repository at this point in the history
  34. Configuration menu
    Copy the full SHA
    ce4e5fe View commit details
    Browse the repository at this point in the history
  35. Configuration menu
    Copy the full SHA
    f421efb View commit details
    Browse the repository at this point in the history
  36. cmake: fix host target detection

    jacobly0 authored and andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    687ea31 View commit details
    Browse the repository at this point in the history
  37. test: remove reference to stage1

    jacobly0 authored and andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    47a2a52 View commit details
    Browse the repository at this point in the history
  38. Configuration menu
    Copy the full SHA
    fdb98c5 View commit details
    Browse the repository at this point in the history
  39. wasm2c: remove unnecessary brackets to reduce max bracket depth

    This avoids the need to pass `-fbracket-depth=512` to clang.
    jacobly0 authored and andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    9f4ef4d View commit details
    Browse the repository at this point in the history
  40. zig.h: fix shift ub for a shlo by 0

    jacobly0 authored and andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    fad2142 View commit details
    Browse the repository at this point in the history
  41. Configuration menu
    Copy the full SHA
    c8541f0 View commit details
    Browse the repository at this point in the history
  42. Configuration menu
    Copy the full SHA
    bd4a1f3 View commit details
    Browse the repository at this point in the history
  43. CMake: remove --color on

    This flag makes CI servers print garbage to the terminal. I started
    implementing detection in wasi.c, but it would have destroyed the
    beautiful code that only touches the C stdlib, does not do any
    conditional compilation based on the operating system, or rely on any
    POSIX functions.
    
    So instead, let's just do without this flag to retain simplicity in this
    step of the build process.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    ec1334d View commit details
    Browse the repository at this point in the history
  44. Configuration menu
    Copy the full SHA
    57995c2 View commit details
    Browse the repository at this point in the history
  45. translate-c: fix wrong logic adjustment

    In ea9ad1e, I incorrectly applied
    boolean logic to one of the pieces of logic, resulting in a regression
    in translate-c.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    cb01249 View commit details
    Browse the repository at this point in the history
  46. std: fix WASI regressions

    This branch largely reverts 58f961f. I
    would like to revisit the proposal to modify the standard library in
    this way and think more carefully about it before adding isAbsolute()
    checks everywhere.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    e73170f View commit details
    Browse the repository at this point in the history
  47. CMake: adjust apple clang compiler check

    Before it was checking if the compiler was AppleClang, however, this did
    not handle the case when using a compiled-from-source Clang on a macOS
    computer, in which case the linker is still Apple ld64, and it is in
    fact the linker that needs to have the different flag to communicate
    stack size.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    aaf95ce View commit details
    Browse the repository at this point in the history
  48. stage2: revert change to handling of --color on

    This change has been extracted to #13786 and can be solved separately
    than this branch.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    a3fadd2 View commit details
    Browse the repository at this point in the history
  49. Configuration menu
    Copy the full SHA
    5dbd28f View commit details
    Browse the repository at this point in the history
  50. Configuration menu
    Copy the full SHA
    4a70149 View commit details
    Browse the repository at this point in the history
  51. add std.heap.wasm_allocator

    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    823d1e7 View commit details
    Browse the repository at this point in the history
  52. Configuration menu
    Copy the full SHA
    4451786 View commit details
    Browse the repository at this point in the history
  53. Configuration menu
    Copy the full SHA
    ee2fb5b View commit details
    Browse the repository at this point in the history
  54. CMake: remove --color on for compiler_rt too

    Missing change from bcd4ea9
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    eef4348 View commit details
    Browse the repository at this point in the history
  55. CI: use a patched tarball

    In the CI system, I copied the old tarball and then applied
    05c21a2 to its compiler_rt
    implementation.
    
    After this is verified we can drop this commit and regenerate the
    tarballs from a master branch commit.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    3fb1b2c View commit details
    Browse the repository at this point in the history
  56. cmake: fix bootstrap dependencies

    jacobly0 authored and andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    9833835 View commit details
    Browse the repository at this point in the history
  57. wasi: add support for windows paths

    jacobly0 authored and andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    793db27 View commit details
    Browse the repository at this point in the history
  58. CI: update tarballs

    In particular, these two changes are relevant:
    
     * zig cc: support -stack in addition to --stack for linker arg
       - Fixes stack overflow when running zig2 on aarch64-macos.
     * compiler_rt: avoid using weak aliases
       - Fixes duplicate symbol when linking zig2 on aarch64-linux.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    f46567e View commit details
    Browse the repository at this point in the history
  59. Configuration menu
    Copy the full SHA
    0f2a7d3 View commit details
    Browse the repository at this point in the history
  60. CI: aarch64-macos tarball patch

    I messed up the spelling of '-stack_size' making it '-stack' instead.
    Will need to fix on master branch. But let's test this here before
    making another master branch commit.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    c58ebfb View commit details
    Browse the repository at this point in the history
  61. Configuration menu
    Copy the full SHA
    c4dc851 View commit details
    Browse the repository at this point in the history
  62. Configuration menu
    Copy the full SHA
    3686787 View commit details
    Browse the repository at this point in the history
  63. Configuration menu
    Copy the full SHA
    3683602 View commit details
    Browse the repository at this point in the history
  64. CBE: avoid trailing space

    jacobly0 authored and andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    91e4891 View commit details
    Browse the repository at this point in the history
  65. Configuration menu
    Copy the full SHA
    2a3d9c3 View commit details
    Browse the repository at this point in the history
  66. cmake: disable stack protector for zig2 to avoid link errors on windows

    On windows we get:
    
        lld-link: error: undefined symbol: __stack_chk_fail
        >>> referenced by CMakeFiles/zig2.dir/zig2.c.obj:(main)
        >>> referenced by CMakeFiles/zig2.dir/zig2.c.obj:(main_main)
        >>> referenced by CMakeFiles/zig2.dir/zig2.c.obj:(log_scoped_28_default_29_err__anon_2764)
        >>> referenced 36192 more times
    
        lld-link: error: undefined symbol: __stack_chk_guard
        >>> referenced by CMakeFiles/zig2.dir/zig2.c.obj:(.refptr.__stack_chk_guard)
        >>> referenced by CMakeFiles/zig2.dir/compiler_rt.c.obj
    jacobly0 authored and andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    a0a2e75 View commit details
    Browse the repository at this point in the history
  67. CI: revert windows script to master branch version

    This commit chickens out and reverts
    02456a3, leaving it for a future
    enhancement.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    106e967 View commit details
    Browse the repository at this point in the history
  68. add zig1.wasm.zst

    This commit adds a 637 KB binary file to the source repository. This
    commit does nothing else, so it should be replaced with a different
    commit before this branch is merged to avoid bloating the git
    repository.
    andrewrk committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    20d86d9 View commit details
    Browse the repository at this point in the history