Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make zig compiler processes live across rebuilds #20633

Merged
merged 6 commits into from
Jul 15, 2024
Merged

Conversation

andrewrk
Copy link
Member

@andrewrk andrewrk commented Jul 15, 2024

$ zig build --watch -fincremental
Build Summary: 5/5 steps succeeded
install success
├─ install abc success
│  └─ zig build-lib abc Debug native success 183ms
└─ install abc success
   └─ zig build-exe abc Debug native success 195ms
watching 67 directories, 2 processes

Closes #20600


This gets the build system fully ready for incremental compilation. The next step is to finish incremental compilation features in the frontend, as you can see from using the results of this PR:

andy@bark ~/t/abc [SIGINT]> ~/dev/zig/build-release/stage4/bin/zig build --watch -fincremental
Build Summary: 5/5 steps succeeded
install success
├─ install abc success
│  └─ zig build-lib abc Debug native success 1s
└─ install abc success
   └─ zig build-exe abc Debug native success 1s
install
└─ install abc
   └─ zig build-exe abc Debug native failure
error: thread 91237 panic: access of union field 'pl_node' while field 'declaration' is active
/home/andy/dev/zig/lib/std/zig/Zir.zig:4027:53: 0x653b360 in getAssociatedSrcHash (zig)
            const pl_node = data[@intFromEnum(inst)].pl_node;
                                                    ^
/home/andy/dev/zig/src/Zcu/PerThread.zig:379:45: 0x653a1f7 in updateZirRefs (zig)
            if (old_zir.getAssociatedSrcHash(old_inst)) |old_hash| hash_changed: {
                                            ^
/home/andy/dev/zig/src/Zcu/PerThread.zig:324:29: 0x654033e in astGenFile (zig)
        try pt.updateZirRefs(file, file_index, prev_zir.*);
                            ^
/home/andy/dev/zig/src/Compilation.zig:4309:18: 0x60a5c52 in workerAstGenFile (zig)
    pt.astGenFile(file, file_index, path_digest, root_decl) catch |err| switch (err) {
                 ^
/home/andy/dev/zig/lib/std/Thread/Pool.zig:178:50: 0x60a64cc in runFn (zig)
            @call(.auto, func, .{id.?} ++ closure.arguments);
                                                 ^
/home/andy/dev/zig/lib/std/Thread/Pool.zig:291:32: 0x6064b8d in worker (zig)
            run_node.data.runFn(&run_node.data, id);
                               ^
/home/andy/dev/zig/lib/std/Thread.zig:409:13: 0x5d837ba in callFn__anon_73354 (zig)
            @call(.auto, f, args);
            ^
/home/andy/dev/zig/lib/std/Thread.zig:678:30: 0x5b016e2 in entryFn (zig)
                return callFn(f, args_ptr.*);
                             ^
???:?:?: 0x7f318dada271 in ??? (libc.so.6)
Unwind information for `libc.so.6:0x7f318dada271` was not available, trace may be incomplete


error: the following command terminated unexpectedly:
/home/andy/dev/zig/build-release/stage4/bin/zig build-exe -fno-llvm -fno-lld -ODebug -Mroot=/home/andy/tmp/abc/src/main.zig --cache-dir /home/andy/tmp/abc/.zig-cache --global-cache-dir /home/andy/.cache/zig --name abc --zig-lib-dir /home/andy/dev/zig/lib/ -fincremental --listen=- 
Build Summary: 2/5 steps succeeded; 1 failed
install transitive failure
└─ install abc transitive failure
   └─ zig build-exe abc Debug native failure
watching 67 directories, 1 processes

Here, the build system has kept the compiler running, and asked for an update. The frontend crashed due to a bug/missing feature.

Users should not use -fincremental yet, in order to avoid triggering these bugs. If you don't pass -fincremental then it doesn't keep the compiler alive.

@andrewrk andrewrk added breaking Implementing this issue could cause existing code to no longer compile or have different behavior. zig build system std.Build, the build runner, `zig build` subcommand, package management release notes This PR should be mentioned in the release notes. labels Jul 15, 2024
Changes the `make` function signature to take an options struct, which
additionally includes `watch: bool`. I intentionally am not exposing
this information to configure phase logic.

Also adds global zig cache to the compiler cache prefixes.

Closes #20600
Remove --debug-incremental

This flag is also added to the build system. Importantly, this tells
Compile step whether or not to keep the compiler running between
rebuilds. It defaults off because it is currently crashing
zirUpdateRefs.
This makes advanced use cases possible such as a long-lived child
process whose progress node gets re-attached to a different parent.
@andrewrk andrewrk added the standard library This issue involves writing Zig code for the standard library. label Jul 15, 2024
Compilation errors now report a failure on rebuilds triggered by file
system watches.

Compiler crashes now report failure correctly on rebuilds triggered by
file system watches.

The compiler subprocess is restarted if a broken pipe is encountered on
a rebuild.
@andrewrk andrewrk merged commit 9d38e82 into master Jul 15, 2024
10 checks passed
@andrewrk andrewrk deleted the long-live-zig branch July 15, 2024 08:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking Implementing this issue could cause existing code to no longer compile or have different behavior. release notes This PR should be mentioned in the release notes. standard library This issue involves writing Zig code for the standard library. zig build system std.Build, the build runner, `zig build` subcommand, package management
Projects
None yet
Development

Successfully merging this pull request may close these issues.

build runner: make watched Compile steps keep the compiler alive
1 participant