Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: detect and abort build when getPath is called outside of any make #21541

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

marler8997
Copy link
Contributor

@marler8997 marler8997 commented Sep 28, 2024

zig's build system now detects when any LazyPath.getPath function is called outside of any make function. This is done by saving the currently running step to a threadlocal global variable. An added benefit of this change is we can use this thread local variable to provide better error messages without having to manage/propagate the currently running step throughout the build.

I also have a follow up change in the works that will be able to detect whenever any step tries to use a lazy path without having dependend on it first, but this requires more changes to the API that I'd rather defer to its own PR. With that branch I had found a few errors and included one of the fixes in this commit (see RemoveDir.zig). Also, with this PR and the follow up, we'll have everything we need to be able to implement #21525 (the ability to only fetch lazy dependencies if they are required by the current set of steps/targets).

zig's build system now detect when any LazyPath.getPath function is called
outside of any make function.  This is done by saving the currently running
step to a threadlocal global variable.  An added benefit of this change
is we can use this thread local variable to provide better error messages
without having to manage/propagate the currently running step throughout
the build.
Copy link
Member

@andrewrk andrewrk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why you think thread local data is required to implement this?

I'd rather work on #20981 than this.

@marler8997
Copy link
Contributor Author

To clarify, this change allows the build system to know which step is currently being executed. By doing this, the build system can verify misconfiguation. In particular, the goal here is to catch out steps that are using lazy paths which they have no dependency on (either direct or indirect).

This PR along with a branch I'm working on has caught missing dependencies in the std build system (see master...marler8997:zig:lazyPathMisuse). There's one in RemoveDir.zig, a few in CheckObject.zig. I've also fixed missing dependencies for lazy paths in the past in this PR: https://github.com/ziglang/zig/pull/20475/files. These missing dependencies never would have made it through CI if we had the verification I've added here and in the branch I linked.

Why is this important? Because implementing #21525 requires that the build system is able to find all "lazy paths" that will be needed during a build so it can find the correct set of required lazy dependencies. Steps today are supposed to declare their dependency on lazy paths via LazyPath.addStepDependencies but not all do, so depending on this would be brittle. Adding a way to enforce/check when steps fail to do this makes it viable.

I'm not seeing how #20981 (running the configure phase in a separate process) fixes this. The same bugs I listed above would still exist whether or not the configure phase is separated out.

@marler8997
Copy link
Contributor Author

marler8997 commented Oct 5, 2024

Actually maybe I do see a way that #20981 could help solve this? If we run the configuration in a separate process, then that has alot of implications, some I'm not so clear on. It means everything in the build graph is "serialized" so maybe we can detect misconfiguration at that time? Not sure though, there's alot of unknowns as to how that would work/look, but, theoretically both the dependence and usage of lazy paths would be codified in that serialized graph and we could verify it somehow?

P.S. then again, I'm still not sure this would solve bugs/problems with zig's internal build steps, which this PR and my other branch both catch and prevent from being added back in the future

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants