Use a two step Fortran and C++ dependency scanner #12539
Commits on Oct 1, 2024
-
environment: build_dir is allowed to be None in the initializer
But we really expect it to be a string once the initializer is done. Therefore, let's set it to `''` just like we do with the scratchdir in the case that it's set to None in the initializer.
-
tests: our fortran order deps are wrong if a new module is introduced
Because we don't set the appropriate dependencies
-
backend/ninja: fortran must fully depend on all linked targets
Because we use this dependency to ensure that any binary module files are generated, we must have a full dependency. Otherwise, if a new module is added to a dependent target, and used in our target, we race the generation of the binary module definition.
-
build: annotate all
get_all_link_depsthe sameAnd fix some internal annotations
-
build: include each target in get_transitive_link_deps
Otherwise only shared libraries get installed, which isn't correct.
-
build: add link_whole_targets to get_transitive_link_deps
Otherwise we again miss out on half of the targets we need
-
backend/ninja: Fortran targets need to -I transitive deps private dirs
Otherwise they won't be able to find their module outputs.
-
backend/ninja: depfile generation needs a full dependency on all scan…
…ned sources It is not sufficient to have an order dependency on generated sources. If any of those sources change we need to rescan, otherwise we may not notice changes to modules.
-
backend/ninja: fix cross module dependencies
This requires that every Fortran target that uses modules have a full dependency on the scan target of it's dependencies. This means that for a three step target `A -> B -> C`, we cannot start compiling any of B until all of A is linked, and cannot start compiling any of C until all of A and B is linked. This fixes various kinds of races, but it serializes the build and makes it slow. This is the best we can do though, since we don't have any sort of portable format for telling C what is in A and B, so C can't know what sources to wait on for it's modules to be fulfilled.
-
backend/ninja: use a two step process for dependency scanning
This splits the scanner into two discrete steps, one that scans the source files, and one that that reads in the dependency information and produces a dyndep. The scanner uses the JSON format from https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p1689r5.html, which is the same format the MSVC and Clang use for C++ modules scanning. This will allow us to more easily move to using MSVC and clang-scan-deps when possible. As an added bonus, this correctly tracks dependencies across TU and Target boundaries, unlike the previous implementation, which assumed that if it couldn't find a provider that everything was good, but could run into issues. Because of that limitation Fortran code had to fully depend on all of it's dependencies, transitive or not. Now, when using the dep scanner, we can remove that restriction, allowing more parallelism.
-
tests/fortran: also test using a generator()
We already test for custom_targets and configure_file, but we should test a generator too
-
tests/fortran: use fs.copyfile
Since the comment saying we need a generic way to do this is a little outdated.
