ignore errors when parsing non-essential source files

[perrette]

Sometimes in the course of the development, or in messy projects, there are files that are half-way written, or with missing dependencies, but not essential for the executable to build. When it encounters such a file, FoBiS just stops with a parsing error. Wouldn't a better behavior just issue a warning and continue the compilation? Or really even ignore it (or possibly list the files that could not be parsed), unless something is missing at the end of parsing (a module is not found).

In an old, big project, I had to manually exclude 6-7 files (and everytime wait for the full re-compilation until the next parsing error / missing dependency error is reached) before the executable was successfully compiled.

[szaghi]

Sure you are right.

Anyhow FoBiS.py already has such an option:

FoBiS.py -h | grep -i exclude

this will show you the exclude option:

→ FoBiS.py build -h | grep -i exclude
                      [-t TARGET] [-o OUTPUT] [-e EXCLUDE [EXCLUDE ...]]
  -e EXCLUDE [EXCLUDE ...], --exclude EXCLUDE [EXCLUDE ...]
                        Exclude a list of files from the building process

Let me know if this is what you are searching for or you need another type of feature.

See you.

[perrette]

Thanks, I had found that option, and that is what I ambiguously called "manually exclude" files (see second paragraph. This is handy (essential), but does not resolve a common use case (especially for messy scientific code) where files may temporarily lie around that are not ready (but might be so an hour later), which will break the build. This makes this otherwise very handy utility not very robust if the build can be broken so easily. In such case editing the build script would be overkill (since these files are unrelated to the project).

So my suggestion: when encountering a parsing or dependency error in one file, ignore it as long as the full dependency tree from the target is intact. A discrete, one-line warning that the file was ignored seems a much better option than throwing an error. A more verbose warning (similar to the error message thrown when a dependency/parsing error is found) could be issued when the dependency tree is incomplete. I would have that as default, but if you judge otherwise with respect to the robustness of FoBiS.py, you may have an option controlling how strict FoBiS.py should be (debug, production, development...).

[szaghi]

Ok,

nice suggestion. However, I must ask you more details to try implement this "relaxed" building:

• what do you mean for "not ready file"? if the file is a module, to build the project it should be enough to avoid its usage, e.g. temporary commenting use not_still_ready_module into other files; more tricky is the file is not a module where probably its compilation failure break the building of the target;
• what do you mean for "parsing or dependency error"?

With more details and a minimal working example the improvement can be implement more quickly.

Thank you again.

P.S. I have just fixed the bug on the recompilation...

[perrette]

• "not ready file": some junk code that is legacy (kept in a corner just in case), or a convulsive "let's start a new module..." which was then given up for some urgent reason (but the file still lies around). Since it is junk file, it is not even in git, so a git stash will not remove it and the build will still fail. Even if the file is not used anywhere, a compilation is attempted.
• "parsing or dependency error" : I simply meant an error thrown thrown by the compiler. That actually surprises me, I would have thought that building the dependency tree does not require any compilation.

Your questions make me think that maybe the behavior is a bug? I mean any file ending with *f90 found in the project is compiled and an error breaks the build. A minimal example if to just drop an invalid file anywhere in the project (pick any one of your existing project).

echo 0 > junk.f90

A compilation error will then occur even when the --target flag is specified on a completely different file (where it is clear that junk.f90 is not needed to fulfill its dependencies starting from the target, given other parsed files).

My guess : this points to a bug in FoBiS.py, which attempts to compile non-essential files.

[szaghi]

@perrette

Indeed, I not completely follow you, but yes, presently FoBiSy.py try to:

• build your target or all program it found:
  • compile all the dependency (if necessary) of the target (or targets in case of all programs found);
  • compile all non-module libraries old-style fortran messy procedures container, that are intended to be non-module container, that have not a compilations hierarchy that must be respected, but that are necessary to build the target(s), i.e. after the compilation they are simply passed to the linker.

This is the simplest way I figured out to handle both modules and non-modules libraries almost automatically.

If I correctly understand you, you are searching for a new class: a not ready file (that can be both a module or non-module library) the compilation of which may fail without the building process is stopped. In case, I think the modification is simple: I have to just relax the check on compilation phase to just print a warning and then try to going on the building. Some problem can then be highlighted into the linking phase... but I can try.

Do I understand your desiderata?

[perrette]

So I guess your highlighted bullet point is indeed the cause of the problem (that make a simple echo 0 > junk.f90 break any project!). I checked that echo -e "module useless\n 0\nend module" > bla.f90 does not break because module tells FoBiS.py the file is not a dependency.

And yes, the solution is simple and simply requires a warning (at most) when compilation of non-module libraries fails. No need to introduce a new class.

[szaghi]

OK, I will try it soon.

[szaghi]

Fixed.

Now if some non-module library fails to compile a warning is printed

Warning: compiling file ./src/junk-nonmodule.f90 fails! Removing from the list of non-module libraries linked into the target!

then its corresponding object is purged out from the list of non-module-compiled-objects passed to the linker.

I have added a unit-test for you: src/unittest/python/build-test14/ there are two junk files:

• junk-nonmodule.f90: it fails to compile thus it triggers the above algorithm;
• junk-module.f90: it being a module that is not used it is simply omitted and never tried to compile.

Obviously, if you try to use a module that does not compile the building is broken as before.

Let me know if this is ok for you.

See you soon.

[perrette]

This sounds perfect !