Skip to content

Commit

Permalink
Build system: add mamake features, start refactoring Mamfiles
Browse files Browse the repository at this point in the history
This adds three interrelated features to mamake and refactors
the Mamfiles to take advantage of them. The new features are:

1. A new strict mode level 2 that activates some more backward
   incompatible tweaks for general sanity. See details below.
2. A new 'shim' command that works much like 'exec', except it
   defines common code prepended to all subsequent shell actions.
3. A new 'loop' block command that repeatedly processes the lines
   between it and 'done', with an iteration variable set to each of
   a series of whitespace-separated values.

The Mamfiles are refactored to take advantage of these. That work
is not done yet, they can be refactored further to make them easier
to read and maintain. But they are now actually starting to look
like they could have been designed by humans, although they remain
very easy to parse by scripts (and I intend to keep them that way).

src/cmd/INIT/mamake.c:
- substitute():
  - Add comments for legibility; some reformatting.
  - At strict >= 2, always keep ${foo:-bar}, ${foo:+bar},
    ${foo:=bar}, ${foo=bar} unexpanded for passing on to shell
    scripts in 'exec'. They're irrelevant for MAM syntax and this
    allows Mamfiles to use common these shell language features.
  - At strict >= 2, disable a absurd hack that made ${mam_lib*}
    variables expand to only the first whitespace-separated word of
    the value after ${AR} was encountered. It was only used once,
    in libcmd/Mamfile (see below).
  - At strict >= 2, disable the insanity that is recursive
    expansion of variable references in variable values.
    All variables now expand to their literal values instead.
- input():
  - Exit on read error.
- run():
  - At strict => 2, prepend 'set -f' (a.k.a. 'set -o noglob') to
    all shell actions, disabling global pathname expansion for
    safer field splitting: any '?', '*' or '[' in variable values
    will no longer cause unexpected pathname expansion.
  - Prepend the shim to shell actions if one was set.
  - If a shim was set, apply the prepended code as well as the
    shell action itself to viewpathing; the shim must be subjected
    to viewpathing as it becomes part of the shell action.
- attributes():
  - At strict => 2, remove the deprectated 'archive', 'generated'
    and 'joint' attributes (note that RULE_generated continues to
    exist as an internal attribute assigend by 'exec').
- make():
  - Add three arguments to prepare for recursive invocation by
    'loop' in the context of the same rule.
  - Rename 'z' (modification time) to 'modtime' for legibility.
  - When recursively invoked by 'loop', share cmd (the 'exec -'
    buffer) with the caller and skip rule initialisation.
  - Fix a bug: do not call bindfile() if the rule is virtual.
    The bug caused a virtual target (e.g. 'all' or 'install')
    to be skipped if a file by that name exists.
  - case KEY('d','o','n','e'):
    - Simply break (enforcing no arguments) when in loop.
    - Make 'mismatched done statement' error message more useful by
      reporting which 'make' it was mismatched with.
    - At strict >= 2, prohibit supplying attributes here
      (deprecated in strict==1).
  - case KEY('e','x','e','c'):
    - If a shim is buffered, set state.shim and reset the buffer;
      any subsequent 'shim' command will start setting a new shim.
  - case KEY('l','o','o','p'):
    - Added. See the code and comments for details.
  - case KEY('s','e','t','v'):
    - At strict => 2, do not remove " from variable values set.
    - Ignore 'setv MAMAKE_STRICT' if it was previously set.
    - When handling 'setv MAMAKE_STRICT', treat the argument as a
      string value representing an integer and store its conversion
      (the strict level) in state.strict.
  - case KEY('s','h','i','m'):
    - Added. This adds line of code to the shim buffer.
  - When in loop, pass both cmd (via the parentcmd pointer) and the
    current rule's modification time back to the caller.

**/Mamfile:
- Major refactoring, taking advantage of the new features.

src/lib/libcmd/Mamfile:
- Fix 'exec - ${AR} x ${mam_libsum} sumlib.o' to avoid needing the
  absurd ${AR} hack. The first word of ${mam_libsum} can trivially
  be extracted by the shell script instead:
	exec - set -- ${mam_libsum}	# set positional parameters
	exec - ${AR} x "$1" sumlib.o	# use only the first one

src/cmd/INIT/mamake.{rt,tst}:
- Add some regression tests.

src/cmd/INIT/utils/Mamfile_indent:
- Add support for 'loop'...'done' indentation.
- Move 'setv AR' so it follows the definitions of the variables it
  depends on. This was the only fix necessary after removing
  support for recursive expansion of variable values.

src/cmd/ksh93/bltins/enum.c:
- Use SH_DICT macro instead of ERROR_CATALOG like every other *.c
  file in src/cmd/ksh93; this allows enum.c to compile with the
  same compiler invocation (with -DSH_DICT="libshell").
  • Loading branch information
McDutchie committed Feb 21, 2024
1 parent 2c37759 commit 4eaf25c
Show file tree
Hide file tree
Showing 16 changed files with 7,519 additions and 6,970 deletions.
12 changes: 5 additions & 7 deletions src/Mamfile
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
note *
note * This build file is in the Make Abstract Machine (MAM) language. It was
note * first generated by nmake, but in the ksh 93u+m distribution we maintain
note * it manually because nmake had too many problems to keep using. The
note * Mamfiles are processed by mamake (src/cmd/INIT/mamake.c); we added
note * indentation to improve readability. The language is documented in
note * src/cmd/INIT/README-mamake.md.
note * This build script is in an extended Make Abstract Machine (MAM)
note * language. Documentation is at: src/cmd/INIT/README-mamake.md
note *
setv MAMAKE_STRICT

setv MAMAKE_STRICT 2

make test virtual
make test_announce virtual
exec - : testing KornShell $KSH_VERSION :
Expand Down
149 changes: 42 additions & 107 deletions src/cmd/INIT/Mamfile
Original file line number Diff line number Diff line change
@@ -1,98 +1,48 @@
note *
note * This build file is in the Make Abstract Machine (MAM) language. It was
note * first generated by nmake, but in the ksh 93u+m distribution we maintain
note * it manually because nmake had too many problems to keep using. The
note * Mamfiles are processed by mamake (src/cmd/INIT/mamake.c); we added
note * indentation to improve readability. The language is documented in
note * src/cmd/INIT/README-mamake.md.
note * This build script is in an extended Make Abstract Machine (MAM)
note * language. Documentation is at: src/cmd/INIT/README-mamake.md
note *
setv MAMAKE_STRICT

setv MAMAKE_STRICT 2
setv INSTALLROOT ../../..
setv PACKAGEROOT ../../../../..
setv CC cc
setv mam_cc_FLAGS
setv CCFLAGS ${-debug-symbols?1?${mam_cc_DEBUG} -D_BLD_DEBUG?${mam_cc_OPTIMIZE}?}
setv CCLDFLAGS ${-strip-symbols?1?${mam_cc_LD_STRIP}??}
setv LDFLAGS

note *
note * initialization for the build system
note *

make install virtual
make iffe
prev iffe.sh
exec - cp ${<} ${@} && chmod u+w,+x ${@}
done
make mktest
prev mktest.sh
exec - cp ${<} ${@} && chmod u+w,+x ${@}
done
make regress
prev regress.sh
exec - cp ${<} ${@} && chmod u+w,+x ${@}
done
make crossexec
prev crossexec.sh
exec - cp ${<} ${@} && chmod u+w,+x ${@}
done
make mkreq
prev mkreq.sh
exec - cp ${<} ${@} && chmod u+w,+x ${@}
done
make mkreq-maplib
prev mkreq-maplib.sh
exec - cp ${<} ${@} && chmod u+w,+x ${@}
done
make mprobe
prev mprobe.sh
exec - cp ${<} ${@} && chmod u+w,+x ${@}
done
make probe
make probe.sh
prev C+probe
prev make.probe
exec - cat ${^} > ${@}
loop DIR bin include/ast lib/lib lib/probe/C/make lib/probe/C/pp lib/probe/C/mam
make ${INSTALLROOT}/${DIR}
exec - mkdir -p ${@}
done
exec - cp ${<} ${@} && chmod u+w,+x ${@}
done
make ${INSTALLROOT}/bin
exec - mkdir -p ${@}
done
make ${INSTALLROOT}/bin/iffe
prev iffe
exec - cp -f ${<} ${@}
done
make ${INSTALLROOT}/bin/mkreq
prev mkreq
exec - cp -f ${<} ${@}
done
make ${INSTALLROOT}/bin/mktest
prev mktest
exec - cp -f ${<} ${@}
done
make ${INSTALLROOT}/bin/regress
prev regress
exec - cp -f ${<} ${@}
done
make ${PACKAGEROOT}/bin
exec - mkdir -p ${PACKAGEROOT}/bin
done
make ${INSTALLROOT}/bin/crossexec
prev crossexec
exec - cp -f ${<} ${@}
done
make ${INSTALLROOT}/bin/proto
note *
note * proto(1) has been removed, but install a backward compatibility stub
note * that allows old Mamfiles containing proto commands to keep working.
note *
prev proto.sh
exec - cp ${<} ${@} && chmod u+w,+x ${@}

note *
note * install helper scripts
note *

make probe.sh
prev C+probe
prev make.probe
exec - cat ${^} > ${@}
done
loop SCRIPT probe iffe mktest regress crossexec mkreq mkreq-maplib mprobe proto
make ${INSTALLROOT}/bin/${SCRIPT}
make ${SCRIPT}
prev ${@}.sh
exec - cp ${<} ${@} && ${STDCHMOD} u+w,+x ${@}
done
exec - cp -f ${<} ${@}
done
done

note *
note * ksh93 function search on PATH
note * ksh93 builtin command library -lcmd
note * ksh93 ld library path search on PATH
note *

make ${INSTALLROOT}/bin/.paths
exec - if test ! -f ${INSTALLROOT}/bin/.paths -o -w ${INSTALLROOT}/bin/.paths
exec - then N='
Expand Down Expand Up @@ -225,25 +175,15 @@ make install virtual
exec - esac
exec - fi
done

note *
note * probe initialization
note *
make ${INSTALLROOT}/lib/probe/C
exec - mkdir -p ${@}
done

make ${INSTALLROOT}/lib/probe/C/probe
prev C+probe
exec - cp -f ${<} ${@}
done
make ${INSTALLROOT}/lib/probe/C/make
exec - mkdir -p ${@}
done
make ${INSTALLROOT}/lib/probe/C/pp
exec - mkdir -p ${@}
done
make ${INSTALLROOT}/lib/probe/C/mam
exec - mkdir -p ${@}
done
make ${INSTALLROOT}/lib/probe/C/mam/probe
prev mprobe
exec - cp -f ${<} ${@}
Expand All @@ -256,12 +196,6 @@ make install virtual
prev probe
exec - cp -f ${<} ${@}
done
make ${INSTALLROOT}/include/ast
exec - mkdir -p ${@}
done
make ${INSTALLROOT}/lib/lib
exec - mkdir -p ${@}
done
note *
note * check if -ldl is required
note *
Expand All @@ -274,8 +208,8 @@ make install virtual
make ${INSTALLROOT}/lib/lib/dl
make dl.req
prev dl.c
exec - mkreq-maplib ${CC} : dl : ${^} : dl
prev mkreq-maplib
exec - mkreq-maplib ${CC} : dl : dl.c : dl
done
exec - cp -f ${<} ${@}
done
Expand All @@ -285,17 +219,17 @@ make install virtual
make ${INSTALLROOT}/lib/lib/iconv
make iconv.req
prev iconv.c
exec - mkreq-maplib ${CC} : iconv : ${^} : iconv
prev mkreq-maplib
exec - mkreq-maplib ${CC} : iconv : iconv.c : iconv
done
exec - cp -f ${<} ${@}
done
make ${INSTALLROOT}/lib/lib/w
make w.req
prev w.c
prev w2.c
exec - mkreq-maplib ${CC} : w : ${^} : w
prev mkreq-maplib
exec - mkreq-maplib ${CC} : w : w.c w2.c : w
done
exec - cp -f ${<} ${@}
done
Expand All @@ -305,8 +239,8 @@ make install virtual
make ${INSTALLROOT}/lib/lib/intl
make intl.req
prev intl.c
exec - mkreq-maplib ${CC} : intl : ${^} : intl
prev mkreq-maplib
exec - mkreq-maplib ${CC} : intl : intl.c : intl
done
exec - cp -f ${<} ${@}
done
Expand All @@ -318,16 +252,16 @@ make install virtual
prev m4.c
prev m5.c
prev m6.c
exec - mkreq-maplib ${CC} : m : ${^} : m
prev mkreq-maplib
exec - mkreq-maplib ${CC} : m : m.c m2.c m3.c m4.c m5.c m6.c : m
done
exec - cp -f ${<} ${@}
done
make ${INSTALLROOT}/lib/lib/nsl
make nsl.req
prev nsl.c
exec - mkreq-maplib ${CC} : nsl : ${^} : nsl
prev mkreq-maplib
exec - mkreq-maplib ${CC} : nsl : nsl.c : nsl
done
exec - cp -f ${<} ${@}
done
Expand All @@ -341,8 +275,8 @@ make install virtual
make socket.req
prev socket.c
prev nsl.c
exec - mkreq-maplib ${CC} : socket : ${^} : socket
prev mkreq-maplib
exec - mkreq-maplib ${CC} : socket : socket.c nsl.c : socket
done
exec - cp -f ${<} ${@}
done
Expand All @@ -364,20 +298,21 @@ make install virtual
prev gdbm.c
prev gdbm1.c
prev gdbm2.c
exec - mkreq-maplib ${CC} : dbm : ${^} : db gdbm_compat gdbm ndbm dbm
prev mkreq-maplib
exec - mkreq-maplib ${CC} : dbm : db.c gdbm.c gdbm1.c gdbm2.c : db gdbm_compat gdbm ndbm dbm
done
exec - cp -f ${<} ${@}
done
done install

make test dontcare virtual
make test.iffe virtual
prev iffe.tst
exec - regress --verbose ${<} iffe
exec - regress iffe.tst iffe
done
make test.mamake virtual
prev mamake.tst
exec - : testing non-libast mamake at $PWD/mamake :
exec - regress --verbose ${<} mamake
exec - regress mamake.tst mamake
done
done test
Loading

0 comments on commit 4eaf25c

Please sign in to comment.