pkg/openthread: improve package Makefile #14282

aabadie · 2020-06-14T10:16:29Z

Contribution description

This PR reworks the main Makefile of the openthread package. In master, the all target contains all calls to bootstrap, configure and make and this results in all these steps to be run for each rebuild of an application that is using this package.

This PR splits all in several targets to prevent this.

There are other minors fixes applied: use $() instead of ${} to access some make variables, move some info messages in the corresponding target (otherwise they are displayed twice at the beginning of the build).

Testing procedure

Verify that not all steps (bootstrap, configure, make) are not re-run for each build:

this PR

first run (output shorten):

make -C examples/openthread
make: Entering directory '/work/riot/RIOT/examples/openthread'
Building application "openthread" for "samr21-xpro" with MCU "samd21".

[INFO] updating openthread /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread/.pkg-state.git-downloaded
echo thread-reference-20180926 > /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread/.pkg-state.git-downloaded
[INFO] patch openthread
"make" -C /work/riot/RIOT/pkg/openthread
OPENTHREAD_ARGS is [--enable-cli --enable-ftd  --enable-application-coap]configure.ac:198: installing 'third_party/nlbuild-autotools/repo/third_party/autoconf/compile'
configure.ac:114: installing 'third_party/nlbuild-autotools/repo/third_party/autoconf/missing'
examples/apps/cli/Makefile.am: installing 'third_party/nlbuild-autotools/repo/third_party/autoconf/depcomp'
configure: WARNING: unrecognized options: --enable-default-logging
checking build system type... x86_64-unknown-linux-gnu
checking host system type... arm-none-eabi
checking filtered build system type... x86_64-unknown-linux-gnu
checking filtered host system type... arm-none-eabi

[...]

OpenThread built for ftd device
rm /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread/output/lib/libmbedcrypto.a /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread/output/lib/libopenthread-ftd.a
"make" -C /work/riot/RIOT/boards/samr21-xpro
"make" -C /work/riot/RIOT/core
"make" -C /work/riot/RIOT/cpu/samd21
"make" -C /work/riot/RIOT/cpu/cortexm_common
"make" -C /work/riot/RIOT/cpu/cortexm_common/periph
"make" -C /work/riot/RIOT/cpu/sam0_common
"make" -C /work/riot/RIOT/cpu/sam0_common/periph
"make" -C /work/riot/RIOT/cpu/samd21/periph
"make" -C /work/riot/RIOT/drivers
"make" -C /work/riot/RIOT/drivers/at86rf2xx
"make" -C /work/riot/RIOT/drivers/netdev_ieee802154
"make" -C /work/riot/RIOT/drivers/periph_common
"make" -C /work/riot/RIOT/pkg/openthread/contrib
"make" -C /work/riot/RIOT/pkg/openthread/contrib/netdev
"make" -C /work/riot/RIOT/sys
"make" -C /work/riot/RIOT/sys/auto_init
"make" -C /work/riot/RIOT/sys/cpp11-compat
"make" -C /work/riot/RIOT/sys/div
"make" -C /work/riot/RIOT/sys/luid
"make" -C /work/riot/RIOT/sys/net/link_layer/ieee802154
"make" -C /work/riot/RIOT/sys/net/link_layer/l2util
"make" -C /work/riot/RIOT/sys/net/netif
"make" -C /work/riot/RIOT/sys/newlib_syscalls_default
"make" -C /work/riot/RIOT/sys/pm_layered
"make" -C /work/riot/RIOT/sys/ps
"make" -C /work/riot/RIOT/sys/random
"make" -C /work/riot/RIOT/sys/random/tinymt32
"make" -C /work/riot/RIOT/sys/stdio_uart
"make" -C /work/riot/RIOT/sys/timex
"make" -C /work/riot/RIOT/sys/xtimer
   text	   data	    bss	    dec	    hex	filename
 165952	    120	  16092	 182164	  2c794	/work/riot/RIOT/examples/openthread/bin/samr21-xpro/openthread.elf

second run:

make -C examples/openthread
make: Entering directory '/work/riot/RIOT/examples/openthread'
Building application "openthread" for "samr21-xpro" with MCU "samd21".

make[1]: Nothing to be done for 'prepare'.
"make" -C /work/riot/RIOT/pkg/openthread
"make" -C /work/riot/RIOT/boards/samr21-xpro
"make" -C /work/riot/RIOT/core
"make" -C /work/riot/RIOT/cpu/samd21
"make" -C /work/riot/RIOT/cpu/cortexm_common
"make" -C /work/riot/RIOT/cpu/cortexm_common/periph
"make" -C /work/riot/RIOT/cpu/sam0_common
"make" -C /work/riot/RIOT/cpu/sam0_common/periph
"make" -C /work/riot/RIOT/cpu/samd21/periph
"make" -C /work/riot/RIOT/drivers
"make" -C /work/riot/RIOT/drivers/at86rf2xx
"make" -C /work/riot/RIOT/drivers/netdev_ieee802154
"make" -C /work/riot/RIOT/drivers/periph_common
"make" -C /work/riot/RIOT/pkg/openthread/contrib
"make" -C /work/riot/RIOT/pkg/openthread/contrib/netdev
"make" -C /work/riot/RIOT/sys
"make" -C /work/riot/RIOT/sys/auto_init
"make" -C /work/riot/RIOT/sys/cpp11-compat
"make" -C /work/riot/RIOT/sys/div
"make" -C /work/riot/RIOT/sys/luid
"make" -C /work/riot/RIOT/sys/net/link_layer/ieee802154
"make" -C /work/riot/RIOT/sys/net/link_layer/l2util
"make" -C /work/riot/RIOT/sys/net/netif
"make" -C /work/riot/RIOT/sys/newlib_syscalls_default
"make" -C /work/riot/RIOT/sys/pm_layered
"make" -C /work/riot/RIOT/sys/ps
"make" -C /work/riot/RIOT/sys/random
"make" -C /work/riot/RIOT/sys/random/tinymt32
"make" -C /work/riot/RIOT/sys/stdio_uart
"make" -C /work/riot/RIOT/sys/timex
"make" -C /work/riot/RIOT/sys/xtimer
   text	   data	    bss	    dec	    hex	filename
 165952	    120	  16092	 182164	  2c794	/work/riot/RIOT/examples/openthread/bin/samr21-xpro/openthread.elf

master

first run (output shorten):

make -C examples/openthread
make: Entering directory '/work/riot/RIOT/examples/openthread'
Building application "openthread" for "samr21-xpro" with MCU "samd21".

Compile OpenThread for FTD device
$OPENTHREAD_ARGS is [--enable-cli --enable-ftd  --enable-application-coap]
[INFO] cloning openthread
Cloning into '/work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread'...
remote: Enumerating objects: 63578, done.
remote: Total 63578 (delta 0), reused 0 (delta 0), pack-reused 63578
Receiving objects: 100% (63578/63578), 66.32 MiB | 738.00 KiB/s, done.
Resolving deltas: 100% (49089/49089), done.
HEAD is now at 677d49b4 [posix-app] platform UDP (#3070)
[INFO] updating openthread /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread/.pkg-state.git-downloaded
echo thread-reference-20180926 > /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread/.pkg-state.git-downloaded
[INFO] patch openthread
"make" -C /work/riot/RIOT/pkg/openthread
Compile OpenThread for FTD device
$OPENTHREAD_ARGS is [--enable-cli --enable-ftd  --enable-application-coap]
cd /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread && PREFIX="/" ./bootstrap
configure.ac:198: installing 'third_party/nlbuild-autotools/repo/third_party/autoconf/compile'
configure.ac:114: installing 'third_party/nlbuild-autotools/repo/third_party/autoconf/missing'
examples/apps/cli/Makefile.am: installing 'third_party/nlbuild-autotools/repo/third_party/autoconf/depcomp'
cd /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread && CPP="arm-none-eabi-gcc -E" CC="arm-none-eabi-gcc" CXX="arm-none-eabi-g++"\
	OBJC="" OBJCXX="" AR="arm-none-eabi-ar" RANLIB="arm-none-eabi-ranlib" NM="arm-none-eabi-nm" \
	STRIP="" \
	CPPFLAGS="-fdata-sections -ffunction-sections -Os -Wno-implicit-fallthrough -Wno-unused-parameter -mcpu=cortex-m0plus -mlittle-endian -mthumb -mfloat-abi=soft -march=armv6s-m -DOPENTHREAD_PROJECT_CORE_CONFIG_FILE='\"platform_config.h\"'" \
	CFLAGS="-fdata-sections -ffunction-sections -Os -Wno-implicit-fallthrough -Wno-unused-parameter -mcpu=cortex-m0plus -mlittle-endian -mthumb -mfloat-abi=soft -march=armv6s-m " \
	CXXFLAGS="-fdata-sections -ffunction-sections -Os -Wno-implicit-fallthrough -Wno-unused-parameter -Wno-class-memaccess \
	          -mcpu=cortex-m0plus -mlittle-endian -mthumb -mfloat-abi=soft -march=armv6s-m -fno-exceptions -fno-rtti " \
	LDFLAGS="-fdata-sections -ffunction-sections -Os -Wno-implicit-fallthrough -Wno-unused-parameter -mcpu=cortex-m0plus -mlittle-endian -mthumb -mfloat-abi=soft -march=armv6s-m -nostartfiles -specs=nano.specs \
	-specs=nosys.specs -Wl,--gc-sections -Wl,-Map=map.map " \
	./configure --disable-docs --host=arm-none-eabi --target=arm-none-eabi \
	--prefix=/ --enable-default-logging --enable-cli --enable-ftd  --enable-application-coap
configure: WARNING: unrecognized options: --enable-default-logging
checking build system type... x86_64-unknown-linux-gnu
checking host system type... arm-none-eabi
checking filtered build system type... x86_64-unknown-linux-gnu
checking filtered host system type... arm-none-eabi

[...]

cp /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread/output/lib/libmbedcrypto.a /work/riot/RIOT/examples/openthread/bin/samr21-xpro/mbedcrypto.a
cp /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread/output/lib/libopenthread-ftd.a /work/riot/RIOT/examples/openthread/bin/samr21-xpro/openthread-ftd.a
cp /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread/output/lib/libopenthread-cli-ftd.a /work/riot/RIOT/examples/openthread/bin/samr21-xpro/openthread-cli.a
"make" -C /work/riot/RIOT/boards/samr21-xpro
"make" -C /work/riot/RIOT/core
"make" -C /work/riot/RIOT/cpu/samd21
"make" -C /work/riot/RIOT/cpu/cortexm_common
"make" -C /work/riot/RIOT/cpu/cortexm_common/periph
"make" -C /work/riot/RIOT/cpu/sam0_common
"make" -C /work/riot/RIOT/cpu/sam0_common/periph
"make" -C /work/riot/RIOT/cpu/samd21/periph
"make" -C /work/riot/RIOT/drivers
"make" -C /work/riot/RIOT/drivers/at86rf2xx
"make" -C /work/riot/RIOT/drivers/netdev_ieee802154
"make" -C /work/riot/RIOT/drivers/periph_common
"make" -C /work/riot/RIOT/pkg/openthread/contrib
"make" -C /work/riot/RIOT/pkg/openthread/contrib/netdev
"make" -C /work/riot/RIOT/sys
"make" -C /work/riot/RIOT/sys/auto_init
"make" -C /work/riot/RIOT/sys/cpp11-compat
"make" -C /work/riot/RIOT/sys/div
"make" -C /work/riot/RIOT/sys/luid
"make" -C /work/riot/RIOT/sys/net/link_layer/ieee802154
"make" -C /work/riot/RIOT/sys/net/link_layer/l2util
"make" -C /work/riot/RIOT/sys/net/netif
"make" -C /work/riot/RIOT/sys/newlib_syscalls_default
"make" -C /work/riot/RIOT/sys/pm_layered
"make" -C /work/riot/RIOT/sys/ps
"make" -C /work/riot/RIOT/sys/random
"make" -C /work/riot/RIOT/sys/random/tinymt32
"make" -C /work/riot/RIOT/sys/stdio_uart
"make" -C /work/riot/RIOT/sys/timex
"make" -C /work/riot/RIOT/sys/xtimer
   text	   data	    bss	    dec	    hex	filename
 165920	    120	  16092	 182132	  2c774	/work/riot/RIOT/examples/openthread/bin/samr21-xpro/openthread.elf

second run (output shorten):

make -C examples/openthread
make: Entering directory '/work/riot/RIOT/examples/openthread'
Building application "openthread" for "samr21-xpro" with MCU "samd21".

Compile OpenThread for FTD device
$OPENTHREAD_ARGS is [--enable-cli --enable-ftd  --enable-application-coap]
make[1]: Nothing to be done for 'prepare'.
"make" -C /work/riot/RIOT/pkg/openthread
Compile OpenThread for FTD device
$OPENTHREAD_ARGS is [--enable-cli --enable-ftd  --enable-application-coap]
cd /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread && PREFIX="/" ./bootstrap
configure.ac:198: installing 'third_party/nlbuild-autotools/repo/third_party/autoconf/compile'
configure.ac:114: installing 'third_party/nlbuild-autotools/repo/third_party/autoconf/missing'
examples/apps/cli/Makefile.am: installing 'third_party/nlbuild-autotools/repo/third_party/autoconf/depcomp'
cd /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread && CPP="arm-none-eabi-gcc -E" CC="arm-none-eabi-gcc" CXX="arm-none-eabi-g++"\
	OBJC="" OBJCXX="" AR="arm-none-eabi-ar" RANLIB="arm-none-eabi-ranlib" NM="arm-none-eabi-nm" \
	STRIP="" \
	CPPFLAGS="-fdata-sections -ffunction-sections -Os -Wno-implicit-fallthrough -Wno-unused-parameter -mcpu=cortex-m0plus -mlittle-endian -mthumb -mfloat-abi=soft -march=armv6s-m -DOPENTHREAD_PROJECT_CORE_CONFIG_FILE='\"platform_config.h\"'" \
	CFLAGS="-fdata-sections -ffunction-sections -Os -Wno-implicit-fallthrough -Wno-unused-parameter -mcpu=cortex-m0plus -mlittle-endian -mthumb -mfloat-abi=soft -march=armv6s-m " \
	CXXFLAGS="-fdata-sections -ffunction-sections -Os -Wno-implicit-fallthrough -Wno-unused-parameter -Wno-class-memaccess \
	          -mcpu=cortex-m0plus -mlittle-endian -mthumb -mfloat-abi=soft -march=armv6s-m -fno-exceptions -fno-rtti " \
	LDFLAGS="-fdata-sections -ffunction-sections -Os -Wno-implicit-fallthrough -Wno-unused-parameter -mcpu=cortex-m0plus -mlittle-endian -mthumb -mfloat-abi=soft -march=armv6s-m -nostartfiles -specs=nano.specs \
	-specs=nosys.specs -Wl,--gc-sections -Wl,-Map=map.map " \
	./configure --disable-docs --host=arm-none-eabi --target=arm-none-eabi \
	--prefix=/ --enable-default-logging --enable-cli --enable-ftd  --enable-application-coap
configure: WARNING: unrecognized options: --enable-default-logging
checking build system type... x86_64-unknown-linux-gnu
checking host system type... arm-none-eabi
checking filtered build system type... x86_64-unknown-linux-gnu
checking filtered host system type... arm-none-eabi

[...]

cp /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread/output/lib/libmbedcrypto.a /work/riot/RIOT/examples/openthread/bin/samr21-xpro/mbedcrypto.a
cp /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread/output/lib/libopenthread-ftd.a /work/riot/RIOT/examples/openthread/bin/samr21-xpro/openthread-ftd.a
cp /work/riot/RIOT/examples/openthread/bin/pkg/samr21-xpro/openthread/output/lib/libopenthread-cli-ftd.a /work/riot/RIOT/examples/openthread/bin/samr21-xpro/openthread-cli.a
"make" -C /work/riot/RIOT/boards/samr21-xpro
"make" -C /work/riot/RIOT/core
"make" -C /work/riot/RIOT/cpu/samd21
"make" -C /work/riot/RIOT/cpu/cortexm_common
"make" -C /work/riot/RIOT/cpu/cortexm_common/periph
"make" -C /work/riot/RIOT/cpu/sam0_common
"make" -C /work/riot/RIOT/cpu/sam0_common/periph
"make" -C /work/riot/RIOT/cpu/samd21/periph
"make" -C /work/riot/RIOT/drivers
"make" -C /work/riot/RIOT/drivers/at86rf2xx
"make" -C /work/riot/RIOT/drivers/netdev_ieee802154
"make" -C /work/riot/RIOT/drivers/periph_common
"make" -C /work/riot/RIOT/pkg/openthread/contrib
"make" -C /work/riot/RIOT/pkg/openthread/contrib/netdev
"make" -C /work/riot/RIOT/sys
"make" -C /work/riot/RIOT/sys/auto_init
"make" -C /work/riot/RIOT/sys/cpp11-compat
"make" -C /work/riot/RIOT/sys/div
"make" -C /work/riot/RIOT/sys/luid
"make" -C /work/riot/RIOT/sys/net/link_layer/ieee802154
"make" -C /work/riot/RIOT/sys/net/link_layer/l2util
"make" -C /work/riot/RIOT/sys/net/netif
"make" -C /work/riot/RIOT/sys/newlib_syscalls_default
"make" -C /work/riot/RIOT/sys/pm_layered
"make" -C /work/riot/RIOT/sys/ps
"make" -C /work/riot/RIOT/sys/random
"make" -C /work/riot/RIOT/sys/random/tinymt32
"make" -C /work/riot/RIOT/sys/stdio_uart
"make" -C /work/riot/RIOT/sys/timex
"make" -C /work/riot/RIOT/sys/xtimer
   text	   data	    bss	    dec	    hex	filename
 165920	    120	  16092	 182132	  2c774	/work/riot/RIOT/examples/openthread/bin/samr21-xpro/openthread.elf

Issues/PRs references

None

This commit improves the target dependencis so that bootstrap and configure command are not run for each rebuild

fjmolinas · 2020-06-16T13:23:07Z

I'm using make 4.1 and it is rebuilding every time I compile :/

fjmolinas · 2020-06-16T13:27:51Z

I'm using make 4.1 and it is rebuilding every time I compile :/

Hmm it does work properly when I build in docker..

fjmolinas · 2020-06-16T14:50:56Z

#14299 fixed my local issues

aabadie · 2020-06-16T18:11:27Z

#14299 is merged, can you give this another try ?

fjmolinas · 2020-06-16T19:12:41Z

I tested by compiling with this PR and master and comparing the generated hex:

git diff openthread.hex examples/openthread/bin/samr21-xpro/openthread.hex

PR

   text	   data	    bss	    dec	    hex	filename
 165904	    120	  16092	 182116	  2c764	/data/riotbuild/riotbase/examples/openthread/bin/samr21-xpro/openthread.elf

cp examples/openthread/bin/samr21-xpro/openthread.hex .

master

   text	   data	    bss	    dec	    hex	filename
 165904	    120	  16092	 182116	  2c764	/data/riotbuild/riotbase/examples/openthread/bin/samr21-xpro/openthread.elf

diff

git diff openthread.hex examples/openthread/bin/samr21-xpro/openthread.hex
#no diff

fjmolinas

ACK.

aabadie added Area: build system Area: Build system Area: pkg Area: External package ports CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Jun 14, 2020

aabadie requested a review from jia200x as a code owner June 14, 2020 10:16

aabadie force-pushed the pr/pkg/openthread_make_rework branch 2 times, most recently from b97e449 to ee77031 Compare June 14, 2020 18:58

pkg/openthread: improve package Makefile

d7ddf57

This commit improves the target dependencis so that bootstrap and configure command are not run for each rebuild

aabadie force-pushed the pr/pkg/openthread_make_rework branch from ee77031 to d7ddf57 Compare June 14, 2020 20:12

aabadie mentioned this pull request Jun 15, 2020

pkg: store packages sources in a global package directory #14289

Merged

aabadie requested a review from fjmolinas June 16, 2020 12:29

aabadie assigned fjmolinas Jun 16, 2020

fjmolinas approved these changes Jun 16, 2020

View reviewed changes

fjmolinas merged commit e3c9894 into RIOT-OS:master Jun 16, 2020

aabadie deleted the pr/pkg/openthread_make_rework branch June 22, 2020 08:45

miri64 added this to the Release 2020.07 milestone Jun 24, 2020

miri64 added the Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation label Jul 16, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pkg/openthread: improve package Makefile #14282

pkg/openthread: improve package Makefile #14282

aabadie commented Jun 14, 2020

fjmolinas commented Jun 16, 2020

fjmolinas commented Jun 16, 2020 •

edited

Loading

fjmolinas commented Jun 16, 2020

aabadie commented Jun 16, 2020

fjmolinas commented Jun 16, 2020 •

edited

Loading

fjmolinas left a comment

pkg/openthread: improve package Makefile #14282

pkg/openthread: improve package Makefile #14282

Conversation

aabadie commented Jun 14, 2020

Contribution description

Testing procedure

Issues/PRs references

fjmolinas commented Jun 16, 2020

fjmolinas commented Jun 16, 2020 • edited Loading

fjmolinas commented Jun 16, 2020

aabadie commented Jun 16, 2020

fjmolinas commented Jun 16, 2020 • edited Loading

fjmolinas left a comment

Choose a reason for hiding this comment

fjmolinas commented Jun 16, 2020 •

edited

Loading

fjmolinas commented Jun 16, 2020 •

edited

Loading