| Message ID | 20180608123307.24773-48-alex.bennee@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | fix building of tests/tcg | expand |
Hi Alex, On 06/08/2018 09:33 AM, Alex Bennée wrote: > One problem with satisfying your docker dependencies in a sub-make it > you might end up trying to satisfy the dependency multiple times. This > is especially a problem with debian-sid based cross compilers and CI > setups. We solve this by doing a docker build pass at the top level > before any sub-makes are called. > > We still need to satisfy dependencies in the Makefile.target call so > people can run tests from individual target directories. We introduce > a new Makefile.probe which gets called for each PROBE_TARGET and > allows us to build up the list. It does require multiply including > config-target.mak which shouldn't cause any issues as it shouldn't > define anything that clashes with config-host.mak. However we undefine > a few key variables each time around. > > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > --- > tests/Makefile.include | 14 +++++++++++++- > tests/tcg/Makefile.probe | 31 +++++++++++++++++++++++++++++++ > 2 files changed, 44 insertions(+), 1 deletion(-) > create mode 100644 tests/tcg/Makefile.probe > > diff --git a/tests/Makefile.include b/tests/Makefile.include > index ca00247e36..049a387436 100644 > --- a/tests/Makefile.include > +++ b/tests/Makefile.include > @@ -931,7 +931,19 @@ BUILD_TCG_TARGET_RULES=$(patsubst %,build-tcg-tests-%, $(LINUX_USER_TARGETS)) > CLEAN_TCG_TARGET_RULES=$(patsubst %,clean-tcg-tests-%, $(LINUX_USER_TARGETS)) > RUN_TCG_TARGET_RULES=$(patsubst %,run-tcg-tests-%, $(LINUX_USER_TARGETS)) > > -build-tcg-tests-%: > +# Probe for the Docker Builds we need for later > +DOCKER_BUILD_TARGETS:= > +$(foreach PROBE_TARGET,$(TARGET_LIST), \ > + $(eval -include $(SRC_PATH)/tests/tcg/Makefile.probe) \ > + $(if $(DOCKER_PREREQ), \ > + $(eval DOCKER_BUILD_TARGETS+=$(DOCKER_PREREQ)))) > + > +$(info DOCKER_BUILD_TARGETS=$(sort $(DOCKER_BUILD_TARGETS))) Is this verbose info on purpose? Else this could be surrounded by something equiv to: ifneq ($(V)$(DEBUG),) ... endif > + > +.PHONY: build-docker-prereqs > +build-docker-prereqs: $(sort $(DOCKER_BUILD_TARGETS)) > + > +build-tcg-tests-%: build-docker-prereqs > $(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $* V="$(V)" TARGET_DIR="$*/" guest-tests,) > > run-tcg-tests-%: build-tcg-tests-% > diff --git a/tests/tcg/Makefile.probe b/tests/tcg/Makefile.probe > new file mode 100644 > index 0000000000..7529e203ad > --- /dev/null > +++ b/tests/tcg/Makefile.probe > @@ -0,0 +1,31 @@ > +# -*- Mode: makefile -*- > +# > +# TCG Compiler Probe > +# > +# This Makefile fragement is included multiple times in the main make > +# script to probe for available compilers. This is used to build up a > +# selection of required docker targets before we invoke a sub-make for > +# each target. > + > +# First we need the target makefile which tells us the target architecture > +-include $(BUILD_DIR)/$(PROBE_TARGET)/config-target.mak > + > +# Then we load up the target architecture makefiles which tell us > +# about the compilers > +undefine CROSS_CC_GUEST > +undefine DOCKER_IMAGE > +DOCKER_PREREQ:= > + > +-include $(SRC_PATH)/tests/tcg/$(TARGET_BASE_ARCH)/Makefile.include > +-include $(SRC_PATH)/tests/tcg/$(TARGET_NAME)/Makefile.include > + > +ifndef CROSS_CC_GUEST > +ifneq ($(DOCKER_IMAGE),) > +DOCKER_PREREQ:=docker-image-$(DOCKER_IMAGE) > +endif > +endif > + > +# Clean-up > +undefine TARGET_NAME > +undefine TARGET_BASE_ARCH > +undefine TARGET_ABI_DIR >
On 06/08/2018 09:33 AM, Alex Bennée wrote: > One problem with satisfying your docker dependencies in a sub-make it > you might end up trying to satisfy the dependency multiple times. This > is especially a problem with debian-sid based cross compilers and CI > setups. We solve this by doing a docker build pass at the top level > before any sub-makes are called. > > We still need to satisfy dependencies in the Makefile.target call so > people can run tests from individual target directories. We introduce > a new Makefile.probe which gets called for each PROBE_TARGET and > allows us to build up the list. It does require multiply including > config-target.mak which shouldn't cause any issues as it shouldn't > define anything that clashes with config-host.mak. However we undefine > a few key variables each time around. > > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > --- > tests/Makefile.include | 14 +++++++++++++- > tests/tcg/Makefile.probe | 31 +++++++++++++++++++++++++++++++ > 2 files changed, 44 insertions(+), 1 deletion(-) > create mode 100644 tests/tcg/Makefile.probe > > diff --git a/tests/Makefile.include b/tests/Makefile.include > index ca00247e36..049a387436 100644 > --- a/tests/Makefile.include > +++ b/tests/Makefile.include > @@ -931,7 +931,19 @@ BUILD_TCG_TARGET_RULES=$(patsubst %,build-tcg-tests-%, $(LINUX_USER_TARGETS)) > CLEAN_TCG_TARGET_RULES=$(patsubst %,clean-tcg-tests-%, $(LINUX_USER_TARGETS)) > RUN_TCG_TARGET_RULES=$(patsubst %,run-tcg-tests-%, $(LINUX_USER_TARGETS)) > > -build-tcg-tests-%: > +# Probe for the Docker Builds we need for later > +DOCKER_BUILD_TARGETS:= > +$(foreach PROBE_TARGET,$(TARGET_LIST), \ > + $(eval -include $(SRC_PATH)/tests/tcg/Makefile.probe) \ > + $(if $(DOCKER_PREREQ), \ > + $(eval DOCKER_BUILD_TARGETS+=$(DOCKER_PREREQ)))) > + > +$(info DOCKER_BUILD_TARGETS=$(sort $(DOCKER_BUILD_TARGETS))) > + > +.PHONY: build-docker-prereqs > +build-docker-prereqs: $(sort $(DOCKER_BUILD_TARGETS)) It seems now all docker images are built, even if you want to test a single arch... Using ~7GB of storage at once. > + > +build-tcg-tests-%: build-docker-prereqs > $(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $* V="$(V)" TARGET_DIR="$*/" guest-tests,) > > run-tcg-tests-%: build-tcg-tests-% > diff --git a/tests/tcg/Makefile.probe b/tests/tcg/Makefile.probe > new file mode 100644 > index 0000000000..7529e203ad > --- /dev/null > +++ b/tests/tcg/Makefile.probe > @@ -0,0 +1,31 @@ > +# -*- Mode: makefile -*- > +# > +# TCG Compiler Probe > +# > +# This Makefile fragement is included multiple times in the main make > +# script to probe for available compilers. This is used to build up a > +# selection of required docker targets before we invoke a sub-make for > +# each target. > + > +# First we need the target makefile which tells us the target architecture > +-include $(BUILD_DIR)/$(PROBE_TARGET)/config-target.mak > + > +# Then we load up the target architecture makefiles which tell us > +# about the compilers > +undefine CROSS_CC_GUEST > +undefine DOCKER_IMAGE > +DOCKER_PREREQ:= > + > +-include $(SRC_PATH)/tests/tcg/$(TARGET_BASE_ARCH)/Makefile.include > +-include $(SRC_PATH)/tests/tcg/$(TARGET_NAME)/Makefile.include > + > +ifndef CROSS_CC_GUEST > +ifneq ($(DOCKER_IMAGE),) > +DOCKER_PREREQ:=docker-image-$(DOCKER_IMAGE) > +endif > +endif > + > +# Clean-up > +undefine TARGET_NAME > +undefine TARGET_BASE_ARCH > +undefine TARGET_ABI_DIR >
Philippe Mathieu-Daudé <f4bug@amsat.org> writes: > Hi Alex, > > On 06/08/2018 09:33 AM, Alex Bennée wrote: >> One problem with satisfying your docker dependencies in a sub-make it >> you might end up trying to satisfy the dependency multiple times. This >> is especially a problem with debian-sid based cross compilers and CI >> setups. We solve this by doing a docker build pass at the top level >> before any sub-makes are called. >> >> We still need to satisfy dependencies in the Makefile.target call so >> people can run tests from individual target directories. We introduce >> a new Makefile.probe which gets called for each PROBE_TARGET and >> allows us to build up the list. It does require multiply including >> config-target.mak which shouldn't cause any issues as it shouldn't >> define anything that clashes with config-host.mak. However we undefine >> a few key variables each time around. >> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> >> --- >> tests/Makefile.include | 14 +++++++++++++- >> tests/tcg/Makefile.probe | 31 +++++++++++++++++++++++++++++++ >> 2 files changed, 44 insertions(+), 1 deletion(-) >> create mode 100644 tests/tcg/Makefile.probe >> >> diff --git a/tests/Makefile.include b/tests/Makefile.include >> index ca00247e36..049a387436 100644 >> --- a/tests/Makefile.include >> +++ b/tests/Makefile.include >> @@ -931,7 +931,19 @@ BUILD_TCG_TARGET_RULES=$(patsubst %,build-tcg-tests-%, $(LINUX_USER_TARGETS)) >> CLEAN_TCG_TARGET_RULES=$(patsubst %,clean-tcg-tests-%, $(LINUX_USER_TARGETS)) >> RUN_TCG_TARGET_RULES=$(patsubst %,run-tcg-tests-%, $(LINUX_USER_TARGETS)) >> >> -build-tcg-tests-%: >> +# Probe for the Docker Builds we need for later >> +DOCKER_BUILD_TARGETS:= >> +$(foreach PROBE_TARGET,$(TARGET_LIST), \ >> + $(eval -include $(SRC_PATH)/tests/tcg/Makefile.probe) \ >> + $(if $(DOCKER_PREREQ), \ >> + $(eval DOCKER_BUILD_TARGETS+=$(DOCKER_PREREQ)))) >> + >> +$(info DOCKER_BUILD_TARGETS=$(sort $(DOCKER_BUILD_TARGETS))) > > Is this verbose info on purpose? Ahh that slipped through, removed now. > > Else this could be surrounded by something equiv to: > > ifneq ($(V)$(DEBUG),) > ... > endif > >> + >> +.PHONY: build-docker-prereqs >> +build-docker-prereqs: $(sort $(DOCKER_BUILD_TARGETS)) >> + >> +build-tcg-tests-%: build-docker-prereqs >> $(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $* V="$(V)" TARGET_DIR="$*/" guest-tests,) >> >> run-tcg-tests-%: build-tcg-tests-% >> diff --git a/tests/tcg/Makefile.probe b/tests/tcg/Makefile.probe >> new file mode 100644 >> index 0000000000..7529e203ad >> --- /dev/null >> +++ b/tests/tcg/Makefile.probe >> @@ -0,0 +1,31 @@ >> +# -*- Mode: makefile -*- >> +# >> +# TCG Compiler Probe >> +# >> +# This Makefile fragement is included multiple times in the main make >> +# script to probe for available compilers. This is used to build up a >> +# selection of required docker targets before we invoke a sub-make for >> +# each target. >> + >> +# First we need the target makefile which tells us the target architecture >> +-include $(BUILD_DIR)/$(PROBE_TARGET)/config-target.mak >> + >> +# Then we load up the target architecture makefiles which tell us >> +# about the compilers >> +undefine CROSS_CC_GUEST >> +undefine DOCKER_IMAGE >> +DOCKER_PREREQ:= >> + >> +-include $(SRC_PATH)/tests/tcg/$(TARGET_BASE_ARCH)/Makefile.include >> +-include $(SRC_PATH)/tests/tcg/$(TARGET_NAME)/Makefile.include >> + >> +ifndef CROSS_CC_GUEST >> +ifneq ($(DOCKER_IMAGE),) >> +DOCKER_PREREQ:=docker-image-$(DOCKER_IMAGE) >> +endif >> +endif >> + >> +# Clean-up >> +undefine TARGET_NAME >> +undefine TARGET_BASE_ARCH >> +undefine TARGET_ABI_DIR >> -- Alex Bennée
Philippe Mathieu-Daudé <f4bug@amsat.org> writes: > On 06/08/2018 09:33 AM, Alex Bennée wrote: >> One problem with satisfying your docker dependencies in a sub-make it >> you might end up trying to satisfy the dependency multiple times. This >> is especially a problem with debian-sid based cross compilers and CI >> setups. We solve this by doing a docker build pass at the top level >> before any sub-makes are called. >> >> We still need to satisfy dependencies in the Makefile.target call so >> people can run tests from individual target directories. We introduce >> a new Makefile.probe which gets called for each PROBE_TARGET and >> allows us to build up the list. It does require multiply including >> config-target.mak which shouldn't cause any issues as it shouldn't >> define anything that clashes with config-host.mak. However we undefine >> a few key variables each time around. >> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> >> --- >> tests/Makefile.include | 14 +++++++++++++- >> tests/tcg/Makefile.probe | 31 +++++++++++++++++++++++++++++++ >> 2 files changed, 44 insertions(+), 1 deletion(-) >> create mode 100644 tests/tcg/Makefile.probe >> >> diff --git a/tests/Makefile.include b/tests/Makefile.include >> index ca00247e36..049a387436 100644 >> --- a/tests/Makefile.include >> +++ b/tests/Makefile.include >> @@ -931,7 +931,19 @@ BUILD_TCG_TARGET_RULES=$(patsubst %,build-tcg-tests-%, $(LINUX_USER_TARGETS)) >> CLEAN_TCG_TARGET_RULES=$(patsubst %,clean-tcg-tests-%, $(LINUX_USER_TARGETS)) >> RUN_TCG_TARGET_RULES=$(patsubst %,run-tcg-tests-%, $(LINUX_USER_TARGETS)) >> >> -build-tcg-tests-%: >> +# Probe for the Docker Builds we need for later >> +DOCKER_BUILD_TARGETS:= >> +$(foreach PROBE_TARGET,$(TARGET_LIST), \ >> + $(eval -include $(SRC_PATH)/tests/tcg/Makefile.probe) \ >> + $(if $(DOCKER_PREREQ), \ >> + $(eval DOCKER_BUILD_TARGETS+=$(DOCKER_PREREQ)))) >> + >> +$(info DOCKER_BUILD_TARGETS=$(sort $(DOCKER_BUILD_TARGETS))) >> + >> +.PHONY: build-docker-prereqs >> +build-docker-prereqs: $(sort $(DOCKER_BUILD_TARGETS)) > > It seems now all docker images are built, even if you want to test a > single arch... Using ~7GB of storage at once. I've moved build-docker-prereqs to build-tcg and made check-tcg depend on that. The single arch test case now relies on the sub-make building things. > >> + >> +build-tcg-tests-%: build-docker-prereqs >> $(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $* V="$(V)" TARGET_DIR="$*/" guest-tests,) >> >> run-tcg-tests-%: build-tcg-tests-% >> diff --git a/tests/tcg/Makefile.probe b/tests/tcg/Makefile.probe >> new file mode 100644 >> index 0000000000..7529e203ad >> --- /dev/null >> +++ b/tests/tcg/Makefile.probe >> @@ -0,0 +1,31 @@ >> +# -*- Mode: makefile -*- >> +# >> +# TCG Compiler Probe >> +# >> +# This Makefile fragement is included multiple times in the main make >> +# script to probe for available compilers. This is used to build up a >> +# selection of required docker targets before we invoke a sub-make for >> +# each target. >> + >> +# First we need the target makefile which tells us the target architecture >> +-include $(BUILD_DIR)/$(PROBE_TARGET)/config-target.mak >> + >> +# Then we load up the target architecture makefiles which tell us >> +# about the compilers >> +undefine CROSS_CC_GUEST >> +undefine DOCKER_IMAGE >> +DOCKER_PREREQ:= >> + >> +-include $(SRC_PATH)/tests/tcg/$(TARGET_BASE_ARCH)/Makefile.include >> +-include $(SRC_PATH)/tests/tcg/$(TARGET_NAME)/Makefile.include >> + >> +ifndef CROSS_CC_GUEST >> +ifneq ($(DOCKER_IMAGE),) >> +DOCKER_PREREQ:=docker-image-$(DOCKER_IMAGE) >> +endif >> +endif >> + >> +# Clean-up >> +undefine TARGET_NAME >> +undefine TARGET_BASE_ARCH >> +undefine TARGET_ABI_DIR >> -- Alex Bennée
diff --git a/tests/Makefile.include b/tests/Makefile.include index ca00247e36..049a387436 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -931,7 +931,19 @@ BUILD_TCG_TARGET_RULES=$(patsubst %,build-tcg-tests-%, $(LINUX_USER_TARGETS)) CLEAN_TCG_TARGET_RULES=$(patsubst %,clean-tcg-tests-%, $(LINUX_USER_TARGETS)) RUN_TCG_TARGET_RULES=$(patsubst %,run-tcg-tests-%, $(LINUX_USER_TARGETS)) -build-tcg-tests-%: +# Probe for the Docker Builds we need for later +DOCKER_BUILD_TARGETS:= +$(foreach PROBE_TARGET,$(TARGET_LIST), \ + $(eval -include $(SRC_PATH)/tests/tcg/Makefile.probe) \ + $(if $(DOCKER_PREREQ), \ + $(eval DOCKER_BUILD_TARGETS+=$(DOCKER_PREREQ)))) + +$(info DOCKER_BUILD_TARGETS=$(sort $(DOCKER_BUILD_TARGETS))) + +.PHONY: build-docker-prereqs +build-docker-prereqs: $(sort $(DOCKER_BUILD_TARGETS)) + +build-tcg-tests-%: build-docker-prereqs $(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $* V="$(V)" TARGET_DIR="$*/" guest-tests,) run-tcg-tests-%: build-tcg-tests-% diff --git a/tests/tcg/Makefile.probe b/tests/tcg/Makefile.probe new file mode 100644 index 0000000000..7529e203ad --- /dev/null +++ b/tests/tcg/Makefile.probe @@ -0,0 +1,31 @@ +# -*- Mode: makefile -*- +# +# TCG Compiler Probe +# +# This Makefile fragement is included multiple times in the main make +# script to probe for available compilers. This is used to build up a +# selection of required docker targets before we invoke a sub-make for +# each target. + +# First we need the target makefile which tells us the target architecture +-include $(BUILD_DIR)/$(PROBE_TARGET)/config-target.mak + +# Then we load up the target architecture makefiles which tell us +# about the compilers +undefine CROSS_CC_GUEST +undefine DOCKER_IMAGE +DOCKER_PREREQ:= + +-include $(SRC_PATH)/tests/tcg/$(TARGET_BASE_ARCH)/Makefile.include +-include $(SRC_PATH)/tests/tcg/$(TARGET_NAME)/Makefile.include + +ifndef CROSS_CC_GUEST +ifneq ($(DOCKER_IMAGE),) +DOCKER_PREREQ:=docker-image-$(DOCKER_IMAGE) +endif +endif + +# Clean-up +undefine TARGET_NAME +undefine TARGET_BASE_ARCH +undefine TARGET_ABI_DIR
One problem with satisfying your docker dependencies in a sub-make it you might end up trying to satisfy the dependency multiple times. This is especially a problem with debian-sid based cross compilers and CI setups. We solve this by doing a docker build pass at the top level before any sub-makes are called. We still need to satisfy dependencies in the Makefile.target call so people can run tests from individual target directories. We introduce a new Makefile.probe which gets called for each PROBE_TARGET and allows us to build up the list. It does require multiply including config-target.mak which shouldn't cause any issues as it shouldn't define anything that clashes with config-host.mak. However we undefine a few key variables each time around. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> --- tests/Makefile.include | 14 +++++++++++++- tests/tcg/Makefile.probe | 31 +++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 1 deletion(-) create mode 100644 tests/tcg/Makefile.probe -- 2.17.1