diff mbox series

[v1,2/5] tests/tcg: fix up test-i386-fprem.ref generation

Message ID 20190705160421.19015-3-alex.bennee@linaro.org
State Superseded
Headers show
Series tcg tests and gdbstub fixes | expand

Commit Message

Alex Bennée July 5, 2019, 4:04 p.m. UTC
We never shipped the reference data in the source tree because it was
quite big (64M). As a result the only option is to generate it
locally. Although we have a rule to generate the reference file we
missed the dependency and location changes, probably because it is
only run for SLOW test runs.

The test still fails with mostly incorrect flags and different than
expected NaNs. I'll leave that for the x86 experts to look at.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Jan Bobek <jan.bobek@gmail.com>
---
 tests/tcg/i386/Makefile.target | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

-- 
2.20.1

Comments

Richard Henderson July 10, 2019, 6:58 a.m. UTC | #1
On 7/5/19 6:04 PM, Alex Bennée wrote:
> We never shipped the reference data in the source tree because it was

> quite big (64M). As a result the only option is to generate it

> locally. Although we have a rule to generate the reference file we

> missed the dependency and location changes, probably because it is

> only run for SLOW test runs.

> 

> The test still fails with mostly incorrect flags and different than

> expected NaNs. I'll leave that for the x86 experts to look at.

> 

> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

> Cc: Paolo Bonzini <pbonzini@redhat.com>

> Cc: Richard Henderson <rth@twiddle.net>

> Cc: Eduardo Habkost <ehabkost@redhat.com>

> Cc: Jan Bobek <jan.bobek@gmail.com>

> ---

>  tests/tcg/i386/Makefile.target | 4 ++--

>  1 file changed, 2 insertions(+), 2 deletions(-)


There's nothing here to guess that test-i386-fprem is in any way natively
runnable.  Otherwise you might just end up testing against the installed qemu.

On the other hand, at the moment this simply won't run at all, since the input
file is missing.  So I suppose this is good enough for SLOW.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>



r~
Philippe Mathieu-Daudé July 10, 2019, 10:41 a.m. UTC | #2
On 7/5/19 6:04 PM, Alex Bennée wrote:
> We never shipped the reference data in the source tree because it was

> quite big (64M). As a result the only option is to generate it


Can we fetch it (with hash verification) or store it compressed?

$ du -ch pc-bios/edk2-*bz2
1.2M    pc-bios/edk2-aarch64-code.fd.bz2
1.2M    pc-bios/edk2-arm-code.fd.bz2
4.0K    pc-bios/edk2-arm-vars.fd.bz2
1.7M    pc-bios/edk2-i386-code.fd.bz2
1.9M    pc-bios/edk2-i386-secure-code.fd.bz2
4.0K    pc-bios/edk2-i386-vars.fd.bz2
1.7M    pc-bios/edk2-x86_64-code.fd.bz2
1.9M    pc-bios/edk2-x86_64-secure-code.fd.bz2
9.3M    total

> locally. Although we have a rule to generate the reference file we

> missed the dependency and location changes, probably because it is

> only run for SLOW test runs.

> 

> The test still fails with mostly incorrect flags and different than

> expected NaNs. I'll leave that for the x86 experts to look at.

> 

> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

> Cc: Paolo Bonzini <pbonzini@redhat.com>

> Cc: Richard Henderson <rth@twiddle.net>

> Cc: Eduardo Habkost <ehabkost@redhat.com>

> Cc: Jan Bobek <jan.bobek@gmail.com>

> ---

>  tests/tcg/i386/Makefile.target | 4 ++--

>  1 file changed, 2 insertions(+), 2 deletions(-)

> 

> diff --git a/tests/tcg/i386/Makefile.target b/tests/tcg/i386/Makefile.target

> index b4033ba3d1..d0eb7023e5 100644

> --- a/tests/tcg/i386/Makefile.target

> +++ b/tests/tcg/i386/Makefile.target

> @@ -35,9 +35,9 @@ test-i386-fprem.ref: test-i386-fprem

>  	$(call quiet-command, ./$< > $@,"GENREF","generating $@")

>  

>  run-test-i386-fprem: TIMEOUT=60

> -run-test-i386-fprem: test-i386-fprem

> +run-test-i386-fprem: test-i386-fprem test-i386-fprem.ref

>  	$(call run-test,test-i386-fprem, $(QEMU) $<,"$< on $(TARGET_NAME)")

> -	$(call diff-out,test-i386-fprem, $(I386_SRC)/$<.ref)

> +	$(call diff-out,test-i386-fprem, test-i386-fprem.ref)

>  else

>  run-test-i386-fprem: test-i386-fprem

>  	$(call skip-test, $<, "SLOW")

>
Alex Bennée July 10, 2019, 11:32 a.m. UTC | #3
Philippe Mathieu-Daudé <philmd@redhat.com> writes:

> On 7/5/19 6:04 PM, Alex Bennée wrote:

>> We never shipped the reference data in the source tree because it was

>> quite big (64M). As a result the only option is to generate it

>

> Can we fetch it (with hash verification) or store it compressed?


Seems like a lot of hoops to jump through. The limitation is that you
need to be on an x86 to generate the reference data but that isn't very
hard these days. If you do happen to be on non-x86 and are not paying
attention it becomes a simple regression test against the system
(binfmt_misc) run QEMU. But as it is currently broken and needing some
love it's just an incremental improvement for where we are at the
moment.

>

> $ du -ch pc-bios/edk2-*bz2

> 1.2M    pc-bios/edk2-aarch64-code.fd.bz2

> 1.2M    pc-bios/edk2-arm-code.fd.bz2

> 4.0K    pc-bios/edk2-arm-vars.fd.bz2

> 1.7M    pc-bios/edk2-i386-code.fd.bz2

> 1.9M    pc-bios/edk2-i386-secure-code.fd.bz2

> 4.0K    pc-bios/edk2-i386-vars.fd.bz2

> 1.7M    pc-bios/edk2-x86_64-code.fd.bz2

> 1.9M    pc-bios/edk2-x86_64-secure-code.fd.bz2

> 9.3M    total

>

>> locally. Although we have a rule to generate the reference file we

>> missed the dependency and location changes, probably because it is

>> only run for SLOW test runs.

>>

>> The test still fails with mostly incorrect flags and different than

>> expected NaNs. I'll leave that for the x86 experts to look at.

>>

>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

>> Cc: Paolo Bonzini <pbonzini@redhat.com>

>> Cc: Richard Henderson <rth@twiddle.net>

>> Cc: Eduardo Habkost <ehabkost@redhat.com>

>> Cc: Jan Bobek <jan.bobek@gmail.com>

>> ---

>>  tests/tcg/i386/Makefile.target | 4 ++--

>>  1 file changed, 2 insertions(+), 2 deletions(-)

>>

>> diff --git a/tests/tcg/i386/Makefile.target b/tests/tcg/i386/Makefile.target

>> index b4033ba3d1..d0eb7023e5 100644

>> --- a/tests/tcg/i386/Makefile.target

>> +++ b/tests/tcg/i386/Makefile.target

>> @@ -35,9 +35,9 @@ test-i386-fprem.ref: test-i386-fprem

>>  	$(call quiet-command, ./$< > $@,"GENREF","generating $@")

>>

>>  run-test-i386-fprem: TIMEOUT=60

>> -run-test-i386-fprem: test-i386-fprem

>> +run-test-i386-fprem: test-i386-fprem test-i386-fprem.ref

>>  	$(call run-test,test-i386-fprem, $(QEMU) $<,"$< on $(TARGET_NAME)")

>> -	$(call diff-out,test-i386-fprem, $(I386_SRC)/$<.ref)

>> +	$(call diff-out,test-i386-fprem, test-i386-fprem.ref)

>>  else

>>  run-test-i386-fprem: test-i386-fprem

>>  	$(call skip-test, $<, "SLOW")

>>



--
Alex Bennée
diff mbox series

Patch

diff --git a/tests/tcg/i386/Makefile.target b/tests/tcg/i386/Makefile.target
index b4033ba3d1..d0eb7023e5 100644
--- a/tests/tcg/i386/Makefile.target
+++ b/tests/tcg/i386/Makefile.target
@@ -35,9 +35,9 @@  test-i386-fprem.ref: test-i386-fprem
 	$(call quiet-command, ./$< > $@,"GENREF","generating $@")
 
 run-test-i386-fprem: TIMEOUT=60
-run-test-i386-fprem: test-i386-fprem
+run-test-i386-fprem: test-i386-fprem test-i386-fprem.ref
 	$(call run-test,test-i386-fprem, $(QEMU) $<,"$< on $(TARGET_NAME)")
-	$(call diff-out,test-i386-fprem, $(I386_SRC)/$<.ref)
+	$(call diff-out,test-i386-fprem, test-i386-fprem.ref)
 else
 run-test-i386-fprem: test-i386-fprem
 	$(call skip-test, $<, "SLOW")