| Message ID | 1412942554-752-10-git-send-email-ian.campbell@citrix.com |
|---|---|
| State | New |
| Headers | show |
Ian Campbell writes ("[PATCH RFC OSSTEST 10/19] make-flight: Run a basic test on each arm platform"):
> Unlike x86 there is enough variation in the ARM platforms that it is
> worth having a basic test on each as part of a standard run. This
> relies on each host having an appropriate equiv-$platform host flag.
I think using equiv-* for this is not correct.
Hosts named equiv-* are supposed to be completely identical, but we
might end up with hosts which are similar enough that they qualify for
this cross-architecture testing, and but not completely identical.
Furthermore, using equiv-* makes it difficult to automatically search
for a list of the different board classes we would want to use.
I would invent a new prefix. Perhaps flags like
platform-midway
etc.
Then you could search the database
SELECT UNIQUE platform
WHERE EXISTS host
SUCHTHAT (host,"arch-$arch") in hostflags
AND (host,"blessed-$whatever") in hostflags
etc.
rather than hardcoding the list in make-flight.
> Currently the existing arm midway boxes are "equiv-marilith", this
> turns out to be a bad choice at least for this purpose, since
> marilith is the host name, not the platform, we should either switch
> to or add equiv-midway. ISTR discussing this at the time and picking
> equiv-marilith pretty much arbitrarily, AFAIK nothing is using it
> explicitly today so I think removing and adding equiv-midway is the
> right choice.
See above. `equiv-marilith' would be correct if we had another
identical Calxeda box.
Ian.
On Fri, 2014-10-10 at 15:20 +0100, Ian Jackson wrote: > Ian Campbell writes ("[PATCH RFC OSSTEST 10/19] make-flight: Run a basic test on each arm platform"): > > Unlike x86 there is enough variation in the ARM platforms that it is > > worth having a basic test on each as part of a standard run. This > > relies on each host having an appropriate equiv-$platform host flag. > > I think using equiv-* for this is not correct. > > Hosts named equiv-* are supposed to be completely identical, but we > might end up with hosts which are similar enough that they qualify for > this cross-architecture testing, and but not completely identical. > > Furthermore, using equiv-* makes it difficult to automatically search > for a list of the different board classes we would want to use. > > I would invent a new prefix. Perhaps flags like > platform-midway > etc. Would I have to add some code somewhere to process this? (I'm often surprised to find that these just work via some generic magic, e.g. the buildjob stuff etc, but I don't think so in this case?) Perhaps in ts-hosts-allocate-Executive alongside the equiv handling? > Then you could search the database > > SELECT UNIQUE platform > WHERE EXISTS host > SUCHTHAT (host,"arch-$arch") in hostflags > AND (host,"blessed-$whatever") in hostflags > etc. > > rather than hardcoding the list in make-flight. I think this wouldn't work for standalone mode, would it? > > Currently the existing arm midway boxes are "equiv-marilith", this > > turns out to be a bad choice at least for this purpose, since > > marilith is the host name, not the platform, we should either switch > > to or add equiv-midway. ISTR discussing this at the time and picking > > equiv-marilith pretty much arbitrarily, AFAIK nothing is using it > > explicitly today so I think removing and adding equiv-midway is the > > right choice. > > See above. `equiv-marilith' would be correct if we had another > identical Calxeda box. Given "marilith" is the host name and not any sort of name which would identify "similar" machines -- really?
Ian Campbell writes ("Re: [PATCH RFC OSSTEST 10/19] make-flight: Run a basic test on each arm platform"):
> Would I have to add some code somewhere to process this? (I'm often
> surprised to find that these just work via some generic magic, e.g. the
> buildjob stuff etc, but I don't think so in this case?)
(We discussed this in person.)
> > See above. `equiv-marilith' would be correct if we had another
> > identical Calxeda box.
>
> Given "marilith" is the host name and not any sort of name which would
> identify "similar" machines -- really?
Modulo that `marilith' is the wrong identifier and that I complained
about that at the time, yes :-).
Ian.
On Fri, 2014-10-10 at 15:20 +0100, Ian Jackson wrote: > Ian Campbell writes ("[PATCH RFC OSSTEST 10/19] make-flight: Run a basic test on each arm platform"): > > Unlike x86 there is enough variation in the ARM platforms that it is > > worth having a basic test on each as part of a standard run. This > > relies on each host having an appropriate equiv-$platform host flag. > > I think using equiv-* for this is not correct. > > Hosts named equiv-* are supposed to be completely identical, but we > might end up with hosts which are similar enough that they qualify for > this cross-architecture testing, and but not completely identical. > > Furthermore, using equiv-* makes it difficult to automatically search > for a list of the different board classes we would want to use. > > I would invent a new prefix. Perhaps flags like > platform-midway > etc. > > Then you could search the database > > SELECT UNIQUE platform > WHERE EXISTS host > SUCHTHAT (host,"arch-$arch") in hostflags > AND (host,"blessed-$whatever") in hostflags > etc. This was just a pseudo idea, right? Because the Internet has never heard of this EXISTS ... SUCHTHAT syntax that appeared so tantalizingly useful. I think the real answer is going to involve some exciting JOIN-s? Something like (using equiv-% as a surrogate for the currently non-existent platform-*): SELECT DISTINCT h4.hostflag FROM hostflags h1 INNER JOIN hostflags h2 ON h1.hostname = h2.hostname AND h2.hostflag = 'arch-amd64' INNER JOIN hostflags h3 ON h1.hostname = h3.hostname AND h3.hostflag = 'blessed-real' INNER JOIN hostflags h4 ON h1.hostname = h4.hostname AND h4.hostflag LIKE 'equiv-%'; which lists: equiv-rackservers-s40670 equiv-r310-moth equiv-dell-r310 equiv-rackservers-s40680 equiv-rackservers-s40679 equiv-rackservers-s40663 equiv-rackservers-q21011 Was it your intention that every machine was given a platform-* property? Or do I need to create code to handle the "everything else" case? i.e. on x86 where there is no platform do we want to retain the exact same set of jobs? Ian.
diff --git a/make-flight b/make-flight index 9963a46..0e8e26a 100755 --- a/make-flight +++ b/make-flight @@ -284,10 +284,19 @@ do_passthrough_tests () { test_matrix_do_one () { # Basic PV Linux test with xl + case $xenarch in + armhf) platforms='midway cubietruck arndale' ;; + *) platforms='' ;; + esac + + for platform in ${platforms:-''} ; do + suffix=${platform:+-$platform} + hostflags=${most_hostflags}${platform:+,equiv-$platform} - job_create_test test-$xenarch$kern-$dom0arch-xl test-debian xl \ + job_create_test test-$xenarch$kern-$dom0arch-xl$suffix test-debian xl \ $xenarch $dom0arch \ - $debian_runvars all_hostflags=$most_hostflags + $debian_runvars all_hostflags=$hostflags + done job_create_test test-$xenarch$kern-$dom0arch-libvirt test-debian libvirt \ $xenarch $dom0arch \
Unlike x86 there is enough variation in the ARM platforms that it is worth having a basic test on each as part of a standard run. This relies on each host having an appropriate equiv-$platform host flag. Currently the existing arm midway boxes are "equiv-marilith", this turns out to be a bad choice at least for this purpose, since marilith is the host name, not the platform, we should either switch to or add equiv-midway. ISTR discussing this at the time and picking equiv-marilith pretty much arbitrarily, AFAIK nothing is using it explicitly today so I think removing and adding equiv-midway is the right choice. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> --- make-flight | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)