mbox series

[v4,0/3] kunit: support running subsets of test suites from kunit.py

Message ID 20210206000854.2037923-1-dlatypov@google.com
Headers show
Series kunit: support running subsets of test suites from kunit.py | expand

Message

Daniel Latypov Feb. 6, 2021, 12:08 a.m. UTC 
When using `kunit.py run` to run tests, users must populate a
`kunitconfig` file to select the options the tests are hidden behind and
all their dependencies.

The patch [1] to allow specifying a path to kunitconfig promises to make
this nicer as we can have checked in files corresponding to different
sets of tests.

But it's still annoying 
1) when trying to run a subet of tests
2) when you want to run tests that don't have such a pre-existing
kunitconfig and selecting all the necessary options is tricky.

This patch series aims to alleviate both:
1) `kunit.py run 'my-suite-*'`
I.e. use my current kunitconfig, but just run suites that match this glob
2) `kunit.py run --alltests 'my-suite-*'`
I.e. use allyesconfig so I don't have to worry about writing a
kunitconfig at all.

See the first commit message for more details and discussion about
future work.

This patch series also includes a bugfix for a latent bug that can't be
triggered right now but has worse consequences as a result of the
changes needed to plumb in this suite name glob.

[1] https://lore.kernel.org/linux-kselftest/20210201205514.3943096-1-dlatypov@google.com/

---
v1 -> v2:
  Fix free of `suites` subarray in suite_set.
  Found by Dan Carpenter and kernel test robot.
v2 -> v3:
  Add MODULE_PARM_DESC() for kunit.filter_glob.
v3 -> v4:
  Rebase on top of kunit_tool_test.py and typing fixes for merging.

Daniel Latypov (3):
  kunit: add kunit.filter_glob cmdline option to filter suites
  kunit: tool: add support for filtering suites by glob
  kunit: tool: fix unintentional statefulness in run_kernel()

 lib/kunit/Kconfig                      |  1 +
 lib/kunit/executor.c                   | 93 +++++++++++++++++++++++---
 tools/testing/kunit/kunit.py           | 21 ++++--
 tools/testing/kunit/kunit_kernel.py    |  6 +-
 tools/testing/kunit/kunit_tool_test.py | 15 +++--
 5 files changed, 115 insertions(+), 21 deletions(-)


base-commit: aa919f3b019d0e10e0c035598546b30cca7bcb19

Comments

Brendan Higgins Feb. 8, 2021, 6:53 p.m. UTC | #1 
On Fri, Feb 5, 2021 at 4:09 PM Daniel Latypov <dlatypov@google.com> wrote:
>

> When using `kunit.py run` to run tests, users must populate a

> `kunitconfig` file to select the options the tests are hidden behind and

> all their dependencies.

>

> The patch [1] to allow specifying a path to kunitconfig promises to make

> this nicer as we can have checked in files corresponding to different

> sets of tests.

>

> But it's still annoying

> 1) when trying to run a subet of tests

> 2) when you want to run tests that don't have such a pre-existing

> kunitconfig and selecting all the necessary options is tricky.

>

> This patch series aims to alleviate both:

> 1) `kunit.py run 'my-suite-*'`

> I.e. use my current kunitconfig, but just run suites that match this glob

> 2) `kunit.py run --alltests 'my-suite-*'`

> I.e. use allyesconfig so I don't have to worry about writing a

> kunitconfig at all.

>

> See the first commit message for more details and discussion about

> future work.

>

> This patch series also includes a bugfix for a latent bug that can't be

> triggered right now but has worse consequences as a result of the

> changes needed to plumb in this suite name glob.

>

> [1] https://lore.kernel.org/linux-kselftest/20210201205514.3943096-1-dlatypov@google.com/


Tested-by: Brendan Higgins <brendanhiggins@google.com>