| Message ID | 20220326003356.487828-1-dlatypov@google.com |
|---|---|
| State | Accepted |
| Commit | b04d1a8dc7e7ff7ca91a20bef053bcc04265d83a |
| Headers | show |
| Series | Documentation: kunit: update kconfig options needed for UML coverage | expand |
On Fri, Mar 25, 2022 at 8:34 PM Daniel Latypov <dlatypov@google.com> wrote: > > Recent changes have made it so the current set is not sufficient. > Namely, CONFIG_DEBUG_INFO is not being set even when explicitly asked. > > Specifying a version of the debug info fixes this. > Pick CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT as an option that's > hopefully less fragile (esp. given we're tied to GCC 6 and lower). > > Signed-off-by: Daniel Latypov <dlatypov@google.com> I agree with David's comment. Otherwise, Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
On Fri, Mar 25, 2022 at 9:56 PM David Gow <davidgow@google.com> wrote: > <snip> > > # Append coverage options to the current config > > - $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig > > + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig > > $ ./tools/testing/kunit/kunit.py run > > Would we want to instead use a chain of --kconfig_add arguments? (I > think there are advantages either way...) I've been considering this ever since the --kconfig_add patch was accepted. It's more compatible w/ commands using --kunitconfig, but it also looks very verbose. E.g. it looks like $ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6 --kconfig_add=CONFIG_DEBUG_INFO=y --kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y --kconfig_add=CONFIG_GCOV=y Neither looks very appealing to me, so I've just kept it as-is for now. Maybe there's something we can do to make this easier (e.g. allowing --kunitconfig to be repeated and mergable)?
On Mon, Mar 28, 2022 at 12:35 PM Daniel Latypov <dlatypov@google.com> wrote: > > On Fri, Mar 25, 2022 at 9:56 PM David Gow <davidgow@google.com> wrote: > > > > <snip> > > > > # Append coverage options to the current config > > > - $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig > > > + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig > > > $ ./tools/testing/kunit/kunit.py run > > > > Would we want to instead use a chain of --kconfig_add arguments? (I > > think there are advantages either way...) > > I've been considering this ever since the --kconfig_add patch was accepted. > It's more compatible w/ commands using --kunitconfig, but it also > looks very verbose. > E.g. it looks like > > $ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6 > --kconfig_add=CONFIG_DEBUG_INFO=y > --kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y > --kconfig_add=CONFIG_GCOV=y I don't think it's *that* much more verbose, but I see your point. I personally prefer this, but not enough to argue about it. > Neither looks very appealing to me, so I've just kept it as-is for now. > > Maybe there's something we can do to make this easier (e.g. allowing > --kunitconfig to be repeated and mergable)? I would like --kunitconfig to be repeadable and mergable.
On Mon, Mar 28, 2022 at 11:54 AM 'Brendan Higgins' via KUnit Development <kunit-dev@googlegroups.com> wrote: > > On Mon, Mar 28, 2022 at 12:35 PM Daniel Latypov <dlatypov@google.com> wrote: > > > > On Fri, Mar 25, 2022 at 9:56 PM David Gow <davidgow@google.com> wrote: > > > > > > > <snip> > > > > > > # Append coverage options to the current config > > > > - $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig > > > > + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig > > > > $ ./tools/testing/kunit/kunit.py run > > > > > > Would we want to instead use a chain of --kconfig_add arguments? (I > > > think there are advantages either way...) > > > > I've been considering this ever since the --kconfig_add patch was accepted. > > It's more compatible w/ commands using --kunitconfig, but it also > > looks very verbose. > > E.g. it looks like > > > > $ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6 > > --kconfig_add=CONFIG_DEBUG_INFO=y > > --kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y > > --kconfig_add=CONFIG_GCOV=y > > I don't think it's *that* much more verbose, but I see your point. I > personally prefer this, but not enough to argue about it. I personally prefer it too, but I'm biased as the person who added --kconfig_add. They're both ugly enough I'd figured I'd save the bikeshedding for another patch. > > > Neither looks very appealing to me, so I've just kept it as-is for now. > > > > Maybe there's something we can do to make this easier (e.g. allowing > > --kunitconfig to be repeated and mergable)? > > I would like --kunitconfig to be repeadable and mergable. Ack. There's some things to consider first. 1. This will conflict w/ https://patchwork.kernel.org/project/linux-kselftest/patch/20220226212325.2984807-1-dlatypov@google.com/, so I'm going to wait until that gets merged first. 2. some kconfigs can be incompatible (e.g. these options only work on UML, can't combine w/ a non-UML compatible file) How do we make this less of a footgun? We'd talked about how it'd be nice if kconfig/"make olddefconfig" could print out *why* options get dropped (either they're not visible, have unmet deps, etc.). If we had that, I'd feel more comfortable w/ repeatable kunitconfig. 3. People have the ability to do this already if they're really sure it's safe $ cat <files...> | ./tools/testing/kunit/kunit.py run --kunitconfig=/dev/stdin 4. are we committed to supporting a "uml_coverage.kunitconfig" file? As shown by the existence of this patch, we've let it get broken for a bit, at least against linux-next (afaik, it was working on torvalds/master up until the 5.18 window opened and we had some patches reworking CONFIG_DEBUG_INFO). These instructions exist so others don't have to try and re-figure out the steps/workarounds. But they're not more formally "part of KUnit" since no one has had the expertise to maintain it (and fix issues like the reliance on gcc-6), etc. Creating a kunitconfig file for this will further imply ownership.
On Mon, Mar 28, 2022 at 2:58 PM 'Daniel Latypov' via KUnit Development <kunit-dev@googlegroups.com> wrote: > > On Mon, Mar 28, 2022 at 11:54 AM 'Brendan Higgins' via KUnit > Development <kunit-dev@googlegroups.com> wrote: > > > > On Mon, Mar 28, 2022 at 12:35 PM Daniel Latypov <dlatypov@google.com> wrote: > > > > > > On Fri, Mar 25, 2022 at 9:56 PM David Gow <davidgow@google.com> wrote: > > > > > > > > > > <snip> > > > > > > > > # Append coverage options to the current config > > > > > - $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig > > > > > + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig > > > > > $ ./tools/testing/kunit/kunit.py run > > > > > > > > Would we want to instead use a chain of --kconfig_add arguments? (I > > > > think there are advantages either way...) > > > > > > I've been considering this ever since the --kconfig_add patch was accepted. > > > It's more compatible w/ commands using --kunitconfig, but it also > > > looks very verbose. > > > E.g. it looks like > > > > > > $ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6 > > > --kconfig_add=CONFIG_DEBUG_INFO=y > > > --kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y > > > --kconfig_add=CONFIG_GCOV=y > > > > I don't think it's *that* much more verbose, but I see your point. I > > personally prefer this, but not enough to argue about it. > > I personally prefer it too, but I'm biased as the person who added > --kconfig_add. > They're both ugly enough I'd figured I'd save the bikeshedding for > another patch. > > > > > > Neither looks very appealing to me, so I've just kept it as-is for now. > > > > > > Maybe there's something we can do to make this easier (e.g. allowing > > > --kunitconfig to be repeated and mergable)? > > > > I would like --kunitconfig to be repeadable and mergable. > > Ack. > There's some things to consider first. I wasn't saying I want you to do it now. I just like the idea. > 1. This will conflict w/ > https://patchwork.kernel.org/project/linux-kselftest/patch/20220226212325.2984807-1-dlatypov@google.com/, > so I'm going to wait until that gets merged first. Agreed. > 2. some kconfigs can be incompatible (e.g. these options only work on > UML, can't combine w/ a non-UML compatible file) > How do we make this less of a footgun? > We'd talked about how it'd be nice if kconfig/"make olddefconfig" > could print out *why* options get dropped (either they're not visible, > have unmet deps, etc.). If we had that, I'd feel more comfortable w/ > repeatable kunitconfig. Good point. Something to think about. > 3. People have the ability to do this already if they're really sure it's safe > $ cat <files...> | ./tools/testing/kunit/kunit.py run --kunitconfig=/dev/stdin Sure, but I still think multiple --kunitconfig s would be a good feature to add. I think it makes it a bit easier to think about mixing and matching kunitconfigs. > 4. are we committed to supporting a "uml_coverage.kunitconfig" file? > As shown by the existence of this patch, we've let it get broken for a > bit, at least against linux-next (afaik, it was working on > torvalds/master up until the 5.18 window opened and we had some > patches reworking CONFIG_DEBUG_INFO). Good point. I don't think we want to get in that business. UML Coverage needs a lot of work, and I don't think we have the time or resources to own that work by ourselves. If someone else wants to add - and then own - such a kunitconfig, I would fully support them and maybe even help them a bit, but I don't want to own such a file until UML coverage gets a bit more stable. > These instructions exist so others don't have to try and re-figure out > the steps/workarounds. > But they're not more formally "part of KUnit" since no one has had the > expertise to maintain it (and fix issues like the reliance on gcc-6), > etc. > > Creating a kunitconfig file for this will further imply ownership. Agreed.
diff --git a/Documentation/dev-tools/kunit/running_tips.rst b/Documentation/dev-tools/kunit/running_tips.rst index 7b6d26a25959..c36f6760087d 100644 --- a/Documentation/dev-tools/kunit/running_tips.rst +++ b/Documentation/dev-tools/kunit/running_tips.rst @@ -114,6 +114,7 @@ Instead of enabling ``CONFIG_GCOV_KERNEL=y``, we can set these options: CONFIG_DEBUG_KERNEL=y CONFIG_DEBUG_INFO=y + CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y CONFIG_GCOV=y @@ -122,7 +123,7 @@ Putting it together into a copy-pastable sequence of commands: .. code-block:: bash # Append coverage options to the current config - $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig $ ./tools/testing/kunit/kunit.py run # Extract the coverage information from the build dir (.kunit/) $ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/
Recent changes have made it so the current set is not sufficient. Namely, CONFIG_DEBUG_INFO is not being set even when explicitly asked. Specifying a version of the debug info fixes this. Pick CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT as an option that's hopefully less fragile (esp. given we're tied to GCC 6 and lower). Signed-off-by: Daniel Latypov <dlatypov@google.com> --- Documentation/dev-tools/kunit/running_tips.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) base-commit: b14ffae378aa1db993e62b01392e70d1e585fb23