[RFC,0/5] systemtap adding sysroot, cross compiling of user land related scripts

Message ID 1520358641-7062-1-git-send-email-kamensky@cisco.com
Headers show
Series
  • systemtap adding sysroot, cross compiling of user land related scripts
Related show

Message

Victor Kamensky (kamensky) March 6, 2018, 5:50 p.m.
Hi Guys,

I am a fan of SystemTap and big fan of OE. Systemtap and OE use case I have
on my mind is the following:

1) small embedded target image built, because of resource constraint such
target could not run full blown systemtap on, it would not have resources
to have debug symbols, toolchain, kernel sources, etc installed on target,
memory size available to to SystemTap script build, etc

2) on host target image is build with debug symbols available somewhere
on the host

3) systemtap operates in cross compilation mode, builds targetscript.ko module,
which could be copied to target device

4) on target tiny systemtap runtime, like staprun, stapio, utilities are
present and they are used to activate systemtap targetscript.ko and observe
SystemTap in action.

In fact OE already has scripts/crosstap script that provides similar to above
experience. But the scripts one can have with crosstap can deal only with
kernel code. If one would like to write SystemTap script that involves
user land process probes crosstap script will not work.

In part the problem was that in SystemTap itself --sysroot options was
effectively broken or never worked. So it was not possible to build user-land
related script in cross compiled mode. I worked on SystemTap fixing --sysroot 
option support and I posted these fixes on systemtap mailing list [1].

Now with --sysroot option support fixed in SystemTap it comes to the
question how it could be integrated into OE related engineers work flow. And
here I need team's feedback. These patch series implements proposal for such
improved SystemTap OE integration, but some parts can cause controversy.

Here is patches overview, with indication of my concerns and ask for
feedback:

  Revert "systemtap: Cross compilation fix"

      Remove previous OE specific patch. It superseded by --sysroot support
      patch series


  systemtap: support --sysroot option in variety of situations in cross
    build

      effectively [1] integrated with OE, these patches will be discussed
      with SystemTap guys directly.


  image: add IMAGE_GEN_COMBINED_DEBUGFS build option

      this one may cause some controversy. For SystemTap to build user-land
      related script, it needs target sysroot tree overlayed with debug
      symbols. OE already has IMAGE_GEN_DEBUGFS option implemented but
      it produces split target and debug symbols tarballs that have to be
      recombined. I claim step of recombination is a drug on developer
      work flow. IMAGE_GEN_COMBINED_DEBUGFS is similar to IMAGE_GEN_DEBUGFS,
      but resulting rootfs-dbg would contain both target and symbols, so
      it could be used right away. Also I feel that implementation of
      IMAGE_GEN_COMBINED_DEBUGFS option might not be very elegant, especially
      with prelink related issue that changes target binaries


  systemtap: introduce utils variant of systemtap package build

      systemtap-utils recipe is special systemtap build that would compile
      only utilities and have minimal dependencies possible. I considered to
      split out those utilities on packaging step of systemtap but with
      dependencies description, it got very messy, so I decided to go
      with separate conflicting with systemtap recipe


  crosstap2: wrapper to build systemtap script on host against given
    target

      Wrapper script similar to crosstap that take SystemTap script and
      image name, retrieve rest of required information from bitbake -e
      command and construct stap invocation with all proper arguments
      passed needed for cross compilation. Script is implemented in python
      it has more advanced logic then existing crosstap tool. But one may
      ask why I did not improved existing crosstap script - I felt that
      new requirements is too much to be implemented in shell script and
      in backward compatible mode without breaking historic interface to
      crosstap.


If someone interested in full patch series my OE tree with these fixes
is here [2]. It was tested on variety of qemu related machines. Steps that
shows how whole work flow OE and SystemTap works are here [3]. Examples of
crosstap2 invocations are here [4].

[1] https://sourceware.org/ml/systemtap/2018-q1/msg00069.html

[2] https://github.com/victorkamensky/openembedded-core

[3] https://github.com/victorkamensky/systemtap-oe-sysroot-manifest

[4] https://github.com/victorkamensky/systemtap-oe-sysroot-manifest/blob/master/EXAMPLES

Victor Kamensky (5):
  Revert "systemtap: Cross compilation fix"
  systemtap: support --sysroot option in variety of situations in cross
    build
  image: add IMAGE_GEN_COMBINED_DEBUGFS build option
  systemtap: introduce utils variant of systemtap package build
  crosstap2: wrapper to build systemtap script on host against given
    target

 meta/classes/image-prelink.bbclass                 |  13 +
 meta/classes/image.bbclass                         |  11 +-
 meta/lib/oe/rootfs.py                              |  59 +--
 .../systemtap/systemtap-utils_git.bb               |  31 ++
 ...-short-release-r-option-handling-in-case-.patch |  61 +++
 ...-short-release-r-option-handling-in-case-.patch |  68 ++++
 ...uginfo-lookup-with-sysroot-case-do-not-re.patch |  97 +++++
 ...ay-adding-sysroot-path-to-module-name-in-.patch | 124 ++++++
 ...dle-symbolic-links-with-absolute-name-rel.patch | 159 ++++++++
 ...p_umodule_relocate-needs-receive-target-f.patch |  58 +++
 .../systemtap/systemtap/system_map_location.patch  |  23 --
 meta/recipes-kernel/systemtap/systemtap_git.inc    |   7 +-
 scripts/crosstap2                                  | 422 +++++++++++++++++++++
 13 files changed, 1081 insertions(+), 52 deletions(-)
 create mode 100644 meta/recipes-kernel/systemtap/systemtap-utils_git.bb
 create mode 100644 meta/recipes-kernel/systemtap/systemtap/0001-sysroot-fix-short-release-r-option-handling-in-case-.patch
 create mode 100644 meta/recipes-kernel/systemtap/systemtap/0002-sysroot-fix-short-release-r-option-handling-in-case-.patch
 create mode 100644 meta/recipes-kernel/systemtap/systemtap/0003-sysroot-debuginfo-lookup-with-sysroot-case-do-not-re.patch
 create mode 100644 meta/recipes-kernel/systemtap/systemtap/0004-sysroot-delay-adding-sysroot-path-to-module-name-in-.patch
 create mode 100644 meta/recipes-kernel/systemtap/systemtap/0005-sysroot-handle-symbolic-links-with-absolute-name-rel.patch
 create mode 100644 meta/recipes-kernel/systemtap/systemtap/0006-sysroot-_stp_umodule_relocate-needs-receive-target-f.patch
 delete mode 100644 meta/recipes-kernel/systemtap/systemtap/system_map_location.patch
 create mode 100755 scripts/crosstap2

-- 
2.7.4

-- 
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Comments

Alexander Kanavin March 6, 2018, 7 p.m. | #1
On 03/06/2018 07:50 PM, Victor Kamensky wrote:
> I am a fan of SystemTap and big fan of OE. Systemtap and OE use case I have

> on my mind is the following:


Hello Victor,

even though I'm listed as the maintainer of systemtap, I do not actually 
use it or know anything about it - my responsibility ends with keeping 
it up to date and fixing build errors. So I can't comment on the essence 
of these patches. I do suggest however that you first get as many 
patches as possible into the upstream, as carrying them in oe-core adds 
a significant maintenance burden.

Also, if it's possible to merge systemtap-utils into the main systemtap 
recipe (as a binary package), and merge together crosstap and crosstap2, 
please do that.

Alex
-- 
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Victor Kamensky (kamensky) March 9, 2018, 8:55 p.m. | #2
Hi Alex,

On Tue, 6 Mar 2018, Alexander Kanavin wrote:

> On 03/06/2018 07:50 PM, Victor Kamensky wrote:

>> I am a fan of SystemTap and big fan of OE. Systemtap and OE use case I have

>> on my mind is the following:

>

> Hello Victor,

>

> even though I'm listed as the maintainer of systemtap, I do not actually use 

> it or know anything about it - my responsibility ends with keeping it up to 

> date and fixing build errors.


Fair enough. It is understood.

> So I can't comment on the essence of these 

> patches. I do suggest however that you first get as many patches as possible 

> into the upstream, as carrying them in oe-core adds a significant maintenance 

> burden.


Yes, it is understood. I am actively working, as we speak,
with SystemTap folks on putting sysroot related patches into SystemTap.

> Also, if it's possible to merge systemtap-utils into the main systemtap 

> recipe (as a binary package),


I've set to write response why systemtap-utils is really
needed and ended up doing below :). Does it look acceptable
direction to you?

From 808fbe3c4b64c734258da08e92fb367a8a20939b Mon Sep 17 00:00:00 2001
From: Victor Kamensky <kamensky@cisco.com>

Date: Wed, 7 Mar 2018 18:33:33 -0800
Subject: [PATCH 08/11] systemtap: create translator packageconfig

For cases when systemap module compilation happens on host in
cross-compilation mode, and it is desirable to minimize systemtap
presense on target we need to have just smallest possible set of
utilties that are required to run compiled modules.

Introduce new "translator" PACKAGECONFIG, if it is not set
it would mean that just minimal set of run-time utilities will
be included in the package.

For run-time only systemtap build variant use
PACKAGECONFIG_pn-systemtap = "" or
PACKAGECONFIG_pn-systemtap = "monitor"

Suggested-by: Taras Kondratiuk <takondra@cisco.com>
Signed-off-by: Victor Kamensky <kamensky@cisco.com>

---
  meta/recipes-kernel/systemtap/systemtap_git.bb | 15 +++++++++++----
  1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-kernel/systemtap/systemtap_git.bb b/meta/recipes-kernel/systemtap/systemtap_git.bb
index 475b207..b280f58 100644
--- a/meta/recipes-kernel/systemtap/systemtap_git.bb
+++ b/meta/recipes-kernel/systemtap/systemtap_git.bb
@@ -3,9 +3,7 @@ HOMEPAGE = "https://sourceware.org/systemtap/"

  require systemtap_git.inc

-DEPENDS = "boost elfutils"
-
-RDEPENDS_${PN} += "python3-core bash perl"
+DEPENDS = "elfutils"

  EXTRA_OECONF += "--with-libelf=${STAGING_DIR_TARGET} --without-rpm \
              --without-nss --without-avahi --without-dyninst \
@@ -18,7 +16,8 @@ STAP_DOCS ?= "--disable-docs --disable-publican --disable-refdocs"

  EXTRA_OECONF += "${STAP_DOCS} "

-PACKAGECONFIG ??= "sqlite monitor python3-probes"
+PACKAGECONFIG ??= "translator sqlite monitor python3-probes"
+PACKAGECONFIG[translator] = "--enable-translator,--disable-translator,boost,python3-core bash perl"
  PACKAGECONFIG[libvirt] = "--enable-libvirt,--disable-libvirt,libvirt"
  PACKAGECONFIG[sqlite] = "--enable-sqlite,--disable-sqlite,sqlite3"
  PACKAGECONFIG[monitor] = "--enable-monitor,--disable-monitor,ncurses json-c"
@@ -26,4 +25,12 @@ PACKAGECONFIG[python3-probes] = "--with-python3-probes,--without-python3-probes,

  inherit autotools gettext pkgconfig distutils3-base

+do_install_append () {
+   if [ ! -f ${D}${bindir}/stap ]; then
+      # translator disabled case, need to leave only minimal runtime
+      rm -rf ${D}${datadir}/${PN}
+      rm ${D}${libexecdir}/${PN}/stap-env
+   fi
+}
+
  BBCLASSEXTEND = "nativesdk"
-- 
2.7.4

> and merge together crosstap and crosstap2,  please do that.


I added code into crosstap2 to provide backward compatible
crosstap legecy invocation interface. I plan to post new crosstap2 code
as total backward compatible replacement of crosstap with superseding 
functionality.

The only controvercial item left from my original ask is
IMAGE_GEN_COMBINED_DEBUGFS build option that is supposed to
build rootfs-dbg containing combined target and debug symbols.

Thanks,
Victor

> Alex

>

-- 
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Alexander Kanavin March 9, 2018, 9:23 p.m. | #3
On 03/09/2018 10:55 PM, Victor Kamensky wrote:
> I've set to write response why systemtap-utils is really

> needed and ended up doing below :). Does it look acceptable

> direction to you?


Yes.

> The only controvercial item left from my original ask is

> IMAGE_GEN_COMBINED_DEBUGFS build option that is supposed to

> build rootfs-dbg containing combined target and debug symbols. 


You can probably repost that as a regular patch.

Alex
-- 
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core