perl: Handle PACKAGES_DYNAMIC for perl-native

Message ID 20191006154747.7699-1-raj.khem@gmail.com
State Superseded
Headers show
Series
  • perl: Handle PACKAGES_DYNAMIC for perl-native
Related show

Commit Message

Khem Raj Oct. 6, 2019, 3:47 p.m.
A perl module recipe extending to provide native version causes target
perl dependencies to be pulled into native build if the module recipe
has RDEPENDS_${PN} = "perl-module-XXXX" e.g. libxml-sax-base-perl
recipe.

The reason is that native bbclass empties out PACKAGES_DYNAMIC and
perl's PACKAGES_DYNAMIC_class-target is greedy enough to usurp native
modules as well.

Eventually we end up with errors like when sstate is used across
machines

* ERROR: libxml-sax-base-perl-native different signature for task do_populate_sysroot.sigdata between qemux86copy and qemuarm

Therefore, to fix this native case needs to handled specially when
re-assigning module dependencies in split_perl_packages(), where the
modules are named correctly for native case and have a single dependency
on perl-native, secondly, PACKAGES_DYNAMIC for target case needs to be
reined in to spare, -native modules, thirdly, let perl-native take over
the case for providing native modules

This will fix several sstate signature errors like above with external
perl modules providing native variants and having runtime dependencies on
modules which are provided by perl proper

Signed-off-by: Khem Raj <raj.khem@gmail.com>

---
 meta/recipes-devtools/perl/perl_5.30.0.bb | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

-- 
2.23.0

-- 
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Comments

Richard Purdie Oct. 7, 2019, 10:15 p.m. | #1
On Sun, 2019-10-06 at 08:47 -0700, Khem Raj wrote:
> A perl module recipe extending to provide native version causes target

> perl dependencies to be pulled into native build if the module recipe

> has RDEPENDS_${PN} = "perl-module-XXXX" e.g. libxml-sax-base-perl

> recipe.

> 

> The reason is that native bbclass empties out PACKAGES_DYNAMIC and

> perl's PACKAGES_DYNAMIC_class-target is greedy enough to usurp native

> modules as well.

> 

> Eventually we end up with errors like when sstate is used across

> machines

> 

> * ERROR: libxml-sax-base-perl-native different signature for task do_populate_sysroot.sigdata between qemux86copy and qemuarm

> 

> Therefore, to fix this native case needs to handled specially when

> re-assigning module dependencies in split_perl_packages(), where the

> modules are named correctly for native case and have a single dependency

> on perl-native, secondly, PACKAGES_DYNAMIC for target case needs to be

> reined in to spare, -native modules, thirdly, let perl-native take over

> the case for providing native modules

> 

> This will fix several sstate signature errors like above with external

> perl modules providing native variants and having runtime dependencies on

> modules which are provided by perl proper

> 

> Signed-off-by: Khem Raj <raj.khem@gmail.com>

> ---

>  meta/recipes-devtools/perl/perl_5.30.0.bb | 13 +++++++++----

>  1 file changed, 9 insertions(+), 4 deletions(-)

> 

> diff --git a/meta/recipes-devtools/perl/perl_5.30.0.bb b/meta/recipes-devtools/perl/perl_5.30.0.bb

> index a221bce52b..9614477982 100644

> --- a/meta/recipes-devtools/perl/perl_5.30.0.bb

> +++ b/meta/recipes-devtools/perl/perl_5.30.0.bb

> @@ -265,13 +265,18 @@ python split_perl_packages () {

>      # Read the pre-generated dependency file, and use it to set module dependecies

>      for line in open(d.expand("${WORKDIR}") + '/perl-rdepends.txt').readlines():

>          splitline = line.split()

> -        module = splitline[0].replace("RDEPENDS_perl", "RDEPENDS_${PN}")

> -        depends = splitline[2].strip('"').replace("perl-module", "${PN}-module")

> +        if bb.data.inherits_class('native', d):

> +            module = splitline[0] + '-native'

> +            depends = "perl-native"

> +        else:

> +            module = splitline[0].replace("RDEPENDS_perl", "RDEPENDS_${PN}")

> +            depends = splitline[2].strip('"').replace("perl-module", "${PN}-module")

>          d.appendVar(d.expand(module), " " + depends)

>  }

>  

> -PACKAGES_DYNAMIC_class-target += "^perl-module-.*"

> -PACKAGES_DYNAMIC_class-nativesdk += "^nativesdk-perl-module-.*"

> +PACKAGES_DYNAMIC_class-native_forcevariable = "^perl-module-.*-native$"

> +PACKAGES_DYNAMIC_class-target = "^perl-module-.*(?<!native)$"

> +PACKAGES_DYNAMIC_class-nativesdk = "^nativesdk-perl-module-.*"

>  

>  RDEPENDS_${PN}-misc += "perl perl-modules"

>  RDEPENDS_${PN}-pod += "perl"


We should never be using _forcevariable in public repos, let alone OE-
Core. Its a tool of last resort. I guess this is because native.bbclass
is clearing it but we need to find a better way.

Cheers,

Richard

-- 
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Khem Raj Oct. 7, 2019, 10:35 p.m. | #2
On Mon, Oct 7, 2019 at 3:15 PM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Sun, 2019-10-06 at 08:47 -0700, Khem Raj wrote:

> > A perl module recipe extending to provide native version causes target

> > perl dependencies to be pulled into native build if the module recipe

> > has RDEPENDS_${PN} = "perl-module-XXXX" e.g. libxml-sax-base-perl

> > recipe.

> >

> > The reason is that native bbclass empties out PACKAGES_DYNAMIC and

> > perl's PACKAGES_DYNAMIC_class-target is greedy enough to usurp native

> > modules as well.

> >

> > Eventually we end up with errors like when sstate is used across

> > machines

> >

> > * ERROR: libxml-sax-base-perl-native different signature for task

> do_populate_sysroot.sigdata between qemux86copy and qemuarm

> >

> > Therefore, to fix this native case needs to handled specially when

> > re-assigning module dependencies in split_perl_packages(), where the

> > modules are named correctly for native case and have a single dependency

> > on perl-native, secondly, PACKAGES_DYNAMIC for target case needs to be

> > reined in to spare, -native modules, thirdly, let perl-native take over

> > the case for providing native modules

> >

> > This will fix several sstate signature errors like above with external

> > perl modules providing native variants and having runtime dependencies on

> > modules which are provided by perl proper

> >

> > Signed-off-by: Khem Raj <raj.khem@gmail.com>

> > ---

> >  meta/recipes-devtools/perl/perl_5.30.0.bb | 13 +++++++++----

> >  1 file changed, 9 insertions(+), 4 deletions(-)

> >

> > diff --git a/meta/recipes-devtools/perl/perl_5.30.0.bb

> b/meta/recipes-devtools/perl/perl_5.30.0.bb

> > index a221bce52b..9614477982 100644

> > --- a/meta/recipes-devtools/perl/perl_5.30.0.bb

> > +++ b/meta/recipes-devtools/perl/perl_5.30.0.bb

> > @@ -265,13 +265,18 @@ python split_perl_packages () {

> >      # Read the pre-generated dependency file, and use it to set module

> dependecies

> >      for line in open(d.expand("${WORKDIR}") +

> '/perl-rdepends.txt').readlines():

> >          splitline = line.split()

> > -        module = splitline[0].replace("RDEPENDS_perl", "RDEPENDS_${PN}")

> > -        depends = splitline[2].strip('"').replace("perl-module",

> "${PN}-module")

> > +        if bb.data.inherits_class('native', d):

> > +            module = splitline[0] + '-native'

> > +            depends = "perl-native"

> > +        else:

> > +            module = splitline[0].replace("RDEPENDS_perl",

> "RDEPENDS_${PN}")

> > +            depends = splitline[2].strip('"').replace("perl-module",

> "${PN}-module")

> >          d.appendVar(d.expand(module), " " + depends)

> >  }

> >

> > -PACKAGES_DYNAMIC_class-target += "^perl-module-.*"

> > -PACKAGES_DYNAMIC_class-nativesdk += "^nativesdk-perl-module-.*"

> > +PACKAGES_DYNAMIC_class-native_forcevariable = "^perl-module-.*-native$"

> > +PACKAGES_DYNAMIC_class-target = "^perl-module-.*(?<!native)$"

> > +PACKAGES_DYNAMIC_class-nativesdk = "^nativesdk-perl-module-.*"

> >

> >  RDEPENDS_${PN}-misc += "perl perl-modules"

> >  RDEPENDS_${PN}-pod += "perl"

>

> We should never be using _forcevariable in public repos, let alone OE-

> Core. Its a tool of last resort. I guess this is because native.bbclass

> is clearing it but we need to find a better way.



Ok and any suggestions that would be better here ?


>

> Cheers,

>

> Richard

>

>
<div><br></div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Oct 7, 2019 at 3:15 PM Richard Purdie &lt;<a href="mailto:richard.purdie@linuxfoundation.org">richard.purdie@linuxfoundation.org</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Sun, 2019-10-06 at 08:47 -0700, Khem Raj wrote:<br>
&gt; A perl module recipe extending to provide native version causes target<br>
&gt; perl dependencies to be pulled into native build if the module recipe<br>
&gt; has RDEPENDS_${PN} = &quot;perl-module-XXXX&quot; e.g. libxml-sax-base-perl<br>
&gt; recipe.<br>
&gt; <br>
&gt; The reason is that native bbclass empties out PACKAGES_DYNAMIC and<br>
&gt; perl&#39;s PACKAGES_DYNAMIC_class-target is greedy enough to usurp native<br>
&gt; modules as well.<br>
&gt; <br>
&gt; Eventually we end up with errors like when sstate is used across<br>
&gt; machines<br>
&gt; <br>
&gt; * ERROR: libxml-sax-base-perl-native different signature for task do_populate_sysroot.sigdata between qemux86copy and qemuarm<br>
&gt; <br>
&gt; Therefore, to fix this native case needs to handled specially when<br>
&gt; re-assigning module dependencies in split_perl_packages(), where the<br>
&gt; modules are named correctly for native case and have a single dependency<br>
&gt; on perl-native, secondly, PACKAGES_DYNAMIC for target case needs to be<br>
&gt; reined in to spare, -native modules, thirdly, let perl-native take over<br>
&gt; the case for providing native modules<br>
&gt; <br>
&gt; This will fix several sstate signature errors like above with external<br>
&gt; perl modules providing native variants and having runtime dependencies on<br>
&gt; modules which are provided by perl proper<br>
&gt; <br>
&gt; Signed-off-by: Khem Raj &lt;<a href="mailto:raj.khem@gmail.com" target="_blank">raj.khem@gmail.com</a>&gt;<br>
&gt; ---<br>
&gt;  meta/recipes-devtools/perl/<a href="http://perl_5.30.0.bb" rel="noreferrer" target="_blank">perl_5.30.0.bb</a> | 13 +++++++++----<br>
&gt;  1 file changed, 9 insertions(+), 4 deletions(-)<br>
&gt; <br>
&gt; diff --git a/meta/recipes-devtools/perl/<a href="http://perl_5.30.0.bb" rel="noreferrer" target="_blank">perl_5.30.0.bb</a> b/meta/recipes-devtools/perl/<a href="http://perl_5.30.0.bb" rel="noreferrer" target="_blank">perl_5.30.0.bb</a><br>
&gt; index a221bce52b..9614477982 100644<br>
&gt; --- a/meta/recipes-devtools/perl/<a href="http://perl_5.30.0.bb" rel="noreferrer" target="_blank">perl_5.30.0.bb</a><br>
&gt; +++ b/meta/recipes-devtools/perl/<a href="http://perl_5.30.0.bb" rel="noreferrer" target="_blank">perl_5.30.0.bb</a><br>
&gt; @@ -265,13 +265,18 @@ python split_perl_packages () {<br>
&gt;      # Read the pre-generated dependency file, and use it to set module dependecies<br>
&gt;      for line in open(d.expand(&quot;${WORKDIR}&quot;) + &#39;/perl-rdepends.txt&#39;).readlines():<br>
&gt;          splitline = line.split()<br>
&gt; -        module = splitline[0].replace(&quot;RDEPENDS_perl&quot;, &quot;RDEPENDS_${PN}&quot;)<br>
&gt; -        depends = splitline[2].strip(&#39;&quot;&#39;).replace(&quot;perl-module&quot;, &quot;${PN}-module&quot;)<br>
&gt; +        if bb.data.inherits_class(&#39;native&#39;, d):<br>
&gt; +            module = splitline[0] + &#39;-native&#39;<br>
&gt; +            depends = &quot;perl-native&quot;<br>
&gt; +        else:<br>
&gt; +            module = splitline[0].replace(&quot;RDEPENDS_perl&quot;, &quot;RDEPENDS_${PN}&quot;)<br>
&gt; +            depends = splitline[2].strip(&#39;&quot;&#39;).replace(&quot;perl-module&quot;, &quot;${PN}-module&quot;)<br>
&gt;          d.appendVar(d.expand(module), &quot; &quot; + depends)<br>
&gt;  }<br>
&gt;  <br>
&gt; -PACKAGES_DYNAMIC_class-target += &quot;^perl-module-.*&quot;<br>
&gt; -PACKAGES_DYNAMIC_class-nativesdk += &quot;^nativesdk-perl-module-.*&quot;<br>
&gt; +PACKAGES_DYNAMIC_class-native_forcevariable = &quot;^perl-module-.*-native$&quot;<br>
&gt; +PACKAGES_DYNAMIC_class-target = &quot;^perl-module-.*(?&lt;!native)$&quot;<br>
&gt; +PACKAGES_DYNAMIC_class-nativesdk = &quot;^nativesdk-perl-module-.*&quot;<br>
&gt;  <br>
&gt;  RDEPENDS_${PN}-misc += &quot;perl perl-modules&quot;<br>
&gt;  RDEPENDS_${PN}-pod += &quot;perl&quot;<br>
<br>
We should never be using _forcevariable in public repos, let alone OE-<br>
Core. Its a tool of last resort. I guess this is because native.bbclass<br>
is clearing it but we need to find a better way.</blockquote><div dir="auto"><br></div><div dir="auto">Ok and any suggestions that would be better here ?</div><div dir="auto"><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
<br>
Cheers,<br>
<br>
Richard<br>
<br>
</blockquote></div></div>
-- 
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Patch

diff --git a/meta/recipes-devtools/perl/perl_5.30.0.bb b/meta/recipes-devtools/perl/perl_5.30.0.bb
index a221bce52b..9614477982 100644
--- a/meta/recipes-devtools/perl/perl_5.30.0.bb
+++ b/meta/recipes-devtools/perl/perl_5.30.0.bb
@@ -265,13 +265,18 @@  python split_perl_packages () {
     # Read the pre-generated dependency file, and use it to set module dependecies
     for line in open(d.expand("${WORKDIR}") + '/perl-rdepends.txt').readlines():
         splitline = line.split()
-        module = splitline[0].replace("RDEPENDS_perl", "RDEPENDS_${PN}")
-        depends = splitline[2].strip('"').replace("perl-module", "${PN}-module")
+        if bb.data.inherits_class('native', d):
+            module = splitline[0] + '-native'
+            depends = "perl-native"
+        else:
+            module = splitline[0].replace("RDEPENDS_perl", "RDEPENDS_${PN}")
+            depends = splitline[2].strip('"').replace("perl-module", "${PN}-module")
         d.appendVar(d.expand(module), " " + depends)
 }
 
-PACKAGES_DYNAMIC_class-target += "^perl-module-.*"
-PACKAGES_DYNAMIC_class-nativesdk += "^nativesdk-perl-module-.*"
+PACKAGES_DYNAMIC_class-native_forcevariable = "^perl-module-.*-native$"
+PACKAGES_DYNAMIC_class-target = "^perl-module-.*(?<!native)$"
+PACKAGES_DYNAMIC_class-nativesdk = "^nativesdk-perl-module-.*"
 
 RDEPENDS_${PN}-misc += "perl perl-modules"
 RDEPENDS_${PN}-pod += "perl"