From patchwork Thu Sep 6 11:15:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 146075 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp371947ljw; Thu, 6 Sep 2018 04:15:30 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbAkELbcTK4UB/gUc2NJofTMvn6xgrmmP+P0Siu5F4wNKUQ7NQsBeXtggSpYmCoqWE2OKn/ X-Received: by 2002:a17:902:934a:: with SMTP id g10-v6mr2099638plp.121.1536232530567; Thu, 06 Sep 2018 04:15:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536232530; cv=none; d=google.com; s=arc-20160816; b=fk1bHUCh7jRNqsXE8Je530bRWzrNzv8ttdlvveBD8so7ZEu19iYRImFonM1zKa0w8R YyN6CnePOZvl/izn44aJKuyOiVkwixJbJWpRopdnpdKA4UQ+TPcubZ0gRZ/UMq7fICtb 0xzf0yXpIJW4z7bRAEkxDX5y3+aq7gNCm2dvqWEqZQinNsLuw8YgMyjnJod2uMCd38WW n5GR91y1ntrH4yGQijJBzy95osAY1UhFeGu1ifpDc+E/4th9Srx6eiRXm22JDZfUKhxb IE5w8GeDMmjW+GG9EHSm8nYwVtlBVUhRSP/WU6JxQH2pTXtbl1UcXIqKinXoywU2imsY EuDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to; bh=XzRW28NDtzWrSIX+J70aQ2yf2L5fxSACniy1iVRV0jY=; b=vWO5sdSXyRlLVInbTqiWGJvmzDbWSStxmq9y8dfJI62toZlD4qU5n5KVYvL1J6JFaH y1xZeXetQAyIGezOoYSVQDsR8EuQu4bUfgmHh494Iho37pJ0W0P6anXjzy1Pql/kSkuL D2psK2bj6K93lJbDCTY2UENRVqAGa8ANPIf7aQIVZLaasRSQO26TM7AfUyINkCTEzQT4 BnVshDMzHqML/zgHYesu27knxs0mWMKydl8FBOEWmrFwkMx5XcSikK9TQOts4o3dq2tK GAw8axk+qVEyDurfHhGXEV/bq//7sYbru9EoTZzue3OVUY7gN+sZ7cFlcuEIT3OuFtsn elIQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=FL0MATUc; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id b7-v6si5135769pfj.245.2018.09.06.04.15.30; Thu, 06 Sep 2018 04:15:30 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=FL0MATUc; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from layers.openembedded.org (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id 424BE7965B; Thu, 6 Sep 2018 11:15:27 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wm0-f65.google.com (mail-wm0-f65.google.com [74.125.82.65]) by mail.openembedded.org (Postfix) with ESMTP id 9BB5D7964D for ; Thu, 6 Sep 2018 11:15:25 +0000 (UTC) Received: by mail-wm0-f65.google.com with SMTP id r1-v6so14107924wmh.0 for ; Thu, 06 Sep 2018 04:15:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id; bh=M6d6AxbpsPG212mwO6vC6M4bg194Ke9Mi63JvScSDZ4=; b=FL0MATUcbk9E+FPc5ePMcIZG9n7Lad0zniYKTBvHOd8+fWbZXPdiO6ZIBfsHNrDA9G iJ43J7eY/YvRvUR4+kQTJUAuoB9QUkuzia0P/dPIUWBNaRSx36maRUTcXfDlnkEuDxzR DRhM2Uh6Vi4LUotghJqmc/rI3D7IzQpNyXwsggF5PMExtlbFtUKFWqoTDdgWG+JJEkib BfA7z4KNO1HE7FKLXT+LGCSwkM4G0GxChiyXS57xmU8rWPv/FmFWZbtGuS2QDOGIakwC tIc/ijF3wR8+6dFF73obZr7upglil6IVX1ul/JPH+LP2ncKPryIRrleWjxx+F2XgJR4i rcxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=M6d6AxbpsPG212mwO6vC6M4bg194Ke9Mi63JvScSDZ4=; b=PHlBeRvKoFll42FQwWHsY44H8624ctzyX7vT4dYGnLomK2htmCkCed+GxemOB1AMWS iIlkwJiKc5pISOmvUTC6KkXBD1Xzf4BMAyxIeac76rpmy1euodYE+iIXYHOleBoX+ymS TDAsYVDWaz8AeSBTCFJLLOoHcLLhB7F6RkyOWXW3/aHiC/PoSCtyFheXRRZ8VbGnQwL1 1J1uL5oAIJdbn7GlytO/TUwfs47nNPx0J6lPvh1C2F/pM+23X2A/7S9m6YpVrLYmWqoi sG0Xz0UAphiEKkmG6Bpe+agc9VRIp2+1vdMh/nrdxTUYEQPN9FXC0S1k/d5D3NYn6NHf ck0w== X-Gm-Message-State: APzg51CQsoe/A0RGhUZgBKwtylsFuIKSZ83S5aMBGxlAZQ+DoH2pFKvx 39+atdb5+3y49BKiGdzHECNc/epW+mU= X-Received: by 2002:a1c:2108:: with SMTP id h8-v6mr1745246wmh.108.1536232525863; Thu, 06 Sep 2018 04:15:25 -0700 (PDT) Received: from flashheart.burtonini.com (35.106.2.81.in-addr.arpa. [81.2.106.35]) by smtp.gmail.com with ESMTPSA id f6-v6sm5519173wrr.68.2018.09.06.04.15.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Sep 2018 04:15:25 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Date: Thu, 6 Sep 2018 12:15:17 +0100 Message-Id: <20180906111521.20997-1-ross.burton@intel.com> X-Mailer: git-send-email 2.11.0 Subject: [OE-core] [PATCH 1/5] python3: generalise make commands X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org We're repeating the same make invocation over and over, twice without setting OPT=${CFLAGS} which doesn't seem right. Centralise the make invocation to clean up the tasks. Signed-off-by: Ross Burton --- meta/recipes-devtools/python/python3_3.5.5.bb | 48 +++++++++------------------ 1 file changed, 16 insertions(+), 32 deletions(-) -- 2.11.0 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core diff --git a/meta/recipes-devtools/python/python3_3.5.5.bb b/meta/recipes-devtools/python/python3_3.5.5.bb index 9e31fa6f299..8e94e38a2d5 100644 --- a/meta/recipes-devtools/python/python3_3.5.5.bb +++ b/meta/recipes-devtools/python/python3_3.5.5.bb @@ -95,6 +95,18 @@ do_configure_append() { autoreconf -Wcross --verbose --install --force --exclude=autopoint ../Python-${PV}/Modules/_ctypes/libffi } +run_make() { + oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \ + HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python3-native/python3 \ + STAGING_LIBDIR=${STAGING_LIBDIR} \ + STAGING_INCDIR=${STAGING_INCDIR} \ + STAGING_BASELIBDIR=${STAGING_BASELIBDIR} \ + LIB=${baselib} \ + ARCH=${TARGET_ARCH} \ + OPT="${CFLAGS}" \ + "$@" +} + do_compile() { # regenerate platform specific files, because they depend on system headers cd ${S}/Lib/plat-linux* @@ -125,14 +137,7 @@ do_compile() { # then call do_install twice we get Makefile.orig == Makefile.sysroot install -m 0644 Makefile Makefile.sysroot - oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \ - HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python3-native/python3 \ - STAGING_LIBDIR=${STAGING_LIBDIR} \ - STAGING_INCDIR=${STAGING_INCDIR} \ - STAGING_BASELIBDIR=${STAGING_BASELIBDIR} \ - LIB=${baselib} \ - ARCH=${TARGET_ARCH} \ - OPT="${CFLAGS}" profile-opt + run_make profile-opt if ${@bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'true', 'false', d)}; then qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_TARGET}', ['${B}', '${STAGING_DIR_TARGET}/${base_libdir}'])}" @@ -145,14 +150,7 @@ EOF ./pgo-image-qemuwrapper ${B}/python ${PYTHON3_PROFILE_TASK} || true fi - oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \ - HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python3-native/python3 \ - STAGING_LIBDIR=${STAGING_LIBDIR} \ - STAGING_INCDIR=${STAGING_INCDIR} \ - STAGING_BASELIBDIR=${STAGING_BASELIBDIR} \ - LIB=${baselib} \ - ARCH=${TARGET_ARCH} \ - OPT="${CFLAGS}" clean_and_use_profile + run_make clean_and_use_profile } do_install() { @@ -165,23 +163,9 @@ do_install() { # rerun the build once again with original makefile this time # run install in a separate step to avoid compile/install race - oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \ - HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python3-native/python3 \ - STAGING_LIBDIR=${STAGING_LIBDIR} \ - STAGING_INCDIR=${STAGING_INCDIR} \ - STAGING_BASELIBDIR=${STAGING_BASELIBDIR} \ - LIB=${baselib} \ - ARCH=${TARGET_ARCH} \ - DESTDIR=${D} LIBDIR=${libdir} build_all_use_profile + run_make DESTDIR=${D} LIBDIR=${libdir} build_all_use_profile - oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \ - HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python3-native/python3 \ - STAGING_LIBDIR=${STAGING_LIBDIR} \ - STAGING_INCDIR=${STAGING_INCDIR} \ - STAGING_BASELIBDIR=${STAGING_BASELIBDIR} \ - LIB=${baselib} \ - ARCH=${TARGET_ARCH} \ - DESTDIR=${D} LIBDIR=${libdir} install + run_make DESTDIR=${D} LIBDIR=${libdir} install # avoid conflict with 2to3 from Python 2 rm -f ${D}/${bindir}/2to3 From patchwork Thu Sep 6 11:15:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 146076 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp372130ljw; Thu, 6 Sep 2018 04:15:41 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbuysL7EDR5HBHCPy/RntzpngdOF0MmU5O3QLMokFMct+q4a30OB9imv5kmd6mjl7eYHw/i X-Received: by 2002:a63:e914:: with SMTP id i20-v6mr2218251pgh.10.1536232541177; Thu, 06 Sep 2018 04:15:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536232541; cv=none; d=google.com; s=arc-20160816; b=ZMhLcDhuuFPWr5y+NS42YWeK0zi+cHnrxM6fE7ZS2xeEi8D5714XLw2ya9mOdHEF5j D1GJPKdS4F80gLSgzUOgRTOLJiANgfYhHQbttCqLAI0VviqQv7w+zCiPHQwcQZef0DcU Sq7EPagdEha+8OTXl/PWobZ4XTUbVflTcUjXTiFrgCmWIUdfdmLAd4m06qHVppGdpE8B BBSTrWLR5+I0FlsO/GiWBPZ0M2gN78GljIRntjbjcijkEKo0PcIFqlw5+DlsLUNMVa3q hTvE+Av3gcvR9H4QRx9DJIgDvlIAJp34R8Jf5w/cQPbUWk6oYN8t4QF3kQCldCdbojAu F2ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=32f+2VMCLyBBF3mQunSV59DjbzmH8aYiqKtzSz0NBVc=; b=MkSTM5uSTF+OhSKc3GzJPUfPvXsKjdrqb6ACOF7dxT+0tmIoisYtu/I88J48XlZp1A NxoeNg8zZxyQlnUheWgrcz5P7bubS0P47KnmA/AFPgWzM0v1Oi8JDdeWz91iuPRS6BIt cX5yKQNaVEQ8IHAUllWHqcd2sIWF3nW5qMf5G6057Tgxyy37jb3dz8Y6MXWUsef5eA8R RjzpqYeSts80VWJvELZBmjxGliE6q3B1IDKq1UoFQ4ooqCiKy8nMUNFnpB1tfX/z3BnO YpQ0X9VsEloIP59Z/bg3irA3N2dgHibd570bCA+sHq7Fco60Ddusj9bAT88m9C7nfYjn FLnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=wh9sbqGE; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id z5-v6si4256606plo.130.2018.09.06.04.15.40; Thu, 06 Sep 2018 04:15:41 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=wh9sbqGE; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from layers.openembedded.org (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id 01CB979662; Thu, 6 Sep 2018 11:15:38 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68]) by mail.openembedded.org (Postfix) with ESMTP id B18957964D for ; Thu, 6 Sep 2018 11:15:26 +0000 (UTC) Received: by mail-wm0-f68.google.com with SMTP id j192-v6so10869634wmj.1 for ; Thu, 06 Sep 2018 04:15:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=0i0DK+ixPmgmsBKhd1OI3gyTn3uhohl1ohgQ5CS7oog=; b=wh9sbqGEka7lzZNKEGwWuQ6VMJxB/9SfBwcDqnck9foMnFduZZ8dNrCspD/UGUJpfb F8n4NAxN8wfJipwLO9gNl1wrtDliONSbWZTAvyM9oCs34IqlsagHRdW7zE6bNfi0zBvq uxJ8l4rnYB9UTW0NCbSQB/yCjeygVCFoaRj3JbFORuY6NquCXekB/sDRjI4lMfn2N+pm N95ghOQYniyZb+fahZuv8ini1InZwmn/bNOyOQ21Xy7n241ryqoxNBCaaFFnPiSlYZ1h +SVLTI6+UH2txW6Wu3TXOokOh6G4FYVHWDaNqNlg6/P0g82OYyAlpEOh8hdp9A4a5sr/ K1qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=0i0DK+ixPmgmsBKhd1OI3gyTn3uhohl1ohgQ5CS7oog=; b=NxdbjtAe8ZpZqmI2oIYN9D04y92IivUBWdLj3XhIRKheo7fknXBotKEOfX9NJ2KfZj h8nDiZA8I2f0IQJ628UpxCwJ7wQP7uuAHu4isGKNT5NApQd0+dwKO1wu+QHq5dBPcODA 8j7bI7jCfrWUe3/ANyQoPybGtgdEh8YWHO3aesOs30tTbKqjngrLvBDRLwfB0kEanr2B VdZU3KfIeiCyv+CCp1U9uO8v2MiVxCNeIojR86GokT07WuKvl3HuyQpgHcjprylbPlQo 1dMtcV4a0wWzINnmMd/5eZ1J9YI1ln4MbnaKsBwyPonhUtgad2D1RBi+oRxXZWSgxlUb f8gQ== X-Gm-Message-State: APzg51BsYwL/tLLfJAHQ+EOeQ4Vyo+HLg9vZoAUKK801kPbDnRjGXw7b rf4oO4937JnfemXyNbfgYlS8FZRc0qs= X-Received: by 2002:a1c:4182:: with SMTP id o124-v6mr1778063wma.101.1536232526909; Thu, 06 Sep 2018 04:15:26 -0700 (PDT) Received: from flashheart.burtonini.com (35.106.2.81.in-addr.arpa. [81.2.106.35]) by smtp.gmail.com with ESMTPSA id f6-v6sm5519173wrr.68.2018.09.06.04.15.25 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Sep 2018 04:15:26 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Date: Thu, 6 Sep 2018 12:15:18 +0100 Message-Id: <20180906111521.20997-2-ross.burton@intel.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180906111521.20997-1-ross.burton@intel.com> References: <20180906111521.20997-1-ross.burton@intel.com> Subject: [OE-core] [PATCH 2/5] python3: add toggle for profile-guided-optimisation X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org Add a PACKAGECONFIG for profile-guided-optimisation, and default to whether qemu-usermode is available. Move --enable-optimizations to the pgo optimisation as all this currently does is use the PGO rules, causing a performance hit if PGO isn't actually used. Signed-off-by: Ross Burton --- meta/recipes-devtools/python/python3_3.5.5.bb | 54 +++++++++++++++------------ 1 file changed, 31 insertions(+), 23 deletions(-) -- 2.11.0 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core diff --git a/meta/recipes-devtools/python/python3_3.5.5.bb b/meta/recipes-devtools/python/python3_3.5.5.bb index 8e94e38a2d5..aad2e3abe60 100644 --- a/meta/recipes-devtools/python/python3_3.5.5.bb +++ b/meta/recipes-devtools/python/python3_3.5.5.bb @@ -73,7 +73,8 @@ CACHED_CONFIGUREVARS = "ac_cv_have_chflags=no \ TARGET_CC_ARCH += "-DNDEBUG -fno-inline" SDK_CC_ARCH += "-DNDEBUG -fno-inline" EXTRA_OEMAKE += "CROSS_COMPILE=yes" -EXTRA_OECONF += "CROSSPYTHONPATH=${STAGING_LIBDIR_NATIVE}/python${PYTHON_MAJMIN}/lib-dynload/ --without-ensurepip --enable-optimizations" +EXTRA_OECONF += "CROSSPYTHONPATH=${STAGING_LIBDIR_NATIVE}/python${PYTHON_MAJMIN}/lib-dynload/ --without-ensurepip" + PYTHON3_PROFILE_TASK ?= "${S}/Tools/pybench/pybench.py -n 1" export CROSS_COMPILE = "${TARGET_PREFIX}" @@ -87,8 +88,10 @@ export CROSSPYTHONPATH = "${STAGING_LIBDIR_NATIVE}/python${PYTHON_MAJMIN}/lib-dy # No ctypes option for python 3 PYTHONLSBOPTS = "" -PACKAGECONFIG ??= "readline" +PACKAGECONFIG ??= "readline ${@bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'pgo', '', d)}" PACKAGECONFIG[readline] = ",,readline" +# Use profile guided optimisation by running PyBench inside qemu-user +PACKAGECONFIG[pgo] = "--enable-optimizations" do_configure_append() { rm -f ${S}/Makefile.orig @@ -108,16 +111,16 @@ run_make() { } do_compile() { - # regenerate platform specific files, because they depend on system headers - cd ${S}/Lib/plat-linux* - include=${STAGING_INCDIR} ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ - ${S}/Tools/scripts/h2py.py -i '(u_long)' \ - ${STAGING_INCDIR}/dlfcn.h \ - ${STAGING_INCDIR}/linux/cdrom.h \ - ${STAGING_INCDIR}/netinet/in.h \ - ${STAGING_INCDIR}/sys/types.h - sed -e 's,${STAGING_DIR_HOST},,g' -i *.py - cd - + # regenerate platform specific files, because they depend on system headers + cd ${S}/Lib/plat-linux* + include=${STAGING_INCDIR} ${STAGING_BINDIR_NATIVE}/python3-native/python3 \ + ${S}/Tools/scripts/h2py.py -i '(u_long)' \ + ${STAGING_INCDIR}/dlfcn.h \ + ${STAGING_INCDIR}/linux/cdrom.h \ + ${STAGING_INCDIR}/netinet/in.h \ + ${STAGING_INCDIR}/sys/types.h + sed -e 's,${STAGING_DIR_HOST},,g' -i *.py + cd - # remove any bogus LD_LIBRARY_PATH sed -i -e s,RUNSHARED=.*,RUNSHARED=, Makefile @@ -137,20 +140,21 @@ do_compile() { # then call do_install twice we get Makefile.orig == Makefile.sysroot install -m 0644 Makefile Makefile.sysroot - run_make profile-opt - - if ${@bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'true', 'false', d)}; then - qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_TARGET}', ['${B}', '${STAGING_DIR_TARGET}/${base_libdir}'])}" - cat > pgo-image-qemuwrapper << EOF + if ${@bb.utils.contains('PACKAGECONFIG', 'pgo', 'true', 'false', d)}; then + run_make profile-opt + qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_TARGET}', ['${B}', '${STAGING_DIR_TARGET}/${base_libdir}'])}" + cat > pgo-image-qemuwrapper << EOF #!/bin/sh set -x $qemu_binary "\$@" EOF - chmod +x pgo-image-qemuwrapper - ./pgo-image-qemuwrapper ${B}/python ${PYTHON3_PROFILE_TASK} || true + chmod +x pgo-image-qemuwrapper + ./pgo-image-qemuwrapper ${B}/python ${PYTHON3_PROFILE_TASK} || true + run_make clean_and_use_profile + else + run_make libpython3.so + run_make fi - - run_make clean_and_use_profile } do_install() { @@ -163,8 +167,12 @@ do_install() { # rerun the build once again with original makefile this time # run install in a separate step to avoid compile/install race - run_make DESTDIR=${D} LIBDIR=${libdir} build_all_use_profile - + if ${@bb.utils.contains('PACKAGECONFIG', 'pgo', 'true', 'false', d)}; then + run_make DESTDIR=${D} LIBDIR=${libdir} build_all_use_profile + else + run_make DESTDIR=${D} LIBDIR=${libdir} + fi + run_make DESTDIR=${D} LIBDIR=${libdir} install # avoid conflict with 2to3 from Python 2 From patchwork Thu Sep 6 11:15:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 146077 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp372241ljw; Thu, 6 Sep 2018 04:15:46 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbHmvAJfo2lq/dNrOVWwjnj7Uvqdccqo9z+epqrqpNr+97HArdePw8/QDLoVh7XzyHLBHF7 X-Received: by 2002:a17:902:4a:: with SMTP id 68-v6mr2087519pla.276.1536232546770; Thu, 06 Sep 2018 04:15:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536232546; cv=none; d=google.com; s=arc-20160816; b=y34m4JHRjOT9YUbAKuVlxl7UAqalUoR+XEtmxiPI3m66e9HopsWhgxBsfT2clP//tR Sj1/w73SoR0POCuK/+dODBxX/XfoTKMikoHsNOZMhCkA31ESYP/++I6AY/XFrdfk0iy0 cBU2ful9VFCr8jsO5Spg3bKMa7rPXNTt58GtGn2gNqbl+I74NDfl9G+YoYStVhLyn4jx 2lF1iTf+uHZ3H4MczWdjgNIse8MWmpRra9XD0U3qctCm8ujhgsQ1aqCz+z6B45gmR6Lk D0bNGbWCK/RnVcCkdTKJUofGWBIA1wLUjPGD1mLgtaF+7ISe0/8OiXgTaIVzOSnPHdax T4rQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=iLeWRutRyKV+CfhBRGzf9pJJ7S9pBV+Js3WlrKjl1os=; b=MNUYNykDSLpBEX/FcbAcOWxEqxwuoJj3kDc5JOiPiGGSbCOXaO033Onbzl6nrx+Ldn RLWrOllL3mWd2BNwc3kdtkkQYGv/PunxA3aunEo0kIuHq+wi573FJj+DJxvWAPQHxzfK OV69un52MfnTLiRI6GU1cFoBru4R8w2QANt5yOJm8c+59oJSQ9F8TF/QpF/ZHdD3Tmyg 5Hh8OaFiZKZ7oi1lUFlnEpJTihnD1jYFYV0jx82ks96Y2E6G4cPmTtsbnHkONRnVZJAP 4m2bOsrjeBNvNxrSk5PwvchsppvhYyLrENs/aStW9AojIQzra6eWFLJ9DC4TuAW4esV0 DrWg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=OIIfEDjW; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id w16-v6si5059853pgj.61.2018.09.06.04.15.46; Thu, 06 Sep 2018 04:15:46 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=OIIfEDjW; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from layers.openembedded.org (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id 380127966A; Thu, 6 Sep 2018 11:15:39 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by mail.openembedded.org (Postfix) with ESMTP id E0C95795CF for ; Thu, 6 Sep 2018 11:15:27 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id j26-v6so10979179wre.2 for ; Thu, 06 Sep 2018 04:15:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=zmCy05G6bh9EjY4CzZGNN8rwPN/nTN+kZ5rmBpaiJoM=; b=OIIfEDjWZyar+sOACsadX34HOtq0Dxx4UxG3bUte265/UekwfhW6K5qrqoe4yGhqer ii0ulvDXAHbrNGElp9oAVM42redt0e9wJIwkacq0T7qsbiF6JAHqBAdB2I8wJOGkKN19 96+g4FmQgzbNmBRqm9lXyFdD6+6QlOvIRXLc5XLlMG8D9QfTz6ZScP/DkaQhH0gaVBzx PaxVRQsU+sOAuJISe+LXtJ9RJcgVn2IQcMvJQ2n0tfy4Ea4h68lIYWDDeXYv9myP4b+8 NlAquCrUhWHRETHt/iFWKLhx4fJ7MxjmbtA55Y9jFGPCYPQiEXA9LBn+3Q4Jy8kmzXyA RViw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=zmCy05G6bh9EjY4CzZGNN8rwPN/nTN+kZ5rmBpaiJoM=; b=A3zdcMzM044Jfxu9zJMjLvclVHdZ20q/m9uTTPdurR4aSQyXoKEDF8wH0QMoK6IEiC YejThGx1O5F9cqh/svZTURAUkPZKE7JfUeHRUIaHbvID0jODTVbo9q2Zj3BHHj90vkJo J9UCbqeaFxGNbQGHms0ifToOfnXcfhjWAolRsXf5cImIjBIXLv0HImk5IjJCHyf7A89Y EQmaiHbCOjbYYrEeLTHtEvX3HucYbVTgh+YpLjehJ77vaQmlEsPEMyLc4UVV7UTz1ik/ j/EatVbkQbE0RJA0it2fjDELzZkO9RcE90OVxBTcTbpNNh/BvGneNxULRRPyUDozlK7L E/Tw== X-Gm-Message-State: APzg51CpVnFSE1Q8c8Xo1E4wpA0tkNY6r3QBN+TpCmHBOG/pnVIEAlPD z/IBjrtrBx4XNKmN4WmhIw4JA8yuVP8= X-Received: by 2002:adf:9051:: with SMTP id h75-v6mr2067206wrh.65.1536232528156; Thu, 06 Sep 2018 04:15:28 -0700 (PDT) Received: from flashheart.burtonini.com (35.106.2.81.in-addr.arpa. [81.2.106.35]) by smtp.gmail.com with ESMTPSA id f6-v6sm5519173wrr.68.2018.09.06.04.15.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Sep 2018 04:15:27 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Date: Thu, 6 Sep 2018 12:15:19 +0100 Message-Id: <20180906111521.20997-3-ross.burton@intel.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180906111521.20997-1-ross.burton@intel.com> References: <20180906111521.20997-1-ross.burton@intel.com> Subject: [OE-core] [PATCH 3/5] python3: trim PGO patch X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org There's no need to delete the line that removes the profile data, as we're not using it after the build. This reduces the size of the patch, making it easier to maintain. Signed-off-by: Ross Burton --- .../0002-Makefile-add-target-to-split-profile-generation.patch | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 2.11.0 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core diff --git a/meta/recipes-devtools/python/python3/0002-Makefile-add-target-to-split-profile-generation.patch b/meta/recipes-devtools/python/python3/0002-Makefile-add-target-to-split-profile-generation.patch index bb01c5bcb46..2b4ba316e45 100644 --- a/meta/recipes-devtools/python/python3/0002-Makefile-add-target-to-split-profile-generation.patch +++ b/meta/recipes-devtools/python/python3/0002-Makefile-add-target-to-split-profile-generation.patch @@ -19,7 +19,7 @@ diff --git a/Makefile.pre.in b/Makefile.pre.in index 84bc3ff..017a2c4 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in -@@ -469,13 +469,11 @@ profile-opt: +@@ -469,13 +469,12 @@ profile-opt: $(MAKE) profile-removal $(MAKE) build_all_generate_profile $(MAKE) profile-removal @@ -31,7 +31,7 @@ index 84bc3ff..017a2c4 100644 @echo "Rebuilding with profile guided optimizations:" $(MAKE) clean $(MAKE) build_all_use_profile -- $(MAKE) profile-removal + $(MAKE) profile-removal build_all_generate_profile: $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS) $(EXTRA_CFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LIBS="$(LIBS)" From patchwork Thu Sep 6 11:15:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 146078 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp372354ljw; Thu, 6 Sep 2018 04:15:53 -0700 (PDT) X-Google-Smtp-Source: ANB0VdaGh1qGFIGJ3uZGQkZ4ctoSdxlMkELIcPKW3oto1ob5Z0VoMQm5SLg8mlqzQXmLexzCNwh1 X-Received: by 2002:a63:b207:: with SMTP id x7-v6mr2191911pge.401.1536232552922; Thu, 06 Sep 2018 04:15:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536232552; cv=none; d=google.com; s=arc-20160816; b=olbQyJkCNDHOpuZNBhJCqc5JGVka66cFqHlvTxlDd6r76MIqhJaimsPRVF5eHMNHmx +I7PdrDR1+aQpQAOzGaZVapwL1JkDn5MIbQ5XOLGjpWC3S0uWFSwS4Bchbw4nMS+qEg6 lshmYdrHMUWzCeBofNbel+J0cOAheBAp811n7o/PDFqEBL8EZLcH1tWsf+bUdhwzcVEh z37HhPdFMQvpGTd1maSDB1rs3zwukG3tSvmHC8+FxtsrzTur7UhBpz6iv2nEQ1MtFeT/ wICUTcPxHkfjTxYpB3qDY1OseSxGws/1ZlAjbu4MnR3BcyuIKIED+W1qZjTbeuupHUyp Un1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=oY4tUQ8x/hw7l/nu6ZfzV2lAPDeTL1MCtvoAn74sGFY=; b=uEEEB2ea9hzn5Oq4mpunuAA/HfDWw22/um+JhQLHrgBslk6kTfyqlNQ72rXbUYk8QT U/L4Lmf4o1md6yfm8WgYb0TySpxvtoRmmcnsAzP17BdI8kwCQv4RywqZDvIf2taC2B2o byWyMtXCud/QJT0tX/dIkVeAFqaNR7fjmOcl47KmXqNntUnE3SVOwUX3kGEtGOSen0k8 JjRn+GIW5qZwBitkKbmZAKJsPpfF3OAFtqxUy3lMxPICl3rfw/c2yh3J7YwnkUYwhuA+ rhYnqg4ROyDiuVca2Sonh5mYY9JsInAkFPkOuWWxzpShS7qi563DkRPSOgbhbhzAQYcW VECw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=OymI5m7i; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id z1-v6si4304871pln.63.2018.09.06.04.15.52; Thu, 06 Sep 2018 04:15:52 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=OymI5m7i; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from layers.openembedded.org (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id 81FFA7966D; Thu, 6 Sep 2018 11:15:40 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by mail.openembedded.org (Postfix) with ESMTP id 5CAD879660 for ; Thu, 6 Sep 2018 11:15:29 +0000 (UTC) Received: by mail-wm0-f67.google.com with SMTP id c14-v6so11005056wmb.4 for ; Thu, 06 Sep 2018 04:15:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=nfSHl7EkOiGBL7G/xExpApOJ+BXqpLg3dZL8oV825gI=; b=OymI5m7ijIrE6a2rDW+mD6puaYrQNVAEqqLhQpyzvzPS81TRgy86Tgch3AUk3K/7SV g7Awa8Gg5HmtR0qTbR63WzjYhpWDds7q9J1MfmrGC0o3xJQhD+3CmjE+bHX3bYMsH2WZ FALbm2X0spdLTw1tdzDKu+K40gb6kMiUAXPNrZ6kBpna0b9SJ7rhqHw1l+DHeoWxC3kH Xu8Nkt8XoZRwP4a1GoukNNIdP0IOvOexayQGENUdwhbefx3W+l3gxjOQZSq5YRZs0SuE zDp4S2s9wzRpWTdm9hnEIl8YNRXwnBKXGjLYR0g5KZ1M9QqELy54vQx5n/dL7LtEXWw3 99Gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=nfSHl7EkOiGBL7G/xExpApOJ+BXqpLg3dZL8oV825gI=; b=CWdyjuL31OOp6SJnH4/oxoC6XkUSPUupg66NGtxR4tFwb4e6coY1mvF2ad3YdBjlE3 rQYqnvdAV++LwsgG6bffMNDg8F1BjljhkoG4Dkfrpw2HXYHWjnXpgHOKzMf/f/doldL7 HQ8sNQSLQTN4QnS1zaoGbk9enJMramwDX+cwkZMP6Se9DdlkC38NEN8ElKOIBMM1eISH FBAzd81PmRcsO1cz286Fx+OqtRov1Pv+FnfFPpa8bVIaAFPVvuH30X+NlnFEaBrlp4QQ Hot63kzibS/vV8T9ETsA+50K6tGWKD0Jmnkf5BMpP6qnr2q2AR9SlGRByx9rTBSTvlAf hZvA== X-Gm-Message-State: APzg51AFhLHU7CgGgWvqek80G9eWOcfJ4xlXq1AYi+HLUWADfHp0HqiQ jVUbD8uiScMEEErQlOokY9tXSMWEP9g= X-Received: by 2002:a1c:8952:: with SMTP id l79-v6mr1783352wmd.7.1536232529384; Thu, 06 Sep 2018 04:15:29 -0700 (PDT) Received: from flashheart.burtonini.com (35.106.2.81.in-addr.arpa. [81.2.106.35]) by smtp.gmail.com with ESMTPSA id f6-v6sm5519173wrr.68.2018.09.06.04.15.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Sep 2018 04:15:28 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Date: Thu, 6 Sep 2018 12:15:20 +0100 Message-Id: <20180906111521.20997-4-ross.burton@intel.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180906111521.20997-1-ross.burton@intel.com> References: <20180906111521.20997-1-ross.burton@intel.com> Subject: [OE-core] [PATCH 4/5] python3: don't use runtime checks to identify float endianism X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org Python uses AC_RUN_IFELSE to determine the byte order for floats and doubles, and falls back onto "I don't know" if it can't run code. This results in crippled floating point numbers in Python, and the regression tests fail. Instead of running code, take a macro from autoconf-archive which compiles C with a special double in which has an ASCII representation, and then greps the binary to identify the format. Signed-off-by: Ross Burton --- .../python/python3/float-endian.patch | 212 +++++++++++++++++++++ meta/recipes-devtools/python/python3_3.5.5.bb | 1 + 2 files changed, 213 insertions(+) create mode 100644 meta/recipes-devtools/python/python3/float-endian.patch -- 2.11.0 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core diff --git a/meta/recipes-devtools/python/python3/float-endian.patch b/meta/recipes-devtools/python/python3/float-endian.patch new file mode 100644 index 00000000000..6ba3f5c252d --- /dev/null +++ b/meta/recipes-devtools/python/python3/float-endian.patch @@ -0,0 +1,212 @@ +Python uses AC_RUN_IFELSE to determine the byte order for floats and doubles, +and falls back onto "I don't know" if it can't run code. This results in +crippled floating point numbers in Python, and the regression tests fail. + +Instead of running code, take a macro from autoconf-archive which compiles C +with a special double in which has an ASCII representation, and then greps the +binary to identify the format. + +Upstream-Status: Submitted [https://bugs.python.org/issue34585] +Signed-off-by: Ross Burton + +From 50df2a4c3a65ed06322be7c26d42b06ce81730c1 Mon Sep 17 00:00:00 2001 +From: Ross Burton +Date: Wed, 5 Sep 2018 11:45:52 +0100 +Subject: [PATCH] Don't do runtime test to get float byte order + +--- + configure.ac | 74 +++++------------------------------ + m4/ax_c_float_words_bigendian.m4 | 83 ++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 92 insertions(+), 65 deletions(-) + create mode 100644 m4/ax_c_float_words_bigendian.m4 + +diff --git a/configure.ac b/configure.ac +index c9b755f0f4..1215969871 100644 +--- a/configure.ac ++++ b/configure.ac +@@ -9,6 +9,8 @@ AC_PREREQ(2.65) + + AC_INIT(python, PYTHON_VERSION, https://bugs.python.org/) + ++AC_CONFIG_MACRO_DIR(m4) ++ + AC_SUBST(BASECPPFLAGS) + if test "$srcdir" != . -a "$srcdir" != "$(pwd)"; then + # If we're building out-of-tree, we need to make sure the following +@@ -4128,77 +4130,19 @@ fi + # * Check for various properties of floating point * + # ************************************************** + +-AC_MSG_CHECKING(whether C doubles are little-endian IEEE 754 binary64) +-AC_CACHE_VAL(ac_cv_little_endian_double, [ +-AC_RUN_IFELSE([AC_LANG_SOURCE([[ +-#include +-int main() { +- double x = 9006104071832581.0; +- if (memcmp(&x, "\x05\x04\x03\x02\x01\xff\x3f\x43", 8) == 0) +- return 0; +- else +- return 1; +-} +-]])], +-[ac_cv_little_endian_double=yes], +-[ac_cv_little_endian_double=no], +-[ac_cv_little_endian_double=no])]) +-AC_MSG_RESULT($ac_cv_little_endian_double) +-if test "$ac_cv_little_endian_double" = yes +-then +- AC_DEFINE(DOUBLE_IS_LITTLE_ENDIAN_IEEE754, 1, +- [Define if C doubles are 64-bit IEEE 754 binary format, stored +- with the least significant byte first]) +-fi +- +-AC_MSG_CHECKING(whether C doubles are big-endian IEEE 754 binary64) +-AC_CACHE_VAL(ac_cv_big_endian_double, [ +-AC_RUN_IFELSE([AC_LANG_SOURCE([[ +-#include +-int main() { +- double x = 9006104071832581.0; +- if (memcmp(&x, "\x43\x3f\xff\x01\x02\x03\x04\x05", 8) == 0) +- return 0; +- else +- return 1; +-} +-]])], +-[ac_cv_big_endian_double=yes], +-[ac_cv_big_endian_double=no], +-[ac_cv_big_endian_double=no])]) +-AC_MSG_RESULT($ac_cv_big_endian_double) +-if test "$ac_cv_big_endian_double" = yes ++AX_C_FLOAT_WORDS_BIGENDIAN ++if test "$ax_cv_c_float_words_bigendian" = "yes" + then + AC_DEFINE(DOUBLE_IS_BIG_ENDIAN_IEEE754, 1, + [Define if C doubles are 64-bit IEEE 754 binary format, stored + with the most significant byte first]) +-fi +- +-# Some ARM platforms use a mixed-endian representation for doubles. +-# While Python doesn't currently have full support for these platforms +-# (see e.g., issue 1762561), we can at least make sure that float <-> string +-# conversions work. +-AC_MSG_CHECKING(whether C doubles are ARM mixed-endian IEEE 754 binary64) +-AC_CACHE_VAL(ac_cv_mixed_endian_double, [ +-AC_RUN_IFELSE([AC_LANG_SOURCE([[ +-#include +-int main() { +- double x = 9006104071832581.0; +- if (memcmp(&x, "\x01\xff\x3f\x43\x05\x04\x03\x02", 8) == 0) +- return 0; +- else +- return 1; +-} +-]])], +-[ac_cv_mixed_endian_double=yes], +-[ac_cv_mixed_endian_double=no], +-[ac_cv_mixed_endian_double=no])]) +-AC_MSG_RESULT($ac_cv_mixed_endian_double) +-if test "$ac_cv_mixed_endian_double" = yes ++elif test "$ax_cv_c_float_words_bigendian" = "no" + then +- AC_DEFINE(DOUBLE_IS_ARM_MIXED_ENDIAN_IEEE754, 1, ++ AC_DEFINE(DOUBLE_IS_LITTLE_ENDIAN_IEEE754, 1, + [Define if C doubles are 64-bit IEEE 754 binary format, stored +- in ARM mixed-endian order (byte order 45670123)]) ++ with the least significant byte first]) ++else ++ AC_MSG_ERROR([Cannot identify floating point byte order]) + fi + + # The short float repr introduced in Python 3.1 requires the +diff --git a/m4/ax_c_float_words_bigendian.m4 b/m4/ax_c_float_words_bigendian.m4 +new file mode 100644 +index 0000000000..216b90d803 +--- /dev/null ++++ b/m4/ax_c_float_words_bigendian.m4 +@@ -0,0 +1,83 @@ ++# =============================================================================== ++# https://www.gnu.org/software/autoconf-archive/ax_c_float_words_bigendian.html ++# =============================================================================== ++# ++# SYNOPSIS ++# ++# AX_C_FLOAT_WORDS_BIGENDIAN([ACTION-IF-TRUE], [ACTION-IF-FALSE], [ACTION-IF-UNKNOWN]) ++# ++# DESCRIPTION ++# ++# Checks the ordering of words within a multi-word float. This check is ++# necessary because on some systems (e.g. certain ARM systems), the float ++# word ordering can be different from the byte ordering. In a multi-word ++# float context, "big-endian" implies that the word containing the sign ++# bit is found in the memory location with the lowest address. This ++# implementation was inspired by the AC_C_BIGENDIAN macro in autoconf. ++# ++# The endianness is detected by first compiling C code that contains a ++# special double float value, then grepping the resulting object file for ++# certain strings of ASCII values. The double is specially crafted to have ++# a binary representation that corresponds with a simple string. In this ++# implementation, the string "noonsees" was selected because the ++# individual word values ("noon" and "sees") are palindromes, thus making ++# this test byte-order agnostic. If grep finds the string "noonsees" in ++# the object file, the target platform stores float words in big-endian ++# order. If grep finds "seesnoon", float words are in little-endian order. ++# If neither value is found, the user is instructed to specify the ++# ordering. ++# ++# LICENSE ++# ++# Copyright (c) 2008 Daniel Amelang ++# ++# Copying and distribution of this file, with or without modification, are ++# permitted in any medium without royalty provided the copyright notice ++# and this notice are preserved. This file is offered as-is, without any ++# warranty. ++ ++#serial 11 ++ ++AC_DEFUN([AX_C_FLOAT_WORDS_BIGENDIAN], ++ [AC_CACHE_CHECK(whether float word ordering is bigendian, ++ ax_cv_c_float_words_bigendian, [ ++ ++ax_cv_c_float_words_bigendian=unknown ++AC_COMPILE_IFELSE([AC_LANG_SOURCE([[ ++ ++double d = 90904234967036810337470478905505011476211692735615632014797120844053488865816695273723469097858056257517020191247487429516932130503560650002327564517570778480236724525140520121371739201496540132640109977779420565776568942592.0; ++ ++]])], [ ++ ++if grep noonsees conftest.$ac_objext >/dev/null ; then ++ ax_cv_c_float_words_bigendian=yes ++fi ++if grep seesnoon conftest.$ac_objext >/dev/null ; then ++ if test "$ax_cv_c_float_words_bigendian" = unknown; then ++ ax_cv_c_float_words_bigendian=no ++ else ++ ax_cv_c_float_words_bigendian=unknown ++ fi ++fi ++ ++])]) ++ ++case $ax_cv_c_float_words_bigendian in ++ yes) ++ m4_default([$1], ++ [AC_DEFINE([FLOAT_WORDS_BIGENDIAN], 1, ++ [Define to 1 if your system stores words within floats ++ with the most significant word first])]) ;; ++ no) ++ $2 ;; ++ *) ++ m4_default([$3], ++ [AC_MSG_ERROR([ ++ ++Unknown float word ordering. You need to manually preset ++ax_cv_c_float_words_bigendian=no (or yes) according to your system. ++ ++ ])]) ;; ++esac ++ ++])# AX_C_FLOAT_WORDS_BIGENDIAN +-- +2.11.0 + diff --git a/meta/recipes-devtools/python/python3_3.5.5.bb b/meta/recipes-devtools/python/python3_3.5.5.bb index aad2e3abe60..b89e3b04c86 100644 --- a/meta/recipes-devtools/python/python3_3.5.5.bb +++ b/meta/recipes-devtools/python/python3_3.5.5.bb @@ -41,6 +41,7 @@ SRC_URI += "\ file://pass-missing-libraries-to-Extension-for-mul.patch \ file://Use-correct-CFLAGS-for-extensions-when-cross-compili.patch \ file://0002-Makefile-add-target-to-split-profile-generation.patch \ + file://float-endian.patch \ " SRC_URI[md5sum] = "f3763edf9824d5d3a15f5f646083b6e0" SRC_URI[sha256sum] = "063d2c3b0402d6191b90731e0f735c64830e7522348aeb7ed382a83165d45009" From patchwork Thu Sep 6 11:15:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 146079 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp372467ljw; Thu, 6 Sep 2018 04:15:59 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZjW8YlGpjJtzjKP3Y97rjSnMwZGQuvIaiTlvPECBXO5vrq2qcMyrOFWiOvlLqld33w8eIX X-Received: by 2002:a17:902:714f:: with SMTP id u15-v6mr2195974plm.154.1536232559743; Thu, 06 Sep 2018 04:15:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536232559; cv=none; d=google.com; s=arc-20160816; b=Vxt4Tv2A5br/1YE1WpNP8gB28awej1e4ddmbcewgQoo4JvxgF4HxhW5MkWqdLoc2bZ 63RTzHp/NzuIWBQWCCxGcWMAq9MITENbpZTDfCAkxZci08Xmu3RLVWwI95nwAizqNk/U m5Tlpm/OUaz12qm0PFop0FTK888CpJevxrsddonEyy2EGnQosb/fit/G5DobUDLA+4jJ 6O0ebDWQyE73ONsrj0Gf4f/vcxkMx46zYO2BETwb70OEvMtqKF/rhXemJRr3vMkVjy9X 7K+LcOPNc0WhmJ9i/ifRjFJfhgH1ccHhWT94kHVc6b6Y/oQJqNiTdsBl/tdWI+c85sBe 5Kzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:delivered-to; bh=RxAsQPyb3/7mAVL3AtCIIvUL0JePRf5iRgE6HRm15vQ=; b=k4IqYqPncABA07ClZUy9dd9C0h4GqmYnb7h1ZL35kOpNydstCeuy+GXfJ5r7LxSOhb Ra1cshMGCcfc20hxQ0HdqtiozdsGoYawTyTBb0lPuuhGCWQeGlUDwPSeTk2JkFRmnbYb L5mDvripbRQ3jzHhJHBI2B6yYf9DJ/THy4frYC7v4kw3C7Zhi3AK+QwkIlGXq5M85ele ll5Xv50UaOdJma/SQ7An87QUGYat86aj9YddaB5nXP9fK/ve+ZAO0+j+cbYXrzime+Zk JEYfMp+Hu3fCgjLSnuMDHrnKYlHf/KnVgkzCP/eH52y1laqvpPTeDxXSb+VJL1KGCWkw rOcg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b="A/6wUSNL"; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id j23-v6si4928827pgn.139.2018.09.06.04.15.59; Thu, 06 Sep 2018 04:15:59 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b="A/6wUSNL"; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from layers.openembedded.org (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id BA2EF79672; Thu, 6 Sep 2018 11:15:41 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by mail.openembedded.org (Postfix) with ESMTP id D18A079659 for ; Thu, 6 Sep 2018 11:15:29 +0000 (UTC) Received: by mail-wr1-f68.google.com with SMTP id u12-v6so10933649wrr.4 for ; Thu, 06 Sep 2018 04:15:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=CxweMLBiCDip5UQcvDozxBNYouu4gPlvzbBZcVFDaK4=; b=A/6wUSNLi2JnHSQm+5zakwFiK7gPN4fksLthMMgkpwte79wHH+Oxu6MGIi2xv9Su2q NzcElKoRXyQxs+Xpq+JCHkMRtWhA19pMaInJY728Y3HJbHN3W6OfxL2WFnJJ3UntPsi0 4u4j3A/r1J3tADUNUVpfAfENhe8sZlz6C4YCtGNXFgLd8YVnbLm+O24fKroPg9EeGcLq il8x6ASNB1EqU4guRFDljveAQmc4NwaG6S6pulAhNs/bercKWIi38Hh4McQeb+orwzYC RNzEdiVHKfp0HpRKOuN+R+jrcqmrmRPo7pvjv5GRbYRDEpYVUXzhB3UuIdRHDnIQXfV+ 9NsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CxweMLBiCDip5UQcvDozxBNYouu4gPlvzbBZcVFDaK4=; b=b61kSezxWuI/Wp4ZFMDOKf0DZiqWjszbR20fdjVfdoX9PLltob5fN4xEi9h4fZQG6A hLwN8CRpAU9pDJ5bFgv4/N0zlNMg7vXernaAsdBd9DoDQfoMSjfnkAwMQgudx8NOfV4Q M4XbDtK0zT8Sh/MnAumhAYQ3JF0c0wANiGUrNdlKXjx3bKk3SKdqY1KssKsyETn3GSvl j8S6vuqRdr12Bpn5488+vVpmqc+Y2vVYJdvKJvoqrpzEbADMKDuGYeOg3/KhiyOdOmwr f2NgYagBuv/Yune/ME6eApIBrSmrRFJB1cIpxhaHREvB15JHu/keVZJJsOBYAcdQ2iA8 Z5CQ== X-Gm-Message-State: APzg51AuQ/+Vc7rg9GEP50jSLrkHClSXr9+KPkKrbX+CFbuhBpy6NeFK FSXCAM7u8xvyTNJNMbbFR/i+2ZWnR+w= X-Received: by 2002:a5d:6604:: with SMTP id n4-v6mr1897563wru.281.1536232530314; Thu, 06 Sep 2018 04:15:30 -0700 (PDT) Received: from flashheart.burtonini.com (35.106.2.81.in-addr.arpa. [81.2.106.35]) by smtp.gmail.com with ESMTPSA id f6-v6sm5519173wrr.68.2018.09.06.04.15.29 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Sep 2018 04:15:29 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Date: Thu, 6 Sep 2018 12:15:21 +0100 Message-Id: <20180906111521.20997-5-ross.burton@intel.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180906111521.20997-1-ross.burton@intel.com> References: <20180906111521.20997-1-ross.burton@intel.com> MIME-Version: 1.0 Subject: [OE-core] [PATCH 5/5] python3: use regrtest instead of PyBench for profile-guided-optimisation X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org PyBench takes a long time to run, also upstream have removed it from Python and instead use test.regrtest —pgo to profile the interpreter. The results are good: not only does Python compile faster (~300s vs ~600s on my machine) but Phoronix’s PyBench test runs in 2130ms compared to 2229ms when using PyBench to train (and 2345ms with PGO disabled). Signed-off-by: Ross Burton --- meta/recipes-devtools/python/python3_3.5.5.bb | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/meta/recipes-devtools/python/python3_3.5.5.bb b/meta/recipes-devtools/python/python3_3.5.5.bb index b89e3b04c86..386b46a763d 100644 --- a/meta/recipes-devtools/python/python3_3.5.5.bb +++ b/meta/recipes-devtools/python/python3_3.5.5.bb @@ -76,7 +76,7 @@ SDK_CC_ARCH += "-DNDEBUG -fno-inline" EXTRA_OEMAKE += "CROSS_COMPILE=yes" EXTRA_OECONF += "CROSSPYTHONPATH=${STAGING_LIBDIR_NATIVE}/python${PYTHON_MAJMIN}/lib-dynload/ --without-ensurepip" -PYTHON3_PROFILE_TASK ?= "${S}/Tools/pybench/pybench.py -n 1" +PYTHON3_PROFILE_TASK ?= "./python -m test.regrtest --pgo test_grammar test_opcodes test_dict test_builtin test_exceptions test_types test_support || true" export CROSS_COMPILE = "${TARGET_PREFIX}" export _PYTHON_PROJECT_BASE = "${B}" @@ -144,13 +144,15 @@ do_compile() { if ${@bb.utils.contains('PACKAGECONFIG', 'pgo', 'true', 'false', d)}; then run_make profile-opt qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_TARGET}', ['${B}', '${STAGING_DIR_TARGET}/${base_libdir}'])}" - cat > pgo-image-qemuwrapper << EOF + cat >pgo-wrapper <