From patchwork Thu Aug 2 16:00:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 143355 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp2338703ljj; Thu, 2 Aug 2018 09:02:22 -0700 (PDT) X-Google-Smtp-Source: AAOMgpdypgXDFCHhPNkrGWt4nPIA297k7GRfkunyQIBqX/uSi9EH6lcArHdiHpuLQg2ULg+tx6Ud X-Received: by 2002:a37:1453:: with SMTP id e80-v6mr54184qkh.239.1533225741937; Thu, 02 Aug 2018 09:02:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533225741; cv=none; d=google.com; s=arc-20160816; b=AQVqPfOooJFcCCGibEbh0SUbZvo/LPqKyEUuqriGcrlez7TAKdGjsjBqKmaBkTD7eP xE0Cg19U14lrdxnqjONd7u9gl25Urj0mcE0UZQt93fI2wf+wJd6eC6sumvm2PXUquWHg XHxlFmktqQKX8NbxZPWkI+3ZZGQVeqSNJxWhtRy1pgz3w++RDxUjoRc6X6za7x37VyMH LNFV26sBFKb7d2GgNfR8KLRTldeyq/4LBtEHMTQiZyo9Wmmh0vg2gPLn0I0HnLmoW8iI svyk2xCXCDPD+ByAmbP+/dtJfB2UeA6A/fFoJdsEcQIZfoiN96MEsT3OO7n9oBY1dbk1 6ESg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=Mj8qFk153vnTfWs+e1tinM0Tia+TE71sVoPyrnLgYH0=; b=C9UQAktP4oQWv0zKj0pyU2mMi+u5CJN1adAMXnC+PKTOTt/XGwfuOaHFBrx8OrtVkp MRtnZ5UCj0poUM+J0Z3BvMzchT8jEp/Sto3gE8YuKyGQQe4WjVBFOstfPANznLjdHVHS ffr+Ky+WTFVm19xrYLlYKswNtgo/v2ZanIoFgEoix2N1JIoLkC9L1FQW2TkzZCqRes6I G515eDOQ0Po21g+3ftoIql5Hfyxvv6cFh8yyWZyH3+clDl4d7G9PNUxgzR0G3kBn65HN vQ66FcYjLurzL4CVy52+KaFXip8pCUIhe4IOf02JGwapnEHCk49c+JwdlPdY2EubTLWA /xzg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (ec2-54-197-127-237.compute-1.amazonaws.com. [54.197.127.237]) by mx.google.com with ESMTP id 65-v6si2245197qkq.393.2018.08.02.09.02.21; Thu, 02 Aug 2018 09:02:21 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) client-ip=54.197.127.237; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 9E21168627; Thu, 2 Aug 2018 16:02:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, MAILING_LIST_MULTI autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id CB21C68609; Thu, 2 Aug 2018 16:01:14 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 8BDA26860A; Thu, 2 Aug 2018 16:01:01 +0000 (UTC) Received: from forward105j.mail.yandex.net (forward105j.mail.yandex.net [5.45.198.248]) by lists.linaro.org (Postfix) with ESMTPS id BC9016860A for ; Thu, 2 Aug 2018 16:00:24 +0000 (UTC) Received: from mxback14g.mail.yandex.net (mxback14g.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:93]) by forward105j.mail.yandex.net (Yandex) with ESMTP id DAFB91838FE for ; Thu, 2 Aug 2018 19:00:17 +0300 (MSK) Received: from smtp1o.mail.yandex.net (smtp1o.mail.yandex.net [2a02:6b8:0:1a2d::25]) by mxback14g.mail.yandex.net (nwsmtp/Yandex) with ESMTP id lkt7iMO5Iv-0HdmUFPp; Thu, 02 Aug 2018 19:00:17 +0300 Received: by smtp1o.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id Gj7hFMLav7-0G9icnI9; Thu, 02 Aug 2018 19:00:16 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Github ODP bot To: lng-odp@lists.linaro.org Date: Thu, 2 Aug 2018 16:00:04 +0000 Message-Id: <1533225614-20192-2-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1533225614-20192-1-git-send-email-odpbot@yandex.ru> References: <1533225614-20192-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 647 Subject: [lng-odp] [PATCH v5 1/11] travis: add docker tests X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Maxim Uvarov initial Travis docker runs Signed-off-by: Maxim Uvarov --- /** Email created from pull request 647 (muvarov:master_docker2) ** https://github.com/Linaro/odp/pull/647 ** Patch: https://github.com/Linaro/odp/pull/647.patch ** Base sha: 6886a2ca3d3dd1157302ddee424c36f632903bd0 ** Merge commit sha: 1ca948a3fc61fc94775495bb529ab02560071b04 **/ .travis.yml | 362 +++++++++------------------------------------ scripts/ci/build_dpdk.sh | 89 +++++++++++ scripts/ci/build_x86_64.sh | 45 ++++++ scripts/ci/coverage.sh | 35 +++++ scripts/ci/distcheck.sh | 25 ++++ 5 files changed, 261 insertions(+), 295 deletions(-) create mode 100755 scripts/ci/build_dpdk.sh create mode 100755 scripts/ci/build_x86_64.sh create mode 100755 scripts/ci/coverage.sh create mode 100755 scripts/ci/distcheck.sh diff --git a/.travis.yml b/.travis.yml index 26bbe1a32..0f0db2931 100644 --- a/.travis.yml +++ b/.travis.yml @@ -7,6 +7,12 @@ # pushing to github/coverity_scan will also launch a static analysis # See https://scan.coverity.com/travis_ci +# +# Travis uses Docker images which mainained here: +# https://github.com/Linaro/odp-docker-images +# CI scirpts are maintained under ./scripts/ci/ directory +# which passed into container during the test run. + language: c sudo: required dist: trusty @@ -49,6 +55,7 @@ env: # you need generated new one at https://codecov.io specific for your repo. - CODECOV_TOKEN=a733c34c-5f5c-4ff1-af4b-e9f5edb1ab5e - DPDK_VERS="17.11.3" + - DOCKER_NAMESPACE="lngodp" matrix: - CONF="" - CONF="--disable-abi-compat" @@ -63,278 +70,45 @@ env: compiler: - gcc - - clang-3.8 - -before_install: - - # Install cross toolchains, etc - # apt-get update may fail thanks to Ubuntu removing Packages/indices while not removing relevant parts of Release file - - if [ -n "$CROSS_ARCH" ] ; - then - BUILD_GNU_TYPE=`dpkg-architecture -a"$CROSS_ARCH" -qDEB_BUILD_GNU_TYPE` ; - CROSS_GNU_TYPE=`dpkg-architecture -a"$CROSS_ARCH" -qDEB_HOST_GNU_TYPE` ; - CROSS_MULTIARCH=`dpkg-architecture -a"$CROSS_ARCH" -qDEB_HOST_MULTIARCH` ; - CROSS="--host="$CROSS_GNU_TYPE" --build="$BUILD_GNU_TYPE"" ; - sudo dpkg --add-architecture "$CROSS_ARCH" ; - PKGS="build-essential libc6-dev:$CROSS_ARCH libssl-dev:$CROSS_ARCH zlib1g-dev:$CROSS_ARCH libconfig-dev:$CROSS_ARCH libstdc++-4.8-dev:$CROSS_ARCH libpcap0.8-dev:$CROSS_ARCH" ; - if [ "$CROSS_ARCH" = "i386" ] ; - then - PKGS="$PKGS g++-multilib" ; - else - PKGS="$PKGS g++-$CROSS_GNU_TYPE" ; - sudo sed -e 's/^deb http/deb [arch=amd64] http/g' /etc/apt/sources.list -i ; - sudo -E apt-add-repository -y "deb [arch=$CROSS_ARCH] http://ports.ubuntu.com trusty main" ; - sudo -E apt-add-repository -y "deb [arch=$CROSS_ARCH] http://ports.ubuntu.com trusty-updates main" ; - fi ; - if [ "$CROSS_ARCH" != "armhf" ] ; - then - PKGS="$PKGS libnuma-dev:$CROSS_ARCH" ; - fi ; - sudo cat /etc/apt/sources.list ; - sudo -E apt-get -y update || true ; - sudo -E apt-get -y --no-install-suggests --no-install-recommends --force-yes install $PKGS || exit 1 ; - export PKG_CONFIG_PATH=/usr/lib/${CROSS_MULTIARCH}/pkgconfig:/usr/${CROSS_MULTIARCH}/lib/pkgconfig ; - fi - - if [ "${CC#clang}" != "${CC}" ] ; - then - if [ -n "$CROSS_ARCH" ] ; - then - export CC="${CC} --target=$CROSS_GNU_TYPE" ; - if [ "$CROSS_ARCH" = "i386" ] ; - then - DPDK_CFLAGS="-m32" ; - else - DPDK_CROSS="$CROSS_GNU_TYPE-" ; - DPDK_CFLAGS="--target=$CROSS_GNU_TYPE" ; - fi - fi ; - export CXX="${CC/clang/clang++}"; - elif [ "$CROSS_ARCH" = "i386" ] ; - then - export CC="gcc -m32" ; - export CXX="g++ -m32" ; - DPDK_CFLAGS="-m32" ; - elif [ -n "$CROSS_ARCH" ] ; - then - export CC="$CROSS_GNU_TYPE"-gcc ; - export CXX="$CROSS_GNU_TYPE"-g++ ; - DPDK_CROSS="$CROSS_GNU_TYPE-" ; - fi - - if test ! -L /usr/lib/ccache/${CC%% *} ; then sudo ln -s -t /usr/lib/ccache/ `which ${CC%% *}` ; fi - - ccache -s - # Install cunit for the validation tests because distro version is too old and fails C99 compile - - sudo apt-get remove libcunit1-dev libcunit1 - - export LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" - - | - if [ ! -f "$HOME/cunit-install/$CROSS_ARCH/lib/libcunit.a" ]; then - export CUNIT_VERSION=2.1-3 - curl -sSOL https://github.com/Linaro/libcunit/releases/download/${CUNIT_VERSION}/CUnit-${CUNIT_VERSION}.tar.bz2 - tar -jxf *.bz2 - pushd CUnit* - libtoolize --force --copy - aclocal - autoheader - automake --add-missing --include-deps --copy - autoconf - ./configure --prefix=$HOME/cunit-install/$CROSS_ARCH --enable-debug --enable-automated --enable-basic --enable-console --enable-examples --enable-test $CROSS || cat config.log - make -j $(nproc) - sudo make install - popd - fi - - export PKG_CONFIG_PATH="$HOME/cunit-install/$CROSS_ARCH/lib/pkgconfig:${PKG_CONFIG_PATH}" - - find $HOME/cunit-install - - # workaround for tap driver issues on recent Travis images - # Allow forwaring on virtual interfaces used for testing - - sudo iptables --policy FORWARD ACCEPT - + - clang install: - - echo 1000 | sudo tee /proc/sys/vm/nr_hugepages - - sudo mkdir -p /mnt/huge - - sudo mount -t hugetlbfs nodev /mnt/huge - - - if [ -z "$CROSS_ARCH" ] ; - then - sudo apt-get -qq update ; - sudo apt-get install linux-headers-`uname -r` ; - fi - - gem install asciidoctor - - # DPDK pktio. Cache will be updated automatically to ${DPDK_VERS} - - | - case "$CROSS_ARCH" in - "arm64") - DPDK_TARGET="arm64-armv8a-linuxapp-" - ;; - "armhf") - DPDK_TARGET="arm-armv7a-linuxapp-" - ;; - "i386") - DPDK_TARGET="i686-native-linuxapp-" - ;; - "") - DPDK_TARGET="x86_64-native-linuxapp-" - DPDK_MACHINE=snb - ;; - esac - - | - if [ -n "$DPDK_TARGET" ] ; then - if [ "${CC#clang}" != "${CC}" ] ; then - DPDKCC=clang ; - else - DPDKCC=gcc ; - fi - if [ -n "$DPDK_SHARED" ] ; then - TARGET="${DPDK_TARGET}$DPDKCC"-shared - LIBDPDKEXT=so - export LD_LIBRARY_PATH="`pwd`/${TARGET}:$LD_LIBRARY_PATH" - echo $LD_LIBRARY_PATH - else - TARGET="${DPDK_TARGET}$DPDKCC" - LIBDPDKEXT=a - fi - DPDK_TARGET="${DPDK_TARGET}gcc" - CACHED_DPDK_VERS=`fgrep Version dpdk/pkg/dpdk.spec | cut -d " " -f 2` - if [ ! -d dpdk -o "${CACHED_DPDK_VERS}" != "${DPDK_VERS}" ]; then - rm -rf dpdk - mkdir dpdk - pushd dpdk - git init - git -c advice.detachedHead=false fetch -q --depth=1 http://dpdk.org/git/dpdk-stable v${DPDK_VERS} - git checkout -f FETCH_HEAD - popd - fi - if [ ! -f "dpdk/${TARGET}/usr/local/lib/libdpdk.$LIBDPDKEXT" ]; then - pushd dpdk - git log --oneline --decorate - # AArch64 && ARMv7 fixup - sed -i -e 's/40900/40800/g' lib/librte_eal/common/include/arch/arm/rte_vect.h - sed -i -e 's/!(/!(defined(__arm__) \&\& defined(__clang__) || /g' lib/librte_eal/common/include/arch/arm/rte_byteorder.h - sed -i -e 's/__GNUC__/defined(__arm__) \&\& defined(__clang__) || __GNUC__/' lib/librte_eal/common/include/generic/rte_byteorder.h - sed -i -e 's,\$(CC),\0 $(EXTRA_CFLAGS),g' lib/librte_acl/Makefile - make config T=${DPDK_TARGET} O=${TARGET} - pushd ${TARGET} - sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_PCAP=).*,\1y,' .config - # OCTEON TX driver includes ARM v8.1 instructions - sed -ri 's,(CONFIG_RTE_LIBRTE_OCTEONTX_PMD=).*,\1n,' .config - sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF=).*,\1n,' .config - sed -ri 's,(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL=).*,\1n,' .config - if test -n "${DPDK_MACHINE}" ; then - sed -ri 's,(CONFIG_RTE_MACHINE=).*,\1"'${DPDK_MACHINE}'",' .config - fi - if test -n "${DPDK_SHARED}" ; then - sed -ri 's,(CONFIG_RTE_BUILD_SHARED_LIB=).*,\1y,' .config - fi - if test -n "$CROSS_ARCH" ; then - sed -ri -e 's,(CONFIG_RTE_EAL_IGB_UIO=).*,\1n,' .config - sed -ri -e 's,(CONFIG_RTE_KNI_KMOD=).*,\1n,' .config - fi - sed -ri -e 's,(CONFIG_RTE_TOOLCHAIN=).*,\1"'${DPDKCC}'",' .config - sed -ri -e '/CONFIG_RTE_TOOLCHAIN_.*/d' .config - echo CONFIG_RTE_TOOLCHAIN_${DPDKCC^^}=y >> .config - popd - make build O=${TARGET} EXTRA_CFLAGS="-fPIC $DPDK_CFLAGS" CROSS="$DPDK_CROSS" CC="$CC" HOSTCC=gcc -j $(nproc) - make install O=${TARGET} DESTDIR=${TARGET} - rm -r ./doc ./${TARGET}/app ./${TARGET}/build - popd - fi - EXTRA_CONF="$EXTRA_CONF --with-dpdk-path=`pwd`/dpdk/${TARGET}/usr/local" - fi - -# Netmap pktio - - | - if [ -z "$CROSS_ARCH" ]; then - if [ ! -f "netmap/LINUX/netmap.ko" ]; then - git -c advice.detachedHead=false clone -q --depth=1 --single-branch --branch=v11.2 https://github.com/luigirizzo/netmap.git - pushd netmap/LINUX - ./configure - make -j $(nproc) - popd - fi - sudo insmod ./netmap/LINUX/netmap.ko - EXTRA_CONF="$EXTRA_CONF --with-netmap-path=`pwd`/netmap" - fi - + - sudo apt-get install linux-headers-`uname -r` script: - - ./bootstrap - - ./configure --prefix=$HOME/odp-install - --enable-user-guides - --enable-debug=full - --enable-helper-linux - $CROSS $EXTRA_CONF $CONF - - make -j $(nproc) - - mkdir /dev/shm/odp - # Run all tests only for default configuration - - if [ -z "$CROSS_ARCH" ] ; then - if [ -n "$CONF" ] ; then - sudo ODP_CONFIG_FILE="`pwd`/config/odp-linux-generic.conf" LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" ODP_SHM_DIR=/dev/shm/odp make check ; - else - sudo ODP_SCHEDULER=basic LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" ODP_SHM_DIR=/dev/shm/odp make check ; - sudo ODP_SCHEDULER=sp LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" ODP_SHM_DIR=/dev/shm/odp make check ; - sudo ODP_SCHEDULER=iquery LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" ODP_SHM_DIR=/dev/shm/odp make check ; - sudo ODP_SCHEDULER=scalable LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" ODP_SHM_DIR=/dev/shm/odp make check ; - fi + - if [ "${CC#clang}" != "${CC}" ] ; then LD=ld CXX=clang++; fi + - if [ -n "${CROSS_ARCH}" ] ; then + docker run -i -t -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" -e DPDK_SHARED="${DPDK_SHARED}" + maximuvarov/odp_ubuntu_14.04.5 /root/build_${CROSS_ARCH}.sh ; + else + docker run --privileged -i -t + -v `pwd`:/odp -v/lib/modules:/lib/modules -v/usr/src:/usr/src --shm-size 8g + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" -e DPDK_SHARED="${DPDK_SHARED}" + ${DOCKER_NAMESPACE}/travis-odp-lng-x86 /odp/scripts/ci/build_x86_64.sh $CONF ; fi - - make install - - - echo "Checking linking and run from install..." - - pushd $HOME - - echo "Dynamic link.." - - ${CC} ${CFLAGS} ${OLDPWD}/example/hello/odp_hello.c -o odp_hello_inst_dynamic `PKG_CONFIG_PATH=${HOME}/odp-install/lib/pkgconfig:${PKG_CONFIG_PATH} pkg-config --cflags --libs libodp-linux` - - if [ -z "$CROSS_ARCH" ] ; then - LD_LIBRARY_PATH="${HOME}/odp-install/lib:$LD_LIBRARY_PATH" ./odp_hello_inst_dynamic ; - fi - - | - # it is not possible to do static linking if we only have shared DPDK library. Compiler complains about missing -ldpdk - if [ -z "$TARGET" -o -z "$DPDK_SHARED" ] ; then - echo "Static link.." - ${CC} ${CFLAGS} ${OLDPWD}/example/hello/odp_hello.c -o odp_hello_inst_static `PKG_CONFIG_PATH=${HOME}/odp-install/lib/pkgconfig:${PKG_CONFIG_PATH} pkg-config --cflags --libs libodp-linux --static` -static || exit 1 - if [ -z "$CROSS_ARCH" ] ; then - ./odp_hello_inst_static; - fi - fi - - popd - - ccache -s - jobs: include: - stage: test env: TEST=coverage compiler: gcc script: - - sudo pip install coverage - - ./bootstrap - - ./configure --prefix=$HOME/odp-install - --enable-user-guides - --with-dpdk-path=`pwd`/dpdk/${TARGET}/usr/local - --with-netmap-path=`pwd`/netmap CFLAGS="-O0 -coverage" - CXXFLAGS="-O0 -coverage" LDFLAGS="--coverage" - --enable-debug=full - --enable-helper-linux - - CCACHE_DISABLE=1 make -j $(nproc) - - mkdir -p /dev/shm/odp - - sudo CCACHE_DISABLE=1 ODP_SCHEDULER=basic ODP_SHM_DIR=/dev/shm/odp LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" make check - - sudo CCACHE_DISABLE=1 ODP_SCHEDULER=sp ODP_SHM_DIR=/dev/shm/odp LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" make check - - sudo CCACHE_DISABLE=1 ODP_SCHEDULER=iquery ODP_SHM_DIR=/dev/shm/odp LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" make check - - sudo CCACHE_DISABLE=1 ODP_SCHEDULER=scalable ODP_SHM_DIR=/dev/shm/odp LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" make check - - bash <(curl -s https://codecov.io/bash) -X coveragepy + - docker run --privileged -i -t + -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src --shm-size 8g + -e CODECOV_TOKEN="${CODECOV_TOKEN}" + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" + ${DOCKER_NAMESPACE}/travis-odp-lng-x86 /odp/scripts/ci/coverage.sh - stage: test env: TEST=distcheck compiler: gcc script: - - ./bootstrap - - ./configure --prefix=$HOME/odp-install - --enable-user-guides - - sudo PATH="$PATH" LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" make distcheck - - stage: test - env: TEST=distcheck-non-abi - compiler: gcc - script: - - ./bootstrap - - ./configure --prefix=$HOME/odp-install - --enable-user-guides - - sudo PATH="$PATH" LD_LIBRARY_PATH="$HOME/cunit-install/$CROSS_ARCH/lib:$LD_LIBRARY_PATH" make distcheck DISTCHECK__CONFIGURE_FLAGS=--disable-abi-compat + - docker run --privileged -i -t + -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src --shm-size 8g + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" + ${DOCKER_NAMESPACE}/travis-odp-lng-x86 /odp/scripts/ci/distcheck.sh - stage: "build only" env: TEST=doxygen compiler: gcc @@ -364,85 +138,83 @@ jobs: true fi - stage: "build only" - env: CONF="" - compiler: gcc - install: true - script: - - ./bootstrap - - ./configure --enable-helper-linux - - make -j $(nproc) - - stage: "build only" - env: CONF="" - compiler: clang-3.8 - install: true - script: - - ./bootstrap - - ./configure --enable-helper-linux - - make -j $(nproc) - - stage: "build only" - env: CROSS_ARCH="i386" - compiler: gcc - install: true - script: - - ./bootstrap - - ./configure --enable-helper-linux $CROSS - - make -j $(nproc) - - stage: "build only" - env: CROSS_ARCH="arm64" + env: Ubuntu14.04_arm64 compiler: gcc - install: true script: - - ./bootstrap - - ./configure --enable-helper-linux $CROSS - - make -j $(nproc) + - docker run -i -t -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src + maximuvarov/odp_ubuntu_14.04.5 /root/build_arm64.sh - stage: test compiler: gcc env: CROSS_ARCH="arm64" - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="arm64" - stage: test compiler: gcc env: CROSS_ARCH="armhf" CFLAGS="-march=armv7-a" - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="armhf" CFLAGS="-march=armv7-a" - stage: test compiler: gcc env: CROSS_ARCH="powerpc" - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="powerpc" - stage: test compiler: gcc env: CROSS_ARCH="i386" + script: + - docker run --privileged -i -t + -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" + maximuvarov/odp_ubuntu_14.04.5_i386 /root/build_${CROSS_ARCH}.sh ; - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="i386" + script: + - docker run --privileged -i -t + -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" + maximuvarov/odp_ubuntu_14.04.5_i386 /root/build_${CROSS_ARCH}.sh ; - stage: test compiler: gcc env: CROSS_ARCH="arm64" CONF="--disable-abi-compat" - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="arm64" CONF="--disable-abi-compat" - stage: test compiler: gcc env: CROSS_ARCH="armhf" CFLAGS="-march=armv7-a" CONF="--disable-abi-compat" - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="armhf" CFLAGS="-march=armv7-a" CONF="--disable-abi-compat" - stage: test compiler: gcc env: CROSS_ARCH="powerpc" CONF="--disable-abi-compat" - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="powerpc" CONF="--disable-abi-compat" - stage: test compiler: gcc env: CROSS_ARCH="i386" CONF="--disable-abi-compat" + script: + - docker run --privileged -i -t + -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" + maximuvarov/odp_ubuntu_14.04.5_i386 /root/build_${CROSS_ARCH}.sh ; - stage: test - compiler: clang-3.8 + compiler: clang env: CROSS_ARCH="i386" CONF="--disable-abi-compat" + script: + - docker run --privileged -i -t + -v `pwd`:/odp -v /lib/modules:/lib/modules -v /usr/src:/usr/src + -e CC="${CC}" -e LD="${LD}" -e CXX="${CXX}" + -e CONF="${CONF}" + maximuvarov/odp_ubuntu_14.04.5_i386 /root/build_${CROSS_ARCH}.sh ; - stage: test canfail: yes env: TEST=checkpatch diff --git a/scripts/ci/build_dpdk.sh b/scripts/ci/build_dpdk.sh new file mode 100755 index 000000000..9ceac4529 --- /dev/null +++ b/scripts/ci/build_dpdk.sh @@ -0,0 +1,89 @@ +#!/bin/bash -x + +set -e + +DPDK_VERS="17.11.2" +CROSS= + + +case "$CROSS_ARCH" in + "arm64") + DPDK_TARGET="arm64-armv8a-linuxapp-" + ;; + "armhf") + DPDK_TARGET="arm-armv7a-linuxapp-" + ;; + "i386") + DPDK_TARGET="i686-native-linuxapp-" + ;; + "") + DPDK_TARGET="x86_64-native-linuxapp-" + DPDK_MACHINE=snb + ;; +esac + + +if [ -n "$DPDK_TARGET" ] ; then + if [ "${CC#clang}" != "${CC}" ] ; then + DPDKCC=clang ; + else + DPDKCC=gcc ; + fi + if [ -n "$DPDK_SHARED" ] ; then + TARGET="${DPDK_TARGET}$DPDKCC"-shared + LIBDPDKEXT=so + export LD_LIBRARY_PATH="`pwd`/${TARGET}:$LD_LIBRARY_PATH" + echo $LD_LIBRARY_PATH + else + TARGET="${DPDK_TARGET}$DPDKCC" + LIBDPDKEXT=a + fi + DPDK_TARGET="${DPDK_TARGET}gcc" + CACHED_DPDK_VERS=`fgrep Version dpdk/pkg/dpdk.spec | cut -d " " -f 2` + if [ ! -d dpdk -o "${CACHED_DPDK_VERS}" != "${DPDK_VERS}" ]; then + rm -rf dpdk + mkdir dpdk + pushd dpdk + git init + git -c advice.detachedHead=false fetch -q --depth=1 http://dpdk.org/git/dpdk-stable v${DPDK_VERS} + git checkout -f FETCH_HEAD + popd + fi + if [ ! -f "dpdk/${TARGET}/usr/local/lib/libdpdk.$LIBDPDKEXT" ]; then + pushd dpdk + git log --oneline --decorate + # AArch64 && ARMv7 fixup + sed -i -e 's/40900/40800/g' lib/librte_eal/common/include/arch/arm/rte_vect.h + sed -i -e 's/!(/!(defined(__arm__) \&\& defined(__clang__) || /g' lib/librte_eal/common/include/arch/arm/rte_byteorder.h + sed -i -e 's/__GNUC__/defined(__arm__) \&\& defined(__clang__) || __GNUC__/' lib/librte_eal/common/include/generic/rte_byteorder.h + sed -i -e 's,\$(CC),\0 $(EXTRA_CFLAGS),g' lib/librte_acl/Makefile + make config T=${DPDK_TARGET} O=${TARGET} + pushd ${TARGET} + sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_PCAP=).*,\1y,' .config + # OCTEON TX driver includes ARM v8.1 instructions + sed -ri 's,(CONFIG_RTE_LIBRTE_OCTEONTX_PMD=).*,\1n,' .config + sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF=).*,\1n,' .config + sed -ri 's,(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL=).*,\1n,' .config + if test -n "${DPDK_MACHINE}" ; then + sed -ri 's,(CONFIG_RTE_MACHINE=).*,\1"'${DPDK_MACHINE}'",' .config + fi + if test -n "${DPDK_SHARED}" ; then + sed -ri 's,(CONFIG_RTE_BUILD_SHARED_LIB=).*,\1y,' .config + fi + if test -n "$CROSS_ARCH" ; then + sed -ri -e 's,(CONFIG_RTE_EAL_IGB_UIO=).*,\1n,' .config + sed -ri -e 's,(CONFIG_RTE_KNI_KMOD=).*,\1n,' .config + fi + sed -ri -e 's,(CONFIG_RTE_TOOLCHAIN=).*,\1"'${DPDKCC}'",' .config + sed -ri -e '/CONFIG_RTE_TOOLCHAIN_.*/d' .config + echo CONFIG_RTE_TOOLCHAIN_${DPDKCC^^}=y >> .config + popd + make build O=${TARGET} EXTRA_CFLAGS="-fPIC $DPDK_CFLAGS" CROSS="$DPDK_CROSS" CC="${CC}" HOSTCC="gcc" -j $(nproc) + make install O=${TARGET} DESTDIR=${TARGET} + pwd + rm -r ./doc ./${TARGET}/app ./${TARGET}/build + popd + fi +fi + +echo "`pwd`/dpdk/${TARGET}/usr/local" > /tmp/dpdk_install_dir diff --git a/scripts/ci/build_x86_64.sh b/scripts/ci/build_x86_64.sh new file mode 100755 index 000000000..1dc039e96 --- /dev/null +++ b/scripts/ci/build_x86_64.sh @@ -0,0 +1,45 @@ +#!/bin/bash +set -e + +# CC LD AR CXX has to be predifubed +# + +export PKG_CONFIG_PATH="$HOME/cunit-install/x86_64/lib/pkgconfig:${PKG_CONFIG_PATH}" + +CWD=$(dirname "$0") +TDIR=`mktemp -d -p ~` + +cd ${TDIR} +export CROSS_ARCH="" + +export TARGET="x86_64$DPDKCC" + +$CWD/build_dpdk.sh +DPDKPATH=`cat /tmp/dpdk_install_dir` + +echo 1000 | tee /proc/sys/vm/nr_hugepages +mkdir -p /mnt/huge +mount -t hugetlbfs nodev /mnt/huge + +git clone ${CWD}/../../ odp +cd ./odp +./bootstrap +./configure ${CONF} \ + --with-dpdk-path=${DPDKPATH} + +make clean +make -j 8 +# Tell some time sensative ODP test that they can be skipped due to not +# isolated environment. +export CI="true" +make check + +if [ $? -ne 0 ]; then + find . -name "*.trs" | xargs grep -l '^.test-result. FAIL' | while read trs ; do echo FAILURE detected at $trs; cat ${trs%%.trs}.log ; done +fi + +cd ~ +rm -rf ${TDIR} + +umount /mnt/huge + diff --git a/scripts/ci/coverage.sh b/scripts/ci/coverage.sh new file mode 100755 index 000000000..4326cb7dd --- /dev/null +++ b/scripts/ci/coverage.sh @@ -0,0 +1,35 @@ +#!/bin/bash +set -e + +# CC LD AR CXX has to be predifubed +# + +export PKG_CONFIG_PATH="$HOME/cunit-install/x86_64/lib/pkgconfig:${PKG_CONFIG_PATH}" + +CWD=$(dirname "$0") +TDIR=`mktemp -d -p ~` + +cd ${TDIR} +export CROSS_ARCH="" +export TARGET="x86_64$DPDKCC" + +$CWD/build_dpdk.sh + +git clone ${CWD}/../../ odp +cd ./odp +./bootstrap +./configure \ + CFLAGS="-O0 -coverage" CXXFLAGS="-O0 -coverage" LDFLAGS="--coverage" \ + --enable-debug=full --enable-helper-linux +make clean +export CCACHE_DISABLE=1 +make -j $(nproc) + +ODP_SCHEDULER=basic make check +ODP_SCHEDULER=sp make check +ODP_SCHEDULER=iquery make check +ODP_SCHEDULER=scalable make check +bash <(curl -s https://codecov.io/bash) -X coveragepy + +cd ~ +rm -rf ${TDIR} diff --git a/scripts/ci/distcheck.sh b/scripts/ci/distcheck.sh new file mode 100755 index 000000000..22013473b --- /dev/null +++ b/scripts/ci/distcheck.sh @@ -0,0 +1,25 @@ +#!/bin/bash +set -e + +# CC LD AR CXX has to be predifubed +# + +export PKG_CONFIG_PATH="$HOME/cunit-install/x86_64/lib/pkgconfig:${PKG_CONFIG_PATH}" + +CWD=$(dirname "$0") +TDIR=`mktemp -d -p ~` + +cd ${TDIR} +git clone ${CWD}/../../ odp +cd ./odp +./bootstrap +./configure --enable-user-guides + +make clean +make distcheck + +make clean +make distcheck DISTCHECK__CONFIGURE_FLAGS=--disable-abi-compat + +cd ~ +rm -rf ${TDIR}