From patchwork Mon Oct 31 23:53:27 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brian Brooks X-Patchwork-Id: 80293 Delivered-To: patch@linaro.org Received: by 10.140.97.247 with SMTP id m110csp371542qge; Mon, 31 Oct 2016 16:54:33 -0700 (PDT) X-Received: by 10.55.20.164 with SMTP id 36mr25748618qku.86.1477958073517; Mon, 31 Oct 2016 16:54:33 -0700 (PDT) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id d36si15317738qte.109.2016.10.31.16.54.32; Mon, 31 Oct 2016 16:54:33 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 99C83617AE; Mon, 31 Oct 2016 23:54:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id C86DC61793; Mon, 31 Oct 2016 23:54:17 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 49D2661795; Mon, 31 Oct 2016 23:54:10 +0000 (UTC) Received: from mail-oi0-f53.google.com (mail-oi0-f53.google.com [209.85.218.53]) by lists.linaro.org (Postfix) with ESMTPS id 5BAC761772 for ; Mon, 31 Oct 2016 23:54:08 +0000 (UTC) Received: by mail-oi0-f53.google.com with SMTP id i127so257582395oia.2 for ; Mon, 31 Oct 2016 16:54:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=310CLnll9rEtSYEZEPerZEDm1rIqTWRK4xNGsSmmgg8=; b=OJrNeOfUdL3ivFA7NOLjYGZUU2rn6koRRopJCPLpvbuNMTcix+wRFa/OSJenEh2pnk nEjO+CJnz58rjdp7/W5OtqspTmyEQfrDeMmH1+PcTnrmtFdn4Zys20TsPU7R9+jJDI/r cEMjG1FcYI6/j6LT+hjMprvRTpBVDqX/APOdYGHaysVfpVjCaOQuFp27cue1+2WiwiVI eoRBRVSwcgGNukHfZ6zIpVhZAAClmfDvKVZbOLvEMKX1A4woZPY/pzmjJo/3UyLbW8Va NYuKapnDA24v3Rst92dRa8lmo1ShQo3WZvDTQZbsUVfgVdqwxMJZ+lZ94d5l4HPIm3aW i+kw== X-Gm-Message-State: ABUngvdoWyq+yicblPwB2YTMdmH4shDpyhRJAlg/gIVB6HVJwc32xelG2/3pEK9S2iSkC+pojl0= X-Received: by 10.157.61.98 with SMTP id a89mr23951047otc.181.1477958047290; Mon, 31 Oct 2016 16:54:07 -0700 (PDT) Received: from localhost.localdomain (68-248-140-212.lightspeed.austtx.sbcglobal.net. [68.248.140.212]) by smtp.gmail.com with ESMTPSA id v126sm8458680oig.20.2016.10.31.16.54.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 31 Oct 2016 16:54:06 -0700 (PDT) From: Brian Brooks To: lng-odp@lists.linaro.org Date: Mon, 31 Oct 2016 18:53:27 -0500 Message-Id: <1477958007-62930-1-git-send-email-brian.brooks@linaro.org> X-Mailer: git-send-email 2.7.4 X-Topics: patch Subject: [lng-odp] [PATCH] Add /scripts/install_build_deps.sh X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" and update build-related documentation. Signed-off-by: Brian Brooks --- DEPENDENCIES | 262 ------------------------------------------ README | 216 ++++++++++++++++++++++++++++++---- platform/linux-generic/README | 17 --- scripts/install_build_deps.sh | 124 ++++++++++++++++++++ 4 files changed, 315 insertions(+), 304 deletions(-) delete mode 100644 DEPENDENCIES delete mode 100644 platform/linux-generic/README create mode 100755 scripts/install_build_deps.sh -- 1.8.3.1 diff --git a/DEPENDENCIES b/DEPENDENCIES deleted file mode 100644 index f1f0edd..0000000 --- a/DEPENDENCIES +++ /dev/null @@ -1,262 +0,0 @@ -Prerequisites for building the OpenDataPlane (ODP) API - -1. Linux kernel >= 2.6.32 - - Earlier versions may or may not work. - -2. autotools - - automake - autoconf - libtool - - On Debian/Ubuntu systems: - $ sudo apt-get install automake autoconf libtool - - On CentOS/RedHat/Fedora systems: - $ sudo yum install automake autoconf libtool - -3. Required libraries - - Libraries currently required to link: openssl - -3.1 OpenSSL native compile - - For native compilation, simply load the necessary libraries using the appropriate - tool set. - - On Debian/Ubuntu systems: - $ sudo apt-get install libssl-dev - - On CentOS/RedHat/Fedora systems: - $ sudo yum install openssl-devel - -3.2 OpenSSL cross compilation - - Cross compilation requires cross compiling the individual libraries. In order for - a cross compiled executable to run on a target system, one must build the same - version as that which is installed on the target rootfs. - - For example, to build openssl for both 32 and 64 bit compilation: - - # Clone openssl repository - $ git clone git://git.openssl.org/openssl.git - $ cd openssl - - # The command "git tag" will list all tags available in the repo. - $ git tag - - # Checkout the specific tag to match openssl library in your target rootfs - $ git checkout - - # Build and install 32 bit version of openssl - $ ./Configure linux-generic32 --cross-compile-prefix=arm-linux-gnueabihf- \ - --prefix=/home/user/src/install-openssl shared - $ make - $ make install - - # Build and install 64 bit version of openssl - $ ./Configure linux-generic64 --cross-compile-prefix=aarch64-linux-gnu- \ - --prefix=/home/user/src/install-openssl-aarch64 shared - $ make - $ make install - - # You may now build either 32 or 64 bit ODP - $ git clone git://git.linaro.org/lng/odp.git odp - $ cd odp - $ ./bootstrap - - # Build 32 bit version of ODP - $ ./configure --host=arm-linux-gnueabihf \ - --with-openssl-path=/home/user/src/install-openssl - $ make - - # Or build 64 bit version of ODP - $ ./configure --host=aarch64-linux-gnu \ - --with-openssl-path=/home/user/src/install-openssl-aarch64 - $ make - -3.3 Netmap packet I/O support (optional) - - Netmap accelerated ODP packet I/O. - -3.3.1 Building netmap kernel modules - - ODP works at least with the latest release version of netmap, which is - currently 11.1. However, if possible one should try to use the latest netmap - master branch commit for the best performance and the latest bug fixes. - - # Checkout netmap code - $ git clone https://github.com/luigirizzo/netmap.git - $ cd netmap - $ git checkout v11.1 (optional) - - This is enough to build ODP. If you don't want to build netmap kernel - modules you can jump to section 3.3.2. - - Netmap consists of a core kernel module (netmap.ko), optional modified - device drivers and user space API headers to access the netmap - functionality. It is recommended to build both the core module and modified - device drivers for optimal performance. - - Netmap builds as an out-of-tree kernel module, you need matching kernel - sources to compile it. General build instructions can be found in the packet - README: https://github.com/luigirizzo/netmap/blob/master/LINUX/README. - - If you are running Ubuntu/Debian with the stock kernel and you want to - compile both netmap.ko and modified drivers, these steps will guide you - through it. - - # Download kernel headers - $ sudo apt-get install linux-headers-$(uname -r) - - # Download kernel source matching to the headers - $ sudo apt-get install linux-source - # or - $ apt-get source linux-image-$(uname -r) - - The source archive will be placed in /usr/src/linux-source- - (or in the current directory if using apt-get source). You will need to - locate it and extract it to a convenient place. - - # Compile netmap - $ cd /LINUX - $ ./configure --kernel-sources= - $ make - -3.3.2 Building ODP - - $ cd - $ ./bootstrap - $ ./configure --with-netmap-path= - $ make - -3.3.3 Inserting netmap kernel modules - - In order to use netmap I/O you need to insert at least the core netmap - kernel module. - - $ cd /LINUX - $ sudo insmod netmap.ko - - To insert the optional modified drivers you first need to remove the - original drivers, if loaded (and if not linked into the kernel). For - example, if using ixgbe: - - $ cd /LINUX - $ sudo rmmod ixgbe - $ sudo insmod ixgbe/ixgbe.ko - - To restore the original drivers you should be able to use modprobe. - -3.3.4 Running ODP with netmap I/O - - ODP applications will use netmap for packet I/O by default as long as the - netmap kernel module is loaded. If socket I/O is desired instead, it can be - activated by setting the environment variable ODP_PKTIO_DISABLE_NETMAP. - -3.4 DPDK packet I/O support (optional) - - Use DPDK for ODP packet I/O. - - Note: only packet I/O is accelerated with DPDK. Use - https://git.linaro.org/lng/odp-dpdk.git - for fully accelerated odp dpdk platform. - -3.4.1 Building DPDK and ODP with DPDK pktio support - - DPDK packet I/O has been tested to work with DPDK v16.07. - - Follow steps in ./scripts/build-pktio-dpdk - -3.4.2 Setup system - - # Load DPDK modules - $ sudo /sbin/modprobe uio - $ cd - $ sudo insmod x86_64-native-linuxapp-gcc/kmod/igb_uio.ko - - Reserve and mount hugepages and bind supported interfaces to DPDK modules - following the DPDK documentation. ODP DPDK packet I/O has been tested with - 512 x 2MB hugepages. All this can be done with the DPDK setup script - (/tools/dpdk-setup.sh). - -3.4.3 Running ODP with DPDK pktio - - ODP applications will try use DPDK for packet I/O by default. If some other - I/O type is desired instead, DPDK I/O can be disabled by setting the - environment variable ODP_PKTIO_DISABLE_DPDK. - - DPDK interfaces are accessed using indices. For example, two first DPDK - interfaces can be used with the odp_l2fwd example as follows: - $ cd - $ sudo ./test/performance/odp_l2fwd -i 0,1 -c 2 -m 0 - - Additionally, DPDK command line options can be passed to the application - using ODP_PKTIO_DPDK_PARAMS environment variable. For example, allocate - 1024MB of memory: - $ sudo ODP_PKTIO_DPDK_PARAMS="-m 1024" ./test/performance/odp_l2fwd -i 0 -c 1 - -4.0 Packages needed to build API tests - - Cunit test framework version 2.1-3 is required - Cunit prvodes a framework to run the API test suite that proves conformance to the - ODP API. The home page http://cunit.sourceforge.net/doc/introduction.html - -4.1 Native Cunit install - - # Debian/Ubuntu - $ apt-get install libcunit1-dev - -4.2 Built from src - - export CUNIT_VERSION=2.1-3 - curl -sSOL http://sourceforge.net/projects/cunit/files/CUnit/${CUNIT_VERSION}/CUnit-${CUNIT_VERSION}.tar.bz2 - tar -jxf *.bz2 - cd CUnit* - ./bootstrap - make install - #In Step 4.4 use --with-cunit-path=/home/${USER}/CUnitHome - -4.3 Cross compile of Cunit - - $ git svn clone http://svn.code.sf.net/p/cunit/code/trunk cunit-code - $ cd cunit-code - $ ./bootstrap - $ ./configure --host=arm-linux-gnueabihf --prefix=/home/${USER}/src/install-cunit - -4.4 Using Cunit with ODP - $ Add the configuration option to the regular configuration options - ./configure --enable-cunit #if cunit is in the PATH - ./configure --with-cunit-path=DIR #only if you need a path to Cunit libs and headers - -5.0 Documentation Images & Doxygen - - Images are stored as svg files. No conversions for these are needed. - - Message squence diagrams are stored as msc files and the svg versions generated when the docs are built - mscgen is used - #Debian/Ubuntu - # apt-get install mscgen - -5.1 API Guide -See http://www.stack.nl/~dimitri/doxygen/manual/install.html - -The tested version of doxygen is 1.8.8 - -5.1.1 HTML - # Debian/Ubuntu - $ apt-get install doxygen graphviz - -5.2 User guides - -5.2.1 HTML - # Debian/Ubuntu - $ apt-get install asciidoctor source-highlight librsvg2-bin - -6.0 Submitting patches - - When submitting patches they should be checked with ./scripts/checkpatch.pl - To have this tool also check spelling you need codespell. - # Debian/Ubuntu - #sudo apt install codespell diff --git a/README b/README index 4350b95..b7771c8 100644 --- a/README +++ b/README @@ -9,31 +9,6 @@ OpenDataPlane (ODP) project source code. Main git repository: git://git.linaro.org/lng/odp.git - How to build: - Look in platform/linux-*/README for more detail how to build. - Directory test contains test applications for ODP API calls and features support. - - For a list of build dependencies, read DEPENDENCIES. - In general you can build: - ./bootstrap - ./configure - Use 'make' to build ODP library and samples and 'make doxygen-doc' to build - ODP API documentation. 'make install' will copy all required headers and - binary files to the install directory. - - Configure options: - ./configure --help - - To execute all the testcases, assuming CUnit is installed for the validation tests: - ./bootstrap - ./configure --enable-test-perf --enable-test-vald - make check - - To generate the users guides: - ./bootstrap - ./configure --enable-user-guide - make - Patches tracking system: http://patches.opendataplane.org/project/lng-odp/list/ @@ -41,8 +16,199 @@ Mailing list: lng-odp@lists.linaro.org Please read CONTRIBUTING file before submitting patches. + When submitting patches they should be checked with ./scripts/checkpatch.pl + To have this tool also check spelling you need codespell. Email prefixes: [PATCH] means patch is for odp.git Bug tracking: https://bugs.linaro.org/describecomponents.cgi?product=OpenDataPlane%20-%20linux-%20generic%20reference + +Build Guide: + + Quickstart: + + $ ./scripts/install_build_deps.sh + $ ./bootstrap + $ ./configure \ + --enable-user-guides \ + --enable-debug \ + --enable-debug-print \ + --enable-cunit-support \ + --enable-test-example \ + --enable-test-helper \ + --enable-test-cpp \ + --enable-test-perf \ + --enable-test-perf-proc \ + --enable-test-vald \ + --enable-abi-compat \ + --enable-netmap-support \ + --enable-dpdk-support + $ make -j`nproc` + $ make check + $ make doxygen-doc + $ make install + + Dependencies: + + See /scripts/install_build_deps.sh + + Runtime + + * Linux kernel >= 2.6.32 + * OpenSSL + * PCAP (optional) + * Netmap (optional) + * DPDK (optional) + + Development + + * Autotools + * Doxygen, Graphviz, Asciidoctor, mscgen + * CUnit == 2.1.3 + * Valgrind, gcov + + Configuration: + + $ ./bootstrap + + $ ./configure --help + + --host= # Cross compiling + + Toolchain prefix: aarch64-linux-gnu + arm-linux-gnueabihf + mips64-linux-gnu + + --enable-user-guides # Docs + + Build /doc/users-guide + /doc/implementers-guide + /doc/process-guide + + --enable-debug + --enable-debug-print + + --enable-cunit # Testing + --enable-cunit-support + --with-cunit-path=DIR + --enable-test-example + --enable-test-helper + --enable-test-cpp + --enable-test-perf + --enable-test-perf-proc + --enable-test-vald + + --enable-valgrind + + --enable-abi-compat + + --with-platform=platform + + Select code in /platform// + + --with-openssl-path=DIR + + --enable-netmap-support, --with-netmap-path=DIR + + --enable-dpdk-support, --with-dpdk-path=DIR + + --enable-pktio_ipc-support + + --enable-schedule-sp + + --with-sdk-install-path=DIR + + Targets: + + make + make check + make doxygen-doc + make install + + Dependencies (from source): + + CUnit + + --with-cunit-path=/home/${USER}/CUnitHome + + $ export CUNIT_VERSION=2.1-3 + $ curl -sSOL http://sourceforge.net/projects/cunit/files/CUnit/${CUNIT_VERSION}/CUnit-${CUNIT_VERSION}.tar.bz2 + $ tar -jxf *.bz2 + $ cd CUnit* + $ ./bootstrap ; make install + + Cross compilation + + $ git svn clone http://svn.code.sf.net/p/cunit/code/trunk cunit-code + $ cd cunit-code + $ ./bootstrap + $ ./configure --host=arm-linux-gnueabihf --prefix=/home/${USER}/src/install-cunit + + OpenSSL + + --with-openssl-path=/home/user/src/install-openssl + + TODO: host/native build + + Cross Compilation + + $ git clone git://git.openssl.org/openssl.git + $ cd openssl + $ git tag + $ git checkout + $ ./configure linux-generic64 --cross-compile-prefix=aarch64-linux-gnu- \ + --prefix=/home/user/src/install-openssl-aarch64 shared + $ make + $ make install + + Netmap + + --with-netmap-path= + + $ KERNEL_VERS=`uname -r | cut -d- -f1 -` + $ wget https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-$KERNEL_VERS.tar.xz + $ xz --decompress linux-$KERNEL_VERS.tar.xz + $ tar -xf linux-$KERNEL_VERS.tar.xz + + $ git clone https://github.com/luigirizzo/netmap.git + $ cd netmap + $ git checkout v11.1 + $ cd LINUX + $ ./configure --kernel-sources=$HOME/linux-$KERNEL_VERS + $ make + + $ sudo insmod netmap.ko + $ sudo rmmod ixgbe + $ sudo insmod ixgbe/ixgbe.ko + + ODP_PKTIO_DISABLE_NETMAP environment variable: use sockets instead of netmap + + DPDK + + See /scripts/build-pktio-dpdk + + ODP applications will try use DPDK for packet I/O by default. If some other + I/O type is desired instead, DPDK I/O can be disabled by setting the + environment variable ODP_PKTIO_DISABLE_DPDK. + + DPDK interfaces are accessed using indices. For example, two first DPDK + interfaces can be used with the odp_l2fwd example as follows: + $ cd + $ sudo ./test/performance/odp_l2fwd -i 0,1 -c 2 -m 0 + + Additionally, DPDK command line options can be passed to the application + using ODP_PKTIO_DPDK_PARAMS environment variable. For example, allocate + 1024MB of memory: + $ sudo ODP_PKTIO_DPDK_PARAMS="-m 1024" ./test/performance/odp_l2fwd -i 0 -c 1 + + Reserve and mount hugepages and bind supported interfaces to DPDK modules + following the DPDK documentation. ODP DPDK packet I/O has been tested with + 512 x 2MB hugepages. All this can be done with the DPDK setup script + (/tools/dpdk-setup.sh). + + $ sudo /sbin/modprobe uio + $ cd + $ sudo insmod x86_64-native-linuxapp-gcc/kmod/igb_uio.ko + + See https://git.linaro.org/lng/odp-dpdk.git for better DPDK integration \ No newline at end of file diff --git a/platform/linux-generic/README b/platform/linux-generic/README deleted file mode 100644 index 3e05dab..0000000 --- a/platform/linux-generic/README +++ /dev/null @@ -1,17 +0,0 @@ -Copyright (c) 2014, Linaro Limited -All rights reserved. - -SPDX-License-Identifier: BSD-3-Clause - -1. Intro - -OpenDataPlane implementation for Linux generic. Directory linux-generic contains ODP headers and implementation -for linux-generic target. This drop does not target high -performance. It is rather proof of ODP API functionality. It still uses -linux-generic's SW scheduler. - -2. Build -# To compile ODP -./bootstrap -./configure -make diff --git a/scripts/install_build_deps.sh b/scripts/install_build_deps.sh new file mode 100755 index 0000000..074661f --- /dev/null +++ b/scripts/install_build_deps.sh @@ -0,0 +1,124 @@ +#!/bin/bash + +# Copyright (c) 2016, Linaro Limited +# All rights reserved. +# +# SPDX-License-Identifier: BSD-3-Clause + +if [ -f /etc/lsb-release ]; then + source /etc/lsb-release +elif [ -f /etc/debian_version ]; then + DISTRIB_ID=Debian +elif [ -f /etc/centos-release ]; then + DISTRIB_ID=CentOS +else + echo "Unknown OS" 1>&2 + exit 1 +fi + +pkgmgr_install() { + local pkgs="$1" + + case $DISTRIB_ID in + Arch) yaourt -S --needed $pkgs ;; + Ubuntu) ;& + Debian) apt-get install $pkgs ;; + Fedora) ;& + CentOS) ;& + RedHat) yum install $pkgs ;; + esac +} + +post_install() { + case $DISTRIB_ID in + RedHat) ;& + CentOS) gem install asciidoctor ;; + esac +} + +main() { + local pkgs="" + + pkgs+="autoconf automake libtool " + pkgs+="make " + + case $DISTRIB_ID in + Arch) pkgs+="pkg-config " ;; + Ubuntu) ;& + Debian) pkgs+="pkg-config " ;; + Fedora) ;& + CentOS) ;& + RedHat) pkgs+="pkgconfig " ;; + + esac + + pkgs+="doxygen " + pkgs+="graphviz " + case $DISTRIB_ID in + Arch) ;& + Ubuntu) ;& + Debian) ;& + Fedora) pkgs+="asciidoctor " ;; + RedHat) ;& + CentOS) pkgs+="gem " ;; + esac + case $DISTRIB_ID in + # FIXME: RPMs are no longer hosted here. Cannot install mscgen + # on these distros. + Fedora) + wget -O /etc/yum.repos.d/mscgen.repo \ + http://www.mcternan.me.uk/mscgen/yum/fedora/mscgen.repo ;; + RedHat) ;& + CentOS) + wget -O /etc/yum.repos.d/mscgen.repo \ + http://www.mcternan.me.uk/mscgen/yum/epel/mscgen.repo ;; + esac + pkgs+="mscgen " + + case $DISTRIB_ID in + Arch) pkgs+="cunit " ;; + Ubuntu) ;& + Debian) pkgs+="libcunit1-dev " ;; + esac + + case $DISTRIB_ID in + Arch) pkgs+="openssl " ;; + Ubuntu) ;& + Debian) pkgs+="libssl-dev " ;; + CentOS) ;& + RedHat) ;& + Fedora) pkgs+="openssl-devel " ;; + esac + + case $DISTRIB_ID in + Arch) pkgs+="linux-headers " ;; + esac + + case $DISTRIB_ID in + Arch) pkgs+="libpcap " ;; + Ubuntu) ;& + Debian) pkgs+="libpcap-dev " ;; + esac + + # Uncomment for distro's version of netmap and dpdk + #pkgs+="netmap " + #pkgs+="dpdk " + + pkgs+="valgrind " + + case $DISTRIB_ID in + Arch) ;& + Ubuntu) ;& + Debian) pkgs+="codespell " ;; + # TODO: Install RPM repos for pip and post_install + # codespell via pip. + Fedora) ;& + RedHat) ;& + CentOS) ;; + esac + + pkgmgr_install "$pkgs" + + post_install +} +main "$@"