From patchwork Fri Sep 8 14:00:12 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 112090 Delivered-To: patch@linaro.org Received: by 10.37.128.210 with SMTP id c18csp1677065ybm; Fri, 8 Sep 2017 07:12:53 -0700 (PDT) X-Google-Smtp-Source: AOwi7QD0m3nIxcEfqr5TYm+ikk+AS4zeODKA3omFaU6IPgGBCFh+nJFExHTFYNh8alEK+KwrCMUR X-Received: by 10.55.97.203 with SMTP id v194mr3665458qkb.305.1504879973258; Fri, 08 Sep 2017 07:12:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1504879973; cv=none; d=google.com; s=arc-20160816; b=UpCSG1tVUCqWjwg96qgACy+xGiaKVu36eWpaoYPaeGditiQRY+EBevJfChsupR8ti8 qfzE9jin7zZmzn6N0ShJk7YRwQ5BldJYZsKroeLgzmn+5t2T6jSVibk9gKuHnC3wWKKi bglamZnPlMrP4wb/5+YaXBDDiEbnnIaDBCA06/3ngtfAMjOmYgWBVDoBQPJXwBi36pJf 7vMR1SRP+YyjK33Rp5JWHA2+HYNhS1UPPsG4r41b9cdlXq+UZUhidkSDCVPWzmCa60Pz IxnmLmBSuy31DU2JJbweX9CnJVwP8OMpijckVATgRLZeYdngcUnVRN9XmF44bNVE6/fV PAzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=YDXWRxlW99Pd6vN0mmkNos+UWCfTfo89GVyQjf20BgA=; b=m2bDtWCXYaA/iRhkoaSuV3bBAVvL1ocd8Kl24LsiPWby6wFGFip9ZiNTZKC7J3743N ZBLRxCjO/sW46QnRnNxo0AKG3ChvT/WcD2+q5/cen18PqX+8a/himb9ndzXOC/BVF3CJ kJu09Jhs/628kxtbPp37PK2DM4Dg/HB85i/eLEvx1k0C4AKTasW6GNGYCWREqESCAd5/ HZyTJLRNQTN3i7V7VQnslEWas54dOkxnHQvLAQtDaA8/L9Koe1bysGCZTQeBgrLKTAI8 iZ1D834oViQFasCaakpTDQhGaEt6eyO0wgOhjNXwNBMW0AXZRhxYHvFYlSh2DZCYn6Po 5ZLA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id i6si2075758qkc.424.2017.09.08.07.12.52; Fri, 08 Sep 2017 07:12:53 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id E0FB660C2D; Fri, 8 Sep 2017 14:12:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id D395560A55; Fri, 8 Sep 2017 14:01:26 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 1E6FB644B0; Fri, 8 Sep 2017 14:00:38 +0000 (UTC) Received: from forward105p.mail.yandex.net (forward105p.mail.yandex.net [77.88.28.108]) by lists.linaro.org (Postfix) with ESMTPS id 0AD6560C4C for ; Fri, 8 Sep 2017 14:00:27 +0000 (UTC) Received: from mxback8o.mail.yandex.net (mxback8o.mail.yandex.net [IPv6:2a02:6b8:0:1a2d::22]) by forward105p.mail.yandex.net (Yandex) with ESMTP id 74F53408505D for ; Fri, 8 Sep 2017 17:00:25 +0300 (MSK) Received: from smtp2p.mail.yandex.net (smtp2p.mail.yandex.net [2a02:6b8:0:1472:2741:0:8b6:7]) by mxback8o.mail.yandex.net (nwsmtp/Yandex) with ESMTP id xApzrBoIRU-0PKicVJd; Fri, 08 Sep 2017 17:00:25 +0300 Received: by smtp2p.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id IwYlQdvFwh-0OKqY48t; Fri, 08 Sep 2017 17:00:25 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Github ODP bot To: lng-odp@lists.linaro.org Date: Fri, 8 Sep 2017 17:00:12 +0300 Message-Id: <1504879212-3689-12-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1504879212-3689-1-git-send-email-odpbot@yandex.ru> References: <1504879212-3689-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 170 Subject: [lng-odp] [PATCH v1 11/11] linux-gen: packet: optimize header layout X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Petri Savolainen Pack most often used data into the first and the last cache line of buffer header, and the first cache line of packet header. Reduce offsets and head-/tailroom fields to 16 bits, since those are in maximum as large as a segment, usually much less. Pack header fields into correct alignments (when possible), so that holes are avoided. These changes reduce packet header size by one cache line. Signed-off-by: Petri Savolainen --- /** Email created from pull request 170 (psavol:master-packet-ref-rework) ** https://github.com/Linaro/odp/pull/170 ** Patch: https://github.com/Linaro/odp/pull/170.patch ** Base sha: 360c1d34b52a2356619b9290811862b9de41de00 ** Merge commit sha: e19cce86ab00ccd959d2c3c82c0786ec6da19ba7 **/ include/odp/arch/default/api/abi/packet.h | 2 +- .../include/odp/api/plat/packet_inlines.h | 4 ++-- .../linux-generic/include/odp/api/plat/packet_types.h | 2 +- platform/linux-generic/include/odp_buffer_internal.h | 16 ++++++++-------- platform/linux-generic/include/odp_packet_internal.h | 19 ++++++++++++------- platform/linux-generic/odp_pool.c | 3 +++ 6 files changed, 27 insertions(+), 19 deletions(-) diff --git a/include/odp/arch/default/api/abi/packet.h b/include/odp/arch/default/api/abi/packet.h index 4aac75b9e..15cf081bf 100644 --- a/include/odp/arch/default/api/abi/packet.h +++ b/include/odp/arch/default/api/abi/packet.h @@ -28,7 +28,7 @@ typedef _odp_abi_packet_seg_t *odp_packet_seg_t; #define ODP_PACKET_INVALID ((odp_packet_t)0xffffffff) #define ODP_PACKET_SEG_INVALID ((odp_packet_seg_t)0xffffffff) -#define ODP_PACKET_OFFSET_INVALID (0x0fffffff) +#define ODP_PACKET_OFFSET_INVALID 0xffff typedef enum { ODP_PACKET_GREEN = 0, diff --git a/platform/linux-generic/include/odp/api/plat/packet_inlines.h b/platform/linux-generic/include/odp/api/plat/packet_inlines.h index d0cf13901..6874d1496 100644 --- a/platform/linux-generic/include/odp/api/plat/packet_inlines.h +++ b/platform/linux-generic/include/odp/api/plat/packet_inlines.h @@ -56,13 +56,13 @@ static inline uint32_t _odp_packet_len(odp_packet_t pkt) /** @internal Inline function @param pkt @return */ static inline uint32_t _odp_packet_headroom(odp_packet_t pkt) { - return _odp_pkt_get(pkt, uint32_t, headroom); + return _odp_pkt_get(pkt, uint16_t, headroom); } /** @internal Inline function @param pkt @return */ static inline uint32_t _odp_packet_tailroom(odp_packet_t pkt) { - return _odp_pkt_get(pkt, uint32_t, tailroom); + return _odp_pkt_get(pkt, uint16_t, tailroom); } /** @internal Inline function @param pkt @return */ diff --git a/platform/linux-generic/include/odp/api/plat/packet_types.h b/platform/linux-generic/include/odp/api/plat/packet_types.h index 96379ebf9..68c66312b 100644 --- a/platform/linux-generic/include/odp/api/plat/packet_types.h +++ b/platform/linux-generic/include/odp/api/plat/packet_types.h @@ -36,7 +36,7 @@ typedef ODP_HANDLE_T(odp_packet_t); #define ODP_PACKET_INVALID _odp_cast_scalar(odp_packet_t, 0) -#define ODP_PACKET_OFFSET_INVALID (0x0fffffff) +#define ODP_PACKET_OFFSET_INVALID 0xffff typedef uint8_t odp_packet_seg_t; diff --git a/platform/linux-generic/include/odp_buffer_internal.h b/platform/linux-generic/include/odp_buffer_internal.h index cd067a08b..5d40303b3 100644 --- a/platform/linux-generic/include/odp_buffer_internal.h +++ b/platform/linux-generic/include/odp_buffer_internal.h @@ -65,13 +65,10 @@ struct odp_buffer_hdr_t { /* Initial buffer data pointer */ uint8_t *base_data; - /* Reference count */ - odp_atomic_u32_t ref_cnt; - - /* Event type. Maybe different than pool type (crypto compl event) */ - int8_t event_type; + /* Pool pointer */ + void *pool_ptr; - /* --- 37 bytes --- */ + /* --- 40 bytes --- */ /* Segments */ seg_entry_t seg[CONFIG_PACKET_MAX_SEGS]; @@ -95,8 +92,11 @@ struct odp_buffer_hdr_t { const void *buf_cctx; /* const alias for ctx */ }; - /* Pool pointer */ - void *pool_ptr; + /* Reference count */ + odp_atomic_u32_t ref_cnt; + + /* Event type. Maybe different than pool type (crypto compl event) */ + int8_t event_type; /* Initial buffer tail pointer */ uint8_t *buf_end; diff --git a/platform/linux-generic/include/odp_packet_internal.h b/platform/linux-generic/include/odp_packet_internal.h index d8d4584a7..d76d7bf08 100644 --- a/platform/linux-generic/include/odp_packet_internal.h +++ b/platform/linux-generic/include/odp_packet_internal.h @@ -88,9 +88,14 @@ typedef struct { error_flags_t error_flags; output_flags_t output_flags; - uint32_t l2_offset; /**< offset to L2 hdr, e.g. Eth */ - uint32_t l3_offset; /**< offset to L3 hdr, e.g. IPv4, IPv6 */ - uint32_t l4_offset; /**< offset to L4 hdr (TCP, UDP, SCTP, also ICMP) */ + /* offset to L2 hdr, e.g. Eth */ + uint16_t l2_offset; + + /* offset to L3 hdr, e.g. IPv4, IPv6 */ + uint16_t l3_offset; + + /* offset to L4 hdr (TCP, UDP, SCTP, also ICMP) */ + uint16_t l4_offset; } packet_parser_t; /* Packet extra data length */ @@ -116,14 +121,14 @@ typedef struct { packet_parser_t p; - uint32_t frame_len; - odp_pktio_t input; - uint32_t headroom; - uint32_t tailroom; + uint32_t frame_len; uint32_t shared_len; + uint16_t headroom; + uint16_t tailroom; + /* * Members below are not initialized by packet_init() */ diff --git a/platform/linux-generic/odp_pool.c b/platform/linux-generic/odp_pool.c index 2f65cb20c..2a0a35e40 100644 --- a/platform/linux-generic/odp_pool.c +++ b/platform/linux-generic/odp_pool.c @@ -41,6 +41,9 @@ ODP_STATIC_ASSERT(CONFIG_POOL_CACHE_SIZE > (2 * CACHE_BURST), ODP_STATIC_ASSERT(CONFIG_PACKET_SEG_LEN_MIN >= 256, "ODP Segment size must be a minimum of 256 bytes"); +ODP_STATIC_ASSERT(CONFIG_PACKET_SEG_SIZE < 0xffff, + "Segment size must be less than 64k (16 bit offsets)"); + /* Thread local variables */ typedef struct pool_local_t { pool_cache_t *cache[ODP_CONFIG_POOLS];