From patchwork Thu Sep 14 09:00:10 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 112523 Delivered-To: patch@linaro.org Received: by 10.140.106.117 with SMTP id d108csp523638qgf; Thu, 14 Sep 2017 02:17:33 -0700 (PDT) X-Google-Smtp-Source: AOwi7QBWIUBJo8aqw620txx4TnxMt059xtt5klHhkUvIykDM/cSt3IsusuBcnLcSOJU2GyZwsr9o X-Received: by 10.55.200.210 with SMTP id t79mr1799463qkl.140.1505380653749; Thu, 14 Sep 2017 02:17:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1505380653; cv=none; d=google.com; s=arc-20160816; b=TpJY+7wbMc/+k03r48SMbC8DA4C7WFen66Y5ikBf3VL7Mcum2P9t1kgbjNjrNk65WG 8O66PjfSF3UHFbtgrrccPbXPxeGrXxgN8dG/1H0D8MHaE/if3I4nWDp0uAUuL/6OsOCI E0KQpq3x5iG1jKRHD58xq361ugTeWGoWPcWc894Jh3c2wpnQuyQvf1G/gUac31dnDv/u jLx2Qjykwyg8O63TEBaN874CxRVjFK3pFfI8I2KaPw1vZD0vm9zdjuN8aP/3xZNBdgTz Bkr6xOO+pBLnMQqHv+6EuBZtStnMr5O/ZWMjwDN3JBlDNdt9MqjvsahlhNh4ZUOeeSUg iZ/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=NkT71emuE/8JJp2STqQjcaNSJQ+jwW25NMf1f2kYD5M=; b=PTFZvK22/hY+Q8JzxDZdweCJnzTNHOmcXrNCjSJR4ESQvRlUZmD2M8zDF+giXG4ZCm 9TjAjVbLwz1NP+p2BUQMOfMRMY69N9aeePTRNKTtNK7fQWbDuLwPJH5boZ2iPF4k84Iv NoxqwOgbhV7HKQTVoN6w0ZeLvtZon6eRgK9ZeamPYKuJZ5ArfJhgz/jI6ugCsLhsPePs EtFM9VP12sgeKRRutt7T5ELnWR2SAAfOQ/knfSOA/IsLaSvPjJ4Gbv8maJMPVn9667pr XMHjOl5o/L2ydXOcni2no5oOd8bAD9/b7Uzt2A9zqczY1YyOr/Jr6+l+55zNp2yaH443 uOPA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id a124si3691354qkc.87.2017.09.14.02.17.33; Thu, 14 Sep 2017 02:17:33 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 5F80862D05; Thu, 14 Sep 2017 09:17:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 4B7B363E50; Thu, 14 Sep 2017 09:05:16 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 41646636AF; Thu, 14 Sep 2017 09:05:06 +0000 (UTC) Received: from forward100j.mail.yandex.net (forward100j.mail.yandex.net [5.45.198.240]) by lists.linaro.org (Postfix) with ESMTPS id 67D70609DA for ; Thu, 14 Sep 2017 09:01:04 +0000 (UTC) Received: from mxback3j.mail.yandex.net (mxback3j.mail.yandex.net [IPv6:2a02:6b8:0:1619::10c]) by forward100j.mail.yandex.net (Yandex) with ESMTP id F3D925D8465F for ; Thu, 14 Sep 2017 12:01:01 +0300 (MSK) Received: from smtp2j.mail.yandex.net (smtp2j.mail.yandex.net [2a02:6b8:0:801::ac]) by mxback3j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id MnSd2YIB11-11rK9OmO; Thu, 14 Sep 2017 12:01:01 +0300 Received: by smtp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id Aoyyu0Wxlx-11Xq8S8v; Thu, 14 Sep 2017 12:01:01 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Github ODP bot To: lng-odp@lists.linaro.org Date: Thu, 14 Sep 2017 12:00:10 +0300 Message-Id: <1505379610-1146-12-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1505379610-1146-1-git-send-email-odpbot@yandex.ru> References: <1505379610-1146-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 170 Subject: [lng-odp] [PATCH v3 11/11] linux-gen: packet: optimize header layout X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Petri Savolainen Pack most often used data into the first and the last cache line of buffer header, and the first cache line of packet header. Reduce offsets and head-/tailroom fields to 16 bits, since those are in maximum as large as a segment, usually much less. Pack header fields into correct alignments (when possible), so that holes are avoided. These changes reduce packet header size by one cache line. Signed-off-by: Petri Savolainen --- /** Email created from pull request 170 (psavol:master-packet-ref-rework) ** https://github.com/Linaro/odp/pull/170 ** Patch: https://github.com/Linaro/odp/pull/170.patch ** Base sha: fb3f36cec108ce9c55241d9f0e66d4832a552b8a ** Merge commit sha: 3baec92b29f68295264bb3a780277f7f1b6cbb83 **/ include/odp/arch/default/api/abi/packet.h | 2 +- .../include/odp/api/plat/packet_inlines.h | 4 ++-- .../linux-generic/include/odp/api/plat/packet_types.h | 2 +- platform/linux-generic/include/odp_buffer_internal.h | 16 ++++++++-------- platform/linux-generic/include/odp_packet_internal.h | 19 ++++++++++++------- platform/linux-generic/odp_pool.c | 3 +++ 6 files changed, 27 insertions(+), 19 deletions(-) diff --git a/include/odp/arch/default/api/abi/packet.h b/include/odp/arch/default/api/abi/packet.h index 4aac75b9..15cf081b 100644 --- a/include/odp/arch/default/api/abi/packet.h +++ b/include/odp/arch/default/api/abi/packet.h @@ -28,7 +28,7 @@ typedef _odp_abi_packet_seg_t *odp_packet_seg_t; #define ODP_PACKET_INVALID ((odp_packet_t)0xffffffff) #define ODP_PACKET_SEG_INVALID ((odp_packet_seg_t)0xffffffff) -#define ODP_PACKET_OFFSET_INVALID (0x0fffffff) +#define ODP_PACKET_OFFSET_INVALID 0xffff typedef enum { ODP_PACKET_GREEN = 0, diff --git a/platform/linux-generic/include/odp/api/plat/packet_inlines.h b/platform/linux-generic/include/odp/api/plat/packet_inlines.h index d0cf1390..6874d149 100644 --- a/platform/linux-generic/include/odp/api/plat/packet_inlines.h +++ b/platform/linux-generic/include/odp/api/plat/packet_inlines.h @@ -56,13 +56,13 @@ static inline uint32_t _odp_packet_len(odp_packet_t pkt) /** @internal Inline function @param pkt @return */ static inline uint32_t _odp_packet_headroom(odp_packet_t pkt) { - return _odp_pkt_get(pkt, uint32_t, headroom); + return _odp_pkt_get(pkt, uint16_t, headroom); } /** @internal Inline function @param pkt @return */ static inline uint32_t _odp_packet_tailroom(odp_packet_t pkt) { - return _odp_pkt_get(pkt, uint32_t, tailroom); + return _odp_pkt_get(pkt, uint16_t, tailroom); } /** @internal Inline function @param pkt @return */ diff --git a/platform/linux-generic/include/odp/api/plat/packet_types.h b/platform/linux-generic/include/odp/api/plat/packet_types.h index 96379ebf..68c66312 100644 --- a/platform/linux-generic/include/odp/api/plat/packet_types.h +++ b/platform/linux-generic/include/odp/api/plat/packet_types.h @@ -36,7 +36,7 @@ typedef ODP_HANDLE_T(odp_packet_t); #define ODP_PACKET_INVALID _odp_cast_scalar(odp_packet_t, 0) -#define ODP_PACKET_OFFSET_INVALID (0x0fffffff) +#define ODP_PACKET_OFFSET_INVALID 0xffff typedef uint8_t odp_packet_seg_t; diff --git a/platform/linux-generic/include/odp_buffer_internal.h b/platform/linux-generic/include/odp_buffer_internal.h index cd067a08..5d40303b 100644 --- a/platform/linux-generic/include/odp_buffer_internal.h +++ b/platform/linux-generic/include/odp_buffer_internal.h @@ -65,13 +65,10 @@ struct odp_buffer_hdr_t { /* Initial buffer data pointer */ uint8_t *base_data; - /* Reference count */ - odp_atomic_u32_t ref_cnt; - - /* Event type. Maybe different than pool type (crypto compl event) */ - int8_t event_type; + /* Pool pointer */ + void *pool_ptr; - /* --- 37 bytes --- */ + /* --- 40 bytes --- */ /* Segments */ seg_entry_t seg[CONFIG_PACKET_MAX_SEGS]; @@ -95,8 +92,11 @@ struct odp_buffer_hdr_t { const void *buf_cctx; /* const alias for ctx */ }; - /* Pool pointer */ - void *pool_ptr; + /* Reference count */ + odp_atomic_u32_t ref_cnt; + + /* Event type. Maybe different than pool type (crypto compl event) */ + int8_t event_type; /* Initial buffer tail pointer */ uint8_t *buf_end; diff --git a/platform/linux-generic/include/odp_packet_internal.h b/platform/linux-generic/include/odp_packet_internal.h index d8d4584a..d76d7bf0 100644 --- a/platform/linux-generic/include/odp_packet_internal.h +++ b/platform/linux-generic/include/odp_packet_internal.h @@ -88,9 +88,14 @@ typedef struct { error_flags_t error_flags; output_flags_t output_flags; - uint32_t l2_offset; /**< offset to L2 hdr, e.g. Eth */ - uint32_t l3_offset; /**< offset to L3 hdr, e.g. IPv4, IPv6 */ - uint32_t l4_offset; /**< offset to L4 hdr (TCP, UDP, SCTP, also ICMP) */ + /* offset to L2 hdr, e.g. Eth */ + uint16_t l2_offset; + + /* offset to L3 hdr, e.g. IPv4, IPv6 */ + uint16_t l3_offset; + + /* offset to L4 hdr (TCP, UDP, SCTP, also ICMP) */ + uint16_t l4_offset; } packet_parser_t; /* Packet extra data length */ @@ -116,14 +121,14 @@ typedef struct { packet_parser_t p; - uint32_t frame_len; - odp_pktio_t input; - uint32_t headroom; - uint32_t tailroom; + uint32_t frame_len; uint32_t shared_len; + uint16_t headroom; + uint16_t tailroom; + /* * Members below are not initialized by packet_init() */ diff --git a/platform/linux-generic/odp_pool.c b/platform/linux-generic/odp_pool.c index 2f65cb20..2a0a35e4 100644 --- a/platform/linux-generic/odp_pool.c +++ b/platform/linux-generic/odp_pool.c @@ -41,6 +41,9 @@ ODP_STATIC_ASSERT(CONFIG_POOL_CACHE_SIZE > (2 * CACHE_BURST), ODP_STATIC_ASSERT(CONFIG_PACKET_SEG_LEN_MIN >= 256, "ODP Segment size must be a minimum of 256 bytes"); +ODP_STATIC_ASSERT(CONFIG_PACKET_SEG_SIZE < 0xffff, + "Segment size must be less than 64k (16 bit offsets)"); + /* Thread local variables */ typedef struct pool_local_t { pool_cache_t *cache[ODP_CONFIG_POOLS];