From patchwork Tue Aug 1 15:00:10 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 109181 Delivered-To: patch@linaro.org Received: by 10.140.101.6 with SMTP id t6csp1615492qge; Tue, 1 Aug 2017 08:09:10 -0700 (PDT) X-Received: by 10.55.191.7 with SMTP id p7mr7614158qkf.288.1501600150770; Tue, 01 Aug 2017 08:09:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501600150; cv=none; d=google.com; s=arc-20160816; b=hmTP3+jf5YUnXPNIUNzRUWcjILf4luUEAJhX73yMQvKHpKyUgG7dVLw8kaPq+AcR+s Ed9B53GAgswG7b1CNZ8Up4IbOiKXOqJtv8Eyc3Cb9d0i1UguYn+uPCvZ5TsJPDdSyPox U6/WKRIh80u30i4uelLGesBR/2BO/awQWVnj5o/vJydTnKyPIooZ3Ah/wAJ89kBfAuop aE3Z9VHHuD0mCaNOqgMiTj+bh7hbi3fm+FStSatabDSfufGdkrdI1BNnfdQQAxuQC1GH TeKuB8Kr5WP25jwJObByxTOw/lk3hqBhhEDXKcSDOa07Eoo7ayVyHgLXGtssawYMbVln 54WQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=DdqBD5yOrNUsElrDHTgKNiZT04w+Gz56aLpOeAPO/5s=; b=VTCPBjwbJ91H8JS+05o0nfOGYHE50SgAvBuIMlJ/HmCVfeY+6Bet5UKeuvodpItnCs b8hx9Uv70dJKfraf8y2VcPsPg+ZGuPN/hXUEfSGROba7pdDBiVCGHHG562FJB0cKnxK9 P+69VlW6U6Sj9EimS+us5EJWqlTKugpqqlqKdMtFzOhE23El82fB4N4Cm++LywXaM0iI VaHNcIsRjCUkWUDsY01fU3Fby6yJ6DrWoH+0SVNYe53MdJevZGsGXQetBg/vA3jozdWP DuRXuIErzO+4X+PYbuLgopUmRQbL4sC0fWmyBAB1rr2Qgpi9N5tcxcjnDWb2ztxCwU7r ttaQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id w6si18627325qti.517.2017.08.01.08.09.10; Tue, 01 Aug 2017 08:09:10 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 6346765BCE; Tue, 1 Aug 2017 15:09:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 10AAA60C06; Tue, 1 Aug 2017 15:03:04 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id B8CEB646DE; Tue, 1 Aug 2017 15:02:53 +0000 (UTC) Received: from forward2m.cmail.yandex.net (forward2m.cmail.yandex.net [5.255.216.20]) by lists.linaro.org (Postfix) with ESMTPS id C3E4765897 for ; Tue, 1 Aug 2017 15:00:40 +0000 (UTC) Received: from smtp2j.mail.yandex.net (smtp2j.mail.yandex.net [95.108.130.60]) by forward2m.cmail.yandex.net (Yandex) with ESMTP id 718CF210E0 for ; Tue, 1 Aug 2017 18:00:39 +0300 (MSK) Received: from smtp2j.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp2j.mail.yandex.net (Yandex) with ESMTP id C77D73EC1192 for ; Tue, 1 Aug 2017 18:00:37 +0300 (MSK) Received: by smtp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id tyzQTD0Qgu-0abOS3pf; Tue, 01 Aug 2017 18:00:36 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Suid-Status: 1 0 From: Github ODP bot To: lng-odp@lists.linaro.org Date: Tue, 1 Aug 2017 18:00:10 +0300 Message-Id: <1501599610-28388-11-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> References: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 90 Subject: [lng-odp] [PATCH v3 10/10] linux-generic: packet: enable zero-copy reference creation X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Bill Fischofer Signed-off-by: Bill Fischofer --- /** Email created from pull request 90 (Bill-Fischofer-Linaro:pktrefs) ** https://github.com/Linaro/odp/pull/90 ** Patch: https://github.com/Linaro/odp/pull/90.patch ** Base sha: bb0ca193f8c28036fdf8b3d6c8aa5d4f11980e6c ** Merge commit sha: dfef320840f96825686124135a6e509db80b7b82 **/ platform/linux-generic/odp_packet.c | 172 +++++++++++++++++++++--------------- 1 file changed, 103 insertions(+), 69 deletions(-) diff --git a/platform/linux-generic/odp_packet.c b/platform/linux-generic/odp_packet.c index 5d4be44e..f18bd4dd 100644 --- a/platform/linux-generic/odp_packet.c +++ b/platform/linux-generic/odp_packet.c @@ -1710,6 +1710,109 @@ int odp_packet_split(odp_packet_t *pkt, uint32_t len, odp_packet_t *tail) } /* + * References + */ + +static inline void packet_ref(odp_packet_hdr_t *pkt_hdr) +{ + uint32_t i; + odp_packet_hdr_t *hdr; + + do { + for (i = 0; i < pkt_hdr->buf_hdr.segcount; i++) { + hdr = pkt_hdr->buf_hdr.seg[i].hdr; + packet_ref_inc(hdr); + } + + pkt_hdr = pkt_hdr->ref_hdr; + } while (pkt_hdr); +} + +static inline odp_packet_t packet_splice(odp_packet_hdr_t *pkt_hdr, + uint32_t offset, + odp_packet_hdr_t *ref_hdr) +{ + /* Catch attempted references to stale handles in debug builds */ + ODP_ASSERT(packet_ref_count(pkt_hdr) > 0); + + /* Splicing is from the last section of src pkt */ + while (ref_hdr->ref_hdr) + ref_hdr = ref_hdr->ref_hdr; + + /* Find section where splice begins */ + while (offset >= pkt_hdr->frame_len && pkt_hdr->ref_hdr) { + offset -= (pkt_hdr->frame_len - pkt_hdr->ref_offset); + offset += (pkt_hdr->ref_hdr->frame_len - pkt_hdr->ref_len); + pkt_hdr = pkt_hdr->ref_hdr; + } + + ref_hdr->ref_hdr = pkt_hdr; + ref_hdr->ref_offset = offset; + ref_hdr->ref_len = pkt_hdr->frame_len; + + if (packet_ref_count(pkt_hdr) == 1 || offset < pkt_hdr->unshared_len) + pkt_hdr->unshared_len = offset; + + packet_ref(pkt_hdr); + return packet_handle(ref_hdr); +} + +odp_packet_t odp_packet_ref_static(odp_packet_t pkt) +{ + odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); + + packet_ref(pkt_hdr); + pkt_hdr->unshared_len = 0; + return pkt; +} + +odp_packet_t odp_packet_ref(odp_packet_t pkt, uint32_t offset) +{ + odp_packet_t hdr; + odp_packet_hdr_t *pkt_hdr; + + if (pkt == ODP_PACKET_INVALID) + return ODP_PACKET_INVALID; + + pkt_hdr = packet_hdr(pkt); + if (offset >= packet_len(pkt_hdr)) + return ODP_PACKET_INVALID; + + hdr = odp_packet_alloc(odp_packet_pool(pkt), 0); + + if (hdr == ODP_PACKET_INVALID) + return ODP_PACKET_INVALID; + + return packet_splice(pkt_hdr, offset, packet_hdr(hdr)); +} + +odp_packet_t odp_packet_ref_pkt(odp_packet_t pkt, uint32_t offset, + odp_packet_t hdr) +{ + odp_packet_hdr_t *pkt_hdr; + + if (pkt == ODP_PACKET_INVALID || + hdr == ODP_PACKET_INVALID || + pkt == hdr) + return ODP_PACKET_INVALID; + + ODP_ASSERT(odp_packet_has_ref(hdr) == 0); + + pkt_hdr = packet_hdr(pkt); + if (offset >= packet_len(pkt_hdr)) + return ODP_PACKET_INVALID; + + return packet_splice(pkt_hdr, offset, packet_hdr(hdr)); +} + +int odp_packet_has_ref(odp_packet_t pkt) +{ + odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); + + return pkt_hdr->ref_hdr != NULL || packet_ref_count(pkt_hdr) > 1; +} + +/* * * Copy * ******************************************************** @@ -2353,75 +2456,6 @@ uint64_t odp_packet_seg_to_u64(odp_packet_seg_t hdl) return _odp_pri(hdl); } -odp_packet_t odp_packet_ref_static(odp_packet_t pkt) -{ - return odp_packet_copy(pkt, odp_packet_pool(pkt)); -} - -odp_packet_t odp_packet_ref(odp_packet_t pkt, uint32_t offset) -{ - odp_packet_t new; - int ret; - - new = odp_packet_copy(pkt, odp_packet_pool(pkt)); - - if (new == ODP_PACKET_INVALID) { - ODP_ERR("copy failed\n"); - return ODP_PACKET_INVALID; - } - - ret = odp_packet_trunc_head(&new, offset, NULL, NULL); - - if (ret < 0) { - ODP_ERR("trunk_head failed\n"); - odp_packet_free(new); - return ODP_PACKET_INVALID; - } - - return new; -} - -odp_packet_t odp_packet_ref_pkt(odp_packet_t pkt, uint32_t offset, - odp_packet_t hdr) -{ - odp_packet_t new; - int ret; - - new = odp_packet_copy(pkt, odp_packet_pool(pkt)); - - if (new == ODP_PACKET_INVALID) { - ODP_ERR("copy failed\n"); - return ODP_PACKET_INVALID; - } - - if (offset) { - ret = odp_packet_trunc_head(&new, offset, NULL, NULL); - - if (ret < 0) { - ODP_ERR("trunk_head failed\n"); - odp_packet_free(new); - return ODP_PACKET_INVALID; - } - } - - ret = odp_packet_concat(&hdr, new); - - if (ret < 0) { - ODP_ERR("concat failed\n"); - odp_packet_free(new); - return ODP_PACKET_INVALID; - } - - return hdr; -} - -int odp_packet_has_ref(odp_packet_t pkt) -{ - (void)pkt; - - return 0; -} - /* Include non-inlined versions of API functions */ #if ODP_ABI_COMPAT == 1 #include