From patchwork Tue Nov 15 14:44:40 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bill Fischofer X-Patchwork-Id: 82745 Delivered-To: patch@linaro.org Received: by 10.140.97.165 with SMTP id m34csp854185qge; Thu, 17 Nov 2016 08:39:17 -0800 (PST) X-Received: by 10.129.138.70 with SMTP id a67mr3639235ywg.290.1479400757504; Thu, 17 Nov 2016 08:39:17 -0800 (PST) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id x39si892470ybi.252.2016.11.17.08.39.17; Thu, 17 Nov 2016 08:39:17 -0800 (PST) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 26F046065A; Thu, 17 Nov 2016 16:39:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id AD7D762DE7; Thu, 17 Nov 2016 16:23:59 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 5209A60C22; Tue, 15 Nov 2016 14:44:53 +0000 (UTC) Received: from mail-yw0-f174.google.com (mail-yw0-f174.google.com [209.85.161.174]) by lists.linaro.org (Postfix) with ESMTPS id 3BF1860C42 for ; Tue, 15 Nov 2016 14:44:48 +0000 (UTC) Received: by mail-yw0-f174.google.com with SMTP id t125so96646515ywc.1 for ; Tue, 15 Nov 2016 06:44:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=7qRH3WBGiPh1Iw3emtkq5Avl3goJ8lCRV/0n7/ekNHs=; b=EzBwgcxc2GQlZlNX1zjRlaccKw7HTbHpGs0EX3/VP55PZBepxaixWgUw7OWkzrvKPy DbaObSnl9mQd/IzXjRZRaG0zGnsWLiZjdwpoTlWXN1eFQxX5GeVBnrzmDJ+BjSJvAdev 6e/JDZpCezymHFLvQ73rYGCAxboUSsfe4q5x1wtvnVEC5NtZsmMNWBaHMsX8ecqJ6LaM ARMIX8eSFqCUu2+RfTwwL3GKd+vjBwfojSYpDduw8RIj63Ltk5CBcVQV8peb7paiMJOR 2IE5N0mimDhTyNkwsoPM40WkZ+gMnfGO0bSVQ80fgelbTZv7ZaOvYSGtnSuMib0tLk73 IWDQ== X-Gm-Message-State: ABUngvdvD7LLXE1iJtQtUXZUEIR8lsUO+8e2iXR1e5sA7dvUiumkpsiuwH40HMhJdws/tovzuCo= X-Received: by 10.202.220.139 with SMTP id t133mr10243472oig.176.1479221087698; Tue, 15 Nov 2016 06:44:47 -0800 (PST) Received: from localhost.localdomain (cpe-70-121-83-241.austin.res.rr.com. [70.121.83.241]) by smtp.gmail.com with ESMTPSA id d30sm5528910otb.4.2016.11.15.06.44.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 15 Nov 2016 06:44:47 -0800 (PST) From: Bill Fischofer To: lng-odp@lists.linaro.org Date: Tue, 15 Nov 2016 08:44:40 -0600 Message-Id: <1479221083-30553-3-git-send-email-bill.fischofer@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1479221083-30553-1-git-send-email-bill.fischofer@linaro.org> References: <1479221083-30553-1-git-send-email-bill.fischofer@linaro.org> X-Topics: patch Subject: [lng-odp] [API-NEXT PATCHv2 2/5] linux-generic: packet: implement reference apis X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" Implement the APIs: - odp_packet_ref_static() - odp_packet_ref() - odp_packet_ref_pkt() - odp_packet_is_ref() - odp_packet_has_ref() - odp_packet_unshared_len() This also involves functional upgrades to the existing packet manipulation APIs to work with packet references as input. Signed-off-by: Bill Fischofer --- .../linux-generic/include/odp_packet_internal.h | 49 ++- platform/linux-generic/odp_packet.c | 331 ++++++++++++++++----- 2 files changed, 308 insertions(+), 72 deletions(-) -- 2.7.4 diff --git a/platform/linux-generic/include/odp_packet_internal.h b/platform/linux-generic/include/odp_packet_internal.h index d09231e..35d1ca2 100644 --- a/platform/linux-generic/include/odp_packet_internal.h +++ b/platform/linux-generic/include/odp_packet_internal.h @@ -167,7 +167,7 @@ typedef struct { * packet_init(). Because of this any new fields added must be reviewed for * initialization requirements. */ -typedef struct { +typedef struct odp_packet_hdr_t { /* common buffer header */ odp_buffer_hdr_t buf_hdr; @@ -178,6 +178,10 @@ typedef struct { uint32_t headroom; uint32_t tailroom; + uint32_t ref_offset; + struct odp_packet_hdr_t *ref_hdr; + odp_atomic_u32_t ref_count; + odp_pktio_t input; /* Members below are not initialized by packet_init() */ @@ -200,6 +204,38 @@ static inline odp_packet_hdr_t *odp_packet_hdr(odp_packet_t pkt) return (odp_packet_hdr_t *)buf_hdl_to_hdr((odp_buffer_t)pkt); } +static inline odp_packet_hdr_t *odp_packet_last_hdr(odp_packet_t pkt, + uint32_t *offset) +{ + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + uint32_t ref_offset = 0; + + while (pkt_hdr->ref_hdr) { + ref_offset = pkt_hdr->ref_offset; + pkt_hdr = pkt_hdr->ref_hdr; + } + + if (offset) + *offset = ref_offset; + return pkt_hdr; +} + +static inline odp_packet_hdr_t *odp_packet_prev_hdr(odp_packet_hdr_t *pkt_hdr, + odp_packet_hdr_t *cur_hdr, + uint32_t *offset) +{ + uint32_t ref_offset = 0; + + while (pkt_hdr->ref_hdr != cur_hdr) { + ref_offset = pkt_hdr->ref_offset; + pkt_hdr = pkt_hdr->ref_hdr; + } + + if (offset) + *offset = ref_offset; + return pkt_hdr; +} + static inline void copy_packet_parser_metadata(odp_packet_hdr_t *src_hdr, odp_packet_hdr_t *dst_hdr) { @@ -227,7 +263,16 @@ static inline void pull_tail(odp_packet_hdr_t *pkt_hdr, uint32_t len) static inline uint32_t packet_len(odp_packet_hdr_t *pkt_hdr) { - return pkt_hdr->frame_len; + uint32_t pkt_len = 0; + uint32_t offset = 0; + + do { + pkt_len += pkt_hdr->frame_len - offset; + offset = pkt_hdr->ref_offset; + pkt_hdr = pkt_hdr->ref_hdr; + } while (pkt_hdr); + + return pkt_len; } static inline void packet_set_len(odp_packet_hdr_t *pkt_hdr, uint32_t len) diff --git a/platform/linux-generic/odp_packet.c b/platform/linux-generic/odp_packet.c index 0d3fd05..0e01c0d 100644 --- a/platform/linux-generic/odp_packet.c +++ b/platform/linux-generic/odp_packet.c @@ -36,7 +36,8 @@ static inline uint32_t packet_seg_len(odp_packet_hdr_t *pkt_hdr, return pkt_hdr->buf_hdr.seg[seg_idx].len; } -static inline void *packet_seg_data(odp_packet_hdr_t *pkt_hdr, uint32_t seg_idx) +static inline uint8_t *packet_seg_data(odp_packet_hdr_t *pkt_hdr, + uint32_t seg_idx) { return pkt_hdr->buf_hdr.seg[seg_idx].data; } @@ -137,7 +138,14 @@ static inline void *packet_map(odp_packet_hdr_t *pkt_hdr, int seg = 0; int seg_count = pkt_hdr->buf_hdr.segcount; - if (odp_unlikely(offset >= pkt_hdr->frame_len)) + /* Special processing for references */ + while (offset >= pkt_hdr->frame_len && pkt_hdr->ref_hdr) { + offset -= (pkt_hdr->frame_len - pkt_hdr->ref_offset); + pkt_hdr = pkt_hdr->ref_hdr; + seg_count = pkt_hdr->buf_hdr.segcount; + } + + if (odp_unlikely(offset > pkt_hdr->frame_len)) return NULL; if (odp_likely(CONFIG_PACKET_MAX_SEGS == 1 || seg_count == 1)) { @@ -231,6 +239,13 @@ static inline void packet_init(odp_packet_hdr_t *pkt_hdr, uint32_t len, CONFIG_PACKET_TAILROOM; pkt_hdr->input = ODP_PKTIO_INVALID; + + /* By default packet has no references */ + pkt_hdr->ref_hdr = NULL; + pkt_hdr->ref_offset = 0; + + /* Start with a ref count of 1 since packet is allocated */ + odp_atomic_init_u32(&pkt_hdr->ref_count, 1); } static inline odp_packet_hdr_t *init_segments(odp_buffer_t buf[], int num) @@ -516,54 +531,49 @@ int odp_packet_alloc_multi(odp_pool_t pool_hdl, uint32_t len, return num; } -void odp_packet_free(odp_packet_t pkt) +static inline void packet_free(odp_packet_hdr_t *pkt_hdr) { - odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); - int num_seg = pkt_hdr->buf_hdr.segcount; + odp_packet_hdr_t *ref_hdr; + int num_seg; - if (odp_likely(CONFIG_PACKET_MAX_SEGS == 1 || num_seg == 1)) { - buffer_free_multi((odp_buffer_t *)&pkt, 1); - } else { - odp_buffer_t buf[num_seg]; - int i; + do { + num_seg = pkt_hdr->buf_hdr.segcount; + ref_hdr = pkt_hdr->ref_hdr; - buf[0] = (odp_buffer_t)pkt; + if (odp_atomic_fetch_dec_u32(&pkt_hdr->ref_count) == 1) { + if (odp_likely(CONFIG_PACKET_MAX_SEGS == 1 || + num_seg == 1)) { + buffer_free_multi + (&pkt_hdr->buf_hdr.handle.handle, 1); + } else { + odp_buffer_t buf[num_seg]; + int i; - for (i = 1; i < num_seg; i++) - buf[i] = buffer_handle(pkt_hdr->buf_hdr.seg[i].hdr); + buf[0] = pkt_hdr->buf_hdr.handle.handle; - buffer_free_multi(buf, num_seg); - } -} + for (i = 1; i < num_seg; i++) + buf[i] = buffer_handle + (pkt_hdr->buf_hdr.seg[i].hdr); -void odp_packet_free_multi(const odp_packet_t pkt[], int num) -{ - if (CONFIG_PACKET_MAX_SEGS == 1) { - buffer_free_multi((const odp_buffer_t * const)pkt, num); - } else { - odp_buffer_t buf[num * CONFIG_PACKET_MAX_SEGS]; - int i, j; - int bufs = 0; - - for (i = 0; i < num; i++) { - odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt[i]); - int num_seg = pkt_hdr->buf_hdr.segcount; - odp_buffer_hdr_t *buf_hdr = &pkt_hdr->buf_hdr; + buffer_free_multi(buf, num_seg); + } + } - buf[bufs] = (odp_buffer_t)pkt[i]; - bufs++; + pkt_hdr = ref_hdr; + } while (pkt_hdr); +} - if (odp_likely(num_seg == 1)) - continue; +void odp_packet_free(odp_packet_t pkt) +{ + packet_free(odp_packet_hdr(pkt)); +} - for (j = 1; j < num_seg; j++) { - buf[bufs] = buffer_handle(buf_hdr->seg[j].hdr); - bufs++; - } - } +void odp_packet_free_multi(const odp_packet_t pkt[], int num) +{ + int i; - buffer_free_multi(buf, bufs); - } + for (i = 0; i < num; i++) + packet_free(odp_packet_hdr(pkt[i])); } int odp_packet_reset(odp_packet_t pkt, uint32_t len) @@ -574,6 +584,9 @@ int odp_packet_reset(odp_packet_t pkt, uint32_t len) if (len > pool->headroom + pool->data_size + pool->tailroom) return -1; + if (pkt_hdr->ref_hdr) + packet_free(pkt_hdr->ref_hdr); + packet_init(pkt_hdr, len, 0); return 0; @@ -616,8 +629,14 @@ void *odp_packet_head(odp_packet_t pkt) uint32_t odp_packet_buf_len(odp_packet_t pkt) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + uint32_t buf_len = 0; + + do { + buf_len += pkt_hdr->buf_hdr.size * pkt_hdr->buf_hdr.segcount; + pkt_hdr = pkt_hdr->ref_hdr; + } while (pkt_hdr); - return pkt_hdr->buf_hdr.size * pkt_hdr->buf_hdr.segcount; + return buf_len; } void *odp_packet_data(odp_packet_t pkt) @@ -636,7 +655,23 @@ uint32_t odp_packet_seg_len(odp_packet_t pkt) uint32_t odp_packet_len(odp_packet_t pkt) { - return odp_packet_hdr(pkt)->frame_len; + return packet_len(odp_packet_hdr(pkt)); +} + +uint32_t odp_packet_unshared_len(odp_packet_t pkt) +{ + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + uint32_t pkt_len = 0, offset = 0; + + do { + if (odp_atomic_load_u32(&pkt_hdr->ref_count) > 1) + break; + pkt_len += pkt_hdr->frame_len - offset; + offset = pkt_hdr->ref_offset; + pkt_hdr = pkt_hdr->ref_hdr; + } while (pkt_hdr); + + return pkt_len; } uint32_t odp_packet_headroom(odp_packet_t pkt) @@ -646,12 +681,12 @@ uint32_t odp_packet_headroom(odp_packet_t pkt) uint32_t odp_packet_tailroom(odp_packet_t pkt) { - return odp_packet_hdr(pkt)->tailroom; + return odp_packet_last_hdr(pkt, NULL)->tailroom; } void *odp_packet_tail(odp_packet_t pkt) { - odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + odp_packet_hdr_t *pkt_hdr = odp_packet_last_hdr(pkt, NULL); return packet_tail(pkt_hdr); } @@ -713,12 +748,24 @@ void *odp_packet_pull_head(odp_packet_t pkt, uint32_t len) int odp_packet_trunc_head(odp_packet_t *pkt, uint32_t len, void **data_ptr, uint32_t *seg_len_out) { - odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(*pkt); + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(*pkt), *nxt_hdr; uint32_t seg_len = packet_first_seg_len(pkt_hdr); - if (len > pkt_hdr->frame_len) + if (len > packet_len(pkt_hdr)) return -1; + /* Special processing for references */ + while (len >= pkt_hdr->frame_len && pkt_hdr->ref_hdr) { + ODP_ASSERT(odp_atomic_load_u32(&pkt_hdr->ref_count) == 1); + len -= (pkt_hdr->frame_len - pkt_hdr->ref_offset); + nxt_hdr = pkt_hdr->ref_hdr; + pkt_hdr->ref_hdr = NULL; + packet_free(pkt_hdr); + pkt_hdr = nxt_hdr; + seg_len = packet_first_seg_len(pkt_hdr); + *pkt = packet_handle(pkt_hdr); + } + if (len < seg_len) { pull_head(pkt_hdr, len); } else if (CONFIG_PACKET_MAX_SEGS != 1) { @@ -747,7 +794,7 @@ int odp_packet_trunc_head(odp_packet_t *pkt, uint32_t len, void *odp_packet_push_tail(odp_packet_t pkt, uint32_t len) { - odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + odp_packet_hdr_t *pkt_hdr = odp_packet_last_hdr(pkt, NULL); void *old_tail; if (len > pkt_hdr->tailroom) @@ -762,7 +809,7 @@ void *odp_packet_push_tail(odp_packet_t pkt, uint32_t len) int odp_packet_extend_tail(odp_packet_t *pkt, uint32_t len, void **data_ptr, uint32_t *seg_len) { - odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(*pkt); + odp_packet_hdr_t *pkt_hdr = odp_packet_last_hdr(*pkt, NULL); void *ret; uint32_t tailroom = pkt_hdr->tailroom; uint32_t tail_off = pkt_hdr->frame_len; @@ -803,11 +850,25 @@ int odp_packet_trunc_tail(odp_packet_t *pkt, uint32_t len, { int last; uint32_t seg_len; - odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(*pkt); + uint32_t offset; + odp_packet_hdr_t *first_hdr = odp_packet_hdr(*pkt); + odp_packet_hdr_t *pkt_hdr, *prev_hdr; - if (len > pkt_hdr->frame_len) + if (len > packet_len(first_hdr)) return -1; + pkt_hdr = odp_packet_last_hdr(*pkt, &offset); + + /* Special processing for references */ + while (len >= pkt_hdr->frame_len - offset && first_hdr->ref_hdr) { + len -= (pkt_hdr->frame_len - offset); + prev_hdr = odp_packet_prev_hdr(first_hdr, pkt_hdr, &offset); + ODP_ASSERT(odp_atomic_load_u32(&pkt_hdr->ref_count) == 1); + prev_hdr->ref_hdr = NULL; + packet_free(pkt_hdr); + pkt_hdr = prev_hdr; + } + last = packet_last_seg(pkt_hdr); seg_len = packet_seg_len(pkt_hdr, last); @@ -1020,35 +1081,50 @@ void odp_packet_ts_set(odp_packet_t pkt, odp_time_t timestamp) int odp_packet_is_segmented(odp_packet_t pkt) { - return odp_packet_hdr(pkt)->buf_hdr.segcount > 1; + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + + return pkt_hdr->buf_hdr.segcount > 1 || pkt_hdr->ref_hdr != NULL; } int odp_packet_num_segs(odp_packet_t pkt) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + uint32_t segcount = 0, i; + uint32_t seg_offset = 0, offset; + + do { + segcount += pkt_hdr->buf_hdr.segcount - seg_offset; + offset = pkt_hdr->ref_offset; + pkt_hdr = pkt_hdr->ref_hdr; + if (pkt_hdr) { + for (i = 0, seg_offset = 0; + i < pkt_hdr->buf_hdr.segcount; + i++, seg_offset++) { + if (offset < pkt_hdr->buf_hdr.seg[i].len) + break; + offset -= pkt_hdr->buf_hdr.seg[i].len; + } + } + } while (pkt_hdr); - return pkt_hdr->buf_hdr.segcount; + return segcount; } -odp_packet_seg_t odp_packet_first_seg(odp_packet_t pkt) +odp_packet_seg_t odp_packet_first_seg(odp_packet_t pkt ODP_UNUSED) { - (void)pkt; - return 0; } odp_packet_seg_t odp_packet_last_seg(odp_packet_t pkt) { - odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); - - return packet_last_seg(pkt_hdr); + return (odp_packet_seg_t)(odp_packet_num_segs(pkt) - 1); } odp_packet_seg_t odp_packet_next_seg(odp_packet_t pkt, odp_packet_seg_t seg) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); - if (odp_unlikely(seg >= (odp_packet_seg_t)packet_last_seg(pkt_hdr))) + if (odp_unlikely(seg >= packet_last_seg(pkt_hdr))) return ODP_PACKET_SEG_INVALID; return seg + 1; @@ -1064,21 +1140,51 @@ odp_packet_seg_t odp_packet_next_seg(odp_packet_t pkt, odp_packet_seg_t seg) void *odp_packet_seg_data(odp_packet_t pkt, odp_packet_seg_t seg) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + uint32_t seg_offset = 0, offset = 0, i; + + while (seg >= pkt_hdr->buf_hdr.segcount - seg_offset && + pkt_hdr->ref_hdr) { + seg -= (pkt_hdr->buf_hdr.segcount - seg_offset); + offset = pkt_hdr->ref_offset; + pkt_hdr = pkt_hdr->ref_hdr; + for (i = 0, seg_offset = 0; + i < pkt_hdr->buf_hdr.segcount; + i++, seg_offset++) { + if (offset < pkt_hdr->buf_hdr.seg[i].len) + break; + offset -= pkt_hdr->buf_hdr.seg[i].len; + } + } - if (odp_unlikely(seg >= pkt_hdr->buf_hdr.segcount)) + if (odp_unlikely(seg + seg_offset >= pkt_hdr->buf_hdr.segcount)) return NULL; - return packet_seg_data(pkt_hdr, seg); + return packet_seg_data(pkt_hdr, seg + seg_offset) + offset; } uint32_t odp_packet_seg_data_len(odp_packet_t pkt, odp_packet_seg_t seg) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + uint32_t seg_offset = 0, offset = 0, i; + + while (seg >= pkt_hdr->buf_hdr.segcount - seg_offset && + pkt_hdr->ref_hdr) { + seg -= (pkt_hdr->buf_hdr.segcount - seg_offset); + offset = pkt_hdr->ref_offset; + pkt_hdr = pkt_hdr->ref_hdr; + for (i = 0, seg_offset = 0; + i < pkt_hdr->buf_hdr.segcount; + i++, seg_offset++) { + if (offset < pkt_hdr->buf_hdr.seg[i].len) + break; + offset -= pkt_hdr->buf_hdr.seg[i].len; + } + } - if (odp_unlikely(seg >= pkt_hdr->buf_hdr.segcount)) + if (odp_unlikely(seg + seg_offset >= pkt_hdr->buf_hdr.segcount)) return 0; - return packet_seg_len(pkt_hdr, seg); + return packet_seg_len(pkt_hdr, seg + seg_offset) - offset; } /* @@ -1092,7 +1198,7 @@ int odp_packet_add_data(odp_packet_t *pkt_ptr, uint32_t offset, uint32_t len) { odp_packet_t pkt = *pkt_ptr; odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); - uint32_t pktlen = pkt_hdr->frame_len; + uint32_t pktlen = packet_len(pkt_hdr); odp_packet_t newpkt; if (offset > pktlen) @@ -1222,6 +1328,91 @@ int odp_packet_split(odp_packet_t *pkt, uint32_t len, odp_packet_t *tail) } /* + * References + */ + +static inline void packet_ref(odp_packet_hdr_t *pkt_hdr) +{ + do { + odp_atomic_inc_u32(&pkt_hdr->ref_count); + pkt_hdr = pkt_hdr->ref_hdr; + } while (pkt_hdr); +} + +static inline odp_packet_t packet_splice(odp_packet_t pkt, uint32_t offset, + odp_packet_t hdr) +{ + odp_packet_hdr_t *ref_hdr, *pkt_hdr; + + pkt_hdr = odp_packet_hdr(pkt); + ref_hdr = odp_packet_hdr(hdr); + + while (ref_hdr->ref_hdr) + ref_hdr = ref_hdr->ref_hdr; + + ref_hdr->ref_hdr = pkt_hdr; + ref_hdr->ref_offset = offset; + + packet_ref(pkt_hdr); + return hdr; +} + +odp_packet_t odp_packet_ref_static(odp_packet_t pkt) +{ + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + + packet_ref(pkt_hdr); + return pkt; +} + +odp_packet_t odp_packet_ref(odp_packet_t pkt, uint32_t offset) +{ + odp_packet_t hdr; + + if (pkt == ODP_PACKET_INVALID || offset >= odp_packet_len(pkt)) + return ODP_PACKET_INVALID; + + hdr = odp_packet_alloc(odp_packet_pool(pkt), 0); + + if (hdr == ODP_PACKET_INVALID) + return ODP_PACKET_INVALID; + + return packet_splice(pkt, offset, hdr); +} + +odp_packet_t odp_packet_ref_pkt(odp_packet_t pkt, uint32_t offset, + odp_packet_t hdr) +{ + if (pkt == ODP_PACKET_INVALID || + pkt == hdr || + offset >= odp_packet_len(pkt)) + return ODP_PACKET_INVALID; + + if (hdr == ODP_PACKET_INVALID) + return odp_packet_ref(pkt, offset); + + return packet_splice(pkt, offset, hdr); +} + +int odp_packet_is_ref(odp_packet_t pkt) +{ + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + uint32_t ref_count = 0; + + while (pkt_hdr->ref_hdr) { + ref_count++; + pkt_hdr = pkt_hdr->ref_hdr; + } + + return ref_count; +} + +int odp_packet_has_ref(odp_packet_t pkt) +{ + return odp_atomic_load_u32(&odp_packet_hdr(pkt)->ref_count) - 1; +} + +/* * * Copy * ******************************************************** @@ -1231,7 +1422,7 @@ int odp_packet_split(odp_packet_t *pkt, uint32_t len, odp_packet_t *tail) odp_packet_t odp_packet_copy(odp_packet_t pkt, odp_pool_t pool) { odp_packet_hdr_t *srchdr = odp_packet_hdr(pkt); - uint32_t pktlen = srchdr->frame_len; + uint32_t pktlen = packet_len(srchdr); odp_packet_t newpkt = odp_packet_alloc(pool, pktlen); if (newpkt != ODP_PACKET_INVALID) { @@ -1270,7 +1461,7 @@ int odp_packet_copy_to_mem(odp_packet_t pkt, uint32_t offset, uint8_t *dstaddr = (uint8_t *)dst; odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); - if (offset + len > pkt_hdr->frame_len) + if (offset + len > packet_len(pkt_hdr)) return -1; while (len > 0) { @@ -1294,7 +1485,7 @@ int odp_packet_copy_from_mem(odp_packet_t pkt, uint32_t offset, const uint8_t *srcaddr = (const uint8_t *)src; odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); - if (offset + len > pkt_hdr->frame_len) + if (offset + len > packet_len(pkt_hdr)) return -1; while (len > 0) { @@ -1322,8 +1513,8 @@ int odp_packet_copy_from_pkt(odp_packet_t dst, uint32_t dst_offset, uint32_t src_seglen = 0; /* GCC */ int overlap; - if (dst_offset + len > dst_hdr->frame_len || - src_offset + len > src_hdr->frame_len) + if (dst_offset + len > packet_len(dst_hdr) || + src_offset + len > packet_len(src_hdr)) return -1; overlap = (dst_hdr == src_hdr && @@ -1408,7 +1599,7 @@ void odp_packet_print(odp_packet_t pkt) len += snprintf(&str[len], n - len, " l4_offset %" PRIu32 "\n", hdr->p.l4_offset); len += snprintf(&str[len], n - len, - " frame_len %" PRIu32 "\n", hdr->frame_len); + " frame_len %" PRIu32 "\n", packet_len(hdr)); len += snprintf(&str[len], n - len, " input %" PRIu64 "\n", odp_pktio_to_u64(hdr->input));