From patchwork Fri Apr 24 02:11:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF698C55186 for ; Fri, 24 Apr 2020 02:11:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9DBAD214AF for ; Fri, 24 Apr 2020 02:11:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694319; bh=gOe6xfSoB9Y0L82Uq8k70E6AaaX0Ms7CPw8ZPtmGWnA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=t+KXyujXhwNkmiJNORSv42aEq5fXjaj5HrVR5b5YbrDLuDzP37mCbVRwyl+1UihS/ pQTOaJWoUYgQb7yX7nXM8Ng1t2WQwKnk+jZruafH3SrbVlTgBio1FeTfY3KcQnLrtx ugn2OfSde5P1NBD1rv/+yLa0f2QRkKbaoGA3c/f0= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726261AbgDXCLz (ORCPT ); Thu, 23 Apr 2020 22:11:55 -0400 Received: from mail.kernel.org ([198.145.29.99]:36920 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725884AbgDXCLx (ORCPT ); Thu, 23 Apr 2020 22:11:53 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 4262921556; Fri, 24 Apr 2020 02:11:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694312; bh=gOe6xfSoB9Y0L82Uq8k70E6AaaX0Ms7CPw8ZPtmGWnA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mmlBwUiInqINR43oFM8Bi0l7gt45JmUZ7bPLs8CdckmqMt01MqRRwmP3GrTtJwwz1 kzQ1+1h8rmAB9TFxg/+RW5RrPr3jE5ku8RfPalG6NLvjRmQk7+zVJ5/zr4jMwst4Ze BAlS5Ai1hvAVMcYwM0Hyih2aFGyv38yeT5Do3GIQ= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v2 bpf-next 01/17] net: Refactor convert_to_xdp_frame Date: Thu, 23 Apr 2020 20:11:32 -0600 Message-Id: <20200424021148.83015-2-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200424021148.83015-1-dsahern@kernel.org> References: <20200424021148.83015-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Move the guts of convert_to_xdp_frame to a new helper, update_xdp_frame so it can be reused in a later patch. Suggested-by: Jesper Dangaard Brouer Signed-off-by: David Ahern --- include/net/xdp.h | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/include/net/xdp.h b/include/net/xdp.h index 40c6d3398458..779313862073 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -93,32 +93,42 @@ static inline void xdp_scrub_frame(struct xdp_frame *frame) struct xdp_frame *xdp_convert_zc_to_xdp_frame(struct xdp_buff *xdp); -/* Convert xdp_buff to xdp_frame */ static inline -struct xdp_frame *convert_to_xdp_frame(struct xdp_buff *xdp) +bool update_xdp_frame(struct xdp_buff *xdp, struct xdp_frame *xdp_frame) { - struct xdp_frame *xdp_frame; int metasize; int headroom; - if (xdp->rxq->mem.type == MEM_TYPE_ZERO_COPY) - return xdp_convert_zc_to_xdp_frame(xdp); - /* Assure headroom is available for storing info */ headroom = xdp->data - xdp->data_hard_start; metasize = xdp->data - xdp->data_meta; metasize = metasize > 0 ? metasize : 0; if (unlikely((headroom - metasize) < sizeof(*xdp_frame))) - return NULL; - - /* Store info in top of packet */ - xdp_frame = xdp->data_hard_start; + return false; xdp_frame->data = xdp->data; xdp_frame->len = xdp->data_end - xdp->data; xdp_frame->headroom = headroom - sizeof(*xdp_frame); xdp_frame->metasize = metasize; + return true; +} + +/* Convert xdp_buff to xdp_frame */ +static inline +struct xdp_frame *convert_to_xdp_frame(struct xdp_buff *xdp) +{ + struct xdp_frame *xdp_frame; + + if (xdp->rxq->mem.type == MEM_TYPE_ZERO_COPY) + return xdp_convert_zc_to_xdp_frame(xdp); + + /* Store info in top of packet */ + xdp_frame = xdp->data_hard_start; + + if (unlikely(!update_xdp_frame(xdp, xdp_frame))) + return NULL; + /* rxq only valid until napi_schedule ends, convert to xdp_mem_info */ xdp_frame->mem = xdp->rxq->mem; From patchwork Fri Apr 24 02:11:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220666 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72AC9C54FCB for ; Fri, 24 Apr 2020 02:11:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 52B91214AF for ; Fri, 24 Apr 2020 02:11:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694316; bh=wq4g1BuXZG0/8MK0MKY2iUKq5TEEaqohzwzVrRNK5rM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=jNz3mvvrbIn3AcNHdVLa+go2MmvoQ5gkhjTvJU9riebJAqGO7hHsBNnSmvE+7dh2h ai4OK1MzY1wxlxB6WdKklj7/PXlyKlhWI2Mi8/COsGA0WEu5p8vS6UHd7zr2AssHCT yr7GhvBKgZKGqnQpDDOVe6e/nABPtVfTHVD0RbgY= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726385AbgDXCLz (ORCPT ); Thu, 23 Apr 2020 22:11:55 -0400 Received: from mail.kernel.org ([198.145.29.99]:36962 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726087AbgDXCLy (ORCPT ); Thu, 23 Apr 2020 22:11:54 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D693821582; Fri, 24 Apr 2020 02:11:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694313; bh=wq4g1BuXZG0/8MK0MKY2iUKq5TEEaqohzwzVrRNK5rM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=zIMHcsbCB0pwFsyTMUGHX+5THi/9t11SRl6jxEBih+Vveui8hcheBZgxecgXB0rp3 9IJl79P/J3Avs9VvXiuaDUzWZfIRvmKJkLZhgOrtvk5xsJ711ndH6IwoHMjuHpEH4j cSHvc3RlJ9+Xff483q4ehAl1O20lMMMJvfPcAr40= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v2 bpf-next 02/17] net: Move handling of IFLA_XDP attribute out of do_setlink Date: Thu, 23 Apr 2020 20:11:33 -0600 Message-Id: <20200424021148.83015-3-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200424021148.83015-1-dsahern@kernel.org> References: <20200424021148.83015-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Later patch re-uses XDP attributes. Move processing of IFLA_XDP to a helper. Code move only; no functional change intended. Signed-off-by: David Ahern --- net/core/rtnetlink.c | 94 +++++++++++++++++++++++--------------------- 1 file changed, 49 insertions(+), 45 deletions(-) diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c index d6f4f4a9e8ba..193da2bb908a 100644 --- a/net/core/rtnetlink.c +++ b/net/core/rtnetlink.c @@ -2487,6 +2487,53 @@ static int do_set_master(struct net_device *dev, int ifindex, #define DO_SETLINK_MODIFIED 0x01 /* notify flag means notify + modified. */ #define DO_SETLINK_NOTIFY 0x03 + +static int do_setlink_xdp(struct net_device *dev, struct nlattr *tb, + int *status, struct netlink_ext_ack *extack) +{ + struct nlattr *xdp[IFLA_XDP_MAX + 1]; + u32 xdp_flags = 0; + int err; + + err = nla_parse_nested_deprecated(xdp, IFLA_XDP_MAX, tb, + ifla_xdp_policy, NULL); + if (err < 0) + return err; + + if (xdp[IFLA_XDP_ATTACHED] || xdp[IFLA_XDP_PROG_ID]) + goto out_einval; + + if (xdp[IFLA_XDP_FLAGS]) { + xdp_flags = nla_get_u32(xdp[IFLA_XDP_FLAGS]); + if (xdp_flags & ~XDP_FLAGS_MASK) + goto out_einval; + if (hweight32(xdp_flags & XDP_FLAGS_MODES) > 1) + goto out_einval; + } + + if (xdp[IFLA_XDP_FD]) { + int expected_fd = -1; + + if (xdp_flags & XDP_FLAGS_REPLACE) { + if (!xdp[IFLA_XDP_EXPECTED_FD]) + goto out_einval; + expected_fd = nla_get_s32(xdp[IFLA_XDP_EXPECTED_FD]); + } + + err = dev_change_xdp_fd(dev, extack, + nla_get_s32(xdp[IFLA_XDP_FD]), + expected_fd, xdp_flags); + if (err) + return err; + + *status |= DO_SETLINK_NOTIFY; + } + + return 0; +out_einval: + return -EINVAL; +} + static int do_setlink(const struct sk_buff *skb, struct net_device *dev, struct ifinfomsg *ifm, struct netlink_ext_ack *extack, @@ -2781,52 +2828,9 @@ static int do_setlink(const struct sk_buff *skb, } if (tb[IFLA_XDP]) { - struct nlattr *xdp[IFLA_XDP_MAX + 1]; - u32 xdp_flags = 0; - - err = nla_parse_nested_deprecated(xdp, IFLA_XDP_MAX, - tb[IFLA_XDP], - ifla_xdp_policy, NULL); - if (err < 0) - goto errout; - - if (xdp[IFLA_XDP_ATTACHED] || xdp[IFLA_XDP_PROG_ID]) { - err = -EINVAL; + err = do_setlink_xdp(dev, tb[IFLA_XDP], &status, extack); + if (err) goto errout; - } - - if (xdp[IFLA_XDP_FLAGS]) { - xdp_flags = nla_get_u32(xdp[IFLA_XDP_FLAGS]); - if (xdp_flags & ~XDP_FLAGS_MASK) { - err = -EINVAL; - goto errout; - } - if (hweight32(xdp_flags & XDP_FLAGS_MODES) > 1) { - err = -EINVAL; - goto errout; - } - } - - if (xdp[IFLA_XDP_FD]) { - int expected_fd = -1; - - if (xdp_flags & XDP_FLAGS_REPLACE) { - if (!xdp[IFLA_XDP_EXPECTED_FD]) { - err = -EINVAL; - goto errout; - } - expected_fd = - nla_get_s32(xdp[IFLA_XDP_EXPECTED_FD]); - } - - err = dev_change_xdp_fd(dev, extack, - nla_get_s32(xdp[IFLA_XDP_FD]), - expected_fd, - xdp_flags); - if (err) - goto errout; - status |= DO_SETLINK_NOTIFY; - } } errout: From patchwork Fri Apr 24 02:11:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220664 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E69AC55186 for ; Fri, 24 Apr 2020 02:12:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0E8B42087E for ; Fri, 24 Apr 2020 02:12:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694326; bh=fx+VVj3jtgcKvjqecXTges10Co820atjUmxQypl4RTc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=bYt/UpC4CNtyPM0fosScLhNBS8km0ScygbcRmgXlfRD5ilQs7avQzUwbzkR9chFLw uJR2ZC3NER/9+NL2vBtjHdLUKyRFuLyjqStiHgOEdlSwVivj9cmTJfkLFUH/fgGILc TYfdvzrnrBnM/5LQkW8SxLJOWoyj+0p9koXCoTZg= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726442AbgDXCMF (ORCPT ); Thu, 23 Apr 2020 22:12:05 -0400 Received: from mail.kernel.org ([198.145.29.99]:37128 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726414AbgDXCMB (ORCPT ); Thu, 23 Apr 2020 22:12:01 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id ED292215A4; Fri, 24 Apr 2020 02:11:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694320; bh=fx+VVj3jtgcKvjqecXTges10Co820atjUmxQypl4RTc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oOGO6M6KrLlsmiaWrbjm0T590Igmj3Dk5WKaIXnxLS0KXuOSUZZO+62C9hMUR7CB+ tID1HR91jWlT/Wb7d7CcF93v7K7pX00RxKp468iOlpLydHh2ZEssYaHWU82qQuQZrA G1PF/QZilN8yEPoyDFmHjmwj52ysdDBbiGGSAqC0= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v2 bpf-next 08/17] net: rename netif_receive_generic_xdp to do_generic_xdp_core Date: Thu, 23 Apr 2020 20:11:39 -0600 Message-Id: <20200424021148.83015-9-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200424021148.83015-1-dsahern@kernel.org> References: <20200424021148.83015-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern In skb generic path, we need a way to run XDP program on skb but to have customized handling of XDP actions. netif_receive_generic_xdp will be more helpful in such cases than do_xdp_generic. This patch prepares netif_receive_generic_xdp() to be used as general purpose function for running xdp programs on skbs by renaming it to do_xdp_generic_core, moving skb_is_redirected and rxq settings as well as XDP return code checks to the callers. This allows this core function to be used from both Rx and Tx paths with rxq and txq set based on context. Signed-off-by: Jason Wang Signed-off-by: Prashant Bhole Signed-off-by: David Ahern --- net/core/dev.c | 52 ++++++++++++++++++++++++-------------------------- 1 file changed, 25 insertions(+), 27 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index 3c38f576ffcc..74610b5cf2da 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4501,25 +4501,17 @@ static struct netdev_rx_queue *netif_get_rxqueue(struct sk_buff *skb) return rxqueue; } -static u32 netif_receive_generic_xdp(struct sk_buff *skb, - struct xdp_buff *xdp, - struct bpf_prog *xdp_prog) +static u32 do_xdp_generic_core(struct sk_buff *skb, struct xdp_buff *xdp, + struct bpf_prog *xdp_prog) { - struct netdev_rx_queue *rxqueue; void *orig_data, *orig_data_end; - u32 metalen, act = XDP_DROP; __be16 orig_eth_type; struct ethhdr *eth; + u32 metalen, act; bool orig_bcast; int hlen, off; u32 mac_len; - /* Reinjected packets coming from act_mirred or similar should - * not get XDP generic processing. - */ - if (skb_is_redirected(skb)) - return XDP_PASS; - /* XDP packets must be linear and must have sufficient headroom * of XDP_PACKET_HEADROOM bytes. This is the guarantee that also * native XDP provides, thus we need to do it here as well. @@ -4535,9 +4527,9 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb, if (pskb_expand_head(skb, hroom > 0 ? ALIGN(hroom, NET_SKB_PAD) : 0, troom > 0 ? troom + 128 : 0, GFP_ATOMIC)) - goto do_drop; + return XDP_DROP; if (skb_linearize(skb)) - goto do_drop; + return XDP_DROP; } /* The XDP program wants to see the packet starting at the MAC @@ -4555,9 +4547,6 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb, orig_bcast = is_multicast_ether_addr_64bits(eth->h_dest); orig_eth_type = eth->h_proto; - rxqueue = netif_get_rxqueue(skb); - xdp->rxq = &rxqueue->xdp_rxq; - act = bpf_prog_run_xdp(xdp_prog, xdp); /* check if bpf_xdp_adjust_head was used */ @@ -4600,16 +4589,6 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb, if (metalen) skb_metadata_set(skb, metalen); break; - default: - bpf_warn_invalid_xdp_action(act); - /* fall through */ - case XDP_ABORTED: - trace_xdp_exception(skb->dev, xdp_prog, act); - /* fall through */ - case XDP_DROP: - do_drop: - kfree_skb(skb); - break; } return act; @@ -4644,12 +4623,22 @@ static DEFINE_STATIC_KEY_FALSE(generic_xdp_needed_key); int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb) { + /* Reinjected packets coming from act_mirred or similar should + * not get XDP generic processing. + */ + if (skb_is_redirected(skb)) + return XDP_PASS; + if (xdp_prog) { + struct netdev_rx_queue *rxqueue; struct xdp_buff xdp; u32 act; int err; - act = netif_receive_generic_xdp(skb, &xdp, xdp_prog); + rxqueue = netif_get_rxqueue(skb); + xdp.rxq = &rxqueue->xdp_rxq; + + act = do_xdp_generic_core(skb, &xdp, xdp_prog); if (act != XDP_PASS) { switch (act) { case XDP_REDIRECT: @@ -4661,6 +4650,15 @@ int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb) case XDP_TX: generic_xdp_tx(skb, xdp_prog); break; + default: + bpf_warn_invalid_xdp_action(act); + /* fall through */ + case XDP_ABORTED: + trace_xdp_exception(skb->dev, xdp_prog, act); + /* fall through */ + case XDP_DROP: + kfree_skb(skb); + break; } return XDP_DROP; } From patchwork Fri Apr 24 02:11:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220662 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DA91C2BA19 for ; Fri, 24 Apr 2020 02:12:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3FD5420CC7 for ; Fri, 24 Apr 2020 02:12:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694336; bh=vpGCe4/vr0DWlY+/1Nl9ZNDIziRd52CinPj79/q/0UM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=nwuP0rxhe1/6Q/jI8UpdnCI2o4zVe4EbhX8zOjpexw4zfqBvjoN10h2xdNCvTCkCc F2oWdUzAUeDVqxWEdOIFjFlYdyTSRDoLH/fNH2lvcM55C8jufKHVQPjcrbV/bUBjsW bG8CdY8hzWj0FsDB70RePrM4/oUXP5VwwMmVFrBs= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726461AbgDXCMJ (ORCPT ); Thu, 23 Apr 2020 22:12:09 -0400 Received: from mail.kernel.org ([198.145.29.99]:37204 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725888AbgDXCMD (ORCPT ); Thu, 23 Apr 2020 22:12:03 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0E362214AF; Fri, 24 Apr 2020 02:12:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694322; bh=vpGCe4/vr0DWlY+/1Nl9ZNDIziRd52CinPj79/q/0UM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=I9i9b+wG/sjkKKD2Lnh1dQqJjA/YrxHCa18sJUFW03LtRxRW/8ny3wcsLsa8jTdr4 eIXPXbcWd4JBFPnQZ/cyjvBCX3YpTiY/83QBOG32Hyoq1Vv5MiMKp4ZOOejIgHOWog I65eH7G+5DP18qt7mxCu9STTblZSMFK+I/DH8wS4= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v2 bpf-next 10/17] net: Support xdp in the Tx path for packets as an skb Date: Thu, 23 Apr 2020 20:11:41 -0600 Message-Id: <20200424021148.83015-11-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200424021148.83015-1-dsahern@kernel.org> References: <20200424021148.83015-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add support to run Tx path program on packets about to hit the ndo_start_xmit function for a device. Only XDP_DROP and XDP_PASS are supported now. Conceptually, XDP_REDIRECT for this path can work the same as it does for the Rx path, but that support is left for a follow on series. Signed-off-by: David Ahern --- include/linux/netdevice.h | 11 +++++++++ net/core/dev.c | 52 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 62 insertions(+), 1 deletion(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index ec177c3e2720..84ef4cc1c2a7 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3715,6 +3715,7 @@ static inline void dev_consume_skb_any(struct sk_buff *skb) void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog); int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb); +u32 do_xdp_egress_skb(struct net_device *dev, struct sk_buff *skb); int netif_rx(struct sk_buff *skb); int netif_rx_ni(struct sk_buff *skb); int netif_receive_skb(struct sk_buff *skb); @@ -4575,6 +4576,16 @@ static inline netdev_tx_t __netdev_start_xmit(const struct net_device_ops *ops, struct sk_buff *skb, struct net_device *dev, bool more) { + if (static_branch_unlikely(&xdp_egress_needed_key)) { + u32 act; + + rcu_read_lock(); + act = do_xdp_egress_skb(dev, skb); + rcu_read_unlock(); + if (act == XDP_DROP) + return NET_XMIT_DROP; + } + __this_cpu_write(softnet_data.xmit.more, more); return ops->ndo_start_xmit(skb, dev); } diff --git a/net/core/dev.c b/net/core/dev.c index f8c74aaa2946..7bf0003e1876 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4620,7 +4620,6 @@ void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog) } static DEFINE_STATIC_KEY_FALSE(generic_xdp_needed_key); -DEFINE_STATIC_KEY_FALSE(xdp_egress_needed_key); int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb) { @@ -4671,6 +4670,57 @@ int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb) } EXPORT_SYMBOL_GPL(do_xdp_generic_rx); +DEFINE_STATIC_KEY_FALSE(xdp_egress_needed_key); +EXPORT_SYMBOL_GPL(xdp_egress_needed_key); + +static u32 handle_xdp_egress_act(u32 act, struct net_device *dev, + struct bpf_prog *xdp_prog) +{ + switch (act) { + case XDP_DROP: + /* fall through */ + case XDP_PASS: + break; + case XDP_TX: + /* fall through */ + case XDP_REDIRECT: + /* fall through */ + default: + bpf_warn_invalid_xdp_action(act); + /* fall through */ + case XDP_ABORTED: + trace_xdp_exception(dev, xdp_prog, act); + act = XDP_DROP; + break; + } + + return act; +} + +u32 do_xdp_egress_skb(struct net_device *dev, struct sk_buff *skb) +{ + struct bpf_prog *xdp_prog; + u32 act = XDP_PASS; + + xdp_prog = rcu_dereference(dev->xdp_egress_prog); + if (xdp_prog) { + struct xdp_txq_info txq = { .dev = dev }; + struct xdp_buff xdp; + + xdp.txq = &txq; + act = do_xdp_generic_core(skb, &xdp, xdp_prog); + act = handle_xdp_egress_act(act, dev, xdp_prog); + if (act == XDP_DROP) { + atomic_long_inc(&dev->tx_dropped); + skb_tx_error(skb); + kfree_skb(skb); + } + } + + return act; +} +EXPORT_SYMBOL_GPL(do_xdp_egress_skb); + static int netif_rx_internal(struct sk_buff *skb) { int ret; From patchwork Fri Apr 24 02:11:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220663 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E75EC54FCB for ; Fri, 24 Apr 2020 02:12:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0A9C921655 for ; Fri, 24 Apr 2020 02:12:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694332; bh=AbQlYd9YWfTONMYbZExVoQXT6jJuo+Rf7Nwuto5So0U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=aiiFQA35f8pkJYEwMzHTPRGufxzr4VZnXR0cFPDku37wi/IC1zBXPcln9HdYSKPYC nxDUAPWdcE/i74Dx8R/yHZTvTTGUgFXuDJuZZ8ySHp0Gn0Ad+9vPwTrRcA5TuQFhNK SSWhate0nrfIUEe9OV8D9hivXLuMd/Q92sGgcqGk= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726474AbgDXCML (ORCPT ); Thu, 23 Apr 2020 22:12:11 -0400 Received: from mail.kernel.org ([198.145.29.99]:37232 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726403AbgDXCME (ORCPT ); Thu, 23 Apr 2020 22:12:04 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0F8F721707; Fri, 24 Apr 2020 02:12:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694324; bh=AbQlYd9YWfTONMYbZExVoQXT6jJuo+Rf7Nwuto5So0U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Oq8N4cPagJnBd3/RvcOq7fyr7CVKsH7/h3sU+G+2Aw5xx976FxNY/6yIjIE7Rh2eH mmUZHQiLgM5yARfbyQrTuKKWMvdw/YCvXrkSQxuYnqEf/vKEorunTRHp2CeVVQrPZO PWwBUgmzdveHj/zrmyTmDYEgMd29j38UPbOrmN1s= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v2 bpf-next 11/17] net: Support xdp in the Tx path for xdp_frames Date: Thu, 23 Apr 2020 20:11:42 -0600 Message-Id: <20200424021148.83015-12-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200424021148.83015-1-dsahern@kernel.org> References: <20200424021148.83015-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add support to run Tx path program on xdp_frames by adding a hook to bq_xmit_all before xdp_frames are passed to ndo_xdp_xmit for the device. If an xdp_frame is dropped by the program, it is removed from the xdp_frames array with subsequent entries moved up. Signed-off-by: David Ahern --- include/linux/netdevice.h | 3 ++ kernel/bpf/devmap.c | 19 +++++++++---- net/core/dev.c | 59 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 76 insertions(+), 5 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 84ef4cc1c2a7..8d77c260c43b 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3716,6 +3716,9 @@ static inline void dev_consume_skb_any(struct sk_buff *skb) void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog); int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb); u32 do_xdp_egress_skb(struct net_device *dev, struct sk_buff *skb); +unsigned int do_xdp_egress_frame(struct net_device *dev, + struct xdp_frame **frames, + unsigned int count); int netif_rx(struct sk_buff *skb); int netif_rx_ni(struct sk_buff *skb); int netif_receive_skb(struct sk_buff *skb); diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 58bdca5d978a..c038c8c3ccdb 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -322,24 +322,33 @@ static int bq_xmit_all(struct xdp_dev_bulk_queue *bq, u32 flags) { struct net_device *dev = bq->dev; int sent = 0, drops = 0, err = 0; + unsigned int count = bq->count; int i; - if (unlikely(!bq->count)) + if (unlikely(!count)) return 0; - for (i = 0; i < bq->count; i++) { + for (i = 0; i < count; i++) { struct xdp_frame *xdpf = bq->q[i]; prefetch(xdpf); } - sent = dev->netdev_ops->ndo_xdp_xmit(dev, bq->count, bq->q, flags); + if (static_branch_unlikely(&xdp_egress_needed_key)) { + count = do_xdp_egress_frame(dev, bq->q, count); + drops += bq->count - count; + /* all frames consumed by the xdp program? */ + if (!count) + goto out; + } + + sent = dev->netdev_ops->ndo_xdp_xmit(dev, count, bq->q, flags); if (sent < 0) { err = sent; sent = 0; goto error; } - drops = bq->count - sent; + drops += count - sent; out: bq->count = 0; @@ -351,7 +360,7 @@ static int bq_xmit_all(struct xdp_dev_bulk_queue *bq, u32 flags) /* If ndo_xdp_xmit fails with an errno, no frames have been * xmit'ed and it's our responsibility to them free all. */ - for (i = 0; i < bq->count; i++) { + for (i = 0; i < count; i++) { struct xdp_frame *xdpf = bq->q[i]; xdp_return_frame_rx_napi(xdpf); diff --git a/net/core/dev.c b/net/core/dev.c index 7bf0003e1876..0567a3e459aa 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4721,6 +4721,65 @@ u32 do_xdp_egress_skb(struct net_device *dev, struct sk_buff *skb) } EXPORT_SYMBOL_GPL(do_xdp_egress_skb); +static u32 __xdp_egress_frame(struct net_device *dev, + struct bpf_prog *xdp_prog, + struct xdp_frame *xdp_frame, + struct xdp_txq_info *txq) +{ + struct xdp_buff xdp; + u32 act; + + xdp.data_hard_start = xdp_frame->data - xdp_frame->headroom + - sizeof(*xdp_frame); + xdp.data = xdp_frame->data; + xdp.data_end = xdp.data + xdp_frame->len; + xdp.data_meta = xdp.data - xdp_frame->metasize; + xdp.txq = txq; + + act = bpf_prog_run_xdp(xdp_prog, &xdp); + act = handle_xdp_egress_act(act, dev, xdp_prog); + + /* if not dropping frame, readjust pointers in case + * program made changes to the buffer + */ + if (act != XDP_DROP) { + if (unlikely(!update_xdp_frame(&xdp, xdp_frame))) + return XDP_DROP; + } + + return act; +} + +unsigned int do_xdp_egress_frame(struct net_device *dev, + struct xdp_frame **frames, + unsigned int count) +{ + struct bpf_prog *xdp_prog; + + xdp_prog = rcu_dereference(dev->xdp_egress_prog); + if (xdp_prog) { + struct xdp_txq_info txq = { .dev = dev }; + unsigned int i, j; + u32 act; + + for (i = 0, j = 0; i < count; i++) { + struct xdp_frame *frame = frames[i]; + + act = __xdp_egress_frame(dev, xdp_prog, frame, &txq); + if (act == XDP_DROP) { + xdp_return_frame_rx_napi(frame); + continue; + } + + frames[j] = frame; + j++; + } + count = j; + } + + return count; +} + static int netif_rx_internal(struct sk_buff *skb) { int ret; From patchwork Fri Apr 24 02:11:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E07C0C2BA19 for ; Fri, 24 Apr 2020 02:12:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BB7562087E for ; Fri, 24 Apr 2020 02:12:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694349; bh=UF9F1WQ7PVYCc1CRybC/5xoCAj/lQmyaUG9Z4k3hBsk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=bth/6GjutYPEJJHsFGTvBr/zWPp1uTW1fMq2DSCKagk8w2vjShbAS2RD0+4VUCcNQ c9Ugy3nvjl7IuW7Lv0cOKuN5/9xYCF5TNx7+yM2w84wXz2zMqQ3f4WdFImkf+Hds22 bEtSLBg68jZMydFE9fjklI4nqibH0Oxmdhz/BMQE= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726527AbgDXCM2 (ORCPT ); Thu, 23 Apr 2020 22:12:28 -0400 Received: from mail.kernel.org ([198.145.29.99]:37394 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726473AbgDXCMM (ORCPT ); Thu, 23 Apr 2020 22:12:12 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 31FBA21569; Fri, 24 Apr 2020 02:12:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694331; bh=UF9F1WQ7PVYCc1CRybC/5xoCAj/lQmyaUG9Z4k3hBsk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NjYaeZYOvZwuiPSSdAa/iUXIH9PRBlHA9CKfRfNYY/+OhMysBMJcNtstHS8E/g4Hg ff2sbSHEmoURA8WijhledpGc4vVaMdhDTzsvf5qQbO0rekrNAT+E29IuVtKu5vZTpY sehbsjbsRrcY8EnKdUv+W7lMHmR6hg1MZQFQy5Ws= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v2 bpf-next 14/17] bpftool: Add support for XDP egress Date: Thu, 23 Apr 2020 20:11:45 -0600 Message-Id: <20200424021148.83015-15-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200424021148.83015-1-dsahern@kernel.org> References: <20200424021148.83015-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add NET_ATTACH_TYPE_XDP_EGRESS and update attach_type_strings to allow a user to specify 'xdp_egress' as the attach or detach point. libbpf handles egress config via bpf_set_link_xdp_fd_opts, so update do_attach_detach_xdp to use it. Specifically, the new API requires old_fd to be set based on any currently loaded program, so use bpf_get_link_xdp_id and bpf_get_link_xdp_egress_id to get an fd to any existing program. Update 'net show' command to dump egress programs. Signed-off-by: David Ahern --- .../bpf/bpftool/Documentation/bpftool-net.rst | 4 +- tools/bpf/bpftool/bash-completion/bpftool | 2 +- tools/bpf/bpftool/main.h | 2 +- tools/bpf/bpftool/net.c | 48 +++++++++++++++++-- tools/bpf/bpftool/netlink_dumper.c | 12 +++-- 5 files changed, 58 insertions(+), 10 deletions(-) diff --git a/tools/bpf/bpftool/Documentation/bpftool-net.rst b/tools/bpf/bpftool/Documentation/bpftool-net.rst index 8651b00b81ea..d7398fb00ec4 100644 --- a/tools/bpf/bpftool/Documentation/bpftool-net.rst +++ b/tools/bpf/bpftool/Documentation/bpftool-net.rst @@ -26,7 +26,8 @@ NET COMMANDS | **bpftool** **net help** | | *PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* } -| *ATTACH_TYPE* := { **xdp** | **xdpgeneric** | **xdpdrv** | **xdpoffload** } +| *ATTACH_TYPE* := +| { **xdp** | **xdpgeneric** | **xdpdrv** | **xdpoffload** | **xdp_egress** } DESCRIPTION =========== @@ -63,6 +64,7 @@ DESCRIPTION **xdpgeneric** - Generic XDP. runs at generic XDP hook when packet already enters receive path as skb; **xdpdrv** - Native XDP. runs earliest point in driver's receive path; **xdpoffload** - Offload XDP. runs directly on NIC on each packet reception; + **xdp_egress** - XDP in egress path. runs at core networking level; **bpftool** **net detach** *ATTACH_TYPE* **dev** *NAME* Detach bpf program attached to network interface *NAME* with diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool index 45ee99b159e2..cc7a36057393 100644 --- a/tools/bpf/bpftool/bash-completion/bpftool +++ b/tools/bpf/bpftool/bash-completion/bpftool @@ -1003,7 +1003,7 @@ _bpftool() ;; net) local PROG_TYPE='id pinned tag name' - local ATTACH_TYPES='xdp xdpgeneric xdpdrv xdpoffload' + local ATTACH_TYPES='xdp xdpgeneric xdpdrv xdpoffload xdp_egress' case $command in show|list) [[ $prev != "$command" ]] && return 0 diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h index 86f14ce26fd7..cbc0cc2257eb 100644 --- a/tools/bpf/bpftool/main.h +++ b/tools/bpf/bpftool/main.h @@ -230,7 +230,7 @@ void btf_dump_linfo_json(const struct btf *btf, struct nlattr; struct ifinfomsg; struct tcmsg; -int do_xdp_dump(struct ifinfomsg *ifinfo, struct nlattr **tb); +int do_xdp_dump(struct ifinfomsg *ifinfo, struct nlattr **tb, bool egress); int do_filter_dump(struct tcmsg *ifinfo, struct nlattr **tb, const char *kind, const char *devname, int ifindex); diff --git a/tools/bpf/bpftool/net.c b/tools/bpf/bpftool/net.c index c5e3895b7c8b..d09272a53734 100644 --- a/tools/bpf/bpftool/net.c +++ b/tools/bpf/bpftool/net.c @@ -32,6 +32,7 @@ struct bpf_netdev_t { int used_len; int array_len; int filter_idx; + bool egress; }; struct tc_kind_handle { @@ -61,6 +62,7 @@ enum net_attach_type { NET_ATTACH_TYPE_XDP_GENERIC, NET_ATTACH_TYPE_XDP_DRIVER, NET_ATTACH_TYPE_XDP_OFFLOAD, + NET_ATTACH_TYPE_XDP_EGRESS, }; static const char * const attach_type_strings[] = { @@ -68,6 +70,7 @@ static const char * const attach_type_strings[] = { [NET_ATTACH_TYPE_XDP_GENERIC] = "xdpgeneric", [NET_ATTACH_TYPE_XDP_DRIVER] = "xdpdrv", [NET_ATTACH_TYPE_XDP_OFFLOAD] = "xdpoffload", + [NET_ATTACH_TYPE_XDP_EGRESS] = "xdp_egress", }; const size_t net_attach_type_size = ARRAY_SIZE(attach_type_strings); @@ -111,7 +114,7 @@ static int dump_link_nlmsg(void *cookie, void *msg, struct nlattr **tb) : ""); netinfo->used_len++; - return do_xdp_dump(ifinfo, tb); + return do_xdp_dump(ifinfo, tb, netinfo->egress); } static int dump_class_qdisc_nlmsg(void *cookie, void *msg, struct nlattr **tb) @@ -276,10 +279,20 @@ static int net_parse_dev(int *argc, char ***argv) static int do_attach_detach_xdp(int progfd, enum net_attach_type attach_type, int ifindex, bool overwrite) { - __u32 flags = 0; + struct bpf_xdp_set_link_opts opts; + __u32 flags = 0, id = 0; + int rc; + + memset(&opts, 0, sizeof(opts)); + opts.sz = sizeof(opts); + opts.old_fd = -1; if (!overwrite) flags = XDP_FLAGS_UPDATE_IF_NOEXIST; + + if (attach_type == NET_ATTACH_TYPE_XDP_EGRESS) + opts.egress = 1; + if (attach_type == NET_ATTACH_TYPE_XDP_GENERIC) flags |= XDP_FLAGS_SKB_MODE; if (attach_type == NET_ATTACH_TYPE_XDP_DRIVER) @@ -287,7 +300,25 @@ static int do_attach_detach_xdp(int progfd, enum net_attach_type attach_type, if (attach_type == NET_ATTACH_TYPE_XDP_OFFLOAD) flags |= XDP_FLAGS_HW_MODE; - return bpf_set_link_xdp_fd(ifindex, progfd, flags); + if (opts.egress) + rc = bpf_get_link_xdp_egress_id(ifindex, &id, flags); + else + rc = bpf_get_link_xdp_id(ifindex, &id, flags); + + if (rc) { + p_err("Failed to get existing prog id for device"); + return rc; + } + + if (id) + opts.old_fd = bpf_prog_get_fd_by_id(id); + + rc = bpf_set_link_xdp_fd_opts(ifindex, progfd, flags, &opts); + + if (opts.old_fd != -1) + close(opts.old_fd); + + return rc; } static int do_attach(int argc, char **argv) @@ -411,6 +442,7 @@ static int do_show(int argc, char **argv) dev_array.used_len = 0; dev_array.array_len = 0; dev_array.filter_idx = filter_idx; + dev_array.egress = 0; if (json_output) jsonw_start_array(json_wtr); @@ -419,6 +451,14 @@ static int do_show(int argc, char **argv) ret = libbpf_nl_get_link(sock, nl_pid, dump_link_nlmsg, &dev_array); NET_END_ARRAY("\n"); + if (!ret) { + dev_array.egress = true; + NET_START_ARRAY("xdp_egress", "%s:\n"); + ret = libbpf_nl_get_link(sock, nl_pid, dump_link_nlmsg, + &dev_array); + NET_END_ARRAY("\n"); + } + if (!ret) { NET_START_ARRAY("tc", "%s:\n"); for (i = 0; i < dev_array.used_len; i++) { @@ -464,7 +504,7 @@ static int do_help(int argc, char **argv) " %s %s help\n" "\n" " " HELP_SPEC_PROGRAM "\n" - " ATTACH_TYPE := { xdp | xdpgeneric | xdpdrv | xdpoffload }\n" + " ATTACH_TYPE := { xdp | xdpgeneric | xdpdrv | xdpoffload | xdp_egress}\n" "\n" "Note: Only xdp and tc attachments are supported now.\n" " For progs attached to cgroups, use \"bpftool cgroup\"\n" diff --git a/tools/bpf/bpftool/netlink_dumper.c b/tools/bpf/bpftool/netlink_dumper.c index 5f65140b003b..e4a2b6f8e50b 100644 --- a/tools/bpf/bpftool/netlink_dumper.c +++ b/tools/bpf/bpftool/netlink_dumper.c @@ -55,6 +55,7 @@ static int do_xdp_dump_one(struct nlattr *attr, unsigned int ifindex, xdp_dump_prog_id(tb, IFLA_XDP_SKB_PROG_ID, "generic", true); xdp_dump_prog_id(tb, IFLA_XDP_DRV_PROG_ID, "driver", true); xdp_dump_prog_id(tb, IFLA_XDP_HW_PROG_ID, "offload", true); + xdp_dump_prog_id(tb, IFLA_XDP_EGRESS_CORE_PROG_ID, "core", true); if (json_output) jsonw_end_array(json_wtr); } else if (mode == XDP_ATTACHED_DRV) { @@ -63,18 +64,23 @@ static int do_xdp_dump_one(struct nlattr *attr, unsigned int ifindex, xdp_dump_prog_id(tb, IFLA_XDP_PROG_ID, "generic", false); } else if (mode == XDP_ATTACHED_HW) { xdp_dump_prog_id(tb, IFLA_XDP_PROG_ID, "offload", false); + } else if (mode == XDP_ATTACHED_EGRESS_CORE) { + xdp_dump_prog_id(tb, IFLA_XDP_EGRESS_CORE_PROG_ID, "core", + false); } NET_END_OBJECT_FINAL; return 0; } -int do_xdp_dump(struct ifinfomsg *ifinfo, struct nlattr **tb) +int do_xdp_dump(struct ifinfomsg *ifinfo, struct nlattr **tb, bool egress) { - if (!tb[IFLA_XDP]) + __u16 atype = egress ? IFLA_XDP_EGRESS : IFLA_XDP; + + if (!tb[atype]) return 0; - return do_xdp_dump_one(tb[IFLA_XDP], ifinfo->ifi_index, + return do_xdp_dump_one(tb[atype], ifinfo->ifi_index, libbpf_nla_getattr_str(tb[IFLA_IFNAME])); } From patchwork Fri Apr 24 02:11:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220660 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6ED4C55186 for ; Fri, 24 Apr 2020 02:12:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 95A9E20CC7 for ; Fri, 24 Apr 2020 02:12:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694346; bh=YEKnfhpruNzG9WjFUG1Pl8waCRTar5/8EQmUHJsdR6Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=fxwuks+QEAiR4XLHQZWvqJF2IjbNkiTBaeE73PLyKcnjRShnlX0OBug0kriOR9QyH tmrRGPggs0h4pkO2oD6EGFe5RRt8aV9agzJNQ3EJlDs0ttcrqNZ7gjFTYrAmnS43EA hmRIzbDVLPQ33pvSdS+JENYoiqnlEHx5X+1/1YNs= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726519AbgDXCMY (ORCPT ); Thu, 23 Apr 2020 22:12:24 -0400 Received: from mail.kernel.org ([198.145.29.99]:37470 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726451AbgDXCMO (ORCPT ); Thu, 23 Apr 2020 22:12:14 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1B12021655; Fri, 24 Apr 2020 02:12:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694334; bh=YEKnfhpruNzG9WjFUG1Pl8waCRTar5/8EQmUHJsdR6Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DaOun2d/MDYkDnuvBli2aW6FavgHM/0/IZOZjidv7QjJYNDTArl+Hd++3RNp/sAcN VngKKPEqHEZXUZfshVSvul8VrMu5BSUk8GPWIbtKwZbTOvyNG60QDW4eSHSlYwguEw GeJnKHoiSwhqul3Uc6KoakEGT0fFYRgZN4GbNXAo= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v2 bpf-next 16/17] selftest: Add xdp_egress attach tests Date: Thu, 23 Apr 2020 20:11:47 -0600 Message-Id: <20200424021148.83015-17-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200424021148.83015-1-dsahern@kernel.org> References: <20200424021148.83015-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add xdp_egress attach tests: 1. verify egress programs cannot access ingress entries in xdp context 2. verify ability to load, attach, and detach xdp egress to a device. Signed-off-by: David Ahern --- .../bpf/prog_tests/xdp_egress_attach.c | 62 +++++++++++++++++++ .../selftests/bpf/progs/test_xdp_egress.c | 12 ++++ .../bpf/progs/test_xdp_egress_fail.c | 16 +++++ 3 files changed, 90 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/xdp_egress_attach.c create mode 100644 tools/testing/selftests/bpf/progs/test_xdp_egress.c create mode 100644 tools/testing/selftests/bpf/progs/test_xdp_egress_fail.c diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_egress_attach.c b/tools/testing/selftests/bpf/prog_tests/xdp_egress_attach.c new file mode 100644 index 000000000000..a8727f82a29d --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/xdp_egress_attach.c @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: GPL-2.0 +#include + +#define IFINDEX_LO 1 + +void test_xdp_egress_attach(void) +{ + struct bpf_prog_load_attr attr = { + .prog_type = BPF_PROG_TYPE_XDP, + .expected_attach_type = BPF_XDP_EGRESS, + }; + struct bpf_xdp_set_link_opts opts; + struct bpf_prog_info info = {}; + __u32 id, len = sizeof(info); + struct bpf_object *obj; + __u32 duration = 0; + int err, fd = -1; + + memset(&opts, 0, sizeof(opts)); + opts.sz = sizeof(opts); + + /* should fail - accesses rx queue info */ + attr.file = "./test_xdp_egress_fail.o", + err = bpf_prog_load_xattr(&attr, &obj, &fd); + if (CHECK(err == 0 && fd >= 0, "xdp_egress with rx failed to load", + "load of xdp_egress with rx succeeded instead of failed")) + return; + + attr.file = "./test_xdp_egress.o", + err = bpf_prog_load_xattr(&attr, &obj, &fd); + if (CHECK_FAIL(err)) + return; + + err = bpf_obj_get_info_by_fd(fd, &info, &len); + if (CHECK_FAIL(err)) + goto out_close; + + opts.old_fd = -1; + opts.egress = 1; + err = bpf_set_link_xdp_fd_opts(IFINDEX_LO, fd, 0, &opts); + if (CHECK(err, "xdp attach", "xdp attach failed")) + goto out_close; + + err = bpf_get_link_xdp_egress_id(IFINDEX_LO, &id, 0); + if (CHECK(err || id != info.id, "id_check", + "loaded prog id %u != id %u, err %d", info.id, id, err)) + goto out; + +out: + opts.old_fd = bpf_prog_get_fd_by_id(id); + err = bpf_set_link_xdp_fd_opts(IFINDEX_LO, -1, 0, &opts); + if (CHECK(err, "xdp detach", "xdp detach failed")) + goto out_close; + + err = bpf_get_link_xdp_egress_id(IFINDEX_LO, &id, 0); + if (CHECK(err || id, "id_check", + "failed to detach program %u", id)) + goto out; + +out_close: + bpf_object__close(obj); +} diff --git a/tools/testing/selftests/bpf/progs/test_xdp_egress.c b/tools/testing/selftests/bpf/progs/test_xdp_egress.c new file mode 100644 index 000000000000..0477e8537b7f --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_xdp_egress.c @@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include + +SEC("xdp_egress") +int xdp_egress_good(struct xdp_md *ctx) +{ + __u32 idx = ctx->egress_ifindex; + + return idx == 1 ? XDP_DROP : XDP_PASS; +} diff --git a/tools/testing/selftests/bpf/progs/test_xdp_egress_fail.c b/tools/testing/selftests/bpf/progs/test_xdp_egress_fail.c new file mode 100644 index 000000000000..76b47b1d3bc3 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_xdp_egress_fail.c @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include + +SEC("xdp_egress") +int xdp_egress_fail(struct xdp_md *ctx) +{ + __u32 rxq = ctx->rx_queue_index; + __u32 idx = ctx->ingress_ifindex; + + if (idx == 1) + return XDP_DROP; + + return rxq ? XDP_DROP : XDP_PASS; +} From patchwork Fri Apr 24 02:11:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 220661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E458C55193 for ; Fri, 24 Apr 2020 02:12:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3B41420CC7 for ; Fri, 24 Apr 2020 02:12:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694338; bh=fW1BC6x7tMfCNt++/v7F2D1iENTMOQg79pviDBE/3s8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=k/BoQRF/vwxrs5VpYKmYYAt3ATi1/Gf/WDuXOH/VN7+g7Ioth/+RQBS1WkYLT57Ou L+9elOpfE3CTSApF5FAhatRfWl0qIj7JyzOsf2Ou45sKpO3gDbEY4orr9fdKsqkYsA aHbdHzO6vdyda/HeKtRg8H7GbmDTibPdzlUZxoiw= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726511AbgDXCMR (ORCPT ); Thu, 23 Apr 2020 22:12:17 -0400 Received: from mail.kernel.org ([198.145.29.99]:37494 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726479AbgDXCMP (ORCPT ); Thu, 23 Apr 2020 22:12:15 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 3978521556; Fri, 24 Apr 2020 02:12:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587694335; bh=fW1BC6x7tMfCNt++/v7F2D1iENTMOQg79pviDBE/3s8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S5ZK1iaIEOL4yUkNq/t9F3YJFzRGPr9hpbIJa8ig1Arj+0Cv4IsA+6zl/kj6PlHgk 8tLwiY+4W+FWc7PcoZCMksynwQHlyY1JoY1fC8LUNBvfRnj7ApmRcGuRGTEF1gNMcy /TzKkkKfG5Sv5i2UxJlcGMk5kvhIgxIiXAHYSh1g= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH v2 bpf-next 17/17] samples/bpf: add XDP egress support to xdp1 Date: Thu, 23 Apr 2020 20:11:48 -0600 Message-Id: <20200424021148.83015-18-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200424021148.83015-1-dsahern@kernel.org> References: <20200424021148.83015-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern xdp1 and xdp2 now accept -E flag to set XDP program in the egress path. Signed-off-by: Prashant Bhole Signed-off-by: David Ahern --- samples/bpf/xdp1_user.c | 39 ++++++++++++++++++++++++++++++++------- 1 file changed, 32 insertions(+), 7 deletions(-) diff --git a/samples/bpf/xdp1_user.c b/samples/bpf/xdp1_user.c index c447ad9e3a1d..9f79bd537763 100644 --- a/samples/bpf/xdp1_user.c +++ b/samples/bpf/xdp1_user.c @@ -20,22 +20,37 @@ static int ifindex; static __u32 xdp_flags = XDP_FLAGS_UPDATE_IF_NOEXIST; +static struct bpf_xdp_set_link_opts opts; static __u32 prog_id; static void int_exit(int sig) { __u32 curr_prog_id = 0; + int rc; - if (bpf_get_link_xdp_id(ifindex, &curr_prog_id, xdp_flags)) { - printf("bpf_get_link_xdp_id failed\n"); + if (opts.egress) + rc = bpf_get_link_xdp_egress_id(ifindex, &curr_prog_id, xdp_flags); + else + rc = bpf_get_link_xdp_id(ifindex, &curr_prog_id, xdp_flags); + + if (rc) { + printf("Failed to get existing prog id for device"); exit(1); } + + if (curr_prog_id) + opts.old_fd = bpf_prog_get_fd_by_id(curr_prog_id); + if (prog_id == curr_prog_id) - bpf_set_link_xdp_fd(ifindex, -1, xdp_flags); + bpf_set_link_xdp_fd_opts(ifindex, -1, xdp_flags, &opts); else if (!curr_prog_id) printf("couldn't find a prog id on a given interface\n"); else printf("program on interface changed, not removing\n"); + + if (opts.old_fd >= 0) + close(opts.old_fd); + exit(0); } @@ -73,7 +88,8 @@ static void usage(const char *prog) "OPTS:\n" " -S use skb-mode\n" " -N enforce native mode\n" - " -F force loading prog\n", + " -F force loading prog\n" + " -E egress path program\n", prog); } @@ -83,15 +99,20 @@ int main(int argc, char **argv) struct bpf_prog_load_attr prog_load_attr = { .prog_type = BPF_PROG_TYPE_XDP, }; + struct bpf_xdp_set_link_opts opts; struct bpf_prog_info info = {}; __u32 info_len = sizeof(info); - const char *optstr = "FSN"; + const char *optstr = "FSNE"; int prog_fd, map_fd, opt; struct bpf_object *obj; struct bpf_map *map; char filename[256]; int err; + memset(&opts, 0, sizeof(opts)); + opts.sz = sizeof(opts); + opts.old_fd = -1; + while ((opt = getopt(argc, argv, optstr)) != -1) { switch (opt) { case 'S': @@ -103,13 +124,17 @@ int main(int argc, char **argv) case 'F': xdp_flags &= ~XDP_FLAGS_UPDATE_IF_NOEXIST; break; + case 'E': + opts.egress = true; + prog_load_attr.expected_attach_type = BPF_XDP_EGRESS; + break; default: usage(basename(argv[0])); return 1; } } - if (!(xdp_flags & XDP_FLAGS_SKB_MODE)) + if (!(xdp_flags & XDP_FLAGS_SKB_MODE) && !opts.egress) xdp_flags |= XDP_FLAGS_DRV_MODE; if (optind == argc) { @@ -149,7 +174,7 @@ int main(int argc, char **argv) signal(SIGINT, int_exit); signal(SIGTERM, int_exit); - if (bpf_set_link_xdp_fd(ifindex, prog_fd, xdp_flags) < 0) { + if (bpf_set_link_xdp_fd_opts(ifindex, prog_fd, xdp_flags, &opts) < 0) { printf("link set xdp fd failed\n"); return 1; }