From patchwork Mon Apr 13 17:17:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 221235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8151C2BA2B for ; Mon, 13 Apr 2020 17:19:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C97E42072D for ; Mon, 13 Apr 2020 17:19:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586798365; bh=5+1Xa0FD36K0m/kxFeoFKPENVaCqW9g44NMe5nr88o0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=gQFEsZf9uhLMd4dWKcC5682dMtxgSPgtvoTLbixts2j7KJX5vPKE2UDu38BHZlgxZ cVuAwJb+/HfFHgJAqHq9gpJSVA8mUIHgKDTsJKWchO2UUDWH8j/uJhwlWZOnKeataH gefkg9rB3O9ecfIlKjijfestc+hnLxSDRQQJt3L4= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732711AbgDMRTW (ORCPT ); Mon, 13 Apr 2020 13:19:22 -0400 Received: from mail.kernel.org ([198.145.29.99]:38624 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732637AbgDMRS1 (ORCPT ); Mon, 13 Apr 2020 13:18:27 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9968E20735; Mon, 13 Apr 2020 17:18:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586798306; bh=5+1Xa0FD36K0m/kxFeoFKPENVaCqW9g44NMe5nr88o0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NB6gqwt6knEB6FGZG+ulMu+7pfRjr+G+mvfhI3MNNbCdKxVn5N9hDaQrVZdXGu/LI +FdCczAhmEhiFOR3avYbMusY+FX1pJ6uhi+a3XERUxO6MbsMjJi+RH+C2Huh8sfOax qzUZn5AsSWnXJP+M6UYYmrg4cM8qxnwt9Xr4wDE4= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH RFC-v5 bpf-next 01/12] net: Add XDP setup and query commands for Tx programs Date: Mon, 13 Apr 2020 11:17:50 -0600 Message-Id: <20200413171801.54406-2-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200413171801.54406-1-dsahern@kernel.org> References: <20200413171801.54406-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add new netdev commands, XDP_SETUP_PROG_EGRESS and XDP_QUERY_PROG_EGRESS, to query and setup egress programs. Update dev_change_xdp_fd and dev_xdp_install to take an egress argument. If egress bool is set, then use XDP_SETUP_PROG_EGRESS in dev_xdp_install and XDP_QUERY_PROG_EGRESS in dev_change_xdp_fd. Update dev_xdp_uninstall to query for XDP_QUERY_PROG_EGRESS and if a program is installed call dev_xdp_install with the egress argument set to true. This enables existing infrastructure to be used for XDP programs in the egress path. Signed-off-by: David Ahern Co-developed-by: Prashant Bhole Signed-off-by: Prashant Bhole --- include/linux/netdevice.h | 4 +++- net/core/dev.c | 34 +++++++++++++++++++++++----------- net/core/rtnetlink.c | 2 +- 3 files changed, 27 insertions(+), 13 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 130a668049ab..d0bb9e09660a 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -871,8 +871,10 @@ enum bpf_netdev_command { */ XDP_SETUP_PROG, XDP_SETUP_PROG_HW, + XDP_SETUP_PROG_EGRESS, XDP_QUERY_PROG, XDP_QUERY_PROG_HW, + XDP_QUERY_PROG_EGRESS, /* BPF program for offload callbacks, invoked at program load time. */ BPF_OFFLOAD_MAP_ALLOC, BPF_OFFLOAD_MAP_FREE, @@ -3777,7 +3779,7 @@ struct sk_buff *dev_hard_start_xmit(struct sk_buff *skb, struct net_device *dev, typedef int (*bpf_op_t)(struct net_device *dev, struct netdev_bpf *bpf); int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack, - int fd, int expected_fd, u32 flags); + int fd, int expected_fd, u32 flags, bool egress); u32 __dev_xdp_query(struct net_device *dev, bpf_op_t xdp_op, enum bpf_netdev_command cmd); int xdp_umem_query(struct net_device *dev, u16 queue_id); diff --git a/net/core/dev.c b/net/core/dev.c index 3ebfecc7b112..06e0872ecdae 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -8589,7 +8589,7 @@ u32 __dev_xdp_query(struct net_device *dev, bpf_op_t bpf_op, static int dev_xdp_install(struct net_device *dev, bpf_op_t bpf_op, struct netlink_ext_ack *extack, u32 flags, - struct bpf_prog *prog) + struct bpf_prog *prog, bool egress) { bool non_hw = !(flags & XDP_FLAGS_HW_MODE); struct bpf_prog *prev_prog = NULL; @@ -8597,8 +8597,10 @@ static int dev_xdp_install(struct net_device *dev, bpf_op_t bpf_op, int err; if (non_hw) { - prev_prog = bpf_prog_by_id(__dev_xdp_query(dev, bpf_op, - XDP_QUERY_PROG)); + enum bpf_netdev_command cmd; + + cmd = egress ? XDP_QUERY_PROG_EGRESS : XDP_QUERY_PROG; + prev_prog = bpf_prog_by_id(__dev_xdp_query(dev, bpf_op, cmd)); if (IS_ERR(prev_prog)) prev_prog = NULL; } @@ -8607,7 +8609,7 @@ static int dev_xdp_install(struct net_device *dev, bpf_op_t bpf_op, if (flags & XDP_FLAGS_HW_MODE) xdp.command = XDP_SETUP_PROG_HW; else - xdp.command = XDP_SETUP_PROG; + xdp.command = egress ? XDP_SETUP_PROG_EGRESS : XDP_SETUP_PROG; xdp.extack = extack; xdp.flags = flags; xdp.prog = prog; @@ -8628,7 +8630,12 @@ static void dev_xdp_uninstall(struct net_device *dev) bpf_op_t ndo_bpf; /* Remove generic XDP */ - WARN_ON(dev_xdp_install(dev, generic_xdp_install, NULL, 0, NULL)); + WARN_ON(dev_xdp_install(dev, generic_xdp_install, NULL, 0, NULL, + false)); + + /* Remove XDP egress */ + WARN_ON(dev_xdp_install(dev, generic_xdp_install, NULL, 0, NULL, + true)); /* Remove from the driver */ ndo_bpf = dev->netdev_ops->ndo_bpf; @@ -8640,14 +8647,14 @@ static void dev_xdp_uninstall(struct net_device *dev) WARN_ON(ndo_bpf(dev, &xdp)); if (xdp.prog_id) WARN_ON(dev_xdp_install(dev, ndo_bpf, NULL, xdp.prog_flags, - NULL)); + NULL, false)); /* Remove HW offload */ memset(&xdp, 0, sizeof(xdp)); xdp.command = XDP_QUERY_PROG_HW; if (!ndo_bpf(dev, &xdp) && xdp.prog_id) WARN_ON(dev_xdp_install(dev, ndo_bpf, NULL, xdp.prog_flags, - NULL)); + NULL, false)); } /** @@ -8661,7 +8668,7 @@ static void dev_xdp_uninstall(struct net_device *dev) * Set or clear a bpf program for a device */ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack, - int fd, int expected_fd, u32 flags) + int fd, int expected_fd, u32 flags, bool egress) { const struct net_device_ops *ops = dev->netdev_ops; enum bpf_netdev_command query; @@ -8674,7 +8681,11 @@ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack, ASSERT_RTNL(); offload = flags & XDP_FLAGS_HW_MODE; - query = offload ? XDP_QUERY_PROG_HW : XDP_QUERY_PROG; + if (egress) + query = XDP_QUERY_PROG_EGRESS; + else + query = offload ? XDP_QUERY_PROG_HW : XDP_QUERY_PROG; + bpf_op = bpf_chk = ops->ndo_bpf; if (!bpf_op && (flags & (XDP_FLAGS_DRV_MODE | XDP_FLAGS_HW_MODE))) { @@ -8704,7 +8715,8 @@ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack, } } if (fd >= 0) { - if (!offload && __dev_xdp_query(dev, bpf_chk, XDP_QUERY_PROG)) { + if (!offload && !egress && + __dev_xdp_query(dev, bpf_chk, XDP_QUERY_PROG)) { NL_SET_ERR_MSG(extack, "native and generic XDP can't be active at the same time"); return -EEXIST; } @@ -8736,7 +8748,7 @@ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack, prog = NULL; } - err = dev_xdp_install(dev, bpf_op, extack, flags, prog); + err = dev_xdp_install(dev, bpf_op, extack, flags, prog, egress); if (err < 0 && prog) bpf_prog_put(prog); diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c index 97e47c292333..dc44af16226a 100644 --- a/net/core/rtnetlink.c +++ b/net/core/rtnetlink.c @@ -2515,7 +2515,7 @@ static int do_setlink_xdp(struct net_device *dev, struct nlattr *tb, err = dev_change_xdp_fd(dev, extack, nla_get_s32(xdp[IFLA_XDP_FD]), - expected_fd, xdp_flags); + expected_fd, xdp_flags, false); if (err) return err; From patchwork Mon Apr 13 17:17:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 221236 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9340EC2BA19 for ; Mon, 13 Apr 2020 17:19:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6BDAD20739 for ; Mon, 13 Apr 2020 17:19:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586798357; bh=pIjVxLC9+D5l2V+Pgm7Qo6a60H/grc0JxSE8AgNPwWI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=dgkmQyU8CEjOy8SJuJxSYcsb8HpOMs9EfNA9KC8YZBaUIlBAhwvUYGcbzF5N8pZUS nhJ+qndYAZEU5ZAeQQtG4ZunZbGDWo57hYMbUifr1PGB7r7obQoyk/JlpFOiZz1Ny9 ICkLgFZWYfRnklZZpHPszCO2oz2LOQPJPSaHkK8I= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732703AbgDMRTP (ORCPT ); Mon, 13 Apr 2020 13:19:15 -0400 Received: from mail.kernel.org ([198.145.29.99]:38898 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732592AbgDMRSc (ORCPT ); Mon, 13 Apr 2020 13:18:32 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 149722073E; Mon, 13 Apr 2020 17:18:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586798311; bh=pIjVxLC9+D5l2V+Pgm7Qo6a60H/grc0JxSE8AgNPwWI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AK4Oz9A4SVuCAak4sszhZuI0edLO4kNUMp7i9xjKIiF8s/Cd0/OsRqZpIiJVjx1LU pyTEyXmch8p0tJPCiA4Vss5QL6D4LaDcOXOJUb9LSkketPmuKlXcGJO0zjNk79Amc0 dBD6xowQUp63nzJi00cQ1mh2ggZcpHdlijbZgTn0= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH RFC-v5 bpf-next 05/12] net: core: rename netif_receive_generic_xdp to do_generic_xdp_core Date: Mon, 13 Apr 2020 11:17:54 -0600 Message-Id: <20200413171801.54406-6-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200413171801.54406-1-dsahern@kernel.org> References: <20200413171801.54406-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern In skb generic path, we need a way to run XDP program on skb but to have customized handling of XDP actions. netif_receive_generic_xdp will be more helpful in such cases than do_xdp_generic. This patch prepares netif_receive_generic_xdp() to be used as general purpose function for running xdp programs on skbs by renaming it to do_xdp_generic_core, moving skb_is_redirected and rxq settings as well as XDP return code checks to the callers. This allows this core function to be used from both Rx and Tx paths with rxq and txq set based on context. Signed-off-by: Jason Wang Signed-off-by: Prashant Bhole Signed-off-by: David Ahern --- net/core/dev.c | 52 ++++++++++++++++++++++++-------------------------- 1 file changed, 25 insertions(+), 27 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index e763b6cea8ff..4f0c4fee1125 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4500,25 +4500,17 @@ static struct netdev_rx_queue *netif_get_rxqueue(struct sk_buff *skb) return rxqueue; } -static u32 netif_receive_generic_xdp(struct sk_buff *skb, - struct xdp_buff *xdp, - struct bpf_prog *xdp_prog) +static u32 do_xdp_generic_core(struct sk_buff *skb, struct xdp_buff *xdp, + struct bpf_prog *xdp_prog) { - struct netdev_rx_queue *rxqueue; void *orig_data, *orig_data_end; - u32 metalen, act = XDP_DROP; __be16 orig_eth_type; struct ethhdr *eth; + u32 metalen, act; bool orig_bcast; int hlen, off; u32 mac_len; - /* Reinjected packets coming from act_mirred or similar should - * not get XDP generic processing. - */ - if (skb_is_redirected(skb)) - return XDP_PASS; - /* XDP packets must be linear and must have sufficient headroom * of XDP_PACKET_HEADROOM bytes. This is the guarantee that also * native XDP provides, thus we need to do it here as well. @@ -4534,9 +4526,9 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb, if (pskb_expand_head(skb, hroom > 0 ? ALIGN(hroom, NET_SKB_PAD) : 0, troom > 0 ? troom + 128 : 0, GFP_ATOMIC)) - goto do_drop; + return XDP_DROP; if (skb_linearize(skb)) - goto do_drop; + return XDP_DROP; } /* The XDP program wants to see the packet starting at the MAC @@ -4554,9 +4546,6 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb, orig_bcast = is_multicast_ether_addr_64bits(eth->h_dest); orig_eth_type = eth->h_proto; - rxqueue = netif_get_rxqueue(skb); - xdp->rxq = &rxqueue->xdp_rxq; - act = bpf_prog_run_xdp(xdp_prog, xdp); /* check if bpf_xdp_adjust_head was used */ @@ -4599,16 +4588,6 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb, if (metalen) skb_metadata_set(skb, metalen); break; - default: - bpf_warn_invalid_xdp_action(act); - /* fall through */ - case XDP_ABORTED: - trace_xdp_exception(skb->dev, xdp_prog, act); - /* fall through */ - case XDP_DROP: - do_drop: - kfree_skb(skb); - break; } return act; @@ -4643,12 +4622,22 @@ static DEFINE_STATIC_KEY_FALSE(generic_xdp_needed_key); int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb) { + /* Reinjected packets coming from act_mirred or similar should + * not get XDP generic processing. + */ + if (skb_is_redirected(skb)) + return XDP_PASS; + if (xdp_prog) { + struct netdev_rx_queue *rxqueue; struct xdp_buff xdp; u32 act; int err; - act = netif_receive_generic_xdp(skb, &xdp, xdp_prog); + rxqueue = netif_get_rxqueue(skb); + xdp.rxq = &rxqueue->xdp_rxq; + + act = do_xdp_generic_core(skb, &xdp, xdp_prog); if (act != XDP_PASS) { switch (act) { case XDP_REDIRECT: @@ -4660,6 +4649,15 @@ int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb) case XDP_TX: generic_xdp_tx(skb, xdp_prog); break; + default: + bpf_warn_invalid_xdp_action(act); + /* fall through */ + case XDP_ABORTED: + trace_xdp_exception(skb->dev, xdp_prog, act); + /* fall through */ + case XDP_DROP: + kfree_skb(skb); + break; } return XDP_DROP; } From patchwork Mon Apr 13 17:17:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 221238 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5177CC2BA19 for ; Mon, 13 Apr 2020 17:18:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2D7612072D for ; Mon, 13 Apr 2020 17:18:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586798339; bh=M+lOrV9SlrvxORyu69vZ9EjwGSUXp3w84j9gxHIDtt0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=gv5nhuNzJvfrDtvmF0rHTJ/w0VkY1yqaB+pw3lk++GP2WWYrBQUtmV3ksg5T2avK9 YHnZxOWmpqu705gx7vk3XbdY0oW6246YZEiaoCPCnDrRl2URjbvzSIBTgZdRe2rhlQ 3XWbv4gHnMcd6j3YM7xPICsqB16ewsf5KONzKsaU= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732685AbgDMRS4 (ORCPT ); Mon, 13 Apr 2020 13:18:56 -0400 Received: from mail.kernel.org ([198.145.29.99]:38966 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732659AbgDMRSd (ORCPT ); Mon, 13 Apr 2020 13:18:33 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C15872074B; Mon, 13 Apr 2020 17:18:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586798312; bh=M+lOrV9SlrvxORyu69vZ9EjwGSUXp3w84j9gxHIDtt0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oTRJo0nOkVXYlurAs5LS2zW+pWV8r2qiUEpoSKuOCqdo+f8JYZhHAyWHNQuHqGzEA lOmHDTFoVw/Fy+hQulUNxPfsGRH0lYRYZPuMP6XyxvOA/Rt9Upgt1VBMybric2PXrk TqmNcnzRsBUq00MuPqpVnZSH/gFb039hO201tDa0= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH RFC-v5 bpf-next 06/12] net: core: Rename do_xdp_generic to do_xdp_generic_rx Date: Mon, 13 Apr 2020 11:17:55 -0600 Message-Id: <20200413171801.54406-7-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200413171801.54406-1-dsahern@kernel.org> References: <20200413171801.54406-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Rename do_xdp_generic to do_xdp_generic_rx to emphasize its use in the Rx path. Signed-off-by: Jason Wang Signed-off-by: Prashant Bhole Signed-off-by: David Ahern --- drivers/net/tun.c | 4 ++-- include/linux/netdevice.h | 2 +- net/core/dev.c | 7 ++++--- 3 files changed, 7 insertions(+), 6 deletions(-) diff --git a/drivers/net/tun.c b/drivers/net/tun.c index 228fe449dc6d..20d94bf79cf7 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -1895,7 +1895,7 @@ static ssize_t tun_get_user(struct tun_struct *tun, struct tun_file *tfile, rcu_read_lock(); xdp_prog = rcu_dereference(tun->xdp_prog); if (xdp_prog) { - ret = do_xdp_generic(xdp_prog, skb); + ret = do_xdp_generic_rx(xdp_prog, skb); if (ret != XDP_PASS) { rcu_read_unlock(); local_bh_enable(); @@ -2459,7 +2459,7 @@ static int tun_xdp_one(struct tun_struct *tun, skb_probe_transport_header(skb); if (skb_xdp) { - err = do_xdp_generic(xdp_prog, skb); + err = do_xdp_generic_rx(xdp_prog, skb); if (err != XDP_PASS) goto out; } diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 3133247681fd..2649f2b36858 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3711,7 +3711,7 @@ static inline void dev_consume_skb_any(struct sk_buff *skb) } void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog); -int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb); +int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb); int netif_rx(struct sk_buff *skb); int netif_rx_ni(struct sk_buff *skb); int netif_receive_skb(struct sk_buff *skb); diff --git a/net/core/dev.c b/net/core/dev.c index 4f0c4fee1125..6da613fb6623 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4620,7 +4620,7 @@ void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog) static DEFINE_STATIC_KEY_FALSE(generic_xdp_needed_key); -int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb) +int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb) { /* Reinjected packets coming from act_mirred or similar should * not get XDP generic processing. @@ -4667,7 +4667,7 @@ int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb) kfree_skb(skb); return XDP_DROP; } -EXPORT_SYMBOL_GPL(do_xdp_generic); +EXPORT_SYMBOL_GPL(do_xdp_generic_rx); static int netif_rx_internal(struct sk_buff *skb) { @@ -5017,7 +5017,8 @@ static int __netif_receive_skb_core(struct sk_buff *skb, bool pfmemalloc, int ret2; preempt_disable(); - ret2 = do_xdp_generic(rcu_dereference(skb->dev->xdp_prog), skb); + ret2 = do_xdp_generic_rx(rcu_dereference(skb->dev->xdp_prog), + skb); preempt_enable(); if (ret2 != XDP_PASS) From patchwork Mon Apr 13 17:17:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 221239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A425DC2BBFD for ; Mon, 13 Apr 2020 17:18:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8134420735 for ; Mon, 13 Apr 2020 17:18:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586798328; bh=4xXg9Ak70ikS6gqWoDvWLciekwD+swq945hMj/6z0To=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=DmY4waRl09ttT3N+uIukd6SEUT2VTK5+Qt0L08XHcPUhkt817gfjACwhkogB5le9K N3ZOIssO5T9b+IDyV/OKFu8d1mxLuKBgtzzTQOFJFwM8yfpOhl/8os6W64JJr9nrCk +xsVDcuBgQG2YzM+zM1a+8vfbZ9sIWP4YwFXk26M= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732677AbgDMRSr (ORCPT ); Mon, 13 Apr 2020 13:18:47 -0400 Received: from mail.kernel.org ([198.145.29.99]:39148 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732635AbgDMRSh (ORCPT ); Mon, 13 Apr 2020 13:18:37 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 981302082E; Mon, 13 Apr 2020 17:18:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586798316; bh=4xXg9Ak70ikS6gqWoDvWLciekwD+swq945hMj/6z0To=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PqinOY4uDzOSsEiTwy0iZ8X7njeVwjzV2EfW3rzT/+3N/LJszCX4UnxYujIPiYB+j HMKJ7eNWKDrPk6eIWPDe5G5z60ZNxX3a/he/chFju0RnlS6Sv+zycJAyACHEZ64pdX 6SdUvBh4xo7SU72EXVIKzFmVjbI0gFOhtOiTHfKA= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH RFC-v5 bpf-next 08/12] dev: Support xdp in the Tx path for packets as an skb Date: Mon, 13 Apr 2020 11:17:57 -0600 Message-Id: <20200413171801.54406-9-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200413171801.54406-1-dsahern@kernel.org> References: <20200413171801.54406-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Add support to run Tx path program on packets about to hit the ndo_start_xmit function for a device. Only XDP_DROP and XDP_PASS are supported now. Conceptually, XDP_REDIRECT for this path can work the same as it does for the Rx path, but that support is left for a follow on series. Signed-off-by: David Ahern --- include/linux/netdevice.h | 11 +++++++++ net/core/dev.c | 52 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 62 insertions(+), 1 deletion(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 0c89996a6bec..39e1b42c042f 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3714,6 +3714,7 @@ static inline void dev_consume_skb_any(struct sk_buff *skb) void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog); int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb); +u32 do_xdp_egress_skb(struct net_device *dev, struct sk_buff *skb); int netif_rx(struct sk_buff *skb); int netif_rx_ni(struct sk_buff *skb); int netif_receive_skb(struct sk_buff *skb); @@ -4534,6 +4535,16 @@ static inline netdev_tx_t __netdev_start_xmit(const struct net_device_ops *ops, struct sk_buff *skb, struct net_device *dev, bool more) { + if (static_branch_unlikely(&xdp_egress_needed_key)) { + u32 act; + + rcu_read_lock(); + act = do_xdp_egress_skb(dev, skb); + rcu_read_unlock(); + if (act == XDP_DROP) + return NET_XMIT_DROP; + } + __this_cpu_write(softnet_data.xmit.more, more); return ops->ndo_start_xmit(skb, dev); } diff --git a/net/core/dev.c b/net/core/dev.c index c879c291244a..1bbaeb8842ed 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4619,7 +4619,6 @@ void generic_xdp_tx(struct sk_buff *skb, struct bpf_prog *xdp_prog) } static DEFINE_STATIC_KEY_FALSE(generic_xdp_needed_key); -DEFINE_STATIC_KEY_FALSE(xdp_egress_needed_key); int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb) { @@ -4670,6 +4669,57 @@ int do_xdp_generic_rx(struct bpf_prog *xdp_prog, struct sk_buff *skb) } EXPORT_SYMBOL_GPL(do_xdp_generic_rx); +DEFINE_STATIC_KEY_FALSE(xdp_egress_needed_key); +EXPORT_SYMBOL_GPL(xdp_egress_needed_key); + +static u32 handle_xdp_egress_act(u32 act, struct net_device *dev, + struct bpf_prog *xdp_prog) +{ + switch (act) { + case XDP_DROP: + /* fall through */ + case XDP_PASS: + break; + case XDP_TX: + /* fall through */ + case XDP_REDIRECT: + /* fall through */ + default: + bpf_warn_invalid_xdp_action(act); + /* fall through */ + case XDP_ABORTED: + trace_xdp_exception(dev, xdp_prog, act); + act = XDP_DROP; + break; + } + + return act; +} + +u32 do_xdp_egress_skb(struct net_device *dev, struct sk_buff *skb) +{ + struct bpf_prog *xdp_prog; + u32 act = XDP_PASS; + + xdp_prog = rcu_dereference(dev->xdp_egress_prog); + if (xdp_prog) { + struct xdp_txq_info txq = { .dev = dev }; + struct xdp_buff xdp; + + xdp.txq = &txq; + act = do_xdp_generic_core(skb, &xdp, xdp_prog); + act = handle_xdp_egress_act(act, dev, xdp_prog); + if (act == XDP_DROP) { + atomic_long_inc(&dev->tx_dropped); + skb_tx_error(skb); + kfree_skb(skb); + } + } + + return act; +} +EXPORT_SYMBOL_GPL(do_xdp_egress_skb); + static int netif_rx_internal(struct sk_buff *skb) { int ret; From patchwork Mon Apr 13 17:17:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 221237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C467C2BA19 for ; Mon, 13 Apr 2020 17:19:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1F75E20735 for ; Mon, 13 Apr 2020 17:19:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586798349; bh=NP/m3h/6ENqZtfvSlbKSZWe1sVTvikOieq2/Ok6Xm9g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=xfE67H6XN0vpkMhF/R8Z4tw09UTtuLoFYfm2UpDlZclwdZi5AP7gU4nmBYG1F/mgs j5xuik4sU94o7W8CIrvZD3GRANqXloQHmly4By9ZQx+l/jLx3kgrvvn5DU+gCXea6i xC857qxi4KNXssZ2VWZIVfIQ+bCAeUG5ycRR/vjg= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732697AbgDMRTF (ORCPT ); Mon, 13 Apr 2020 13:19:05 -0400 Received: from mail.kernel.org ([198.145.29.99]:39318 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732666AbgDMRSj (ORCPT ); Mon, 13 Apr 2020 13:18:39 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 573022084D; Mon, 13 Apr 2020 17:18:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586798318; bh=NP/m3h/6ENqZtfvSlbKSZWe1sVTvikOieq2/Ok6Xm9g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sNJQgnyZB44ZbFk3RGv5m162ZAcb3RMNpPWJszFcNPRec0K3GGGVCYwhKDXWabzbN cFb+Bs2AJNu4qogNdU+FGNNS2knOlzY5kqqlRck0CCZoRhekbNLhTDuQtA3e23I9Vf 5S3+Z8iqIYmiGJcc+KwPmEwr09W+GNnw9bqngtZI= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH RFC-v5 bpf-next 10/12] libbpf: Add egress XDP support Date: Mon, 13 Apr 2020 11:17:59 -0600 Message-Id: <20200413171801.54406-11-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200413171801.54406-1-dsahern@kernel.org> References: <20200413171801.54406-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern Patch adds egress XDP support in libbpf. New section name hint, xdp_egress, is added to set expected attach type at program load. Programs can use xdp_egress as the prefix in the SEC statement to load the program with the BPF_XDP_EGRESS attach type set. egress is added to bpf_xdp_set_link_opts to specify egress type for use with bpf_set_link_xdp_fd_opts. Update library side to check for flag and set nla_type to IFLA_XDP_EGRESS. Add egress version of bpf_get_link_xdp* info and id apis with core code refactored to handle both rx and tx paths. Signed-off-by: Prashant Bhole Co-developed-by: David Ahern Signed-off-by: David Ahern --- tools/lib/bpf/libbpf.c | 2 ++ tools/lib/bpf/libbpf.h | 9 +++++- tools/lib/bpf/libbpf.map | 2 ++ tools/lib/bpf/netlink.c | 63 +++++++++++++++++++++++++++++++++++----- 4 files changed, 67 insertions(+), 9 deletions(-) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index ff9174282a8c..463f55e6e82f 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -6322,6 +6322,8 @@ static const struct bpf_sec_def section_defs[] = { .is_attach_btf = true, .expected_attach_type = BPF_LSM_MAC, .attach_fn = attach_lsm), + BPF_EAPROG_SEC("xdp_egress", BPF_PROG_TYPE_XDP, + BPF_XDP_EGRESS), BPF_PROG_SEC("xdp", BPF_PROG_TYPE_XDP), BPF_PROG_SEC("perf_event", BPF_PROG_TYPE_PERF_EVENT), BPF_PROG_SEC("lwt_in", BPF_PROG_TYPE_LWT_IN), diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index 44df1d3e7287..2de56b9c6397 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -453,14 +453,16 @@ struct xdp_link_info { __u32 drv_prog_id; __u32 hw_prog_id; __u32 skb_prog_id; + __u32 egress_core_prog_id; __u8 attach_mode; }; struct bpf_xdp_set_link_opts { size_t sz; __u32 old_fd; + __u8 egress; }; -#define bpf_xdp_set_link_opts__last_field old_fd +#define bpf_xdp_set_link_opts__last_field egress LIBBPF_API int bpf_set_link_xdp_fd(int ifindex, int fd, __u32 flags); LIBBPF_API int bpf_set_link_xdp_fd_opts(int ifindex, int fd, __u32 flags, @@ -468,6 +470,11 @@ LIBBPF_API int bpf_set_link_xdp_fd_opts(int ifindex, int fd, __u32 flags, LIBBPF_API int bpf_get_link_xdp_id(int ifindex, __u32 *prog_id, __u32 flags); LIBBPF_API int bpf_get_link_xdp_info(int ifindex, struct xdp_link_info *info, size_t info_size, __u32 flags); +LIBBPF_API int bpf_get_link_xdp_egress_id(int ifindex, __u32 *prog_id, + __u32 flags); +LIBBPF_API int bpf_get_link_xdp_egress_info(int ifindex, + struct xdp_link_info *info, + size_t info_size, __u32 flags); struct perf_buffer; diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index bb8831605b25..51576c8a02fe 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -253,4 +253,6 @@ LIBBPF_0.0.8 { bpf_program__set_attach_target; bpf_program__set_lsm; bpf_set_link_xdp_fd_opts; + bpf_get_link_xdp_egress_id; + bpf_get_link_xdp_egress_info; } LIBBPF_0.0.7; diff --git a/tools/lib/bpf/netlink.c b/tools/lib/bpf/netlink.c index 18b5319025e1..31c4252a6908 100644 --- a/tools/lib/bpf/netlink.c +++ b/tools/lib/bpf/netlink.c @@ -28,6 +28,7 @@ typedef int (*__dump_nlmsg_t)(struct nlmsghdr *nlmsg, libbpf_dump_nlmsg_t, struct xdp_id_md { int ifindex; __u32 flags; + __u16 nla_type; struct xdp_link_info info; }; @@ -133,7 +134,7 @@ static int bpf_netlink_recv(int sock, __u32 nl_pid, int seq, } static int __bpf_set_link_xdp_fd_replace(int ifindex, int fd, int old_fd, - __u32 flags) + __u32 flags, __u16 nla_type) { int sock, seq = 0, ret; struct nlattr *nla, *nla_xdp; @@ -160,7 +161,7 @@ static int __bpf_set_link_xdp_fd_replace(int ifindex, int fd, int old_fd, /* started nested attribute for XDP */ nla = (struct nlattr *)(((char *)&req) + NLMSG_ALIGN(req.nh.nlmsg_len)); - nla->nla_type = NLA_F_NESTED | IFLA_XDP; + nla->nla_type = NLA_F_NESTED | nla_type; nla->nla_len = NLA_HDRLEN; /* add XDP fd */ @@ -203,6 +204,7 @@ static int __bpf_set_link_xdp_fd_replace(int ifindex, int fd, int old_fd, int bpf_set_link_xdp_fd_opts(int ifindex, int fd, __u32 flags, const struct bpf_xdp_set_link_opts *opts) { + __u16 nla_type = IFLA_XDP; int old_fd = -1; if (!OPTS_VALID(opts, bpf_xdp_set_link_opts)) @@ -213,14 +215,22 @@ int bpf_set_link_xdp_fd_opts(int ifindex, int fd, __u32 flags, flags |= XDP_FLAGS_REPLACE; } + if (OPTS_HAS(opts, egress)) { + __u8 egress = OPTS_GET(opts, egress, 0); + + if (egress) + nla_type = IFLA_XDP_EGRESS; + } + return __bpf_set_link_xdp_fd_replace(ifindex, fd, old_fd, - flags); + flags, + nla_type); } int bpf_set_link_xdp_fd(int ifindex, int fd, __u32 flags) { - return __bpf_set_link_xdp_fd_replace(ifindex, fd, 0, flags); + return __bpf_set_link_xdp_fd_replace(ifindex, fd, 0, flags, IFLA_XDP); } static int __dump_link_nlmsg(struct nlmsghdr *nlh, @@ -243,15 +253,16 @@ static int get_xdp_info(void *cookie, void *msg, struct nlattr **tb) struct nlattr *xdp_tb[IFLA_XDP_MAX + 1]; struct xdp_id_md *xdp_id = cookie; struct ifinfomsg *ifinfo = msg; + __u16 atype = xdp_id->nla_type; int ret; if (xdp_id->ifindex && xdp_id->ifindex != ifinfo->ifi_index) return 0; - if (!tb[IFLA_XDP]) + if (!tb[atype]) return 0; - ret = libbpf_nla_parse_nested(xdp_tb, IFLA_XDP_MAX, tb[IFLA_XDP], NULL); + ret = libbpf_nla_parse_nested(xdp_tb, IFLA_XDP_MAX, tb[atype], NULL); if (ret) return ret; @@ -280,11 +291,16 @@ static int get_xdp_info(void *cookie, void *msg, struct nlattr **tb) xdp_id->info.hw_prog_id = libbpf_nla_getattr_u32( xdp_tb[IFLA_XDP_HW_PROG_ID]); + if (xdp_tb[IFLA_XDP_EGRESS_CORE_PROG_ID]) + xdp_id->info.egress_core_prog_id = libbpf_nla_getattr_u32( + xdp_tb[IFLA_XDP_EGRESS_CORE_PROG_ID]); + return 0; } -int bpf_get_link_xdp_info(int ifindex, struct xdp_link_info *info, - size_t info_size, __u32 flags) +static int __bpf_get_link_xdp_info(int ifindex, struct xdp_link_info *info, + size_t info_size, __u32 flags, + __u16 nla_type) { struct xdp_id_md xdp_id = {}; int sock, ret; @@ -306,6 +322,7 @@ int bpf_get_link_xdp_info(int ifindex, struct xdp_link_info *info, xdp_id.ifindex = ifindex; xdp_id.flags = flags; + xdp_id.nla_type = nla_type; ret = libbpf_nl_get_link(sock, nl_pid, get_xdp_info, &xdp_id); if (!ret) { @@ -319,6 +336,20 @@ int bpf_get_link_xdp_info(int ifindex, struct xdp_link_info *info, return ret; } +int bpf_get_link_xdp_info(int ifindex, struct xdp_link_info *info, + size_t info_size, __u32 flags) +{ + return __bpf_get_link_xdp_info(ifindex, info, info_size, flags, + IFLA_XDP); +} + +int bpf_get_link_xdp_egress_info(int ifindex, struct xdp_link_info *info, + size_t info_size, __u32 flags) +{ + return __bpf_get_link_xdp_info(ifindex, info, info_size, flags, + IFLA_XDP_EGRESS); +} + static __u32 get_xdp_id(struct xdp_link_info *info, __u32 flags) { if (info->attach_mode != XDP_ATTACHED_MULTI) @@ -345,6 +376,22 @@ int bpf_get_link_xdp_id(int ifindex, __u32 *prog_id, __u32 flags) return ret; } +int bpf_get_link_xdp_egress_id(int ifindex, __u32 *prog_id, __u32 flags) +{ + struct xdp_link_info info; + int ret; + + /* egress path does not support SKB, DRV or HW mode */ + if (flags & XDP_FLAGS_MODES) + return -EINVAL; + + ret = bpf_get_link_xdp_egress_info(ifindex, &info, sizeof(info), flags); + if (!ret) + *prog_id = get_xdp_id(&info, flags); + + return ret; +} + int libbpf_nl_get_link(int sock, unsigned int nl_pid, libbpf_dump_nlmsg_t dump_link_nlmsg, void *cookie) { From patchwork Mon Apr 13 17:18:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 221234 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36574C2BA2B for ; Mon, 13 Apr 2020 17:19:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 17B102072D for ; Mon, 13 Apr 2020 17:19:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586798370; bh=diygsHrujtkxdOjgK42/j6clPIEVUq33QjY8ljsaJZ8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=rNlzZrXY/JoxXWFvJZ4Sz6ZdbjeObzSfJruAn2zIhVkC9+7Slngln9FO0o2KuoMq7 y0A9Ajsud2vpuXdYxRYs3e4oAfDRVPcAlP2LquNrPfhDPjwMTL5+Tnrg5G044zItjP c68hTVt2q499pV9PCjp6yjUMWVXltR5mENU3WaaU= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732719AbgDMRT2 (ORCPT ); Mon, 13 Apr 2020 13:19:28 -0400 Received: from mail.kernel.org ([198.145.29.99]:39318 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732670AbgDMRSl (ORCPT ); Mon, 13 Apr 2020 13:18:41 -0400 Received: from C02YQ0RWLVCF.internal.digitalocean.com (c-73-181-34-237.hsd1.co.comcast.net [73.181.34.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AB9C72076A; Mon, 13 Apr 2020 17:18:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586798320; bh=diygsHrujtkxdOjgK42/j6clPIEVUq33QjY8ljsaJZ8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eyMJxX8HXiN3TbKYlFUIjxCPxJOvQl3I89PUQ66/DVXl6hZnEErGslg8MdgtNCtdI pTDSfn1yb69U3ckJvl+vc4KfVxMSU+3TfNcDTmHxJcM5I4A4UyzLlugFPsgtJ8MWFN GzzGTTB9FxIVyKNgp9K9gxp5JjPXCk0eU0OyUd9M= From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, prashantbhole.linux@gmail.com, jasowang@redhat.com, brouer@redhat.com, toke@redhat.com, toshiaki.makita1@gmail.com, daniel@iogearbox.net, john.fastabend@gmail.com, ast@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, andriin@fb.com, dsahern@gmail.com, David Ahern Subject: [PATCH RFC-v5 bpf-next 12/12] samples/bpf: add XDP egress support to xdp1 Date: Mon, 13 Apr 2020 11:18:01 -0600 Message-Id: <20200413171801.54406-13-dsahern@kernel.org> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200413171801.54406-1-dsahern@kernel.org> References: <20200413171801.54406-1-dsahern@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: David Ahern xdp1 and xdp2 now accept -E flag to set XDP program in the egress path. Signed-off-by: Prashant Bhole Signed-off-by: David Ahern --- samples/bpf/xdp1_user.c | 38 +++++++++++++++++++++++++++++++------- 1 file changed, 31 insertions(+), 7 deletions(-) diff --git a/samples/bpf/xdp1_user.c b/samples/bpf/xdp1_user.c index c447ad9e3a1d..1e74020203ca 100644 --- a/samples/bpf/xdp1_user.c +++ b/samples/bpf/xdp1_user.c @@ -21,18 +21,32 @@ static int ifindex; static __u32 xdp_flags = XDP_FLAGS_UPDATE_IF_NOEXIST; static __u32 prog_id; +static bool egress; static void int_exit(int sig) { + struct bpf_xdp_set_link_opts opts = { .sz = sizeof(opts) }; __u32 curr_prog_id = 0; + int err, old_fd; - if (bpf_get_link_xdp_id(ifindex, &curr_prog_id, xdp_flags)) { + if (egress) + err = bpf_get_link_xdp_egress_id(ifindex, &curr_prog_id, + xdp_flags); + else + err = bpf_get_link_xdp_id(ifindex, &curr_prog_id, xdp_flags); + if (err) { printf("bpf_get_link_xdp_id failed\n"); exit(1); } - if (prog_id == curr_prog_id) - bpf_set_link_xdp_fd(ifindex, -1, xdp_flags); - else if (!curr_prog_id) + if (prog_id == curr_prog_id) { + if (egress) + opts.egress = 1; + + old_fd = bpf_prog_get_fd_by_id(prog_id); + opts.old_fd = old_fd; + bpf_set_link_xdp_fd_opts(ifindex, -1, xdp_flags, &opts); + close(old_fd); + } else if (!curr_prog_id) printf("couldn't find a prog id on a given interface\n"); else printf("program on interface changed, not removing\n"); @@ -73,19 +87,21 @@ static void usage(const char *prog) "OPTS:\n" " -S use skb-mode\n" " -N enforce native mode\n" - " -F force loading prog\n", + " -F force loading prog\n" + " -E egress path program\n", prog); } int main(int argc, char **argv) { + struct bpf_xdp_set_link_opts opts = { .sz = sizeof(opts) }; struct rlimit r = {RLIM_INFINITY, RLIM_INFINITY}; struct bpf_prog_load_attr prog_load_attr = { .prog_type = BPF_PROG_TYPE_XDP, }; struct bpf_prog_info info = {}; __u32 info_len = sizeof(info); - const char *optstr = "FSN"; + const char *optstr = "FSNE"; int prog_fd, map_fd, opt; struct bpf_object *obj; struct bpf_map *map; @@ -103,6 +119,9 @@ int main(int argc, char **argv) case 'F': xdp_flags &= ~XDP_FLAGS_UPDATE_IF_NOEXIST; break; + case 'E': + egress = true; + break; default: usage(basename(argv[0])); return 1; @@ -130,6 +149,10 @@ int main(int argc, char **argv) snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]); prog_load_attr.file = filename; + if (egress) { + opts.egress = 1; + prog_load_attr.expected_attach_type = BPF_XDP_EGRESS; + } if (bpf_prog_load_xattr(&prog_load_attr, &obj, &prog_fd)) return 1; @@ -149,7 +172,8 @@ int main(int argc, char **argv) signal(SIGINT, int_exit); signal(SIGTERM, int_exit); - if (bpf_set_link_xdp_fd(ifindex, prog_fd, xdp_flags) < 0) { + err = bpf_set_link_xdp_fd_opts(ifindex, prog_fd, xdp_flags, &opts); + if (err < 0) { printf("link set xdp fd failed\n"); return 1; }