From patchwork Mon Nov 16 09:34:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marek Majtyka X-Patchwork-Id: 325155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E9DDC5519F for ; Mon, 16 Nov 2020 10:09:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AEF3B216C4 for ; Mon, 16 Nov 2020 10:09:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WlEbLFYd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728567AbgKPJfR (ORCPT ); Mon, 16 Nov 2020 04:35:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55282 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726621AbgKPJfQ (ORCPT ); Mon, 16 Nov 2020 04:35:16 -0500 Received: from mail-lf1-x144.google.com (mail-lf1-x144.google.com [IPv6:2a00:1450:4864:20::144]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A386C0613CF; Mon, 16 Nov 2020 01:35:16 -0800 (PST) Received: by mail-lf1-x144.google.com with SMTP id 74so24195985lfo.5; Mon, 16 Nov 2020 01:35:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DRogxLAWeILeQ6O5RXo7aniYeaKUoqbSS7azBUAy5Kw=; b=WlEbLFYdpEyjkHcsw64/Ldawgo53paFUAqNMf76aMfyLQzOm1q6hBp1Wx/Qw6UGvyy U6Mlfa0SXIZBGPX88KZHQxhWv8pz+DCJxINMnz/nBk2RsnzA5/Y7cGXqUk0G6Abp3XuO 6n1oQLC85mIbPWdXc4e/6/6FWZxPiL90YX07i1QWcgUFqOx7RvbXSJd4/LRX/0OJvAFs NuTDcYrMl0mIWdvpEpG9NiFf/GbVSDbxLTfzZNOe35Gb1z0GdIBO82kvgTAUg7cadFqX K/kF/OcH7+1VDnoSG5z4bL5+FZ0Gk1ymdYj7/4vGDdps8/ObgNI08jJ4reLErhA8wlf9 UjQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DRogxLAWeILeQ6O5RXo7aniYeaKUoqbSS7azBUAy5Kw=; b=RBfWoft8C3ICyOJudMYETZDViqT3Deiia7JsEgQGBO46h5YW7j4l2dlHlS/K3ToOWF b4blDtONvMm96t5Zfd9rFPj0/ZXlrFlyrDPyQr8wK6h3WAyPpmGAz1AMELyaUZV0FS4i 8WtAOwQIlioH9Pw1XVtQVqL2EQCxAys+oBZM8h5hfTfQZTh3F7QLILeqqfaVNg5nhpvm nU/ES4BaplK4feSgO7AQPXihabalbRyEue8YZIVg+kPrP+ta+IrklWzTn58XBTmLoukS ih3R3uVeBS/bKoXv3tpdzaEa8vjPCOuMjxfoMyKsCLYS2vHW2pTtIvll341b/5YGNP0n dEHQ== X-Gm-Message-State: AOAM530v/KWF0wV25/4695PP8v2i7tIzGDAlrAj7tL/Tak8N//F8CUbz xDNMPknP0Cxo9NOJB7HsmSo= X-Google-Smtp-Source: ABdhPJwZSitmq4q2+XMdKps3x1Y/CBuSJ5fG9gJYLmD19asV8v9AL897u2KYDgbUKBIpY8c3dmg/iA== X-Received: by 2002:a05:6512:74e:: with SMTP id c14mr5468117lfs.463.1605519314602; Mon, 16 Nov 2020 01:35:14 -0800 (PST) Received: from localhost.localdomain (87-205-71-93.adsl.inetia.pl. [87.205.71.93]) by smtp.gmail.com with ESMTPSA id t26sm2667986lfp.296.2020.11.16.01.35.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 01:35:13 -0800 (PST) From: alardam@gmail.com X-Google-Original-From: marekx.majtyka@intel.com To: magnus.karlsson@intel.com, bjorn.topel@intel.com, andrii.nakryiko@gmail.com, kuba@kernel.org, ast@kernel.org, daniel@iogearbox.net, netdev@vger.kernel.org, davem@davemloft.net, john.fastabend@gmail.com, hawk@kernel.org, toke@redhat.com Cc: maciej.fijalkowski@intel.com, jonathan.lemon@gmail.com, bpf@vger.kernel.org, jeffrey.t.kirsher@intel.com, maciejromanfijalkowski@gmail.com, intel-wired-lan@lists.osuosl.org, Marek Majtyka Subject: [PATCH 2/8] drivers/net: turn XDP flags on Date: Mon, 16 Nov 2020 10:34:46 +0100 Message-Id: <20201116093452.7541-3-marekx.majtyka@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201116093452.7541-1-marekx.majtyka@intel.com> References: <20201116093452.7541-1-marekx.majtyka@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Marek Majtyka Turn 'xdp' and 'af-xdp-zc' feature flags on for: - i40e - ice - ixgbe - mlx5. Turn 'xdp' feature flag on for: - igb - tun - veth - dpaa2 - mvneta - mvpp2 - qede - sfc - netsec - cpsw - xen - virtio_net. The first group of NICs is currently visible with ethtool as: $ ethtool -k enp1s0f0 Features for enp1s0f0: .. xdp: on [fixed] af-xdp-zc: on [fixed] whereas for the second group output is as: $ ethtool -k enp1s0f0 Features for enp1s0f0: .. xdp: on [fixed] af-xdp-zc: off [fixed] Signed-off-by: Marek Majtyka --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 1 + drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c | 1 + drivers/net/ethernet/intel/i40e/i40e_main.c | 2 ++ drivers/net/ethernet/intel/ice/ice_main.c | 4 ++++ drivers/net/ethernet/intel/igb/igb_main.c | 2 ++ drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 3 +++ drivers/net/ethernet/marvell/mvneta.c | 1 + drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 1 + drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 ++ drivers/net/ethernet/qlogic/qede/qede_main.c | 1 + drivers/net/ethernet/sfc/efx.c | 1 + drivers/net/ethernet/socionext/netsec.c | 1 + drivers/net/ethernet/ti/cpsw.c | 2 ++ drivers/net/ethernet/ti/cpsw_new.c | 2 ++ drivers/net/tun.c | 4 ++++ drivers/net/veth.c | 1 + drivers/net/virtio_net.c | 1 + drivers/net/xen-netfront.c | 1 + 18 files changed, 31 insertions(+) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 7975f59735d6..9f689717319d 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -12604,6 +12604,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) dev->features |= dev->hw_features | NETIF_F_HIGHDMA; if (dev->features & NETIF_F_GRO_HW) dev->features &= ~NETIF_F_LRO; + xdp_set_feature_flag(&dev->features); dev->priv_flags |= IFF_UNICAST_FLT; #ifdef CONFIG_BNXT_SRIOV diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c index cf9400a9886d..418ec3dae1dd 100644 --- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c +++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c @@ -4014,6 +4014,7 @@ static int dpaa2_eth_netdev_init(struct net_device *net_dev) NETIF_F_SG | NETIF_F_HIGHDMA | NETIF_F_LLTX | NETIF_F_HW_TC; net_dev->hw_features = net_dev->features; + xdp_set_feature_flag(&net_dev->features); return 0; } diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c index 4f8a2154b93f..0b7825f629c6 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_main.c +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c @@ -12873,6 +12873,8 @@ static int i40e_config_netdev(struct i40e_vsi *vsi) netdev->hw_features |= hw_features; netdev->features |= hw_features | NETIF_F_HW_VLAN_CTAG_FILTER; + xdp_set_feature_flag(&netdev->features); + xsk_set_feature_flag(&netdev->features); netdev->hw_enc_features |= NETIF_F_TSO_MANGLEID; if (vsi->type == I40E_VSI_MAIN) { diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index 2dea4d0e9415..7b932ba42f09 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -13,6 +13,7 @@ #include "ice_dcb_lib.h" #include "ice_dcb_nl.h" #include "ice_devlink.h" +#include #define DRV_SUMMARY "Intel(R) Ethernet Connection E800 Series Linux Driver" static const char ice_driver_string[] = DRV_SUMMARY; @@ -2941,6 +2942,9 @@ static void ice_set_netdev_features(struct net_device *netdev) /* enable features */ netdev->features |= netdev->hw_features; + xdp_set_feature_flag(&netdev->features); + xsk_set_feature_flag(&netdev->features); + /* encap and VLAN devices inherit default, csumo and tso features */ netdev->hw_enc_features |= dflt_features | csumo_features | tso_features; diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 5fc2c381da55..e89a0442606f 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -3293,6 +3293,8 @@ static int igb_probe(struct pci_dev *pdev, const struct pci_device_id *ent) NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_HW_VLAN_CTAG_TX; + xdp_set_feature_flag(&netdev->features); + netdev->priv_flags |= IFF_SUPP_NOFCS; netdev->priv_flags |= IFF_UNICAST_FLT; diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index 45ae33e15303..fccc282a9492 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -10880,6 +10880,9 @@ static int ixgbe_probe(struct pci_dev *pdev, const struct pci_device_id *ent) if (adapter->flags2 & IXGBE_FLAG2_RSC_ENABLED) netdev->features |= NETIF_F_LRO; + xdp_set_feature_flag(&netdev->features); + xsk_set_feature_flag(&netdev->features); + if (ixgbe_check_fw_error(adapter)) { err = -EIO; goto err_sw_init; diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c index 183530ed4d1d..b80c1eca820e 100644 --- a/drivers/net/ethernet/marvell/mvneta.c +++ b/drivers/net/ethernet/marvell/mvneta.c @@ -5262,6 +5262,7 @@ static int mvneta_probe(struct platform_device *pdev) NETIF_F_TSO | NETIF_F_RXCSUM; dev->hw_features |= dev->features; dev->vlan_features |= dev->features; + xdp_set_feature_flag(&dev->features); dev->priv_flags |= IFF_LIVE_ADDR_CHANGE; dev->gso_max_segs = MVNETA_MAX_TSO_SEGS; diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c index 3069e192d773..6add63ef9ac0 100644 --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c @@ -6475,6 +6475,7 @@ static int mvpp2_port_probe(struct platform_device *pdev, mvpp2_set_hw_csum(port, port->pool_long->id); dev->vlan_features |= features; + xdp_set_feature_flag(&dev->features); dev->gso_max_segs = MVPP2_MAX_TSO_SEGS; dev->priv_flags |= IFF_UNICAST_FLT; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 527c5f12c5af..67aaec330816 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -4980,6 +4980,8 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev) netdev->features |= NETIF_F_HIGHDMA; netdev->features |= NETIF_F_HW_VLAN_STAG_FILTER; + xdp_set_feature_flag(&netdev->features); + xsk_set_feature_flag(&netdev->features); netdev->priv_flags |= IFF_UNICAST_FLT; diff --git a/drivers/net/ethernet/qlogic/qede/qede_main.c b/drivers/net/ethernet/qlogic/qede/qede_main.c index 05e3a3b60269..add1b820adea 100644 --- a/drivers/net/ethernet/qlogic/qede/qede_main.c +++ b/drivers/net/ethernet/qlogic/qede/qede_main.c @@ -839,6 +839,7 @@ static void qede_init_ndev(struct qede_dev *edev) ndev->features = hw_features | NETIF_F_RXHASH | NETIF_F_RXCSUM | NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_HIGHDMA | NETIF_F_HW_VLAN_CTAG_FILTER | NETIF_F_HW_VLAN_CTAG_TX; + xdp_set_feature_flag(&ndev->features); ndev->hw_features = hw_features; diff --git a/drivers/net/ethernet/sfc/efx.c b/drivers/net/ethernet/sfc/efx.c index 718308076341..ca5a582ce26b 100644 --- a/drivers/net/ethernet/sfc/efx.c +++ b/drivers/net/ethernet/sfc/efx.c @@ -1101,6 +1101,7 @@ static int efx_pci_probe(struct pci_dev *pci_dev, efx = netdev_priv(net_dev); efx->type = (const struct efx_nic_type *) entry->driver_data; efx->fixed_features |= NETIF_F_HIGHDMA; + xdp_set_feature_flag(&efx->fixed_features); pci_set_drvdata(pci_dev, efx); SET_NETDEV_DEV(net_dev, &pci_dev->dev); diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c index 1503cc9ec6e2..8a8cca53821d 100644 --- a/drivers/net/ethernet/socionext/netsec.c +++ b/drivers/net/ethernet/socionext/netsec.c @@ -2099,6 +2099,7 @@ static int netsec_probe(struct platform_device *pdev) ndev->features |= NETIF_F_HIGHDMA | NETIF_F_RXCSUM | NETIF_F_GSO | NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM; ndev->hw_features = ndev->features; + xdp_set_feature_flag(&ndev->features); priv->rx_cksum_offload_flag = true; diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c index 9fd1f77190ad..ca5548f92b56 100644 --- a/drivers/net/ethernet/ti/cpsw.c +++ b/drivers/net/ethernet/ti/cpsw.c @@ -1475,6 +1475,7 @@ static int cpsw_probe_dual_emac(struct cpsw_priv *priv) priv_sl2->emac_port = 1; cpsw->slaves[1].ndev = ndev; ndev->features |= NETIF_F_HW_VLAN_CTAG_FILTER | NETIF_F_HW_VLAN_CTAG_RX; + xdp_set_feature_flag(&ndev->features); ndev->netdev_ops = &cpsw_netdev_ops; ndev->ethtool_ops = &cpsw_ethtool_ops; @@ -1654,6 +1655,7 @@ static int cpsw_probe(struct platform_device *pdev) cpsw->slaves[0].ndev = ndev; ndev->features |= NETIF_F_HW_VLAN_CTAG_FILTER | NETIF_F_HW_VLAN_CTAG_RX; + xdp_set_feature_flag(&ndev->features); ndev->netdev_ops = &cpsw_netdev_ops; ndev->ethtool_ops = &cpsw_ethtool_ops; diff --git a/drivers/net/ethernet/ti/cpsw_new.c b/drivers/net/ethernet/ti/cpsw_new.c index f779d2e1b5c5..9a8fb3dd8a19 100644 --- a/drivers/net/ethernet/ti/cpsw_new.c +++ b/drivers/net/ethernet/ti/cpsw_new.c @@ -1416,6 +1416,8 @@ static int cpsw_create_ports(struct cpsw_common *cpsw) ndev->features |= NETIF_F_HW_VLAN_CTAG_FILTER | NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_NETNS_LOCAL; + xdp_set_feature_flag(&ndev->features); + ndev->netdev_ops = &cpsw_netdev_ops; ndev->ethtool_ops = &cpsw_ethtool_ops; SET_NETDEV_DEV(ndev, dev); diff --git a/drivers/net/tun.c b/drivers/net/tun.c index 3d45d56172cb..2187714598d6 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -2721,6 +2721,10 @@ static int tun_set_iff(struct net *net, struct file *file, struct ifreq *ifr) ~(NETIF_F_HW_VLAN_CTAG_TX | NETIF_F_HW_VLAN_STAG_TX); + /* Currently tap does not support XDP, only tun does. */ + if (tun->flags == IFF_TUN) + xdp_set_feature_flag(&dev->features); + tun->flags = (tun->flags & ~TUN_FEATURES) | (ifr->ifr_flags & TUN_FEATURES); diff --git a/drivers/net/veth.c b/drivers/net/veth.c index 8c737668008a..26f2e83d9a2c 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -1266,6 +1266,7 @@ static void veth_setup(struct net_device *dev) dev->hw_features = VETH_FEATURES; dev->hw_enc_features = VETH_FEATURES; dev->mpls_features = NETIF_F_HW_CSUM | NETIF_F_GSO_SOFTWARE; + xdp_set_feature_flag(&dev->features); } /* diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 21b71148c532..ee33e9fce4c5 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -3017,6 +3017,7 @@ static int virtnet_probe(struct virtio_device *vdev) dev->hw_features |= NETIF_F_LRO; dev->vlan_features = dev->features; + xdp_set_feature_flag(&dev->features); /* MTU range: 68 - 65535 */ dev->min_mtu = MIN_MTU; diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c index 920cac4385bf..abe5f0104c73 100644 --- a/drivers/net/xen-netfront.c +++ b/drivers/net/xen-netfront.c @@ -1555,6 +1555,7 @@ static struct net_device *xennet_create_dev(struct xenbus_device *dev) * negotiate with the backend regarding supported features. */ netdev->features |= netdev->hw_features; + xdp_set_feature_flag(&netdev->features); netdev->ethtool_ops = &xennet_ethtool_ops; netdev->min_mtu = ETH_MIN_MTU; From patchwork Mon Nov 16 09:34:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marek Majtyka X-Patchwork-Id: 325154 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7749C4742C for ; Mon, 16 Nov 2020 10:11:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 752E420709 for ; Mon, 16 Nov 2020 10:11:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TTaM8tw7" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728588AbgKPJfV (ORCPT ); Mon, 16 Nov 2020 04:35:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55294 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726621AbgKPJfU (ORCPT ); Mon, 16 Nov 2020 04:35:20 -0500 Received: from mail-lj1-x244.google.com (mail-lj1-x244.google.com [IPv6:2a00:1450:4864:20::244]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8261C0613CF; Mon, 16 Nov 2020 01:35:19 -0800 (PST) Received: by mail-lj1-x244.google.com with SMTP id 142so5715836ljj.10; Mon, 16 Nov 2020 01:35:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2VYsHDkEtXhvGJF/LEvo7CUCE7h8wXtuPkmjNp3cbdQ=; b=TTaM8tw7A8IMkmIx3/v5UFetBijSnludtH0u+gJ//DkgAAralma1mlrsk+20Die7A9 4bt5h9k4UStoLngAZZ/f97SCGRnYbAwZiJhxTXR/mIDnp+Ts6tScTzcTmwW0Z9jxgSF2 9aCNnQfSGtUdMixpD+CcsUv0CzqFVirTHvqE4LTgRJfl3beTeR0J2urW4oNH2TU/s0wI r1QkR8cR7Lj+VDTHyDKk+RzaNxr1BkobuI7Q5B20+0N+0OQe/ILeAwfEiLtuv7H8vBJL WBkPnEEai7Vq6q29iJWzGzVgmg4S5apNvq6kTL/7YenLGosp0noiQ7e0i/f+TQ8YpcgD PnmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2VYsHDkEtXhvGJF/LEvo7CUCE7h8wXtuPkmjNp3cbdQ=; b=TmcysCl7eFbj1KfElBbtdHaREhhGK1uu86KB/8St4V1sWGc4G2DWUiScjew+bhwXFK bXtUozptuv9GfAZrkNmg4jfsq0zUKYCPNUuI+q8KXZy/3mJslmn0N1M0ly0P+lkIf56u 34aPLrkHYKmHvSTFsp9tUH+5WfhDknzCd/GYs/zylla1eV43uF5qHXk5z4UkYuTxYLMu jgzwG73ofEzvMzS8PqHBgSUyl64/3te2OAbTDk9EkCg8wFrGZViZ3yexCV8x19Anvmbd i39g8SzmAVx33h520wfoE1JBuNmvIeDo+XPVhsonaHqKvsAtdyysWZvvBeJekwkGmfcP N6NQ== X-Gm-Message-State: AOAM530jK34C0OZfGxNidhmrNaHwr8/2CHDJV8jpMwOxRCKZr3QlflS4 Aiqx+oX4+x6CYT7LuGCOYoU= X-Google-Smtp-Source: ABdhPJzsKM/DellFePfB2VFYSH7lxGLEfJsoHHTb68fxyrXgLEt6otDaDkoPAdKi4cY1awp6Ea65QA== X-Received: by 2002:a2e:b4c2:: with SMTP id r2mr6096274ljm.216.1605519318181; Mon, 16 Nov 2020 01:35:18 -0800 (PST) Received: from localhost.localdomain (87-205-71-93.adsl.inetia.pl. [87.205.71.93]) by smtp.gmail.com with ESMTPSA id t26sm2667986lfp.296.2020.11.16.01.35.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 01:35:17 -0800 (PST) From: alardam@gmail.com X-Google-Original-From: marekx.majtyka@intel.com To: magnus.karlsson@intel.com, bjorn.topel@intel.com, andrii.nakryiko@gmail.com, kuba@kernel.org, ast@kernel.org, daniel@iogearbox.net, netdev@vger.kernel.org, davem@davemloft.net, john.fastabend@gmail.com, hawk@kernel.org, toke@redhat.com Cc: maciej.fijalkowski@intel.com, jonathan.lemon@gmail.com, bpf@vger.kernel.org, jeffrey.t.kirsher@intel.com, maciejromanfijalkowski@gmail.com, intel-wired-lan@lists.osuosl.org, Marek Majtyka Subject: [PATCH 4/8] xsk: add check for full support of XDP in bind Date: Mon, 16 Nov 2020 10:34:48 +0100 Message-Id: <20201116093452.7541-5-marekx.majtyka@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201116093452.7541-1-marekx.majtyka@intel.com> References: <20201116093452.7541-1-marekx.majtyka@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Marek Majtyka Add check for full support of XDP in AF_XDP socket bind. To be able to use an AF_XDP socket with zero-copy, there needs to be support for both XDP_REDIRECT in the driver (XDP native mode) and the driver needs to support zero-copy. The problem is that there are drivers out there that only support XDP partially, so it is possible to successfully load the XDP program in native mode, but it will still not be able to support zero-copy as it does not have XDP_REDIRECT support. We can now alleviate this problem by using the new XDP netdev capability that signifies if full XDP support is indeed present. This check can be triggered by a new bind flag called XDP_CHECK_NATIVE_MODE. To simplify usage, this check is triggered automatically from inside libbpf library via turning on the new XDP_CHECK_NATIVE_MODE flag if and only if the driver mode is selected for the socket. As a result, the xsk_bind function decides if the native mode for a given interface makes sense or not using xdp netdev feature flags. Eventually the xsk socket is bound or an error is returned. Apart from this change and to catch all invalid inputs in a single place, an additional check is set to forbid sbk mode and zero copy settings at the same time as that combination makes no sense. Signed-off-by: Marek Majtyka --- include/uapi/linux/if_xdp.h | 1 + net/xdp/xsk.c | 4 ++-- net/xdp/xsk_buff_pool.c | 17 ++++++++++++++++- tools/include/uapi/linux/if_xdp.h | 1 + tools/lib/bpf/xsk.c | 3 +++ 5 files changed, 23 insertions(+), 3 deletions(-) diff --git a/include/uapi/linux/if_xdp.h b/include/uapi/linux/if_xdp.h index a78a8096f4ce..8f47754dacce 100644 --- a/include/uapi/linux/if_xdp.h +++ b/include/uapi/linux/if_xdp.h @@ -25,6 +25,7 @@ * application. */ #define XDP_USE_NEED_WAKEUP (1 << 3) +#define XDP_CHECK_NATIVE_MODE (1 << 4) /* Flags for xsk_umem_config flags */ #define XDP_UMEM_UNALIGNED_CHUNK_FLAG (1 << 0) diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index cfbec3989a76..a9c386083377 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -658,7 +658,7 @@ static int xsk_bind(struct socket *sock, struct sockaddr *addr, int addr_len) flags = sxdp->sxdp_flags; if (flags & ~(XDP_SHARED_UMEM | XDP_COPY | XDP_ZEROCOPY | - XDP_USE_NEED_WAKEUP)) + XDP_USE_NEED_WAKEUP | XDP_CHECK_NATIVE_MODE)) return -EINVAL; rtnl_lock(); @@ -686,7 +686,7 @@ static int xsk_bind(struct socket *sock, struct sockaddr *addr, int addr_len) struct socket *sock; if ((flags & XDP_COPY) || (flags & XDP_ZEROCOPY) || - (flags & XDP_USE_NEED_WAKEUP)) { + (flags & XDP_USE_NEED_WAKEUP) || (flags & XDP_CHECK_NATIVE_MODE)) { /* Cannot specify flags for shared sockets. */ err = -EINVAL; goto out_unlock; diff --git a/net/xdp/xsk_buff_pool.c b/net/xdp/xsk_buff_pool.c index 76922696ad3c..231d88ddd978 100644 --- a/net/xdp/xsk_buff_pool.c +++ b/net/xdp/xsk_buff_pool.c @@ -123,7 +123,7 @@ static void xp_disable_drv_zc(struct xsk_buff_pool *pool) static int __xp_assign_dev(struct xsk_buff_pool *pool, struct net_device *netdev, u16 queue_id, u16 flags) { - bool force_zc, force_copy; + bool force_zc, force_copy, force_check; struct netdev_bpf bpf; int err = 0; @@ -131,10 +131,24 @@ static int __xp_assign_dev(struct xsk_buff_pool *pool, force_zc = flags & XDP_ZEROCOPY; force_copy = flags & XDP_COPY; + force_check = flags & XDP_CHECK_NATIVE_MODE; + if (force_zc && force_copy) return -EINVAL; + if (!(flags & XDP_SHARED_UMEM)) { + if (force_check) { + /* forbid driver mode without full XDP support */ + if (!(NETIF_F_XDP & netdev->features)) + return -EOPNOTSUPP; + } else { + /* forbid skb mode and zero copy */ + if (force_zc) + return -EINVAL; + } + } + if (xsk_get_pool_from_qid(netdev, queue_id)) return -EBUSY; @@ -206,6 +220,7 @@ int xp_assign_dev_shared(struct xsk_buff_pool *pool, struct xdp_umem *umem, return -EINVAL; flags = umem->zc ? XDP_ZEROCOPY : XDP_COPY; + flags |= XDP_SHARED_UMEM; if (pool->uses_need_wakeup) flags |= XDP_USE_NEED_WAKEUP; diff --git a/tools/include/uapi/linux/if_xdp.h b/tools/include/uapi/linux/if_xdp.h index a78a8096f4ce..8f47754dacce 100644 --- a/tools/include/uapi/linux/if_xdp.h +++ b/tools/include/uapi/linux/if_xdp.h @@ -25,6 +25,7 @@ * application. */ #define XDP_USE_NEED_WAKEUP (1 << 3) +#define XDP_CHECK_NATIVE_MODE (1 << 4) /* Flags for xsk_umem_config flags */ #define XDP_UMEM_UNALIGNED_CHUNK_FLAG (1 << 0) diff --git a/tools/lib/bpf/xsk.c b/tools/lib/bpf/xsk.c index 9bc537d0b92d..7951f7ea6db3 100644 --- a/tools/lib/bpf/xsk.c +++ b/tools/lib/bpf/xsk.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -827,6 +828,8 @@ int xsk_socket__create_shared(struct xsk_socket **xsk_ptr, sxdp.sxdp_shared_umem_fd = umem->fd; } else { sxdp.sxdp_flags = xsk->config.bind_flags; + if (xsk->config.xdp_flags & XDP_FLAGS_DRV_MODE) + sxdp.sxdp_flags |= XDP_CHECK_NATIVE_MODE; } err = bind(xsk->fd, (struct sockaddr *)&sxdp, sizeof(sxdp)); From patchwork Mon Nov 16 09:34:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marek Majtyka X-Patchwork-Id: 325153 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EC0FC4742C for ; Mon, 16 Nov 2020 10:12:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9DF0520709 for ; Mon, 16 Nov 2020 10:12:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YlR2C9TZ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728599AbgKPJfZ (ORCPT ); Mon, 16 Nov 2020 04:35:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55310 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728593AbgKPJfY (ORCPT ); Mon, 16 Nov 2020 04:35:24 -0500 Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com [IPv6:2a00:1450:4864:20::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB8F4C0613D1; Mon, 16 Nov 2020 01:35:23 -0800 (PST) Received: by mail-lf1-x133.google.com with SMTP id 74so24196431lfo.5; Mon, 16 Nov 2020 01:35:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FxyaWGigiA6CBHv9VZdDh747TfZt+Mrn5SZjiJ5kOxE=; b=YlR2C9TZc9+c0VsAQukQIUtdQa2AHSCzdssCyjRnZa0cqzoYGbxtieUshDtOU80SxZ VQZjjVCXbjejyiNiwKs7FIz3SofUICk1uA3ujS+SKJzFpiu6zYNwulyb3SlZ4oIbti2H n4dGRv02HywLJMC/HxKszCyB1RGP1hm4aF66jRM/cBJtf/mvfXoatGoNiwuoy7RjVVc3 3RG9Dj17bX6OR0chsSnzswbDAg+YDz0jNzJ/pDeLzDcpxe7otzpmQk4rR5hizcaZapRT dEpngkf1mzrhYuXM+XEviPO2Agd9QGwaIAzXA1TfUn9IV2ztVrse5d6Wr681LqeERrbl DLCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FxyaWGigiA6CBHv9VZdDh747TfZt+Mrn5SZjiJ5kOxE=; b=fgnYY16S4syRjCz+ktiWroa5N0FyRZLmwpiUhQmrkSBQJkcu/O2cpRfyX4+EWEa0YZ K+7nIVSBEj39ahAf4rnTn8MD/+wcn5xMjlZUm6vUB9aldwcTVJ6gBUztTyUFgWE3IYvY go6bmeBIf5tTM7TtRSaGy2Sg9wXev2AfsMjarNc2PwqDj+aK/uxuOL8/BtVrfCuYASvv p0uF6SjMwD3N5EwEraQqYUTHrNJ+MR2uCLoZk4Evlw1UjSZIY9m1Kc46j8KAFL/1rec6 yzsUMXfbF0PsHNRjoUdRwFKVMhYJP1gJGy2DBLMyXDzJ6u0EwnWX++TQ1J2/2trzhiFt AkiA== X-Gm-Message-State: AOAM532Obo1tXdpX38wMcYE6+PAICM491d0GL6e4alvY1eA5TC9Kozq9 M3zBQD6Nl2eBuiDDs5UjRhE= X-Google-Smtp-Source: ABdhPJwWiQhkPSOX7+DP+n1wbQypCgrpozRFyxo785vqUBp9Xwas5JPp0symuh9I+57dA4ysUgHvSg== X-Received: by 2002:a19:a05:: with SMTP id 5mr4940806lfk.291.1605519322241; Mon, 16 Nov 2020 01:35:22 -0800 (PST) Received: from localhost.localdomain (87-205-71-93.adsl.inetia.pl. [87.205.71.93]) by smtp.gmail.com with ESMTPSA id t26sm2667986lfp.296.2020.11.16.01.35.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 01:35:21 -0800 (PST) From: alardam@gmail.com X-Google-Original-From: marekx.majtyka@intel.com To: magnus.karlsson@intel.com, bjorn.topel@intel.com, andrii.nakryiko@gmail.com, kuba@kernel.org, ast@kernel.org, daniel@iogearbox.net, netdev@vger.kernel.org, davem@davemloft.net, john.fastabend@gmail.com, hawk@kernel.org, toke@redhat.com Cc: maciej.fijalkowski@intel.com, jonathan.lemon@gmail.com, bpf@vger.kernel.org, jeffrey.t.kirsher@intel.com, maciejromanfijalkowski@gmail.com, intel-wired-lan@lists.osuosl.org, Marek Majtyka Subject: [PATCH 6/8] libbpf: add functions to get XSK modes Date: Mon, 16 Nov 2020 10:34:50 +0100 Message-Id: <20201116093452.7541-7-marekx.majtyka@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201116093452.7541-1-marekx.majtyka@intel.com> References: <20201116093452.7541-1-marekx.majtyka@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Marek Majtyka Add functions to get XDP/XSK modes from netdev feature flags over netlink ethtool family interface. These functions provide functionalities that are going to be used in upcoming changes together constituting new libbpf public API function which informs about key xsk capabilities of given network interface. Signed-off-by: Marek Majtyka Reported-by: kernel test robot --- tools/include/uapi/linux/ethtool.h | 44 ++++ tools/lib/bpf/ethtool.h | 49 ++++ tools/lib/bpf/libbpf.h | 1 + tools/lib/bpf/netlink.c | 379 ++++++++++++++++++++++++++++- 4 files changed, 469 insertions(+), 4 deletions(-) create mode 100644 tools/lib/bpf/ethtool.h diff --git a/tools/include/uapi/linux/ethtool.h b/tools/include/uapi/linux/ethtool.h index c86c3e942df9..cf3041d302e4 100644 --- a/tools/include/uapi/linux/ethtool.h +++ b/tools/include/uapi/linux/ethtool.h @@ -48,4 +48,48 @@ struct ethtool_channels { __u32 combined_count; }; +#define ETH_GSTRING_LEN 32 + +/** + * enum ethtool_stringset - string set ID + * @ETH_SS_TEST: Self-test result names, for use with %ETHTOOL_TEST + * @ETH_SS_STATS: Statistic names, for use with %ETHTOOL_GSTATS + * @ETH_SS_PRIV_FLAGS: Driver private flag names, for use with + * %ETHTOOL_GPFLAGS and %ETHTOOL_SPFLAGS + * @ETH_SS_NTUPLE_FILTERS: Previously used with %ETHTOOL_GRXNTUPLE; + * now deprecated + * @ETH_SS_FEATURES: Device feature names + * @ETH_SS_RSS_HASH_FUNCS: RSS hush function names + * @ETH_SS_PHY_STATS: Statistic names, for use with %ETHTOOL_GPHYSTATS + * @ETH_SS_PHY_TUNABLES: PHY tunable names + * @ETH_SS_LINK_MODES: link mode names + * @ETH_SS_MSG_CLASSES: debug message class names + * @ETH_SS_WOL_MODES: wake-on-lan modes + * @ETH_SS_SOF_TIMESTAMPING: SOF_TIMESTAMPING_* flags + * @ETH_SS_TS_TX_TYPES: timestamping Tx types + * @ETH_SS_TS_RX_FILTERS: timestamping Rx filters + * @ETH_SS_UDP_TUNNEL_TYPES: UDP tunnel types + */ +enum ethtool_stringset { + ETH_SS_TEST = 0, + ETH_SS_STATS, + ETH_SS_PRIV_FLAGS, + ETH_SS_NTUPLE_FILTERS, + ETH_SS_FEATURES, + ETH_SS_RSS_HASH_FUNCS, + ETH_SS_TUNABLES, + ETH_SS_PHY_STATS, + ETH_SS_PHY_TUNABLES, + ETH_SS_LINK_MODES, + ETH_SS_MSG_CLASSES, + ETH_SS_WOL_MODES, + ETH_SS_SOF_TIMESTAMPING, + ETH_SS_TS_TX_TYPES, + ETH_SS_TS_RX_FILTERS, + ETH_SS_UDP_TUNNEL_TYPES, + + /* add new constants above here */ + ETH_SS_COUNT +}; + #endif /* _UAPI_LINUX_ETHTOOL_H */ diff --git a/tools/lib/bpf/ethtool.h b/tools/lib/bpf/ethtool.h new file mode 100644 index 000000000000..14b2ae47bc26 --- /dev/null +++ b/tools/lib/bpf/ethtool.h @@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ + +/* + * Generic netlink ethtool family required defines + * + * Copyright (c) 2020 Intel + */ + +#ifndef __LIBBPF_ETHTOOL_H_ +#define __LIBBPF_ETHTOOL_H_ + +#include + +#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d)) +#define FEATURE_BITS_TO_BLOCKS(n_bits) DIV_ROUND_UP(n_bits, 32U) + +#define FEATURE_WORD(blocks, index) ((blocks)[(index) / 32U]) +#define FEATURE_FIELD_FLAG(index) (1U << (index) % 32U) +#define FEATURE_BIT_IS_SET(blocks, index) \ + (FEATURE_WORD(blocks, index) & FEATURE_FIELD_FLAG(index)) + +#define NETDEV_XDP_STR "xdp" +#define NETDEV_XDP_LEN 4 + +#define NETDEV_AF_XDP_ZC_STR "af-xdp-zc" +#define NETDEV_AF_XDP_ZC_LEN 10 + +#define BUF_SIZE_4096 4096 +#define BUF_SIZE_8192 8192 + +#define MAX_FEATURES 500 + +struct ethnl_params { + const char *ifname; + const char *nl_family; + int features; + int xdp_idx; + int xdp_zc_idx; + int xdp_flags; + int xdp_zc_flags; + __u16 fam_id; +}; + +int libbpf_ethnl_get_ethtool_family_id(struct ethnl_params *param); +int libbpf_ethnl_get_netdev_features(struct ethnl_params *param); +int libbpf_ethnl_get_active_bits(struct ethnl_params *param); + +#endif /* __LIBBPF_ETHTOOL_H_ */ + diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index 6909ee81113a..4f0656716eee 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -41,6 +41,7 @@ enum libbpf_errno { LIBBPF_ERRNO__WRNGPID, /* Wrong pid in netlink message */ LIBBPF_ERRNO__INVSEQ, /* Invalid netlink sequence */ LIBBPF_ERRNO__NLPARSE, /* netlink parsing error */ + LIBBPF_ERRNO__INVXDP, /* Invalid XDP data */ __LIBBPF_ERRNO__END, }; diff --git a/tools/lib/bpf/netlink.c b/tools/lib/bpf/netlink.c index 4dd73de00b6f..a5344401b842 100644 --- a/tools/lib/bpf/netlink.c +++ b/tools/lib/bpf/netlink.c @@ -6,6 +6,8 @@ #include #include #include +#include +#include #include #include #include @@ -14,6 +16,7 @@ #include "libbpf.h" #include "libbpf_internal.h" #include "nlattr.h" +#include "ethtool.h" #ifndef SOL_NETLINK #define SOL_NETLINK 270 @@ -23,6 +26,11 @@ typedef int (*libbpf_dump_nlmsg_t)(void *cookie, void *msg, struct nlattr **tb); typedef int (*__dump_nlmsg_t)(struct nlmsghdr *nlmsg, libbpf_dump_nlmsg_t, void *cookie); +struct ethnl_msg { + struct nlmsghdr nlh; + struct genlmsghdr genlhdr; + char msg[BUF_SIZE_4096]; +}; struct xdp_id_md { int ifindex; @@ -30,7 +38,7 @@ struct xdp_id_md { struct xdp_link_info info; }; -static int libbpf_netlink_open(__u32 *nl_pid) +static int libbpf_netlink_open(__u32 *nl_pid, int protocol) { struct sockaddr_nl sa; socklen_t addrlen; @@ -40,7 +48,7 @@ static int libbpf_netlink_open(__u32 *nl_pid) memset(&sa, 0, sizeof(sa)); sa.nl_family = AF_NETLINK; - sock = socket(AF_NETLINK, SOCK_RAW, NETLINK_ROUTE); + sock = socket(AF_NETLINK, SOCK_RAW, protocol); if (sock < 0) return -errno; @@ -143,7 +151,7 @@ static int __bpf_set_link_xdp_fd_replace(int ifindex, int fd, int old_fd, } req; __u32 nl_pid = 0; - sock = libbpf_netlink_open(&nl_pid); + sock = libbpf_netlink_open(&nl_pid, NETLINK_ROUTE); if (sock < 0) return sock; @@ -302,7 +310,7 @@ int bpf_get_link_xdp_info(int ifindex, struct xdp_link_info *info, if (flags && flags & mask) return -EINVAL; - sock = libbpf_netlink_open(&nl_pid); + sock = libbpf_netlink_open(&nl_pid, NETLINK_ROUTE); if (sock < 0) return sock; @@ -370,3 +378,366 @@ int libbpf_nl_get_link(int sock, unsigned int nl_pid, return bpf_netlink_recv(sock, nl_pid, seq, __dump_link_nlmsg, dump_link_nlmsg, cookie); } + +static int libbpf_ethtool_parse_feature_strings(struct nlattr *start, int elem, + int *xdp, int *xdp_zc) +{ + struct nlattr *tbs[__ETHTOOL_A_STRING_CNT + 1]; + struct nlattr *tab[elem > 0 ? elem : 0]; + struct libbpf_nla_policy policy[] = { + [ETHTOOL_A_STRING_UNSPEC] = { + .type = LIBBPF_NLA_UNSPEC, + .minlen = 0, + .maxlen = 0, + }, + [ETHTOOL_A_STRING_INDEX] = { + .type = LIBBPF_NLA_U32, + .minlen = sizeof(uint32_t), + .maxlen = sizeof(uint32_t), + }, + [ETHTOOL_A_STRING_VALUE] = { + .type = LIBBPF_NLA_STRING, + .minlen = 1, + .maxlen = ETH_GSTRING_LEN, + } + }; + const char *f; + int n = 0; + __u32 v; + int ret; + int i; + + if (!xdp || !xdp_zc || !start || elem <= 0) + return -EINVAL; + + *xdp = -1; + *xdp_zc = -1; + + ret = libbpf_nla_parse_table(tab, elem, start, 0, NULL); + if (ret) + goto cleanup; + + for (i = 0; tab[i] && i < elem; ++i) { + ret = libbpf_nla_parse_nested(tbs, __ETHTOOL_A_STRING_CNT, tab[i], policy); + if (ret) + break; + + if (tbs[ETHTOOL_A_STRING_INDEX] && tbs[ETHTOOL_A_STRING_VALUE]) { + f = libbpf_nla_getattr_str(tbs[ETHTOOL_A_STRING_VALUE]); + v = libbpf_nla_getattr_u32(tbs[ETHTOOL_A_STRING_INDEX]); + + if (!strncmp(NETDEV_XDP_STR, f, NETDEV_XDP_LEN)) { + *xdp = v; + n++; + } + + if (!strncmp(NETDEV_AF_XDP_ZC_STR, f, NETDEV_AF_XDP_ZC_LEN)) { + *xdp_zc = v; + n++; + } + } else { + ret = -LIBBPF_ERRNO__NLPARSE; + break; + } + } + +cleanup: + /* If error occurred return it. */ + if (ret) + return ret; + + /* + * If zero or two xdp flags found that is okay. + * Zero means older kernel without any xdp flags added. + * Two means newer kernel with xdp flags added. + * Both flags were added in single commit, so that + * n == 1 is a faulty value. + */ + if (n == 2 || n == 0) + return 0; + + /* If no error and one or more than 2 xdp flags found return error */ + return -LIBBPF_ERRNO__INVXDP; +} + +static int libbpf_ethnl_send(int sock, __u32 seq, __u32 nl_pid, struct ethnl_msg *req) +{ + ssize_t written; + + req->nlh.nlmsg_pid = nl_pid; + req->nlh.nlmsg_seq = seq; + + written = send(sock, req, req->nlh.nlmsg_len, 0); + if (written < 0) + return -errno; + + if (written == req->nlh.nlmsg_len) + return 0; + else + return -errno; +} + +static int libbpf_ethnl_validate(int len, __u16 fam_id, __u32 nl_pid, __u32 seq, + struct ethnl_msg *req) +{ + if (!NLMSG_OK(&req->nlh, (unsigned int)len)) + return -ENOMSG; + + if (req->nlh.nlmsg_pid != nl_pid) + return -LIBBPF_ERRNO__WRNGPID; + + if (req->nlh.nlmsg_seq != seq) + return -LIBBPF_ERRNO__INVSEQ; + + if (req->nlh.nlmsg_type != fam_id) { + int ret = -ENOMSG; + + if (req->nlh.nlmsg_type == NLMSG_ERROR) { + struct nlmsgerr *err = (struct nlmsgerr *)&req->genlhdr; + + if (err->error) + ret = err->error; + libbpf_nla_dump_errormsg(&req->nlh); + } + return ret; + } + + return 0; +} + +static int libbpf_ethnl_send_recv(struct ethnl_msg *req, struct ethnl_params *param) +{ + __u32 nl_pid; + __u32 seq; + int sock; + int ret; + int len; + + sock = libbpf_netlink_open(&nl_pid, NETLINK_GENERIC); + if (sock < 0) { + ret = sock; + goto cleanup; + } + + seq = time(NULL); + ret = libbpf_ethnl_send(sock, seq, nl_pid, req); + if (ret) + goto cleanup; + + len = recv(sock, req, sizeof(struct ethnl_msg), 0); + if (len < 0) { + ret = -errno; + goto cleanup; + } + + ret = libbpf_ethnl_validate(len, param->fam_id, nl_pid, seq, req); + if (ret < 0) + goto cleanup; + + ret = len; + +cleanup: + if (sock >= 0) + close(sock); + + return ret; +} + +int libbpf_ethnl_get_netdev_features(struct ethnl_params *param) +{ + struct ethnl_msg req = { + .nlh.nlmsg_len = NLMSG_LENGTH(sizeof(struct genlmsghdr)), + .nlh.nlmsg_flags = NLM_F_REQUEST, + .nlh.nlmsg_type = param->fam_id, + .nlh.nlmsg_pid = 0, + .genlhdr.version = ETHTOOL_GENL_VERSION, + .genlhdr.cmd = ETHTOOL_MSG_STRSET_GET, + .genlhdr.reserved = 0, + }; + struct nlattr *tbn[__ETHTOOL_A_STRINGSETS_CNT + 1]; + struct nlattr *tbnn[__ETHTOOL_A_STRINGSET_CNT + 1]; + struct nlattr *tb[__ETHTOOL_A_STRSET_CNT + 1]; + struct nlattr *nla, *nla_next, *nla_set; + int string_set = ETH_SS_FEATURES; + int ret; + int len; + + memset(&req.msg, 0, BUF_SIZE_4096); + + nla = (struct nlattr *)req.msg; + nla_next = libbpf_nla_nest_start(nla, ETHTOOL_A_STRSET_HEADER); + nla_next = libbpf_nla_put_str(nla_next, ETHTOOL_A_HEADER_DEV_NAME, + param->ifname, IFNAMSIZ); + libbpf_nla_nest_end(nla, nla_next); + + nla = nla_next; + nla_set = libbpf_nla_nest_start(nla, ETHTOOL_A_STRSET_STRINGSETS); + nla_next = libbpf_nla_nest_start(nla_set, ETHTOOL_A_STRINGSETS_STRINGSET); + nla_next = libbpf_nla_put_u32(nla_next, ETHTOOL_A_STRINGSET_ID, string_set); + libbpf_nla_nest_end(nla_set, nla_next); + libbpf_nla_nest_end(nla, nla_next); + if (!param->features) + nla_next = libbpf_nla_put_flag(nla_next, ETHTOOL_A_STRSET_COUNTS_ONLY); + + req.nlh.nlmsg_len += libbpf_nla_attrs_length((struct nlattr *)req.msg, nla_next); + + len = libbpf_ethnl_send_recv(&req, param); + if (len < 0) + return len; + + /* set parsing error, and change if succeeded */ + ret = -LIBBPF_ERRNO__NLPARSE; + nla = (struct nlattr *)req.msg; + len = len - NLMSG_HDRLEN - GENL_HDRLEN; + + if (libbpf_nla_parse(tb, __ETHTOOL_A_STRSET_CNT, nla, len, NULL)) + return ret; + + if (!tb[ETHTOOL_A_STRSET_STRINGSETS]) + return ret; + + if (libbpf_nla_parse_nested(tbn, __ETHTOOL_A_STRINGSETS_CNT, + tb[ETHTOOL_A_STRSET_STRINGSETS], NULL)) + return ret; + + if (!tbn[ETHTOOL_A_STRINGSETS_STRINGSET]) + return ret; + + if (libbpf_nla_parse_nested(tbnn, __ETHTOOL_A_STRINGSET_CNT, + tbn[ETHTOOL_A_STRINGSETS_STRINGSET], NULL)) + return ret; + + if (param->features == 0) { + if (!tbnn[ETHTOOL_A_STRINGSET_COUNT]) + return ret; + + param->features = libbpf_nla_getattr_u32(tbnn[ETHTOOL_A_STRINGSET_COUNT]); + + /* success */ + ret = 0; + } else if (param->features > 0) { + if (!tbnn[ETHTOOL_A_STRINGSET_STRINGS]) + return ret; + + /* + * Upper boundary is known, but it is input from socket stream. + * Let's perform upper limit check anyway, and limit it up to + * MAX_FEATURES (which is still far more than is actually needed). + */ + if (param->features > MAX_FEATURES) + param->features = MAX_FEATURES; + + /* success if returns 0 */ + ret = libbpf_ethtool_parse_feature_strings(tbnn[ETHTOOL_A_STRINGSET_STRINGS], + param->features, ¶m->xdp_idx, + ¶m->xdp_zc_idx); + } + + return ret; +} + +int libbpf_ethnl_get_ethtool_family_id(struct ethnl_params *param) +{ + struct ethnl_msg req = { + .nlh.nlmsg_len = NLMSG_LENGTH(sizeof(struct genlmsghdr)), + .nlh.nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK, + .nlh.nlmsg_type = GENL_ID_CTRL, + .nlh.nlmsg_pid = 0, + .genlhdr.version = ETHTOOL_GENL_VERSION, + .genlhdr.cmd = CTRL_CMD_GETFAMILY, + .genlhdr.reserved = 0, + }; + struct nlattr *tb[__CTRL_ATTR_MAX + 1] = {0}; + struct nlattr *nla, *nla_next; + int ret = -1; + int len; + + memset(&req.msg, 0, BUF_SIZE_4096); + param->fam_id = GENL_ID_CTRL; + + nla = (struct nlattr *)req.msg; + nla_next = libbpf_nla_put_str(nla, CTRL_ATTR_FAMILY_NAME, param->nl_family, GENL_NAMSIZ); + req.nlh.nlmsg_len += libbpf_nla_attrs_length(nla, nla_next); + + len = libbpf_ethnl_send_recv(&req, param); + if (len < 0) + return len; + + /* set parsing error, and change if succeeded */ + ret = -LIBBPF_ERRNO__NLPARSE; + len = len - NLMSG_HDRLEN - GENL_HDRLEN; + if (!libbpf_nla_parse(tb, __CTRL_ATTR_MAX, nla, len, NULL)) { + if (tb[CTRL_ATTR_FAMILY_ID]) { + param->fam_id = libbpf_nla_getattr_u16(tb[CTRL_ATTR_FAMILY_ID]); + ret = 0; + } + } + + return ret; +} + +int libbpf_ethnl_get_active_bits(struct ethnl_params *param) +{ + struct ethnl_msg req = { + .nlh.nlmsg_len = NLMSG_LENGTH(sizeof(struct genlmsghdr)), + .nlh.nlmsg_flags = NLM_F_REQUEST, + .nlh.nlmsg_type = param->fam_id, + .nlh.nlmsg_pid = 0, + .genlhdr.cmd = ETHTOOL_MSG_FEATURES_GET, + .genlhdr.version = ETHTOOL_GENL_VERSION, + .genlhdr.reserved = 0, + }; + __u32 active[FEATURE_BITS_TO_BLOCKS(param->features)]; + struct nlattr *tb[__ETHTOOL_A_FEATURES_CNT + 1]; + struct nlattr *tbn[__ETHTOOL_A_BITSET_CNT + 1]; + int flags = ETHTOOL_FLAG_COMPACT_BITSETS; + struct nlattr *nla, *nla_next; + int ret = -1; + int len; + + memset(&req.msg, 0, BUF_SIZE_4096); + + nla = (struct nlattr *)req.msg; + nla_next = libbpf_nla_nest_start(nla, ETHTOOL_A_FEATURES_HEADER); + nla_next = libbpf_nla_put_str(nla_next, ETHTOOL_A_HEADER_DEV_NAME, param->ifname, IFNAMSIZ); + nla_next = libbpf_nla_put_u32(nla_next, ETHTOOL_A_HEADER_FLAGS, flags); + libbpf_nla_nest_end(nla, nla_next); + req.nlh.nlmsg_len += libbpf_nla_attrs_length(nla, nla_next); + + len = libbpf_ethnl_send_recv(&req, param); + if (len < 0) + return len; + + ret = -LIBBPF_ERRNO__NLPARSE; + nla = (struct nlattr *)req.msg; + len = len - NLMSG_HDRLEN - GENL_HDRLEN; + if (libbpf_nla_parse(tb, __ETHTOOL_A_FEATURES_CNT, nla, len, NULL)) + return ret; + + if (!tb[ETHTOOL_A_FEATURES_ACTIVE]) + return ret; + + if (libbpf_nla_parse_nested(tbn, __ETHTOOL_A_BITSET_CNT, + tb[ETHTOOL_A_FEATURES_ACTIVE], NULL)) + return ret; + + if (!tbn[ETHTOOL_A_BITSET_VALUE]) + return ret; + + for (unsigned int i = 0; i < FEATURE_BITS_TO_BLOCKS(param->features); ++i) + active[i] = libbpf_nla_getattr_u32(tbn[ETHTOOL_A_BITSET_VALUE] + i); + + /* mark successful parsing */ + ret = 0; + if (FEATURE_BIT_IS_SET(active, param->xdp_idx)) { + param->xdp_flags = 1; + if (FEATURE_BIT_IS_SET(active, param->xdp_zc_idx)) + param->xdp_zc_flags = 1; + } else { + /* zero copy without driver mode makes no sense */ + if (FEATURE_BIT_IS_SET(active, param->xdp_zc_idx)) + ret = -LIBBPF_ERRNO__INVXDP; + } + + return ret; +} From patchwork Mon Nov 16 09:34:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marek Majtyka X-Patchwork-Id: 325152 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNWANTED_LANGUAGE_BODY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A538C4742C for ; Mon, 16 Nov 2020 10:14:00 +0000 (UTC) Received: from vger.kernel.org (unknown [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B13F12074B for ; Mon, 16 Nov 2020 10:13:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BF6kPRoi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B13F12074B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=netdev-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728612AbgKPJf3 (ORCPT ); Mon, 16 Nov 2020 04:35:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55324 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728600AbgKPJf3 (ORCPT ); Mon, 16 Nov 2020 04:35:29 -0500 Received: from mail-lj1-x241.google.com (mail-lj1-x241.google.com [IPv6:2a00:1450:4864:20::241]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55420C0613CF; Mon, 16 Nov 2020 01:35:27 -0800 (PST) Received: by mail-lj1-x241.google.com with SMTP id p12so19343525ljc.9; Mon, 16 Nov 2020 01:35:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xAOG2YTYieBXm2sG2XBQpYDkkTHEFtWVxAivRZSh5+4=; b=BF6kPRoiWEsCarjzX6qu05+3yRz/euXkOGWNZ+GCEtih36CLPMJgLcX+VaZqz6GyU1 MXYpmI5pBI66cny7Q7KyiuGcOtig7QLydG58MC91I7PN5W1dNeQPYLf5hb1gytU/DXIz dAhukOv8S0Tj52wuzeCLpyGFUyEQ8CTWpBCTHcSR5hrzi3hWV30qzhrgJRalfRtmARS7 /Yj5WW1CXWmVrkDdyxOr4ogYUPAfEvFVfMf8Y0O2a0T6k+wfxJN+RlG99ClD7dxuN5dK nn3eYqaBDc4jw6ohqStJSTRizmkJKSqgzXcixxXxnKNN1/ycZWdIMoVeJecP6Ihuo0aC 8Sfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xAOG2YTYieBXm2sG2XBQpYDkkTHEFtWVxAivRZSh5+4=; b=jnZLHsBnwhoutOPtTvSk9ko8wHgmGoI9mSn/l3/f3+gGGJdpXlhtMyUejToPEsSmcg vYtH/ltASTxHOdUVnkFc+AbBxevcTBNytgzjfj5AZ/Y62hEK9NbfK17eEiAencl2YId3 70lIUzjMzM8bENxFmUv5MO4mN4AXUs9+ON2GmcdV1SYMR7EeRIWarfFi5qqsJcwjr3sw pejbqVzCCE70F1jOhp3Q/glrf/1YjZYnQzc+nkMmWf7xW5Z2rSRRkvShYvr6w7QWB1S3 K55UElQtVjEv2paFxbjZYmpNgAJlBUhcrYPHvVOKoHXMgar1hJhwm4kfm/a13+5hRnot Nxww== X-Gm-Message-State: AOAM533AHYTl/vrhBN5aGGDvsmH+QZShwgfiEIlQcxKNh9AaHomkjUWU J28cEQG3EHo2XVGqzf1n11A= X-Google-Smtp-Source: ABdhPJyUWgn/WvrBtKIDpXtZkttYTdLPotBqxAe3e8OLTuto3qC7FmjD65bFqDqMvN1ZDEcZvHKrFQ== X-Received: by 2002:a05:651c:1105:: with SMTP id d5mr6437413ljo.265.1605519325838; Mon, 16 Nov 2020 01:35:25 -0800 (PST) Received: from localhost.localdomain (87-205-71-93.adsl.inetia.pl. [87.205.71.93]) by smtp.gmail.com with ESMTPSA id t26sm2667986lfp.296.2020.11.16.01.35.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 01:35:25 -0800 (PST) From: alardam@gmail.com X-Google-Original-From: marekx.majtyka@intel.com To: magnus.karlsson@intel.com, bjorn.topel@intel.com, andrii.nakryiko@gmail.com, kuba@kernel.org, ast@kernel.org, daniel@iogearbox.net, netdev@vger.kernel.org, davem@davemloft.net, john.fastabend@gmail.com, hawk@kernel.org, toke@redhat.com Cc: maciej.fijalkowski@intel.com, jonathan.lemon@gmail.com, bpf@vger.kernel.org, jeffrey.t.kirsher@intel.com, maciejromanfijalkowski@gmail.com, intel-wired-lan@lists.osuosl.org, Marek Majtyka Subject: [PATCH 8/8] samples/bpf/xdp: apply netdev XDP/XSK modes info Date: Mon, 16 Nov 2020 10:34:52 +0100 Message-Id: <20201116093452.7541-9-marekx.majtyka@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201116093452.7541-1-marekx.majtyka@intel.com> References: <20201116093452.7541-1-marekx.majtyka@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Marek Majtyka Update xdpsock sample so that it utilizes netlink ethtool interface to get available XDP/XSK modes. This allows to automatically choose the best available mode of operation, if these are not provided explicitly. Signed-off-by: Marek Majtyka --- samples/bpf/xdpsock_user.c | 117 ++++++++++++++++++++++++++++++++++--- 1 file changed, 108 insertions(+), 9 deletions(-) diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c index 1149e94ca32f..780e5d1d73a0 100644 --- a/samples/bpf/xdpsock_user.c +++ b/samples/bpf/xdpsock_user.c @@ -53,6 +53,9 @@ #define DEBUG_HEXDUMP 0 +#define XDP_MODES (XDP_FLAGS_SKB_MODE | XDP_FLAGS_DRV_MODE) +#define XSK_MODES (XDP_COPY | XDP_ZEROCOPY) + typedef __u64 u64; typedef __u32 u32; typedef __u16 u16; @@ -86,7 +89,7 @@ static u32 irq_no; static int irqs_at_init = -1; static int opt_poll; static int opt_interval = 1; -static u32 opt_xdp_bind_flags = XDP_USE_NEED_WAKEUP; +static u16 opt_xdp_bind_flags = XDP_USE_NEED_WAKEUP; static u32 opt_umem_flags; static int opt_unaligned_chunks; static int opt_mmap_flags; @@ -95,6 +98,8 @@ static int opt_timeout = 1000; static bool opt_need_wakeup = true; static u32 opt_num_xsks = 1; static u32 prog_id; +static u32 xdp_caps; +static u16 bind_caps; struct xsk_ring_stats { unsigned long rx_npkts; @@ -957,6 +962,26 @@ static void usage(const char *prog) exit(EXIT_FAILURE); } +static inline void set_drv_mode(void) +{ + opt_xdp_flags |= XDP_FLAGS_DRV_MODE; +} + +static inline void set_skb_mode(void) +{ + opt_xdp_flags |= XDP_FLAGS_SKB_MODE; +} + +static inline void set_zc_mode(void) +{ + opt_xdp_bind_flags |= XDP_ZEROCOPY; +} + +static inline void set_copy_mode(void) +{ + opt_xdp_bind_flags |= XDP_COPY; +} + static void parse_command_line(int argc, char **argv) { int option_index, c; @@ -989,20 +1014,19 @@ static void parse_command_line(int argc, char **argv) opt_poll = 1; break; case 'S': - opt_xdp_flags |= XDP_FLAGS_SKB_MODE; - opt_xdp_bind_flags |= XDP_COPY; + set_skb_mode(); break; case 'N': - /* default, set below */ + set_drv_mode(); break; case 'n': opt_interval = atoi(optarg); break; case 'z': - opt_xdp_bind_flags |= XDP_ZEROCOPY; + set_zc_mode(); break; case 'c': - opt_xdp_bind_flags |= XDP_COPY; + set_copy_mode(); break; case 'u': opt_umem_flags |= XDP_UMEM_UNALIGNED_CHUNK_FLAG; @@ -1069,9 +1093,6 @@ static void parse_command_line(int argc, char **argv) } } - if (!(opt_xdp_flags & XDP_FLAGS_SKB_MODE)) - opt_xdp_flags |= XDP_FLAGS_DRV_MODE; - opt_ifindex = if_nametoindex(opt_if); if (!opt_ifindex) { fprintf(stderr, "ERROR: interface \"%s\" does not exist\n", @@ -1461,6 +1482,76 @@ static void enter_xsks_into_map(struct bpf_object *obj) } } +static inline u32 xdp_mode_not_set(void) +{ + return (opt_xdp_flags & XDP_MODES) == 0; +} + +static inline u16 bind_mode_not_set(void) +{ + return (opt_xdp_bind_flags & XSK_MODES) == 0; +} + +static inline u16 zc_mode_set(void) +{ + return opt_xdp_bind_flags & XDP_ZEROCOPY; +} + +static inline u32 drv_mode_set(void) +{ + return opt_xdp_flags & XDP_FLAGS_DRV_MODE; +} + +static inline u16 zc_mode_available(void) +{ + return bind_caps & XDP_ZEROCOPY; +} + +static inline u32 drv_mode_available(void) +{ + return xdp_caps & XDP_FLAGS_DRV_MODE; +} + +static void set_xsk_default_flags(void) +{ + if (drv_mode_available()) { + set_drv_mode(); + + if (zc_mode_available()) + set_zc_mode(); + else + set_copy_mode(); + } else { + set_skb_mode(); + set_copy_mode(); + } +} + +static void adjust_missing_flags(void) +{ + if (xdp_mode_not_set()) { + if (bind_mode_not_set()) { + set_xsk_default_flags(); + } else { + if (zc_mode_set()) { + set_drv_mode(); + } else { + if (drv_mode_available()) + set_drv_mode(); + else + set_skb_mode(); + } + } + } else { + if (bind_mode_not_set()) { + if (drv_mode_set() && zc_mode_available()) + set_zc_mode(); + else + set_copy_mode(); + } + } +} + int main(int argc, char **argv) { struct rlimit r = {RLIM_INFINITY, RLIM_INFINITY}; @@ -1473,6 +1564,14 @@ int main(int argc, char **argv) parse_command_line(argc, argv); + ret = xsk_socket__get_caps(opt_if, &xdp_caps, &bind_caps); + if (ret) { + fprintf(stderr, "ERROR: xsk_socket__get_caps\n"); + exit(EXIT_FAILURE); + } + + adjust_missing_flags(); + if (setrlimit(RLIMIT_MEMLOCK, &r)) { fprintf(stderr, "ERROR: setrlimit(RLIMIT_MEMLOCK) \"%s\"\n", strerror(errno));