From patchwork Wed Nov 4 14:08:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Magnus Karlsson X-Patchwork-Id: 315639 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29ECDC2D0A3 for ; Wed, 4 Nov 2020 14:09:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B9908223AB for ; Wed, 4 Nov 2020 14:09:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="pqfALFjy" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730222AbgKDOJq (ORCPT ); Wed, 4 Nov 2020 09:09:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41604 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730160AbgKDOJm (ORCPT ); Wed, 4 Nov 2020 09:09:42 -0500 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A91D3C0613D3; Wed, 4 Nov 2020 06:09:42 -0800 (PST) Received: by mail-pg1-x544.google.com with SMTP id i7so14774000pgh.6; Wed, 04 Nov 2020 06:09:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=8+wmdl8dxGrL4PiYjqO1IGkwfNXPjaWfQTVtL5mk3TU=; b=pqfALFjyvFpVFRkFsJUEQWBG1QOKGsIgGOga8VgN6ZqRXRoEarHrMOg3EaA/cevxMJ OHh7oDXHKItfvhHKg1l8hH4ze1C88whzbodX2iRIa36ANhUrOvAkHVPt5BWpIugKsvgx VV4yfRMElhX7WfIRm8xopCdT3bqnm84kNIf924oric/ZSsQ9VQ/iwMkjlnRFjWiZII1r Deq1joxcO65nhHdaoBOG7PErCT7dPxbW4UbOJ6cQ3vtMkywVEY4RpwtMNswx79Df8JYO /vwb+mOSe90WdBa2G3BzTlALfjrh/VOCiE0k7TUNEZT37Xb7hIwUKvxxKWAilBDqqxeG Yz8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=8+wmdl8dxGrL4PiYjqO1IGkwfNXPjaWfQTVtL5mk3TU=; b=c4I1Vo9UBvJCDzjo/qKnqABkTA/sNyA0v6yRb/MqacmW0T3pMhgyLjPNRrPRQ2yFIK 91KOYfzUfjHBZtfkhkiVL8+LMOJ054MtIXVfwLS9JLd2XiXqR9+ur3FVNwOEMXrK+75Z Hzgf8nbgEPBG+0E1sZYz5vJtsq4LHvj2T2kV9cmNiiWp+r9T3VEj1gPvBjiJmqhkktrK +SiLlx67ZVKampCLkmbvgoAEKRQ8pgkDe2WC3B5TCFgRp+84lCfS5uV8B1w/dcB7fpmh Qc1rrJvwUQHJdK1pD0BTFW0G5g1fdwPNWjI40dYBAZj844Av5nyuMKpuL/F23YpLo93C EKvA== X-Gm-Message-State: AOAM5305mjPozRKSmu/Ks3rfZSPS2GGDAd3JTy2/YG8ucw6ION9+soLD nGAcx97EMbtMkuz7Ti6F4nKfwZACes63kFboZGs= X-Google-Smtp-Source: ABdhPJzIoXz/gBu1yr5pXzZGVVq8Z08Ksz/3VbhzpwCKerO50ZqQ6h9oFGmPWePptpisDK0xiKadmg== X-Received: by 2002:a17:90a:e658:: with SMTP id ep24mr3478312pjb.171.1604498982258; Wed, 04 Nov 2020 06:09:42 -0800 (PST) Received: from VM.ger.corp.intel.com ([192.55.55.43]) by smtp.gmail.com with ESMTPSA id q123sm2724818pfq.56.2020.11.04.06.09.38 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Nov 2020 06:09:41 -0800 (PST) From: Magnus Karlsson To: magnus.karlsson@intel.com, bjorn.topel@intel.com, ast@kernel.org, daniel@iogearbox.net, netdev@vger.kernel.org, jonathan.lemon@gmail.com Cc: bpf@vger.kernel.org, jeffrey.t.kirsher@intel.com, anthony.l.nguyen@intel.com, maciej.fijalkowski@intel.com, maciejromanfijalkowski@gmail.com, intel-wired-lan@lists.osuosl.org Subject: [PATCH bpf-next 2/6] samples/bpf: increment Tx stats at sending Date: Wed, 4 Nov 2020 15:08:58 +0100 Message-Id: <1604498942-24274-3-git-send-email-magnus.karlsson@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1604498942-24274-1-git-send-email-magnus.karlsson@gmail.com> References: <1604498942-24274-1-git-send-email-magnus.karlsson@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Magnus Karlsson Increment the statistics over how many Tx packets have been sent at the time of sending instead of at the time of completion. This as a completion event means that the buffer has been sent AND returned to user space. The packet always gets sent shortly after sendto() is called. The kernel might, for performance reasons, decide to not return every single buffer to user space immediately after sending, for example, only after a batch of packets have been transmitted. Incrementing the number of packets sent at completion, will in that case be confusing as if you send a single packet, the counter might show zero for a while even though the packet has been transmitted. Signed-off-by: Magnus Karlsson Acked-by: John Fastabend --- samples/bpf/xdpsock_user.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c index 1149e94..2567f0d 100644 --- a/samples/bpf/xdpsock_user.c +++ b/samples/bpf/xdpsock_user.c @@ -1146,7 +1146,6 @@ static inline void complete_tx_l2fwd(struct xsk_socket_info *xsk, xsk_ring_prod__submit(&xsk->umem->fq, rcvd); xsk_ring_cons__release(&xsk->umem->cq, rcvd); xsk->outstanding_tx -= rcvd; - xsk->ring_stats.tx_npkts += rcvd; } } @@ -1168,7 +1167,6 @@ static inline void complete_tx_only(struct xsk_socket_info *xsk, if (rcvd > 0) { xsk_ring_cons__release(&xsk->umem->cq, rcvd); xsk->outstanding_tx -= rcvd; - xsk->ring_stats.tx_npkts += rcvd; } } @@ -1260,6 +1258,7 @@ static void tx_only(struct xsk_socket_info *xsk, u32 *frame_nb, int batch_size) } xsk_ring_prod__submit(&xsk->tx, batch_size); + xsk->ring_stats.tx_npkts += batch_size; xsk->outstanding_tx += batch_size; *frame_nb += batch_size; *frame_nb %= NUM_FRAMES; @@ -1348,6 +1347,7 @@ static void l2fwd(struct xsk_socket_info *xsk, struct pollfd *fds) } return; } + xsk->ring_stats.rx_npkts += rcvd; ret = xsk_ring_prod__reserve(&xsk->tx, rcvd, &idx_tx); while (ret != rcvd) { @@ -1379,7 +1379,7 @@ static void l2fwd(struct xsk_socket_info *xsk, struct pollfd *fds) xsk_ring_prod__submit(&xsk->tx, rcvd); xsk_ring_cons__release(&xsk->rx, rcvd); - xsk->ring_stats.rx_npkts += rcvd; + xsk->ring_stats.tx_npkts += rcvd; xsk->outstanding_tx += rcvd; } From patchwork Wed Nov 4 14:09:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Magnus Karlsson X-Patchwork-Id: 315638 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28570C2D0A3 for ; Wed, 4 Nov 2020 14:09:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C1402223AB for ; Wed, 4 Nov 2020 14:09:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iInZt+7v" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730249AbgKDOJy (ORCPT ); Wed, 4 Nov 2020 09:09:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730237AbgKDOJw (ORCPT ); Wed, 4 Nov 2020 09:09:52 -0500 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7BD3CC061A4A; Wed, 4 Nov 2020 06:09:51 -0800 (PST) Received: by mail-pf1-x441.google.com with SMTP id c20so17355649pfr.8; Wed, 04 Nov 2020 06:09:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=SjS5TEp5z8cEIBGlzyQMwfGUqpE+fsLHvYNxytW1zhE=; b=iInZt+7vo/656g3R+HqOPQwJLDpnDzLZdTp/NyL9cIzNchStu1zbL61zamGqJi1Ymy 6JE480O5LJ8U+EtaZtqVenWwm6nuiLxFhlk6+T0SCkqJj+FRtTW2K4JDGt5PYfNvSHFD 49CAxFvktj6Vti/2g4zzxuQ7xUcT/Dhr2sCCVh7ZbB3qFJ9WaE62x9HrCfQwg7RmWjMW 5KqicybSFNNgSX3YTuBECODXL6x9gaHBgMg2h1AHzbro/VbwfCxGENPPkg+k7GzUnY+D MzTdbGrsvQfbsMi/YRUO9DzLCjB0xRp5m4UeRyQrrYOsqqX14WyMLgDAgufN1ZNmv5xn 0y9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=SjS5TEp5z8cEIBGlzyQMwfGUqpE+fsLHvYNxytW1zhE=; b=KSqVB+hR3yxq7XIjEkXqNvpikXyrD7C6Zp7pUiBp00y0YT0Dt43hXxgQPie22EyGZg VP8+skP+pw+vzjM8rGcQ+zF8oQAr2oPOLwzX8t/aR9eMGkEKQEGUhNRKgnvzWLqi2JEu TwQqQojUHCgZKYH8jZP9bqKyjOTmchc0aPbi3fouDM/i+F+3wpE+izB3Y6HFRu/Lwdxh cOFD7cvYcxFYzsVuixEjPEmDzAoTsGJxz7FEvSNVQY0PUMMBi96StKz9bJdZhl2Dorhl XVU4DwMRf8YHreFdNbk5jHG2mrJYutmtLgsRH2GVYEIHOR9q9EUtTPpHTp3K3nN5EylO H/LA== X-Gm-Message-State: AOAM530iwjFECXKzKi3k4ePFQrsaoJU/dZ3p2B0qmaSbFt5ZNmIKhxMX fq4SMIsNuvPeIZFJNVyCR27dewui4Qyjxf5p2AQ= X-Google-Smtp-Source: ABdhPJwNBSJJDYLtTsXrcjAt4hqpccMQ5da+iMvYUZs5LnvSwt+eTwObKsSYWc1EFBYyXl1kjAiLQw== X-Received: by 2002:a63:1924:: with SMTP id z36mr21497231pgl.354.1604498991045; Wed, 04 Nov 2020 06:09:51 -0800 (PST) Received: from VM.ger.corp.intel.com ([192.55.55.43]) by smtp.gmail.com with ESMTPSA id q123sm2724818pfq.56.2020.11.04.06.09.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Nov 2020 06:09:50 -0800 (PST) From: Magnus Karlsson To: magnus.karlsson@intel.com, bjorn.topel@intel.com, ast@kernel.org, daniel@iogearbox.net, netdev@vger.kernel.org, jonathan.lemon@gmail.com Cc: bpf@vger.kernel.org, jeffrey.t.kirsher@intel.com, anthony.l.nguyen@intel.com, maciej.fijalkowski@intel.com, maciejromanfijalkowski@gmail.com, intel-wired-lan@lists.osuosl.org Subject: [PATCH bpf-next 4/6] xsk: introduce padding between more ring pointers Date: Wed, 4 Nov 2020 15:09:00 +0100 Message-Id: <1604498942-24274-5-git-send-email-magnus.karlsson@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1604498942-24274-1-git-send-email-magnus.karlsson@gmail.com> References: <1604498942-24274-1-git-send-email-magnus.karlsson@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Magnus Karlsson Introduce one cache line worth of padding between the consumer pointer and the flags field as well as between the flags field and the start of the descriptors in all the lockless rings. This so that the x86 HW adjacency prefetcher will not prefetch the adjacent pointer/field when only one pointer/field is going to be used. This improves throughput performance for the l2fwd sample app with 1% on my machine with HW prefetching turned on in the BIOS. Signed-off-by: Magnus Karlsson Acked-by: John Fastabend --- net/xdp/xsk_queue.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/net/xdp/xsk_queue.h b/net/xdp/xsk_queue.h index cdb9cf3..74fac80 100644 --- a/net/xdp/xsk_queue.h +++ b/net/xdp/xsk_queue.h @@ -18,9 +18,11 @@ struct xdp_ring { /* Hinder the adjacent cache prefetcher to prefetch the consumer * pointer if the producer pointer is touched and vice versa. */ - u32 pad ____cacheline_aligned_in_smp; + u32 pad1 ____cacheline_aligned_in_smp; u32 consumer ____cacheline_aligned_in_smp; + u32 pad2 ____cacheline_aligned_in_smp; u32 flags; + u32 pad3 ____cacheline_aligned_in_smp; }; /* Used for the RX and TX queues for packets */