From patchwork Tue Jun 22 22:50:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 465543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50459C48BDF for ; Tue, 22 Jun 2021 22:54:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 302D46135A for ; Tue, 22 Jun 2021 22:54:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230380AbhFVW4z (ORCPT ); Tue, 22 Jun 2021 18:56:55 -0400 Received: from mail.kernel.org ([198.145.29.99]:51358 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229800AbhFVW4y (ORCPT ); Tue, 22 Jun 2021 18:56:54 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id BE39D61107; Tue, 22 Jun 2021 22:54:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1624402478; bh=o7seISUNB4f/eTt96A55r1Jq41yJV74Z9zrPMMzgLo0=; h=From:To:Cc:Subject:Date:From; b=brMxoqcPCnJAtAvYG7R1qoZtfF0NqFQMqbm83gB3CGdtdu8eNiKVVbF158RYEEsz8 cxHMumVVOrUB/2fO3JRoAgYbYcyHKwTD3J+ZpyPjE0lHFW8abGCjm3SacUxm9v7ZXE RZ0Igme8T4I2hWYeF207G2jRIjUpqUSNaigTmuKq7Uc6T+QtDAEJfb5HlT8nqJ2zNb 5B0QpAaSt1OXlHWKGNRpXO0gMh2RCFFy5Khkhxl9b/O2pumNbhhVF1Y5RXZ+eDDXyY dNSs3hUT5eOpcn3VSN9A3zOuiSjOl475+wi3BFhtvEqwPV11BSwhBsEYrLnaDi63SQ Voz/WpWEdqbjA== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, willemb@google.com, eric.dumazet@gmail.com, dsahern@gmail.com, yoshfuji@linux-ipv6.org, Jakub Kicinski Subject: [PATCH net-next v2 1/2] net: ip: refactor SG checks Date: Tue, 22 Jun 2021 15:50:56 -0700 Message-Id: <20210622225057.2108592-1-kuba@kernel.org> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org There is a number of rt->dst.dev->features & NETIF_F_SG checks scattered throughout the code. Shorten the lines by caching the result of this check. Signed-off-by: Jakub Kicinski --- net/ipv4/ip_output.c | 13 ++++++------- net/ipv6/ip6_output.c | 13 ++++++------- 2 files changed, 12 insertions(+), 14 deletions(-) diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index c3efc7d658f6..90031f5446bd 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -981,12 +981,14 @@ static int __ip_append_data(struct sock *sk, unsigned int maxfraglen, fragheaderlen, maxnonfragsize; int csummode = CHECKSUM_NONE; struct rtable *rt = (struct rtable *)cork->dst; + bool has_sg, paged, extra_uref = false; unsigned int wmem_alloc_delta = 0; - bool paged, extra_uref = false; u32 tskey = 0; skb = skb_peek_tail(queue); + has_sg = rt->dst.dev->features & NETIF_F_SG; + exthdrlen = !skb ? rt->dst.header_len : 0; mtu = cork->gso_size ? IP_MAX_MTU : cork->fragsize; paged = !!cork->gso_size; @@ -1023,8 +1025,7 @@ static int __ip_append_data(struct sock *sk, if (!uarg) return -ENOBUFS; extra_uref = !skb_zcopy(skb); /* only ref on new uarg */ - if (rt->dst.dev->features & NETIF_F_SG && - csummode == CHECKSUM_PARTIAL) { + if (has_sg && csummode == CHECKSUM_PARTIAL) { paged = true; } else { uarg->zerocopy = 0; @@ -1074,8 +1075,7 @@ static int __ip_append_data(struct sock *sk, fraglen = datalen + fragheaderlen; pagedlen = 0; - if ((flags & MSG_MORE) && - !(rt->dst.dev->features&NETIF_F_SG)) + if ((flags & MSG_MORE) && !has_sg) alloclen = mtu; else if (!paged) alloclen = fraglen; @@ -1174,8 +1174,7 @@ static int __ip_append_data(struct sock *sk, if (copy > length) copy = length; - if (!(rt->dst.dev->features&NETIF_F_SG) && - skb_tailroom(skb) >= copy) { + if (!has_sg && skb_tailroom(skb) >= copy) { unsigned int off; off = skb->len; diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index ff4f9ebcf7f6..c667b7e2856f 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1444,8 +1444,8 @@ static int __ip6_append_data(struct sock *sk, struct ipv6_txoptions *opt = v6_cork->opt; int csummode = CHECKSUM_NONE; unsigned int maxnonfragsize, headersize; + bool has_sg, paged, extra_uref = false; unsigned int wmem_alloc_delta = 0; - bool paged, extra_uref = false; skb = skb_peek_tail(queue); if (!skb) { @@ -1453,6 +1453,8 @@ static int __ip6_append_data(struct sock *sk, dst_exthdrlen = rt->dst.header_len - rt->rt6i_nfheader_len; } + has_sg = rt->dst.dev->features & NETIF_F_SG; + paged = !!cork->gso_size; mtu = cork->gso_size ? IP6_MAX_MTU : cork->fragsize; orig_mtu = mtu; @@ -1515,8 +1517,7 @@ static int __ip6_append_data(struct sock *sk, if (!uarg) return -ENOBUFS; extra_uref = !skb_zcopy(skb); /* only ref on new uarg */ - if (rt->dst.dev->features & NETIF_F_SG && - csummode == CHECKSUM_PARTIAL) { + if (has_sg && csummode == CHECKSUM_PARTIAL) { paged = true; } else { uarg->zerocopy = 0; @@ -1582,8 +1583,7 @@ static int __ip6_append_data(struct sock *sk, fraglen = datalen + fragheaderlen; pagedlen = 0; - if ((flags & MSG_MORE) && - !(rt->dst.dev->features&NETIF_F_SG)) + if ((flags & MSG_MORE) && !has_sg) alloclen = mtu; else if (!paged) alloclen = fraglen; @@ -1698,8 +1698,7 @@ static int __ip6_append_data(struct sock *sk, if (copy > length) copy = length; - if (!(rt->dst.dev->features&NETIF_F_SG) && - skb_tailroom(skb) >= copy) { + if (!has_sg && skb_tailroom(skb) >= copy) { unsigned int off; off = skb->len; From patchwork Tue Jun 22 22:50:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 466267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEF09C2B9F4 for ; Tue, 22 Jun 2021 22:54:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C10FB6102A for ; Tue, 22 Jun 2021 22:54:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230445AbhFVW44 (ORCPT ); Tue, 22 Jun 2021 18:56:56 -0400 Received: from mail.kernel.org ([198.145.29.99]:51376 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229955AbhFVW4y (ORCPT ); Tue, 22 Jun 2021 18:56:54 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 2A4BE61042; Tue, 22 Jun 2021 22:54:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1624402478; bh=tdijdXDmNYtsiY2V/IjRRNu5X1ZuOn4QcSnK5odX/LI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qrzbYTg4qc9SJtYOu5AWZ8veVXfekZ8KZapS8uZbAkaPcKQzTHx6R1c9mS6xrKPMn RwRLtNFg0Yere99q3TT5Q8eqmPdp53VlR7i9EVoOShePC4UAoaWMRG0/70b18RfSuL 1gvuPmi92mUCkOuTxYW5OClcSnaogwHsy5row1QpZ2Bz62uuoO415iZ2HUhcO19Kd7 CHi0X9V1C0W5sOecHN67/0OIDA2Nh+KBzuzF8c1u4l17DDgAhIBX1VgIsFrrf7gfx+ ES2/aPPQ18dm8mrhG1ejY3VGtvY1IHz2wv8ZcKG5sAfTIV63fkf4SRJ3/1M8gl6AUt sb3fwEbyEVJrA== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, willemb@google.com, eric.dumazet@gmail.com, dsahern@gmail.com, yoshfuji@linux-ipv6.org, Jakub Kicinski , Dave Jones Subject: [PATCH net-next v2 2/2] net: ip: avoid OOM kills with large UDP sends over loopback Date: Tue, 22 Jun 2021 15:50:57 -0700 Message-Id: <20210622225057.2108592-2-kuba@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210622225057.2108592-1-kuba@kernel.org> References: <20210622225057.2108592-1-kuba@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Dave observed number of machines hitting OOM on the UDP send path. The workload seems to be sending large UDP packets over loopback. Since loopback has MTU of 64k kernel will try to allocate an skb with up to 64k of head space. This has a good chance of failing under memory pressure. What's worse if the message length is <32k the allocation may trigger an OOM killer. This is entirely avoidable, we can use an skb with frags. af_unix solves a similar problem by limiting the head length to SKB_MAX_ALLOC. This seems like a good and simple approach. It means that UDP messages > 16kB will now use fragments if underlying device supports SG, if extra allocator pressure causes regressions in real workloads we can switch to trying the large allocation first and falling back. Reported-by: Dave Jones Signed-off-by: Jakub Kicinski --- net/ipv4/ip_output.c | 2 +- net/ipv6/ip6_output.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index 90031f5446bd..1ab140c173d0 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -1077,7 +1077,7 @@ static int __ip_append_data(struct sock *sk, if ((flags & MSG_MORE) && !has_sg) alloclen = mtu; - else if (!paged) + else if (!paged && (fraglen < SKB_MAX_ALLOC || !has_sg)) alloclen = fraglen; else { alloclen = min_t(int, fraglen, MAX_HEADER); diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index c667b7e2856f..46d805097a79 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1585,7 +1585,7 @@ static int __ip6_append_data(struct sock *sk, if ((flags & MSG_MORE) && !has_sg) alloclen = mtu; - else if (!paged) + else if (!paged && (fraglen < SKB_MAX_ALLOC || !has_sg)) alloclen = fraglen; else { alloclen = min_t(int, fraglen, MAX_HEADER);