From patchwork Wed May 14 13:56:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Chia-Yu Chang \(Nokia\)" X-Patchwork-Id: 890052 Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2053.outbound.protection.outlook.com [40.107.21.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4EAFF27A465; Wed, 14 May 2025 13:56:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.21.53 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231013; cv=fail; b=PuNXhI9LbhyusOK+/cZuQ53r6GCz4wI2Cq+TTZzOvJi9Xm4bHBbffkHkce3fCWMQpjgnFVo9nwP+pBI+aXN+ssMn0htmnsIFK1502wbI5+trPmhA1W6cdAS3rdY5maDOdJVKBiOd1FbCLSud3BbJZ57PuIo/53MRWPoSscnRiL4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231013; c=relaxed/simple; bh=PYyCXUuKVKRWlC5fYTcacULQXbuy70otEK62ZKgXQqA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=IVFSM4wB9FN6qQy0UMpbXEojmONFy1CemD7kqy4jLYgBIVGoyXADaC/+qLcOgZfdh99UWac0zM7qvvBfz34HBJV78tPKQ+blN3+ADLblDxwPMCfqyKNCi8uYcdV1TqbcS7TOC56V9tiyYYy+MMqEjLJvbFWtAzNFTNJHycGGE6w= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com; spf=fail smtp.mailfrom=nokia-bell-labs.com; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b=rLgYvlLn; arc=fail smtp.client-ip=40.107.21.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b="rLgYvlLn" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=daTvxasa0sf9HwFuqPpibnN6CWWyCQSF4zexmwsrVZ+lNgFNxr1kGlNtu1CzKtiAhR/XRgjrOOxIf70p3Fdw1vTTAFclLtdrGT0RsGOLIPhBBzNSN/QfPHtiybKn9uN0sTqaMslaaE9NwUv2WtEZ91mFDnXJ+A9kQJyX6bSkG7xiPaphofWaETes6Jbo60sPNrjAYWRzYOWs5uOHzVUcgv5MnIHxr4su1BklqopezHhkogn4a4QUrATr3Frwo3tTbhjM2mQSTqLDKqiY9HGJ1pBAIdTMvVWd4917MU46yq1AK15aIrNx7GyuEpxeCRMY3bAI4BdPQYrWE/qLcn0I/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Jbp9mDaGjGTen8BidXf0OT4su1XnPFAKk8A8EBFXEZQ=; b=xl80tEDQYfOqbxA/EPgKeRufqqEJnV4mXN+yKUut6W+fjWxr5A9ZbVnr9rdyCkygX3+YRTbXQXktlYcRSmEB7lVEpJTBQtkiYR7V+oDr2/J7lA11tIZJBe5o5ebdhXy7TGQTw/DUEsCqnBRy2iXWQHZ2bXBT4D1qKkBu+ZHxeTFpMKwqa/mDzXAmBQyBFKoHXRYx1k81mwWFv6ZW/8e44auox+QjWGUlcxxxSAXgnUMAzL3Mgk2t5IZsWb90koskKvcwFBDDLkF/G+eNmh9Z0SpvmlEjKUJCkpOKXOyJxiB6MQAIg+Qk4rrKA9yaRkP9nCvZi3vZhFH/aY5ebHqWyA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 131.228.2.240) smtp.rcpttodomain=amazon.com smtp.mailfrom=nokia-bell-labs.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nokia-bell-labs.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia-bell-labs.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Jbp9mDaGjGTen8BidXf0OT4su1XnPFAKk8A8EBFXEZQ=; b=rLgYvlLnxG06HInm4/CI3yhhgmB3uL6/U+xBjuYMCqOu/uRatC1Rh8j+GSzv4O9Ku8/xbi6ueVT8CSID3Rnb4I2aiZaPaBL1TGudXYc8YyC7ObJUagTiJYT7KPdvRV12IVH57FL7YqpIrnrErKR3PQLxTcqrVo2RnZwG/BLUb5+DwWjATiBu61FKAHWkFyrcRc0DHtcJqghX/dKaPva/YqugAX3DvuwYA0Ne8HmU9i/0702W6TQEWFCZSqSjrcNjw5zz1iV7ae4onw2MbTnnCXPlMBO5uuoC0SzmToxCYtQKEJMTsAePTBmHFXmU39Zwq2w8v4O/qiGXnu1zKAPc5A== Received: from DU7P194CA0010.EURP194.PROD.OUTLOOK.COM (2603:10a6:10:553::13) by DB8PR07MB6394.eurprd07.prod.outlook.com (2603:10a6:10:13b::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8722.30; Wed, 14 May 2025 13:56:47 +0000 Received: from DU2PEPF00028CFD.eurprd03.prod.outlook.com (2603:10a6:10:553:cafe::90) by DU7P194CA0010.outlook.office365.com (2603:10a6:10:553::13) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8746.17 via Frontend Transport; Wed, 14 May 2025 13:56:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 131.228.2.240) smtp.mailfrom=nokia-bell-labs.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nokia-bell-labs.com; Received-SPF: Pass (protection.outlook.com: domain of nokia-bell-labs.com designates 131.228.2.240 as permitted sender) receiver=protection.outlook.com; client-ip=131.228.2.240; helo=fihe3nok0735.emea.nsn-net.net; pr=C Received: from fihe3nok0735.emea.nsn-net.net (131.228.2.240) by DU2PEPF00028CFD.mail.protection.outlook.com (10.167.242.181) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8722.18 via Frontend Transport; Wed, 14 May 2025 13:56:47 +0000 Received: from sarah.nbl.nsn-rdnet.net (sarah.nbl.nsn-rdnet.net [10.0.73.150]) by fihe3nok0735.emea.nsn-net.net (Postfix) with ESMTP id 339822009B; Wed, 14 May 2025 16:56:46 +0300 (EEST) From: chia-yu.chang@nokia-bell-labs.com To: linux-doc@vger.kernel.org, corbet@lwn.net, horms@kernel.org, dsahern@kernel.org, kuniyu@amazon.com, bpf@vger.kernel.org, netdev@vger.kernel.org, dave.taht@gmail.com, pabeni@redhat.com, jhs@mojatatu.com, kuba@kernel.org, stephen@networkplumber.org, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, andrew+netdev@lunn.ch, donald.hunter@gmail.com, ast@fiberby.net, liuhangbin@gmail.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, ij@kernel.org, ncardwell@google.com, koen.de_schepper@nokia-bell-labs.com, g.white@cablelabs.com, ingemar.s.johansson@ericsson.com, mirja.kuehlewind@ericsson.com, cheshire@apple.com, rs.ietf@gmx.at, Jason_Livingood@comcast.com, vidhi_goel@apple.com Cc: Chia-Yu Chang Subject: [PATCH v7 net-next 01/15] tcp: reorganize SYN ECN code Date: Wed, 14 May 2025 15:56:28 +0200 Message-Id: <20250514135642.11203-2-chia-yu.chang@nokia-bell-labs.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> References: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU2PEPF00028CFD:EE_|DB8PR07MB6394:EE_ X-MS-Office365-Filtering-Correlation-Id: 46520629-f505-4ae8-942d-08dd92ef2b5b X-LD-Processed: 5d471751-9675-428d-917b-70f44f9630b0,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|36860700013|82310400026|376014|7416014|921020; X-Microsoft-Antispam-Message-Info: =?utf-8?q?lTZNjFn9ZxlGzwis7lZFYMan5E5Dk3x?= =?utf-8?q?Ig6+CEQtQWz1x4Xs7AIv4AKAO/JbSQQbkHg3ra4fryOFxDbd1JKLXXgBSdx8a4vjF?= =?utf-8?q?iHxmM36MPr1oubr8zW+NPyqvu45aLuUQTCGg3wITmN5QfrBo10rUqwxg41KVkPiQ1?= =?utf-8?q?h0r0GKxV+wsTkVO1he+WhFCH1C7fStolKzXbu4w7oj23VOrkZRnvuK+nBi5H/AqF9?= =?utf-8?q?oHNtpvup6+gQ4hv0weTKsYYkJwbzYypCYk29kTW/8deR/t1zGWH2ONuAfOXourXH7?= =?utf-8?q?pq32d/ppP0Ox6TMc3b2Jt0kjZA5OrxodyQxZ07eXvGsBdXz/MOzCkmN7C1FYfa0GX?= =?utf-8?q?2JSnO+xIXqRc/owJ+OCM/OyFpLWt05BZgh7ZbDk34LSdKh4oWDLw/tdKRudeW10j6?= =?utf-8?q?+NZ9slj5PW64naIe6Evkg13aEgbIMXC+/3nDN/vRD2S8f7K9JqhPaB9wd3StdGP9X?= =?utf-8?q?GEO9CC12AYOAAyOjmwb7AEf6yFjOv36byy2VDhmrxn+XnlEGazTUUKza/xHTS3x69?= =?utf-8?q?WGyc+s5IbJCPj52xocCAGzVqsHzsuxWGG+/lsPSWFAj057yNLwcAgKY35/nEdgW4n?= =?utf-8?q?6K9DTWYbZOk78tHALMXgW3RUiRqYIkFD9ZTF572tsPptlSqdoBto72RUTUmQwA/At?= =?utf-8?q?w4Ap33/HnELX1RZd5t/hI7P89Aq/xRdGUylIP6B1SNaNp/I87nqF/nIgILqMCQzWG?= =?utf-8?q?bfsySoKObZKEySayv1tAEHNTzi1MZ+i71OTcDQGOrKqzfzNbVCNji9d5y7uvtPUEW?= =?utf-8?q?OQ1dwVVmU6A6i8+tCaQS+40DylIOHdl7SEhGW/U5FvP9d72UFyjKmY4Lhk8VuBWJ6?= =?utf-8?q?nTgtgov7vQoodJskUuyxyLANS6SrS+KUTxkJKOgBrt3ignKL4xooU0v/PJbWNeuZe?= =?utf-8?q?fT7JfwO0VwL/kNpNbTUdodIYWbOcdsrdT2duG8Y2ndiqQpQqelSvXhDfY792KEkCL?= =?utf-8?q?jWzDcsVQkDbwqHUyHo2U8jKhcONye2ALgZoPNceSIv5OK7o628ZrAbUoAqG9RvLMj?= =?utf-8?q?+M4WZNtSqp84ho0w7Q5M/194d+4tmZKAE09MX9cDwgPxZhIuBoEDVSebJAVR+/IX8?= =?utf-8?q?kBaF6uy4V5CkQwqB8Hec0ifSANR+QYOO/CrIymdT1L1UB9SvJxSEgueUuB4IH2sq4?= =?utf-8?q?RjMOEhPafYbZGt/1TIzeXT1pPbt7cI3LgRCUCEoJe8GLAlh2ufKAjD7rnEBMq7An/?= =?utf-8?q?nUyfNMtAAChFWsbpPRXbQi0kDwllGgZhLj2mm6llWKy2uVXPvGd9tAFhThDRbi+7s?= =?utf-8?q?DXdQiaF1kdyKx/XElAK+fBbgxxt5CPb/NoLtG3nSLoOlEe3X4UovsfMc/XBc5Ri7s?= =?utf-8?q?kXZ6bfDQkGpbv2Z/5S3xca89bH4ksMIrvK2vubKmbJsN2O2OfW5QicmKdEgs8Z08D?= =?utf-8?q?ZaGL589XSp1004ka2Pfbpns0glpyeM01sLzDDkaABrC6ngRnm5BFoKqWY+47er2jf?= =?utf-8?q?58a4kqReSKd1X8a7rYHUH8zbYGH0cSGw=3D=3D?= X-Forefront-Antispam-Report: CIP:131.228.2.240; CTRY:FI; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:fihe3nok0735.emea.nsn-net.net; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(1800799024)(36860700013)(82310400026)(376014)(7416014)(921020); DIR:OUT; SFP:1101; X-OriginatorOrg: nokia-bell-labs.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2025 13:56:47.7937 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 46520629-f505-4ae8-942d-08dd92ef2b5b X-MS-Exchange-CrossTenant-Id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5d471751-9675-428d-917b-70f44f9630b0; Ip=[131.228.2.240]; Helo=[fihe3nok0735.emea.nsn-net.net] X-MS-Exchange-CrossTenant-AuthSource: DU2PEPF00028CFD.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR07MB6394 From: Ilpo Järvinen Prepare for AccECN that needs to have access here on IP ECN field value which is only available after INET_ECN_xmit(). No functional changes. Signed-off-by: Ilpo Järvinen Signed-off-by: Chia-Yu Chang Reviewed-by: Eric Dumazet --- net/ipv4/tcp_output.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 13295a59d22e..9a1ab946ff62 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -350,10 +350,11 @@ static void tcp_ecn_send_syn(struct sock *sk, struct sk_buff *skb) tp->ecn_flags = 0; if (use_ecn) { - TCP_SKB_CB(skb)->tcp_flags |= TCPHDR_ECE | TCPHDR_CWR; - tcp_ecn_mode_set(tp, TCP_ECN_MODE_RFC3168); if (tcp_ca_needs_ecn(sk) || bpf_needs_ecn) INET_ECN_xmit(sk); + + TCP_SKB_CB(skb)->tcp_flags |= TCPHDR_ECE | TCPHDR_CWR; + tcp_ecn_mode_set(tp, TCP_ECN_MODE_RFC3168); } } From patchwork Wed May 14 13:56:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chia-Yu Chang \(Nokia\)" X-Patchwork-Id: 890051 Received: from DB3PR0202CU003.outbound.protection.outlook.com (mail-northeuropeazon11011052.outbound.protection.outlook.com [52.101.65.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 726CE27C175; Wed, 14 May 2025 13:56:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.65.52 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231021; cv=fail; b=q1V2lsLpwLVnpyLNLubK3aiBDrrU1Cq59oAiJnAQEA3ewJsgqQKhYFARbBFLJgY2wCDetWM+vRxHJInufNtHldr1YrLvPCUZhPbyBKmsqN7+4rfrHL2MF7pvRMVA3hrHP4+YkkXLPTlwk9T8POaSA6/V4XVgS/iBe2Pn7VEfFwc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231021; c=relaxed/simple; bh=iFKd1UJp9PQqL0tER0fodJJRNiOr86sZy0ZnvNT38nM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=o6GoVl6GdckcQ0AjNB1YscGZX0jV8pinQjyMkoN9lZLGbztPbz913Hw2ERi/pMkEgNIHwzIG+Qxn6BdWsbQS8R2yPDm+1i5mCkBE+JyXBXe7o9fewcp6CepnFTPRi+MiqkFeTeALWIEXpWBYCflrNkgoxrM30RUCaUZ29+oebYQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com; spf=fail smtp.mailfrom=nokia-bell-labs.com; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b=M0sVSZhR; arc=fail smtp.client-ip=52.101.65.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b="M0sVSZhR" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KrnH5Tzm0hvpJpcIFFBrzivs4s73lfY9m8zLuCQR44A6/DUp1Zq1EdQyIjlUa9NboDyLGNc+LEmjp7LoNyAi6coWlUX36LwV9nFuEL2QISUrmMuDUvWPLH2WYFZOxqJy/Hmm6uF7IZUJW4StTfZ5VTyFb5lgIQcXCddhO0KQoVGQ77HID1Vo5GgVqvmyTX7cO1rEen6AYlgTNnUMyYuJARh2i+YGsx8pMPkDbe3PnHXJID2HOeuhfD0/2gyerNSk8BGF3019rp+g3E1NByogpb9OeokHUDxHZfnTnd3eCYHUQivyacAGRC4o1PG7wA1icqJui/nP9hxLwTb+p2LMOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YSVuX+M5utpsiNsGLb8bRjvC3kcK1sHHaC0wWy0l5vI=; b=r4xA7IpRFrZnCFDG/c4PsnQ2JRIhjssylxOdSfT/tQ0qPbn6Awxo1RM1nDcNmjjVVK+Eb+nh5U2RgeWtAEni+bavY2vfoLMZ1bArDDBYLrYleAg12Oi3Oj7BebNuFEegK0IjmFbvuyfP8Lx8MChi5wB1LZlwUJfUzJIdY2nYamFLThgXs6qnMM20dzSeh/U74zSgeQUVEAlyj2lqu5hrnBCArawn+sN345A1LI2ZPz8orPpAXrGKWTAorqXOPxGlZtPDe1Kf/dBYn7gdnUelNkIKW698XqdUVtbh/ROqvbjsZTBm/VOgfex0y7o66ov+ruRSWAFBISMC9qjt7hD9ZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=temperror (sender ip is 131.228.2.240) smtp.rcpttodomain=amazon.com smtp.mailfrom=nokia-bell-labs.com; dmarc=temperror action=none header.from=nokia-bell-labs.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia-bell-labs.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YSVuX+M5utpsiNsGLb8bRjvC3kcK1sHHaC0wWy0l5vI=; b=M0sVSZhRayPyEqN1iSk39bRX8kMzUIOnlLq/4X8rMojXA/HXbESlppzIzK75tIEwuzWvbBA00KWwbF5HOnLxw8/Ux40Pyn689nwbd3ApXDCWgQJYkWjoGZsNL3OdH6rDZjQurX1JzU3siR0KaIFp2cQs+QHraUAJkpe2rusOuHnQqNeq8V5M+ZgqUbfi7MDkInCunef3JKyswydCoQcSVAHSJtNQBqFT2yTJ/9GZ7Xuj1hkfFzrRA1L1q/L5nvgU+XNGfHBrfL5g3ZR7wjqXrfHAi5tLLrSsK7AU4lqMl6WMplzKXHOmEgPWi8glfxPBPQZHwJYmzOVKv0RXso2+dA== Received: from CWLP265CA0437.GBRP265.PROD.OUTLOOK.COM (2603:10a6:400:1d7::13) by DB9PR07MB7100.eurprd07.prod.outlook.com (2603:10a6:10:21a::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8722.31; Wed, 14 May 2025 13:56:54 +0000 Received: from AM1PEPF000252DB.eurprd07.prod.outlook.com (2603:10a6:400:1d7:cafe::ca) by CWLP265CA0437.outlook.office365.com (2603:10a6:400:1d7::13) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8746.17 via Frontend Transport; Wed, 14 May 2025 13:56:54 +0000 X-MS-Exchange-Authentication-Results: spf=temperror (sender IP is 131.228.2.240) smtp.mailfrom=nokia-bell-labs.com; dkim=none (message not signed) header.d=none;dmarc=temperror action=none header.from=nokia-bell-labs.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of nokia-bell-labs.com: DNS Timeout) Received: from fihe3nok0735.emea.nsn-net.net (131.228.2.240) by AM1PEPF000252DB.mail.protection.outlook.com (10.167.16.53) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8722.18 via Frontend Transport; Wed, 14 May 2025 13:56:53 +0000 Received: from sarah.nbl.nsn-rdnet.net (sarah.nbl.nsn-rdnet.net [10.0.73.150]) by fihe3nok0735.emea.nsn-net.net (Postfix) with ESMTP id C80AE2009B; Wed, 14 May 2025 16:56:51 +0300 (EEST) From: chia-yu.chang@nokia-bell-labs.com To: linux-doc@vger.kernel.org, corbet@lwn.net, horms@kernel.org, dsahern@kernel.org, kuniyu@amazon.com, bpf@vger.kernel.org, netdev@vger.kernel.org, dave.taht@gmail.com, pabeni@redhat.com, jhs@mojatatu.com, kuba@kernel.org, stephen@networkplumber.org, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, andrew+netdev@lunn.ch, donald.hunter@gmail.com, ast@fiberby.net, liuhangbin@gmail.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, ij@kernel.org, ncardwell@google.com, koen.de_schepper@nokia-bell-labs.com, g.white@cablelabs.com, ingemar.s.johansson@ericsson.com, mirja.kuehlewind@ericsson.com, cheshire@apple.com, rs.ietf@gmx.at, Jason_Livingood@comcast.com, vidhi_goel@apple.com Cc: Chia-Yu Chang Subject: [PATCH v7 net-next 03/15] tcp: reorganize tcp_sock_write_txrx group for variables later Date: Wed, 14 May 2025 15:56:30 +0200 Message-Id: <20250514135642.11203-4-chia-yu.chang@nokia-bell-labs.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> References: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM1PEPF000252DB:EE_|DB9PR07MB7100:EE_ X-MS-Office365-Filtering-Correlation-Id: 431ef7b3-8cdf-4bb2-8178-08dd92ef2eae X-LD-Processed: 5d471751-9675-428d-917b-70f44f9630b0,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|36860700013|82310400026|7416014|376014|921020; X-Microsoft-Antispam-Message-Info: Wm10JTyTdCkbo54nvEQPG6y721aDiOI/HFwwyr2d1FC3JDHVDauUGWi+qK8bGXFNWiQev+PRsNTGKbEDs9K1YOqxcYQkYdS7WMB/cdLCOHTRml/N1ccg4+bev5JJ9XnAFhxnj1KJDXmDGpATScnEVjCVeZ6rpuu1pwg83XhcfM0FvPwjcyn1IDzE26ToV9iLd+dIrYlPyvm6eFygiC+rQ0f8ApqWsyG4cFLpocr6zuPS2kCsC5whMTfG2r/czhmhOSmKYK4lhiGktzqOEYaLG8h5V45twCqEcxDTgwDHsU3Ici8IaJFqrVojnREL/L755EBIw88LQkundxopV3ELftT/unO9oxyg9I6t8yYE6MvBAnclxrrWRifRyI70Pl0D0chvnAOaeET3SdwhugBj1CZ9xPjomACK4ASYZoc6OBE/jRmGC6GlQ7DThajcOdgrK3AmHGQmrG3ERM2wG6tTrJjiJkETFLk6iRLlw0d20rgXtgv3BsYZ+Ipyfq1G07DnhdN3tNGWJlDoQxfERZKJhLKIP6440eeEQLV8vxm5QcPBAVe9y1boPgFdI7x6fmF7fAra9RmpAUkNqacvXWN60CHNE+RTJred9kap2QEtAa1/GdfhAcYhUGaBSFetRi5z+C7YtKvgewFXn0n2ZuueODE2CZgsxZ6KabjZ13ZpQBdqHT2hd1LL8WX/BQjdUGiIIvBLTZwaIVYsYln+LqMKWOso06cZIXmkg/BE02PqyuE9H0fSvN6sOUb/n9t0tihzjaqDYAn+rOvJwHHINKNNcrt3EzfSI6F9VM6Rm1WEmQWDM87+0k7fvgu5pnnanvIiwbzk80iNekqTFv1hYF/pWTqViRbeOWpIlgfJ37DWYePdL4iiMOne7+0nd94R6edCCn9jArUme59zajvICcVD4ItqfMgF/SnVhGvJUQIWCKis/LuA4sURx4I6rPh2eAW+Pi5pZQg/1hx4vyadsF4Po1FflYSuRwfaarLOoDorsMPGyj6ZOLUw910fIK6D8XRbUqKxz3DZmuwGaRoesTSX0mZcl0cXI7nvjLO2UZyX9tM7+uOUJd+lxsuKKvcwk2vMjEgIPqnBJLIbOSk8vxnewNBikthRUoT7P50sB77zwH/ieYIWT42DIWJeZCrfJBQMRqQFn05/uhjmxX2GEtqvK/HpoN/1su3vU0Aznlmw699NP2SfmWNnWVg9nKxJMFYVCMBbR29nmA6TaxU/c1uEMKQ2ZN1VK+k+6GrWRucZpzhirlXfjm8a1zYMg3vrbqK2h59IPr+Jd1fIfe0rGRmJLwSPWWcXq/b5KKs3zDxPdhXPMb/VUOv4hZeQU8exJWNj4Je1NpknfPletaG5DYTjsdRtfetJ3uZhkGHk6RrjLoqypWErpqiu96St7Bq9TtOa9mj0rlG7lJM/nccS7Ez/+vL6/E0tFB56o7ABwnOy+OGPhtxs4KScrP8icAIO40vz6FECqDN9C+bYwk7Y0eGQCzFuOhG13OlGhPNwpTBzsdWd0stZRndRr8b1ax8Ru7BknY8QwVTWGdXzlvAgKHGTBg== X-Forefront-Antispam-Report: CIP:131.228.2.240; CTRY:FI; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:fihe3nok0735.emea.nsn-net.net; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(1800799024)(36860700013)(82310400026)(7416014)(376014)(921020); DIR:OUT; SFP:1101; X-OriginatorOrg: nokia-bell-labs.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2025 13:56:53.2639 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 431ef7b3-8cdf-4bb2-8178-08dd92ef2eae X-MS-Exchange-CrossTenant-Id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5d471751-9675-428d-917b-70f44f9630b0; Ip=[131.228.2.240]; Helo=[fihe3nok0735.emea.nsn-net.net] X-MS-Exchange-CrossTenant-AuthSource: AM1PEPF000252DB.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR07MB7100 From: Chia-Yu Chang Use the first 3-byte hole at the beginning of the tcp_sock_write_txrx group for 'noneagle'/'rate_app_limited' to fill in the existing hole in later patches. Therefore, the group size of tcp_sock_write_txrx is reduced from 92 + 4 to 91 + 4. In addition, the group size of tcp_sock_write_rx is changed to 96 to fit in the pahole outcome. Below are the trimmed pahole outcomes before and after this patch: [BEFORE THIS PATCH] struct tcp_sock { [...] __cacheline_group_begin__tcp_sock_write_txrx[0]; /* 2585 0 */ /* XXX 3 bytes hole, try to pack */ [...] struct tcp_options_received rx_opt; /* 2652 24 */ u8 nonagle:4; /* 2676: 0 1 */ u8 rate_app_limited:1; /* 2676: 4 1 */ /* XXX 3 bits hole, try to pack */ __cacheline_group_end__tcp_sock_write_txrx[0]; /* 2677 0 */ /* XXX 3 bytes hole, try to pack */ __cacheline_group_begin__tcp_sock_write_rx[0] __attribute__((__aligned__(8))); /* 2680 0 */ [...] __cacheline_group_end__tcp_sock_write_rx[0]; /* 2776 0 */ [...] /* size: 3264, cachelines: 51, members: 163 */ } [AFTER THIS PATCH] struct tcp_sock { [...] __cacheline_group_begin__tcp_sock_write_txrx[0]; /* 2585 0 */ u8 nonagle:4; /* 2585: 0 1 */ u8 rate_app_limited:1; /* 2585: 4 1 */ /* XXX 3 bits hole, try to pack */ /* XXX 2 bytes hole, try to pack */ [...] struct tcp_options_received rx_opt; /* 2652 24 */ __cacheline_group_end__tcp_sock_write_txrx[0]; /* 2676 0 */ /* XXX 4 bytes hole, try to pack */ __cacheline_group_begin__tcp_sock_write_rx[0] __attribute__((__aligned__(8))); /* 2680 0 */ [...] __cacheline_group_end__tcp_sock_write_rx[0]; /* 2776 0 */ [...] /* size: 3264, cachelines: 51, members: 163 */ } Signed-off-by: Chia-Yu Chang --- include/linux/tcp.h | 4 ++-- net/ipv4/tcp.c | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/linux/tcp.h b/include/linux/tcp.h index a8af71623ba7..0c82a8c942dc 100644 --- a/include/linux/tcp.h +++ b/include/linux/tcp.h @@ -286,6 +286,8 @@ struct tcp_sock { * Header prediction flags * 0x5?10 << 16 + snd_wnd in net byte order */ + u8 nonagle : 4,/* Disable Nagle algorithm? */ + rate_app_limited:1; /* rate_{delivered,interval_us} limited? */ __be32 pred_flags; u64 tcp_clock_cache; /* cache last tcp_clock_ns() (see tcp_mstamp_refresh()) */ u64 tcp_mstamp; /* most recent packet received/sent */ @@ -304,8 +306,6 @@ struct tcp_sock { * Options received (usually on last packet, some only on SYN packets). */ struct tcp_options_received rx_opt; - u8 nonagle : 4,/* Disable Nagle algorithm? */ - rate_app_limited:1; /* rate_{delivered,interval_us} limited? */ __cacheline_group_end(tcp_sock_write_txrx); /* RX read-write hotpath cache lines */ diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 0ae265d39184..bc9ab715fc2b 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -5130,7 +5130,7 @@ static void __init tcp_struct_check(void) /* 32bit arches with 8byte alignment on u64 fields might need padding * before tcp_clock_cache. */ - CACHELINE_ASSERT_GROUP_SIZE(struct tcp_sock, tcp_sock_write_txrx, 92 + 4); + CACHELINE_ASSERT_GROUP_SIZE(struct tcp_sock, tcp_sock_write_txrx, 91 + 4); /* RX read-write hotpath cache lines */ CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, bytes_received); @@ -5147,7 +5147,7 @@ static void __init tcp_struct_check(void) CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, bytes_acked); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, rcv_rtt_est); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, rcvq_space); - CACHELINE_ASSERT_GROUP_SIZE(struct tcp_sock, tcp_sock_write_rx, 99); + CACHELINE_ASSERT_GROUP_SIZE(struct tcp_sock, tcp_sock_write_rx, 96); } void __init tcp_init(void) From patchwork Wed May 14 13:56:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Chia-Yu Chang \(Nokia\)" X-Patchwork-Id: 890049 Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on2074.outbound.protection.outlook.com [40.107.104.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2235727B4FF; Wed, 14 May 2025 13:57:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.104.74 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231029; cv=fail; b=nz+iQZIqoJsc3DcGqCVj7j67aekjVyeMF9wiUYEQDEkxSN7UcOX9l39QloFepoBH1IhrvNqGfrDLL7b928vODOrqKX1VBLRFRDl7aVv/W+qH+Yo88rSwCRftVLHZ0u0sGfIE75ajel8NXqiu77a34/ZOb1aPfuX9exaIxeuulKE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231029; c=relaxed/simple; bh=qitPsQQeT20VB3iwN4hlffqF1BmzCYGbkM9sU8xH3c4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=AJ6fPTn+FmDminJmJ5pdOEpHdchnxwewa6qAYbzUggrWkwH5TDqaRDD3zmnQh2RWcRCpndFTWT39JPpRCS9vleKdL+KVWQgyKf12JURBGIpT32G1Xe7GaADseugXMyloEcrraS4rB8ACGIVOMxnflFAuMDwRfVxpQSFHMz3cMFI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com; spf=fail smtp.mailfrom=nokia-bell-labs.com; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b=qOkx790O; arc=fail smtp.client-ip=40.107.104.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b="qOkx790O" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=cOoOTsQ+fJ4LLBrmvZR/Yfwqs2zt5THe6SnScwlilut1/BTTPFG35PXneSNnGTYTUJe3cBovZ0+3lTbxGYO9XzAZrtq3ZUOP/+sKYSHxdQzpzbX+X54ldv1cJFlFpB94mVRpKJusCL33XFSmN6bnrlbFLs+4f8YS8juD2oB1ArpypHzQZNDApwLGqT15FAlVjp/3Ot/5mhNBFUPA0wlbUAnFIvnlT+CuwZ9E4nBMrpuPTVNYajo9DiIC0Wy1IvjrYeF4h4FOYJU1NgQpLVEy0tAzUEPzJabQZV7To48haolEn5fmqqU72Md3mVfG/ZL35cTfN6pjGyfvB2un1b9uOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Cq4W/qiqOubJXdbyF8jB58x3uBzfo5Cy9QLZ80KIoF4=; b=UOexvdEQpmx9BhvLxKGWoXOM0XHd6LirPofMhc0AU6gD81U4AD5vqt75um5/TagznxtaOsdSMOfOcuOe8iQZGnU9GEaEWvHhpQC0GCEsMmDz9tHZxHciNCEvM1ZFQ1msLYbJFyodbuO8NpE1uiBUFNDUu0O/tD+ptw31FsUAEbdZyQ0wtYaYHGxDcz3oalibw3er8RBxMTNldP8IBr9MV1faJXDqJ7EeJdiB5r700+dla/LHP9AABZguyDmE3V1UGSbOjcjh/B2yeKbgXYKXIzsoU/pHhhhbCA4UbH0JWgv4c/jC/Gg0dPWOPCe63UNciFSlzMpp3OfJeVZgnRxBig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 131.228.2.240) smtp.rcpttodomain=amazon.com smtp.mailfrom=nokia-bell-labs.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nokia-bell-labs.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia-bell-labs.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Cq4W/qiqOubJXdbyF8jB58x3uBzfo5Cy9QLZ80KIoF4=; b=qOkx790OiV7v05cPv7tGJe0QjyBw3AT6IPp2IH3i62cKkuR3PoN1vbC9jJ+5nazHNUJ3P/fSHXYM/X6b5uAHA+TCT3wJAywdxt09ISwwLiOXrIBUQ5f3Gu4BbpylL8SkL6/iDvvYyvX3h1a2SLtdKllw0IXrGvwkK78PE+48d8cwnO8q1M/zPFRBuGDd+l3x8oZzgoW/uA+BT4Cu+XEIMi9FTCwC/rh6qHU14qWAr8lyINaKVZp3WorSrHUXdzY+d/rGSg4HSgHDthd5viaNgD9t76LELqTLuZCaHNC680yTUpCV2sLBN+DVQvi1Jgz0jSnpFc2uhNFBzvhdkq/7dw== Received: from AM8P190CA0027.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:219::32) by AM7PR07MB6482.eurprd07.prod.outlook.com (2603:10a6:20b:1a5::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8722.31; Wed, 14 May 2025 13:56:58 +0000 Received: from AMS0EPF0000019F.eurprd05.prod.outlook.com (2603:10a6:20b:219:cafe::f5) by AM8P190CA0027.outlook.office365.com (2603:10a6:20b:219::32) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8722.30 via Frontend Transport; Wed, 14 May 2025 13:56:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 131.228.2.240) smtp.mailfrom=nokia-bell-labs.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nokia-bell-labs.com; Received-SPF: Pass (protection.outlook.com: domain of nokia-bell-labs.com designates 131.228.2.240 as permitted sender) receiver=protection.outlook.com; client-ip=131.228.2.240; helo=fihe3nok0735.emea.nsn-net.net; pr=C Received: from fihe3nok0735.emea.nsn-net.net (131.228.2.240) by AMS0EPF0000019F.mail.protection.outlook.com (10.167.16.251) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8722.18 via Frontend Transport; Wed, 14 May 2025 13:56:57 +0000 Received: from sarah.nbl.nsn-rdnet.net (sarah.nbl.nsn-rdnet.net [10.0.73.150]) by fihe3nok0735.emea.nsn-net.net (Postfix) with ESMTP id 3764720094; Wed, 14 May 2025 16:56:56 +0300 (EEST) From: chia-yu.chang@nokia-bell-labs.com To: linux-doc@vger.kernel.org, corbet@lwn.net, horms@kernel.org, dsahern@kernel.org, kuniyu@amazon.com, bpf@vger.kernel.org, netdev@vger.kernel.org, dave.taht@gmail.com, pabeni@redhat.com, jhs@mojatatu.com, kuba@kernel.org, stephen@networkplumber.org, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, andrew+netdev@lunn.ch, donald.hunter@gmail.com, ast@fiberby.net, liuhangbin@gmail.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, ij@kernel.org, ncardwell@google.com, koen.de_schepper@nokia-bell-labs.com, g.white@cablelabs.com, ingemar.s.johansson@ericsson.com, mirja.kuehlewind@ericsson.com, cheshire@apple.com, rs.ietf@gmx.at, Jason_Livingood@comcast.com, vidhi_goel@apple.com Cc: Olivier Tilmans , Chia-Yu Chang Subject: [PATCH v7 net-next 05/15] tcp: accecn: AccECN negotiation Date: Wed, 14 May 2025 15:56:32 +0200 Message-Id: <20250514135642.11203-6-chia-yu.chang@nokia-bell-labs.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> References: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AMS0EPF0000019F:EE_|AM7PR07MB6482:EE_ X-MS-Office365-Filtering-Correlation-Id: 2d1ca57a-f677-4a52-eb7a-08dd92ef3151 X-LD-Processed: 5d471751-9675-428d-917b-70f44f9630b0,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700013|1800799024|82310400026|376014|7416014|921020|13003099007; X-Microsoft-Antispam-Message-Info: =?utf-8?q?WuGCrMJWOYsMC1s7ywrqj3o+RP2T7L6?= =?utf-8?q?4I+8wHWesdXLjQxA721Fc+jJyU038yaqIx8lvZflNJiq1Us/HqBdJiGFkAx7H6G6/?= =?utf-8?q?dtVadH2IGOK4mV1M2HfEzzVb5Rr77/Er6rq65h3eKZ6n4bgH6uBAPhKysi94U4bVK?= =?utf-8?q?sSeRodwMp972dYcBd46ygWkZlHUrEgmJ6ySmXKVbE8PMd+W4OsWly1kXAetGtax8U?= =?utf-8?q?O4X0LU8gFiq38OvVPYNL/lqWXtooYBZ0+wFGUvRkrr5ELeQhfH+8qJllGbA2XwtN6?= =?utf-8?q?z+glyBNaTnseJvU02sQgiJdGUyZVODOV457JezW45NeJ8nw2wx1wo2KoNxsVwDH2+?= =?utf-8?q?ZWXc4I7MBO0FqZeKfAYdyPUOqKxWv+kehuUrUL8RGz7P4/iFA9aPZQqLNv4GJwj5V?= =?utf-8?q?vRlaot8yftOWTF6254Fg4NLPt2xpye/bHQeZ0OKM7zGspe6aOn53E36jvmMg7FYZI?= =?utf-8?q?hLNR/Eff23oWiNZq6rYfuKKKRgJaEPuUKsNP0CPkvhLdj8yAFKLtUnVYla4sp87PO?= =?utf-8?q?UouJWNUe7lj/lqZ+0dDkdVW0QZ0F0uzLKNhujFSo+iu/tjvhtctzWLzy7cWZUCsQ7?= =?utf-8?q?t+dCK0S9W1TEiCP15L/keqGI5oTtWcJsKLEWfeklZzPusy3DSicW1CGT/3KtPLkRf?= =?utf-8?q?G/w7DW3Pni+7HKx2WsnJHZsLR5dE2EWRO/RIrudQdakDgyL09VgC51alulo2VOgwS?= =?utf-8?q?qqnf+LEe7AddkWbn6qHopbt+5LIWAhv48X8qrbRhxH0jHPpEuWKyGc1uljVv3QDMO?= =?utf-8?q?ynDy/GP9JoUUEiuJFA2ByfnBQa7eh3If/+d85a9hdE2LaIosxUiyu2su5RBgbZGEN?= =?utf-8?q?ZB7X5e+W5l7S6xghKZIsKupKpHHOBoj5co/yiz37nVNKYfm9LF7N4slOTFNe1miDN?= =?utf-8?q?pdIeU3ovAQg7Wyp42P+sdvQFgc2HqznyNY3nsvij98dPBfUt+UMs1em23Qu5FlAOa?= =?utf-8?q?bf6664pceAI1zMUOJsOS3pfyklgAV28TRWdKJte9wWuLAEl+alGF4C3JiK2qqtUtz?= =?utf-8?q?gHWm9nIT8Q4sCGXoJPfSo0wD9YFqX4PjSxe7o+I9N5CbL4e921tkq6n/PyEuGcMYW?= =?utf-8?q?C+B1bKst088sOcl1H/SWyGAUNXw8OHxELx1P99U2ZX9d4eaDQ0vsrdA5gf6VVOZjc?= =?utf-8?q?zgtA3wWrkG6JgBWxdTQumVCwwn/YLZQUtyJHPMXuYKp8HgsRubGmeBHMw41GiSNdb?= =?utf-8?q?ttUCt3Fc6T5tb6pyRviObvEQGYAC/KtOE6qgrEtloYb4s+SuBMie5uPVUa1YgXeCq?= =?utf-8?q?mToIvdxCJ5PAbgnjvu5NxqcieV8/uhO4Lubw5zzWZ3vhybEQmcMUXkJHkLh2hZdod?= =?utf-8?q?mpN8o/xUj1BrcV59s6sN5YMKURRJ/MyV0voX6aK8Sx5Gv6XSy/BVNmLESbnXoCQPA?= =?utf-8?q?jo/JI9gU/lTihs5quZUbpHyY13M564kQ+GWuFoxLyCWUf3Vb4dnnU0=3D?= X-Forefront-Antispam-Report: CIP:131.228.2.240; CTRY:FI; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:fihe3nok0735.emea.nsn-net.net; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(36860700013)(1800799024)(82310400026)(376014)(7416014)(921020)(13003099007); DIR:OUT; SFP:1101; X-OriginatorOrg: nokia-bell-labs.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2025 13:56:57.7840 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2d1ca57a-f677-4a52-eb7a-08dd92ef3151 X-MS-Exchange-CrossTenant-Id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5d471751-9675-428d-917b-70f44f9630b0; Ip=[131.228.2.240]; Helo=[fihe3nok0735.emea.nsn-net.net] X-MS-Exchange-CrossTenant-AuthSource: AMS0EPF0000019F.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR07MB6482 From: Ilpo Järvinen Accurate ECN negotiation parts based on the specification: https://tools.ietf.org/id/draft-ietf-tcpm-accurate-ecn-28.txt Accurate ECN is negotiated using ECE, CWR and AE flags in the TCP header. TCP falls back into using RFC3168 ECN if one of the ends supports only RFC3168-style ECN. The AccECN negotiation includes reflecting IP ECN field value seen in SYN and SYNACK back using the same bits as negotiation to allow responding to SYN CE marks and to detect ECN field mangling. CE marks should not occur currently because SYN=1 segments are sent with Non-ECT in IP ECN field (but proposal exists to remove this restriction). Reflecting SYN IP ECN field in SYNACK is relatively simple. Reflecting SYNACK IP ECN field in the final/third ACK of the handshake is more challenging. Linux TCP code is not well prepared for using the final/third ACK a signalling channel which makes things somewhat complicated here. tcp_ecn sysctl can be used to select the highest ECN variant (Accurate ECN, ECN, No ECN) that is attemped to be negotiated and requested for incoming connection and outgoing connection: TCP_ECN_IN_NOECN_OUT_NOECN, TCP_ECN_IN_ECN_OUT_ECN, TCP_ECN_IN_ECN_OUT_NOECN, TCP_ECN_IN_ACCECN_OUT_ACCECN, TCP_ECN_IN_ACCECN_OUT_ECN, and TCP_ECN_IN_ACCECN_OUT_NOECN. After this patch, the size of tcp_request_sock remains unchanged and no new holes are added. Below are the pahole outcomes before and after this patch: [BEFORE THIS PATCH] struct tcp_request_sock { [...] u32 rcv_nxt; /* 352 4 */ u8 syn_tos; /* 356 1 */ /* size: 360, cachelines: 6, members: 16 */ } [AFTER THIS PATCH] struct tcp_request_sock { [...] u32 rcv_nxt; /* 352 4 */ u8 syn_tos; /* 356 1 */ bool accecn_ok; /* 357 1 */ u8 syn_ect_snt:2; /* 358: 0 1 */ u8 syn_ect_rcv:2; /* 358: 2 1 */ u8 accecn_fail_mode:4; /* 358: 4 1 */ /* size: 360, cachelines: 6, members: 20 */ } After this patch, the size of tcp_sock remains unchanged and no new holes are added. Also, 4 bits of the existing 2-byte hole are exploited. Below are the pahole outcomes before and after this patch: [BEFORE THIS PATCH] struct tcp_sock { [...] u8 dup_ack_counter:2; /* 2825: 0 1 */ u8 tlp_retrans:1; /* 2825: 2 1 */ u8 unused:5; /* 2825: 3 1 */ u8 thin_lto:1; /* 2826: 0 1 */ u8 fastopen_connect:1; /* 2826: 1 1 */ u8 fastopen_no_cookie:1; /* 2826: 2 1 */ u8 fastopen_client_fail:2; /* 2826: 3 1 */ u8 frto:1; /* 2826: 5 1 */ /* XXX 2 bits hole, try to pack */ [...] u8 keepalive_probes; /* 2829 1 */ /* XXX 2 bytes hole, try to pack */ [...] /* size: 3264, cachelines: 51, members: 166 */ } [AFTER THIS PATCH] struct tcp_sock { [...] u8 dup_ack_counter:2; /* 2825: 0 1 */ u8 tlp_retrans:1; /* 2825: 2 1 */ u8 syn_ect_snt:2; /* 2825: 3 1 */ u8 syn_ect_rcv:2; /* 2825: 5 1 */ u8 wait_third_ack:1; /* 2825: 7 1 */ u8 thin_lto:1; /* 2826: 0 1 */ u8 fastopen_connect:1; /* 2826: 1 1 */ u8 fastopen_no_cookie:1; /* 2826: 2 1 */ u8 fastopen_client_fail:2; /* 2826: 3 1 */ u8 frto:1; /* 2826: 5 1 */ /* XXX 2 bits hole, try to pack */ [...] u8 keepalive_probes; /* 2829 1 */ u8 accecn_fail_mode:4; /* 2830: 0 1 */ /* XXX 4 bits hole, try to pack */ /* XXX 1 byte hole, try to pack */ [...] /* size: 3264, cachelines: 51, members: 169 */ } Signed-off-by: Ilpo Järvinen Co-developed-by: Olivier Tilmans Signed-off-by: Olivier Tilmans Co-developed-by: Chia-Yu Chang Signed-off-by: Chia-Yu Chang --- .../networking/net_cachelines/tcp_sock.rst | 4 + include/linux/tcp.h | 9 +- include/net/tcp.h | 93 ++++++++- net/ipv4/syncookies.c | 3 + net/ipv4/sysctl_net_ipv4.c | 3 +- net/ipv4/tcp.c | 2 + net/ipv4/tcp_input.c | 178 ++++++++++++++++-- net/ipv4/tcp_ipv4.c | 5 +- net/ipv4/tcp_minisocks.c | 51 ++++- net/ipv4/tcp_output.c | 78 ++++++-- net/ipv6/syncookies.c | 1 + net/ipv6/tcp_ipv6.c | 1 + 12 files changed, 384 insertions(+), 44 deletions(-) diff --git a/Documentation/networking/net_cachelines/tcp_sock.rst b/Documentation/networking/net_cachelines/tcp_sock.rst index 9bf507c349a0..eef8700085de 100644 --- a/Documentation/networking/net_cachelines/tcp_sock.rst +++ b/Documentation/networking/net_cachelines/tcp_sock.rst @@ -103,6 +103,10 @@ u32 delivered read_mostly read_w u32 delivered_ce read_mostly read_write tcp_rate_skb_sent(tx);tcp_rate_gen(rx) u32 received_ce read_mostly read_write u8:4 received_ce_pending read_mostly read_write +u8:2 syn_ect_snt write_mostly read_write +u8:2 syn_ect_rcv read_mostly read_write +u8:1 wait_third_ack read_write +u8:4 accecn_fail_mode u32 lost read_mostly tcp_ack u32 app_limited read_write read_mostly tcp_rate_check_app_limited,tcp_rate_skb_sent(tx);tcp_rate_gen(rx) u64 first_tx_mstamp read_write tcp_rate_skb_sent diff --git a/include/linux/tcp.h b/include/linux/tcp.h index e1884df45a27..b0e6a11a3fa1 100644 --- a/include/linux/tcp.h +++ b/include/linux/tcp.h @@ -168,6 +168,10 @@ struct tcp_request_sock { * after data-in-SYN. */ u8 syn_tos; + bool accecn_ok; + u8 syn_ect_snt: 2, + syn_ect_rcv: 2, + accecn_fail_mode:4; #ifdef CONFIG_TCP_AO u8 ao_keyid; u8 ao_rcv_next; @@ -376,7 +380,9 @@ struct tcp_sock { u8 compressed_ack; u8 dup_ack_counter:2, tlp_retrans:1, /* TLP is a retransmission */ - unused:5; + syn_ect_snt:2, /* AccECN ECT memory, only */ + syn_ect_rcv:2, /* ... needed durign 3WHS + first seqno */ + wait_third_ack:1; /* Wait 3rd ACK in simultaneous open */ u8 thin_lto : 1,/* Use linear timeouts for thin streams */ fastopen_connect:1, /* FASTOPEN_CONNECT sockopt */ fastopen_no_cookie:1, /* Allow send/recv SYN+data without a cookie */ @@ -392,6 +398,7 @@ struct tcp_sock { syn_fastopen_child:1; /* created TFO passive child socket */ u8 keepalive_probes; /* num of allowed keep alive probes */ + u8 accecn_fail_mode:4; /* AccECN failure handling */ u32 tcp_tx_delay; /* delay (in usec) added to TX packets */ /* RTT measurement */ diff --git a/include/net/tcp.h b/include/net/tcp.h index cc28255deef7..5fce4b49e132 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -27,6 +27,7 @@ #include #include #include +#include #include #include @@ -234,6 +235,37 @@ static_assert((1 << ATO_BITS) > TCP_DELACK_MAX); #define TCPOLEN_MSS_ALIGNED 4 #define TCPOLEN_EXP_SMC_BASE_ALIGNED 8 +/* tp->accecn_fail_mode */ +#define TCP_ACCECN_ACE_FAIL_SEND BIT(0) +#define TCP_ACCECN_ACE_FAIL_RECV BIT(1) +#define TCP_ACCECN_OPT_FAIL_SEND BIT(2) +#define TCP_ACCECN_OPT_FAIL_RECV BIT(3) + +static inline bool tcp_accecn_ace_fail_send(const struct tcp_sock *tp) +{ + return tp->accecn_fail_mode & TCP_ACCECN_ACE_FAIL_SEND; +} + +static inline bool tcp_accecn_ace_fail_recv(const struct tcp_sock *tp) +{ + return tp->accecn_fail_mode & TCP_ACCECN_ACE_FAIL_RECV; +} + +static inline bool tcp_accecn_opt_fail_send(const struct tcp_sock *tp) +{ + return tp->accecn_fail_mode & TCP_ACCECN_OPT_FAIL_SEND; +} + +static inline bool tcp_accecn_opt_fail_recv(const struct tcp_sock *tp) +{ + return tp->accecn_fail_mode & TCP_ACCECN_OPT_FAIL_RECV; +} + +static inline void tcp_accecn_fail_mode_set(struct tcp_sock *tp, u8 mode) +{ + tp->accecn_fail_mode |= mode; +} + /* Flags in tp->nonagle */ #define TCP_NAGLE_OFF 1 /* Nagle's algo is disabled */ #define TCP_NAGLE_CORK 2 /* Socket is corked */ @@ -420,6 +452,23 @@ static inline u8 tcp_accecn_ace(const struct tcphdr *th) return (th->ae << 2) | (th->cwr << 1) | th->ece; } +/* Infer the ECT value our SYN arrived with from the echoed ACE field */ +static inline int tcp_accecn_extract_syn_ect(u8 ace) +{ + if (ace & 0x1) + return INET_ECN_ECT_1; + if (!(ace & 0x2)) + return INET_ECN_ECT_0; + if (ace & 0x4) + return INET_ECN_CE; + return INET_ECN_NOT_ECT; +} + +bool tcp_accecn_validate_syn_feedback(struct sock *sk, u8 ace, u8 sent_ect); +void tcp_accecn_third_ack(struct sock *sk, const struct sk_buff *skb, + u8 syn_ect_snt); +void tcp_ecn_received_counters(struct sock *sk, const struct sk_buff *skb); + enum tcp_tw_status { TCP_TW_SUCCESS = 0, TCP_TW_RST = 1, @@ -657,6 +706,15 @@ static inline bool cookie_ecn_ok(const struct net *net, const struct dst_entry * dst_feature(dst, RTAX_FEATURE_ECN); } +/* AccECN specification, 5.1: [...] a server can determine that it + * negotiated AccECN as [...] if the ACK contains an ACE field with + * the value 0b010 to 0b111 (decimal 2 to 7). + */ +static inline bool cookie_accecn_ok(const struct tcphdr *th) +{ + return tcp_accecn_ace(th) > 0x1; +} + #if IS_ENABLED(CONFIG_BPF) static inline bool cookie_bpf_ok(struct sk_buff *skb) { @@ -968,6 +1026,7 @@ static inline u32 tcp_rsk_tsval(const struct tcp_request_sock *treq) #define TCPHDR_ACE (TCPHDR_ECE | TCPHDR_CWR | TCPHDR_AE) #define TCPHDR_SYN_ECN (TCPHDR_SYN | TCPHDR_ECE | TCPHDR_CWR) +#define TCPHDR_SYNACK_ACCECN (TCPHDR_SYN | TCPHDR_ACK | TCPHDR_CWR) #define TCP_ACCECN_CEP_ACE_MASK 0x7 #define TCP_ACCECN_ACE_MAX_DELTA 6 @@ -977,6 +1036,19 @@ static inline u32 tcp_rsk_tsval(const struct tcp_request_sock *treq) */ #define TCP_ACCECN_CEP_INIT_OFFSET 5 +/* The highest ECN variant (Accurate ECN, ECN, or no ECN) that is + * attemped to be negotiated and requested for incoming connection + * and outgoing connection, respectively. + */ +enum tcp_ecn_mode { + TCP_ECN_IN_NOECN_OUT_NOECN = 0, + TCP_ECN_IN_ECN_OUT_ECN = 1, + TCP_ECN_IN_ECN_OUT_NOECN = 2, + TCP_ECN_IN_ACCECN_OUT_ACCECN = 3, + TCP_ECN_IN_ACCECN_OUT_ECN = 4, + TCP_ECN_IN_ACCECN_OUT_NOECN = 5, +}; + static inline void tcp_accecn_init_counters(struct tcp_sock *tp) { tp->received_ce = 0; @@ -1051,6 +1123,15 @@ struct tcp_skb_cb { #define TCP_SKB_CB(__skb) ((struct tcp_skb_cb *)&((__skb)->cb[0])) +static inline u16 tcp_accecn_reflector_flags(u8 ect) +{ + u32 flags = ect + 2; + + if (ect == 3) + flags++; + return FIELD_PREP(TCPHDR_ACE, flags); +} + extern const struct inet_connection_sock_af_ops ipv4_specific; #if IS_ENABLED(CONFIG_IPV6) @@ -1173,7 +1254,10 @@ enum tcp_ca_ack_event_flags { #define TCP_CONG_NON_RESTRICTED BIT(0) /* Requires ECN/ECT set on all packets */ #define TCP_CONG_NEEDS_ECN BIT(1) -#define TCP_CONG_MASK (TCP_CONG_NON_RESTRICTED | TCP_CONG_NEEDS_ECN) +/* Require successfully negotiated AccECN capability */ +#define TCP_CONG_NEEDS_ACCECN BIT(2) +#define TCP_CONG_MASK (TCP_CONG_NON_RESTRICTED | TCP_CONG_NEEDS_ECN | \ + TCP_CONG_NEEDS_ACCECN) union tcp_cc_info; @@ -1305,6 +1389,13 @@ static inline bool tcp_ca_needs_ecn(const struct sock *sk) return icsk->icsk_ca_ops->flags & TCP_CONG_NEEDS_ECN; } +static inline bool tcp_ca_needs_accecn(const struct sock *sk) +{ + const struct inet_connection_sock *icsk = inet_csk(sk); + + return icsk->icsk_ca_ops->flags & TCP_CONG_NEEDS_ACCECN; +} + static inline void tcp_ca_event(struct sock *sk, const enum tcp_ca_event event) { const struct inet_connection_sock *icsk = inet_csk(sk); diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c index 5459a78b9809..3a44eb9c1d1a 100644 --- a/net/ipv4/syncookies.c +++ b/net/ipv4/syncookies.c @@ -403,6 +403,7 @@ struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb) struct tcp_sock *tp = tcp_sk(sk); struct inet_request_sock *ireq; struct net *net = sock_net(sk); + struct tcp_request_sock *treq; struct request_sock *req; struct sock *ret = sk; struct flowi4 fl4; @@ -428,6 +429,7 @@ struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb) } ireq = inet_rsk(req); + treq = tcp_rsk(req); sk_rcv_saddr_set(req_to_sk(req), ip_hdr(skb)->daddr); sk_daddr_set(req_to_sk(req), ip_hdr(skb)->saddr); @@ -482,6 +484,7 @@ struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb) if (!req->syncookie) ireq->rcv_wscale = rcv_wscale; ireq->ecn_ok &= cookie_ecn_ok(net, &rt->dst); + treq->accecn_ok = ireq->ecn_ok && cookie_accecn_ok(th); ret = tcp_get_cookie_sock(sk, skb, req, &rt->dst); /* ip_queue_xmit() depends on our flow being setup diff --git a/net/ipv4/sysctl_net_ipv4.c b/net/ipv4/sysctl_net_ipv4.c index 3a43010d726f..75ec1a599b52 100644 --- a/net/ipv4/sysctl_net_ipv4.c +++ b/net/ipv4/sysctl_net_ipv4.c @@ -47,6 +47,7 @@ static unsigned int udp_child_hash_entries_max = UDP_HTABLE_SIZE_MAX; static int tcp_plb_max_rounds = 31; static int tcp_plb_max_cong_thresh = 256; static unsigned int tcp_tw_reuse_delay_max = TCP_PAWS_MSL * MSEC_PER_SEC; +static int tcp_ecn_mode_max = 5; /* obsolete */ static int sysctl_tcp_low_latency __read_mostly; @@ -728,7 +729,7 @@ static struct ctl_table ipv4_net_table[] = { .mode = 0644, .proc_handler = proc_dou8vec_minmax, .extra1 = SYSCTL_ZERO, - .extra2 = SYSCTL_TWO, + .extra2 = &tcp_ecn_mode_max, }, { .procname = "tcp_ecn_fallback", diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 0c8a391d90fe..29146bb3585a 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -3396,6 +3396,8 @@ int tcp_disconnect(struct sock *sk, int flags) tp->window_clamp = 0; tp->delivered = 0; tp->delivered_ce = 0; + tp->wait_third_ack = 0; + tp->accecn_fail_mode = 0; tcp_accecn_init_counters(tp); if (icsk->icsk_ca_initialized && icsk->icsk_ca_ops->release) icsk->icsk_ca_ops->release(sk); diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index 8dbb625f5e8a..f0475f95e112 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -401,14 +401,114 @@ static void tcp_data_ecn_check(struct sock *sk, const struct sk_buff *skb) } } -static void tcp_ecn_rcv_synack(struct tcp_sock *tp, const struct tcphdr *th) +/* AccECN specificaiton, 3.1.2: If a TCP server that implements AccECN + * receives a SYN with the three TCP header flags (AE, CWR and ECE) set + * to any combination other than 000, 011 or 111, it MUST negotiate the + * use of AccECN as if they had been set to 111. + */ +static bool tcp_accecn_syn_requested(const struct tcphdr *th) +{ + u8 ace = tcp_accecn_ace(th); + + return ace && ace != 0x3; +} + +/* Check ECN field transition to detect invalid transitions */ +static bool tcp_ect_transition_valid(u8 snt, u8 rcv) { - if (tcp_ecn_mode_rfc3168(tp) && (!th->ece || th->cwr)) + if (rcv == snt) + return true; + + /* Non-ECT altered to something or something became non-ECT */ + if (snt == INET_ECN_NOT_ECT || rcv == INET_ECN_NOT_ECT) + return false; + /* CE -> ECT(0/1)? */ + if (snt == INET_ECN_CE) + return false; + return true; +} + +bool tcp_accecn_validate_syn_feedback(struct sock *sk, u8 ace, u8 sent_ect) +{ + u8 ect = tcp_accecn_extract_syn_ect(ace); + struct tcp_sock *tp = tcp_sk(sk); + + if (!sock_net(sk)->ipv4.sysctl_tcp_ecn_fallback) + return true; + + if (!tcp_ect_transition_valid(sent_ect, ect)) { + tcp_accecn_fail_mode_set(tp, TCP_ACCECN_ACE_FAIL_RECV); + return false; + } + + return true; +} + +/* See Table 2 of the AccECN draft */ + +static void tcp_ecn_rcv_synack(struct sock *sk, const struct tcphdr *th, + u8 ip_dsfield) +{ + struct tcp_sock *tp = tcp_sk(sk); + u8 ace = tcp_accecn_ace(th); + + switch (ace) { + case 0x0: + case 0x7: + /* +========+========+============+=============+ + * | A | B | SYN/ACK | Feedback | + * | | | B->A | Mode of A | + * | | | AE CWR ECE | | + * +========+========+============+=============+ + * | AccECN | No ECN | 0 0 0 | Not ECN | + * | AccECN | Broken | 1 1 1 | Not ECN | + * +--------+--------+------------+-------------+ + */ tcp_ecn_mode_set(tp, TCP_ECN_DISABLED); + break; + case 0x1: + case 0x5: + /* +========+========+============+=============+ + * | A | B | SYN/ACK | Feedback | + * | | | B->A | Mode of A | + * | | | AE CWR ECE | | + * +========+========+============+=============+ + * | AccECN | Nonce | 1 0 1 | (Reserved) | + * | AccECN | ECN | 0 0 1 | Classic ECN | + * | Nonce | AccECN | 0 0 1 | Classic ECN | + * | ECN | AccECN | 0 0 1 | Classic ECN | + * +--------+--------+------------+-------------+ + */ + if (tcp_ecn_mode_pending(tp)) + /* Downgrade from AccECN, or requested initially */ + tcp_ecn_mode_set(tp, TCP_ECN_MODE_RFC3168); + break; + default: + tcp_ecn_mode_set(tp, TCP_ECN_MODE_ACCECN); + tp->syn_ect_rcv = ip_dsfield & INET_ECN_MASK; + if (INET_ECN_is_ce(ip_dsfield) && + tcp_accecn_validate_syn_feedback(sk, ace, + tp->syn_ect_snt)) { + tp->received_ce++; + tp->received_ce_pending++; + } + break; + } } -static void tcp_ecn_rcv_syn(struct tcp_sock *tp, const struct tcphdr *th) +static void tcp_ecn_rcv_syn(struct tcp_sock *tp, const struct tcphdr *th, + const struct sk_buff *skb) { + if (tcp_ecn_mode_pending(tp)) { + if (!tcp_accecn_syn_requested(th)) { + /* Downgrade to classic ECN feedback */ + tcp_ecn_mode_set(tp, TCP_ECN_MODE_RFC3168); + } else { + tp->syn_ect_rcv = TCP_SKB_CB(skb)->ip_dsfield & + INET_ECN_MASK; + tcp_ecn_mode_set(tp, TCP_ECN_MODE_ACCECN); + } + } if (tcp_ecn_mode_rfc3168(tp) && (!th->ece || !th->cwr)) tcp_ecn_mode_set(tp, TCP_ECN_DISABLED); } @@ -3833,8 +3933,18 @@ bool tcp_oow_rate_limited(struct net *net, const struct sk_buff *skb, return __tcp_oow_rate_limited(net, mib_idx, last_oow_ack_time); } +static void tcp_send_ack_reflect_ect(struct sock *sk, bool accecn_reflector) +{ + struct tcp_sock *tp = tcp_sk(sk); + u16 flags = 0; + + if (accecn_reflector) + flags = tcp_accecn_reflector_flags(tp->syn_ect_rcv); + __tcp_send_ack(sk, tp->rcv_nxt, flags); +} + /* RFC 5961 7 [ACK Throttling] */ -static void tcp_send_challenge_ack(struct sock *sk) +static void tcp_send_challenge_ack(struct sock *sk, bool accecn_reflector) { struct tcp_sock *tp = tcp_sk(sk); struct net *net = sock_net(sk); @@ -3864,7 +3974,7 @@ static void tcp_send_challenge_ack(struct sock *sk) WRITE_ONCE(net->ipv4.tcp_challenge_count, count - 1); send_ack: NET_INC_STATS(net, LINUX_MIB_TCPCHALLENGEACK); - tcp_send_ack(sk); + tcp_send_ack_reflect_ect(sk, accecn_reflector); } } @@ -4031,7 +4141,7 @@ static int tcp_ack(struct sock *sk, const struct sk_buff *skb, int flag) /* RFC 5961 5.2 [Blind Data Injection Attack].[Mitigation] */ if (before(ack, prior_snd_una - max_window)) { if (!(flag & FLAG_NO_CHALLENGE_ACK)) - tcp_send_challenge_ack(sk); + tcp_send_challenge_ack(sk, false); return -SKB_DROP_REASON_TCP_TOO_OLD_ACK; } goto old_ack; @@ -6025,8 +6135,7 @@ static void tcp_urg(struct sock *sk, struct sk_buff *skb, const struct tcphdr *t } /* Updates Accurate ECN received counters from the received IP ECN field */ -static void tcp_ecn_received_counters(struct sock *sk, - const struct sk_buff *skb) +void tcp_ecn_received_counters(struct sock *sk, const struct sk_buff *skb) { u8 ecnfield = TCP_SKB_CB(skb)->ip_dsfield & INET_ECN_MASK; u8 is_ce = INET_ECN_is_ce(ecnfield); @@ -6068,6 +6177,7 @@ static bool tcp_validate_incoming(struct sock *sk, struct sk_buff *skb, const struct tcphdr *th, int syn_inerr) { struct tcp_sock *tp = tcp_sk(sk); + bool accecn_reflector = false; SKB_DR(reason); /* RFC1323: H1. Apply PAWS check first. */ @@ -6160,7 +6270,7 @@ static bool tcp_validate_incoming(struct sock *sk, struct sk_buff *skb, if (tp->syn_fastopen && !tp->data_segs_in && sk->sk_state == TCP_ESTABLISHED) tcp_fastopen_active_disable(sk); - tcp_send_challenge_ack(sk); + tcp_send_challenge_ack(sk, false); SKB_DR_SET(reason, TCP_RESET); goto discard; } @@ -6171,16 +6281,23 @@ static bool tcp_validate_incoming(struct sock *sk, struct sk_buff *skb, * RFC 5961 4.2 : Send a challenge ack */ if (th->syn) { + if (tcp_ecn_mode_accecn(tp)) + accecn_reflector = true; if (sk->sk_state == TCP_SYN_RECV && sk->sk_socket && th->ack && TCP_SKB_CB(skb)->seq + 1 == TCP_SKB_CB(skb)->end_seq && TCP_SKB_CB(skb)->seq + 1 == tp->rcv_nxt && - TCP_SKB_CB(skb)->ack_seq == tp->snd_nxt) + TCP_SKB_CB(skb)->ack_seq == tp->snd_nxt) { + if (!tcp_ecn_disabled(tp)) { + tp->wait_third_ack = true; + tcp_send_ack_reflect_ect(sk, accecn_reflector); + } goto pass; + } syn_challenge: if (syn_inerr) TCP_INC_STATS(sock_net(sk), TCP_MIB_INERRS); NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPSYNCHALLENGE); - tcp_send_challenge_ack(sk); + tcp_send_challenge_ack(sk, accecn_reflector); SKB_DR_SET(reason, TCP_INVALID_SYN); goto discard; } @@ -6393,6 +6510,12 @@ void tcp_rcv_established(struct sock *sk, struct sk_buff *skb) return; step5: + if (unlikely(tp->wait_third_ack)) { + tp->wait_third_ack = 0; + if (tcp_ecn_mode_accecn(tp)) + tcp_accecn_third_ack(sk, skb, tp->syn_ect_snt); + tcp_fast_path_on(tp); + } tcp_ecn_received_counters(sk, skb); reason = tcp_ack(sk, skb, FLAG_SLOWPATH | FLAG_UPDATE_TS_RECENT); @@ -6645,7 +6768,8 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb, * state to ESTABLISHED..." */ - tcp_ecn_rcv_synack(tp, th); + if (tcp_ecn_mode_any(tp)) + tcp_ecn_rcv_synack(sk, th, TCP_SKB_CB(skb)->ip_dsfield); tcp_init_wl(tp, TCP_SKB_CB(skb)->seq); tcp_try_undo_spurious_syn(sk); @@ -6717,7 +6841,7 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb, TCP_DELACK_MAX, false); goto consume; } - tcp_send_ack(sk); + tcp_send_ack_reflect_ect(sk, tcp_ecn_mode_accecn(tp)); return -1; } @@ -6776,7 +6900,7 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb, tp->snd_wl1 = TCP_SKB_CB(skb)->seq; tp->max_window = tp->snd_wnd; - tcp_ecn_rcv_syn(tp, th); + tcp_ecn_rcv_syn(tp, th, skb); tcp_mtup_init(sk); tcp_sync_mss(sk, icsk->icsk_pmtu_cookie); @@ -6958,7 +7082,7 @@ tcp_rcv_state_process(struct sock *sk, struct sk_buff *skb) } /* accept old ack during closing */ if ((int)reason < 0) { - tcp_send_challenge_ack(sk); + tcp_send_challenge_ack(sk, false); reason = -reason; goto discard; } @@ -7005,9 +7129,16 @@ tcp_rcv_state_process(struct sock *sk, struct sk_buff *skb) tp->lsndtime = tcp_jiffies32; tcp_initialize_rcv_mss(sk); - tcp_fast_path_on(tp); + if (likely(!tp->wait_third_ack)) { + if (tcp_ecn_mode_accecn(tp)) + tcp_accecn_third_ack(sk, skb, tp->syn_ect_snt); + tcp_fast_path_on(tp); + } if (sk->sk_shutdown & SEND_SHUTDOWN) tcp_shutdown(sk, SEND_SHUTDOWN); + + if (sk->sk_socket && tp->wait_third_ack) + goto consume; break; case TCP_FIN_WAIT1: { @@ -7177,6 +7308,15 @@ static void tcp_ecn_create_request(struct request_sock *req, bool ect, ecn_ok; u32 ecn_ok_dst; + if (tcp_accecn_syn_requested(th) && + (net->ipv4.sysctl_tcp_ecn >= 3 || tcp_ca_needs_accecn(listen_sk))) { + inet_rsk(req)->ecn_ok = 1; + tcp_rsk(req)->accecn_ok = 1; + tcp_rsk(req)->syn_ect_rcv = TCP_SKB_CB(skb)->ip_dsfield & + INET_ECN_MASK; + return; + } + if (!th_ecn) return; @@ -7184,7 +7324,8 @@ static void tcp_ecn_create_request(struct request_sock *req, ecn_ok_dst = dst_feature(dst, DST_FEATURE_ECN_MASK); ecn_ok = READ_ONCE(net->ipv4.sysctl_tcp_ecn) || ecn_ok_dst; - if (((!ect || th->res1) && ecn_ok) || tcp_ca_needs_ecn(listen_sk) || + if (((!ect || th->res1 || th->ae) && ecn_ok) || + tcp_ca_needs_ecn(listen_sk) || (ecn_ok_dst & DST_FEATURE_ECN_CA) || tcp_bpf_ca_needs_ecn((struct sock *)req)) inet_rsk(req)->ecn_ok = 1; @@ -7202,6 +7343,9 @@ static void tcp_openreq_init(struct request_sock *req, tcp_rsk(req)->snt_synack = 0; tcp_rsk(req)->snt_tsval_first = 0; tcp_rsk(req)->last_oow_ack_time = 0; + tcp_rsk(req)->accecn_ok = 0; + tcp_rsk(req)->syn_ect_rcv = 0; + tcp_rsk(req)->syn_ect_snt = 0; req->mss = rx_opt->mss_clamp; req->ts_recent = rx_opt->saw_tstamp ? rx_opt->rcv_tsval : 0; ireq->tstamp_ok = rx_opt->tstamp_ok; diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index d5b5c32115d2..5cec04ed6a8e 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -1189,7 +1189,7 @@ static int tcp_v4_send_synack(const struct sock *sk, struct dst_entry *dst, enum tcp_synack_type synack_type, struct sk_buff *syn_skb) { - const struct inet_request_sock *ireq = inet_rsk(req); + struct inet_request_sock *ireq = inet_rsk(req); struct flowi4 fl4; int err = -1; struct sk_buff *skb; @@ -1202,6 +1202,7 @@ static int tcp_v4_send_synack(const struct sock *sk, struct dst_entry *dst, skb = tcp_make_synack(sk, dst, req, foc, synack_type, syn_skb); if (skb) { + tcp_rsk(req)->syn_ect_snt = inet_sk(sk)->tos & INET_ECN_MASK; __tcp_v4_send_check(skb, ireq->ir_loc_addr, ireq->ir_rmt_addr); tos = READ_ONCE(inet_sk(sk)->tos); @@ -3448,7 +3449,7 @@ static void __net_init tcp_set_hashinfo(struct net *net) static int __net_init tcp_sk_init(struct net *net) { - net->ipv4.sysctl_tcp_ecn = 2; + net->ipv4.sysctl_tcp_ecn = TCP_ECN_IN_ECN_OUT_NOECN; net->ipv4.sysctl_tcp_ecn_fallback = 1; net->ipv4.sysctl_tcp_base_mss = TCP_BASE_MSS; diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c index 43d7852ce07e..779a206a5ca6 100644 --- a/net/ipv4/tcp_minisocks.c +++ b/net/ipv4/tcp_minisocks.c @@ -461,12 +461,51 @@ void tcp_openreq_init_rwin(struct request_sock *req, ireq->rcv_wscale = rcv_wscale; } -static void tcp_ecn_openreq_child(struct tcp_sock *tp, - const struct request_sock *req) +void tcp_accecn_third_ack(struct sock *sk, const struct sk_buff *skb, + u8 syn_ect_snt) { - tcp_ecn_mode_set(tp, inet_rsk(req)->ecn_ok ? - TCP_ECN_MODE_RFC3168 : - TCP_ECN_DISABLED); + u8 ace = tcp_accecn_ace(tcp_hdr(skb)); + struct tcp_sock *tp = tcp_sk(sk); + + switch (ace) { + case 0x0: + tcp_accecn_fail_mode_set(tp, TCP_ACCECN_ACE_FAIL_RECV); + break; + case 0x7: + case 0x5: + case 0x1: + /* Unused but legal values */ + break; + default: + /* Validation only applies to first non-data packet */ + if (TCP_SKB_CB(skb)->seq == TCP_SKB_CB(skb)->end_seq && + !TCP_SKB_CB(skb)->sacked && + tcp_accecn_validate_syn_feedback(sk, ace, syn_ect_snt)) { + if ((tcp_accecn_extract_syn_ect(ace) == INET_ECN_CE) && + !tp->delivered_ce) + tp->delivered_ce++; + } + break; + } +} + +static void tcp_ecn_openreq_child(struct sock *sk, + const struct request_sock *req, + const struct sk_buff *skb) +{ + const struct tcp_request_sock *treq = tcp_rsk(req); + struct tcp_sock *tp = tcp_sk(sk); + + if (treq->accecn_ok) { + tcp_ecn_mode_set(tp, TCP_ECN_MODE_ACCECN); + tp->syn_ect_snt = treq->syn_ect_snt; + tcp_accecn_third_ack(sk, skb, treq->syn_ect_snt); + tcp_ecn_received_counters(sk, skb); + } else { + tcp_ecn_mode_set(tp, inet_rsk(req)->ecn_ok ? + TCP_ECN_MODE_RFC3168 : + TCP_ECN_DISABLED); + } } void tcp_ca_openreq_child(struct sock *sk, const struct dst_entry *dst) @@ -631,7 +670,7 @@ struct sock *tcp_create_openreq_child(const struct sock *sk, if (skb->len >= TCP_MSS_DEFAULT + newtp->tcp_header_len) newicsk->icsk_ack.last_seg_size = skb->len - newtp->tcp_header_len; newtp->rx_opt.mss_clamp = req->mss; - tcp_ecn_openreq_child(newtp, req); + tcp_ecn_openreq_child(newsk, req, skb); newtp->fastopen_req = NULL; RCU_INIT_POINTER(newtp->fastopen_rsk, NULL); diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 63352372248b..d0f0fee8335e 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -322,7 +322,7 @@ static u16 tcp_select_window(struct sock *sk) /* Packet ECN state for a SYN-ACK */ static void tcp_ecn_send_synack(struct sock *sk, struct sk_buff *skb) { - const struct tcp_sock *tp = tcp_sk(sk); + struct tcp_sock *tp = tcp_sk(sk); TCP_SKB_CB(skb)->tcp_flags &= ~TCPHDR_CWR; if (tcp_ecn_disabled(tp)) @@ -330,6 +330,13 @@ static void tcp_ecn_send_synack(struct sock *sk, struct sk_buff *skb) else if (tcp_ca_needs_ecn(sk) || tcp_bpf_ca_needs_ecn(sk)) INET_ECN_xmit(sk); + + if (tp->ecn_flags & TCP_ECN_MODE_ACCECN) { + TCP_SKB_CB(skb)->tcp_flags &= ~TCPHDR_ACE; + TCP_SKB_CB(skb)->tcp_flags |= + tcp_accecn_reflector_flags(tp->syn_ect_rcv); + tp->syn_ect_snt = inet_sk(sk)->tos & INET_ECN_MASK; + } } /* Packet ECN state for a SYN. */ @@ -337,8 +344,20 @@ static void tcp_ecn_send_syn(struct sock *sk, struct sk_buff *skb) { struct tcp_sock *tp = tcp_sk(sk); bool bpf_needs_ecn = tcp_bpf_ca_needs_ecn(sk); - bool use_ecn = READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_ecn) == 1 || - tcp_ca_needs_ecn(sk) || bpf_needs_ecn; + bool use_ecn, use_accecn; + u8 tcp_ecn = READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_ecn); + + /* ============== ========================== + * tcp_ecn values Outgoing connections + * ============== ========================== + * 0,2,5 Do not request ECN + * 1,4 Request ECN connection + * 3 Request AccECN connection + * ============== ========================== + */ + use_accecn = tcp_ecn == 3 || tcp_ca_needs_accecn(sk); + use_ecn = tcp_ecn == 1 || tcp_ecn == 4 || + tcp_ca_needs_ecn(sk) || bpf_needs_ecn || use_accecn; if (!use_ecn) { const struct dst_entry *dst = __sk_dst_get(sk); @@ -354,35 +373,58 @@ static void tcp_ecn_send_syn(struct sock *sk, struct sk_buff *skb) INET_ECN_xmit(sk); TCP_SKB_CB(skb)->tcp_flags |= TCPHDR_ECE | TCPHDR_CWR; - tcp_ecn_mode_set(tp, TCP_ECN_MODE_RFC3168); + if (use_accecn) { + TCP_SKB_CB(skb)->tcp_flags |= TCPHDR_AE; + tcp_ecn_mode_set(tp, TCP_ECN_MODE_PENDING); + tp->syn_ect_snt = inet_sk(sk)->tos & INET_ECN_MASK; + } else { + tcp_ecn_mode_set(tp, TCP_ECN_MODE_RFC3168); + } } } static void tcp_ecn_clear_syn(struct sock *sk, struct sk_buff *skb) { - if (READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_ecn_fallback)) + if (READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_ecn_fallback)) { /* tp->ecn_flags are cleared at a later point in time when * SYN ACK is ultimatively being received. */ - TCP_SKB_CB(skb)->tcp_flags &= ~(TCPHDR_ECE | TCPHDR_CWR); + TCP_SKB_CB(skb)->tcp_flags &= ~TCPHDR_ACE; + } +} + +static void tcp_accecn_echo_syn_ect(struct tcphdr *th, u8 ect) +{ + th->ae = !!(ect & INET_ECN_ECT_0); + th->cwr = ect != INET_ECN_ECT_0; + th->ece = ect == INET_ECN_ECT_1; } static void tcp_ecn_make_synack(const struct request_sock *req, struct tcphdr *th) { - if (inet_rsk(req)->ecn_ok) + if (tcp_rsk(req)->accecn_ok) + tcp_accecn_echo_syn_ect(th, tcp_rsk(req)->syn_ect_rcv); + else if (inet_rsk(req)->ecn_ok) th->ece = 1; } -static void tcp_accecn_set_ace(struct tcphdr *th, struct tcp_sock *tp) +static void tcp_accecn_set_ace(struct tcp_sock *tp, struct sk_buff *skb, + struct tcphdr *th) { u32 wire_ace; - wire_ace = tp->received_ce + TCP_ACCECN_CEP_INIT_OFFSET; - th->ece = !!(wire_ace & 0x1); - th->cwr = !!(wire_ace & 0x2); - th->ae = !!(wire_ace & 0x4); - tp->received_ce_pending = 0; + /* The final packet of the 3WHS or anything like it must reflect + * the SYN/ACK ECT instead of putting CEP into ACE field, such + * case show up in tcp_flags. + */ + if (likely(!(TCP_SKB_CB(skb)->tcp_flags & TCPHDR_ACE))) { + wire_ace = tp->received_ce + TCP_ACCECN_CEP_INIT_OFFSET; + th->ece = !!(wire_ace & 0x1); + th->cwr = !!(wire_ace & 0x2); + th->ae = !!(wire_ace & 0x4); + tp->received_ce_pending = 0; + } } /* Set up ECN state for a packet on a ESTABLISHED socket that is about to @@ -397,8 +439,9 @@ static void tcp_ecn_send(struct sock *sk, struct sk_buff *skb, return; if (tcp_ecn_mode_accecn(tp)) { - INET_ECN_xmit(sk); - tcp_accecn_set_ace(th, tp); + if (!tcp_accecn_ace_fail_recv(tp)) + INET_ECN_xmit(sk); + tcp_accecn_set_ace(tp, skb, th); skb_shinfo(skb)->gso_type |= SKB_GSO_TCP_ACCECN; } else { /* Not-retransmitted data segment: set ECT and inject CWR. */ @@ -3415,7 +3458,10 @@ int __tcp_retransmit_skb(struct sock *sk, struct sk_buff *skb, int segs) tcp_retrans_try_collapse(sk, skb, avail_wnd); } - /* RFC3168, section 6.1.1.1. ECN fallback */ + /* RFC3168, section 6.1.1.1. ECN fallback + * As AccECN uses the same SYN flags (+ AE), this check covers both + * cases. + */ if ((TCP_SKB_CB(skb)->tcp_flags & TCPHDR_SYN_ECN) == TCPHDR_SYN_ECN) tcp_ecn_clear_syn(sk, skb); diff --git a/net/ipv6/syncookies.c b/net/ipv6/syncookies.c index 9d83eadd308b..50046460ee0b 100644 --- a/net/ipv6/syncookies.c +++ b/net/ipv6/syncookies.c @@ -264,6 +264,7 @@ struct sock *cookie_v6_check(struct sock *sk, struct sk_buff *skb) if (!req->syncookie) ireq->rcv_wscale = rcv_wscale; ireq->ecn_ok &= cookie_ecn_ok(net, dst); + tcp_rsk(req)->accecn_ok = ireq->ecn_ok && cookie_accecn_ok(th); ret = tcp_get_cookie_sock(sk, skb, req, dst); if (!ret) { diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c index e8e68a142649..eb7dc7fb08cc 100644 --- a/net/ipv6/tcp_ipv6.c +++ b/net/ipv6/tcp_ipv6.c @@ -544,6 +544,7 @@ static int tcp_v6_send_synack(const struct sock *sk, struct dst_entry *dst, skb = tcp_make_synack(sk, dst, req, foc, synack_type, syn_skb); if (skb) { + tcp_rsk(req)->syn_ect_snt = np->tclass & INET_ECN_MASK; __tcp_v6_send_check(skb, &ireq->ir_v6_loc_addr, &ireq->ir_v6_rmt_addr); From patchwork Wed May 14 13:56:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Chia-Yu Chang \(Nokia\)" X-Patchwork-Id: 890050 Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2042.outbound.protection.outlook.com [40.107.22.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8A1F275853; Wed, 14 May 2025 13:57:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.22.42 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231026; cv=fail; b=tkcQQJcgCQkA9M4HJKeWc8bCdDN24weNTQ8WdfYhfbrlj97qWFIF2OCDCI0QixQ1mASlFdfhnPMpfGrbmOaKcrvOS3pne5xzOd8FI3C0jYxbhuKx7iugz9aqvUEtAydx4+zMOEgKAjss9iluB1Z71WeGH5E8OefJFVmSUYPuKhU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231026; c=relaxed/simple; bh=9qmujAjk4RWg91NaBZA23iQgM+Gxiha+wF+iwobw0uA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=n75Dc1RzRGbh7oXtLv5cjMgeoCPLbrTX5+zbz36ZxrXwlnKZAbeclgLKRuXeQlOaAX9K8nNphAUaGdu67EBNNkOwkYo4j47s+13RQmY7u9B8cUbDI3AkhQzGXGLToWZzKLQr/qOnxBzQKJcS4nCSNCUtAC9x8bdd2CKReRIrd9I= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com; spf=fail smtp.mailfrom=nokia-bell-labs.com; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b=ApIdg+GC; arc=fail smtp.client-ip=40.107.22.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b="ApIdg+GC" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KE+8Ssk8w0DbDswbSoJcz2wwEI+XvvorQIia3k69iaywrBKeWmqOZzBZSyQExD/RBomVGlSSeKjiwFCaivsXKxqCKsqsPYKWbfMNeuZcEs7BNBmSlp3Vcq1GhfZ/BrMeN9R0iOZoTRw5nvakVoM3Oa368gaBGhi+4g2vrMkEacQVIHlt2r2veeCGlWXzdMKHupR2Voy/PMDRad+tkp2IdZD49rdPrAQyCTWgJRE5uIIkq2D0ypnqr/VgG7pgresjRcJbV5NUL1V0CvVTcKNiimxBAyXSfFDsORQmkezSJNXprv7gLyaAEZIRnS2M1b64r2QPoasVP0xxnhJOQDdxgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=D7F71i36/NPvNde2+sbLfpGH2LLOkU+UOArXF07/SJI=; b=T/49ek0LwyEHqYH9GRxrwcAb7CtrWopTkkD4t5lF1v+toTls87lOGN9GDvouq6Kt3MWxFNL6Q2n5zRd3DK4ZlA/kRDs47o9ltRq/zw1IDqlFHESO/qReRZZqiTwLAsTA5TLpVbJg3pT3lYt6AanoWrvVQnbvyyLg96PmxSAKm7Fj+B2RpJ31MNO6mRvV/l2P1p+CLYFSVGAIqeDs4d8vQABJaOm7DXAFlOoksPjWgkR6H5FDBQ7QEdCKZQmZ7RB8htypvm3kUrQjJMozCy172PPWLRDZl/40gAddp3JcmptLXYIqZg/saa+5/eJ3SVZOfuJERJO/t0ORFHd/bVIyGg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 131.228.2.240) smtp.rcpttodomain=amazon.com smtp.mailfrom=nokia-bell-labs.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nokia-bell-labs.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia-bell-labs.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=D7F71i36/NPvNde2+sbLfpGH2LLOkU+UOArXF07/SJI=; b=ApIdg+GCwwnTbB9dtWUFQEdFYT8oEbqhXUPlERM38d++ND+62kJjDTfLvwbCKuqio7G0my9T9Z71In7qDW+FTpzmdCsfs28h3as7T2P/5vz0kx0bp80AUlz24VxahXLyX2jaRLptF4Bnm/w02DkM00+sEh88Iu0cjYqq5UWrTq/oIPLtABqcjxViB5UTqVGo6FBxH161L2rlifSOEA+5jB0cryxR4Gs2Z58l/xIQnD9u7+Lch2MtNKcrOulaT9hJ3lLaNjrKOUk32KrcjibfpFkV5Ly5lawNMYbsX5o9ADYMwudKHariJXeYUTK1oCk5M82nksMt9Lgst9cAEbBpmA== Received: from AS4P191CA0027.EURP191.PROD.OUTLOOK.COM (2603:10a6:20b:5d9::10) by DB8PR07MB6284.eurprd07.prod.outlook.com (2603:10a6:10:131::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8722.30; Wed, 14 May 2025 13:57:00 +0000 Received: from AMS1EPF0000004E.eurprd04.prod.outlook.com (2603:10a6:20b:5d9:cafe::6c) by AS4P191CA0027.outlook.office365.com (2603:10a6:20b:5d9::10) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8746.17 via Frontend Transport; Wed, 14 May 2025 13:56:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 131.228.2.240) smtp.mailfrom=nokia-bell-labs.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nokia-bell-labs.com; Received-SPF: Pass (protection.outlook.com: domain of nokia-bell-labs.com designates 131.228.2.240 as permitted sender) receiver=protection.outlook.com; client-ip=131.228.2.240; helo=fihe3nok0735.emea.nsn-net.net; pr=C Received: from fihe3nok0735.emea.nsn-net.net (131.228.2.240) by AMS1EPF0000004E.mail.protection.outlook.com (10.167.16.139) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8722.18 via Frontend Transport; Wed, 14 May 2025 13:56:59 +0000 Received: from sarah.nbl.nsn-rdnet.net (sarah.nbl.nsn-rdnet.net [10.0.73.150]) by fihe3nok0735.emea.nsn-net.net (Postfix) with ESMTP id 0DA292004A; Wed, 14 May 2025 16:56:58 +0300 (EEST) From: chia-yu.chang@nokia-bell-labs.com To: linux-doc@vger.kernel.org, corbet@lwn.net, horms@kernel.org, dsahern@kernel.org, kuniyu@amazon.com, bpf@vger.kernel.org, netdev@vger.kernel.org, dave.taht@gmail.com, pabeni@redhat.com, jhs@mojatatu.com, kuba@kernel.org, stephen@networkplumber.org, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, andrew+netdev@lunn.ch, donald.hunter@gmail.com, ast@fiberby.net, liuhangbin@gmail.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, ij@kernel.org, ncardwell@google.com, koen.de_schepper@nokia-bell-labs.com, g.white@cablelabs.com, ingemar.s.johansson@ericsson.com, mirja.kuehlewind@ericsson.com, cheshire@apple.com, rs.ietf@gmx.at, Jason_Livingood@comcast.com, vidhi_goel@apple.com Cc: Chia-Yu Chang Subject: [PATCH v7 net-next 06/15] tcp: accecn: add AccECN rx byte counters Date: Wed, 14 May 2025 15:56:33 +0200 Message-Id: <20250514135642.11203-7-chia-yu.chang@nokia-bell-labs.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> References: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AMS1EPF0000004E:EE_|DB8PR07MB6284:EE_ X-MS-Office365-Filtering-Correlation-Id: 94d02c6a-63b4-420e-b17b-08dd92ef325a X-LD-Processed: 5d471751-9675-428d-917b-70f44f9630b0,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|7416014|376014|36860700013|82310400026|1800799024|921020; X-Microsoft-Antispam-Message-Info: =?utf-8?q?pkckVYfhGxHXkGTjZ0RqMMnakbiBwyg?= =?utf-8?q?mhSxNWNdakOT7g2Deu/g2GINWzezF09QneQEaINXKMbp2M/u8MUZlK/r2GfLyJstk?= =?utf-8?q?us27CH7GiPC61NBgcteHonVQ3BgLW5NoSFzTLD+vzmt1b1BJaXxV/eVIx9v5/QWHa?= =?utf-8?q?FUKsL3kfPHnr3Ju4iDI3F/+yiLBL7iQcXsd7OxfwVCdwOAXWhIV4VrwT32eFhS6lq?= =?utf-8?q?cJoRbm6F+V37i8PTchnzOjtO+Bfaq3LaL9wWyGGuyBGiuHEWIiuMVhxiqQy0iSKKG?= =?utf-8?q?5bN9tdJ6bkvgp+RT1df9NM6F8mJZNPnHER0aXyTdC9g8fSahuccabcGtfA0VOTSTC?= =?utf-8?q?vsdjfgAZfS8yIYE6aq/CA/BQyO19erQKgc4hddWGrDwAyGnjTKQcf2yiUD+oMpYB7?= =?utf-8?q?FfisSYlNQbUZKNRJr2jko+IPkg/sf8a5srPTLgAtnXTmt7UlbDwVgAh5HxR30UdWB?= =?utf-8?q?4aSzB/mjOxGrTMSCcfExly821OS7osP7W8DzmE58GeYXVIjHi5+BUfHZm3ULMMS5x?= =?utf-8?q?oV20Ox33BwZQkhF512B3ABPubqTwr66ChWjKRO/LpkHf8ABTeFB4ur8CgXQU8bk/m?= =?utf-8?q?b7gSuedN3JrT8EpJjoFpF4mHOop46AblMIDCMDra+qhh8gBFldUQTkdWIX+g2WT3X?= =?utf-8?q?2hZPqnuzGhFapUGANUYm8mZuDhAQPKedBLBjBghpLKA41ELTiRvKXHoIFCucM51Ad?= =?utf-8?q?EUSGCXXlA8x9WWY52vtPkYTeZrM1ywuuA5Gay8r5yKReCxJ3LMN/T7Oro0YjaCfvz?= =?utf-8?q?Q4j1Hm7/Q7R5cJlPorN1thENw3no92eFxlL39SkYb1K+D1/jo4esTrBqFWoEU3zjm?= =?utf-8?q?yS5nEKy9FoJlF6HNEH6pvKZEkLWNJFgVUoTDVzXaKiabAnuwbXqaBef6m+x7MDjKN?= =?utf-8?q?/J6hcXxAFvfS8Bz7Li9GP5gOCLfGI37M4AcHlBlP7fI2teWw3myjbs1cU/bXdURk5?= =?utf-8?q?CzXvuFQgXL2mv9IUz/EWvHVqTXUS2/nllSfdnpHXBHbDMWKAIUCALguYE6RcVPhD8?= =?utf-8?q?47nWl6HNe2QNHNjum9wkNzCfTKulYnFfNOSKl5jxkyk/KS8dmuzim1OsJ/fS8M5Ch?= =?utf-8?q?0yS2MyM/B3ZFQxRmnRusGkffkZmRaokXxzaK6vUzdaHycVLkhY2ofHtutfVUt2SEp?= =?utf-8?q?DB26nFuwF7xodq8xT9T6oUOBzsC6XM3as4AEOiW2ltHdIam0J4nB6BlxBVay91K9+?= =?utf-8?q?ooGaSmhYUxN/juzAbjP02PWBpcct9mqNMbyNmKY4XtTvOXYMQgSoaUgUUU3aTVvBL?= =?utf-8?q?c7YhgkZj87BulN4NXOeLUibjqLBi6hPDrFP4Hbi7Rie45VyZrYRBBoJGgl5wDposF?= =?utf-8?q?ZGPuS4aEmzoU1fI/85F3AnUr/aU20GZOtvU4Mu5AD6HzajMSzruk4q8qYCyAYcK9o?= =?utf-8?q?YZ5E96V6If5PpFNJPbvL3MKn1iXuWJZPnWLpz5CEK16ycBAecB4lPcUUMER7ply24?= =?utf-8?q?FJ7dfm9k+a6pkGTjvY3zhlqWn5JQf3KA=3D=3D?= X-Forefront-Antispam-Report: CIP:131.228.2.240; CTRY:FI; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:fihe3nok0735.emea.nsn-net.net; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(7416014)(376014)(36860700013)(82310400026)(1800799024)(921020); DIR:OUT; SFP:1101; X-OriginatorOrg: nokia-bell-labs.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2025 13:56:59.5469 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 94d02c6a-63b4-420e-b17b-08dd92ef325a X-MS-Exchange-CrossTenant-Id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5d471751-9675-428d-917b-70f44f9630b0; Ip=[131.228.2.240]; Helo=[fihe3nok0735.emea.nsn-net.net] X-MS-Exchange-CrossTenant-AuthSource: AMS1EPF0000004E.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR07MB6284 From: Ilpo Järvinen These three byte counters track IP ECN field payload byte sums for all arriving (acceptable) packets for ECT0, ECT1, and CE. The AccECN option (added by a later patch in the series) echoes these counters back to sender side; therefore, it is placed within the group of tcp_sock_write_txrx. Below are the pahole outcomes before and after this patch, in which the group size of tcp_sock_write_txrx is increased from 95 + 4 to 107 + 4 and an extra 4-byte hole is created but will be exploited in later patches: [BEFORE THIS PATCH] struct tcp_sock { [...] u32 delivered_ce; /* 2640 4 */ u32 received_ce; /* 2644 4 */ u32 app_limited; /* 2648 4 */ u32 rcv_wnd; /* 2652 4 */ struct tcp_options_received rx_opt; /* 2656 24 */ __cacheline_group_end__tcp_sock_write_txrx[0]; /* 2680 0 */ [...] /* size: 3264, cachelines: 51, members: 169 */ } [AFTER THIS PATCH] struct tcp_sock { [...] u32 delivered_ce; /* 2640 4 */ u32 received_ce; /* 2644 4 */ u32 received_ecn_bytes[3];/* 2648 12 */ u32 app_limited; /* 2660 4 */ u32 rcv_wnd; /* 2664 4 */ struct tcp_options_received rx_opt; /* 2668 24 */ __cacheline_group_end__tcp_sock_write_txrx[0]; /* 2692 0 */ /* XXX 4 bytes hole, try to pack */ [...] /* size: 3264, cachelines: 51, members: 170 */ } Signed-off-by: Ilpo Järvinen Signed-off-by: Neal Cardwell Co-developed-by: Chia-Yu Chang Signed-off-by: Chia-Yu Chang --- .../networking/net_cachelines/tcp_sock.rst | 1 + include/linux/tcp.h | 4 ++++ include/net/tcp.h | 18 +++++++++++++++++- net/ipv4/tcp.c | 3 ++- net/ipv4/tcp_input.c | 13 +++++++++---- net/ipv4/tcp_minisocks.c | 3 ++- 6 files changed, 35 insertions(+), 7 deletions(-) diff --git a/Documentation/networking/net_cachelines/tcp_sock.rst b/Documentation/networking/net_cachelines/tcp_sock.rst index eef8700085de..64306ae096cd 100644 --- a/Documentation/networking/net_cachelines/tcp_sock.rst +++ b/Documentation/networking/net_cachelines/tcp_sock.rst @@ -102,6 +102,7 @@ u32 prr_out read_mostly read_m u32 delivered read_mostly read_write tcp_rate_skb_sent, tcp_newly_delivered(tx);tcp_ack, tcp_rate_gen, tcp_clean_rtx_queue (rx) u32 delivered_ce read_mostly read_write tcp_rate_skb_sent(tx);tcp_rate_gen(rx) u32 received_ce read_mostly read_write +u32[3] received_ecn_bytes read_mostly read_write u8:4 received_ce_pending read_mostly read_write u8:2 syn_ect_snt write_mostly read_write u8:2 syn_ect_rcv read_mostly read_write diff --git a/include/linux/tcp.h b/include/linux/tcp.h index b0e6a11a3fa1..427f2adab247 100644 --- a/include/linux/tcp.h +++ b/include/linux/tcp.h @@ -307,6 +307,10 @@ struct tcp_sock { u32 delivered; /* Total data packets delivered incl. rexmits */ u32 delivered_ce; /* Like the above but only ECE marked packets */ u32 received_ce; /* Like the above but for rcvd CE marked pkts */ + u32 received_ecn_bytes[3]; /* received byte counters for three ECN + * types: INET_ECN_ECT_1, INET_ECN_ECT_0, + * and INET_ECN_CE + */ u32 app_limited; /* limited until "delivered" reaches this val */ u32 rcv_wnd; /* Current receiver window */ /* diff --git a/include/net/tcp.h b/include/net/tcp.h index 5fce4b49e132..101e07b8a089 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -467,7 +467,8 @@ static inline int tcp_accecn_extract_syn_ect(u8 ace) bool tcp_accecn_validate_syn_feedback(struct sock *sk, u8 ace, u8 sent_ect); void tcp_accecn_third_ack(struct sock *sk, const struct sk_buff *skb, u8 syn_ect_snt); -void tcp_ecn_received_counters(struct sock *sk, const struct sk_buff *skb); +void tcp_ecn_received_counters(struct sock *sk, const struct sk_buff *skb, + u32 payload_len); enum tcp_tw_status { TCP_TW_SUCCESS = 0, @@ -1035,6 +1036,20 @@ static inline u32 tcp_rsk_tsval(const struct tcp_request_sock *treq) * See draft-ietf-tcpm-accurate-ecn for the latest values. */ #define TCP_ACCECN_CEP_INIT_OFFSET 5 +#define TCP_ACCECN_E1B_INIT_OFFSET 1 +#define TCP_ACCECN_E0B_INIT_OFFSET 1 +#define TCP_ACCECN_CEB_INIT_OFFSET 0 + +static inline void __tcp_accecn_init_bytes_counters(int *counter_array) +{ + BUILD_BUG_ON(INET_ECN_ECT_1 != 0x1); + BUILD_BUG_ON(INET_ECN_ECT_0 != 0x2); + BUILD_BUG_ON(INET_ECN_CE != 0x3); + + counter_array[INET_ECN_ECT_1 - 1] = 0; + counter_array[INET_ECN_ECT_0 - 1] = 0; + counter_array[INET_ECN_CE - 1] = 0; +} /* The highest ECN variant (Accurate ECN, ECN, or no ECN) that is * attemped to be negotiated and requested for incoming connection @@ -1053,6 +1068,7 @@ static inline void tcp_accecn_init_counters(struct tcp_sock *tp) { tp->received_ce = 0; tp->received_ce_pending = 0; + __tcp_accecn_init_bytes_counters(tp->received_ecn_bytes); } /* State flags for sacked in struct tcp_skb_cb */ diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 29146bb3585a..af52ebfdd734 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -5127,6 +5127,7 @@ static void __init tcp_struct_check(void) CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_txrx, delivered); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_txrx, delivered_ce); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_txrx, received_ce); + CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_txrx, received_ecn_bytes); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_txrx, app_limited); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_txrx, rcv_wnd); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_txrx, rx_opt); @@ -5134,7 +5135,7 @@ static void __init tcp_struct_check(void) /* 32bit arches with 8byte alignment on u64 fields might need padding * before tcp_clock_cache. */ - CACHELINE_ASSERT_GROUP_SIZE(struct tcp_sock, tcp_sock_write_txrx, 95 + 4); + CACHELINE_ASSERT_GROUP_SIZE(struct tcp_sock, tcp_sock_write_txrx, 107 + 4); /* RX read-write hotpath cache lines */ CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, bytes_received); diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index f0475f95e112..76fda2a63add 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -6135,7 +6135,8 @@ static void tcp_urg(struct sock *sk, struct sk_buff *skb, const struct tcphdr *t } /* Updates Accurate ECN received counters from the received IP ECN field */ -void tcp_ecn_received_counters(struct sock *sk, const struct sk_buff *skb) +void tcp_ecn_received_counters(struct sock *sk, const struct sk_buff *skb, + u32 payload_len) { u8 ecnfield = TCP_SKB_CB(skb)->ip_dsfield & INET_ECN_MASK; u8 is_ce = INET_ECN_is_ce(ecnfield); @@ -6150,6 +6151,9 @@ void tcp_ecn_received_counters(struct sock *sk, const struct sk_buff *skb) tp->received_ce += pcount; tp->received_ce_pending = min(tp->received_ce_pending + pcount, 0xfU); + + if (payload_len > 0) + tp->received_ecn_bytes[ecnfield - 1] += payload_len; } } @@ -6423,7 +6427,7 @@ void tcp_rcv_established(struct sock *sk, struct sk_buff *skb) flag |= __tcp_replace_ts_recent(tp, delta); - tcp_ecn_received_counters(sk, skb); + tcp_ecn_received_counters(sk, skb, 0); /* We know that such packets are checksummed * on entry. @@ -6469,7 +6473,8 @@ void tcp_rcv_established(struct sock *sk, struct sk_buff *skb) /* Bulk data transfer: receiver */ tcp_cleanup_skb(skb); __skb_pull(skb, tcp_header_len); - tcp_ecn_received_counters(sk, skb); + tcp_ecn_received_counters(sk, skb, + len - tcp_header_len); eaten = tcp_queue_rcv(sk, skb, &fragstolen); tcp_event_data_recv(sk, skb); @@ -6516,7 +6521,7 @@ void tcp_rcv_established(struct sock *sk, struct sk_buff *skb) tcp_accecn_third_ack(sk, skb, tp->syn_ect_snt); tcp_fast_path_on(tp); } - tcp_ecn_received_counters(sk, skb); + tcp_ecn_received_counters(sk, skb, len - th->doff * 4); reason = tcp_ack(sk, skb, FLAG_SLOWPATH | FLAG_UPDATE_TS_RECENT); if ((int)reason < 0) { diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c index 779a206a5ca6..3f8225bae49f 100644 --- a/net/ipv4/tcp_minisocks.c +++ b/net/ipv4/tcp_minisocks.c @@ -497,10 +497,11 @@ static void tcp_ecn_openreq_child(struct sock *sk, struct tcp_sock *tp = tcp_sk(sk); if (treq->accecn_ok) { + const struct tcphdr *th = (const struct tcphdr *)skb->data; tcp_ecn_mode_set(tp, TCP_ECN_MODE_ACCECN); tp->syn_ect_snt = treq->syn_ect_snt; tcp_accecn_third_ack(sk, skb, treq->syn_ect_snt); - tcp_ecn_received_counters(sk, skb); + tcp_ecn_received_counters(sk, skb, skb->len - th->doff * 4); } else { tcp_ecn_mode_set(tp, inet_rsk(req)->ecn_ok ? TCP_ECN_MODE_RFC3168 : From patchwork Wed May 14 13:56:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Chia-Yu Chang \(Nokia\)" X-Patchwork-Id: 890048 Received: from AM0PR02CU008.outbound.protection.outlook.com (mail-westeuropeazon11013017.outbound.protection.outlook.com [52.101.72.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 205D627F72C; Wed, 14 May 2025 13:57:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.72.17 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231032; cv=fail; b=LVHm+ccxWN6iEhWVq2JiQZUWMF68x9unHwR6njInLdEcOCrpcSWpdJNvBxhtkmpEzhrwT895CwqUkqLBPij4l9nwpfppKE4FqdtRXr06dT/yNpUibwb9B5Tf3Q/t3KWCZwhJcIVTi3F27rWGUKXfYF5zP00Z6ejrMQa2YYMDSWk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231032; c=relaxed/simple; bh=sywBYSVQmStUZmxTxp3pUP0KmLv81U3yCY/vWwS3BtU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=YfSaZz4PXD8c3kHTWdU0wA6UqPd7+rr7NCNObrxAfIng/YqUDx+FwUhvzGar8hgTMqZSfVhEuHoDf4vKnlwQTUbyUmmshtVvtgQJqnXDsMOcdfxjTr2ApD+qVfq9xLOiIAPXTtchhVqVumdz/67/PyC95PYZq+zQ+ES7bpu9aGQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com; spf=fail smtp.mailfrom=nokia-bell-labs.com; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b=JNjcWj6C; arc=fail smtp.client-ip=52.101.72.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b="JNjcWj6C" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=r7OXaZWOjKGPqX3GhdQtZBGzG1LH2shfNOjxwchrkqNAM5wQmw8rsbTm5YDXza5xmb56LyqQJPGXeBwb66Fen+CClaTVTT8dMJEbqxwWE4NI/gjQwxLH+1kveuO1iT7c4QyX7DDfIrRPCI2mbrYghf48cSdcl3rfC7oTTFu68vf65Yqwlv795XkFZzlGco/S2vFrC4xt/550NaSS288lJZLRnHfc8Qy2BAi8oQlzAePR3/kClFOLL9piC3W0OkbPhgQWdoJUAFYgcByJ7pqNKxH6aYHlz9EZIJ2sKYnpKOBXatA5zpOGAStO5aF5OGANU7BgvdlrMNY3cmzWtNrkLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nE88s/9KM8F2HIqiCNvKLeOvudM+htBcoca+HWhkMMo=; b=bEDynMARk2/jmjGorImmkaGlwvy9PaY6LQ1FXbPKkMEBuNfl8QvTI37Kz5PzltHFzT0b9y0RitjAOTD5dihwR06hbeMHhMk/ieHW6b1aWX2VIAp2V2ZHmyMel686/8Lw+NqFac44DsjinBnvQtLR3dtB6Zq+F0OJCaL+OXDy6Q0DTZ2yA1Rrg671Yb7FiutE4nH/etqgUUG/IQO5oznAz4nL2+9zwbyyU2VzYWmnEzWhOEJ/CG7t1NQfDpp8zYGI5yp0gt8Qg5qr2TucEuHcHXxhiR6RfjyKGvZvfJk9K9JuL6QqkJjpVBy6gcZ7jSRhI3OzVRNhCN7gL5jMfRVpRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 131.228.2.240) smtp.rcpttodomain=amazon.com smtp.mailfrom=nokia-bell-labs.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nokia-bell-labs.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia-bell-labs.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nE88s/9KM8F2HIqiCNvKLeOvudM+htBcoca+HWhkMMo=; b=JNjcWj6CTJMOvCUJkKhzcbROfrDXdvjbPftB7vvGJ0WPiWyGJS6ZEp0ajY4TAYo7fs3xfE9aU1/YSVPNN+AmRHB5ffE/nc/FFYOfFShXWHM3UlZMi0dkNf8nJ6GGlAaOhRYYshcajP5wNopTz1hpsGm/iWy0n0EifgnKgJK14NpfJL+dTnaUx/B75LsH6g7TsleCa0h7hkZXkG15ELpyt7/MyPtxFs7/Bn6+4+rdfisQicuDJYkt05ocV8ivJ4S2ZdjnskNK/z5Mioxru86La3TZrQ+XTpFFlUQJGev7HyT/I0aQPtdhF2x9PMqYMNkbfzygxOK165zeTHjF+eEDyg== Received: from AS4P192CA0029.EURP192.PROD.OUTLOOK.COM (2603:10a6:20b:5e1::14) by AS5PR07MB10057.eurprd07.prod.outlook.com (2603:10a6:20b:67d::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8722.30; Wed, 14 May 2025 13:57:06 +0000 Received: from AMS1EPF00000048.eurprd04.prod.outlook.com (2603:10a6:20b:5e1::4) by AS4P192CA0029.outlook.office365.com (2603:10a6:20b:5e1::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8746.17 via Frontend Transport; Wed, 14 May 2025 13:57:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 131.228.2.240) smtp.mailfrom=nokia-bell-labs.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nokia-bell-labs.com; Received-SPF: Pass (protection.outlook.com: domain of nokia-bell-labs.com designates 131.228.2.240 as permitted sender) receiver=protection.outlook.com; client-ip=131.228.2.240; helo=fihe3nok0735.emea.nsn-net.net; pr=C Received: from fihe3nok0735.emea.nsn-net.net (131.228.2.240) by AMS1EPF00000048.mail.protection.outlook.com (10.167.16.132) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8722.18 via Frontend Transport; Wed, 14 May 2025 13:57:06 +0000 Received: from sarah.nbl.nsn-rdnet.net (sarah.nbl.nsn-rdnet.net [10.0.73.150]) by fihe3nok0735.emea.nsn-net.net (Postfix) with ESMTP id C43A1200CA; Wed, 14 May 2025 16:57:04 +0300 (EEST) From: chia-yu.chang@nokia-bell-labs.com To: linux-doc@vger.kernel.org, corbet@lwn.net, horms@kernel.org, dsahern@kernel.org, kuniyu@amazon.com, bpf@vger.kernel.org, netdev@vger.kernel.org, dave.taht@gmail.com, pabeni@redhat.com, jhs@mojatatu.com, kuba@kernel.org, stephen@networkplumber.org, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, andrew+netdev@lunn.ch, donald.hunter@gmail.com, ast@fiberby.net, liuhangbin@gmail.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, ij@kernel.org, ncardwell@google.com, koen.de_schepper@nokia-bell-labs.com, g.white@cablelabs.com, ingemar.s.johansson@ericsson.com, mirja.kuehlewind@ericsson.com, cheshire@apple.com, rs.ietf@gmx.at, Jason_Livingood@comcast.com, vidhi_goel@apple.com Cc: Chia-Yu Chang Subject: [PATCH v7 net-next 09/15] tcp: accecn: AccECN option Date: Wed, 14 May 2025 15:56:36 +0200 Message-Id: <20250514135642.11203-10-chia-yu.chang@nokia-bell-labs.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> References: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AMS1EPF00000048:EE_|AS5PR07MB10057:EE_ X-MS-Office365-Filtering-Correlation-Id: c0212812-ce76-4d22-d8e1-08dd92ef3662 X-LD-Processed: 5d471751-9675-428d-917b-70f44f9630b0,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|7416014|1800799024|36860700013|82310400026|921020; X-Microsoft-Antispam-Message-Info: =?utf-8?q?zsOF7YxGm+VXvNv+p6oS+4O6sjNEZZx?= =?utf-8?q?MuYnNtbaBvyPwKcVYED1p5FPmQFhfTWJqBhdlpYWbmDkyf62aBIEYur+VETNVhEze?= =?utf-8?q?O3LpXwO3D3SJqtjr+RuxUAA9UPDSfWjRnstb21ES8HYPL+suyLeJAf9ijfnXYQPHF?= =?utf-8?q?T4chWOTO0Q0tEl9I7MufX5K4VbD21/nR5+SfmhFiMjBFlUQ0sm73k2MCf9kv1BuKa?= =?utf-8?q?XlxYL1xnnR5xSWTtTbqAB2RwttfeIqmLtGIonDmaxaeaLbBVm3N+CTFufbLeoi0g/?= =?utf-8?q?ugSezflg6IFROJwLcurxOUdpG1EXVVasf8ErJtwV4cRaaBg3/cVvSaMOuQE4gDvRN?= =?utf-8?q?SUsHn8qEsC2BvxCdQv9ui4CgnCwrTt3hHyROnijP9Y4vkSW5nIahHvMvsKAj8Z2jY?= =?utf-8?q?iIfY0UcUIUXYxMQ0fW5PSskZpd0H4kVPqdbTJ9Jc3N4Y+tuZ9qQWGDw9M7kXyWRfT?= =?utf-8?q?yfcwmW6e0sKGZnmwnM3qR0Pvu//fFHXOnfUWar/cRdDooxJVg3v4XxezDWSjHMlI7?= =?utf-8?q?nvQz7Ys80xihJi8rJ0LpUl6pNrtk2e6hXd8HhsyQexhvdiKRiu37dBI4xEkWSs3t7?= =?utf-8?q?xtK0bw7SqHA0kQRNvAhi4lWVW2VVDpnF+D7s0gJjgWJSfhV+LmITicNBG13kP4A8K?= =?utf-8?q?0iCCzC6mFiD4RQTehwle86b1Nx9Up57yZ5vUSq75HWor4RYazzd8qBlxs1zWJRUm6?= =?utf-8?q?bOT7DlX9LAo6qqI/Ee5hnW6Mj65odbGjDsLvVuCg3hKagTaJBMWNN1rhQMBMsTe8e?= =?utf-8?q?HhMKYxV6v7VG7KEXRbNtRMqeYBptAvZBU0jYSTw6sDdNs6TijjN//LBTigu3LDp95?= =?utf-8?q?U6lsnqgtyFr+rFwJq1eU2j0CQvPJQf4SHdaDDLTNK86O4fxHS6DE5dleXtLzApuzC?= =?utf-8?q?55FoCn2z0OvzjvEN2tJ7uiIvNN1gH9rPfxB4pd5rQvYFTNh2DsWJuO/7oakwdgYTN?= =?utf-8?q?yc63is3206MAau1qBC3FVVLQCq+YHf4ArNQyA1sugAdmZHMuFm3d3RNYp+UM+Au8A?= =?utf-8?q?fzsdEt59/9+mMl2Bnj+ukIoeBYnjuazcb9iXd7rMG/8pPCltlyxWgkTMCGcAmVLES?= =?utf-8?q?qKjiIRWP7gMudF4SYNVUr0dxOcnTU9rssMklBATIuw4wAMj8Uw39PkyeNmEheynKi?= =?utf-8?q?pXyhK0A+ZgO8wy6oxQjOQd9gqIp20cizOwXapqP1ustiEtZoZQvhYl1GvmrTsbJg6?= =?utf-8?q?8bQo11IoqHecgwa3JLCfSC0AyrmOEmuPjC0Jh2l1S9nW0H8Y4PIDDPZxDNVUWI9o+?= =?utf-8?q?W3xV/26RRNDrEoa6EfOBE09xaOyOLFOVFJ9PR26H6z5+ChSGmsgtrxAdH2WIR5gek?= =?utf-8?q?8HPRMSSPX0oFT1sa79rj/X2Bi6yP6VtW7qdKX0WbYu9DhJOIX14/F88iLVitYvjQM?= =?utf-8?q?WxztalRa2oX0x60tOMPH7l7sz7frgSontkmOtmS/cHQAUVs6fY10OU=3D?= X-Forefront-Antispam-Report: CIP:131.228.2.240; CTRY:FI; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:fihe3nok0735.emea.nsn-net.net; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(376014)(7416014)(1800799024)(36860700013)(82310400026)(921020); DIR:OUT; SFP:1101; X-OriginatorOrg: nokia-bell-labs.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2025 13:57:06.2828 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c0212812-ce76-4d22-d8e1-08dd92ef3662 X-MS-Exchange-CrossTenant-Id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5d471751-9675-428d-917b-70f44f9630b0; Ip=[131.228.2.240]; Helo=[fihe3nok0735.emea.nsn-net.net] X-MS-Exchange-CrossTenant-AuthSource: AMS1EPF00000048.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS5PR07MB10057 From: Ilpo Järvinen The Accurate ECN allows echoing back the sum of bytes for each IP ECN field value in the received packets using AccECN option. This change implements AccECN option tx & rx side processing without option send control related features that are added by a later change. Based on specification: https://tools.ietf.org/id/draft-ietf-tcpm-accurate-ecn-28.txt (Some features of the spec will be added in the later changes rather than in this one). A full-length AccECN option is always attempted but if it does not fit, the minimum length is selected based on the counters that have changed since the last update. The AccECN option (with 24-bit fields) often ends in odd sizes so the option write code tries to take advantage of some nop used to pad the other TCP options. The delivered_ecn_bytes pairs with received_ecn_bytes similar to how delivered_ce pairs with received_ce. In contrast to ACE field, however, the option is not always available to update delivered_ecn_bytes. For ACK w/o AccECN option, the delivered bytes calculated based on the cumulative ACK+SACK information are assigned to one of the counters using an estimation heuristic to select the most likely ECN byte counter. Any estimation error is corrected when the next AccECN option arrives. It may occur that the heuristic gets too confused when there are enough different byte counter deltas between ACKs with the AccECN option in which case the heuristic just gives up on updating the counters for a while. tcp_ecn_option sysctl can be used to select option sending mode for AccECN: TCP_ECN_OPTION_DISABLED, TCP_ECN_OPTION_MINIMUM, and TCP_ECN_OPTION_FULL. This patch increases the size of tcp_info struct, as there is no existing holes for new u32 variables. Below are the pahole outcomes before and after this patch: [BEFORE THIS PATCH] struct tcp_info { [...] __u32 tcpi_total_rto_time; /* 244 4 */ /* size: 248, cachelines: 4, members: 61 */ } [AFTER THIS PATCH] struct tcp_info { [...] __u32 tcpi_total_rto_time; /* 244 4 */ __u32 tcpi_received_ce; /* 248 4 */ __u32 tcpi_delivered_e1_bytes; /* 252 4 */ __u32 tcpi_delivered_e0_bytes; /* 256 4 */ __u32 tcpi_delivered_ce_bytes; /* 260 4 */ __u32 tcpi_received_e1_bytes; /* 264 4 */ __u32 tcpi_received_e0_bytes; /* 268 4 */ __u32 tcpi_received_ce_bytes; /* 272 4 */ /* size: 280, cachelines: 5, members: 68 */ } This patch uses the existing 1-byte holes in the tcp_sock_write_txrx group for new u8 members, but adds a 4-byte hole in tcp_sock_write_rx group after the new u32 delivered_ecn_bytes[3] member. Therefore, the group size of tcp_sock_write_rx is increased from 96 to 112. Below are the pahole outcomes before and after this patch: [BEFORE THIS PATCH] struct tcp_sock { [...] u8 received_ce_pending:4; /* 2586: 0 1 */ u8 unused2:4; /* 2586: 4 1 */ /* XXX 1 byte hole, try to pack */ [...] u32 rcv_rtt_last_tsecr; /* 2732 4 */ [...] __cacheline_group_end__tcp_sock_write_rx[0]; /* 2792 0 */ [...] /* size: 3264, cachelines: 51, members: 170 */ } [AFTER THIS PATCH] struct tcp_sock { [...] u8 received_ce_pending:4;/* 2586: 0 1 */ u8 unused2:4; /* 2586: 4 1 */ u8 accecn_minlen:2; /* 2587: 0 1 */ u8 est_ecnfield:2; /* 2587: 2 1 */ u8 unused3:4; /* 2587: 4 1 */ [...] u32 rcv_rtt_last_tsecr; /* 2732 4 */ u32 delivered_ecn_bytes[3];/* 2736 12 */ /* XXX 4 bytes hole, try to pack */ [...] __cacheline_group_end__tcp_sock_write_rx[0]; /* 2808 0 */ [...] /* size: 3264, cachelines: 51, members: 174 */ } Signed-off-by: Ilpo Järvinen Signed-off-by: Neal Cardwell Co-developed-by: Chia-Yu Chang Signed-off-by: Chia-Yu Chang --- .../networking/net_cachelines/tcp_sock.rst | 3 + include/linux/tcp.h | 9 +- include/net/netns/ipv4.h | 1 + include/net/tcp.h | 20 +++ include/uapi/linux/tcp.h | 7 + net/ipv4/sysctl_net_ipv4.c | 9 + net/ipv4/tcp.c | 15 +- net/ipv4/tcp_input.c | 166 +++++++++++++++++- net/ipv4/tcp_ipv4.c | 1 + net/ipv4/tcp_output.c | 151 +++++++++++++++- 10 files changed, 369 insertions(+), 13 deletions(-) diff --git a/Documentation/networking/net_cachelines/tcp_sock.rst b/Documentation/networking/net_cachelines/tcp_sock.rst index 64306ae096cd..c74f939f32cc 100644 --- a/Documentation/networking/net_cachelines/tcp_sock.rst +++ b/Documentation/networking/net_cachelines/tcp_sock.rst @@ -104,9 +104,12 @@ u32 delivered_ce read_mostly read_w u32 received_ce read_mostly read_write u32[3] received_ecn_bytes read_mostly read_write u8:4 received_ce_pending read_mostly read_write +u32[3] delivered_ecn_bytes read_write u8:2 syn_ect_snt write_mostly read_write u8:2 syn_ect_rcv read_mostly read_write u8:1 wait_third_ack read_write +u8:2 accecn_minlen write_mostly read_write +u8:2 est_ecnfield read_write u8:4 accecn_fail_mode u32 lost read_mostly tcp_ack u32 app_limited read_write read_mostly tcp_rate_check_app_limited,tcp_rate_skb_sent(tx);tcp_rate_gen(rx) diff --git a/include/linux/tcp.h b/include/linux/tcp.h index 427f2adab247..afb1a5ac65bf 100644 --- a/include/linux/tcp.h +++ b/include/linux/tcp.h @@ -122,8 +122,9 @@ struct tcp_options_received { smc_ok : 1, /* SMC seen on SYN packet */ snd_wscale : 4, /* Window scaling received from sender */ rcv_wscale : 4; /* Window scaling to send to receiver */ - u8 saw_unknown:1, /* Received unknown option */ - unused:7; + u8 accecn:6, /* AccECN index in header, 0=no options */ + saw_unknown:1, /* Received unknown option */ + unused:1; u8 num_sacks; /* Number of SACK blocks */ u16 user_mss; /* mss requested by user in ioctl */ u16 mss_clamp; /* Maximal mss, negotiated at connection setup */ @@ -294,6 +295,9 @@ struct tcp_sock { rate_app_limited:1; /* rate_{delivered,interval_us} limited? */ u8 received_ce_pending:4, /* Not yet transmit cnt of received_ce */ unused2:4; + u8 accecn_minlen:2,/* Minimum length of AccECN option sent */ + est_ecnfield:2,/* ECN field for AccECN delivered estimates */ + unused3:4; __be32 pred_flags; u64 tcp_clock_cache; /* cache last tcp_clock_ns() (see tcp_mstamp_refresh()) */ u64 tcp_mstamp; /* most recent packet received/sent */ @@ -338,6 +342,7 @@ struct tcp_sock { u32 rate_delivered; /* saved rate sample: packets delivered */ u32 rate_interval_us; /* saved rate sample: time elapsed */ u32 rcv_rtt_last_tsecr; + u32 delivered_ecn_bytes[3]; u64 first_tx_mstamp; /* start of window send phase */ u64 delivered_mstamp; /* time we reached "delivered" */ u64 bytes_acked; /* RFC4898 tcpEStatsAppHCThruOctetsAcked diff --git a/include/net/netns/ipv4.h b/include/net/netns/ipv4.h index 6373e3f17da8..4569a9ef4fb8 100644 --- a/include/net/netns/ipv4.h +++ b/include/net/netns/ipv4.h @@ -148,6 +148,7 @@ struct netns_ipv4 { struct local_ports ip_local_ports; u8 sysctl_tcp_ecn; + u8 sysctl_tcp_ecn_option; u8 sysctl_tcp_ecn_fallback; u8 sysctl_ip_default_ttl; diff --git a/include/net/tcp.h b/include/net/tcp.h index 101e07b8a089..50ad6e33d52d 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -204,6 +204,8 @@ static_assert((1 << ATO_BITS) > TCP_DELACK_MAX); #define TCPOPT_AO 29 /* Authentication Option (RFC5925) */ #define TCPOPT_MPTCP 30 /* Multipath TCP (RFC6824) */ #define TCPOPT_FASTOPEN 34 /* Fast open (RFC7413) */ +#define TCPOPT_ACCECN0 172 /* 0xAC: Accurate ECN Order 0 */ +#define TCPOPT_ACCECN1 174 /* 0xAE: Accurate ECN Order 1 */ #define TCPOPT_EXP 254 /* Experimental */ /* Magic number to be after the option value for sharing TCP * experimental options. See draft-ietf-tcpm-experimental-options-00.txt @@ -221,6 +223,7 @@ static_assert((1 << ATO_BITS) > TCP_DELACK_MAX); #define TCPOLEN_TIMESTAMP 10 #define TCPOLEN_MD5SIG 18 #define TCPOLEN_FASTOPEN_BASE 2 +#define TCPOLEN_ACCECN_BASE 2 #define TCPOLEN_EXP_FASTOPEN_BASE 4 #define TCPOLEN_EXP_SMC_BASE 6 @@ -234,6 +237,13 @@ static_assert((1 << ATO_BITS) > TCP_DELACK_MAX); #define TCPOLEN_MD5SIG_ALIGNED 20 #define TCPOLEN_MSS_ALIGNED 4 #define TCPOLEN_EXP_SMC_BASE_ALIGNED 8 +#define TCPOLEN_ACCECN_PERFIELD 3 + +/* Maximum number of byte counters in AccECN option + size */ +#define TCP_ACCECN_NUMFIELDS 3 +#define TCP_ACCECN_MAXSIZE (TCPOLEN_ACCECN_BASE + \ + TCPOLEN_ACCECN_PERFIELD * \ + TCP_ACCECN_NUMFIELDS) /* tp->accecn_fail_mode */ #define TCP_ACCECN_ACE_FAIL_SEND BIT(0) @@ -1064,11 +1074,21 @@ enum tcp_ecn_mode { TCP_ECN_IN_ACCECN_OUT_NOECN = 5, }; +/* AccECN option sending when AccECN has been successfully negotiated */ +enum tcp_ecn_option { + TCP_ECN_OPTION_DISABLED = 0, + TCP_ECN_OPTION_MINIMUM = 1, + TCP_ECN_OPTION_FULL = 2, +}; + static inline void tcp_accecn_init_counters(struct tcp_sock *tp) { tp->received_ce = 0; tp->received_ce_pending = 0; __tcp_accecn_init_bytes_counters(tp->received_ecn_bytes); + __tcp_accecn_init_bytes_counters(tp->delivered_ecn_bytes); + tp->accecn_minlen = 0; + tp->est_ecnfield = 0; } /* State flags for sacked in struct tcp_skb_cb */ diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h index bdac8c42fa82..53e0e85b52be 100644 --- a/include/uapi/linux/tcp.h +++ b/include/uapi/linux/tcp.h @@ -316,6 +316,13 @@ struct tcp_info { * in milliseconds, including any * unfinished recovery. */ + __u32 tcpi_received_ce; /* # of CE marks received */ + __u32 tcpi_delivered_e1_bytes; /* Accurate ECN byte counters */ + __u32 tcpi_delivered_e0_bytes; + __u32 tcpi_delivered_ce_bytes; + __u32 tcpi_received_e1_bytes; + __u32 tcpi_received_e0_bytes; + __u32 tcpi_received_ce_bytes; }; /* netlink attributes types for SCM_TIMESTAMPING_OPT_STATS */ diff --git a/net/ipv4/sysctl_net_ipv4.c b/net/ipv4/sysctl_net_ipv4.c index 75ec1a599b52..1d7fd86ca7b9 100644 --- a/net/ipv4/sysctl_net_ipv4.c +++ b/net/ipv4/sysctl_net_ipv4.c @@ -731,6 +731,15 @@ static struct ctl_table ipv4_net_table[] = { .extra1 = SYSCTL_ZERO, .extra2 = &tcp_ecn_mode_max, }, + { + .procname = "tcp_ecn_option", + .data = &init_net.ipv4.sysctl_tcp_ecn_option, + .maxlen = sizeof(u8), + .mode = 0644, + .proc_handler = proc_dou8vec_minmax, + .extra1 = SYSCTL_ZERO, + .extra2 = SYSCTL_TWO, + }, { .procname = "tcp_ecn_fallback", .data = &init_net.ipv4.sysctl_tcp_ecn_fallback, diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index af52ebfdd734..a71260838ccf 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -270,6 +270,7 @@ #include #include +#include #include #include #include @@ -4142,6 +4143,9 @@ void tcp_get_info(struct sock *sk, struct tcp_info *info) { const struct tcp_sock *tp = tcp_sk(sk); /* iff sk_type == SOCK_STREAM */ const struct inet_connection_sock *icsk = inet_csk(sk); + const u8 ect1_idx = INET_ECN_ECT_1 - 1; + const u8 ect0_idx = INET_ECN_ECT_0 - 1; + const u8 ce_idx = INET_ECN_CE - 1; unsigned long rate; u32 now; u64 rate64; @@ -4268,6 +4272,14 @@ void tcp_get_info(struct sock *sk, struct tcp_info *info) if (tp->rto_stamp) info->tcpi_total_rto_time += tcp_clock_ms() - tp->rto_stamp; + info->tcpi_received_ce = tp->received_ce; + info->tcpi_delivered_e1_bytes = tp->delivered_ecn_bytes[ect1_idx]; + info->tcpi_delivered_e0_bytes = tp->delivered_ecn_bytes[ect0_idx]; + info->tcpi_delivered_ce_bytes = tp->delivered_ecn_bytes[ce_idx]; + info->tcpi_received_e1_bytes = tp->received_ecn_bytes[ect1_idx]; + info->tcpi_received_e0_bytes = tp->received_ecn_bytes[ect0_idx]; + info->tcpi_received_ce_bytes = tp->received_ecn_bytes[ce_idx]; + unlock_sock_fast(sk, slow); } EXPORT_SYMBOL_GPL(tcp_get_info); @@ -5147,12 +5159,13 @@ static void __init tcp_struct_check(void) CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, rate_delivered); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, rate_interval_us); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, rcv_rtt_last_tsecr); + CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, delivered_ecn_bytes); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, first_tx_mstamp); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, delivered_mstamp); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, bytes_acked); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, rcv_rtt_est); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, rcvq_space); - CACHELINE_ASSERT_GROUP_SIZE(struct tcp_sock, tcp_sock_write_rx, 96); + CACHELINE_ASSERT_GROUP_SIZE(struct tcp_sock, tcp_sock_write_rx, 112); } void __init tcp_init(void) diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index e710ccf22dc8..3de8cc56c5cd 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -70,6 +70,7 @@ #include #include #include +#include #include #include #include @@ -520,6 +521,139 @@ static bool tcp_ecn_rcv_ecn_echo(const struct tcp_sock *tp, const struct tcphdr return false; } +/* Maps IP ECN field ECT/CE code point to AccECN option field number, given + * we are sending fields with Accurate ECN Order 1: ECT(1), CE, ECT(0). + */ +static u8 tcp_ecnfield_to_accecn_optfield(u8 ecnfield) +{ + switch (ecnfield & INET_ECN_MASK) { + case INET_ECN_NOT_ECT: + return 0; /* AccECN does not send counts of NOT_ECT */ + case INET_ECN_ECT_1: + return 1; + case INET_ECN_CE: + return 2; + case INET_ECN_ECT_0: + return 3; + } + return 0; +} + +/* Maps IP ECN field ECT/CE code point to AccECN option field value offset. + * Some fields do not start from zero, to detect zeroing by middleboxes. + */ +static u32 tcp_accecn_field_init_offset(u8 ecnfield) +{ + switch (ecnfield & INET_ECN_MASK) { + case INET_ECN_NOT_ECT: + return 0; /* AccECN does not send counts of NOT_ECT */ + case INET_ECN_ECT_1: + return TCP_ACCECN_E1B_INIT_OFFSET; + case INET_ECN_CE: + return TCP_ACCECN_CEB_INIT_OFFSET; + case INET_ECN_ECT_0: + return TCP_ACCECN_E0B_INIT_OFFSET; + } + return 0; +} + +/* Maps AccECN option field #nr to IP ECN field ECT/CE bits */ +static unsigned int tcp_accecn_optfield_to_ecnfield(unsigned int optfield, + bool order) +{ + u8 tmp; + + optfield = order ? 2 - optfield : optfield; + tmp = optfield + 2; + + return (tmp + (tmp >> 2)) & INET_ECN_MASK; +} + +/* Handles AccECN option ECT and CE 24-bit byte counters update into + * the u32 value in tcp_sock. As we're processing TCP options, it is + * safe to access from - 1. + */ +static s32 tcp_update_ecn_bytes(u32 *cnt, const char *from, u32 init_offset) +{ + u32 truncated = (get_unaligned_be32(from - 1) - init_offset) & + 0xFFFFFFU; + u32 delta = (truncated - *cnt) & 0xFFFFFFU; + + /* If delta has the highest bit set (24th bit) indicating + * negative, sign extend to correct an estimation using + * sign_extend32(delta, 24 - 1) + */ + delta = sign_extend32(delta, 23); + *cnt += delta; + return (s32)delta; +} + +/* Returns true if the byte counters can be used */ +static bool tcp_accecn_process_option(struct tcp_sock *tp, + const struct sk_buff *skb, + u32 delivered_bytes, int flag) +{ + u8 estimate_ecnfield = tp->est_ecnfield; + bool ambiguous_ecn_bytes_incr = false; + bool first_changed = false; + unsigned int optlen; + bool order1, res; + unsigned int i; + u8 *ptr; + + if (!(flag & FLAG_SLOWPATH) || !tp->rx_opt.accecn) { + if (estimate_ecnfield) { + u8 ecnfield = estimate_ecnfield - 1; + + tp->delivered_ecn_bytes[ecnfield] += delivered_bytes; + return true; + } + return false; + } + + ptr = skb_transport_header(skb) + tp->rx_opt.accecn; + optlen = ptr[1] - 2; + WARN_ON_ONCE(ptr[0] != TCPOPT_ACCECN0 && ptr[0] != TCPOPT_ACCECN1); + order1 = (ptr[0] == TCPOPT_ACCECN1); + ptr += 2; + + res = !!estimate_ecnfield; + for (i = 0; i < 3; i++) { + if (optlen < TCPOLEN_ACCECN_PERFIELD) + continue; + + u32 init_offset; + u8 ecnfield; + s32 delta; + u32 *cnt; + + ecnfield = tcp_accecn_optfield_to_ecnfield(i, order1); + init_offset = tcp_accecn_field_init_offset(ecnfield); + cnt = &tp->delivered_ecn_bytes[ecnfield - 1]; + delta = tcp_update_ecn_bytes(cnt, ptr, init_offset); + if (delta && delta < 0) { + res = false; + ambiguous_ecn_bytes_incr = true; + } + if (delta && ecnfield != estimate_ecnfield) { + if (!first_changed) { + tp->est_ecnfield = ecnfield; + first_changed = true; + } else { + res = false; + ambiguous_ecn_bytes_incr = true; + } + } + + optlen -= TCPOLEN_ACCECN_PERFIELD; + ptr += TCPOLEN_ACCECN_PERFIELD; + } + if (ambiguous_ecn_bytes_incr) + tp->est_ecnfield = 0; + + return res; +} + static void tcp_count_delivered_ce(struct tcp_sock *tp, u32 ecn_count) { tp->delivered_ce += ecn_count; @@ -536,7 +670,8 @@ static void tcp_count_delivered(struct tcp_sock *tp, u32 delivered, /* Returns the ECN CE delta */ static u32 __tcp_accecn_process(struct sock *sk, const struct sk_buff *skb, - u32 delivered_pkts, int flag) + u32 delivered_pkts, u32 delivered_bytes, + int flag) { const struct tcphdr *th = tcp_hdr(skb); struct tcp_sock *tp = tcp_sk(sk); @@ -547,6 +682,8 @@ static u32 __tcp_accecn_process(struct sock *sk, const struct sk_buff *skb, if (!(flag & (FLAG_FORWARD_PROGRESS | FLAG_TS_PROGRESS))) return 0; + tcp_accecn_process_option(tp, skb, delivered_bytes, flag); + if (!(flag & FLAG_SLOWPATH)) { /* AccECN counter might overflow on large ACKs */ if (delivered_pkts <= TCP_ACCECN_CEP_ACE_MASK) @@ -572,12 +709,14 @@ static u32 __tcp_accecn_process(struct sock *sk, const struct sk_buff *skb, } static u32 tcp_accecn_process(struct sock *sk, const struct sk_buff *skb, - u32 delivered_pkts, int *flag) + u32 delivered_pkts, u32 delivered_bytes, + int *flag) { struct tcp_sock *tp = tcp_sk(sk); u32 delta; - delta = __tcp_accecn_process(sk, skb, delivered_pkts, *flag); + delta = __tcp_accecn_process(sk, skb, delivered_pkts, + delivered_bytes, *flag); if (delta > 0) { tcp_count_delivered_ce(tp, delta); *flag |= FLAG_ECE; @@ -4241,6 +4380,7 @@ static int tcp_ack(struct sock *sk, const struct sk_buff *skb, int flag) if (tcp_ecn_mode_accecn(tp)) ecn_count = tcp_accecn_process(sk, skb, tp->delivered - delivered, + sack_state.delivered_bytes, &flag); tcp_in_ack_event(sk, flag); @@ -4280,6 +4420,7 @@ static int tcp_ack(struct sock *sk, const struct sk_buff *skb, int flag) if (tcp_ecn_mode_accecn(tp)) ecn_count = tcp_accecn_process(sk, skb, tp->delivered - delivered, + sack_state.delivered_bytes, &flag); tcp_in_ack_event(sk, flag); /* If data was DSACKed, see if we can undo a cwnd reduction. */ @@ -4407,6 +4548,7 @@ void tcp_parse_options(const struct net *net, ptr = (const unsigned char *)(th + 1); opt_rx->saw_tstamp = 0; + opt_rx->accecn = 0; opt_rx->saw_unknown = 0; while (length > 0) { @@ -4498,6 +4640,12 @@ void tcp_parse_options(const struct net *net, ptr, th->syn, foc, false); break; + case TCPOPT_ACCECN0: + case TCPOPT_ACCECN1: + /* Save offset of AccECN option in TCP header */ + opt_rx->accecn = (ptr - 2) - (__u8 *)th; + break; + case TCPOPT_EXP: /* Fast Open option shares code 254 using a * 16 bits magic number. @@ -4558,11 +4706,14 @@ static bool tcp_fast_parse_options(const struct net *net, */ if (th->doff == (sizeof(*th) / 4)) { tp->rx_opt.saw_tstamp = 0; + tp->rx_opt.accecn = 0; return false; } else if (tp->rx_opt.tstamp_ok && th->doff == ((sizeof(*th) + TCPOLEN_TSTAMP_ALIGNED) / 4)) { - if (tcp_parse_aligned_timestamp(tp, th)) + if (tcp_parse_aligned_timestamp(tp, th)) { + tp->rx_opt.accecn = 0; return true; + } } tcp_parse_options(net, skb, &tp->rx_opt, 1, NULL); @@ -6162,8 +6313,12 @@ void tcp_ecn_received_counters(struct sock *sk, const struct sk_buff *skb, tp->received_ce_pending = min(tp->received_ce_pending + pcount, 0xfU); - if (payload_len > 0) + if (payload_len > 0) { + u8 minlen = tcp_ecnfield_to_accecn_optfield(ecnfield); tp->received_ecn_bytes[ecnfield - 1] += payload_len; + tp->accecn_minlen = max_t(u8, tp->accecn_minlen, + minlen); + } } } @@ -6383,6 +6538,7 @@ void tcp_rcv_established(struct sock *sk, struct sk_buff *skb) */ tp->rx_opt.saw_tstamp = 0; + tp->rx_opt.accecn = 0; /* pred_flags is 0xS?10 << 16 + snd_wnd * if header_prediction is to be made diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index 5cec04ed6a8e..cc3049bb9d02 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -3450,6 +3450,7 @@ static void __net_init tcp_set_hashinfo(struct net *net) static int __net_init tcp_sk_init(struct net *net) { net->ipv4.sysctl_tcp_ecn = TCP_ECN_IN_ECN_OUT_NOECN; + net->ipv4.sysctl_tcp_ecn_option = TCP_ECN_OPTION_FULL; net->ipv4.sysctl_tcp_ecn_fallback = 1; net->ipv4.sysctl_tcp_base_mss = TCP_BASE_MSS; diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index d7fef3d2698b..0a4d9791b4a2 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -492,6 +492,7 @@ static inline bool tcp_urg_mode(const struct tcp_sock *tp) #define OPTION_SMC BIT(9) #define OPTION_MPTCP BIT(10) #define OPTION_AO BIT(11) +#define OPTION_ACCECN BIT(12) static void smc_options_write(__be32 *ptr, u16 *options) { @@ -513,12 +514,14 @@ struct tcp_out_options { u16 mss; /* 0 to disable */ u8 ws; /* window scale, 0 to disable */ u8 num_sack_blocks; /* number of SACK blocks to include */ + u8 num_accecn_fields; /* number of AccECN fields needed */ u8 hash_size; /* bytes in hash_location */ u8 bpf_opt_len; /* length of BPF hdr option */ __u8 *hash_location; /* temporary pointer, overloaded */ __u32 tsval, tsecr; /* need to include OPTION_TS */ struct tcp_fastopen_cookie *fastopen_cookie; /* Fast open cookie */ struct mptcp_out_options mptcp; + u32 *ecn_bytes; /* AccECN ECT/CE byte counters */ }; static void mptcp_options_write(struct tcphdr *th, __be32 *ptr, @@ -710,6 +713,8 @@ static __be32 *process_tcp_ao_options(struct tcp_sock *tp, return ptr; } +#define NOP_LEFTOVER ((TCPOPT_NOP << 8) | TCPOPT_NOP) + /* Write previously computed TCP options to the packet. * * Beware: Something in the Internet is very sensitive to the ordering of @@ -728,8 +733,10 @@ static void tcp_options_write(struct tcphdr *th, struct tcp_sock *tp, struct tcp_out_options *opts, struct tcp_key *key) { + u16 leftover_bytes = NOP_LEFTOVER; /* replace next NOPs if avail */ __be32 *ptr = (__be32 *)(th + 1); u16 options = opts->options; /* mungable copy */ + int leftover_size = 2; if (tcp_key_is_md5(key)) { *ptr++ = htonl((TCPOPT_NOP << 24) | (TCPOPT_NOP << 16) | @@ -763,18 +770,64 @@ static void tcp_options_write(struct tcphdr *th, struct tcp_sock *tp, *ptr++ = htonl(opts->tsecr); } + if (OPTION_ACCECN & options) { + const u8 ect0_idx = INET_ECN_ECT_0 - 1; + const u8 ect1_idx = INET_ECN_ECT_1 - 1; + const u8 ce_idx = INET_ECN_CE - 1; + u32 e0b; + u32 e1b; + u32 ceb; + u8 len; + + e0b = opts->ecn_bytes[ect0_idx] + TCP_ACCECN_E0B_INIT_OFFSET; + e1b = opts->ecn_bytes[ect1_idx] + TCP_ACCECN_E1B_INIT_OFFSET; + ceb = opts->ecn_bytes[ce_idx] + TCP_ACCECN_CEB_INIT_OFFSET; + len = TCPOLEN_ACCECN_BASE + + opts->num_accecn_fields * TCPOLEN_ACCECN_PERFIELD; + + if (opts->num_accecn_fields == 2) { + *ptr++ = htonl((TCPOPT_ACCECN1 << 24) | (len << 16) | + ((e1b >> 8) & 0xffff)); + *ptr++ = htonl(((e1b & 0xff) << 24) | + (ceb & 0xffffff)); + } else if (opts->num_accecn_fields == 1) { + *ptr++ = htonl((TCPOPT_ACCECN1 << 24) | (len << 16) | + ((e1b >> 8) & 0xffff)); + leftover_bytes = ((e1b & 0xff) << 8) | + TCPOPT_NOP; + leftover_size = 1; + } else if (opts->num_accecn_fields == 0) { + leftover_bytes = (TCPOPT_ACCECN1 << 8) | len; + leftover_size = 2; + } else if (opts->num_accecn_fields == 3) { + *ptr++ = htonl((TCPOPT_ACCECN1 << 24) | (len << 16) | + ((e1b >> 8) & 0xffff)); + *ptr++ = htonl(((e1b & 0xff) << 24) | + (ceb & 0xffffff)); + *ptr++ = htonl(((e0b & 0xffffff) << 8) | + TCPOPT_NOP); + } + if (tp) + tp->accecn_minlen = 0; + } + if (unlikely(OPTION_SACK_ADVERTISE & options)) { - *ptr++ = htonl((TCPOPT_NOP << 24) | - (TCPOPT_NOP << 16) | + *ptr++ = htonl((leftover_bytes << 16) | (TCPOPT_SACK_PERM << 8) | TCPOLEN_SACK_PERM); + leftover_bytes = NOP_LEFTOVER; } if (unlikely(OPTION_WSCALE & options)) { - *ptr++ = htonl((TCPOPT_NOP << 24) | + u8 highbyte = TCPOPT_NOP; + + if (unlikely(leftover_size == 1)) + highbyte = leftover_bytes >> 8; + *ptr++ = htonl((highbyte << 24) | (TCPOPT_WINDOW << 16) | (TCPOLEN_WINDOW << 8) | opts->ws); + leftover_bytes = NOP_LEFTOVER; } if (unlikely(opts->num_sack_blocks)) { @@ -782,8 +835,7 @@ static void tcp_options_write(struct tcphdr *th, struct tcp_sock *tp, tp->duplicate_sack : tp->selective_acks; int this_sack; - *ptr++ = htonl((TCPOPT_NOP << 24) | - (TCPOPT_NOP << 16) | + *ptr++ = htonl((leftover_bytes << 16) | (TCPOPT_SACK << 8) | (TCPOLEN_SACK_BASE + (opts->num_sack_blocks * TCPOLEN_SACK_PERBLOCK))); @@ -795,6 +847,10 @@ static void tcp_options_write(struct tcphdr *th, struct tcp_sock *tp, } tp->rx_opt.dsack = 0; + } else if (unlikely(leftover_bytes != NOP_LEFTOVER)) { + *ptr++ = htonl((leftover_bytes << 16) | + (TCPOPT_NOP << 8) | + TCPOPT_NOP); } if (unlikely(OPTION_FAST_OPEN_COOKIE & options)) { @@ -875,6 +931,60 @@ static void mptcp_set_option_cond(const struct request_sock *req, } } +/* Initial values for AccECN option, ordered is based on ECN field bits + * similar to received_ecn_bytes. Used for SYN/ACK AccECN option. + */ +static u32 synack_ecn_bytes[3] = { 0, 0, 0 }; + +static u32 tcp_synack_options_combine_saving(struct tcp_out_options *opts) +{ + /* How much there's room for combining with the alignment padding? */ + if ((opts->options & (OPTION_SACK_ADVERTISE | OPTION_TS)) == + OPTION_SACK_ADVERTISE) + return 2; + else if (opts->options & OPTION_WSCALE) + return 1; + return 0; +} + +/* Calculates how long AccECN option will fit to @remaining option space. + * + * AccECN option can sometimes replace NOPs used for alignment of other + * TCP options (up to @max_combine_saving available). + * + * Only solutions with at least @required AccECN fields are accepted. + * + * Returns: The size of the AccECN option excluding space repurposed from + * the alignment of the other options. + */ +static int tcp_options_fit_accecn(struct tcp_out_options *opts, int required, + int remaining, int max_combine_saving) +{ + int size = TCP_ACCECN_MAXSIZE; + + opts->num_accecn_fields = TCP_ACCECN_NUMFIELDS; + + while (opts->num_accecn_fields >= required) { + int leftover_size = size & 0x3; + /* Pad to dword if cannot combine */ + if (leftover_size > max_combine_saving) + leftover_size = -((4 - leftover_size) & 0x3); + + if (remaining >= size - leftover_size) { + size -= leftover_size; + break; + } + + opts->num_accecn_fields--; + size -= TCPOLEN_ACCECN_PERFIELD; + } + if (opts->num_accecn_fields < required) + return 0; + + opts->options |= OPTION_ACCECN; + return size; +} + /* Compute TCP options for SYN packets. This is not the final * network wire format yet. */ @@ -957,6 +1067,17 @@ static unsigned int tcp_syn_options(struct sock *sk, struct sk_buff *skb, } } + /* Simultaneous open SYN/ACK needs AccECN option but not SYN */ + if (unlikely((TCP_SKB_CB(skb)->tcp_flags & TCPHDR_ACK) && + tcp_ecn_mode_accecn(tp) && + sock_net(sk)->ipv4.sysctl_tcp_ecn_option && + remaining >= TCPOLEN_ACCECN_BASE)) { + u32 saving = tcp_synack_options_combine_saving(opts); + + opts->ecn_bytes = synack_ecn_bytes; + remaining -= tcp_options_fit_accecn(opts, 0, remaining, saving); + } + bpf_skops_hdr_opt_len(sk, skb, NULL, NULL, 0, opts, &remaining); return MAX_TCP_OPTION_SPACE - remaining; @@ -974,6 +1095,7 @@ static unsigned int tcp_synack_options(const struct sock *sk, { struct inet_request_sock *ireq = inet_rsk(req); unsigned int remaining = MAX_TCP_OPTION_SPACE; + struct tcp_request_sock *treq = tcp_rsk(req); if (tcp_key_is_md5(key)) { opts->options |= OPTION_MD5; @@ -1036,6 +1158,14 @@ static unsigned int tcp_synack_options(const struct sock *sk, smc_set_option_cond(tcp_sk(sk), ireq, opts, &remaining); + if (treq->accecn_ok && sock_net(sk)->ipv4.sysctl_tcp_ecn_option && + remaining >= TCPOLEN_ACCECN_BASE) { + u32 saving = tcp_synack_options_combine_saving(opts); + + opts->ecn_bytes = synack_ecn_bytes; + remaining -= tcp_options_fit_accecn(opts, 0, remaining, saving); + } + bpf_skops_hdr_opt_len((struct sock *)sk, skb, req, syn_skb, synack_type, opts, &remaining); @@ -1106,6 +1236,17 @@ static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb opts->num_sack_blocks = 0; } + if (tcp_ecn_mode_accecn(tp) && + sock_net(sk)->ipv4.sysctl_tcp_ecn_option) { + int saving = opts->num_sack_blocks > 0 ? 2 : 0; + int remaining = MAX_TCP_OPTION_SPACE - size; + + opts->ecn_bytes = tp->received_ecn_bytes; + size += tcp_options_fit_accecn(opts, tp->accecn_minlen, + remaining, + saving); + } + if (unlikely(BPF_SOCK_OPS_TEST_FLAG(tp, BPF_SOCK_OPS_WRITE_HDR_OPT_CB_FLAG))) { unsigned int remaining = MAX_TCP_OPTION_SPACE - size; From patchwork Wed May 14 13:56:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Chia-Yu Chang \(Nokia\)" X-Patchwork-Id: 890047 Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2088.outbound.protection.outlook.com [40.107.20.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F37FB204F9B; Wed, 14 May 2025 13:57:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.20.88 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231039; cv=fail; b=FHuOoLzuK4Lx/VVybdXAVI4YXJCg/LdAKHHgam00uyFBivOCdCfXkqCorG7SHRaiUjA12oyzBdri+gs56YoCiUIUoQbCKPln35PBnc5Hq2WehuEOOKzeAGIuFCVlFxPXZHBTAPk/Mh+ZwZyKbO6EC4sZwTsGkpIBcDCWj2ys8Hc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231039; c=relaxed/simple; bh=1pCiRVrGhv4R0XrPgS61RGmSnDL2HVJVzufoYbtWiTY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=Lc3lchbHWAUcn2k1GlABELorj/H6h6y4cA6OietC2YjMcy/rVYF5rikOxd05doMGSom7Pidc0f9OA/1d+GP4bWC6T8JuIPd9i5J0OaNAEtapBShpL/9P9JRCsAqtHf5Mq+3oGxKL9PcuftTrBrpQ2GP0/yG6YLTsqxJ45aIRxwg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com; spf=fail smtp.mailfrom=nokia-bell-labs.com; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b=OdG64JhN; arc=fail smtp.client-ip=40.107.20.88 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b="OdG64JhN" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kcx4nYqGTTdz1CS63k8tuWgJAID5/VVSG0WVu2uAtPnH9RcPYoYiN/kkVUkqnEGeEyRgYd+6Wmuk9GVCAwatgKVuQarC79YEnYzeoNzseyYfcktkEPUixK1KrlF6i5d8G7HngRBr3zEJHfE02Jr+p8+qIn9gFlKCs2qD9CDY0ZIyQrqyg4afXdKpLhfPfMn0ksUcqr+/LVFqTLJ4xaTjpeLgu2BtjyoT4pyDxU3QoEWNIg7v9m4g5ZO5GqnIxgYfpp0cBlzQjkyhq9ZhWNWrmw/++a9ms6SU4XheH7Mlq6za/767HMK1TM2F5prrrg9xsg+LoGbC8X0sKWZeUe+eJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PoGNrcdrYXgWHVsxSZNFPxnoXNEmlUXIofckmi/u/Yk=; b=MPpA527dHKiTFhGnFKXKUv1ZQMMnaSTdkYxTyruU4h1NDmWc84kfB964CUEM4dj+B8a8yAYOOEp20wdsXCxtxVtd8gM8aytI1yxQkZqIfPSuLPHIrX6QPv9xCcYWZhBfdhdyVI+WB0evV9HPQ4/9kR/4Jw1gJ4GsOyQiqRxxfCQzjmH+YS/+OM0fwvWmalmCmdO4scLSpfgcHJjcxU+TCfcKbaIbs73Veo6mVfagEp2wgX9hc/31XRTVo9VhVP41udX70dfQhi7YnVQNzImbVvVercVyxoxQrluaktt7umXg79CkaDX4MZ0qcFOhrMAcGs5svIQUos1k2qX+0oxzmw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 131.228.2.240) smtp.rcpttodomain=amazon.com smtp.mailfrom=nokia-bell-labs.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nokia-bell-labs.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia-bell-labs.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PoGNrcdrYXgWHVsxSZNFPxnoXNEmlUXIofckmi/u/Yk=; b=OdG64JhN2c15HYaQ6sm5tdb0iRORz/0S8oerK9bMxKqM8aiy4+ih9mDoU3a4tLZFPecaJm0dwFXbiE1vvNiQXrGM/RfAWsx02R1/TopVTZdHuYaLd5YhdfbfRrWOzGvm3ding//dzjNhxzIOEmoerxCT0tJS1ZzF94eutGDAdNNRdMPovOHqQJbEPNGvSqksF3qcQN2AHWv042RotP+D4pPCzrIs7U+0rcEUoqzBYwufU74mZigBnMIfcNztNbX5NcQYcWKsbPPOGN3KshHmdaM5N+E3jvh3iDBxfSU/4MwWLn4LsG6ZxoeUn8P+s2rXlQg6SvJPLeMwOebuQa4gjQ== Received: from AS8PR04CA0165.eurprd04.prod.outlook.com (2603:10a6:20b:331::20) by DBAPR07MB6888.eurprd07.prod.outlook.com (2603:10a6:10:194::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8722.31; Wed, 14 May 2025 13:57:13 +0000 Received: from AM3PEPF0000A78E.eurprd04.prod.outlook.com (2603:10a6:20b:331:cafe::f8) by AS8PR04CA0165.outlook.office365.com (2603:10a6:20b:331::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8746.17 via Frontend Transport; Wed, 14 May 2025 13:57:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 131.228.2.240) smtp.mailfrom=nokia-bell-labs.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nokia-bell-labs.com; Received-SPF: Pass (protection.outlook.com: domain of nokia-bell-labs.com designates 131.228.2.240 as permitted sender) receiver=protection.outlook.com; client-ip=131.228.2.240; helo=fihe3nok0735.emea.nsn-net.net; pr=C Received: from fihe3nok0735.emea.nsn-net.net (131.228.2.240) by AM3PEPF0000A78E.mail.protection.outlook.com (10.167.16.117) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8722.18 via Frontend Transport; Wed, 14 May 2025 13:57:12 +0000 Received: from sarah.nbl.nsn-rdnet.net (sarah.nbl.nsn-rdnet.net [10.0.73.150]) by fihe3nok0735.emea.nsn-net.net (Postfix) with ESMTP id 3613520094; Wed, 14 May 2025 16:57:11 +0300 (EEST) From: chia-yu.chang@nokia-bell-labs.com To: linux-doc@vger.kernel.org, corbet@lwn.net, horms@kernel.org, dsahern@kernel.org, kuniyu@amazon.com, bpf@vger.kernel.org, netdev@vger.kernel.org, dave.taht@gmail.com, pabeni@redhat.com, jhs@mojatatu.com, kuba@kernel.org, stephen@networkplumber.org, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, andrew+netdev@lunn.ch, donald.hunter@gmail.com, ast@fiberby.net, liuhangbin@gmail.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, ij@kernel.org, ncardwell@google.com, koen.de_schepper@nokia-bell-labs.com, g.white@cablelabs.com, ingemar.s.johansson@ericsson.com, mirja.kuehlewind@ericsson.com, cheshire@apple.com, rs.ietf@gmx.at, Jason_Livingood@comcast.com, vidhi_goel@apple.com Cc: Chia-Yu Chang Subject: [PATCH v7 net-next 12/15] tcp: accecn: AccECN option ceb/cep heuristic Date: Wed, 14 May 2025 15:56:39 +0200 Message-Id: <20250514135642.11203-13-chia-yu.chang@nokia-bell-labs.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> References: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM3PEPF0000A78E:EE_|DBAPR07MB6888:EE_ X-MS-Office365-Filtering-Correlation-Id: 7297405a-a82c-419b-2bb5-08dd92ef3a2b X-LD-Processed: 5d471751-9675-428d-917b-70f44f9630b0,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700013|1800799024|7416014|82310400026|376014|921020; X-Microsoft-Antispam-Message-Info: =?utf-8?q?R38Xoy+z2ETkyyOLHaMfxFkfWGpXLGi?= =?utf-8?q?nDKveNSKmbqSmKAo2bgP6fhZho1yrqjZEYtXK53BMe5kjjOBhma2O8AymigwgDm+t?= =?utf-8?q?kKcyNjEEytsMfE+HmnnlktTHWf74yYk4MC+WAD/nBB12GcBotAO13DsBqiIKE3V6o?= =?utf-8?q?CIHZfDHIRzBy8azbanoqwXwq9Jj1Wjd4qkjt3R12Ea4fgt3+QdKuuc/JCShw10vQM?= =?utf-8?q?FR+GSZ++5C6Lj0n15HtgNngfPZCxf5lqQGmBHjTR2Y4hF9VSjudcYi+t4fldLbBrx?= =?utf-8?q?nre9a1bf4xMfKd18pnj9rxYQRDA3o10j55KJRKSuYl4ClVpB6piEx3YAh9mNzbVWR?= =?utf-8?q?vtEhxhrFAAod1gWKIChOsgmedywbYS1A4Fm6yMxY0TsSZyy792ABj3VYzhzNjzOMX?= =?utf-8?q?TNJOhKsALTnRws2YuGqvTjjFCBMfyfWhyTxxP9Raic9jG2oju1am1b7p1zc9RiaHC?= =?utf-8?q?7lZS80YABdNZjwynlEWwovin/qUZY0fTDpV1WTrClX6mKYaxsVGbPVNqLQz8KKOI2?= =?utf-8?q?BkOumIwgkf6vX9HgzCg+m2T9OnJnyvvFmm50E4Q3RKDvgGaYQwWHOVtDJVSRkuIAK?= =?utf-8?q?LqF6CqHgBgw+az4/Zz2fgkx7cNK2dpnhHCvhvEGLekVgr7FW80VJ3bIU0l33rEyRW?= =?utf-8?q?zRNWYDg45yhe81Xrztydv/Wb9soJWrmWDTHwfzzJ4lhcf9DQXWqfR1Z2m26vYAa2I?= =?utf-8?q?B4M8x0rsOj4ii77nOlXmVWuJFxC2BTXu8CufFP2PT0WopHEtrleE+XYhQEpNnpzg6?= =?utf-8?q?d0O3VaF3GABwwWSMXG99d5wrJDpZcuNM+LYyrYNmN1qLgtgQJ7NNIpp5ZuHbvCZyW?= =?utf-8?q?WptXInXWBB8GVpEaYU6DZNPpJyq0U2vq8sF/M8m/6rCRU236XwSBt4tpJC6JPENWR?= =?utf-8?q?7qmiISBdJVfOLsXP9vSMIfp00DsE97vcq9A5zYbmphUdSVnTXn6/0SteHh4kFBuMd?= =?utf-8?q?uYdZHK5eK4042aQd5kIZBC4AUXcuaiLBPRa+n8wMMkUkC2EX/G9rPYOJe33DJsiTQ?= =?utf-8?q?Yy0k9wirQC9kPJeq3DGz/Ws+i9QxhJXF9ut4YZ8/vEmp9TPVuaHEhJze7oCBK4vvx?= =?utf-8?q?oYC2FZn2DhXoail0V2L7JcHUBXS1GyLWnIx2+f2poOtTh4Tp0p+qPcAy0iMgoirLH?= =?utf-8?q?obbfBq+W0HsKb1z3OvVipcZLQPo2V1yZG1huT8NbI0bHRO1B4u3PjNL4P8c0OwCTo?= =?utf-8?q?yHLrj0LQ7Juf034ZVTyB5YO4R0Vr4LOjM430e2pa+IMzE2XWzZWWO4XeeKIHjIasP?= =?utf-8?q?5bvctHnP9okgE6FomlshXlsD7V4qHD903qmr9tBMkdQiOQqEK+QkFcOBsN+b8SRyG?= =?utf-8?q?1BPZMJbFoLQXujAsHHHJmQIuNzZ3L2iY/cCclF+TthbLnw8ojak9zAVLNaiIqhpyJ?= =?utf-8?q?Xe3vAzSOYlVOwx3EtZ/z1nJ24CWMEszn5i0inoOAJnSUYjHJK65OIa5j3/QT2KbLr?= =?utf-8?q?K3tHYrNiSlMw5DcflK/1wPVz5L9tBh7w=3D=3D?= X-Forefront-Antispam-Report: CIP:131.228.2.240; CTRY:FI; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:fihe3nok0735.emea.nsn-net.net; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(36860700013)(1800799024)(7416014)(82310400026)(376014)(921020); DIR:OUT; SFP:1101; X-OriginatorOrg: nokia-bell-labs.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2025 13:57:12.6514 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7297405a-a82c-419b-2bb5-08dd92ef3a2b X-MS-Exchange-CrossTenant-Id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5d471751-9675-428d-917b-70f44f9630b0; Ip=[131.228.2.240]; Helo=[fihe3nok0735.emea.nsn-net.net] X-MS-Exchange-CrossTenant-AuthSource: AM3PEPF0000A78E.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR07MB6888 From: Ilpo Järvinen The heuristic algorithm from draft-11 Appendix A.2.2 to mitigate against false ACE field overflows. Signed-off-by: Ilpo Järvinen Signed-off-by: Chia-Yu Chang --- include/net/tcp.h | 1 + net/ipv4/tcp_input.c | 18 ++++++++++++++++-- 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/include/net/tcp.h b/include/net/tcp.h index 5e4593e39de4..201326b10396 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -247,6 +247,7 @@ static_assert((1 << ATO_BITS) > TCP_DELACK_MAX); #define TCP_ACCECN_MAXSIZE (TCPOLEN_ACCECN_BASE + \ TCPOLEN_ACCECN_PERFIELD * \ TCP_ACCECN_NUMFIELDS) +#define TCP_ACCECN_SAFETY_SHIFT 1 /* SAFETY_FACTOR in accecn draft */ /* tp->accecn_fail_mode */ #define TCP_ACCECN_ACE_FAIL_SEND BIT(0) diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index c93e4bffb23e..f206c1a93a56 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -709,16 +709,19 @@ static u32 __tcp_accecn_process(struct sock *sk, const struct sk_buff *skb, u32 delivered_pkts, u32 delivered_bytes, int flag) { + u32 old_ceb = tcp_sk(sk)->delivered_ecn_bytes[INET_ECN_CE - 1]; const struct tcphdr *th = tcp_hdr(skb); struct tcp_sock *tp = tcp_sk(sk); - u32 delta, safe_delta; + u32 delta, safe_delta, d_ceb; + bool opt_deltas_valid; u32 corrected_ace; /* Reordered ACK or uncertain due to lack of data to send and ts */ if (!(flag & (FLAG_FORWARD_PROGRESS | FLAG_TS_PROGRESS))) return 0; - tcp_accecn_process_option(tp, skb, delivered_bytes, flag); + opt_deltas_valid = tcp_accecn_process_option(tp, skb, + delivered_bytes, flag); if (!(flag & FLAG_SLOWPATH)) { /* AccECN counter might overflow on large ACKs */ @@ -741,6 +744,17 @@ static u32 __tcp_accecn_process(struct sock *sk, const struct sk_buff *skb, safe_delta = delivered_pkts - ((delivered_pkts - delta) & TCP_ACCECN_CEP_ACE_MASK); + if (opt_deltas_valid) { + d_ceb = tp->delivered_ecn_bytes[INET_ECN_CE - 1] - old_ceb; + if (!d_ceb) + return delta; + if (d_ceb > delta * tp->mss_cache) + return safe_delta; + if (d_ceb < + safe_delta * tp->mss_cache >> TCP_ACCECN_SAFETY_SHIFT) + return delta; + } + return safe_delta; } From patchwork Wed May 14 13:56:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Chia-Yu Chang \(Nokia\)" X-Patchwork-Id: 890046 Received: from MRWPR03CU001.outbound.protection.outlook.com (mail-francesouthazon11011059.outbound.protection.outlook.com [40.107.130.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6CF6283FD4; Wed, 14 May 2025 13:57:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.130.59 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231042; cv=fail; b=aAGfHHLegN7zuPKY/GUD8yoIL81+k/7CJUkFRcBdX0Pl1RFiSJyH5qGBUhbIzI3aK9fkmBsc6yDnHRuVHAFF2tQFDOqQbE3VYaY4eqYg3GcG9oISaEpaXfM772jUkgZaOnJ7RgnyS0QI3hM+SV8Tdg5ZDW7ros7ABnr62mD35eU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231042; c=relaxed/simple; bh=fzdmbMlDFSfz9D5JclJq+MMfeINB6U5xwyzRoWOoF7Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=cBii0xnwerMVCtVjfGra07Jgwb6GJTOk+c+HwFcWcqabq2tRR5kHRlKf/kFG6Rlx40kNzKoYOwDLZvhloM0ljz2+37s6T/9gNzMDRhzNhXcUkrp9FIv36qk8M/wzSYbqougITRhrdrD6pdbmY/ZAeI4GJeDHBrBYFSQRH/yUGKM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com; spf=fail smtp.mailfrom=nokia-bell-labs.com; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b=U6i3oinL; arc=fail smtp.client-ip=40.107.130.59 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b="U6i3oinL" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=K+BOJjHpVd6o4gA2PlrUDH7+UXi8Lv8kavIbWuMtds3vXjh11ttAaTEF24EUKB9y1C4mu3H3jmMmJUok6OQab7IWrObA1SvsAkPSZW3fzbi4YZWXBvsp6OIVgxRseCp2YXeMu6ohvb5H6N/OsJ/ocRJo9IunaMspojJB92TjJIPq/5hsajNpxojiFpnbyu3CMgxsLP/FIWfbzdmGWy+0rt11qkr02+n1I/mU+fYntKgYSMkSEuT+7TQvYOSQKNuchpl+BPJ9kaEat+9G0TBD2OKbroNrgjLuNKYVa5L4hkHe1VdQvX2qIaSYbsp/BHbBdGvhgBzbvn4R8B/3DffnZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zDrl+/6MKZU6clhwC3c3K9/dYY3Jj2BGOr7YXH5GBCg=; b=AUXMNyESsdz4Axc4kgpD/IiKfMKFgjX0bDgxb1oWkl6MRcHk93c4VTARAR69D/vEKT873hegsGVIhu15tYl6h15mtWkhTPH8Qm8y846s/GXKI7RwxCevZIo929HihLN+S7FttU6yn+43vC178CagtAmy17ji9kr7m16J0CPyI2qwYA5ErkKciA6wEGYiKSFgE9K5r48xdaehjI4rAYPaebkeaqkekVM+9+xiW2OU2suuT+YQjdwMK/ebog6d2rHkJ2GOInHFrtPYiQNrOtHBig/8n9qG51FuRrpue1XkyCJNBWeiEF3KzK+Ashptg7mMLf+1HnTTy6ub1pHvBeHixw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 131.228.2.240) smtp.rcpttodomain=amazon.com smtp.mailfrom=nokia-bell-labs.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nokia-bell-labs.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia-bell-labs.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zDrl+/6MKZU6clhwC3c3K9/dYY3Jj2BGOr7YXH5GBCg=; b=U6i3oinLAGDkfSV3XKqO19nnmGEW0qiodkrf7r3bWQAjI3jwPLCxkmuwfz/iMbf1P7ZI5L1V8jFJHjSZExS2YepvM/DXcu2m4WLDe+MI7B6kOGlGW8/wzbhbAPyc3RG4t8fIfn+MlMRjLK6Uey7yIg2r1MuoTwHLhzpyU/SL1UdJ0F0xaZAp0p0xHg4Dk7cMRnsl+IVgl++TnCHbTnKpRLHDelHD2RcKN/QZ0R0mEvGVXkhsOyiR/i0FC/x5A+AKV+EzSIwqzg2e/LMfbxMiKiIZEeZE0ACzqwN8S3K5xKEWKjK5rAmUUPrsI2dsSSOwjy+AIm8SqfihIAJ15J8ndA== Received: from CWLP123CA0058.GBRP123.PROD.OUTLOOK.COM (2603:10a6:401:59::22) by AM9PR07MB7842.eurprd07.prod.outlook.com (2603:10a6:20b:300::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8722.31; Wed, 14 May 2025 13:57:17 +0000 Received: from AM3PEPF0000A790.eurprd04.prod.outlook.com (2603:10a6:401:59:cafe::82) by CWLP123CA0058.outlook.office365.com (2603:10a6:401:59::22) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8746.17 via Frontend Transport; Wed, 14 May 2025 13:57:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 131.228.2.240) smtp.mailfrom=nokia-bell-labs.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nokia-bell-labs.com; Received-SPF: Pass (protection.outlook.com: domain of nokia-bell-labs.com designates 131.228.2.240 as permitted sender) receiver=protection.outlook.com; client-ip=131.228.2.240; helo=fihe3nok0735.emea.nsn-net.net; pr=C Received: from fihe3nok0735.emea.nsn-net.net (131.228.2.240) by AM3PEPF0000A790.mail.protection.outlook.com (10.167.16.119) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8722.18 via Frontend Transport; Wed, 14 May 2025 13:57:17 +0000 Received: from sarah.nbl.nsn-rdnet.net (sarah.nbl.nsn-rdnet.net [10.0.73.150]) by fihe3nok0735.emea.nsn-net.net (Postfix) with ESMTP id 940B5200BA; Wed, 14 May 2025 16:57:15 +0300 (EEST) From: chia-yu.chang@nokia-bell-labs.com To: linux-doc@vger.kernel.org, corbet@lwn.net, horms@kernel.org, dsahern@kernel.org, kuniyu@amazon.com, bpf@vger.kernel.org, netdev@vger.kernel.org, dave.taht@gmail.com, pabeni@redhat.com, jhs@mojatatu.com, kuba@kernel.org, stephen@networkplumber.org, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, andrew+netdev@lunn.ch, donald.hunter@gmail.com, ast@fiberby.net, liuhangbin@gmail.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, ij@kernel.org, ncardwell@google.com, koen.de_schepper@nokia-bell-labs.com, g.white@cablelabs.com, ingemar.s.johansson@ericsson.com, mirja.kuehlewind@ericsson.com, cheshire@apple.com, rs.ietf@gmx.at, Jason_Livingood@comcast.com, vidhi_goel@apple.com Cc: Chia-Yu Chang Subject: [PATCH v7 net-next 14/15] tcp: accecn: try to fit AccECN option with SACK Date: Wed, 14 May 2025 15:56:41 +0200 Message-Id: <20250514135642.11203-15-chia-yu.chang@nokia-bell-labs.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> References: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM3PEPF0000A790:EE_|AM9PR07MB7842:EE_ X-MS-Office365-Filtering-Correlation-Id: 0c2c459e-3fc1-46f3-2eed-08dd92ef3cd0 X-LD-Processed: 5d471751-9675-428d-917b-70f44f9630b0,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|7416014|376014|1800799024|36860700013|82310400026|921020; X-Microsoft-Antispam-Message-Info: =?utf-8?q?LPO4RbtjQUTULDvYjKkkYNN+3xek1gd?= =?utf-8?q?pJhzQBYoz9VJN9Vs+6GlMHyNB9cSAEA+R+lR6792/fCPmScxb+ttcsTRRQBfOgkpM?= =?utf-8?q?yRNCzyiJNNnwO+tIn4dAo6/BIDa6SosKuWMwaorwbyBYeN/rpsGd960bzRmOfogd/?= =?utf-8?q?PSFrHrlf+Dt/L/HO50E8GkBN2gkJl6EFscON/B6EXvRbXauQq4Kk50Tsbe1WsOzv4?= =?utf-8?q?/jUBE5dWgdm4qBocjghSSaGP9sdmM7VXwXvLwFbhdqXkNJD4c0tnJ2btANLM44IRi?= =?utf-8?q?cFnz4JTkA+5htNFRXdjcDxyzZAzAuZJNElgLZidEv2molppq0DlKvkqSSQiGNSedh?= =?utf-8?q?D/aNT82WiFU02bZjWMjq1LnCxecp6cnJd/9DHlfdq4bErEaqcOIuStSPqipZ01VwJ?= =?utf-8?q?53PEZaNHFIslSOw/Kf2fOchsf33vCiWqGn99QiVMHzdT31/RudM2fat7zzswx8Hk1?= =?utf-8?q?99fm45D1RNOy5UDc2R7HB5DQrGMJ8zPFTmEusHHn1AtUAgQ21/wGS8OptIz7ISYE1?= =?utf-8?q?rBajnQuoiB+tvNM5E6WvtWoReczCXxS/mLNTCI2+bART6lml1cRZL7qccen/+fp48?= =?utf-8?q?QoscG0FhEXT9QcREd/UIoVfFN8CD/0tdEt6GP8mvT2vSaAairO7wWi/bmAdEGQmfh?= =?utf-8?q?m03JMheY9RCE70K7d16oSDbu60VlA/prlag4IiudiO0IPF9uB802tZhj1g+iMaiTo?= =?utf-8?q?BAV3e9nQCvdDP8lFytTHRON+Ew6lo38CiEsEd2yjHMlzxLonCJkBCRd6TbSyAp9eW?= =?utf-8?q?cEavR449zKj3wL2XJM5zIiVyP5nRlb/t9k+5X+nnl9sVEJhC8ie0bswBk4fEGw7I3?= =?utf-8?q?yi7ePTfSkABTLG8vzwBby8vHRds6W1LRxgVXImofNsNAO8RAiV8N1A+WP0915RsSm?= =?utf-8?q?HHPInOymNenUK+dF7tQ3pbY2v36JJHO8iACPJE2EX543ZsYvsWHCIntXeWrWrZK7d?= =?utf-8?q?Q/wx5ryNdHtpt7wpRIKCyBGjzZuCC/pa5dcOJcOu48kPPWDPLpt3s/HSw4cAlIYmZ?= =?utf-8?q?BTFr9g2RLMBNjf1V4D3JHX4ek3tr437rkQOoBJM130f68PK9g+KoQFqucb67dDI0E?= =?utf-8?q?KGEIeMfd3h075Va+j3C3xGESgDQRmKGL8k9LIF/HZ93TrEEbXQRMEfxREExfzZPSS?= =?utf-8?q?e0uq0UV7EWYDqLT6QN+61R2qbKn2azZwMjklzQztGwBbhQwIRPvdP3ksr+55GS0zF?= =?utf-8?q?FJ+RpRCfnzvgpOymPQvp0mRZ9q1+s2BqDFTBql6WVQ1HTJgRh25M19j8jDm5+N3h7?= =?utf-8?q?ywuFxBKN/fKiJL/p8kHbEVSuyw7msHSNhT7hHZFJm1thFDsw8W5xnSu5dOb5/cOR2?= =?utf-8?q?GB0bS+//vKEJivMICorWBthUHb44ynyq7zbJJoizwG2a7e7oUSIpXiSj0UAxcz9u0?= =?utf-8?q?WTJnXSBWDfuXWLqa+SI2m950zzEk9DXH7EvAf3oSZkiyhNEvpGAsqIMEo/iLdR88n?= =?utf-8?q?+PdKMHgNyJ147k5Ld7e0UCjP5x9Or9Ug=3D=3D?= X-Forefront-Antispam-Report: CIP:131.228.2.240; CTRY:FI; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:fihe3nok0735.emea.nsn-net.net; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(7416014)(376014)(1800799024)(36860700013)(82310400026)(921020); DIR:OUT; SFP:1101; X-OriginatorOrg: nokia-bell-labs.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2025 13:57:17.0919 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0c2c459e-3fc1-46f3-2eed-08dd92ef3cd0 X-MS-Exchange-CrossTenant-Id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5d471751-9675-428d-917b-70f44f9630b0; Ip=[131.228.2.240]; Helo=[fihe3nok0735.emea.nsn-net.net] X-MS-Exchange-CrossTenant-AuthSource: AM3PEPF0000A790.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR07MB7842 From: Ilpo Järvinen As SACK blocks tend to eat all option space when there are many holes, it is useful to compromise on sending many SACK blocks in every ACK and try to fit AccECN option there by reduction the number of SACK blocks. But never go below two SACK blocks because of AccECN option. As AccECN option is often not put to every ACK, the space hijack is usually only temporary. Signed-off-by: Ilpo Järvinen Signed-off-by: Chia-Yu Chang --- net/ipv4/tcp_output.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index b630923c4cef..d9d3cc8dbb5b 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -982,8 +982,21 @@ static int tcp_options_fit_accecn(struct tcp_out_options *opts, int required, opts->num_accecn_fields--; size -= TCPOLEN_ACCECN_PERFIELD; } - if (opts->num_accecn_fields < required) + if (opts->num_accecn_fields < required) { + if (opts->num_sack_blocks > 2) { + /* Try to fit the option by removing one SACK block */ + opts->num_sack_blocks--; + size = tcp_options_fit_accecn(opts, required, + remaining + + TCPOLEN_SACK_PERBLOCK, + max_combine_saving); + if (opts->options & OPTION_ACCECN) + return size - TCPOLEN_SACK_PERBLOCK; + + opts->num_sack_blocks++; + } return 0; + } opts->options |= OPTION_ACCECN; return size; From patchwork Wed May 14 13:56:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Chia-Yu Chang \(Nokia\)" X-Patchwork-Id: 890045 Received: from AM0PR83CU005.outbound.protection.outlook.com (mail-westeuropeazon11010032.outbound.protection.outlook.com [52.101.69.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A44527CCDB; Wed, 14 May 2025 13:57:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.69.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231050; cv=fail; b=oUnZB3H16dKGHjS9r+vn6oSbrFkP95AGMbHTbeUofpVMP2BPju/LXHkmP7s+subT2IS0q0WMcgl4nF+6ANclY5W7lK57daCGgmjYt2gtVBiU+2wRsN40Sf8jJGzxN0vk9uwtXx0/VfbuPHfO6NrjZB/mF08Bwa0xVsssS9oK//M= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747231050; c=relaxed/simple; bh=Rpc4gI7FN20PeVrV/bFAaFUlv9cSGLN/wMwrNcsSIh4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=qoW8AcMzqBGyZlvccZspGbfhlol+vlJX/Z5mH7+bo1QhZAFK3lvXz5zUYvRcv43r1VqaKPXVRAXFmD6zPfZJjBIOXzrMioYFte0S0nF7w/gB1AeHCUZzabeUeyyzpzNLGPKQI3E7VQ/PK7ZF3gmfubAYZXxWZIxW67l4HsLbGVY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com; spf=fail smtp.mailfrom=nokia-bell-labs.com; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b=ae8ozgiw; arc=fail smtp.client-ip=52.101.69.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nokia-bell-labs.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=nokia-bell-labs.com header.i=@nokia-bell-labs.com header.b="ae8ozgiw" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=fG1UACtFu9/xr7tqkoTUob+mtxAu6E2VfakqtGFHHdOBS/dP+timCvlgDve4FeIXxyE96XpyMzhl3TPZ76Ul3phKBeyTyZAM7ieyIQNlRbRxtnd91GGdZPfNdkP3rwBMb0I5MTY0Gyn+lBhPaR2jH8A50NT9jcGnCTEZLrAmuC4Pz492JJ2PLvTsUsO0YbMMFdlSEq4IMd2EqF4LxIlCDoFn/8iCwvl2PGc94PVgL2h3L4EUDCh1rrbHFCkT5cIujpkZNzKkFPhR088NuD8zXFzdazAoYePHmzO3423IpIZUvvEMZMCc5w6+lnQrl3jcPZscDCafcWFQdOAQg46x+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FeVAX8+J+Pzn97ZH7ktHpXkSmO6s4Os2BtLfBHe1RX0=; b=V7sK5n1y9yzrmImBEmQCJ0IzD1kj7284wyo1dSkQWXps3FiM24Yb7xtlMMZfwbYG0KYCXMUnPcYUG2RBIYgmKpczOXQG81o3dnfg8LYbY64PBv/hXM7l81PFdqTvOOBUmksK5AF1b02uJUR5Pyf3gpips+dYq7a818o2+LNlJhgQU3ELkj6kb7CpTrtvWAaE3VbGOJzvtlr8kCylRYPntB821guUpn78A0UDzAowAFGnEpDlg71W4tMBceqOGUX9bx2EDFrB0OVxym2XlGcUisfWa+DSoSBuSwVz66Pqh2GK6jLK9dtlPwnXKFLz0EP+okzu96CuKZ0nVtQ0tL9gEg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=temperror (sender ip is 131.228.2.240) smtp.rcpttodomain=amazon.com smtp.mailfrom=nokia-bell-labs.com; dmarc=temperror action=none header.from=nokia-bell-labs.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia-bell-labs.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FeVAX8+J+Pzn97ZH7ktHpXkSmO6s4Os2BtLfBHe1RX0=; b=ae8ozgiwTB3iGYm4q9AMCb2pJq6bRAZlX+UeFhoOneu7fw8w+Eh+FtOA0gbtHF6nw6ByeJAls+2+qCu0q7keVaSSyNJE3vyGSiuPQQfwTF0JUb/+uyF5kOc0iXsYrmpxuCVcBio0iJ+dEyNZLZqwdVm5pNUCX1ovwordt6uAltf19TIO3eDahFrmiV8wZjB8FoTg9x9KE0Zvcxvt1TP4//NTwC7roT8v4FCPzIW6LjtdGgFMF7SqWu/+cFUQx1XnlbQYGAtHkpOBADzVwYUECwuIiKGY8tM/tPNd4Tk3e/5x4u4XPmbc7Xrsks54eSnGcH0VCMSW9p4xdi/B6LqSfg== Received: from AM9P195CA0024.EURP195.PROD.OUTLOOK.COM (2603:10a6:20b:21f::29) by VI1PR07MB9472.eurprd07.prod.outlook.com (2603:10a6:800:1c6::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8722.31; Wed, 14 May 2025 13:57:20 +0000 Received: from AMS0EPF000001A5.eurprd05.prod.outlook.com (2603:10a6:20b:21f:cafe::25) by AM9P195CA0024.outlook.office365.com (2603:10a6:20b:21f::29) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8722.31 via Frontend Transport; Wed, 14 May 2025 13:57:20 +0000 X-MS-Exchange-Authentication-Results: spf=temperror (sender IP is 131.228.2.240) smtp.mailfrom=nokia-bell-labs.com; dkim=none (message not signed) header.d=none;dmarc=temperror action=none header.from=nokia-bell-labs.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of nokia-bell-labs.com: DNS Timeout) Received: from fihe3nok0735.emea.nsn-net.net (131.228.2.240) by AMS0EPF000001A5.mail.protection.outlook.com (10.167.16.232) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8722.18 via Frontend Transport; Wed, 14 May 2025 13:57:19 +0000 Received: from sarah.nbl.nsn-rdnet.net (sarah.nbl.nsn-rdnet.net [10.0.73.150]) by fihe3nok0735.emea.nsn-net.net (Postfix) with ESMTP id B267D200C2; Wed, 14 May 2025 16:57:17 +0300 (EEST) From: chia-yu.chang@nokia-bell-labs.com To: linux-doc@vger.kernel.org, corbet@lwn.net, horms@kernel.org, dsahern@kernel.org, kuniyu@amazon.com, bpf@vger.kernel.org, netdev@vger.kernel.org, dave.taht@gmail.com, pabeni@redhat.com, jhs@mojatatu.com, kuba@kernel.org, stephen@networkplumber.org, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, andrew+netdev@lunn.ch, donald.hunter@gmail.com, ast@fiberby.net, liuhangbin@gmail.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, ij@kernel.org, ncardwell@google.com, koen.de_schepper@nokia-bell-labs.com, g.white@cablelabs.com, ingemar.s.johansson@ericsson.com, mirja.kuehlewind@ericsson.com, cheshire@apple.com, rs.ietf@gmx.at, Jason_Livingood@comcast.com, vidhi_goel@apple.com Cc: Chia-Yu Chang Subject: [PATCH v7 net-next 15/15] tcp: try to avoid safer when ACKs are thinned Date: Wed, 14 May 2025 15:56:42 +0200 Message-Id: <20250514135642.11203-16-chia-yu.chang@nokia-bell-labs.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> References: <20250514135642.11203-1-chia-yu.chang@nokia-bell-labs.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AMS0EPF000001A5:EE_|VI1PR07MB9472:EE_ X-MS-Office365-Filtering-Correlation-Id: 48f4ec08-97ca-41d8-c8b0-08dd92ef3e29 X-LD-Processed: 5d471751-9675-428d-917b-70f44f9630b0,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|1800799024|7416014|82310400026|36860700013|921020; X-Microsoft-Antispam-Message-Info: =?utf-8?q?eC/gfWmDJN4fU3rEtbzt6YcFAPLm0kd?= =?utf-8?q?ytUzuJLWmF8N5sE5yxCLkmxUnU0gKn0H9tRrnRN5fd7y1Oq8ox9l4XcrRd2Yw4ehW?= =?utf-8?q?ivwdAIro+qpuqcDFw4gUQ3skPvoW5cNWoz/IX4Cn6jp3R1+tyoWA6XaIV/Cp9ynyT?= =?utf-8?q?Yzk1d2RLh15CfIbk0S3njTQmFXO7ejHNue5sb3UOay8GsykFG3N1NuCCQRfJ6Kq9k?= =?utf-8?q?8XyFRFFE1YsuKEnWA8uEVNX8WivaR+PgDllC6URovAc0rASKdq0+tw/wmH2nFmE9Y?= =?utf-8?q?mJnrWClB1x/cK88aQkJ6E3i4laCgymnOr09UrJRiorPTIwpLyEfXrq0aJFCno5HJk?= =?utf-8?q?bpC6lfkePLJqsSKbWa+0RzteAOJt/bu5IgfNnNPjyLrBrPUhidUzYm7rCJPHnGIpn?= =?utf-8?q?fEXK8IbIz6mWlmjOdwe3j5FG4LqW1lI/d+BvuPs+3HeuhKQvsEhB7ZU9V9tCOKPP1?= =?utf-8?q?2ng0FlXJ9D7WTPpp1rJ+X458X5W7nti2KzmSPQsNHStQBA1LpqwyDUxgraM0hXVZ8?= =?utf-8?q?WkcnIyPdLCVd5BqFJrvO4xY9GGVxSLLktP0IiFLDre3oMUJIVALX8+7AhfJNrel2L?= =?utf-8?q?HhONEfi3VyoB5cxp1Nnq6Z6aQbcIjzbZ/Z6zAXu1k2oq9+nlP8u1fsOEbm0sQFMzH?= =?utf-8?q?k2asKBVOZHHk/NmCV0CGPGGflFn5alauuBDDsrADO+nzWmnL7FF4nkGQwttHcbfLh?= =?utf-8?q?D9nodzsxYsx7sf54rjs8zhamYZ6zaIQhSzT6FRC4Y8V3nNkGHNCHXjMBHx6tLi75U?= =?utf-8?q?3cMNzqYTIYVvMxbx+dnrOPi1ZwaA21r8NLyQS/kEjnIW2sBKM+yWzLJiy396Of6tQ?= =?utf-8?q?RXhq3A+jloVLUcgWEAFZkA+DNsQJlmNVaVYZnAWjdpBVbmWnyiaHx8zD6h/dS7NLL?= =?utf-8?q?jEf93tTZxpE8l/8gUKE3RQSpitHx9+xMHYntLt+LtpUi4QulR3UQs1FKVnIHzLtB+?= =?utf-8?q?szY1ZQRM3TqgtoTXOh9uk3GxL2XFZvte7eAJ5zDVidtOTXcwwMquqgjAec8f6aB2B?= =?utf-8?q?SmGfn7W8dHpsBC6oDLo56X6oViXsPS03wB4iBugjF6bnx6zxdZaW85kffD9SEz0iL?= =?utf-8?q?DVa1cLn6Q6O1mrLT4+A8zshe5GvLB2dLBC8vjrqwUVD/Un+tl9+q4pdpWJvlxsFiI?= =?utf-8?q?MeUO0TW3cj7NqSACcG+zWfLAoGzjJO3T0mJ9N8OFYze8Ifx0dUR51qWz2RA3swGtW?= =?utf-8?q?6uGos60Co4Y6APnXhxm/9XaTbxUnvFkOM9Tmkwp7FAzrUdvPhEdF4meFLcHvtD/lc?= =?utf-8?q?XinpY4jfaZ8gGyUxKK12ayd5fSlrMKX8VD1MDvyg3VAz9nQR7seIKpiKjs/yzYLmu?= =?utf-8?q?SPyFrwfa3RJE6djyTjPgPPKQCrR1TBYtw2GbRv80v/DtikG+H/n4Xaei/RdfuzZ2/?= =?utf-8?q?PzzeInZvOZWciY1Lg3PddRyuUCLwc8Xi5DDTI++FeZNKUknub3W6JDWZn+pxg32Lb?= =?utf-8?q?V25iqt4A7mdOGsd9UG4ceN+7yV9bC10A=3D=3D?= X-Forefront-Antispam-Report: CIP:131.228.2.240; CTRY:FI; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:fihe3nok0735.emea.nsn-net.net; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(376014)(1800799024)(7416014)(82310400026)(36860700013)(921020); DIR:OUT; SFP:1101; X-OriginatorOrg: nokia-bell-labs.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2025 13:57:19.2184 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 48f4ec08-97ca-41d8-c8b0-08dd92ef3e29 X-MS-Exchange-CrossTenant-Id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5d471751-9675-428d-917b-70f44f9630b0; Ip=[131.228.2.240]; Helo=[fihe3nok0735.emea.nsn-net.net] X-MS-Exchange-CrossTenant-AuthSource: AMS0EPF000001A5.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR07MB9472 From: Ilpo Järvinen Add newly acked pkts EWMA. When ACK thinning occurs, select between safer and unsafe cep delta in AccECN processing based on it. If the packets ACKed per ACK tends to be large, don't conservatively assume ACE field overflow. This patch uses the existing 2-byte holes in the rx group for new u16 variables withtout creating more holes. Below are the pahole outcomes before and after this patch: [BEFORE THIS PATCH] struct tcp_sock { [...] u32 delivered_ecn_bytes[3]; /* 2744 12 */ /* XXX 4 bytes hole, try to pack */ [...] __cacheline_group_end__tcp_sock_write_rx[0]; /* 2816 0 */ [...] /* size: 3264, cachelines: 51, members: 177 */ } [AFTER THIS PATCH] struct tcp_sock { [...] u32 delivered_ecn_bytes[3]; /* 2744 12 */ u16 pkts_acked_ewma; /* 2756 2 */ /* XXX 2 bytes hole, try to pack */ [...] __cacheline_group_end__tcp_sock_write_rx[0]; /* 2816 0 */ [...] /* size: 3264, cachelines: 51, members: 178 */ } Signed-off-by: Ilpo Järvinen Co-developed-by: Chia-Yu Chang Signed-off-by: Chia-Yu Chang --- .../networking/net_cachelines/tcp_sock.rst | 1 + include/linux/tcp.h | 1 + net/ipv4/tcp.c | 2 ++ net/ipv4/tcp_input.c | 20 ++++++++++++++++++- 4 files changed, 23 insertions(+), 1 deletion(-) diff --git a/Documentation/networking/net_cachelines/tcp_sock.rst b/Documentation/networking/net_cachelines/tcp_sock.rst index af76d4f9b33a..2e6ef8290aff 100644 --- a/Documentation/networking/net_cachelines/tcp_sock.rst +++ b/Documentation/networking/net_cachelines/tcp_sock.rst @@ -105,6 +105,7 @@ u32 received_ce read_mostly read_w u32[3] received_ecn_bytes read_mostly read_write u8:4 received_ce_pending read_mostly read_write u32[3] delivered_ecn_bytes read_write +u16 pkts_acked_ewma read_write u8:2 syn_ect_snt write_mostly read_write u8:2 syn_ect_rcv read_mostly read_write u8:1 wait_third_ack read_write diff --git a/include/linux/tcp.h b/include/linux/tcp.h index b5066eef8782..b22914a4f3cb 100644 --- a/include/linux/tcp.h +++ b/include/linux/tcp.h @@ -346,6 +346,7 @@ struct tcp_sock { u32 rate_interval_us; /* saved rate sample: time elapsed */ u32 rcv_rtt_last_tsecr; u32 delivered_ecn_bytes[3]; + u16 pkts_acked_ewma;/* Pkts acked EWMA for AccECN cep heuristic */ u64 first_tx_mstamp; /* start of window send phase */ u64 delivered_mstamp; /* time we reached "delivered" */ u64 bytes_acked; /* RFC4898 tcpEStatsAppHCThruOctetsAcked diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index e68b9706eeff..b1ee5b6618f0 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -3403,6 +3403,7 @@ int tcp_disconnect(struct sock *sk, int flags) tcp_accecn_init_counters(tp); tp->prev_ecnfield = 0; tp->accecn_opt_tstamp = 0; + tp->pkts_acked_ewma = 0; if (icsk->icsk_ca_initialized && icsk->icsk_ca_ops->release) icsk->icsk_ca_ops->release(sk); memset(icsk->icsk_ca_priv, 0, sizeof(icsk->icsk_ca_priv)); @@ -5164,6 +5165,7 @@ static void __init tcp_struct_check(void) CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, rate_interval_us); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, rcv_rtt_last_tsecr); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, delivered_ecn_bytes); + CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, pkts_acked_ewma); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, first_tx_mstamp); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, delivered_mstamp); CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_write_rx, bytes_acked); diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index deced87f31f8..7535de493d43 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -704,6 +704,10 @@ static void tcp_count_delivered(struct tcp_sock *tp, u32 delivered, tcp_count_delivered_ce(tp, delivered); } +#define PKTS_ACKED_WEIGHT 6 +#define PKTS_ACKED_PREC 6 +#define ACK_COMP_THRESH 4 + /* Returns the ECN CE delta */ static u32 __tcp_accecn_process(struct sock *sk, const struct sk_buff *skb, u32 delivered_pkts, u32 delivered_bytes, @@ -723,6 +727,19 @@ static u32 __tcp_accecn_process(struct sock *sk, const struct sk_buff *skb, opt_deltas_valid = tcp_accecn_process_option(tp, skb, delivered_bytes, flag); + if (delivered_pkts) { + if (!tp->pkts_acked_ewma) { + tp->pkts_acked_ewma = delivered_pkts << PKTS_ACKED_PREC; + } else { + u32 ewma = tp->pkts_acked_ewma; + + ewma = (((ewma << PKTS_ACKED_WEIGHT) - ewma) + + (delivered_pkts << PKTS_ACKED_PREC)) >> + PKTS_ACKED_WEIGHT; + tp->pkts_acked_ewma = min_t(u32, ewma, 0xFFFFU); + } + } + if (!(flag & FLAG_SLOWPATH)) { /* AccECN counter might overflow on large ACKs */ if (delivered_pkts <= TCP_ACCECN_CEP_ACE_MASK) @@ -771,7 +788,8 @@ static u32 __tcp_accecn_process(struct sock *sk, const struct sk_buff *skb, if (d_ceb < safe_delta * tp->mss_cache >> TCP_ACCECN_SAFETY_SHIFT) return delta; - } + } else if (tp->pkts_acked_ewma > (ACK_COMP_THRESH << PKTS_ACKED_PREC)) + return delta; return safe_delta; }