From patchwork Mon Feb 1 19:47:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ido Schimmel X-Patchwork-Id: 374509 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2A5EC433E0 for ; Mon, 1 Feb 2021 19:52:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8353B64EA2 for ; Mon, 1 Feb 2021 19:52:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232770AbhBATvn (ORCPT ); Mon, 1 Feb 2021 14:51:43 -0500 Received: from new1-smtp.messagingengine.com ([66.111.4.221]:55215 "EHLO new1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231868AbhBATtr (ORCPT ); Mon, 1 Feb 2021 14:49:47 -0500 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.nyi.internal (Postfix) with ESMTP id 363A8580436; Mon, 1 Feb 2021 14:48:41 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Mon, 01 Feb 2021 14:48:41 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=Fnl4B91G1tjsob2Ju8/a/wq2zj1vfaoTlf0/Nw3FNL8=; b=rLS7RMjt dKBrTjkHUfAbfqHGCZjBZmNtQGO1ncgInDewtV3l01zdEWO6HupiRiT/0YUQZVeU 0daAtBLtrrXZynBsfu2GiQPv2c/2XAPM663qSa4JTQ88XxCVuZ0AilP9rXhVIQYU P5lrJOUna+a70xwz1eypFXRsfT/RxeVVBTQsxfg16cVVDAE7QuYvU7kqL145GdEz WjpQOL1C/anFddsvYeO0gFr9/evBU4+4fNIKur/xuxe3IpYHls7v6FSCIuI5Kl61 GgryZUmkalmnhgtJDNRNrmcm3ejvQvHtmNgau/yo5+BzNhKrlkJOj80zVdXfMOAM JwT6Ci5tbxBLNw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrfeekgddufedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkofgjfhgggfestdekre dtredttdenucfhrhhomhepkfguohcuufgthhhimhhmvghluceoihguohhstghhsehiugho shgthhdrohhrgheqnecuggftrfgrthhtvghrnhepudetieevffffveelkeeljeffkefhke ehgfdtffethfelvdejgffghefgveejkefhnecukfhppeekgedrvddvledrudehfedrgeeg necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepihguoh hstghhsehiughoshgthhdrohhrgh X-ME-Proxy: Received: from shredder.mellanox.com (igld-84-229-153-44.inter.net.il [84.229.153.44]) by mail.messagingengine.com (Postfix) with ESMTPA id 5882924005C; Mon, 1 Feb 2021 14:48:38 -0500 (EST) From: Ido Schimmel To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, dsahern@gmail.com, yoshfuji@linux-ipv6.org, jiri@nvidia.com, amcohen@nvidia.com, roopa@nvidia.com, bpoirier@nvidia.com, sharpd@nvidia.com, mlxsw@nvidia.com, Ido Schimmel Subject: [PATCH net-next v2 01/10] netdevsim: fib: Convert the current occupancy to an atomic variable Date: Mon, 1 Feb 2021 21:47:48 +0200 Message-Id: <20210201194757.3463461-2-idosch@idosch.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210201194757.3463461-1-idosch@idosch.org> References: <20210201194757.3463461-1-idosch@idosch.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Amit Cohen When route is added/deleted, the appropriate counter is increased/decreased to maintain number of routes. User can limit the number of routes and then according to the appropriate counter, adding more routes than the limitation is forbidden. Currently, there is one lock which protects hashtable, list and accounting. Handling the counters will be performed from both atomic context and non-atomic context, while the hashtable and the list will be used only from non-atomic context and therefore will be protected by a separate lock. Protect accounting by using an atomic variable, so lock is not needed. v2: * Use atomic64_sub() in nsim_nexthop_account()'s error path Signed-off-by: Amit Cohen Signed-off-by: Ido Schimmel Reviewed-by: David Ahern --- drivers/net/netdevsim/fib.c | 55 ++++++++++++++++++------------------- 1 file changed, 27 insertions(+), 28 deletions(-) diff --git a/drivers/net/netdevsim/fib.c b/drivers/net/netdevsim/fib.c index f140bbca98c5..7be603e06769 100644 --- a/drivers/net/netdevsim/fib.c +++ b/drivers/net/netdevsim/fib.c @@ -31,7 +31,7 @@ struct nsim_fib_entry { u64 max; - u64 num; + atomic64_t num; }; struct nsim_per_fib_data { @@ -46,7 +46,7 @@ struct nsim_fib_data { struct nsim_fib_entry nexthops; struct rhashtable fib_rt_ht; struct list_head fib_rt_list; - spinlock_t fib_lock; /* Protects hashtable, list and accounting */ + spinlock_t fib_lock; /* Protects hashtable and list */ struct notifier_block nexthop_nb; struct rhashtable nexthop_ht; struct devlink *devlink; @@ -128,7 +128,7 @@ u64 nsim_fib_get_val(struct nsim_fib_data *fib_data, return 0; } - return max ? entry->max : entry->num; + return max ? entry->max : atomic64_read(&entry->num); } static void nsim_fib_set_max(struct nsim_fib_data *fib_data, @@ -165,14 +165,12 @@ static int nsim_fib_rule_account(struct nsim_fib_entry *entry, bool add, int err = 0; if (add) { - if (entry->num < entry->max) { - entry->num++; - } else { + if (!atomic64_add_unless(&entry->num, 1, entry->max)) { err = -ENOSPC; NL_SET_ERR_MSG_MOD(extack, "Exceeded number of supported fib rule entries"); } } else { - entry->num--; + atomic64_dec_if_positive(&entry->num); } return err; @@ -202,14 +200,12 @@ static int nsim_fib_account(struct nsim_fib_entry *entry, bool add, int err = 0; if (add) { - if (entry->num < entry->max) { - entry->num++; - } else { + if (!atomic64_add_unless(&entry->num, 1, entry->max)) { err = -ENOSPC; NL_SET_ERR_MSG_MOD(extack, "Exceeded number of supported fib entries"); } } else { - entry->num--; + atomic64_dec_if_positive(&entry->num); } return err; @@ -769,25 +765,22 @@ static int nsim_fib_event_nb(struct notifier_block *nb, unsigned long event, struct fib_notifier_info *info = ptr; int err = 0; - /* IPv6 routes can be added via RAs from softIRQ. */ - spin_lock_bh(&data->fib_lock); - switch (event) { case FIB_EVENT_RULE_ADD: case FIB_EVENT_RULE_DEL: err = nsim_fib_rule_event(data, info, event == FIB_EVENT_RULE_ADD); break; - case FIB_EVENT_ENTRY_REPLACE: case FIB_EVENT_ENTRY_APPEND: case FIB_EVENT_ENTRY_DEL: + /* IPv6 routes can be added via RAs from softIRQ. */ + spin_lock_bh(&data->fib_lock); err = nsim_fib_event(data, info, event); + spin_unlock_bh(&data->fib_lock); break; } - spin_unlock_bh(&data->fib_lock); - return notifier_from_errno(err); } @@ -847,8 +840,8 @@ static void nsim_fib_dump_inconsistent(struct notifier_block *nb) nsim_fib_rt_free(fib_rt, data); } - data->ipv4.rules.num = 0ULL; - data->ipv6.rules.num = 0ULL; + atomic64_set(&data->ipv4.rules.num, 0ULL); + atomic64_set(&data->ipv6.rules.num, 0ULL); } static struct nsim_nexthop *nsim_nexthop_create(struct nsim_fib_data *data, @@ -894,22 +887,28 @@ static void nsim_nexthop_destroy(struct nsim_nexthop *nexthop) static int nsim_nexthop_account(struct nsim_fib_data *data, u64 occ, bool add, struct netlink_ext_ack *extack) { - int err = 0; + int i, err = 0; if (add) { - if (data->nexthops.num + occ <= data->nexthops.max) { - data->nexthops.num += occ; - } else { - err = -ENOSPC; - NL_SET_ERR_MSG_MOD(extack, "Exceeded number of supported nexthops"); - } + for (i = 0; i < occ; i++) + if (!atomic64_add_unless(&data->nexthops.num, 1, + data->nexthops.max)) { + err = -ENOSPC; + NL_SET_ERR_MSG_MOD(extack, "Exceeded number of supported nexthops"); + goto err_num_decrease; + } } else { - if (WARN_ON(occ > data->nexthops.num)) + if (WARN_ON(occ > atomic64_read(&data->nexthops.num))) return -EINVAL; - data->nexthops.num -= occ; + atomic64_sub(occ, &data->nexthops.num); } return err; + +err_num_decrease: + atomic64_sub(i, &data->nexthops.num); + return err; + } static int nsim_nexthop_add(struct nsim_fib_data *data, From patchwork Mon Feb 1 19:47:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ido Schimmel X-Patchwork-Id: 374503 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2317C433E0 for ; Mon, 1 Feb 2021 19:58:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5B860601FB for ; Mon, 1 Feb 2021 19:58:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232564AbhBAT6Q (ORCPT ); Mon, 1 Feb 2021 14:58:16 -0500 Received: from new1-smtp.messagingengine.com ([66.111.4.221]:42425 "EHLO new1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232626AbhBATtw (ORCPT ); Mon, 1 Feb 2021 14:49:52 -0500 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.nyi.internal (Postfix) with ESMTP id 7DA2F580508; Mon, 1 Feb 2021 14:48:44 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Mon, 01 Feb 2021 14:48:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=EoRTweBmHMtEPOISSYlh+0N+awSDk4O0htJH2VRUa7Q=; b=fkLd6j18 Woc0hxbX+CZR3liiq2lrfgrwE11mA8LJjAbTlXHI6YRg57MPUp3JWWzgNJQSu4Ak nUtIL/54QjI0DUJ9bxGZaFy5JEG2DV8JTS6VXYQTDHahmeJpaU9X3R7YvQOkQAga XVURoneMVcm/DWlTKx2eFDWiY6XMUaYI4tfqW7YiFQrrkaQ1dNUdGOQrvHA9UaCk kDQWAGXLqhhCpVu1CXQEiX928BJ6SlpvFOXX/3zC2erYZO/m6TCbLPuvpgE26fB6 i5AeK1+0KIVb+pTm6LnlAihDWyPAnFRdsp76IYxIZRYgVAU/SPNP/bXlNu1DY2xQ FJCRfCIWDlaR6g== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrfeekgddufedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkofgjfhgggfestdekre dtredttdenucfhrhhomhepkfguohcuufgthhhimhhmvghluceoihguohhstghhsehiugho shgthhdrohhrgheqnecuggftrfgrthhtvghrnhepudetieevffffveelkeeljeffkefhke ehgfdtffethfelvdejgffghefgveejkefhnecukfhppeekgedrvddvledrudehfedrgeeg necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepihguoh hstghhsehiughoshgthhdrohhrgh X-ME-Proxy: Received: from shredder.mellanox.com (igld-84-229-153-44.inter.net.il [84.229.153.44]) by mail.messagingengine.com (Postfix) with ESMTPA id 69F0F24005D; Mon, 1 Feb 2021 14:48:41 -0500 (EST) From: Ido Schimmel To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, dsahern@gmail.com, yoshfuji@linux-ipv6.org, jiri@nvidia.com, amcohen@nvidia.com, roopa@nvidia.com, bpoirier@nvidia.com, sharpd@nvidia.com, mlxsw@nvidia.com, Ido Schimmel Subject: [PATCH net-next v2 02/10] netdevsim: fib: Perform the route programming in a non-atomic context Date: Mon, 1 Feb 2021 21:47:49 +0200 Message-Id: <20210201194757.3463461-3-idosch@idosch.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210201194757.3463461-1-idosch@idosch.org> References: <20210201194757.3463461-1-idosch@idosch.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Amit Cohen Currently, netdevsim implements dummy FIB offload and marks notified routes with RTM_F_TRAP flag. netdevsim does not defer route notifications to a work queue because it does not need to program any hardware. Given that netdevsim's purpose is to both give an example implementation and allow developers to test their code, align netdevsim to a "real" hardware device driver like mlxsw and have it also perform the route "programming" in a non-atomic context. It will be used to test route flags notifications which will be added in the next patches. The following changes are needed when route handling is performed in WQ: - Handle the accounting in the main context, to be able to return an error for adding route when all the routes are used. For FIB_EVENT_ENTRY_REPLACE increase the counter before scheduling the delayed work, and in case that this event replaces an existing route, decrease the counter as part of the delayed work. - For IPv6, cannot use fen6_info->rt->fib6_siblings list because it might be changed during handling the delayed work. Save an array with the nexthops as part of fib6_event struct, and take a reference for each nexthop to prevent them from being freed while event is queued. - Change GFP_ATOMIC allocations to GFP_KERNEL. - Use single work item that is handling a list of ordered routes. Handling routes must be processed in the order they were submitted to avoid logical errors that could lead to unexpected failures. Signed-off-by: Amit Cohen Signed-off-by: Ido Schimmel Acked-by: David Ahern --- drivers/net/netdevsim/fib.c | 467 +++++++++++++++++++++++++----------- 1 file changed, 327 insertions(+), 140 deletions(-) diff --git a/drivers/net/netdevsim/fib.c b/drivers/net/netdevsim/fib.c index 7be603e06769..d557533d43dd 100644 --- a/drivers/net/netdevsim/fib.c +++ b/drivers/net/netdevsim/fib.c @@ -46,10 +46,13 @@ struct nsim_fib_data { struct nsim_fib_entry nexthops; struct rhashtable fib_rt_ht; struct list_head fib_rt_list; - spinlock_t fib_lock; /* Protects hashtable and list */ + struct mutex fib_lock; /* Protects hashtable and list */ struct notifier_block nexthop_nb; struct rhashtable nexthop_ht; struct devlink *devlink; + struct work_struct fib_event_work; + struct list_head fib_event_queue; + spinlock_t fib_event_queue_lock; /* Protects fib event queue list */ }; struct nsim_fib_rt_key { @@ -83,6 +86,22 @@ struct nsim_fib6_rt_nh { struct fib6_info *rt; }; +struct nsim_fib6_event { + struct fib6_info **rt_arr; + unsigned int nrt6; +}; + +struct nsim_fib_event { + struct list_head list; /* node in fib queue */ + union { + struct fib_entry_notifier_info fen_info; + struct nsim_fib6_event fib6_event; + }; + struct nsim_fib_data *data; + unsigned long event; + int family; +}; + static const struct rhashtable_params nsim_fib_rt_ht_params = { .key_offset = offsetof(struct nsim_fib_rt, key), .head_offset = offsetof(struct nsim_fib_rt, ht_node), @@ -194,16 +213,13 @@ static int nsim_fib_rule_event(struct nsim_fib_data *data, return err; } -static int nsim_fib_account(struct nsim_fib_entry *entry, bool add, - struct netlink_ext_ack *extack) +static int nsim_fib_account(struct nsim_fib_entry *entry, bool add) { int err = 0; if (add) { - if (!atomic64_add_unless(&entry->num, 1, entry->max)) { + if (!atomic64_add_unless(&entry->num, 1, entry->max)) err = -ENOSPC; - NL_SET_ERR_MSG_MOD(extack, "Exceeded number of supported fib entries"); - } } else { atomic64_dec_if_positive(&entry->num); } @@ -250,7 +266,7 @@ nsim_fib4_rt_create(struct nsim_fib_data *data, { struct nsim_fib4_rt *fib4_rt; - fib4_rt = kzalloc(sizeof(*fib4_rt), GFP_ATOMIC); + fib4_rt = kzalloc(sizeof(*fib4_rt), GFP_KERNEL); if (!fib4_rt) return NULL; @@ -307,51 +323,52 @@ static void nsim_fib4_rt_hw_flags_set(struct net *net, } static int nsim_fib4_rt_add(struct nsim_fib_data *data, - struct nsim_fib4_rt *fib4_rt, - struct netlink_ext_ack *extack) + struct nsim_fib4_rt *fib4_rt) { struct net *net = devlink_net(data->devlink); int err; - err = nsim_fib_account(&data->ipv4.fib, true, extack); - if (err) - return err; - err = rhashtable_insert_fast(&data->fib_rt_ht, &fib4_rt->common.ht_node, nsim_fib_rt_ht_params); - if (err) { - NL_SET_ERR_MSG_MOD(extack, "Failed to insert IPv4 route"); + if (err) goto err_fib_dismiss; - } + /* Simulate hardware programming latency. */ + msleep(1); nsim_fib4_rt_hw_flags_set(net, fib4_rt, true); return 0; err_fib_dismiss: - nsim_fib_account(&data->ipv4.fib, false, extack); + /* Drop the accounting that was increased from the notification + * context when FIB_EVENT_ENTRY_REPLACE was triggered. + */ + nsim_fib_account(&data->ipv4.fib, false); return err; } static int nsim_fib4_rt_replace(struct nsim_fib_data *data, struct nsim_fib4_rt *fib4_rt, - struct nsim_fib4_rt *fib4_rt_old, - struct netlink_ext_ack *extack) + struct nsim_fib4_rt *fib4_rt_old) { struct net *net = devlink_net(data->devlink); int err; - /* We are replacing a route, so no need to change the accounting. */ + /* We are replacing a route, so need to remove the accounting which + * was increased when FIB_EVENT_ENTRY_REPLACE was triggered. + */ + err = nsim_fib_account(&data->ipv4.fib, false); + if (err) + return err; err = rhashtable_replace_fast(&data->fib_rt_ht, &fib4_rt_old->common.ht_node, &fib4_rt->common.ht_node, nsim_fib_rt_ht_params); - if (err) { - NL_SET_ERR_MSG_MOD(extack, "Failed to replace IPv4 route"); + if (err) return err; - } + msleep(1); nsim_fib4_rt_hw_flags_set(net, fib4_rt, true); nsim_fib4_rt_hw_flags_set(net, fib4_rt_old, false); @@ -363,7 +380,6 @@ static int nsim_fib4_rt_replace(struct nsim_fib_data *data, static int nsim_fib4_rt_insert(struct nsim_fib_data *data, struct fib_entry_notifier_info *fen_info) { - struct netlink_ext_ack *extack = fen_info->info.extack; struct nsim_fib4_rt *fib4_rt, *fib4_rt_old; int err; @@ -373,9 +389,9 @@ static int nsim_fib4_rt_insert(struct nsim_fib_data *data, fib4_rt_old = nsim_fib4_rt_lookup(&data->fib_rt_ht, fen_info); if (!fib4_rt_old) - err = nsim_fib4_rt_add(data, fib4_rt, extack); + err = nsim_fib4_rt_add(data, fib4_rt); else - err = nsim_fib4_rt_replace(data, fib4_rt, fib4_rt_old, extack); + err = nsim_fib4_rt_replace(data, fib4_rt, fib4_rt_old); if (err) nsim_fib4_rt_destroy(fib4_rt); @@ -386,7 +402,6 @@ static int nsim_fib4_rt_insert(struct nsim_fib_data *data, static void nsim_fib4_rt_remove(struct nsim_fib_data *data, const struct fib_entry_notifier_info *fen_info) { - struct netlink_ext_ack *extack = fen_info->info.extack; struct nsim_fib4_rt *fib4_rt; fib4_rt = nsim_fib4_rt_lookup(&data->fib_rt_ht, fen_info); @@ -395,19 +410,15 @@ static void nsim_fib4_rt_remove(struct nsim_fib_data *data, rhashtable_remove_fast(&data->fib_rt_ht, &fib4_rt->common.ht_node, nsim_fib_rt_ht_params); - nsim_fib_account(&data->ipv4.fib, false, extack); nsim_fib4_rt_destroy(fib4_rt); } static int nsim_fib4_event(struct nsim_fib_data *data, - struct fib_notifier_info *info, + struct fib_entry_notifier_info *fen_info, unsigned long event) { - struct fib_entry_notifier_info *fen_info; int err = 0; - fen_info = container_of(info, struct fib_entry_notifier_info, info); - switch (event) { case FIB_EVENT_ENTRY_REPLACE: err = nsim_fib4_rt_insert(data, fen_info); @@ -441,7 +452,7 @@ static int nsim_fib6_rt_nh_add(struct nsim_fib6_rt *fib6_rt, { struct nsim_fib6_rt_nh *fib6_rt_nh; - fib6_rt_nh = kzalloc(sizeof(*fib6_rt_nh), GFP_ATOMIC); + fib6_rt_nh = kzalloc(sizeof(*fib6_rt_nh), GFP_KERNEL); if (!fib6_rt_nh) return -ENOMEM; @@ -453,6 +464,17 @@ static int nsim_fib6_rt_nh_add(struct nsim_fib6_rt *fib6_rt, return 0; } +#if IS_ENABLED(CONFIG_IPV6) +static void nsim_rt6_release(struct fib6_info *rt) +{ + fib6_info_release(rt); +} +#else +static void nsim_rt6_release(struct fib6_info *rt) +{ +} +#endif + static void nsim_fib6_rt_nh_del(struct nsim_fib6_rt *fib6_rt, const struct fib6_info *rt) { @@ -464,22 +486,20 @@ static void nsim_fib6_rt_nh_del(struct nsim_fib6_rt *fib6_rt, fib6_rt->nhs--; list_del(&fib6_rt_nh->list); -#if IS_ENABLED(CONFIG_IPV6) - fib6_info_release(fib6_rt_nh->rt); -#endif + nsim_rt6_release(fib6_rt_nh->rt); kfree(fib6_rt_nh); } static struct nsim_fib6_rt * nsim_fib6_rt_create(struct nsim_fib_data *data, - struct fib6_entry_notifier_info *fen6_info) + struct fib6_info **rt_arr, unsigned int nrt6) { - struct fib6_info *iter, *rt = fen6_info->rt; + struct fib6_info *rt = rt_arr[0]; struct nsim_fib6_rt *fib6_rt; int i = 0; int err; - fib6_rt = kzalloc(sizeof(*fib6_rt), GFP_ATOMIC); + fib6_rt = kzalloc(sizeof(*fib6_rt), GFP_KERNEL); if (!fib6_rt) return ERR_PTR(-ENOMEM); @@ -493,32 +513,18 @@ nsim_fib6_rt_create(struct nsim_fib_data *data, */ INIT_LIST_HEAD(&fib6_rt->nh_list); - err = nsim_fib6_rt_nh_add(fib6_rt, rt); - if (err) - goto err_fib_rt_fini; - - if (!fen6_info->nsiblings) - return fib6_rt; - - list_for_each_entry(iter, &rt->fib6_siblings, fib6_siblings) { - if (i == fen6_info->nsiblings) - break; - - err = nsim_fib6_rt_nh_add(fib6_rt, iter); + for (i = 0; i < nrt6; i++) { + err = nsim_fib6_rt_nh_add(fib6_rt, rt_arr[i]); if (err) goto err_fib6_rt_nh_del; - i++; } - WARN_ON_ONCE(i != fen6_info->nsiblings); return fib6_rt; err_fib6_rt_nh_del: - list_for_each_entry_continue_reverse(iter, &rt->fib6_siblings, - fib6_siblings) - nsim_fib6_rt_nh_del(fib6_rt, iter); - nsim_fib6_rt_nh_del(fib6_rt, rt); -err_fib_rt_fini: + for (i--; i >= 0; i--) { + nsim_fib6_rt_nh_del(fib6_rt, rt_arr[i]); + }; nsim_fib_rt_fini(&fib6_rt->common); kfree(fib6_rt); return ERR_PTR(err); @@ -551,47 +557,31 @@ nsim_fib6_rt_lookup(struct rhashtable *fib_rt_ht, const struct fib6_info *rt) } static int nsim_fib6_rt_append(struct nsim_fib_data *data, - struct fib6_entry_notifier_info *fen6_info) + struct nsim_fib6_event *fib6_event) { - struct fib6_info *iter, *rt = fen6_info->rt; + struct fib6_info *rt = fib6_event->rt_arr[0]; struct nsim_fib6_rt *fib6_rt; - int i = 0; - int err; + int i, err; fib6_rt = nsim_fib6_rt_lookup(&data->fib_rt_ht, rt); if (WARN_ON_ONCE(!fib6_rt)) return -EINVAL; - err = nsim_fib6_rt_nh_add(fib6_rt, rt); - if (err) - return err; - rt->trap = true; - - if (!fen6_info->nsiblings) - return 0; - - list_for_each_entry(iter, &rt->fib6_siblings, fib6_siblings) { - if (i == fen6_info->nsiblings) - break; - - err = nsim_fib6_rt_nh_add(fib6_rt, iter); + for (i = 0; i < fib6_event->nrt6; i++) { + err = nsim_fib6_rt_nh_add(fib6_rt, fib6_event->rt_arr[i]); if (err) goto err_fib6_rt_nh_del; - iter->trap = true; - i++; + + fib6_event->rt_arr[i]->trap = true; } - WARN_ON_ONCE(i != fen6_info->nsiblings); return 0; err_fib6_rt_nh_del: - list_for_each_entry_continue_reverse(iter, &rt->fib6_siblings, - fib6_siblings) { - iter->trap = false; - nsim_fib6_rt_nh_del(fib6_rt, iter); + for (i--; i >= 0; i--) { + fib6_event->rt_arr[i]->trap = false; + nsim_fib6_rt_nh_del(fib6_rt, fib6_event->rt_arr[i]); } - rt->trap = false; - nsim_fib6_rt_nh_del(fib6_rt, rt); return err; } @@ -605,49 +595,52 @@ static void nsim_fib6_rt_hw_flags_set(const struct nsim_fib6_rt *fib6_rt, } static int nsim_fib6_rt_add(struct nsim_fib_data *data, - struct nsim_fib6_rt *fib6_rt, - struct netlink_ext_ack *extack) + struct nsim_fib6_rt *fib6_rt) { int err; - err = nsim_fib_account(&data->ipv6.fib, true, extack); - if (err) - return err; - err = rhashtable_insert_fast(&data->fib_rt_ht, &fib6_rt->common.ht_node, nsim_fib_rt_ht_params); - if (err) { - NL_SET_ERR_MSG_MOD(extack, "Failed to insert IPv6 route"); + + if (err) goto err_fib_dismiss; - } + msleep(1); nsim_fib6_rt_hw_flags_set(fib6_rt, true); return 0; err_fib_dismiss: - nsim_fib_account(&data->ipv6.fib, false, extack); + /* Drop the accounting that was increased from the notification + * context when FIB_EVENT_ENTRY_REPLACE was triggered. + */ + nsim_fib_account(&data->ipv6.fib, false); return err; } static int nsim_fib6_rt_replace(struct nsim_fib_data *data, struct nsim_fib6_rt *fib6_rt, - struct nsim_fib6_rt *fib6_rt_old, - struct netlink_ext_ack *extack) + struct nsim_fib6_rt *fib6_rt_old) { int err; - /* We are replacing a route, so no need to change the accounting. */ + /* We are replacing a route, so need to remove the accounting which + * was increased when FIB_EVENT_ENTRY_REPLACE was triggered. + */ + err = nsim_fib_account(&data->ipv6.fib, false); + if (err) + return err; + err = rhashtable_replace_fast(&data->fib_rt_ht, &fib6_rt_old->common.ht_node, &fib6_rt->common.ht_node, nsim_fib_rt_ht_params); - if (err) { - NL_SET_ERR_MSG_MOD(extack, "Failed to replace IPv6 route"); + + if (err) return err; - } + msleep(1); nsim_fib6_rt_hw_flags_set(fib6_rt, true); nsim_fib6_rt_hw_flags_set(fib6_rt_old, false); @@ -657,21 +650,22 @@ static int nsim_fib6_rt_replace(struct nsim_fib_data *data, } static int nsim_fib6_rt_insert(struct nsim_fib_data *data, - struct fib6_entry_notifier_info *fen6_info) + struct nsim_fib6_event *fib6_event) { - struct netlink_ext_ack *extack = fen6_info->info.extack; + struct fib6_info *rt = fib6_event->rt_arr[0]; struct nsim_fib6_rt *fib6_rt, *fib6_rt_old; int err; - fib6_rt = nsim_fib6_rt_create(data, fen6_info); + fib6_rt = nsim_fib6_rt_create(data, fib6_event->rt_arr, + fib6_event->nrt6); if (IS_ERR(fib6_rt)) return PTR_ERR(fib6_rt); - fib6_rt_old = nsim_fib6_rt_lookup(&data->fib_rt_ht, fen6_info->rt); + fib6_rt_old = nsim_fib6_rt_lookup(&data->fib_rt_ht, rt); if (!fib6_rt_old) - err = nsim_fib6_rt_add(data, fib6_rt, extack); + err = nsim_fib6_rt_add(data, fib6_rt); else - err = nsim_fib6_rt_replace(data, fib6_rt, fib6_rt_old, extack); + err = nsim_fib6_rt_replace(data, fib6_rt, fib6_rt_old); if (err) nsim_fib6_rt_destroy(fib6_rt); @@ -679,59 +673,100 @@ static int nsim_fib6_rt_insert(struct nsim_fib_data *data, return err; } -static void -nsim_fib6_rt_remove(struct nsim_fib_data *data, - const struct fib6_entry_notifier_info *fen6_info) +static void nsim_fib6_rt_remove(struct nsim_fib_data *data, + struct nsim_fib6_event *fib6_event) { - struct netlink_ext_ack *extack = fen6_info->info.extack; + struct fib6_info *rt = fib6_event->rt_arr[0]; struct nsim_fib6_rt *fib6_rt; + int i; /* Multipath routes are first added to the FIB trie and only then * notified. If we vetoed the addition, we will get a delete * notification for a route we do not have. Therefore, do not warn if * route was not found. */ - fib6_rt = nsim_fib6_rt_lookup(&data->fib_rt_ht, fen6_info->rt); + fib6_rt = nsim_fib6_rt_lookup(&data->fib_rt_ht, rt); if (!fib6_rt) return; /* If not all the nexthops are deleted, then only reduce the nexthop * group. */ - if (fen6_info->nsiblings + 1 != fib6_rt->nhs) { - nsim_fib6_rt_nh_del(fib6_rt, fen6_info->rt); + if (fib6_event->nrt6 != fib6_rt->nhs) { + for (i = 0; i < fib6_event->nrt6; i++) + nsim_fib6_rt_nh_del(fib6_rt, fib6_event->rt_arr[i]); return; } rhashtable_remove_fast(&data->fib_rt_ht, &fib6_rt->common.ht_node, nsim_fib_rt_ht_params); - nsim_fib_account(&data->ipv6.fib, false, extack); nsim_fib6_rt_destroy(fib6_rt); } +static int nsim_fib6_event_init(struct nsim_fib6_event *fib6_event, + struct fib6_entry_notifier_info *fen6_info) +{ + struct fib6_info *rt = fen6_info->rt; + struct fib6_info **rt_arr; + struct fib6_info *iter; + unsigned int nrt6; + int i = 0; + + nrt6 = fen6_info->nsiblings + 1; + + rt_arr = kcalloc(nrt6, sizeof(struct fib6_info *), GFP_ATOMIC); + if (!rt_arr) + return -ENOMEM; + + fib6_event->rt_arr = rt_arr; + fib6_event->nrt6 = nrt6; + + rt_arr[0] = rt; + fib6_info_hold(rt); + + if (!fen6_info->nsiblings) + return 0; + + list_for_each_entry(iter, &rt->fib6_siblings, fib6_siblings) { + if (i == fen6_info->nsiblings) + break; + + rt_arr[i + 1] = iter; + fib6_info_hold(iter); + i++; + } + WARN_ON_ONCE(i != fen6_info->nsiblings); + + return 0; +} + +static void nsim_fib6_event_fini(struct nsim_fib6_event *fib6_event) +{ + int i; + + for (i = 0; i < fib6_event->nrt6; i++) + nsim_rt6_release(fib6_event->rt_arr[i]); + kfree(fib6_event->rt_arr); +} + static int nsim_fib6_event(struct nsim_fib_data *data, - struct fib_notifier_info *info, + struct nsim_fib6_event *fib6_event, unsigned long event) { - struct fib6_entry_notifier_info *fen6_info; int err = 0; - fen6_info = container_of(info, struct fib6_entry_notifier_info, info); - - if (fen6_info->rt->fib6_src.plen) { - NL_SET_ERR_MSG_MOD(info->extack, "IPv6 source-specific route is not supported"); + if (fib6_event->rt_arr[0]->fib6_src.plen) return 0; - } switch (event) { case FIB_EVENT_ENTRY_REPLACE: - err = nsim_fib6_rt_insert(data, fen6_info); + err = nsim_fib6_rt_insert(data, fib6_event); break; case FIB_EVENT_ENTRY_APPEND: - err = nsim_fib6_rt_append(data, fen6_info); + err = nsim_fib6_rt_append(data, fib6_event); break; case FIB_EVENT_ENTRY_DEL: - nsim_fib6_rt_remove(data, fen6_info); + nsim_fib6_rt_remove(data, fib6_event); break; default: break; @@ -740,48 +775,165 @@ static int nsim_fib6_event(struct nsim_fib_data *data, return err; } -static int nsim_fib_event(struct nsim_fib_data *data, - struct fib_notifier_info *info, unsigned long event) +static int nsim_fib_event(struct nsim_fib_event *fib_event) { int err = 0; - switch (info->family) { + switch (fib_event->family) { case AF_INET: - err = nsim_fib4_event(data, info, event); + nsim_fib4_event(fib_event->data, &fib_event->fen_info, + fib_event->event); + fib_info_put(fib_event->fen_info.fi); break; case AF_INET6: - err = nsim_fib6_event(data, info, event); + nsim_fib6_event(fib_event->data, &fib_event->fib6_event, + fib_event->event); + nsim_fib6_event_fini(&fib_event->fib6_event); break; } return err; } +static int nsim_fib4_prepare_event(struct fib_notifier_info *info, + struct nsim_fib_event *fib_event, + unsigned long event) +{ + struct nsim_fib_data *data = fib_event->data; + struct fib_entry_notifier_info *fen_info; + struct netlink_ext_ack *extack; + int err = 0; + + fen_info = container_of(info, struct fib_entry_notifier_info, + info); + fib_event->fen_info = *fen_info; + extack = info->extack; + + switch (event) { + case FIB_EVENT_ENTRY_REPLACE: + err = nsim_fib_account(&data->ipv4.fib, true); + if (err) { + NL_SET_ERR_MSG_MOD(extack, "Exceeded number of supported fib entries"); + return err; + } + break; + case FIB_EVENT_ENTRY_DEL: + nsim_fib_account(&data->ipv4.fib, false); + break; + } + + /* Take reference on fib_info to prevent it from being + * freed while event is queued. Release it afterwards. + */ + fib_info_hold(fib_event->fen_info.fi); + + return 0; +} + +static int nsim_fib6_prepare_event(struct fib_notifier_info *info, + struct nsim_fib_event *fib_event, + unsigned long event) +{ + struct nsim_fib_data *data = fib_event->data; + struct fib6_entry_notifier_info *fen6_info; + struct netlink_ext_ack *extack; + int err = 0; + + fen6_info = container_of(info, struct fib6_entry_notifier_info, + info); + + err = nsim_fib6_event_init(&fib_event->fib6_event, fen6_info); + if (err) + return err; + + extack = info->extack; + switch (event) { + case FIB_EVENT_ENTRY_REPLACE: + err = nsim_fib_account(&data->ipv6.fib, true); + if (err) { + NL_SET_ERR_MSG_MOD(extack, "Exceeded number of supported fib entries"); + goto err_fib6_event_fini; + } + break; + case FIB_EVENT_ENTRY_DEL: + nsim_fib_account(&data->ipv6.fib, false); + break; + } + + return 0; + +err_fib6_event_fini: + nsim_fib6_event_fini(&fib_event->fib6_event); + return err; +} + +static int nsim_fib_event_schedule_work(struct nsim_fib_data *data, + struct fib_notifier_info *info, + unsigned long event) +{ + struct nsim_fib_event *fib_event; + int err; + + if (info->family != AF_INET && info->family != AF_INET6) + /* netdevsim does not support 'RTNL_FAMILY_IP6MR' and + * 'RTNL_FAMILY_IPMR' and should ignore them. + */ + return NOTIFY_DONE; + + fib_event = kzalloc(sizeof(*fib_event), GFP_ATOMIC); + if (!fib_event) + return NOTIFY_BAD; + + fib_event->data = data; + fib_event->event = event; + fib_event->family = info->family; + + switch (info->family) { + case AF_INET: + err = nsim_fib4_prepare_event(info, fib_event, event); + break; + case AF_INET6: + err = nsim_fib6_prepare_event(info, fib_event, event); + break; + } + + if (err) + goto err_fib_prepare_event; + + /* Enqueue the event and trigger the work */ + spin_lock_bh(&data->fib_event_queue_lock); + list_add_tail(&fib_event->list, &data->fib_event_queue); + spin_unlock_bh(&data->fib_event_queue_lock); + schedule_work(&data->fib_event_work); + + return NOTIFY_DONE; + +err_fib_prepare_event: + kfree(fib_event); + return NOTIFY_BAD; +} + static int nsim_fib_event_nb(struct notifier_block *nb, unsigned long event, void *ptr) { struct nsim_fib_data *data = container_of(nb, struct nsim_fib_data, fib_nb); struct fib_notifier_info *info = ptr; - int err = 0; + int err; switch (event) { case FIB_EVENT_RULE_ADD: case FIB_EVENT_RULE_DEL: err = nsim_fib_rule_event(data, info, event == FIB_EVENT_RULE_ADD); - break; + return notifier_from_errno(err); case FIB_EVENT_ENTRY_REPLACE: case FIB_EVENT_ENTRY_APPEND: case FIB_EVENT_ENTRY_DEL: - /* IPv6 routes can be added via RAs from softIRQ. */ - spin_lock_bh(&data->fib_lock); - err = nsim_fib_event(data, info, event); - spin_unlock_bh(&data->fib_lock); - break; + return nsim_fib_event_schedule_work(data, info, event); } - return notifier_from_errno(err); + return NOTIFY_DONE; } static void nsim_fib4_rt_free(struct nsim_fib_rt *fib_rt, @@ -792,7 +944,7 @@ static void nsim_fib4_rt_free(struct nsim_fib_rt *fib_rt, fib4_rt = container_of(fib_rt, struct nsim_fib4_rt, common); nsim_fib4_rt_hw_flags_set(devlink_net(devlink), fib4_rt, false); - nsim_fib_account(&data->ipv4.fib, false, NULL); + nsim_fib_account(&data->ipv4.fib, false); nsim_fib4_rt_destroy(fib4_rt); } @@ -803,7 +955,7 @@ static void nsim_fib6_rt_free(struct nsim_fib_rt *fib_rt, fib6_rt = container_of(fib_rt, struct nsim_fib6_rt, common); nsim_fib6_rt_hw_flags_set(fib6_rt, false); - nsim_fib_account(&data->ipv6.fib, false, NULL); + nsim_fib_account(&data->ipv6.fib, false); nsim_fib6_rt_destroy(fib6_rt); } @@ -831,6 +983,9 @@ static void nsim_fib_dump_inconsistent(struct notifier_block *nb) fib_nb); struct nsim_fib_rt *fib_rt, *fib_rt_tmp; + /* Flush the work to make sure there is no race with notifications. */ + flush_work(&data->fib_event_work); + /* The notifier block is still not registered, so we do not need to * take any locks here. */ @@ -1101,6 +1256,29 @@ static void nsim_fib_set_max_all(struct nsim_fib_data *data, } } +static void nsim_fib_event_work(struct work_struct *work) +{ + struct nsim_fib_data *data = container_of(work, struct nsim_fib_data, + fib_event_work); + struct nsim_fib_event *fib_event, *next_fib_event; + + LIST_HEAD(fib_event_queue); + + spin_lock_bh(&data->fib_event_queue_lock); + list_splice_init(&data->fib_event_queue, &fib_event_queue); + spin_unlock_bh(&data->fib_event_queue_lock); + + mutex_lock(&data->fib_lock); + list_for_each_entry_safe(fib_event, next_fib_event, &fib_event_queue, + list) { + nsim_fib_event(fib_event); + list_del(&fib_event->list); + kfree(fib_event); + cond_resched(); + } + mutex_unlock(&data->fib_lock); +} + struct nsim_fib_data *nsim_fib_create(struct devlink *devlink, struct netlink_ext_ack *extack) { @@ -1116,12 +1294,16 @@ struct nsim_fib_data *nsim_fib_create(struct devlink *devlink, if (err) goto err_data_free; - spin_lock_init(&data->fib_lock); + mutex_init(&data->fib_lock); INIT_LIST_HEAD(&data->fib_rt_list); err = rhashtable_init(&data->fib_rt_ht, &nsim_fib_rt_ht_params); if (err) goto err_rhashtable_nexthop_destroy; + INIT_WORK(&data->fib_event_work, nsim_fib_event_work); + INIT_LIST_HEAD(&data->fib_event_queue); + spin_lock_init(&data->fib_event_queue_lock); + nsim_fib_set_max_all(data, devlink); data->nexthop_nb.notifier_call = nsim_nexthop_event_nb; @@ -1165,11 +1347,13 @@ struct nsim_fib_data *nsim_fib_create(struct devlink *devlink, err_nexthop_nb_unregister: unregister_nexthop_notifier(devlink_net(devlink), &data->nexthop_nb); err_rhashtable_fib_destroy: + flush_work(&data->fib_event_work); rhashtable_free_and_destroy(&data->fib_rt_ht, nsim_fib_rt_free, data); err_rhashtable_nexthop_destroy: rhashtable_free_and_destroy(&data->nexthop_ht, nsim_nexthop_free, data); + mutex_destroy(&data->fib_lock); err_data_free: kfree(data); return ERR_PTR(err); @@ -1189,10 +1373,13 @@ void nsim_fib_destroy(struct devlink *devlink, struct nsim_fib_data *data) NSIM_RESOURCE_IPV4_FIB); unregister_fib_notifier(devlink_net(devlink), &data->fib_nb); unregister_nexthop_notifier(devlink_net(devlink), &data->nexthop_nb); + flush_work(&data->fib_event_work); rhashtable_free_and_destroy(&data->fib_rt_ht, nsim_fib_rt_free, data); rhashtable_free_and_destroy(&data->nexthop_ht, nsim_nexthop_free, data); + WARN_ON_ONCE(!list_empty(&data->fib_event_queue)); WARN_ON_ONCE(!list_empty(&data->fib_rt_list)); + mutex_destroy(&data->fib_lock); kfree(data); } From patchwork Mon Feb 1 19:47:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ido Schimmel X-Patchwork-Id: 374508 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76A1BC433DB for ; Mon, 1 Feb 2021 19:52:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3A9DD64EA2 for ; Mon, 1 Feb 2021 19:52:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232757AbhBATwy (ORCPT ); Mon, 1 Feb 2021 14:52:54 -0500 Received: from new1-smtp.messagingengine.com ([66.111.4.221]:42651 "EHLO new1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232830AbhBATwB (ORCPT ); Mon, 1 Feb 2021 14:52:01 -0500 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.nyi.internal (Postfix) with ESMTP id CF80658050D; Mon, 1 Feb 2021 14:48:52 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Mon, 01 Feb 2021 14:48:52 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=+lWoQT5XNEZ4fTxttEVbaXY/68CwGOEy0JypdjT014M=; b=cGI3RhdI KSdleJ+x6Lz68OlYszzRnOkoqOWKpndrUsJ0+UD3/jB3hCuCFoRTxDpOegKHWaue khD94ZpEnoWugUNqDSuYdH6jfuPCyMVYfMI/6nulYqwdHvjcmFJCAF/fyT8hk5xP Aaqr39sPvnzGRtqX/aZuH1x34wIepY/SYqV3Ht4+uodVYWyGY13oxMXoKeaztuOg AcSO5wz2NxXwXTptWQoQtNYu0+4EOrcydpY3fw7uFvqqY67FRtPE/rfRQg9DlNQ1 4uQuGv3KTWSU30Oqaqq96P+tf1SBc5X00H+4okdMC2fYP6Fz267y9S2ayLG3glzu yZy8IVtChFKi9w== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrfeekgddufedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkofgjfhgggfestdekre dtredttdenucfhrhhomhepkfguohcuufgthhhimhhmvghluceoihguohhstghhsehiugho shgthhdrohhrgheqnecuggftrfgrthhtvghrnhepudetieevffffveelkeeljeffkefhke ehgfdtffethfelvdejgffghefgveejkefhnecukfhppeekgedrvddvledrudehfedrgeeg necuvehluhhsthgvrhfuihiivgepgeenucfrrghrrghmpehmrghilhhfrhhomhepihguoh hstghhsehiughoshgthhdrohhrgh X-ME-Proxy: Received: from shredder.mellanox.com (igld-84-229-153-44.inter.net.il [84.229.153.44]) by mail.messagingengine.com (Postfix) with ESMTPA id 523D624005C; Mon, 1 Feb 2021 14:48:50 -0500 (EST) From: Ido Schimmel To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, dsahern@gmail.com, yoshfuji@linux-ipv6.org, jiri@nvidia.com, amcohen@nvidia.com, roopa@nvidia.com, bpoirier@nvidia.com, sharpd@nvidia.com, mlxsw@nvidia.com, Ido Schimmel Subject: [PATCH net-next v2 05/10] net: ipv4: Emit notification when fib hardware flags are changed Date: Mon, 1 Feb 2021 21:47:52 +0200 Message-Id: <20210201194757.3463461-6-idosch@idosch.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210201194757.3463461-1-idosch@idosch.org> References: <20210201194757.3463461-1-idosch@idosch.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Amit Cohen After installing a route to the kernel, user space receives an acknowledgment, which means the route was installed in the kernel, but not necessarily in hardware. The asynchronous nature of route installation in hardware can lead to a routing daemon advertising a route before it was actually installed in hardware. This can result in packet loss or mis-routed packets until the route is installed in hardware. It is also possible for a route already installed in hardware to change its action and therefore its flags. For example, a host route that is trapping packets can be "promoted" to perform decapsulation following the installation of an IPinIP/VXLAN tunnel. Emit RTM_NEWROUTE notifications whenever RTM_F_OFFLOAD/RTM_F_TRAP flags are changed. The aim is to provide an indication to user-space (e.g., routing daemons) about the state of the route in hardware. Introduce a sysctl that controls this behavior. Keep the default value at 0 (i.e., do not emit notifications) for several reasons: - Multiple RTM_NEWROUTE notification per-route might confuse existing routing daemons. - Convergence reasons in routing daemons. - The extra notifications will negatively impact the insertion rate. - Not all users are interested in these notifications. Signed-off-by: Amit Cohen Acked-by: Roopa Prabhu Signed-off-by: Ido Schimmel Reviewed-by: David Ahern --- Documentation/networking/ip-sysctl.rst | 20 +++++++++++++++++++ include/net/netns/ipv4.h | 2 ++ net/ipv4/af_inet.c | 2 ++ net/ipv4/fib_trie.c | 27 ++++++++++++++++++++++++++ net/ipv4/sysctl_net_ipv4.c | 9 +++++++++ 5 files changed, 60 insertions(+) diff --git a/Documentation/networking/ip-sysctl.rst b/Documentation/networking/ip-sysctl.rst index f0353fb751d1..06568aceb223 100644 --- a/Documentation/networking/ip-sysctl.rst +++ b/Documentation/networking/ip-sysctl.rst @@ -178,6 +178,26 @@ min_adv_mss - INTEGER The advertised MSS depends on the first hop route MTU, but will never be lower than this setting. +fib_notify_on_flag_change - INTEGER + Whether to emit RTM_NEWROUTE notifications whenever RTM_F_OFFLOAD/ + RTM_F_TRAP flags are changed. + + After installing a route to the kernel, user space receives an + acknowledgment, which means the route was installed in the kernel, + but not necessarily in hardware. + It is also possible for a route already installed in hardware to change + its action and therefore its flags. For example, a host route that is + trapping packets can be "promoted" to perform decapsulation following + the installation of an IPinIP/VXLAN tunnel. + The notifications will indicate to user-space the state of the route. + + Default: 0 (Do not emit notifications.) + + Possible values: + + - 0 - Do not emit notifications. + - 1 - Emit notifications. + IP Fragmentation: ipfrag_high_thresh - LONG INTEGER diff --git a/include/net/netns/ipv4.h b/include/net/netns/ipv4.h index 8e4fcac4df72..70a2a085dd1a 100644 --- a/include/net/netns/ipv4.h +++ b/include/net/netns/ipv4.h @@ -188,6 +188,8 @@ struct netns_ipv4 { int sysctl_udp_wmem_min; int sysctl_udp_rmem_min; + int sysctl_fib_notify_on_flag_change; + #ifdef CONFIG_NET_L3_MASTER_DEV int sysctl_udp_l3mdev_accept; #endif diff --git a/net/ipv4/af_inet.c b/net/ipv4/af_inet.c index b94fa8eb831b..ab42f6404fc6 100644 --- a/net/ipv4/af_inet.c +++ b/net/ipv4/af_inet.c @@ -1871,6 +1871,8 @@ static __net_init int inet_init_net(struct net *net) net->ipv4.sysctl_igmp_llm_reports = 1; net->ipv4.sysctl_igmp_qrv = 2; + net->ipv4.sysctl_fib_notify_on_flag_change = 0; + return 0; } diff --git a/net/ipv4/fib_trie.c b/net/ipv4/fib_trie.c index 28117c05dc35..60559b708158 100644 --- a/net/ipv4/fib_trie.c +++ b/net/ipv4/fib_trie.c @@ -1038,6 +1038,8 @@ fib_find_matching_alias(struct net *net, const struct fib_rt_info *fri) void fib_alias_hw_flags_set(struct net *net, const struct fib_rt_info *fri) { struct fib_alias *fa_match; + struct sk_buff *skb; + int err; rcu_read_lock(); @@ -1045,9 +1047,34 @@ void fib_alias_hw_flags_set(struct net *net, const struct fib_rt_info *fri) if (!fa_match) goto out; + if (fa_match->offload == fri->offload && fa_match->trap == fri->trap) + goto out; + fa_match->offload = fri->offload; fa_match->trap = fri->trap; + if (!net->ipv4.sysctl_fib_notify_on_flag_change) + goto out; + + skb = nlmsg_new(fib_nlmsg_size(fa_match->fa_info), GFP_ATOMIC); + if (!skb) { + err = -ENOBUFS; + goto errout; + } + + err = fib_dump_info(skb, 0, 0, RTM_NEWROUTE, fri, 0); + if (err < 0) { + /* -EMSGSIZE implies BUG in fib_nlmsg_size() */ + WARN_ON(err == -EMSGSIZE); + kfree_skb(skb); + goto errout; + } + + rtnl_notify(skb, net, 0, RTNLGRP_IPV4_ROUTE, NULL, GFP_ATOMIC); + goto out; + +errout: + rtnl_set_sk_err(net, RTNLGRP_IPV4_ROUTE, err); out: rcu_read_unlock(); } diff --git a/net/ipv4/sysctl_net_ipv4.c b/net/ipv4/sysctl_net_ipv4.c index 3e5f4f2e705e..e5798b3b59d2 100644 --- a/net/ipv4/sysctl_net_ipv4.c +++ b/net/ipv4/sysctl_net_ipv4.c @@ -1354,6 +1354,15 @@ static struct ctl_table ipv4_net_table[] = { .proc_handler = proc_dointvec_minmax, .extra1 = SYSCTL_ONE }, + { + .procname = "fib_notify_on_flag_change", + .data = &init_net.ipv4.sysctl_fib_notify_on_flag_change, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = proc_dointvec_minmax, + .extra1 = SYSCTL_ZERO, + .extra2 = SYSCTL_ONE, + }, { } }; From patchwork Mon Feb 1 19:47:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ido Schimmel X-Patchwork-Id: 374507 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAC8EC43381 for ; Mon, 1 Feb 2021 19:53:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8B9A764EA2 for ; Mon, 1 Feb 2021 19:53:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232937AbhBATxV (ORCPT ); Mon, 1 Feb 2021 14:53:21 -0500 Received: from new1-smtp.messagingengine.com ([66.111.4.221]:60589 "EHLO new1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232832AbhBATwC (ORCPT ); Mon, 1 Feb 2021 14:52:02 -0500 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.nyi.internal (Postfix) with ESMTP id 99620580512; Mon, 1 Feb 2021 14:48:57 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Mon, 01 Feb 2021 14:48:57 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=zD84T9Sw3RVG4yqvX6HApKCcETwSXu4ZVwlDkoq0KMo=; b=LF3bMYA5 BEVS30DEmtOlykzMEmGophtZzFJ4epEhVubhBeXGoGNbLOdyrdMHyvEIKo2kud98 ENLrys5Ooasph/6ub6mCm7/x1fi00i8sbDqnOBp+I409n0z80YIF5t1mKVSg0pU/ FXfv19udS6E2kYSqXlBI72n9YoOqN6SxCE7ne0NP3v2YCtT62qfmFqwqoHINEBhK y0fGI9B293JbsCFTZw0YAQYmZR2gZ5y5gh+C/goR5FJELgEmq8TLqSFROHv3UxxL d8H3NdBidCJ2OlQi5x/utYHJp0aiGXZsLhEgyzIELucq4sLM0iR9K6qgVg3xXgXI GI5Hsp6p3LPK0g== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrfeekgddufedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkofgjfhgggfestdekre dtredttdenucfhrhhomhepkfguohcuufgthhhimhhmvghluceoihguohhstghhsehiugho shgthhdrohhrgheqnecuggftrfgrthhtvghrnhepudetieevffffveelkeeljeffkefhke ehgfdtffethfelvdejgffghefgveejkefhnecukfhppeekgedrvddvledrudehfedrgeeg necuvehluhhsthgvrhfuihiivgepgeenucfrrghrrghmpehmrghilhhfrhhomhepihguoh hstghhsehiughoshgthhdrohhrgh X-ME-Proxy: Received: from shredder.mellanox.com (igld-84-229-153-44.inter.net.il [84.229.153.44]) by mail.messagingengine.com (Postfix) with ESMTPA id 59A5324005C; Mon, 1 Feb 2021 14:48:55 -0500 (EST) From: Ido Schimmel To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, dsahern@gmail.com, yoshfuji@linux-ipv6.org, jiri@nvidia.com, amcohen@nvidia.com, roopa@nvidia.com, bpoirier@nvidia.com, sharpd@nvidia.com, mlxsw@nvidia.com, Ido Schimmel Subject: [PATCH net-next v2 07/10] net: Do not call fib6_info_hw_flags_set() when IPv6 is disabled Date: Mon, 1 Feb 2021 21:47:54 +0200 Message-Id: <20210201194757.3463461-8-idosch@idosch.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210201194757.3463461-1-idosch@idosch.org> References: <20210201194757.3463461-1-idosch@idosch.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Amit Cohen With the next patch mlxsw and netdevsim will fail in compilation if CONFIG_IPV6 is disabled. Do not call fib6_info_hw_flags_set() when IPv6 is disabled. Signed-off-by: Amit Cohen Signed-off-by: Ido Schimmel --- .../ethernet/mellanox/mlxsw/spectrum_router.c | 16 ++++++++++++++++ drivers/net/netdevsim/fib.c | 8 ++++++++ 2 files changed, 24 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c index 5516e141f5bf..cf111e73f81e 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c @@ -4988,6 +4988,7 @@ mlxsw_sp_fib4_entry_hw_flags_clear(struct mlxsw_sp *mlxsw_sp, fib_alias_hw_flags_set(mlxsw_sp_net(mlxsw_sp), &fri); } +#if IS_ENABLED(CONFIG_IPV6) static void mlxsw_sp_fib6_entry_hw_flags_set(struct mlxsw_sp *mlxsw_sp, struct mlxsw_sp_fib_entry *fib_entry) @@ -5007,7 +5008,15 @@ mlxsw_sp_fib6_entry_hw_flags_set(struct mlxsw_sp *mlxsw_sp, fib6_info_hw_flags_set(mlxsw_sp_net(mlxsw_sp), mlxsw_sp_rt6->rt, should_offload, !should_offload); } +#else +static void +mlxsw_sp_fib6_entry_hw_flags_set(struct mlxsw_sp *mlxsw_sp, + struct mlxsw_sp_fib_entry *fib_entry) +{ +} +#endif +#if IS_ENABLED(CONFIG_IPV6) static void mlxsw_sp_fib6_entry_hw_flags_clear(struct mlxsw_sp *mlxsw_sp, struct mlxsw_sp_fib_entry *fib_entry) @@ -5021,6 +5030,13 @@ mlxsw_sp_fib6_entry_hw_flags_clear(struct mlxsw_sp *mlxsw_sp, fib6_info_hw_flags_set(mlxsw_sp_net(mlxsw_sp), mlxsw_sp_rt6->rt, false, false); } +#else +static void +mlxsw_sp_fib6_entry_hw_flags_clear(struct mlxsw_sp *mlxsw_sp, + struct mlxsw_sp_fib_entry *fib_entry) +{ +} +#endif static void mlxsw_sp_fib_entry_hw_flags_set(struct mlxsw_sp *mlxsw_sp, diff --git a/drivers/net/netdevsim/fib.c b/drivers/net/netdevsim/fib.c index cb68f0cc6740..1779146926a5 100644 --- a/drivers/net/netdevsim/fib.c +++ b/drivers/net/netdevsim/fib.c @@ -585,6 +585,7 @@ static int nsim_fib6_rt_append(struct nsim_fib_data *data, return err; } +#if IS_ENABLED(CONFIG_IPV6) static void nsim_fib6_rt_hw_flags_set(struct nsim_fib_data *data, const struct nsim_fib6_rt *fib6_rt, bool trap) @@ -595,6 +596,13 @@ static void nsim_fib6_rt_hw_flags_set(struct nsim_fib_data *data, list_for_each_entry(fib6_rt_nh, &fib6_rt->nh_list, list) fib6_info_hw_flags_set(net, fib6_rt_nh->rt, false, trap); } +#else +static void nsim_fib6_rt_hw_flags_set(struct nsim_fib_data *data, + const struct nsim_fib6_rt *fib6_rt, + bool trap) +{ +} +#endif static int nsim_fib6_rt_add(struct nsim_fib_data *data, struct nsim_fib6_rt *fib6_rt) From patchwork Mon Feb 1 19:47:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ido Schimmel X-Patchwork-Id: 374506 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 534C8C433E0 for ; Mon, 1 Feb 2021 19:54:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 048BF64EC3 for ; Mon, 1 Feb 2021 19:54:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232374AbhBATws (ORCPT ); Mon, 1 Feb 2021 14:52:48 -0500 Received: from new1-smtp.messagingengine.com ([66.111.4.221]:51599 "EHLO new1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232840AbhBATwC (ORCPT ); Mon, 1 Feb 2021 14:52:02 -0500 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.nyi.internal (Postfix) with ESMTP id 5C271580518; Mon, 1 Feb 2021 14:49:02 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Mon, 01 Feb 2021 14:49:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=4ns3nr/ZKGSjpfRNiHT1j2e5Yq7+hcHMqH9HTf52zRs=; b=WDf21+hE ZHC3rcYGi3iReWUxPrmiOii7w4oI9lv2L3Si8X8SJeaaTRGNP0zcSVMohrMiq2PG 2yzQU/pwmZkp6DHfISoYdrBif0BEvLig50g64RxRjy68bOPMpwQrfQVEMBu5iSTb a7mPmCvS0X6Z8sV1NwzeWf/S929pLEfrN6yFdmaDIQ7Nc2x26WcyS2KCmmMWktzj qi5/C54GwwiN2B/Juv9FKNq7jmKepeRoG+hRWBhi+2bdhSnvMmy4mNRil8UnmKTi Q9s2nOeeM1hy8/wgh0w1z/DKgQkdqFEvnc5Io0BQIEcU9M5Nc3diqr6YeeC6CuyM m5NJ73wdi22Zzw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrfeekgddufedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkofgjfhgggfestdekre dtredttdenucfhrhhomhepkfguohcuufgthhhimhhmvghluceoihguohhstghhsehiugho shgthhdrohhrgheqnecuggftrfgrthhtvghrnhepudetieevffffveelkeeljeffkefhke ehgfdtffethfelvdejgffghefgveejkefhnecukfhppeekgedrvddvledrudehfedrgeeg necuvehluhhsthgvrhfuihiivgepjeenucfrrghrrghmpehmrghilhhfrhhomhepihguoh hstghhsehiughoshgthhdrohhrgh X-ME-Proxy: Received: from shredder.mellanox.com (igld-84-229-153-44.inter.net.il [84.229.153.44]) by mail.messagingengine.com (Postfix) with ESMTPA id 0742B24005B; Mon, 1 Feb 2021 14:48:59 -0500 (EST) From: Ido Schimmel To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, dsahern@gmail.com, yoshfuji@linux-ipv6.org, jiri@nvidia.com, amcohen@nvidia.com, roopa@nvidia.com, bpoirier@nvidia.com, sharpd@nvidia.com, mlxsw@nvidia.com, Ido Schimmel Subject: [PATCH net-next v2 09/10] selftests: Extend fib tests to run with and without flags notifications Date: Mon, 1 Feb 2021 21:47:56 +0200 Message-Id: <20210201194757.3463461-10-idosch@idosch.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210201194757.3463461-1-idosch@idosch.org> References: <20210201194757.3463461-1-idosch@idosch.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Amit Cohen Run the test cases with both `fib_notify_on_flag_change` sysctls set to '1', and then with both sysctls set to '0' to verify there are no regressions in the test when notifications are added. Signed-off-by: Amit Cohen Signed-off-by: Ido Schimmel --- tools/testing/selftests/drivers/net/mlxsw/fib.sh | 14 ++++++++++++++ .../testing/selftests/drivers/net/netdevsim/fib.sh | 14 ++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/tools/testing/selftests/drivers/net/mlxsw/fib.sh b/tools/testing/selftests/drivers/net/mlxsw/fib.sh index eab79b9e58cd..dcbf32b99bb6 100755 --- a/tools/testing/selftests/drivers/net/mlxsw/fib.sh +++ b/tools/testing/selftests/drivers/net/mlxsw/fib.sh @@ -225,6 +225,16 @@ ipv6_local_replace() ip -n $ns link del dev dummy1 } +fib_notify_on_flag_change_set() +{ + local notify=$1; shift + + ip netns exec testns1 sysctl -qw net.ipv4.fib_notify_on_flag_change=$notify + ip netns exec testns1 sysctl -qw net.ipv6.fib_notify_on_flag_change=$notify + + log_info "Set fib_notify_on_flag_change to $notify" +} + setup_prepare() { ip netns add testns1 @@ -251,6 +261,10 @@ trap cleanup EXIT setup_prepare +fib_notify_on_flag_change_set 1 +tests_run + +fib_notify_on_flag_change_set 0 tests_run exit $EXIT_STATUS diff --git a/tools/testing/selftests/drivers/net/netdevsim/fib.sh b/tools/testing/selftests/drivers/net/netdevsim/fib.sh index 2f87c3be76a9..251f228ce63e 100755 --- a/tools/testing/selftests/drivers/net/netdevsim/fib.sh +++ b/tools/testing/selftests/drivers/net/netdevsim/fib.sh @@ -302,6 +302,16 @@ ipv6_error_path() ipv6_error_path_replay } +fib_notify_on_flag_change_set() +{ + local notify=$1; shift + + ip netns exec testns1 sysctl -qw net.ipv4.fib_notify_on_flag_change=$notify + ip netns exec testns1 sysctl -qw net.ipv6.fib_notify_on_flag_change=$notify + + log_info "Set fib_notify_on_flag_change to $notify" +} + setup_prepare() { local netdev @@ -336,6 +346,10 @@ trap cleanup EXIT setup_prepare +fib_notify_on_flag_change_set 1 +tests_run + +fib_notify_on_flag_change_set 0 tests_run exit $EXIT_STATUS From patchwork Mon Feb 1 19:47:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ido Schimmel X-Patchwork-Id: 374505 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF2ACC433E0 for ; Mon, 1 Feb 2021 19:55:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7914964ECA for ; Mon, 1 Feb 2021 19:55:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232605AbhBATzb (ORCPT ); Mon, 1 Feb 2021 14:55:31 -0500 Received: from new1-smtp.messagingengine.com ([66.111.4.221]:47619 "EHLO new1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232842AbhBATwD (ORCPT ); Mon, 1 Feb 2021 14:52:03 -0500 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailnew.nyi.internal (Postfix) with ESMTP id 081B858050F; Mon, 1 Feb 2021 14:49:05 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Mon, 01 Feb 2021 14:49:05 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=DTAro0Iq7h1cN1i5oc1r8aEXcJoMDGIT1BJIVT2juF8=; b=gRr6slEc IKmNsM5+YmCKN64RktILg9imRpFzvhiTpqk9epjttwZ/Vy3nZ0lUD2pPRJ8Voznx uxPYVmMTvDxuhqAMkMjz3pk+Kv+U1E4oQcIb7RfqGtB4gkKYXgE6iVRvh7r86OGZ HBoIUChRgjHLfCNWuiHzX1gAaUrBjMvs0sYJjTS6bYODBrUqMa97qvL52S7oXplU uugvXTqNRFslfWUQg26Kf2erh1L8j5JuA068my/BJ6ZZmFv5pgfjeXpODI3SyXiv jFldl+rBCQvhPwRJoP8x9BVxb3ic6GKTSQfkE+adp/NwSUyGTRc0m1SFtk0ThPOA hgwBg58mdecWpw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrfeekgddufedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkofgjfhgggfestdekre dtredttdenucfhrhhomhepkfguohcuufgthhhimhhmvghluceoihguohhstghhsehiugho shgthhdrohhrgheqnecuggftrfgrthhtvghrnhepudetieevffffveelkeeljeffkefhke ehgfdtffethfelvdejgffghefgveejkefhnecukfhppeekgedrvddvledrudehfedrgeeg necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepihguoh hstghhsehiughoshgthhdrohhrgh X-ME-Proxy: Received: from shredder.mellanox.com (igld-84-229-153-44.inter.net.il [84.229.153.44]) by mail.messagingengine.com (Postfix) with ESMTPA id 8FD2624005B; Mon, 1 Feb 2021 14:49:02 -0500 (EST) From: Ido Schimmel To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, dsahern@gmail.com, yoshfuji@linux-ipv6.org, jiri@nvidia.com, amcohen@nvidia.com, roopa@nvidia.com, bpoirier@nvidia.com, sharpd@nvidia.com, mlxsw@nvidia.com, Ido Schimmel Subject: [PATCH net-next v2 10/10] selftests: netdevsim: Add fib_notifications test Date: Mon, 1 Feb 2021 21:47:57 +0200 Message-Id: <20210201194757.3463461-11-idosch@idosch.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210201194757.3463461-1-idosch@idosch.org> References: <20210201194757.3463461-1-idosch@idosch.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Amit Cohen Add test to check fib notifications behavior. The test checks route addition, route deletion and route replacement for both IPv4 and IPv6. When fib_notify_on_flag_change=0, expect single notification for route addition/deletion/replacement. When fib_notify_on_flag_change=1, expect: - two notification for route addition/replacement, first without RTM_F_TRAP and second with RTM_F_TRAP. - single notification for route deletion. $ ./fib_notifications.sh TEST: IPv4 route addition [ OK ] TEST: IPv4 route deletion [ OK ] TEST: IPv4 route replacement [ OK ] TEST: IPv6 route addition [ OK ] TEST: IPv6 route deletion [ OK ] TEST: IPv6 route replacement [ OK ] Signed-off-by: Amit Cohen Signed-off-by: Ido Schimmel Reviewed-by: David Ahern --- .../net/netdevsim/fib_notifications.sh | 300 ++++++++++++++++++ 1 file changed, 300 insertions(+) create mode 100755 tools/testing/selftests/drivers/net/netdevsim/fib_notifications.sh diff --git a/tools/testing/selftests/drivers/net/netdevsim/fib_notifications.sh b/tools/testing/selftests/drivers/net/netdevsim/fib_notifications.sh new file mode 100755 index 000000000000..16a9dd43aefc --- /dev/null +++ b/tools/testing/selftests/drivers/net/netdevsim/fib_notifications.sh @@ -0,0 +1,300 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +lib_dir=$(dirname $0)/../../../net/forwarding + +ALL_TESTS=" + ipv4_route_addition_test + ipv4_route_deletion_test + ipv4_route_replacement_test + ipv6_route_addition_test + ipv6_route_deletion_test + ipv6_route_replacement_test +" + +NETDEVSIM_PATH=/sys/bus/netdevsim/ +DEV_ADDR=1337 +DEV=netdevsim${DEV_ADDR} +DEVLINK_DEV=netdevsim/${DEV} +SYSFS_NET_DIR=/sys/bus/netdevsim/devices/$DEV/net/ +NUM_NETIFS=0 +source $lib_dir/lib.sh + +check_rt_trap() +{ + local outfile=$1; shift + local line + + # Make sure that the first notification was emitted without RTM_F_TRAP + # flag and the second with RTM_F_TRAP flag + head -n 1 $outfile | grep -q "rt_trap" + if [[ $? -eq 0 ]]; then + return 1 + fi + + head -n 2 $outfile | tail -n 1 | grep -q "rt_trap" +} + +route_notify_check() +{ + local outfile=$1; shift + local expected_num_lines=$1; shift + + # check the monitor results + lines=`wc -l $outfile | cut "-d " -f1` + test $lines -eq $expected_num_lines + check_err $? "$expected_num_lines notifications were expected but $lines were received" + + if [[ $expected_num_lines -eq 2 ]]; then + check_rt_trap $outfile + check_err $? "Wrong RTM_F_TRAP flags in notifications" + fi +} + +route_addition_check() +{ + local ip=$1; shift + local notify=$1; shift + local route=$1; shift + local expected_num_notifications=$1; shift + + ip netns exec testns1 sysctl -qw net.$ip.fib_notify_on_flag_change=$notify + + local outfile=$(mktemp) + + $IP monitor route &> $outfile & + sleep 1 + $IP route add $route dev dummy1 + sleep 1 + kill %% && wait %% &> /dev/null + + route_notify_check $outfile $expected_num_notifications + rm -f $outfile + + $IP route del $route dev dummy1 +} + +ipv4_route_addition_test() +{ + RET=0 + + local ip="ipv4" + local route=192.0.2.0/24 + + # Make sure a single notification will be emitted for the programmed + # route. + local notify=0 + local expected_num_notifications=1 + # route_addition_check will assign value to RET. + route_addition_check $ip $notify $route $expected_num_notifications + + # Make sure two notifications will be emitted for the programmed route. + notify=1 + expected_num_notifications=2 + route_addition_check $ip $notify $route $expected_num_notifications + + log_test "IPv4 route addition" +} + +route_deletion_check() +{ + local ip=$1; shift + local notify=$1; shift + local route=$1; shift + local expected_num_notifications=$1; shift + + ip netns exec testns1 sysctl -qw net.$ip.fib_notify_on_flag_change=$notify + $IP route add $route dev dummy1 + sleep 1 + + local outfile=$(mktemp) + + $IP monitor route &> $outfile & + sleep 1 + $IP route del $route dev dummy1 + sleep 1 + kill %% && wait %% &> /dev/null + + route_notify_check $outfile $expected_num_notifications + rm -f $outfile +} + +ipv4_route_deletion_test() +{ + RET=0 + + local ip="ipv4" + local route=192.0.2.0/24 + local expected_num_notifications=1 + + # Make sure a single notification will be emitted for the deleted route, + # regardless of fib_notify_on_flag_change value. + local notify=0 + # route_deletion_check will assign value to RET. + route_deletion_check $ip $notify $route $expected_num_notifications + + notify=1 + route_deletion_check $ip $notify $route $expected_num_notifications + + log_test "IPv4 route deletion" +} + +route_replacement_check() +{ + local ip=$1; shift + local notify=$1; shift + local route=$1; shift + local expected_num_notifications=$1; shift + + ip netns exec testns1 sysctl -qw net.$ip.fib_notify_on_flag_change=$notify + $IP route add $route dev dummy1 + sleep 1 + + local outfile=$(mktemp) + + $IP monitor route &> $outfile & + sleep 1 + $IP route replace $route dev dummy2 + sleep 1 + kill %% && wait %% &> /dev/null + + route_notify_check $outfile $expected_num_notifications + rm -f $outfile + + $IP route del $route dev dummy2 +} + +ipv4_route_replacement_test() +{ + RET=0 + + local ip="ipv4" + local route=192.0.2.0/24 + + $IP link add name dummy2 type dummy + $IP link set dev dummy2 up + + # Make sure a single notification will be emitted for the new route. + local notify=0 + local expected_num_notifications=1 + # route_replacement_check will assign value to RET. + route_replacement_check $ip $notify $route $expected_num_notifications + + # Make sure two notifications will be emitted for the new route. + notify=1 + expected_num_notifications=2 + route_replacement_check $ip $notify $route $expected_num_notifications + + $IP link del name dummy2 + + log_test "IPv4 route replacement" +} + +ipv6_route_addition_test() +{ + RET=0 + + local ip="ipv6" + local route=2001:db8:1::/64 + + # Make sure a single notification will be emitted for the programmed + # route. + local notify=0 + local expected_num_notifications=1 + route_addition_check $ip $notify $route $expected_num_notifications + + # Make sure two notifications will be emitted for the programmed route. + notify=1 + expected_num_notifications=2 + route_addition_check $ip $notify $route $expected_num_notifications + + log_test "IPv6 route addition" +} + +ipv6_route_deletion_test() +{ + RET=0 + + local ip="ipv6" + local route=2001:db8:1::/64 + local expected_num_notifications=1 + + # Make sure a single notification will be emitted for the deleted route, + # regardless of fib_notify_on_flag_change value. + local notify=0 + route_deletion_check $ip $notify $route $expected_num_notifications + + notify=1 + route_deletion_check $ip $notify $route $expected_num_notifications + + log_test "IPv6 route deletion" +} + +ipv6_route_replacement_test() +{ + RET=0 + + local ip="ipv6" + local route=2001:db8:1::/64 + + $IP link add name dummy2 type dummy + $IP link set dev dummy2 up + + # Make sure a single notification will be emitted for the new route. + local notify=0 + local expected_num_notifications=1 + route_replacement_check $ip $notify $route $expected_num_notifications + + # Make sure two notifications will be emitted for the new route. + notify=1 + expected_num_notifications=2 + route_replacement_check $ip $notify $route $expected_num_notifications + + $IP link del name dummy2 + + log_test "IPv6 route replacement" +} + +setup_prepare() +{ + modprobe netdevsim &> /dev/null + echo "$DEV_ADDR 1" > ${NETDEVSIM_PATH}/new_device + while [ ! -d $SYSFS_NET_DIR ] ; do :; done + + ip netns add testns1 + + if [ $? -ne 0 ]; then + echo "Failed to add netns \"testns1\"" + exit 1 + fi + + devlink dev reload $DEVLINK_DEV netns testns1 + + if [ $? -ne 0 ]; then + echo "Failed to reload into netns \"testns1\"" + exit 1 + fi + + IP="ip -n testns1" + + $IP link add name dummy1 type dummy + $IP link set dev dummy1 up +} + +cleanup() +{ + pre_cleanup + + $IP link del name dummy1 + ip netns del testns1 + echo "$DEV_ADDR" > ${NETDEVSIM_PATH}/del_device + modprobe -r netdevsim &> /dev/null +} + +trap cleanup EXIT + +setup_prepare + +tests_run + +exit $EXIT_STATUS