From patchwork Fri May 5 19:46:26 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arnd Bergmann X-Patchwork-Id: 98676 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp292740qge; Fri, 5 May 2017 12:48:27 -0700 (PDT) X-Received: by 10.98.75.6 with SMTP id y6mr12273505pfa.157.1494013707853; Fri, 05 May 2017 12:48:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1494013707; cv=none; d=google.com; s=arc-20160816; b=iDrvNO1G1YGiCDHcsTiaJevKe7YK0zXxrHjk04eBsxE5v72zO7we+f1Mi0KQJG9z5T OyqiE9LxELJQipM62wfbyIqR2kZJE1LzlT9ZYmVB/0xkmNmOQPP3Ni97LMJcXnNlmn0h fygjmzErOjzzGozRt2rGR4ToS64O5MwY536PYWB1dj69eVHfFur5C7OPcd5VJkzAL/pj uguAe9CKdUKji6iQm4gAsZvnzn6NIM0XyIK0D/8sS+oJRuV0KBKUZxPtLm1rCXErnZ8c yLnTEx4Xm1xIdQn0bUVDLhwIBHcRmLFedt8wsfTmSEvRfyJ1pfqeIp6ogHQXrznORUmU O+rA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=cROrFUYhqoLti1CGtx6njl34q75B1Z+iblvv2ZqYCj0=; b=IhT4Y9Y79z52REMIMUr2ZlhEE0wG+NMVaYYRNrypu/ZjmJqnAkiixuz+YV0ob2459y 1p1GJ04DompdpsciNq0TR+If74Ww631K+YE0AyiYRlNDEGZ1XrtzeSt22cjbm3nsAmiN fTOFb2XR6XfNRgoOhOq9uP+8BkVrWHJqHPImYfJt5RoTMr/a384AE+vff8QM8iqlNSJ+ NuLXg6Neh4mEtYh79e7Al3w6mCtukBZBWoHgXKuycMTe9KZABl5UC1Pnn7ltY5q/6Emd ZqXOh2rZQR8qLTJHAL8GQOH26bzGgVMWwkpNqT9FQIiWyMzdRug/54aTb/m6+rCWEJtu H90g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 144si6012850pfa.118.2017.05.05.12.48.27; Fri, 05 May 2017 12:48:27 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751529AbdEETs0 (ORCPT + 6 others); Fri, 5 May 2017 15:48:26 -0400 Received: from mout.kundenserver.de ([212.227.126.131]:61510 "EHLO mout.kundenserver.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755427AbdEETsP (ORCPT ); Fri, 5 May 2017 15:48:15 -0400 Received: from wuerfel.lan ([78.42.17.5]) by mrelayeu.kundenserver.de (mreue002 [212.227.15.129]) with ESMTPA (Nemesis) id 0MNyfL-1d9k8z2xwp-007Vby; Fri, 05 May 2017 21:48:06 +0200 From: Arnd Bergmann To: Ben Hutchings Cc: stable@vger.kernel.org, Rusty Russell , Arnd Bergmann Subject: [PATCH 3.16-stable 08/87] cpumask_set_cpu_local_first => cpumask_local_spread, lament Date: Fri, 5 May 2017 21:46:26 +0200 Message-Id: <20170505194745.3627137-9-arnd@arndb.de> X-Mailer: git-send-email 2.9.0 In-Reply-To: <20170505194745.3627137-1-arnd@arndb.de> References: <20170505194745.3627137-1-arnd@arndb.de> X-Provags-ID: V03:K0:+BrylGxQtJBiTvLTDVCFA9DW3ZOHtx6I8j+Dy3AfEwo8xP+he/T m4oS2q/Jq+l4vIEKulGWCOsPjzV1XTknopFbI+1QrpgJdqkvJRAEgSNTr+TBAm3I9k8O4vx CrqSLU1cW5R3NijJwGN/XL3bnraQQ9Py12PGaMMdJ+aMZlxuNil5iNDCFqhQGDrG01qcPNs VC4rsRBqoXu8TA9w1BzBw== X-UI-Out-Filterresults: notjunk:1; V01:K0:gihr+IDaejw=:fuJRGAQBGhIsXv68L6qzgA Tf3FJHb3v3ZY6EdbwHXYGaujZWpCls/D2wPCxM7lue0m451QCSY812euN6QLBPkpvr/+QApsa SPwrWrJTKR3Vb7bw+qdYi1ljjtsFdcGFXFmP/OpSW2Xd/hSj0WNQlMgF5sBY5pJG0ZRgjQIbo LVCTHY7jOOh6WswDTWxAhKcDht2Q2CGUXFcU/Vuskb0E+ifHNyxh06mjszQZyTXi7WdC4Ah5k O0uHS7mtxQoCHnnkkS4TzWNzh0PXF+7IMZylgcUzr+kAbGXDL1cLseKG4zaBgJuXll0qNQ9R9 QkyKU6LK42Ucul1hqJkID5PFa4nFn781kj7lGGRaOviVkidUvOhSgHBfiKcdS/THRO4x+tT4w 2NWBR0QF2E9sdj65rweh9qch0EmVDl3WOYgDfP4iL5JgSwnxVJrtE4m/dAdal+U8UghQDt+cv 0PqgeiGD5jEhIvb4vCIf65gaY8+mnptFbayE3279g7Xq7aIjN2Cob9kpnb1pf3ZKGNSOHgyKO zZKwP8OI+bTgsYXXuOnl809y0yh6DcSwaeV1/P01LYOhcetOkL8XQy6IvmgmC+9axKlQF1/qN aCLnrqhPU7FX+cPzM2RE1bTkkOVlUbKMcdQPVIE/5H8vu9DM6NxtxM5t+PbJq2+zfojQpPILo tEtFjAWereVjTCE2jMXwPQQQC9SIPDZZ5TVYPLCt2pdv8ZsWhiocOwGEOMrstwtI4irg= Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Rusty Russell Commit f36963c9d3f6f415732710da3acdd8608a9fa0e upstream. da91309e0a7e (cpumask: Utility function to set n'th cpu...) created a genuinely weird function. I never saw it before, it went through DaveM. (He only does this to make us other maintainers feel better about our own mistakes.) cpumask_set_cpu_local_first's purpose is say "I need to spread things across N online cpus, choose the ones on this numa node first"; you call it in a loop. It can fail. One of the two callers ignores this, the other aborts and fails the device open. It can fail in two ways: allocating the off-stack cpumask, or through a convoluted codepath which AFAICT can only occur if cpu_online_mask changes. Which shouldn't happen, because if cpu_online_mask can change while you call this, it could return a now-offline cpu anyway. It contains a nonsensical test "!cpumask_of_node(numa_node)". This was drawn to my attention by Geert, who said this causes a warning on Sparc. It sets a single bit in a cpumask instead of returning a cpu number, because that's what the callers want. It could be made more efficient by passing the previous cpu rather than an index, but that would be more invasive to the callers. Fixes: da91309e0a7e8966d916a74cce42ed170fde06bf Signed-off-by: Rusty Russell (then rebased) Tested-by: Amir Vadai Acked-by: Amir Vadai Acked-by: David S. Miller Signed-off-by: Arnd Bergmann --- drivers/net/ethernet/mellanox/mlx4/en_netdev.c | 10 ++-- drivers/net/ethernet/mellanox/mlx4/en_tx.c | 6 +-- include/linux/cpumask.h | 6 +-- lib/cpumask.c | 74 +++++++++----------------- 4 files changed, 34 insertions(+), 62 deletions(-) -- 2.9.0 diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c index 6bf4ea400643..f9056192e81c 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c @@ -1535,17 +1535,13 @@ static int mlx4_en_init_affinity_hint(struct mlx4_en_priv *priv, int ring_idx) { struct mlx4_en_rx_ring *ring = priv->rx_ring[ring_idx]; int numa_node = priv->mdev->dev->numa_node; - int ret = 0; if (!zalloc_cpumask_var(&ring->affinity_mask, GFP_KERNEL)) return -ENOMEM; - ret = cpumask_set_cpu_local_first(ring_idx, numa_node, - ring->affinity_mask); - if (ret) - free_cpumask_var(ring->affinity_mask); - - return ret; + cpumask_set_cpu(cpumask_local_spread(ring_idx, numa_node), + ring->affinity_mask); + return 0; } static void mlx4_en_free_affinity_hint(struct mlx4_en_priv *priv, int ring_idx) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_tx.c b/drivers/net/ethernet/mellanox/mlx4/en_tx.c index c5be6d890e94..a3d86e31f34c 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_tx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_tx.c @@ -133,9 +133,9 @@ int mlx4_en_create_tx_ring(struct mlx4_en_priv *priv, ring->queue_index = queue_index; if (queue_index < priv->num_tx_rings_p_up) - cpumask_set_cpu_local_first(queue_index, - priv->mdev->dev->numa_node, - &ring->affinity_mask); + cpumask_set_cpu(cpumask_local_spread(queue_index, + priv->mdev->dev->numa_node), + &ring->affinity_mask); *pring = ring; return 0; diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h index 2997af6d2ccd..f7a172e31aec 100644 --- a/include/linux/cpumask.h +++ b/include/linux/cpumask.h @@ -142,10 +142,8 @@ static inline unsigned int cpumask_any_but(const struct cpumask *mask, return 1; } -static inline int cpumask_set_cpu_local_first(int i, int numa_node, cpumask_t *dstp) +static inline unsigned int cpumask_local_spread(unsigned int i, int node) { - set_bit(0, cpumask_bits(dstp)); - return 0; } @@ -199,7 +197,7 @@ static inline unsigned int cpumask_next_zero(int n, const struct cpumask *srcp) int cpumask_next_and(int n, const struct cpumask *, const struct cpumask *); int cpumask_any_but(const struct cpumask *mask, unsigned int cpu); -int cpumask_set_cpu_local_first(int i, int numa_node, cpumask_t *dstp); +unsigned int cpumask_local_spread(unsigned int i, int node); /** * for_each_cpu - iterate over every cpu in a mask diff --git a/lib/cpumask.c b/lib/cpumask.c index b6513a9f2892..c0bd0df01e3d 100644 --- a/lib/cpumask.c +++ b/lib/cpumask.c @@ -166,64 +166,42 @@ void __init free_bootmem_cpumask_var(cpumask_var_t mask) #endif /** - * cpumask_set_cpu_local_first - set i'th cpu with local numa cpu's first - * + * cpumask_local_spread - select the i'th cpu with local numa cpu's first * @i: index number - * @numa_node: local numa_node - * @dstp: cpumask with the relevant cpu bit set according to the policy + * @node: local numa_node * - * This function sets the cpumask according to a numa aware policy. - * cpumask could be used as an affinity hint for the IRQ related to a - * queue. When the policy is to spread queues across cores - local cores - * first. + * This function selects an online CPU according to a numa aware policy; + * local cpus are returned first, followed by non-local ones, then it + * wraps around. * - * Returns 0 on success, -ENOMEM for no memory, and -EAGAIN when failed to set - * the cpu bit and need to re-call the function. + * It's not very efficient, but useful for setup. */ -int cpumask_set_cpu_local_first(int i, int numa_node, cpumask_t *dstp) +unsigned int cpumask_local_spread(unsigned int i, int node) { - cpumask_var_t mask; int cpu; - int ret = 0; - - if (!zalloc_cpumask_var(&mask, GFP_KERNEL)) - return -ENOMEM; + /* Wrap: we always want a cpu. */ i %= num_online_cpus(); - if (numa_node == -1 || !cpumask_of_node(numa_node)) { - /* Use all online cpu's for non numa aware system */ - cpumask_copy(mask, cpu_online_mask); + if (node == -1) { + for_each_cpu(cpu, cpu_online_mask) + if (i-- == 0) + return cpu; } else { - int n; - - cpumask_and(mask, - cpumask_of_node(numa_node), cpu_online_mask); - - n = cpumask_weight(mask); - if (i >= n) { - i -= n; - - /* If index > number of local cpu's, mask out local - * cpu's - */ - cpumask_andnot(mask, cpu_online_mask, mask); + /* NUMA first. */ + for_each_cpu_and(cpu, cpumask_of_node(node), cpu_online_mask) + if (i-- == 0) + return cpu; + + for_each_cpu(cpu, cpu_online_mask) { + /* Skip NUMA nodes, done above. */ + if (cpumask_test_cpu(cpu, cpumask_of_node(node))) + continue; + + if (i-- == 0) + return cpu; } } - - for_each_cpu(cpu, mask) { - if (--i < 0) - goto out; - } - - ret = -EAGAIN; - -out: - free_cpumask_var(mask); - - if (!ret) - cpumask_set_cpu(cpu, dstp); - - return ret; + BUG(); } -EXPORT_SYMBOL(cpumask_set_cpu_local_first); +EXPORT_SYMBOL(cpumask_local_spread);