From patchwork Tue Apr 15 09:58:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Rafael J. Wysocki" X-Patchwork-Id: 881587 Received: from cloudserver094114.home.pl (cloudserver094114.home.pl [79.96.170.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 039E028E614; Tue, 15 Apr 2025 10:17:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=79.96.170.134 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744712266; cv=none; b=H/iLjq8H4R8PZchNSDFq2d2VScKyr+7Gb9fzUf4luSvRPWrs9ZvbU+qJ2RoWP2VjH1ROhXffU12EsY9WSoBmfudWFf0dyXbUhG1z4hiGHnRLZJrCz0ayD5cIlb5NLUSty7dJzAbgANHmzfx2CTmnneOkAtCpd2RRrLct7FCuwmI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744712266; c=relaxed/simple; bh=2exky6TpNJmCLb/70+qqtFeNC9qkVf1vIRZV7J1dmfk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Y/sXRo+TbzxEM+90Ty5v8e5/56YI9EtmtDTafxwfZd/B+Fz1QUh/TmBSsm8F8jkd8bRHbZvo+boqjJ02jFDR+zYxpDUqYq4hmNiYgCLWgSjbvjXO2thSs6kSQ6PEFrJwo5aBFoTfAagpGI160C0UsANgHmrqz+/kd9fpQ1NquoU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net; spf=pass smtp.mailfrom=rjwysocki.net; dkim=pass (2048-bit key) header.d=rjwysocki.net header.i=@rjwysocki.net header.b=nDRtK2vw; arc=none smtp.client-ip=79.96.170.134 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rjwysocki.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rjwysocki.net header.i=@rjwysocki.net header.b="nDRtK2vw" Received: from kreacher.localnet (unknown [195.136.19.94]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by cloudserver094114.home.pl (Postfix) with ESMTPSA id B60F066266C; Tue, 15 Apr 2025 12:17:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rjwysocki.net; s=dkim; t=1744712258; bh=2exky6TpNJmCLb/70+qqtFeNC9qkVf1vIRZV7J1dmfk=; h=From:Subject:Date; b=nDRtK2vwcj3AT6VQk3u0CALZ2q8GF5zyptE73PgvppblNA9kGlL/rk3TyENqRAgnW HzxLu1G35UImbEo+NfozHS7PT+IVCxHzmB9/Z//4CRyeU6RoSD6YVulBvtXA5OAlih Igp8HbOQr5nU44sxM+dwudkLuw20qQJnNZh+r7egBySy/Uffn1Z0TATwBjvKwom6hj PJHIQnAo22XQnetI3tq7DS26gwrlSCgMKMu9YsnuoEvsLXGHdk+VP/YQ+IYK4mFJVd lto0OsSAbMwrvmFXOIbxw8diw9I4J9vuyothn0dmypea633Zkcc31SDeINu6rGYhbo Q/rv/rRfnQe1Q== From: "Rafael J. Wysocki" To: Linux PM Cc: LKML , Viresh Kumar , Srinivas Pandruvada , Mario Limonciello , Vincent Guittot , Christian Loehle , Sultan Alsawaf , Peter Zijlstra , Valentin Schneider , Ingo Molnar Subject: [PATCH v2 1/6] cpufreq/sched: Fix the usage of CPUFREQ_NEED_UPDATE_LIMITS Date: Tue, 15 Apr 2025 11:58:08 +0200 Message-ID: <3010358.e9J7NaK4W3@rjwysocki.net> In-Reply-To: <6171293.lOV4Wx5bFT@rjwysocki.net> References: <6171293.lOV4Wx5bFT@rjwysocki.net> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CLIENT-IP: 195.136.19.94 X-CLIENT-HOSTNAME: 195.136.19.94 X-VADE-SPAMSTATE: clean X-VADE-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvvdefvddvucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecujffqoffgrffnpdggtffipffknecuuegrihhlohhuthemucduhedtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhvfevufffkfgjfhgggfgtsehtufertddttdejnecuhfhrohhmpedftfgrfhgrvghlucflrdcuhgihshhotghkihdfuceorhhjfiesrhhjfiihshhotghkihdrnhgvtheqnecuggftrfgrthhtvghrnhepfeduudeutdeugfelffduieegiedtueefledvjeegffdttefhhffhtefhleejgfetnecuffhomhgrihhnpehkvghrnhgvlhdrohhrghenucfkphepudelhedrudefiedrudelrdelgeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpeduleehrddufeeirdduledrleegpdhhvghlohepkhhrvggrtghhvghrrdhlohgtrghlnhgvthdpmhgrihhlfhhrohhmpehrjhifsehrjhifhihsohgtkhhirdhnvghtpdhnsggprhgtphhtthhopeduuddprhgtphhtthhopehlihhnuhigqdhpmhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehlihhnuhigqdhkvghrnhgvlhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehvihhrvghshhdrkhhumhgrrheslhhinhgrrhhordhorhhgpdhrtghpthhtohepshhrihhnihhvrghsrdhprghnughruhhvrggurgeslhhinhhugidrihhnthgvlhd X-DCC--Metrics: v370.home.net.pl 1024; Body=11 Fuz1=11 Fuz2=11 From: Rafael J. Wysocki Commit 8e461a1cb43d ("cpufreq: schedutil: Fix superfluous updates caused by need_freq_update") modified sugov_should_update_freq() to set the need_freq_update flag only for drivers with CPUFREQ_NEED_UPDATE_LIMITS set, but that flag generally needs to be set when the policy limits change because the driver callback may need to be invoked for the new limits to take effect. However, if the return value of cpufreq_driver_resolve_freq() after applying the new limits is still equal to the previously selected frequency, the driver callback needs to be invoked only in the case when CPUFREQ_NEED_UPDATE_LIMITS is set (which means that the driver specifically wants its callback to be invoked every time the policy limits change). Update the code accordingly to avoid missing policy limits changes for drivers without CPUFREQ_NEED_UPDATE_LIMITS. Fixes: 8e461a1cb43d ("cpufreq: schedutil: Fix superfluous updates caused by need_freq_update") Closes: https://lore.kernel.org/lkml/Z_Tlc6Qs-tYpxWYb@linaro.org/ Reported-by: Stephan Gerhold Signed-off-by: Rafael J. Wysocki --- v1 -> v2: * Always set need_freq_update when limits_changed is set. * Take CPUFREQ_NEED_UPDATE_LIMITS into account in sugov_update_next_freq(). --- kernel/sched/cpufreq_schedutil.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -83,7 +83,7 @@ if (unlikely(sg_policy->limits_changed)) { sg_policy->limits_changed = false; - sg_policy->need_freq_update = cpufreq_driver_test_flags(CPUFREQ_NEED_UPDATE_LIMITS); + sg_policy->need_freq_update = true; return true; } @@ -95,10 +95,22 @@ static bool sugov_update_next_freq(struct sugov_policy *sg_policy, u64 time, unsigned int next_freq) { - if (sg_policy->need_freq_update) + if (sg_policy->need_freq_update) { sg_policy->need_freq_update = false; - else if (sg_policy->next_freq == next_freq) + /* + * The policy limits have changed, but if the return value of + * cpufreq_driver_resolve_freq() after applying the new limits + * is still equal to the previously selected frequency, the + * driver callback need not be invoked unless the driver + * specifically wants that to happen on every update of the + * policy limits. + */ + if (sg_policy->next_freq == next_freq && + !cpufreq_driver_test_flags(CPUFREQ_NEED_UPDATE_LIMITS)) + return false; + } else if (sg_policy->next_freq == next_freq) { return false; + } sg_policy->next_freq = next_freq; sg_policy->last_freq_update_time = time; From patchwork Tue Apr 15 09:59:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Rafael J. Wysocki" X-Patchwork-Id: 881785 Received: from cloudserver094114.home.pl (cloudserver094114.home.pl [79.96.170.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1219F28DF14; Tue, 15 Apr 2025 10:17:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=79.96.170.134 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744712264; cv=none; b=l8BVNOMwb7mSCmKqbPuAmnOpq1ST85blUKbZ3Kp6UtXqbE8VVYAeRaOKgsFiD6AytBVhw2V4QG7Xr43RAHTTm+eww0zBqAy8w1qFvkNNnlJvki9AjY/lJc5zm/9YwBgRBQIQzfY4TKUpMkRL4YIgfjjqHZvwZ3gne7TfAyassY4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744712264; c=relaxed/simple; bh=OblOeeP2J+XD6Bwadli1Fx0x3GrdHKJhYb8cr0j/ohQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MlwplJaJGdk6Ng17uQBc8o0jyAThVOMXr4hnUBhmPNQSQeI1kGmHojUP9YQFASf0ARagoweswFM+/mdjsiRqN2hAijq2YPA1FB1Cr/IFaqN+4Yfzb4Daalm5t/1qhjlpRAyiWg2YZLQh6KrWcV5mgzG39BwZ/3++0sa7T/ku2wQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net; spf=pass smtp.mailfrom=rjwysocki.net; dkim=pass (2048-bit key) header.d=rjwysocki.net header.i=@rjwysocki.net header.b=qin2wJru; arc=none smtp.client-ip=79.96.170.134 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rjwysocki.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rjwysocki.net header.i=@rjwysocki.net header.b="qin2wJru" Received: from kreacher.localnet (unknown [195.136.19.94]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by cloudserver094114.home.pl (Postfix) with ESMTPSA id CB42F66266B; Tue, 15 Apr 2025 12:17:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rjwysocki.net; s=dkim; t=1744712257; bh=OblOeeP2J+XD6Bwadli1Fx0x3GrdHKJhYb8cr0j/ohQ=; h=From:Subject:Date; b=qin2wJruoS/mHMufsjJpphRcplxPhpwQXF3pdy0RyWY3mu8pH5TR6Ycrse1ojDGV3 DDxWGyUYBmghrBM409WU8V3y0s9QQgO8yD7pcVHE5HCjZMuQFdsEYUypBPPDyOAOl0 gATDyruW8Q1HCyixoDIBHHW2powp4HNW9U3TbqYezXHI0L9AlgWjW7ck/Vc1GI7lId gHvUChHYvJPpALNsGvyI1KeEbY5PTjaOjlWzFE6AhK33vDBgRialtKoxZsbYIisY6h 6VTSWJ7g9V6l6nmcsBkrTZeYY++t8xKhuX+KRz3RnLUoIj/3HJWzFkIxb3qQA0LUql 8uU/iTUHIpG9A== From: "Rafael J. Wysocki" To: Linux PM Cc: LKML , Viresh Kumar , Srinivas Pandruvada , Mario Limonciello , Vincent Guittot , Christian Loehle , Sultan Alsawaf , Peter Zijlstra , Valentin Schneider , Ingo Molnar Subject: [PATCH v2 2/6] cpufreq/sched: Explicitly synchronize limits_changed flag handling Date: Tue, 15 Apr 2025 11:59:15 +0200 Message-ID: <3376719.44csPzL39Z@rjwysocki.net> In-Reply-To: <6171293.lOV4Wx5bFT@rjwysocki.net> References: <6171293.lOV4Wx5bFT@rjwysocki.net> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CLIENT-IP: 195.136.19.94 X-CLIENT-HOSTNAME: 195.136.19.94 X-VADE-SPAMSTATE: clean X-VADE-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvvdefvddvucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecujffqoffgrffnpdggtffipffknecuuegrihhlohhuthemucduhedtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhvfevufffkfgjfhgggfgtsehtufertddttdejnecuhfhrohhmpedftfgrfhgrvghlucflrdcuhgihshhotghkihdfuceorhhjfiesrhhjfiihshhotghkihdrnhgvtheqnecuggftrfgrthhtvghrnhepvdffueeitdfgvddtudegueejtdffteetgeefkeffvdeftddttdeuhfegfedvjefhnecukfhppeduleehrddufeeirdduledrleegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepudelhedrudefiedrudelrdelgedphhgvlhhopehkrhgvrggthhgvrhdrlhhotggrlhhnvghtpdhmrghilhhfrhhomheprhhjfiesrhhjfiihshhotghkihdrnhgvthdpnhgspghrtghpthhtohepuddupdhrtghpthhtoheplhhinhhugidqphhmsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqkhgvrhhnvghlsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepvhhirhgvshhhrdhkuhhmrghrsehlihhnrghrohdrohhrghdprhgtphhtthhopehsrhhinhhivhgrshdrphgrnhgurhhuvhgruggrsehlihhnuhigrdhinhhtvghlrdgtohhmpdhrtghpthhtohepmhgrrhhiohdrlhh X-DCC--Metrics: v370.home.net.pl 1024; Body=11 Fuz1=11 Fuz2=11 From: Rafael J. Wysocki The handling of the limits_changed flag in struct sugov_policy needs to be explicitly synchronized to ensure that cpufreq policy limits updates will not be missed in some cases. Without that synchronization it is theoretically possible that the limits_changed update in sugov_should_update_freq() will be reordered with respect to the reads of the policy limits in cpufreq_driver_resolve_freq() and in that case, if the limits_changed update in sugov_limits() clobbers the one in sugov_should_update_freq(), the new policy limits may not take effect for a long time. Likewise, the limits_changed update in sugov_limits() may theoretically get reordered with respect to the updates of the policy limits in cpufreq_set_policy() and if sugov_should_update_freq() runs between them, the policy limits change may be missed. To ensure that the above situations will not take place, add memory barriers preventing the reordering in question from taking place and add READ_ONCE() and WRITE_ONCE() annotations around all of the limits_changed flag updates to prevent the compiler from messing up with that code. Fixes: 600f5badb78c ("cpufreq: schedutil: Don't skip freq update when limits change") Cc: 5.3+ # 5.3+ Signed-off-by: Rafael J. Wysocki --- v1 -> v2: * Rebase on top of the new [1/6]. --- kernel/sched/cpufreq_schedutil.c | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -81,9 +81,20 @@ if (!cpufreq_this_cpu_can_update(sg_policy->policy)) return false; - if (unlikely(sg_policy->limits_changed)) { - sg_policy->limits_changed = false; + if (unlikely(READ_ONCE(sg_policy->limits_changed))) { + WRITE_ONCE(sg_policy->limits_changed, false); sg_policy->need_freq_update = true; + + /* + * The above limits_changed update must occur before the reads + * of policy limits in cpufreq_driver_resolve_freq() or a policy + * limits update might be missed, so use a memory barrier to + * ensure it. + * + * This pairs with the write memory barrier in sugov_limits(). + */ + smp_mb(); + return true; } @@ -377,7 +388,7 @@ static inline void ignore_dl_rate_limit(struct sugov_cpu *sg_cpu) { if (cpu_bw_dl(cpu_rq(sg_cpu->cpu)) > sg_cpu->bw_min) - sg_cpu->sg_policy->limits_changed = true; + WRITE_ONCE(sg_cpu->sg_policy->limits_changed, true); } static inline bool sugov_update_single_common(struct sugov_cpu *sg_cpu, @@ -883,7 +894,16 @@ mutex_unlock(&sg_policy->work_lock); } - sg_policy->limits_changed = true; + /* + * The limits_changed update below must take place before the updates + * of policy limits in cpufreq_set_policy() or a policy limits update + * might be missed, so use a memory barrier to ensure it. + * + * This pairs with the memory barrier in sugov_should_update_freq(). + */ + smp_wmb(); + + WRITE_ONCE(sg_policy->limits_changed, true); } struct cpufreq_governor schedutil_gov = { From patchwork Tue Apr 15 10:00:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Rafael J. Wysocki" X-Patchwork-Id: 881588 Received: from cloudserver094114.home.pl (cloudserver094114.home.pl [79.96.170.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 64A4F28DEE1; Tue, 15 Apr 2025 10:17:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=79.96.170.134 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744712263; cv=none; b=GNfU3UNOQec2UVWlsyjssmHH38yBN8N/GKJ2kk/U41acF25j0QuNgFkLCKhz7HKBE4P54j/h3iCAJ0zdNqjohe7rXJ6VzuVRYGU5hyfT7ASup/S//hfbsjfv09gotxtJqG52cL2xlOXKPaJcyO9+O+PxnGjCrA25MXrBRlzXP0c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744712263; c=relaxed/simple; bh=qb6MiwdjSvOZkaf22QO5qTTORGUfV38HfJ+qetBWDVw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ki/Us/hAjqJ42RsTwujligZBWHDWBEMvBIJD3QZLAftHJTurJ3Lf+JaeNmS5Fs8BcNB0Ml5PFDQYjnEXJnkPbDJocjHIf+CpJmj/FANn/i/+ci4pOhJh5nfHha94qe9IjzqHXZsewBaJj8S4NU5SmaePikQTK7cfFcnXwAdc8r8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net; spf=pass smtp.mailfrom=rjwysocki.net; dkim=pass (2048-bit key) header.d=rjwysocki.net header.i=@rjwysocki.net header.b=xZcGktxB; arc=none smtp.client-ip=79.96.170.134 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rjwysocki.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rjwysocki.net header.i=@rjwysocki.net header.b="xZcGktxB" Received: from kreacher.localnet (unknown [195.136.19.94]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by cloudserver094114.home.pl (Postfix) with ESMTPSA id 6F92966266A; Tue, 15 Apr 2025 12:17:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rjwysocki.net; s=dkim; t=1744712256; bh=qb6MiwdjSvOZkaf22QO5qTTORGUfV38HfJ+qetBWDVw=; h=From:Subject:Date; b=xZcGktxBm0FjbON8Jcl+Glc6/JhiCuKMfsoC5jbseWkDVPxLKtlvOgdpXIjuPE+SJ BxrP6F2PN5Bd42wZVbtmSzxIWMXsD8rqni6+cW5vzix64yNnFnA+8VmNTPXtjSvzC3 3w97a2EB6ddxSbSsJpvsk4EEGJfbhO4pyCAe7H4f576w4gA2kTuotbtsFPC2Q8FoGM lD+2YW5Ckc7brba6+C1hdZ7oDV/G/gYWwhskUbZsdZqdKJWSY747e7gn/jnWZkqnqs g6hWRH2XRNoeHWTd2GbIbNQ2fF+UkWMVGMTScJoNBRlUuUFXScsoywXxefUU4oTxyY CZG73+RYhQSwA== From: "Rafael J. Wysocki" To: Linux PM Cc: LKML , Viresh Kumar , Srinivas Pandruvada , Mario Limonciello , Vincent Guittot , Christian Loehle , Sultan Alsawaf , Peter Zijlstra , Valentin Schneider , Ingo Molnar Subject: [PATCH v2 3/6] cpufreq/sched: Set need_freq_update in ignore_dl_rate_limit() Date: Tue, 15 Apr 2025 12:00:52 +0200 Message-ID: <10666429.nUPlyArG6x@rjwysocki.net> In-Reply-To: <6171293.lOV4Wx5bFT@rjwysocki.net> References: <6171293.lOV4Wx5bFT@rjwysocki.net> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CLIENT-IP: 195.136.19.94 X-CLIENT-HOSTNAME: 195.136.19.94 X-VADE-SPAMSTATE: clean X-VADE-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvvdefvddvucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecujffqoffgrffnpdggtffipffknecuuegrihhlohhuthemucduhedtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhvfevufffkfgjfhgggfgtsehtufertddttdejnecuhfhrohhmpedftfgrfhgrvghlucflrdcuhgihshhotghkihdfuceorhhjfiesrhhjfiihshhotghkihdrnhgvtheqnecuggftrfgrthhtvghrnhepvdffueeitdfgvddtudegueejtdffteetgeefkeffvdeftddttdeuhfegfedvjefhnecukfhppeduleehrddufeeirdduledrleegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepudelhedrudefiedrudelrdelgedphhgvlhhopehkrhgvrggthhgvrhdrlhhotggrlhhnvghtpdhmrghilhhfrhhomheprhhjfiesrhhjfiihshhotghkihdrnhgvthdpnhgspghrtghpthhtohepuddupdhrtghpthhtoheplhhinhhugidqphhmsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqkhgvrhhnvghlsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepvhhirhgvshhhrdhkuhhmrghrsehlihhnrghrohdrohhrghdprhgtphhtthhopehsrhhinhhivhgrshdrphgrnhgurhhuvhgruggrsehlihhnuhigrdhinhhtvghlrdgtohhmpdhrtghpthhtohepmhgrrhhiohdrlhh X-DCC--Metrics: v370.home.net.pl 1024; Body=11 Fuz1=11 Fuz2=11 From: Rafael J. Wysocki Notice that ignore_dl_rate_limit() need not piggy back on the limits_changed handling to achieve its goal (which is to enforce a frequency update before its due time). Namely, if sugov_should_update_freq() is updated to check sg_policy->need_freq_update and return 'true' if it is set when sg_policy->limits_changed is not set, ignore_dl_rate_limit() may set the former directly instead of setting the latter, so it can avoid hitting the memory barrier in sugov_should_update_freq(). Update the code accordingly. Signed-off-by: Rafael J. Wysocki Reviewed-by: Christian Loehle --- kernel/sched/cpufreq_schedutil.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -96,6 +96,9 @@ smp_mb(); return true; + } else if (sg_policy->need_freq_update) { + /* ignore_dl_rate_limit() wants a new frequency to be found. */ + return true; } delta_ns = time - sg_policy->last_freq_update_time; @@ -388,7 +391,7 @@ static inline void ignore_dl_rate_limit(struct sugov_cpu *sg_cpu) { if (cpu_bw_dl(cpu_rq(sg_cpu->cpu)) > sg_cpu->bw_min) - WRITE_ONCE(sg_cpu->sg_policy->limits_changed, true); + sg_cpu->sg_policy->need_freq_update = true; } static inline bool sugov_update_single_common(struct sugov_cpu *sg_cpu, From patchwork Tue Apr 15 10:02:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Rafael J. Wysocki" X-Patchwork-Id: 881786 Received: from cloudserver094114.home.pl (cloudserver094114.home.pl [79.96.170.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD80128B4F0; Tue, 15 Apr 2025 10:17:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=79.96.170.134 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744712261; cv=none; b=dn+S0hYFt64JsO7LRAb7UKe3VDxRctjpLPWRhdy/hy0UgktjMiRBvXxmRrc4p0xNqv1xrT79pBsbypDL4Sl0MXDVJVWlkw4/YwblOh82wOhsrcL5xKb1+FtVBXHYQxio7YKaa3xyuwDtj7u6KoY3ueAI+Fo5fysyJ4a1S8PFUOQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744712261; c=relaxed/simple; bh=68OXra1Dp9DBMRyBvdCpFfLAw1vqwtRy9di/sd4TVG0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=KVUucCmrvulqWwfccvBTpJfpMQs/ktN4DummCUzXJjzW58pBYCAYYQGGoCh/sZaZgM0wdgEHHWJZhsCqFsBuNfS6KYSFUQmN0fPWH9uwD54ySXRXvs0+DMlSonnuFXG+82+fGYRTqnn8xbCSxBPksnrktbc+o+iRS1o61Tr6eAY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net; spf=pass smtp.mailfrom=rjwysocki.net; dkim=pass (2048-bit key) header.d=rjwysocki.net header.i=@rjwysocki.net header.b=K+KrZHow; arc=none smtp.client-ip=79.96.170.134 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rjwysocki.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rjwysocki.net header.i=@rjwysocki.net header.b="K+KrZHow" Received: from kreacher.localnet (unknown [195.136.19.94]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by cloudserver094114.home.pl (Postfix) with ESMTPSA id 95FC0662669; Tue, 15 Apr 2025 12:17:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rjwysocki.net; s=dkim; t=1744712255; bh=68OXra1Dp9DBMRyBvdCpFfLAw1vqwtRy9di/sd4TVG0=; h=From:Subject:Date; b=K+KrZHowyeP1rTpFLhZOnQ3hoXooHVexHpcYnC4bhcFhx/mui/HRTU1x49Nym1hUC /lsscjxmUUmRdZ6dVyrCRx3kI7BPizTL2UCZ05h4R62YFwHFTit49lirbitxcZLVgj 9x9h1ak7WU02OhPklyObSTorykrQSDEsF7fnDUiD1AUdTm8DS+1FuS1UU4HHI7ijGe WQQrCFcfJ0Lw/Ppm0UunSzKC6B5gmBdWbY7TGIS7yIWtr+S5tTqMgu0aIei7bKtfq7 snNRiXkh24JcLUjB8K37fSmoc4UFBPch+8PZaOc0NAzlPJaZPjPiPk5JhLKb4Zfttj 8q1+3XWa95lIQ== From: "Rafael J. Wysocki" To: Linux PM Cc: LKML , Viresh Kumar , Srinivas Pandruvada , Mario Limonciello , Vincent Guittot , Christian Loehle , Sultan Alsawaf , Peter Zijlstra , Valentin Schneider , Ingo Molnar Subject: [PATCH v2 4/6] cpufreq: Rename __resolve_freq() to clamp_and_resolve_freq() Date: Tue, 15 Apr 2025 12:02:04 +0200 Message-ID: <2024183.PYKUYFuaPT@rjwysocki.net> In-Reply-To: <6171293.lOV4Wx5bFT@rjwysocki.net> References: <6171293.lOV4Wx5bFT@rjwysocki.net> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CLIENT-IP: 195.136.19.94 X-CLIENT-HOSTNAME: 195.136.19.94 X-VADE-SPAMSTATE: clean X-VADE-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvvdefvddvucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecujffqoffgrffnpdggtffipffknecuuegrihhlohhuthemucduhedtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhvfevufffkfgjfhgggfgtsehtufertddttdejnecuhfhrohhmpedftfgrfhgrvghlucflrdcuhgihshhotghkihdfuceorhhjfiesrhhjfiihshhotghkihdrnhgvtheqnecuggftrfgrthhtvghrnhepvdffueeitdfgvddtudegueejtdffteetgeefkeffvdeftddttdeuhfegfedvjefhnecukfhppeduleehrddufeeirdduledrleegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepudelhedrudefiedrudelrdelgedphhgvlhhopehkrhgvrggthhgvrhdrlhhotggrlhhnvghtpdhmrghilhhfrhhomheprhhjfiesrhhjfiihshhotghkihdrnhgvthdpnhgspghrtghpthhtohepuddupdhrtghpthhtoheplhhinhhugidqphhmsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqkhgvrhhnvghlsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepvhhirhgvshhhrdhkuhhmrghrsehlihhnrghrohdrohhrghdprhgtphhtthhopehsrhhinhhivhgrshdrphgrnhgurhhuvhgruggrsehlihhnuhigrdhinhhtvghlrdgtohhmpdhrtghpthhtohepmhgrrhhiohdrlhh X-DCC--Metrics: v370.home.net.pl 1024; Body=11 Fuz1=11 Fuz2=11 From: Rafael J. Wysocki In preparation for subsequent changes rename a function in the cpufreq core as per the subject and while at it, clean up some white space around the declaration for that function. No functional impact. Signed-off-by: Rafael J. Wysocki --- v1 -> v2: No changes --- drivers/cpufreq/cpufreq.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) --- a/drivers/cpufreq/cpufreq.c +++ b/drivers/cpufreq/cpufreq.c @@ -490,8 +490,9 @@ } EXPORT_SYMBOL_GPL(cpufreq_disable_fast_switch); -static unsigned int __resolve_freq(struct cpufreq_policy *policy, - unsigned int target_freq, unsigned int relation) +static unsigned int clamp_and_resolve_freq(struct cpufreq_policy *policy, + unsigned int target_freq, + unsigned int relation) { unsigned int idx; @@ -520,7 +521,7 @@ unsigned int cpufreq_driver_resolve_freq(struct cpufreq_policy *policy, unsigned int target_freq) { - return __resolve_freq(policy, target_freq, CPUFREQ_RELATION_LE); + return clamp_and_resolve_freq(policy, target_freq, CPUFREQ_RELATION_LE); } EXPORT_SYMBOL_GPL(cpufreq_driver_resolve_freq); @@ -2338,7 +2339,7 @@ if (cpufreq_disabled()) return -ENODEV; - target_freq = __resolve_freq(policy, target_freq, relation); + target_freq = clamp_and_resolve_freq(policy, target_freq, relation); pr_debug("target for CPU %u: %u kHz, relation %u, requested %u kHz\n", policy->cpu, target_freq, relation, old_target_freq); @@ -2634,8 +2635,8 @@ */ policy->min = new_data.min; policy->max = new_data.max; - policy->min = __resolve_freq(policy, policy->min, CPUFREQ_RELATION_L); - policy->max = __resolve_freq(policy, policy->max, CPUFREQ_RELATION_H); + policy->min = clamp_and_resolve_freq(policy, policy->min, CPUFREQ_RELATION_L); + policy->max = clamp_and_resolve_freq(policy, policy->max, CPUFREQ_RELATION_H); trace_cpu_frequency_limits(policy); cpufreq_update_pressure(policy); From patchwork Tue Apr 15 10:04:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Rafael J. Wysocki" X-Patchwork-Id: 881589 Received: from cloudserver094114.home.pl (cloudserver094114.home.pl [79.96.170.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A1D7A1CAA6D; Tue, 15 Apr 2025 10:17:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=79.96.170.134 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744712259; cv=none; b=pchg2b5jiuaKqIKxBNfxuyWPBC2fHECcWIjdhxK/w06XJc1V50+sukESBbr9jBTPUSesdKqWtAQ73yCzWg4r6FpBwoXAV1GLQ9IqJX+0rn5eUZbj9sxdrbkMEFoB3geNKbviixAVHEAoiKsFqNCqpwSOqF9cb3cRLOpsX7RtKss= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744712259; c=relaxed/simple; bh=yfxZC1j8MoskcJ1HEXZgO0OGbDvcF6mnmjMlnsHGWp4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ijXDHOVu9inin73hpz7HnxlnQyQEf8NodbG+CTddo7FWhdDNKi7ew9fX2CAucdj91JyK6LiylelEw4tLUp9DNxIk+1Rrsi5XeJchQl8G7C9P/XhwnxHAgNFoDwCd1xrfFt05nka1aXv8gmyIJqNcPsHnkqqTu0J4mVS/73HADOc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net; spf=pass smtp.mailfrom=rjwysocki.net; dkim=pass (2048-bit key) header.d=rjwysocki.net header.i=@rjwysocki.net header.b=VpC8rBD1; arc=none smtp.client-ip=79.96.170.134 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rjwysocki.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rjwysocki.net header.i=@rjwysocki.net header.b="VpC8rBD1" Received: from kreacher.localnet (unknown [195.136.19.94]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by cloudserver094114.home.pl (Postfix) with ESMTPSA id B67D9662668; Tue, 15 Apr 2025 12:17:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rjwysocki.net; s=dkim; t=1744712254; bh=yfxZC1j8MoskcJ1HEXZgO0OGbDvcF6mnmjMlnsHGWp4=; h=From:Subject:Date; b=VpC8rBD12t96jhMkhy2V+uTSZ3rVq/tMV1xZgQNWQL90Y8KwqmgCGkLUUFyR6UEX5 axUs2zOvvi3qzx83+OzXK3+QjqG3DFtMBS+uxsJl5yfvUaFwaHxTKt942eWOknexDQ WLus4fJSTe9zZTXSenrRAvuDuov/yIMeojCCaMogWfXk/z6qBIcJvVL7QcKTB8y6J5 7mzPPHf1V5szZHp9c7GgXWEAt0nO3a6JWINpZsPvDg7ZWYYrDZeX4LlR9NAgKsZ/vN 9HTvynGmYH1vYIxB0ruMeKC4KlR/7nzQ+9/Bq6qCif3H7XAwK0IqeDyE8yZ1Kyi6ma y9raQwC4SufEA== From: "Rafael J. Wysocki" To: Linux PM Cc: LKML , Viresh Kumar , Srinivas Pandruvada , Mario Limonciello , Vincent Guittot , Christian Loehle , Sultan Alsawaf , Peter Zijlstra , Valentin Schneider , Ingo Molnar Subject: [PATCH v2 5/6] cpufreq: Avoid using inconsistent policy->min and policy->max Date: Tue, 15 Apr 2025 12:04:21 +0200 Message-ID: <9458818.CDJkKcVGEf@rjwysocki.net> In-Reply-To: <6171293.lOV4Wx5bFT@rjwysocki.net> References: <6171293.lOV4Wx5bFT@rjwysocki.net> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CLIENT-IP: 195.136.19.94 X-CLIENT-HOSTNAME: 195.136.19.94 X-VADE-SPAMSTATE: clean X-VADE-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvvdefvdefucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecujffqoffgrffnpdggtffipffknecuuegrihhlohhuthemucduhedtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhvfevufffkfgjfhgggfgtsehtufertddttdejnecuhfhrohhmpedftfgrfhgrvghlucflrdcuhgihshhotghkihdfuceorhhjfiesrhhjfiihshhotghkihdrnhgvtheqnecuggftrfgrthhtvghrnhepvdffueeitdfgvddtudegueejtdffteetgeefkeffvdeftddttdeuhfegfedvjefhnecukfhppeduleehrddufeeirdduledrleegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepudelhedrudefiedrudelrdelgedphhgvlhhopehkrhgvrggthhgvrhdrlhhotggrlhhnvghtpdhmrghilhhfrhhomheprhhjfiesrhhjfiihshhotghkihdrnhgvthdpnhgspghrtghpthhtohepuddupdhrtghpthhtoheplhhinhhugidqphhmsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqkhgvrhhnvghlsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepvhhirhgvshhhrdhkuhhmrghrsehlihhnrghrohdrohhrghdprhgtphhtthhopehsrhhinhhivhgrshdrphgrnhgurhhuvhgruggrsehlihhnuhigrdhinhhtvghlrdgtohhmpdhrtghpthhtohepmhgrrhhiohdrlhh X-DCC--Metrics: v370.home.net.pl 1024; Body=11 Fuz1=11 Fuz2=11 From: Rafael J. Wysocki Since cpufreq_driver_resolve_freq() can run in parallel with cpufreq_set_policy() and there is no synchronization between them, the former may access policy->min and policy->max while the latter is updating them and it may see intermediate values of them due to the way the update is carried out. Also the compiler is free to apply any optimizations it wants both to the stores in cpufreq_set_policy() and to the loads in cpufreq_driver_resolve_freq() which may result in additional inconsistencies. To address this, use WRITE_ONCE() when updating policy->min and policy->max in cpufreq_set_policy() and use READ_ONCE() for reading them in cpufreq_driver_resolve_freq(). Moreover, rearrange the update in cpufreq_set_policy() to avoid storing intermediate values in policy->min and policy->max with the help of the observation that their new values are expected to be properly ordered upfront. Also modify cpufreq_driver_resolve_freq() to take the possible reverse ordering of policy->min and policy->max, which may happen depending on the ordering of operations when this function and cpufreq_set_policy() run concurrently, into account by always honoring the max when it turns out to be less than the min (in case it comes from thermal throttling or similar). Fixes: 151717690694 ("cpufreq: Make policy min/max hard requirements") Cc: 5.16+ # 5.16+ Signed-off-by: Rafael J. Wysocki --- v1 -> v2: Minor edit in the subject --- drivers/cpufreq/cpufreq.c | 46 ++++++++++++++++++++++++++++++++++++---------- 1 file changed, 36 insertions(+), 10 deletions(-) --- a/drivers/cpufreq/cpufreq.c +++ b/drivers/cpufreq/cpufreq.c @@ -490,14 +490,12 @@ } EXPORT_SYMBOL_GPL(cpufreq_disable_fast_switch); -static unsigned int clamp_and_resolve_freq(struct cpufreq_policy *policy, - unsigned int target_freq, - unsigned int relation) +static unsigned int __resolve_freq(struct cpufreq_policy *policy, + unsigned int target_freq, + unsigned int relation) { unsigned int idx; - target_freq = clamp_val(target_freq, policy->min, policy->max); - if (!policy->freq_table) return target_freq; @@ -507,6 +505,15 @@ return policy->freq_table[idx].frequency; } +static unsigned int clamp_and_resolve_freq(struct cpufreq_policy *policy, + unsigned int target_freq, + unsigned int relation) +{ + target_freq = clamp_val(target_freq, policy->min, policy->max); + + return __resolve_freq(policy, target_freq, relation); +} + /** * cpufreq_driver_resolve_freq - Map a target frequency to a driver-supported * one. @@ -521,7 +528,22 @@ unsigned int cpufreq_driver_resolve_freq(struct cpufreq_policy *policy, unsigned int target_freq) { - return clamp_and_resolve_freq(policy, target_freq, CPUFREQ_RELATION_LE); + unsigned int min = READ_ONCE(policy->min); + unsigned int max = READ_ONCE(policy->max); + + /* + * If this function runs in parallel with cpufreq_set_policy(), it may + * read policy->min before the update and policy->max after the update + * or the other way around, so there is no ordering guarantee. + * + * Resolve this by always honoring the max (in case it comes from + * thermal throttling or similar). + */ + if (unlikely(min > max)) + min = max; + + return __resolve_freq(policy, clamp_val(target_freq, min, max), + CPUFREQ_RELATION_LE); } EXPORT_SYMBOL_GPL(cpufreq_driver_resolve_freq); @@ -2632,11 +2654,15 @@ * Resolve policy min/max to available frequencies. It ensures * no frequency resolution will neither overshoot the requested maximum * nor undershoot the requested minimum. + * + * Avoid storing intermediate values in policy->max or policy->min and + * compiler optimizations around them because them may be accessed + * concurrently by cpufreq_driver_resolve_freq() during the update. */ - policy->min = new_data.min; - policy->max = new_data.max; - policy->min = clamp_and_resolve_freq(policy, policy->min, CPUFREQ_RELATION_L); - policy->max = clamp_and_resolve_freq(policy, policy->max, CPUFREQ_RELATION_H); + WRITE_ONCE(policy->max, __resolve_freq(policy, new_data.max, CPUFREQ_RELATION_H)); + new_data.min = __resolve_freq(policy, new_data.min, CPUFREQ_RELATION_L); + WRITE_ONCE(policy->min, new_data.min > policy->max ? policy->max : new_data.min); + trace_cpu_frequency_limits(policy); cpufreq_update_pressure(policy); From patchwork Tue Apr 15 10:05:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Rafael J. Wysocki" X-Patchwork-Id: 881787 Received: from cloudserver094114.home.pl (cloudserver094114.home.pl [79.96.170.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E6E1B5A79B; Tue, 15 Apr 2025 10:17:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=79.96.170.134 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744712258; cv=none; b=DkfL4WlWzt3eClC1pIyF2t4rniimzSR1vmWnxdg5W30dcgLzCCM8wN3RQ9N53Vb/33NbjgoGuGfarDUcQN+57UvsnG4SOlJnpfRs5r1rcQfEbVa4el5ssgB3Nc3HgsFSp/gQhESlsCLoCkPifXyr6d0YfannHUHuOn6T5+WUhFM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744712258; c=relaxed/simple; bh=3evzT+2AlWEasWA69QIuvzmFxJexD9EPpHYlvvq7348=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=syWV4Q1yjJARsxnaojj8iGQJRZu16BEYtvITYqTx7VX+cwsVMxnffdwwwvY31rhNvdZeXSQ5UVAM4HJLkTrxpyXvRmQ30Vjj33nXJFASOhCkrPYpKJRZrHG5gSPFzFjZT8mTJ3MIga2xKoRxH4kvKDyF9LGr+gWQLt2eGISz3Y4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net; spf=pass smtp.mailfrom=rjwysocki.net; dkim=pass (2048-bit key) header.d=rjwysocki.net header.i=@rjwysocki.net header.b=hkgA6vXv; arc=none smtp.client-ip=79.96.170.134 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rjwysocki.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rjwysocki.net header.i=@rjwysocki.net header.b="hkgA6vXv" Received: from kreacher.localnet (unknown [195.136.19.94]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by cloudserver094114.home.pl (Postfix) with ESMTPSA id CEB07662667; Tue, 15 Apr 2025 12:17:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rjwysocki.net; s=dkim; t=1744712253; bh=3evzT+2AlWEasWA69QIuvzmFxJexD9EPpHYlvvq7348=; h=From:Subject:Date; b=hkgA6vXvx0L+GnrpLy9yz0KXM3zrTG4Tu9dJ6nh9BJPX/t/Cur5EkPg0HNk34Z5ua HY4Y6HWtnEPHFAUcQkxcbTpdFMjNufQZIFXHdSmP4CJ4ckf21xog5LIrLYdNpW6LL/ 5BztGouqq4H8hpvL6+TIH8Ei6AC3+7urxUAvF2rFH7ZX44/KZoZZZUa7pGq+xP/HOX rM04R/tsP+Lb17uX9+ke00/xdZO4C6Bsqo6TWLTZzcr5P7T0jiFyBW2+2Ww4Hp6h+U NZVimO3nIx2+OQGMTEpqHgQmoDqnmD6StSLKOS6FVKT/B8IMzrXt5KSgJrDqcuZs6s oN3wetJBzeBLA== From: "Rafael J. Wysocki" To: Linux PM Cc: LKML , Viresh Kumar , Srinivas Pandruvada , Mario Limonciello , Vincent Guittot , Christian Loehle , Sultan Alsawaf , Peter Zijlstra , Valentin Schneider , Ingo Molnar Subject: [PATCH v2 6/6] cpufreq: Eliminate clamp_and_resolve_freq() Date: Tue, 15 Apr 2025 12:05:09 +0200 Message-ID: <3301325.5fSG56mABF@rjwysocki.net> In-Reply-To: <6171293.lOV4Wx5bFT@rjwysocki.net> References: <6171293.lOV4Wx5bFT@rjwysocki.net> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CLIENT-IP: 195.136.19.94 X-CLIENT-HOSTNAME: 195.136.19.94 X-VADE-SPAMSTATE: clean X-VADE-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvvdefvddvucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecujffqoffgrffnpdggtffipffknecuuegrihhlohhuthemucduhedtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhvfevufffkfgjfhgggfgtsehtufertddttdejnecuhfhrohhmpedftfgrfhgrvghlucflrdcuhgihshhotghkihdfuceorhhjfiesrhhjfiihshhotghkihdrnhgvtheqnecuggftrfgrthhtvghrnhepvdffueeitdfgvddtudegueejtdffteetgeefkeffvdeftddttdeuhfegfedvjefhnecukfhppeduleehrddufeeirdduledrleegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepudelhedrudefiedrudelrdelgedphhgvlhhopehkrhgvrggthhgvrhdrlhhotggrlhhnvghtpdhmrghilhhfrhhomheprhhjfiesrhhjfiihshhotghkihdrnhgvthdpnhgspghrtghpthhtohepuddupdhrtghpthhtoheplhhinhhugidqphhmsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqkhgvrhhnvghlsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepvhhirhgvshhhrdhkuhhmrghrsehlihhnrghrohdrohhrghdprhgtphhtthhopehsrhhinhhivhgrshdrphgrnhgurhhuvhgruggrsehlihhnuhigrdhinhhtvghlrdgtohhmpdhrtghpthhtohepmhgrrhhiohdrlhh X-DCC--Metrics: v370.home.net.pl 1024; Body=11 Fuz1=11 Fuz2=11 From: Rafael J. Wysocki Fold clamp_and_resolve_freq() into __cpufreq_driver_target() which is its only remaining caller. No intentional functional impact. Signed-off-by: Rafael J. Wysocki --- v1 -> v2: No changes --- drivers/cpufreq/cpufreq.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) --- a/drivers/cpufreq/cpufreq.c +++ b/drivers/cpufreq/cpufreq.c @@ -505,15 +505,6 @@ return policy->freq_table[idx].frequency; } -static unsigned int clamp_and_resolve_freq(struct cpufreq_policy *policy, - unsigned int target_freq, - unsigned int relation) -{ - target_freq = clamp_val(target_freq, policy->min, policy->max); - - return __resolve_freq(policy, target_freq, relation); -} - /** * cpufreq_driver_resolve_freq - Map a target frequency to a driver-supported * one. @@ -2361,7 +2352,8 @@ if (cpufreq_disabled()) return -ENODEV; - target_freq = clamp_and_resolve_freq(policy, target_freq, relation); + target_freq = clamp_val(target_freq, policy->min, policy->max); + target_freq = __resolve_freq(policy, target_freq, relation); pr_debug("target for CPU %u: %u kHz, relation %u, requested %u kHz\n", policy->cpu, target_freq, relation, old_target_freq);