[V3,2/3] cpufreq: intel_pstate: Migrate away from ->stop_cpu() callback

Message ID c31424b7962608eb13f946a665ba6848c4986856.1623986349.git.viresh.kumar@linaro.org
State Superseded
Headers show
Series
  • Untitled series #137411
Related show

Commit Message

Viresh Kumar June 18, 2021, 3:22 a.m.
commit 367dc4aa932b ("cpufreq: Add stop CPU callback to cpufreq_driver
interface") added the stop_cpu() callback to allow the drivers to do
clean up before the CPU is completely down and its state can't be
modified.

At that time the CPU hotplug framework used to call the cpufreq core's
registered notifier for different events like CPU_DOWN_PREPARE and
CPU_POST_DEAD. The stop_cpu() callback was called during the
CPU_DOWN_PREPARE event.

This is no longer the case, cpuhp_cpufreq_offline() is called only once
by the CPU hotplug core now and we don't really need to separately
call stop_cpu() for cpufreq drivers.

Migrate to using the exit() and offline() callbacks instead of
stop_cpu().

We need to clear util hook from both the callbacks, exit() and
offline(), since it is possible that only exit() gets called sometimes
(specially on errors) or both get called at other times.
intel_pstate_clear_update_util_hook() anyway have enough protection in
place if it gets called a second time and will return early then.

Cc: Dirk Brandewie <dirk.brandewie@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

---
V2->V3:
- Update intel_pstate_cpu_offline() as well.
- Improved commit log.

 drivers/cpufreq/intel_pstate.c | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

-- 
2.31.1.272.g89b43f80a514

Comments

Rafael J. Wysocki June 18, 2021, noon | #1
On Fri, Jun 18, 2021 at 5:22 AM Viresh Kumar <viresh.kumar@linaro.org> wrote:
>

> commit 367dc4aa932b ("cpufreq: Add stop CPU callback to cpufreq_driver

> interface") added the stop_cpu() callback to allow the drivers to do

> clean up before the CPU is completely down and its state can't be

> modified.

>

> At that time the CPU hotplug framework used to call the cpufreq core's

> registered notifier for different events like CPU_DOWN_PREPARE and

> CPU_POST_DEAD. The stop_cpu() callback was called during the

> CPU_DOWN_PREPARE event.

>

> This is no longer the case, cpuhp_cpufreq_offline() is called only once

> by the CPU hotplug core now and we don't really need to separately

> call stop_cpu() for cpufreq drivers.

>

> Migrate to using the exit() and offline() callbacks instead of

> stop_cpu().

>

> We need to clear util hook from both the callbacks, exit() and

> offline(), since it is possible that only exit() gets called sometimes

> (specially on errors) or both get called at other times.

> intel_pstate_clear_update_util_hook() anyway have enough protection in

> place if it gets called a second time and will return early then.

>

> Cc: Dirk Brandewie <dirk.brandewie@gmail.com>

> Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

> ---

> V2->V3:

> - Update intel_pstate_cpu_offline() as well.

> - Improved commit log.

>

>  drivers/cpufreq/intel_pstate.c | 12 ++++--------

>  1 file changed, 4 insertions(+), 8 deletions(-)

>

> diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c

> index 0e69dffd5a76..8f8a2d9d7daa 100644

> --- a/drivers/cpufreq/intel_pstate.c

> +++ b/drivers/cpufreq/intel_pstate.c

> @@ -2335,6 +2335,8 @@ static int intel_pstate_cpu_offline(struct cpufreq_policy *policy)

>

>         pr_debug("CPU %d going offline\n", cpu->cpu);

>

> +       intel_pstate_clear_update_util_hook(policy->cpu);

> +

>         if (cpu->suspended)

>                 return 0;

>

> @@ -2374,17 +2376,12 @@ static int intel_pstate_cpu_online(struct cpufreq_policy *policy)

>         return 0;

>  }

>

> -static void intel_pstate_stop_cpu(struct cpufreq_policy *policy)

> -{

> -       pr_debug("CPU %d stopping\n", policy->cpu);

> -

> -       intel_pstate_clear_update_util_hook(policy->cpu);

> -}

> -

>  static int intel_pstate_cpu_exit(struct cpufreq_policy *policy)

>  {

>         pr_debug("CPU %d exiting\n", policy->cpu);

>

> +       intel_pstate_clear_update_util_hook(policy->cpu);


This change is not needed now, because ->offline always runs before
->exit if present.

> +

>         policy->fast_switch_possible = false;

>

>         return 0;

> @@ -2451,7 +2448,6 @@ static struct cpufreq_driver intel_pstate = {

>         .resume         = intel_pstate_resume,

>         .init           = intel_pstate_cpu_init,

>         .exit           = intel_pstate_cpu_exit,

> -       .stop_cpu       = intel_pstate_stop_cpu,

>         .offline        = intel_pstate_cpu_offline,

>         .online         = intel_pstate_cpu_online,

>         .update_limits  = intel_pstate_update_limits,

> --

> 2.31.1.272.g89b43f80a514

>
Viresh Kumar June 21, 2021, 3:09 a.m. | #2
On 18-06-21, 14:00, Rafael J. Wysocki wrote:
> On Fri, Jun 18, 2021 at 5:22 AM Viresh Kumar <viresh.kumar@linaro.org> wrote:

> >

> > commit 367dc4aa932b ("cpufreq: Add stop CPU callback to cpufreq_driver

> > interface") added the stop_cpu() callback to allow the drivers to do

> > clean up before the CPU is completely down and its state can't be

> > modified.

> >

> > At that time the CPU hotplug framework used to call the cpufreq core's

> > registered notifier for different events like CPU_DOWN_PREPARE and

> > CPU_POST_DEAD. The stop_cpu() callback was called during the

> > CPU_DOWN_PREPARE event.

> >

> > This is no longer the case, cpuhp_cpufreq_offline() is called only once

> > by the CPU hotplug core now and we don't really need to separately

> > call stop_cpu() for cpufreq drivers.

> >

> > Migrate to using the exit() and offline() callbacks instead of

> > stop_cpu().

> >

> > We need to clear util hook from both the callbacks, exit() and

> > offline(), since it is possible that only exit() gets called sometimes

> > (specially on errors) or both get called at other times.

> > intel_pstate_clear_update_util_hook() anyway have enough protection in

> > place if it gets called a second time and will return early then.

> >

> > Cc: Dirk Brandewie <dirk.brandewie@gmail.com>

> > Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

> > ---

> > V2->V3:

> > - Update intel_pstate_cpu_offline() as well.

> > - Improved commit log.

> >

> >  drivers/cpufreq/intel_pstate.c | 12 ++++--------

> >  1 file changed, 4 insertions(+), 8 deletions(-)

> >

> > diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c

> > index 0e69dffd5a76..8f8a2d9d7daa 100644

> > --- a/drivers/cpufreq/intel_pstate.c

> > +++ b/drivers/cpufreq/intel_pstate.c

> > @@ -2335,6 +2335,8 @@ static int intel_pstate_cpu_offline(struct cpufreq_policy *policy)

> >

> >         pr_debug("CPU %d going offline\n", cpu->cpu);

> >

> > +       intel_pstate_clear_update_util_hook(policy->cpu);

> > +

> >         if (cpu->suspended)

> >                 return 0;

> >

> > @@ -2374,17 +2376,12 @@ static int intel_pstate_cpu_online(struct cpufreq_policy *policy)

> >         return 0;

> >  }

> >

> > -static void intel_pstate_stop_cpu(struct cpufreq_policy *policy)

> > -{

> > -       pr_debug("CPU %d stopping\n", policy->cpu);

> > -

> > -       intel_pstate_clear_update_util_hook(policy->cpu);

> > -}

> > -

> >  static int intel_pstate_cpu_exit(struct cpufreq_policy *policy)

> >  {

> >         pr_debug("CPU %d exiting\n", policy->cpu);

> >

> > +       intel_pstate_clear_update_util_hook(policy->cpu);

> 

> This change is not needed now, because ->offline always runs before

> ->exit if present.


Not necessarily, we don't call ->offline() for many error paths in
cpufreq_online(). offline() only comes into play after driver is registered
properly once.

-- 
viresh
Rafael J. Wysocki June 21, 2021, 2:17 p.m. | #3
On Mon, Jun 21, 2021 at 5:09 AM Viresh Kumar <viresh.kumar@linaro.org> wrote:
>

> On 18-06-21, 14:00, Rafael J. Wysocki wrote:

> > On Fri, Jun 18, 2021 at 5:22 AM Viresh Kumar <viresh.kumar@linaro.org> wrote:

> > >

> > > commit 367dc4aa932b ("cpufreq: Add stop CPU callback to cpufreq_driver

> > > interface") added the stop_cpu() callback to allow the drivers to do

> > > clean up before the CPU is completely down and its state can't be

> > > modified.

> > >

> > > At that time the CPU hotplug framework used to call the cpufreq core's

> > > registered notifier for different events like CPU_DOWN_PREPARE and

> > > CPU_POST_DEAD. The stop_cpu() callback was called during the

> > > CPU_DOWN_PREPARE event.

> > >

> > > This is no longer the case, cpuhp_cpufreq_offline() is called only once

> > > by the CPU hotplug core now and we don't really need to separately

> > > call stop_cpu() for cpufreq drivers.

> > >

> > > Migrate to using the exit() and offline() callbacks instead of

> > > stop_cpu().

> > >

> > > We need to clear util hook from both the callbacks, exit() and

> > > offline(), since it is possible that only exit() gets called sometimes

> > > (specially on errors) or both get called at other times.

> > > intel_pstate_clear_update_util_hook() anyway have enough protection in

> > > place if it gets called a second time and will return early then.

> > >

> > > Cc: Dirk Brandewie <dirk.brandewie@gmail.com>

> > > Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

> > > ---

> > > V2->V3:

> > > - Update intel_pstate_cpu_offline() as well.

> > > - Improved commit log.

> > >

> > >  drivers/cpufreq/intel_pstate.c | 12 ++++--------

> > >  1 file changed, 4 insertions(+), 8 deletions(-)

> > >

> > > diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c

> > > index 0e69dffd5a76..8f8a2d9d7daa 100644

> > > --- a/drivers/cpufreq/intel_pstate.c

> > > +++ b/drivers/cpufreq/intel_pstate.c

> > > @@ -2335,6 +2335,8 @@ static int intel_pstate_cpu_offline(struct cpufreq_policy *policy)

> > >

> > >         pr_debug("CPU %d going offline\n", cpu->cpu);

> > >

> > > +       intel_pstate_clear_update_util_hook(policy->cpu);

> > > +

> > >         if (cpu->suspended)

> > >                 return 0;

> > >

> > > @@ -2374,17 +2376,12 @@ static int intel_pstate_cpu_online(struct cpufreq_policy *policy)

> > >         return 0;

> > >  }

> > >

> > > -static void intel_pstate_stop_cpu(struct cpufreq_policy *policy)

> > > -{

> > > -       pr_debug("CPU %d stopping\n", policy->cpu);

> > > -

> > > -       intel_pstate_clear_update_util_hook(policy->cpu);

> > > -}

> > > -

> > >  static int intel_pstate_cpu_exit(struct cpufreq_policy *policy)

> > >  {

> > >         pr_debug("CPU %d exiting\n", policy->cpu);

> > >

> > > +       intel_pstate_clear_update_util_hook(policy->cpu);

> >

> > This change is not needed now, because ->offline always runs before

> > ->exit if present.

>

> Not necessarily, we don't call ->offline() for many error paths in

> cpufreq_online().


I guess you mean the error paths in cpufreq_offline()?

IMO this is confusing/broken, because ->offline should always be
called after ->online has returned success.

> offline() only comes into play after driver is registered properly once.


The relevant intel_pstate case is a ->setpolicy driver where
->setpolicy or ->online, if successful, need to be followed by
->offline.

If ->setpolicy is successful in the cpufreq_online() path, the entire
cpufreq_online() is successful and the error paths in question are not
executed.

So the change I was talking about is not needed AFAICS.
Rafael J. Wysocki June 21, 2021, 2:26 p.m. | #4
On Mon, Jun 21, 2021 at 4:17 PM Rafael J. Wysocki <rafael@kernel.org> wrote:
>

> On Mon, Jun 21, 2021 at 5:09 AM Viresh Kumar <viresh.kumar@linaro.org> wrote:

> >

> > On 18-06-21, 14:00, Rafael J. Wysocki wrote:

> > > On Fri, Jun 18, 2021 at 5:22 AM Viresh Kumar <viresh.kumar@linaro.org> wrote:

> > > >

> > > > commit 367dc4aa932b ("cpufreq: Add stop CPU callback to cpufreq_driver

> > > > interface") added the stop_cpu() callback to allow the drivers to do

> > > > clean up before the CPU is completely down and its state can't be

> > > > modified.

> > > >

> > > > At that time the CPU hotplug framework used to call the cpufreq core's

> > > > registered notifier for different events like CPU_DOWN_PREPARE and

> > > > CPU_POST_DEAD. The stop_cpu() callback was called during the

> > > > CPU_DOWN_PREPARE event.

> > > >

> > > > This is no longer the case, cpuhp_cpufreq_offline() is called only once

> > > > by the CPU hotplug core now and we don't really need to separately

> > > > call stop_cpu() for cpufreq drivers.

> > > >

> > > > Migrate to using the exit() and offline() callbacks instead of

> > > > stop_cpu().

> > > >

> > > > We need to clear util hook from both the callbacks, exit() and

> > > > offline(), since it is possible that only exit() gets called sometimes

> > > > (specially on errors) or both get called at other times.

> > > > intel_pstate_clear_update_util_hook() anyway have enough protection in

> > > > place if it gets called a second time and will return early then.

> > > >

> > > > Cc: Dirk Brandewie <dirk.brandewie@gmail.com>

> > > > Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

> > > > ---

> > > > V2->V3:

> > > > - Update intel_pstate_cpu_offline() as well.

> > > > - Improved commit log.

> > > >

> > > >  drivers/cpufreq/intel_pstate.c | 12 ++++--------

> > > >  1 file changed, 4 insertions(+), 8 deletions(-)

> > > >

> > > > diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c

> > > > index 0e69dffd5a76..8f8a2d9d7daa 100644

> > > > --- a/drivers/cpufreq/intel_pstate.c

> > > > +++ b/drivers/cpufreq/intel_pstate.c

> > > > @@ -2335,6 +2335,8 @@ static int intel_pstate_cpu_offline(struct cpufreq_policy *policy)

> > > >

> > > >         pr_debug("CPU %d going offline\n", cpu->cpu);

> > > >

> > > > +       intel_pstate_clear_update_util_hook(policy->cpu);

> > > > +

> > > >         if (cpu->suspended)

> > > >                 return 0;

> > > >

> > > > @@ -2374,17 +2376,12 @@ static int intel_pstate_cpu_online(struct cpufreq_policy *policy)

> > > >         return 0;

> > > >  }

> > > >

> > > > -static void intel_pstate_stop_cpu(struct cpufreq_policy *policy)

> > > > -{

> > > > -       pr_debug("CPU %d stopping\n", policy->cpu);

> > > > -

> > > > -       intel_pstate_clear_update_util_hook(policy->cpu);

> > > > -}

> > > > -

> > > >  static int intel_pstate_cpu_exit(struct cpufreq_policy *policy)

> > > >  {

> > > >         pr_debug("CPU %d exiting\n", policy->cpu);

> > > >

> > > > +       intel_pstate_clear_update_util_hook(policy->cpu);

> > >

> > > This change is not needed now, because ->offline always runs before

> > > ->exit if present.

> >

> > Not necessarily, we don't call ->offline() for many error paths in

> > cpufreq_online().

>

> I guess you mean the error paths in cpufreq_offline()?


s/offline/online/

> IMO this is confusing/broken, because ->offline should always be

> called after ->online has returned success.

>

> > offline() only comes into play after driver is registered properly once.

>

> The relevant intel_pstate case is a ->setpolicy driver where

> ->setpolicy or ->online, if successful, need to be followed by

> ->offline.

>

> If ->setpolicy is successful in the cpufreq_online() path, the entire

> cpufreq_online() is successful and the error paths in question are not

> executed.

>

> So the change I was talking about is not needed AFAICS.
Rafael J. Wysocki June 21, 2021, 5:14 p.m. | #5
On Mon, Jun 21, 2021 at 4:26 PM Rafael J. Wysocki <rafael@kernel.org> wrote:
>

> On Mon, Jun 21, 2021 at 4:17 PM Rafael J. Wysocki <rafael@kernel.org> wrote:

> >

> > On Mon, Jun 21, 2021 at 5:09 AM Viresh Kumar <viresh.kumar@linaro.org> wrote:

> > >

> > > On 18-06-21, 14:00, Rafael J. Wysocki wrote:

> > > > On Fri, Jun 18, 2021 at 5:22 AM Viresh Kumar <viresh.kumar@linaro.org> wrote:

> > > > >

> > > > > commit 367dc4aa932b ("cpufreq: Add stop CPU callback to cpufreq_driver

> > > > > interface") added the stop_cpu() callback to allow the drivers to do

> > > > > clean up before the CPU is completely down and its state can't be

> > > > > modified.

> > > > >

> > > > > At that time the CPU hotplug framework used to call the cpufreq core's

> > > > > registered notifier for different events like CPU_DOWN_PREPARE and

> > > > > CPU_POST_DEAD. The stop_cpu() callback was called during the

> > > > > CPU_DOWN_PREPARE event.

> > > > >

> > > > > This is no longer the case, cpuhp_cpufreq_offline() is called only once

> > > > > by the CPU hotplug core now and we don't really need to separately

> > > > > call stop_cpu() for cpufreq drivers.

> > > > >

> > > > > Migrate to using the exit() and offline() callbacks instead of

> > > > > stop_cpu().

> > > > >

> > > > > We need to clear util hook from both the callbacks, exit() and

> > > > > offline(), since it is possible that only exit() gets called sometimes

> > > > > (specially on errors) or both get called at other times.

> > > > > intel_pstate_clear_update_util_hook() anyway have enough protection in

> > > > > place if it gets called a second time and will return early then.

> > > > >

> > > > > Cc: Dirk Brandewie <dirk.brandewie@gmail.com>

> > > > > Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

> > > > > ---

> > > > > V2->V3:

> > > > > - Update intel_pstate_cpu_offline() as well.

> > > > > - Improved commit log.

> > > > >

> > > > >  drivers/cpufreq/intel_pstate.c | 12 ++++--------

> > > > >  1 file changed, 4 insertions(+), 8 deletions(-)

> > > > >

> > > > > diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c

> > > > > index 0e69dffd5a76..8f8a2d9d7daa 100644

> > > > > --- a/drivers/cpufreq/intel_pstate.c

> > > > > +++ b/drivers/cpufreq/intel_pstate.c

> > > > > @@ -2335,6 +2335,8 @@ static int intel_pstate_cpu_offline(struct cpufreq_policy *policy)

> > > > >

> > > > >         pr_debug("CPU %d going offline\n", cpu->cpu);

> > > > >

> > > > > +       intel_pstate_clear_update_util_hook(policy->cpu);

> > > > > +

> > > > >         if (cpu->suspended)

> > > > >                 return 0;

> > > > >

> > > > > @@ -2374,17 +2376,12 @@ static int intel_pstate_cpu_online(struct cpufreq_policy *policy)

> > > > >         return 0;

> > > > >  }

> > > > >

> > > > > -static void intel_pstate_stop_cpu(struct cpufreq_policy *policy)

> > > > > -{

> > > > > -       pr_debug("CPU %d stopping\n", policy->cpu);

> > > > > -

> > > > > -       intel_pstate_clear_update_util_hook(policy->cpu);

> > > > > -}

> > > > > -

> > > > >  static int intel_pstate_cpu_exit(struct cpufreq_policy *policy)

> > > > >  {

> > > > >         pr_debug("CPU %d exiting\n", policy->cpu);

> > > > >

> > > > > +       intel_pstate_clear_update_util_hook(policy->cpu);

> > > >

> > > > This change is not needed now, because ->offline always runs before

> > > > ->exit if present.

> > >

> > > Not necessarily, we don't call ->offline() for many error paths in

> > > cpufreq_online().

> >

> > I guess you mean the error paths in cpufreq_offline()?

>

> s/offline/online/

>

> > IMO this is confusing/broken, because ->offline should always be

> > called after ->online has returned success.

> >

> > > offline() only comes into play after driver is registered properly once.

> >

> > The relevant intel_pstate case is a ->setpolicy driver where

> > ->setpolicy or ->online, if successful, need to be followed by

> > ->offline.

> >

> > If ->setpolicy is successful in the cpufreq_online() path, the entire

> > cpufreq_online() is successful and the error paths in question are not

> > executed.

> >

> > So the change I was talking about is not needed AFAICS.


Regardless of all of the above, the ->online, ->offline and ->exit
callback routines are used by intel_pstate both in the active mode and
the passive mode, so some more work is needed to migrate it away from
using ->stop_cpu.

Patch

diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c
index 0e69dffd5a76..8f8a2d9d7daa 100644
--- a/drivers/cpufreq/intel_pstate.c
+++ b/drivers/cpufreq/intel_pstate.c
@@ -2335,6 +2335,8 @@  static int intel_pstate_cpu_offline(struct cpufreq_policy *policy)
 
 	pr_debug("CPU %d going offline\n", cpu->cpu);
 
+	intel_pstate_clear_update_util_hook(policy->cpu);
+
 	if (cpu->suspended)
 		return 0;
 
@@ -2374,17 +2376,12 @@  static int intel_pstate_cpu_online(struct cpufreq_policy *policy)
 	return 0;
 }
 
-static void intel_pstate_stop_cpu(struct cpufreq_policy *policy)
-{
-	pr_debug("CPU %d stopping\n", policy->cpu);
-
-	intel_pstate_clear_update_util_hook(policy->cpu);
-}
-
 static int intel_pstate_cpu_exit(struct cpufreq_policy *policy)
 {
 	pr_debug("CPU %d exiting\n", policy->cpu);
 
+	intel_pstate_clear_update_util_hook(policy->cpu);
+
 	policy->fast_switch_possible = false;
 
 	return 0;
@@ -2451,7 +2448,6 @@  static struct cpufreq_driver intel_pstate = {
 	.resume		= intel_pstate_resume,
 	.init		= intel_pstate_cpu_init,
 	.exit		= intel_pstate_cpu_exit,
-	.stop_cpu	= intel_pstate_stop_cpu,
 	.offline	= intel_pstate_cpu_offline,
 	.online		= intel_pstate_cpu_online,
 	.update_limits	= intel_pstate_update_limits,