| Message ID | 20191211154343.29765-1-ulf.hansson@linaro.org |
|---|---|
| Headers | show |
| Series | cpuidle: psci: Support hierarchical CPU arrangement | expand |
Sudeep, Lorenzo, On Wed, 11 Dec 2019 at 16:43, Ulf Hansson <ulf.hansson@linaro.org> wrote: > > Changes in v4: > - Mover the check for OSI support from psci_dt_attach_cpu() to the > caller's side of it. > - Add comment in the code about using the deepest idle state as the > triggering point for the domain state selection. > - Folded in a patch to enable support for CPU hotplug. I believe I should have addressed all your provided inputs for this version, unless you find something new, of course. Then, would it be possible to get your blessing for this, before Christmas, to allow this to cook for a while in linux-next via Rafael's tree? Kind regards Uffe > > Changes in v3: > - Take one step further to completely avoid executing any OSI specific > code from the ->enter() callback, while operating in the default PSCI > Platform Coordinated mode. > - Update example for the PSCI DT bindings to make it compile with > "make dt_binding_check" > > Changes in v2: > - Avoid to affect the non-OSI path with specific changes for OSI. This > forced me to re-order the series and a caused more or less minor changes > to most of the patches. > - Updated the DT bindings for PSCI to clarify and to include the "psci" > name of the PM domain to attach to. > - Replaced patch1 with another patch from Sudeep, solving the same > problem, but in a different way. > > This series enables initial support for hierarchical CPU arrangement, managed > by PSCI and its corresponding cpuidle driver. It's based on using the generic > PM domain (genpd), which nowadays also supports devices belonging to CPUs. > > The last DTS patch enables the hierarchical topology to be used for the Qcom > 410c Dragonboard, which supports the PSCI OS-initiated mode. > > More detailed background can be found from previous submissions [1]. > > The series is also available at: > git.linaro.org/people/ulf.hansson/linux-pm.git next > > Kind regards > Ulf Hansson > > [1] > https://lwn.net/Articles/788306/ > > > Lina Iyer (1): > cpuidle: dt: Support hierarchical CPU idle states > > Sudeep Holla (1): > cpuidle: psci: Align psci_power_state count with idle state count > > Ulf Hansson (12): > dt: psci: Update DT bindings to support hierarchical PSCI states > firmware: psci: Export functions to manage the OSI mode > of: base: Add of_get_cpu_state_node() to get idle states for a CPU > node > cpuidle: psci: Simplify OF parsing of CPU idle state nodes > cpuidle: psci: Support hierarchical CPU idle states > cpuidle: psci: Add a helper to attach a CPU to its PM domain > cpuidle: psci: Attach CPU devices to their PM domains > cpuidle: psci: Prepare to use OS initiated suspend mode via PM domains > cpuidle: psci: Manage runtime PM in the idle path > cpuidle: psci: Support CPU hotplug for the hierarchical model > cpuidle: psci: Add support for PM domains by using genpd > arm64: dts: Convert to the hierarchical CPU topology layout for > MSM8916 > > .../devicetree/bindings/arm/cpus.yaml | 15 + > .../devicetree/bindings/arm/psci.yaml | 104 ++++++ > arch/arm64/boot/dts/qcom/msm8916.dtsi | 57 +++- > drivers/cpuidle/Makefile | 4 +- > drivers/cpuidle/cpuidle-psci-domain.c | 298 ++++++++++++++++++ > drivers/cpuidle/cpuidle-psci.c | 161 ++++++++-- > drivers/cpuidle/cpuidle-psci.h | 17 + > drivers/cpuidle/dt_idle_states.c | 5 +- > drivers/firmware/psci/psci.c | 18 +- > drivers/of/base.c | 36 +++ > include/linux/cpuhotplug.h | 1 + > include/linux/of.h | 8 + > include/linux/psci.h | 2 + > 13 files changed, 691 insertions(+), 35 deletions(-) > create mode 100644 drivers/cpuidle/cpuidle-psci-domain.c > create mode 100644 drivers/cpuidle/cpuidle-psci.h > > -- > 2.17.1 >
On Thu, 19 Dec 2019 at 15:32, Sudeep Holla <sudeep.holla@arm.com> wrote: > > On Wed, Dec 11, 2019 at 04:43:39PM +0100, Ulf Hansson wrote: > > The per CPU variable psci_power_state, contains an array of fixed values, > > which reflects the corresponding arm,psci-suspend-param parsed from DT, for > > each of the available CPU idle states. > > > > This isn't sufficient when using the hierarchical CPU topology in DT, in > > combination with having PSCI OS initiated (OSI) mode enabled. More > > precisely, in OSI mode, Linux is responsible of telling the PSCI FW what > > idle state the cluster (a group of CPUs) should enter, while in PSCI > > Platform Coordinated (PC) mode, each CPU independently votes for an idle > > state of the cluster. > > > > For this reason, introduce a per CPU variable called domain_state and > > implement two helper functions to read/write its value. Then let the > > domain_state take precedence over the regular selected state, when entering > > and idle state. > > > > To avoid executing the above OSI specific code in the ->enter() callback, > > while operating in the default PSCI Platform Coordinated mode, let's also > > add a new enter-function and use it for OSI. > > > > Co-developed-by: Lina Iyer <lina.iyer@linaro.org> > > Signed-off-by: Lina Iyer <lina.iyer@linaro.org> > > Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> > > --- > > > > Changes in v4: > > - Rebased on top of earlier changes. > > - Add comment about using the deepest cpuidle state for the domain state > > selection. > > > > --- > > drivers/cpuidle/cpuidle-psci.c | 56 ++++++++++++++++++++++++++++++---- > > 1 file changed, 50 insertions(+), 6 deletions(-) > > > > diff --git a/drivers/cpuidle/cpuidle-psci.c b/drivers/cpuidle/cpuidle-psci.c > > index 6a87848be3c3..9600fe674a89 100644 > > --- a/drivers/cpuidle/cpuidle-psci.c > > +++ b/drivers/cpuidle/cpuidle-psci.c > > @@ -29,14 +29,47 @@ struct psci_cpuidle_data { > > }; > > > > static DEFINE_PER_CPU_READ_MOSTLY(struct psci_cpuidle_data, psci_cpuidle_data); > > +static DEFINE_PER_CPU(u32, domain_state); > > + > > [...] > > > +static int psci_enter_domain_idle_state(struct cpuidle_device *dev, > > + struct cpuidle_driver *drv, int idx) > > +{ > > + struct psci_cpuidle_data *data = this_cpu_ptr(&psci_cpuidle_data); > > + u32 *states = data->psci_states; > > Why can't the above be like this for consistency(see below in > psci_enter_idle_state) ? You have a point, however in patch11 I am adding this line below. struct device *pd_dev = data->dev; So I don't think it matters much, agree? > > u32 *states = __this_cpu_read(psci_cpuidle_data.psci_states); > > > + u32 state = psci_get_domain_state(); > > + int ret; > > + > > + if (!state) > > + state = states[idx]; > > + > > + ret = psci_enter_state(idx, state); > > + > > + /* Clear the domain state to start fresh when back from idle. */ > > + psci_set_domain_state(0); > > + return ret; > > +} > > > > [...] > > > @@ -118,6 +152,15 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, > > ret = PTR_ERR(data->dev); > > goto free_mem; > > } > > + > > + /* > > + * Using the deepest state for the CPU to trigger a potential > > + * selection of a shared state for the domain, assumes the > > + * domain states are all deeper states. > > + */ > > + if (data->dev) > > You can drop this check as return on error above. Actually not, because if OSI is supported, there is still a possibility that the PM domain topology isn't used. This means ->data->dev is NULL. > > > + drv->states[state_count - 1].enter = > > + psci_enter_domain_idle_state; > > I see the comment above but this potential blocks retention mode at > cluster level when all cpu enter retention at CPU level. I don't like > this assumption, but I don't have any better suggestion. Please add the > note that we can't enter RETENTION state at cluster/domain level when > all CPUs enter at CPU level. You are correct, but I think the comment a few lines above (agreed to be added by Lorenzo in the previous version) should be enough to explain that. No? The point is, this is only a problem if cluster RETENTION is considered to be a shallower state that CPU power off, for example. > > As I wrote above I got another doubt. What if platform specifies just > RETENTION state at CPU as well as Cluster/domain ? I think it should be > fine, just asking it out loud. It's fine. However, I am looking at what future improvements that can be made. This is one of them, but let's discuss that later on. Kind regards Uffe
On Thu, Dec 19, 2019 at 04:48:13PM +0100, Ulf Hansson wrote: > On Thu, 19 Dec 2019 at 15:32, Sudeep Holla <sudeep.holla@arm.com> wrote: > > > > On Wed, Dec 11, 2019 at 04:43:39PM +0100, Ulf Hansson wrote: > > > The per CPU variable psci_power_state, contains an array of fixed values, > > > which reflects the corresponding arm,psci-suspend-param parsed from DT, for > > > each of the available CPU idle states. > > > > > > This isn't sufficient when using the hierarchical CPU topology in DT, in > > > combination with having PSCI OS initiated (OSI) mode enabled. More > > > precisely, in OSI mode, Linux is responsible of telling the PSCI FW what > > > idle state the cluster (a group of CPUs) should enter, while in PSCI > > > Platform Coordinated (PC) mode, each CPU independently votes for an idle > > > state of the cluster. > > > > > > For this reason, introduce a per CPU variable called domain_state and > > > implement two helper functions to read/write its value. Then let the > > > domain_state take precedence over the regular selected state, when entering > > > and idle state. > > > > > > To avoid executing the above OSI specific code in the ->enter() callback, > > > while operating in the default PSCI Platform Coordinated mode, let's also > > > add a new enter-function and use it for OSI. > > > > > > Co-developed-by: Lina Iyer <lina.iyer@linaro.org> > > > Signed-off-by: Lina Iyer <lina.iyer@linaro.org> > > > Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> > > > --- > > > > > > Changes in v4: > > > - Rebased on top of earlier changes. > > > - Add comment about using the deepest cpuidle state for the domain state > > > selection. > > > > > > --- > > > drivers/cpuidle/cpuidle-psci.c | 56 ++++++++++++++++++++++++++++++---- > > > 1 file changed, 50 insertions(+), 6 deletions(-) > > > > > > diff --git a/drivers/cpuidle/cpuidle-psci.c b/drivers/cpuidle/cpuidle-psci.c > > > index 6a87848be3c3..9600fe674a89 100644 > > > --- a/drivers/cpuidle/cpuidle-psci.c > > > +++ b/drivers/cpuidle/cpuidle-psci.c > > > @@ -29,14 +29,47 @@ struct psci_cpuidle_data { > > > }; > > > > > > static DEFINE_PER_CPU_READ_MOSTLY(struct psci_cpuidle_data, psci_cpuidle_data); > > > +static DEFINE_PER_CPU(u32, domain_state); > > > + > > > > [...] > > > > > +static int psci_enter_domain_idle_state(struct cpuidle_device *dev, > > > + struct cpuidle_driver *drv, int idx) > > > +{ > > > + struct psci_cpuidle_data *data = this_cpu_ptr(&psci_cpuidle_data); > > > + u32 *states = data->psci_states; > > > > Why can't the above be like this for consistency(see below in > > psci_enter_idle_state) ? > > You have a point, however in patch11 I am adding this line below. > > struct device *pd_dev = data->dev; > > So I don't think it matters much, agree? > Ah OK, looked odd as part of this patch, may be you could have moved this change into that patch. Anyways fine as is. > > > > u32 *states = __this_cpu_read(psci_cpuidle_data.psci_states); > > > > > + u32 state = psci_get_domain_state(); > > > + int ret; > > > + > > > + if (!state) > > > + state = states[idx]; > > > + > > > + ret = psci_enter_state(idx, state); > > > + > > > + /* Clear the domain state to start fresh when back from idle. */ > > > + psci_set_domain_state(0); > > > + return ret; > > > +} > > > > > > > [...] > > > > > @@ -118,6 +152,15 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, > > > ret = PTR_ERR(data->dev); > > > goto free_mem; > > > } > > > + > > > + /* > > > + * Using the deepest state for the CPU to trigger a potential > > > + * selection of a shared state for the domain, assumes the > > > + * domain states are all deeper states. > > > + */ > > > + if (data->dev) > > > > You can drop this check as return on error above. > > Actually not, because if OSI is supported, there is still a > possibility that the PM domain topology isn't used. > And how do we support that ? I am missing something here. > This means ->data->dev is NULL. > I don't get that. > > > > > + drv->states[state_count - 1].enter = > > > + psci_enter_domain_idle_state; > > > > I see the comment above but this potential blocks retention mode at > > cluster level when all cpu enter retention at CPU level. I don't like > > this assumption, but I don't have any better suggestion. Please add the > > note that we can't enter RETENTION state at cluster/domain level when > > all CPUs enter at CPU level. > > You are correct, but I think the comment a few lines above (agreed to > be added by Lorenzo in the previous version) should be enough to > explain that. No? > > The point is, this is only a problem if cluster RETENTION is > considered to be a shallower state that CPU power off, for example. > Yes, but give examples makes it better and helps people who may be wondering why cluster retention state is not being entered. You can just add to the above comment: "e.g. If CPU Retention is one of the shallower state, then we can't enter any of the allowed domain states." > > > > As I wrote above I got another doubt. What if platform specifies just > > RETENTION state at CPU as well as Cluster/domain ? I think it should be > > fine, just asking it out loud. > > It's fine. > > However, I am looking at what future improvements that can be made. > This is one of them, but let's discuss that later on. > OK -- Regards, Sudeep
Changes in v4: - Mover the check for OSI support from psci_dt_attach_cpu() to the caller's side of it. - Add comment in the code about using the deepest idle state as the triggering point for the domain state selection. - Folded in a patch to enable support for CPU hotplug. Changes in v3: - Take one step further to completely avoid executing any OSI specific code from the ->enter() callback, while operating in the default PSCI Platform Coordinated mode. - Update example for the PSCI DT bindings to make it compile with "make dt_binding_check" Changes in v2: - Avoid to affect the non-OSI path with specific changes for OSI. This forced me to re-order the series and a caused more or less minor changes to most of the patches. - Updated the DT bindings for PSCI to clarify and to include the "psci" name of the PM domain to attach to. - Replaced patch1 with another patch from Sudeep, solving the same problem, but in a different way. This series enables initial support for hierarchical CPU arrangement, managed by PSCI and its corresponding cpuidle driver. It's based on using the generic PM domain (genpd), which nowadays also supports devices belonging to CPUs. The last DTS patch enables the hierarchical topology to be used for the Qcom 410c Dragonboard, which supports the PSCI OS-initiated mode. More detailed background can be found from previous submissions [1]. The series is also available at: git.linaro.org/people/ulf.hansson/linux-pm.git next Kind regards Ulf Hansson [1] https://lwn.net/Articles/788306/ Lina Iyer (1): cpuidle: dt: Support hierarchical CPU idle states Sudeep Holla (1): cpuidle: psci: Align psci_power_state count with idle state count Ulf Hansson (12): dt: psci: Update DT bindings to support hierarchical PSCI states firmware: psci: Export functions to manage the OSI mode of: base: Add of_get_cpu_state_node() to get idle states for a CPU node cpuidle: psci: Simplify OF parsing of CPU idle state nodes cpuidle: psci: Support hierarchical CPU idle states cpuidle: psci: Add a helper to attach a CPU to its PM domain cpuidle: psci: Attach CPU devices to their PM domains cpuidle: psci: Prepare to use OS initiated suspend mode via PM domains cpuidle: psci: Manage runtime PM in the idle path cpuidle: psci: Support CPU hotplug for the hierarchical model cpuidle: psci: Add support for PM domains by using genpd arm64: dts: Convert to the hierarchical CPU topology layout for MSM8916 .../devicetree/bindings/arm/cpus.yaml | 15 + .../devicetree/bindings/arm/psci.yaml | 104 ++++++ arch/arm64/boot/dts/qcom/msm8916.dtsi | 57 +++- drivers/cpuidle/Makefile | 4 +- drivers/cpuidle/cpuidle-psci-domain.c | 298 ++++++++++++++++++ drivers/cpuidle/cpuidle-psci.c | 161 ++++++++-- drivers/cpuidle/cpuidle-psci.h | 17 + drivers/cpuidle/dt_idle_states.c | 5 +- drivers/firmware/psci/psci.c | 18 +- drivers/of/base.c | 36 +++ include/linux/cpuhotplug.h | 1 + include/linux/of.h | 8 + include/linux/psci.h | 2 + 13 files changed, 691 insertions(+), 35 deletions(-) create mode 100644 drivers/cpuidle/cpuidle-psci-domain.c create mode 100644 drivers/cpuidle/cpuidle-psci.h -- 2.17.1