[v6,00/25] PM / Domains: Support hierarchical CPU arrangement (PSCI/ARM)

Message ID 1521046715-30683-1-git-send-email-ulf.hansson@linaro.org
Headers show
Series
  • PM / Domains: Support hierarchical CPU arrangement (PSCI/ARM)
Related show

Message

Ulf Hansson March 14, 2018, 4:58 p.m.
This series is a re-worked version from Lina Iyer's two series [1] that got
posted more than a year ago by now. I have picked up the series and done a
significant re-work of it and here's the result. All patches have been changed,
some have been dropped, some are entirely new. For this reason I decided to not
include a version history, as I think people need a fresh start anyway.

For those few changes that are kind of similar, I have kept Lina's authorship,
the rest are now authored by me.

 - Lina (or anybody else for that matter) please tell if you think I should
   change, in regards to what tag/authorship should be for each commit!

Some background:

Overall this series have been discussed over years at various Linux conferences
and LKML, although let me give a brief introduction and then the rest can be
read in each changelog.

For ARM, the PSCI firmware interface may be managing the power to the CPUs.
Depending on the SoC, CPUs may also be arranged in hierarchical manner, which
could add another level of complexity from a CPU idle management point of view.

PSCI v1.0+ adds support for the so called OS initiated CPU suspend mode, which
enables a more fine grained method, allowing Linux to get more control, in
regards to being energy efficient. This is typically useful for these kind of
complex battery driven platforms.

Now, in principle what is missing today around CPU idle management for these
SoCs arranging CPUs in a hierarchical manner, that is what this series intends
to address.

I could potentially have split the series into two parts, but for completeness
I decided to post it as one. Please tell if want me to change!

 - The first part are changes to generic PM code (genpd, cpu_pm and timers):
   Patch 1 -> Patch 9.

 - The second part, which depends on the first, are changes specific to
   PSCI/ARM64 (psci, of, arm64): Patch 10 -> Patch 25.

The series is based on v4.16-rc5, but applies also as of today to Rafael's
linux-pm.git next branch. The code has been tested on a QCOM 410c dragonboard
and I except that test coverage should be increased by some more boards
shortly (for ARM64 with PSCI OSI support - only a few DT changes are needed).
You may find the code at:

git.linaro.org/people/ulf.hansson/linux-pm.git next


Kind regards
Ulf Hansson

[1]
https://lwn.net/Articles/716300/
https://www.spinics.net/lists/arm-kernel/msg566200.html


Lina Iyer (6):
  PM / Domains: Add generic data pointer to genpd_power_state struct
  timer: Export next wakeup time of a CPU
  dt: psci: Update DT bindings to support hierarchical PSCI states
  cpuidle: dt: Support hierarchical CPU idle states
  drivers: firmware: psci: Support hierarchical CPU idle states
  arm64: dts: Convert to the hierarchical CPU topology layout for
    MSM8916

Ulf Hansson (19):
  PM / Domains: Don't treat zero found compatible idle states as an
    error
  PM / Domains: Deal with multiple states but no governor in genpd
  PM / Domains: Add support for CPU devices to genpd
  PM / Domains: Add helper functions to attach/detach CPUs to/from genpd
  PM / Domains: Add genpd governor for CPUs
  PM / Domains: Extend genpd CPU governor to cope with QoS constraints
  kernel/cpu_pm: Manage runtime PM in the idle path for CPUs
  of: base: Add of_get_cpu_state_node() to get idle states for a CPU
    node
  drivers: firmware: psci: Split psci_dt_cpu_init_idle()
  drivers: firmware: psci: Simplify error path of psci_dt_init()
  drivers: firmware: psci: Announce support for OS initiated suspend
    mode
  drivers: firmware: psci: Prepare to use OS initiated suspend mode
  drivers: firmware: psci: Share a few internal PSCI functions
  drivers: firmware: psci: Add support for PM domains using genpd
  drivers: firmware: psci: Introduce psci_dt_topology_init()
  drivers: firmware: psci: Try to attach CPU devices to their PM domains
  drivers: firmware: psci: Deal with CPU hotplug when using OSI mode
  MAINTAINERS: Update files for PSCI
  arm64: kernel: Respect the hierarchical CPU topology in DT for PSCI

 Documentation/devicetree/bindings/arm/psci.txt | 156 +++++++++++++++++++++
 MAINTAINERS                                    |   2 +-
 arch/arm64/boot/dts/qcom/msm8916.dtsi          |  53 +++++++-
 arch/arm64/kernel/setup.c                      |   3 +
 drivers/base/power/domain.c                    | 156 ++++++++++++++++++++-
 drivers/base/power/domain_governor.c           |  65 ++++++++-
 drivers/cpuidle/dt_idle_states.c               |   5 +-
 drivers/firmware/Makefile                      |   2 +-
 drivers/firmware/psci.c                        | 169 +++++++++++++++++------
 drivers/firmware/psci.h                        |  19 +++
 drivers/firmware/psci_pm_domain.c              | 180 +++++++++++++++++++++++++
 drivers/of/base.c                              |  35 +++++
 include/linux/of.h                             |   8 ++
 include/linux/pm_domain.h                      |  16 +++
 include/linux/psci.h                           |   2 +
 include/linux/tick.h                           |  10 ++
 include/uapi/linux/psci.h                      |   5 +
 kernel/cpu_pm.c                                |  11 ++
 kernel/time/tick-sched.c                       |  11 ++
 19 files changed, 853 insertions(+), 55 deletions(-)
 create mode 100644 drivers/firmware/psci.h
 create mode 100644 drivers/firmware/psci_pm_domain.c

-- 
2.7.4

Comments

Geert Uytterhoeven March 15, 2018, 11 a.m. | #1
Hi Ulf,

On Wed, Mar 14, 2018 at 5:58 PM, Ulf Hansson <ulf.hansson@linaro.org> wrote:
> This series is a re-worked version from Lina Iyer's two series [1] that got

> posted more than a year ago by now. I have picked up the series and done a

> significant re-work of it and here's the result. All patches have been changed,

> some have been dropped, some are entirely new. For this reason I decided to not

> include a version history, as I think people need a fresh start anyway.


Thanks for your series!

I gave it a try on a few Renesas boards and SoCs, without adding any new DT
descriptions.  On all of them it triggers

    BUG: sleeping function called from invalid context at
drivers/base/power/runtime.c:1057

during system suspend (s2ram).

On R-Car Gen2 and SH-Mobile AG5 (arm32, no PSCI):

Disabling non-boot CPUs ...
BUG: sleeping function called from invalid context at
drivers/base/power/runtime.c:1057
in_atomic(): 0, irqs_disabled(): 128, pid: 1725, name: s2ram
CPU: 0 PID: 1725 Comm: s2ram Not tainted
4.16.0-rc5-koelsch-00475-gaa69fc46cc44c3d7 #4001
Hardware name: Generic R-Car Gen2 (Flattened Device Tree)
[<c020f6a0>] (unwind_backtrace) from [<c020b324>] (show_stack+0x10/0x14)
[<c020b324>] (show_stack) from [<c07693c8>] (dump_stack+0x7c/0x9c)
[<c07693c8>] (dump_stack) from [<c02422ec>] (___might_sleep+0x128/0x164)
[<c02422ec>] (___might_sleep) from [<c0508efc>] (__pm_runtime_suspend+0x70/0xa8)
[<c0508efc>] (__pm_runtime_suspend) from [<c0299144>] (cpu_pm_enter+0x78/0x9c)
[<c0299144>] (cpu_pm_enter) from [<c0299170>] (cpu_pm_suspend+0x8/0x18)
[<c0299170>] (cpu_pm_suspend) from [<c04ff8c4>] (syscore_suspend+0x88/0x138)
[<c04ff8c4>] (syscore_suspend) from [<c0263010>]
(suspend_devices_and_enter+0x21c/0x564)
[<c0263010>] (suspend_devices_and_enter) from [<c02635a8>]
(pm_suspend+0x250/0x2c8)
[<c02635a8>] (pm_suspend) from [<c0262054>] (state_store+0xac/0xcc)
[<c0262054>] (state_store) from [<c035f238>] (kernfs_fop_write+0x170/0x1b0)
[<c035f238>] (kernfs_fop_write) from [<c02f8a5c>] (__vfs_write+0x2c/0x140)
[<c02f8a5c>] (__vfs_write) from [<c02f8ce4>] (vfs_write+0xb8/0x144)
[<c02f8ce4>] (vfs_write) from [<c02f8ea4>] (SyS_write+0x54/0xac)
[<c02f8ea4>] (SyS_write) from [<c0201000>] (ret_fast_syscall+0x0/0x4c)
Exception stack(0xeabcbfa8 to 0xeabcbff0)
bfa0:                   00000004 000ce408 00000001 000ce408 00000004 00000000
bfc0: 00000004 000ce408 b6e80b50 00000004 00000004 00000000 000c5758 00000000
bfe0: 00000000 be866754 b6de3c85 b6e1ef26


On R-Car Gen3 (arm64, PSCI):

Disabling non-boot CPUs ...
CPU1: shutdown
psci: CPU1 killed.
CPU2: shutdown
psci: CPU2 killed.
CPU3: shutdown
psci: CPU3 killed.
CPU4: shutdown
psci: CPU4 killed.
CPU5: shutdown
psci: CPU5 killed.
CPU6: shutdown
psci: CPU6 killed.
CPU7: shutdown
psci: CPU7 killed.
BUG: sleeping function called from invalid context at
drivers/base/power/runtime.c:1057
in_atomic(): 0, irqs_disabled(): 128, pid: 2592, name: s2ram
4 locks held by s2ram/2592:
 #0:  (sb_writers#7){.+.+}, at: [<00000000cae1f0e5>] vfs_write+0xb0/0x164
 #1:  (&of->mutex){+.+.}, at: [<000000003002e527>] kernfs_fop_write+0x114/0x1bc
 #2:  (kn->count#71){.+.+}, at: [<000000008c0217e1>]
kernfs_fop_write+0x11c/0x1bc
 #3:  (pm_mutex){+.+.}, at: [<000000009a6c23e2>] pm_suspend+0x194/0xb10
irq event stamp: 69308
hardirqs last  enabled at (69307): [<00000000e9d09767>]
_raw_spin_unlock_irq+0x2c/0x4c
hardirqs last disabled at (69308): [<00000000619169c4>]
arch_suspend_disable_irqs+0x10/0x18
softirqs last  enabled at (69246): [<00000000b8a7706e>]
hrtimers_dead_cpu+0x2b8/0x2f0
softirqs last disabled at (69242): [<000000004dee0c40>]
hrtimers_dead_cpu+0x48/0x2f0
CPU: 0 PID: 2592 Comm: s2ram Not tainted
4.16.0-rc5-salvator-x-00470-g319cfb3643965f46 #1685
Hardware name: Renesas Salvator-X 2nd version board based on r8a7795 ES2.0+ (DT)
Call trace:
 dump_backtrace+0x0/0x140
 show_stack+0x14/0x1c
 dump_stack+0xb4/0xf0
 ___might_sleep+0x1fc/0x218
 __might_sleep+0x70/0x80
 __pm_runtime_suspend+0x6c/0xac
 cpu_pm_enter+0x74/0x9c
 cpu_pm_suspend+0xc/0x1c
 syscore_suspend+0x1b8/0x410
 suspend_devices_and_enter+0x210/0xd9c
 pm_suspend+0x9a4/0xb10
 state_store+0xd4/0xf8
 kobj_attr_store+0x18/0x28
 sysfs_kf_write+0x50/0x5c
 kernfs_fop_write+0x178/0x1bc
 __vfs_write+0x38/0x140
 vfs_write+0xc4/0x164
 SyS_write+0x54/0xa4
 el0_svc_naked+0x30/0x34


I've bisected this to "[PATCH v6 09/25] kernel/cpu_pm: Manage runtime PM
in the idle path for CPUs".

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds