arm/dt: Don't add disabled CPUs to system topology

Message ID 1370538685-22386-1-git-send-email-james.king@linaro.org
State New
Headers show

Commit Message

James King June 6, 2013, 5:11 p.m.
If CPUs are marked as disabled in the devicetree, make sure they do
not exist in the system CPU information and CPU topology information.
In this case these CPUs will not be able to be added to the system later
using hot-plug. This allows a single chip with many CPUs to be easily
used in a variety of hardware devices where they may have different
actual processing requirements (eg for thermal/cost reasons).

- Change devicetree.c to ignore any cpu nodes marked as disabled,
  this effectively limits the number of active cpu cores so no need
  for the max_cpus=x in the chosen node.
- Change topology.c to ignore any cpu nodes marked as disabled, this
  is where the scheduler would learn about big/LITTLE cores so this
  effectively keeps the scheduler in sync.

Signed-off-by: James King <james.king@linaro.org>
---
 Documentation/devicetree/bindings/arm/cpus.txt | 5 +++++
 arch/arm/kernel/devtree.c                      | 6 ++++++
 arch/arm/kernel/topology.c                     | 4 ++++
 3 files changed, 15 insertions(+)

Comments

Nicolas Pitre June 6, 2013, 5:35 p.m. | #1
On Thu, 6 Jun 2013, James King wrote:

> If CPUs are marked as disabled in the devicetree, make sure they do
> not exist in the system CPU information and CPU topology information.
> In this case these CPUs will not be able to be added to the system later
> using hot-plug. This allows a single chip with many CPUs to be easily
> used in a variety of hardware devices where they may have different
> actual processing requirements (eg for thermal/cost reasons).
> 
> - Change devicetree.c to ignore any cpu nodes marked as disabled,
>   this effectively limits the number of active cpu cores so no need
>   for the max_cpus=x in the chosen node.
> - Change topology.c to ignore any cpu nodes marked as disabled, this
>   is where the scheduler would learn about big/LITTLE cores so this
>   effectively keeps the scheduler in sync.
> 
> Signed-off-by: James King <james.king@linaro.org>

Acked-by: Nicolas Pitre <nico@linaro.org>


> ---
>  Documentation/devicetree/bindings/arm/cpus.txt | 5 +++++
>  arch/arm/kernel/devtree.c                      | 6 ++++++
>  arch/arm/kernel/topology.c                     | 4 ++++
>  3 files changed, 15 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index f32494d..9fbcbc5 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -45,6 +45,10 @@ For the ARM architecture every CPU node must contain the following properties:
>  		"marvell,xsc3"
>  		"marvell,xscale"
>  
> +And optionally set the following properties:
> +
> +- status:	can be set to "disabled" to remove that CPU from the system CPU topology
> +
>  Example:
>  
>  	cpus {
> @@ -73,5 +77,6 @@ Example:
>  			device_type = "cpu";
>  			compatible = "arm,cortex-a7";
>  			reg = <0x101>;
> +			status = "disabled";
>  		};
>  	};
> diff --git a/arch/arm/kernel/devtree.c b/arch/arm/kernel/devtree.c
> index 5af04f6..f4ba8ee 100644
> --- a/arch/arm/kernel/devtree.c
> +++ b/arch/arm/kernel/devtree.c
> @@ -112,6 +112,12 @@ void __init arm_dt_init_cpu_maps(void)
>  			return;
>  
>  		/*
> +		 * Check if the cpu is marked as "disabled", if so ignore.
> +		 */
> +		if (!of_device_is_available(cpu))
> +			continue;
> +
> +		/*
>  		 * Duplicate MPIDRs are a recipe for disaster.
>  		 * Scan all initialized entries and check for
>  		 * duplicates. If any is found just bail out.
> diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
> index f10316b..90f8fb3 100644
> --- a/arch/arm/kernel/topology.c
> +++ b/arch/arm/kernel/topology.c
> @@ -116,6 +116,10 @@ static void __init parse_dt_topology(void)
>  		if (cpu_eff->compatible == NULL)
>  			continue;
>  
> +		/* Check if the cpu is marked as "disabled", if so ignore. */
> +		if (!of_device_is_available(cn))
> +			continue;
> +
>  		rate = of_get_property(cn, "clock-frequency", &len);
>  		if (!rate || len != 4) {
>  			pr_err("%s missing clock-frequency property\n",
> -- 
> 1.8.1.2
>
Lorenzo Pieralisi June 7, 2013, 10:23 a.m. | #2
Hi James,

On Thu, Jun 06, 2013 at 06:11:25PM +0100, James King wrote:
> If CPUs are marked as disabled in the devicetree, make sure they do
> not exist in the system CPU information and CPU topology information.
> In this case these CPUs will not be able to be added to the system later
> using hot-plug. This allows a single chip with many CPUs to be easily
> used in a variety of hardware devices where they may have different
> actual processing requirements (eg for thermal/cost reasons).
> 
> - Change devicetree.c to ignore any cpu nodes marked as disabled,
>   this effectively limits the number of active cpu cores so no need
>   for the max_cpus=x in the chosen node.
> - Change topology.c to ignore any cpu nodes marked as disabled, this
>   is where the scheduler would learn about big/LITTLE cores so this
>   effectively keeps the scheduler in sync.
> 

I have two questions:

1) Since with this approach the DT should change anyway if on different
   hardware devices based on the same chip you want to allow booting a
   different number of CPUs, why do not we remove the cpu nodes instead of
   disabling them ? Put it another way: cpu nodes define a cpu as
   possible (currently), we can simply remove the node if we do not want
   that cpu to be seen by the kernel.
2) If we go for the "status" property, why do not we use it to set present
   mask ? That way the cpu is possible but not present, you cannot
   hotplug it in. It is a bit of a stretch, granted, the cpu _is_ present,
   we just want to disable it, do not know how this is handled in x86
   and other archs though.

I am just asking, since it is something I thought about while writing
code that parses the DT cpu map, basically we do not have a way to
"disable" a cpu in the DT and that's what you are doing, I just would like
to understand the best way to put it into DT bindings.

Thanks,
Lorenzo
James King June 7, 2013, 11:48 a.m. | #3
Hi Lorenzo,

On 7 June 2013 11:23, Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> wrote:
> Hi James,
>
> On Thu, Jun 06, 2013 at 06:11:25PM +0100, James King wrote:
>> If CPUs are marked as disabled in the devicetree, make sure they do
>> not exist in the system CPU information and CPU topology information.
>> In this case these CPUs will not be able to be added to the system later
>> using hot-plug. This allows a single chip with many CPUs to be easily
>> used in a variety of hardware devices where they may have different
>> actual processing requirements (eg for thermal/cost reasons).
>>
>> - Change devicetree.c to ignore any cpu nodes marked as disabled,
>>   this effectively limits the number of active cpu cores so no need
>>   for the max_cpus=x in the chosen node.
>> - Change topology.c to ignore any cpu nodes marked as disabled, this
>>   is where the scheduler would learn about big/LITTLE cores so this
>>   effectively keeps the scheduler in sync.
>>
>
> I have two questions:
>
> 1) Since with this approach the DT should change anyway if on different
>    hardware devices based on the same chip you want to allow booting a
>    different number of CPUs, why do not we remove the cpu nodes instead of
>    disabling them ? Put it another way: cpu nodes define a cpu as
>    possible (currently), we can simply remove the node if we do not want
>    that cpu to be seen by the kernel.

The reason we want disabled status rather than just remove the nodes
is to use a common soc.dtsi file which is included in many board.dts
files - eg:

file soc.dtsi contains:

                cpus {
                                cpu0: cpu@0 {
                                                device_type = "cpu";
                                                compatible = "arm,cortex-a7";
                                                reg = <0>;
                                                cluster = <&cluster0>;
                                                core = <&core0>;
                                };

                                cpu1: cpu@1 {
                                                device_type = "cpu";
                                                compatible = "arm,cortex-a7";
                                                reg = <1>;
                                                cluster = <&cluster0>;
                                                core = <&core1>;
                                };

                                cpu2: cpu@2 {
                                                device_type = "cpu";
                                                compatible = "arm,cortex-a15";
                                                reg = <2>;
                                                cluster = <&cluster0>;
                                                core = <&core2>;
                                };
                };

file board1.dts where we want the A15 disabled contains:

/include/ "soc.dtsi"

                cpus {
                                cpu2: cpu@2 {
                                                status = "disabled";
                                };
                };

> 2) If we go for the "status" property, why do not we use it to set present
>    mask ? That way the cpu is possible but not present, you cannot
>    hotplug it in. It is a bit of a stretch, granted, the cpu _is_ present,
>    we just want to disable it, do not know how this is handled in x86
>    and other archs though.

I have been struggling to find any equivalent example for another
arch, so just tried to solve our problem. I guess in the x86 world it
is less likely to want to disable processors in a SoC for heat/battery
issues so this has just never arisen. In this case the cpu is
physically present but not possible, but I am not sure it should be in
a present mask (giving the impression it can be used). Perhaps you
could elaborate with an example what you are thinking about here?

Thanks,
James
>
> I am just asking, since it is something I thought about while writing
> code that parses the DT cpu map, basically we do not have a way to
> "disable" a cpu in the DT and that's what you are doing, I just would like
> to understand the best way to put it into DT bindings.
>
> Thanks,
> Lorenzo
>
Lorenzo Pieralisi June 7, 2013, 4:41 p.m. | #4
On Fri, Jun 07, 2013 at 12:48:58PM +0100, James King wrote:
> Hi Lorenzo,
> 
> On 7 June 2013 11:23, Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> wrote:
> > Hi James,
> >
> > On Thu, Jun 06, 2013 at 06:11:25PM +0100, James King wrote:
> >> If CPUs are marked as disabled in the devicetree, make sure they do
> >> not exist in the system CPU information and CPU topology information.
> >> In this case these CPUs will not be able to be added to the system later
> >> using hot-plug. This allows a single chip with many CPUs to be easily
> >> used in a variety of hardware devices where they may have different
> >> actual processing requirements (eg for thermal/cost reasons).
> >>
> >> - Change devicetree.c to ignore any cpu nodes marked as disabled,
> >>   this effectively limits the number of active cpu cores so no need
> >>   for the max_cpus=x in the chosen node.
> >> - Change topology.c to ignore any cpu nodes marked as disabled, this
> >>   is where the scheduler would learn about big/LITTLE cores so this
> >>   effectively keeps the scheduler in sync.
> >>
> >
> > I have two questions:
> >
> > 1) Since with this approach the DT should change anyway if on different
> >    hardware devices based on the same chip you want to allow booting a
> >    different number of CPUs, why do not we remove the cpu nodes instead of
> >    disabling them ? Put it another way: cpu nodes define a cpu as
> >    possible (currently), we can simply remove the node if we do not want
> >    that cpu to be seen by the kernel.
> 
> The reason we want disabled status rather than just remove the nodes
> is to use a common soc.dtsi file which is included in many board.dts
> files - eg:
> 
> file soc.dtsi contains:
> 
>                 cpus {
>                                 cpu0: cpu@0 {
>                                                 device_type = "cpu";
>                                                 compatible = "arm,cortex-a7";
>                                                 reg = <0>;
>                                                 cluster = <&cluster0>;
>                                                 core = <&core0>;

Minor nit, "cluster" and "core" phandles are not part of cpu the bindings
that will be merged this cycle, I know it is just an example.

>                                 };
> 
>                                 cpu1: cpu@1 {
>                                                 device_type = "cpu";
>                                                 compatible = "arm,cortex-a7";
>                                                 reg = <1>;
>                                                 cluster = <&cluster0>;
>                                                 core = <&core1>;
>                                 };
> 
>                                 cpu2: cpu@2 {
>                                                 device_type = "cpu";
>                                                 compatible = "arm,cortex-a15";
>                                                 reg = <2>;
>                                                 cluster = <&cluster0>;
>                                                 core = <&core2>;
>                                 };
>                 };
> 
> file board1.dts where we want the A15 disabled contains:
> 
> /include/ "soc.dtsi"
> 
>                 cpus {
>                                 cpu2: cpu@2 {
>                                                 status = "disabled";
>                                 };
>                 };

Understood, see the other reply as far as the status property is concerned.

> > 2) If we go for the "status" property, why do not we use it to set present
> >    mask ? That way the cpu is possible but not present, you cannot
> >    hotplug it in. It is a bit of a stretch, granted, the cpu _is_ present,
> >    we just want to disable it, do not know how this is handled in x86
> >    and other archs though.
> 
> I have been struggling to find any equivalent example for another
> arch, so just tried to solve our problem. I guess in the x86 world it
> is less likely to want to disable processors in a SoC for heat/battery
> issues so this has just never arisen. In this case the cpu is
> physically present but not possible, but I am not sure it should be in
> a present mask (giving the impression it can be used). Perhaps you
> could elaborate with an example what you are thinking about here?

I was thinking about using status == "disabled" to mark a cpu as
possible but not present; that is a bad idea for the reason you mentioned
and also for the point Rob raised related to the ePAPR.

I am not sure how we can solve this issue, as I mentioned the easiest
solution consists in not defining cpu nodes in the DT or probably add
an additional property != status with proper bindings attached to it.

Lorenzo
James King June 10, 2013, 10:48 a.m. | #5
On 7 June 2013 17:41, Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> wrote:
> On Fri, Jun 07, 2013 at 12:48:58PM +0100, James King wrote:
>> Hi Lorenzo,
>>
>> On 7 June 2013 11:23, Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> wrote:
>> > Hi James,
>> >
>> > On Thu, Jun 06, 2013 at 06:11:25PM +0100, James King wrote:
>> >> If CPUs are marked as disabled in the devicetree, make sure they do
>> >> not exist in the system CPU information and CPU topology information.
>> >> In this case these CPUs will not be able to be added to the system later
>> >> using hot-plug. This allows a single chip with many CPUs to be easily
>> >> used in a variety of hardware devices where they may have different
>> >> actual processing requirements (eg for thermal/cost reasons).
>> >>
>> >> - Change devicetree.c to ignore any cpu nodes marked as disabled,
>> >>   this effectively limits the number of active cpu cores so no need
>> >>   for the max_cpus=x in the chosen node.
>> >> - Change topology.c to ignore any cpu nodes marked as disabled, this
>> >>   is where the scheduler would learn about big/LITTLE cores so this
>> >>   effectively keeps the scheduler in sync.
>> >>
>> >
>> > I have two questions:
>> >
>> > 1) Since with this approach the DT should change anyway if on different
>> >    hardware devices based on the same chip you want to allow booting a
>> >    different number of CPUs, why do not we remove the cpu nodes instead of
>> >    disabling them ? Put it another way: cpu nodes define a cpu as
>> >    possible (currently), we can simply remove the node if we do not want
>> >    that cpu to be seen by the kernel.
>>
>> The reason we want disabled status rather than just remove the nodes
>> is to use a common soc.dtsi file which is included in many board.dts
>> files - eg:
>>
>> file soc.dtsi contains:
>>
>>                 cpus {
>>                                 cpu0: cpu@0 {
>>                                                 device_type = "cpu";
>>                                                 compatible = "arm,cortex-a7";
>>                                                 reg = <0>;
>>                                                 cluster = <&cluster0>;
>>                                                 core = <&core0>;
>
> Minor nit, "cluster" and "core" phandles are not part of cpu the bindings
> that will be merged this cycle, I know it is just an example.
>
>>                                 };
>>
>>                                 cpu1: cpu@1 {
>>                                                 device_type = "cpu";
>>                                                 compatible = "arm,cortex-a7";
>>                                                 reg = <1>;
>>                                                 cluster = <&cluster0>;
>>                                                 core = <&core1>;
>>                                 };
>>
>>                                 cpu2: cpu@2 {
>>                                                 device_type = "cpu";
>>                                                 compatible = "arm,cortex-a15";
>>                                                 reg = <2>;
>>                                                 cluster = <&cluster0>;
>>                                                 core = <&core2>;
>>                                 };
>>                 };
>>
>> file board1.dts where we want the A15 disabled contains:
>>
>> /include/ "soc.dtsi"
>>
>>                 cpus {
>>                                 cpu2: cpu@2 {
>>                                                 status = "disabled";
>>                                 };
>>                 };
>
> Understood, see the other reply as far as the status property is concerned.
>
>> > 2) If we go for the "status" property, why do not we use it to set present
>> >    mask ? That way the cpu is possible but not present, you cannot
>> >    hotplug it in. It is a bit of a stretch, granted, the cpu _is_ present,
>> >    we just want to disable it, do not know how this is handled in x86
>> >    and other archs though.
>>
>> I have been struggling to find any equivalent example for another
>> arch, so just tried to solve our problem. I guess in the x86 world it
>> is less likely to want to disable processors in a SoC for heat/battery
>> issues so this has just never arisen. In this case the cpu is
>> physically present but not possible, but I am not sure it should be in
>> a present mask (giving the impression it can be used). Perhaps you
>> could elaborate with an example what you are thinking about here?
>
> I was thinking about using status == "disabled" to mark a cpu as
> possible but not present; that is a bad idea for the reason you mentioned
> and also for the point Rob raised related to the ePAPR.
>
> I am not sure how we can solve this issue, as I mentioned the easiest
> solution consists in not defining cpu nodes in the DT or probably add
> an additional property != status with proper bindings attached to it.
>

So what the ePAPR says is:
"disabled" == Indicates that the device is not presently operational,
but it might
become operational in the future (for example, something is not
plugged in, or switched off).
Refer to the device binding for details on what disabled means for
a given device.

So, would "disabled" be ok to use if those cores 'could' be re-added
to the system using hot-plug?

This would probably be fine for our usage as the user would have to
have the right permissions to hot-plug in a cpu anyway (so is just as
strong a protection as the DT anyway - if you root your device you can
burn it). The main thing is the scheduler will not try and use it by
default.

If that sounds ok - I need to go back and look at the changes to see
what will still allow a core to be hot-plugged in.

Thanks,
James

> Lorenzo
>

Patch

diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
index f32494d..9fbcbc5 100644
--- a/Documentation/devicetree/bindings/arm/cpus.txt
+++ b/Documentation/devicetree/bindings/arm/cpus.txt
@@ -45,6 +45,10 @@  For the ARM architecture every CPU node must contain the following properties:
 		"marvell,xsc3"
 		"marvell,xscale"
 
+And optionally set the following properties:
+
+- status:	can be set to "disabled" to remove that CPU from the system CPU topology
+
 Example:
 
 	cpus {
@@ -73,5 +77,6 @@  Example:
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
 			reg = <0x101>;
+			status = "disabled";
 		};
 	};
diff --git a/arch/arm/kernel/devtree.c b/arch/arm/kernel/devtree.c
index 5af04f6..f4ba8ee 100644
--- a/arch/arm/kernel/devtree.c
+++ b/arch/arm/kernel/devtree.c
@@ -112,6 +112,12 @@  void __init arm_dt_init_cpu_maps(void)
 			return;
 
 		/*
+		 * Check if the cpu is marked as "disabled", if so ignore.
+		 */
+		if (!of_device_is_available(cpu))
+			continue;
+
+		/*
 		 * Duplicate MPIDRs are a recipe for disaster.
 		 * Scan all initialized entries and check for
 		 * duplicates. If any is found just bail out.
diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
index f10316b..90f8fb3 100644
--- a/arch/arm/kernel/topology.c
+++ b/arch/arm/kernel/topology.c
@@ -116,6 +116,10 @@  static void __init parse_dt_topology(void)
 		if (cpu_eff->compatible == NULL)
 			continue;
 
+		/* Check if the cpu is marked as "disabled", if so ignore. */
+		if (!of_device_is_available(cn))
+			continue;
+
 		rate = of_get_property(cn, "clock-frequency", &len);
 		if (!rate || len != 4) {
 			pr_err("%s missing clock-frequency property\n",