| Message ID | 20240210030549.4048795-4-saravanak@google.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Add post-init-supplier binding to improve suspend/resume stability | expand |
On Sat, Feb 10, 2024 at 3:06 AM Saravana Kannan <saravanak@google.com> wrote: > > The post-init-supplier property can be used to break a dependency cycle by > marking some supplier(s) as a post device initialization supplier(s). This > allows the kernel to do a better job at ordering initialization and s/the kernel/an OS/ > suspend/resume of the devices in a dependency cycle. > > Signed-off-by: Saravana Kannan <saravanak@google.com> > --- > .../bindings/post-init-supplier.yaml | 99 +++++++++++++++++++ This should probably go into dtschema instead, but fine here now for reviewing. We have to consider if this property should be automatically allowed on any node or nodes with specific suppliers, or if it should be explicit for users. The former needs tool support. I'm leaning towards the latter as I want to know when this is needed. > MAINTAINERS | 3 +- > 2 files changed, 101 insertions(+), 1 deletion(-) > create mode 100644 Documentation/devicetree/bindings/post-init-supplier.yaml > > diff --git a/Documentation/devicetree/bindings/post-init-supplier.yaml b/Documentation/devicetree/bindings/post-init-supplier.yaml > new file mode 100644 > index 000000000000..cf9071ecd06e > --- /dev/null > +++ b/Documentation/devicetree/bindings/post-init-supplier.yaml > @@ -0,0 +1,99 @@ > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) > +# Copyright 2018 Linaro Ltd. ? > +%YAML 1.2 > +--- > +$id: http://devicetree.org/schemas/post-init-supplier.yaml# > +$schema: http://devicetree.org/meta-schemas/core.yaml# > + > +title: Post device initialization supplier > + > +maintainers: > + - Saravana Kannan <saravanak@google.com> > + > +description: | > + This property is used to indicate that the device(s) pointed to by the > + property are not needed for the initialization of the device that lists this > + property. > + > + A device can list its suppliers in devicetree using one or more of the > + standard devicetree bindings. By default, it would be safe to assume the > + supplier device can be initialized before the consumer device is initialized. > + > + However, that assumption cannot be made when there are cyclic dependecies typo > + between devices. Since each device is a supplier (directly or indirectly) of > + the others in the cycle, there is no guaranteed safe order for initalizing typo > + the devices in a cycle. We can try to initialize them in an arbitrary order > + and eventually successfully initialize all of them, but that doesn't always > + work well. > + > + For example, say, > + * The device tree has the following cyclic dependency X -> Y -> Z -> X (where > + -> denotes "depends on"). > + * But X is not needed to fully initialize Z (X might be needed only when a > + specific functionality if requested post initialization). > + > + If all the other -> are mandatory initialization dependencies, then trying to > + initialize the devices in a loop (or arbitrarily) will always eventually end > + up with the devices being initialized in the order Z, Y and X. > + > + However, if Y is an optional supplier for X (where X provides limited > + functionality when Y is not initialized and providing its services), then > + trying to initialize the devices in a loop (or arbitrarily) could end up with > + the devices being initialized in the following order: > + > + * Z, Y and X - All devices provide full functionality > + * Z, X and Y - X provides partial functionality > + * X, Z and Y - X provides partial functionality > + > + However, we always want to initialize the devices in the order Z, Y and X > + since that provides the full functionality without interruptions. > + > + One alternate option that might be suggested is to have the driver for X > + notice that Y became available at a later point and adjust the functionality > + it provides. However, other userspace applications could have started using X > + with the limited functionality before Y was available and it might not be > + possible to transparently transition X or the users of X to full > + functionality while X is in use. > + > + Similarly, when it comes to suspend (resume) ordering, it's unclear which > + device in a dependency cycle needs to be suspended/resumed first and trying > + arbitrary orders can result in system crashes or instability. > + > + Explicitly calling out which link in a cycle needs to be broken when > + determining the order, simplifies things a lot, improves efficiency, makes > + the behavior more deterministic and maximizes the functionality that can be > + provided without interruption. > + > + This property is used to provide this additional information between devices > + in a cycle by telling which supplier(s) is not needed for initializing the > + device that lists this property. > + > + In the example above, Z would list X as a post-init-supplier and the > + initialization dependency would become X -> Y -> Z -/-> X. So the best order > + to initialize them become clear: Z, Y and then X. > + select: true Otherwise, this is never applied. > +properties: > + # A dictionary of DT properties for this binding schema Drop > + post-init-supplier: > + # One or more suppliers can be marked as post initialization supplier > + minItems: 1 That's the default. > + description: > + List of phandles to suppliers that are not needed for initializing or > + resuming this device. > + $ref: /schemas/types.yaml#/definitions/phandle Should be phandle-array plus: items: maxItems: 1 (as each entry is a single phandle) > + > +examples: > + - | > + gcc: general-clock-controller@1000 { clock-controller@1000 > + compatible = "vendor,soc4-gcc", "vendor,soc1-gcc"; > + reg = <0x1000 0x80>;> + clocks = <&dispcc 0x1> > + #clock-cells = <1>; > + post-init-supplier = <&dispcc>; > + }; > + dispcc: display-clock-controller@2000 { clock-controller@2000 > + compatible = "vendor,soc4-dispcc", "vendor,soc1-dispcc"; > + reg = <0x2000 0x80>; > + clocks = <&gcc 0xdd> > + #clock-cells = <1>; > + }; > diff --git a/MAINTAINERS b/MAINTAINERS > index 3dfe7ea25320..40fd498543a5 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -6055,10 +6055,11 @@ S: Maintained > F: drivers/base/devcoredump.c > F: include/linux/devcoredump.h > > -DEVICE DEPENDENCY HELPER SCRIPT > +FIRMWARE DEVICE LINK (fw_devlink) > M: Saravana Kannan <saravanak@google.com> > L: linux-kernel@vger.kernel.org > S: Maintained > +F: Documentation/devicetree/bindings/post-init-supplier.yaml > F: scripts/dev-needs.sh > > DEVICE DIRECT ACCESS (DAX) > -- > 2.43.0.687.g38aa6559b0-goog >
On Fri, 09 Feb 2024 19:05:46 -0800, Saravana Kannan wrote: > The post-init-supplier property can be used to break a dependency cycle by > marking some supplier(s) as a post device initialization supplier(s). This > allows the kernel to do a better job at ordering initialization and > suspend/resume of the devices in a dependency cycle. > > Signed-off-by: Saravana Kannan <saravanak@google.com> > --- > .../bindings/post-init-supplier.yaml | 99 +++++++++++++++++++ > MAINTAINERS | 3 +- > 2 files changed, 101 insertions(+), 1 deletion(-) > create mode 100644 Documentation/devicetree/bindings/post-init-supplier.yaml > My bot found errors running 'make DT_CHECKER_FLAGS=-m dt_binding_check' on your patch (DT_CHECKER_FLAGS is new in v5.13): yamllint warnings/errors: dtschema/dtc warnings/errors: /builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/post-init-supplier.yaml: properties:post-init-supplier:minItems: False schema does not allow 1 hint: Scalar properties should not have array keywords from schema $id: http://devicetree.org/meta-schemas/keywords.yaml# /builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/post-init-supplier.yaml: 'oneOf' conditional failed, one must be fixed: 'unevaluatedProperties' is a required property 'additionalProperties' is a required property hint: Either unevaluatedProperties or additionalProperties must be present from schema $id: http://devicetree.org/meta-schemas/core.yaml# Error: Documentation/devicetree/bindings/post-init-supplier.example.dts:22.13-14 syntax error FATAL ERROR: Unable to parse input tree make[2]: *** [scripts/Makefile.lib:419: Documentation/devicetree/bindings/post-init-supplier.example.dtb] Error 1 make[2]: *** Waiting for unfinished jobs.... make[1]: *** [/builds/robherring/dt-review-ci/linux/Makefile:1428: dt_binding_check] Error 2 make: *** [Makefile:240: __sub-make] Error 2 doc reference errors (make refcheckdocs): See https://patchwork.ozlabs.org/project/devicetree-bindings/patch/20240210030549.4048795-4-saravanak@google.com The base for the series is generally the latest rc1. A different dependency should be noted in *this* patch. If you already ran 'make dt_binding_check' and didn't see the above error(s), then make sure 'yamllint' is installed and dt-schema is up to date: pip3 install dtschema --upgrade Please check and re-submit after running the above command yourself. Note that DT_SCHEMA_FILES can be set to your schema file to speed up checking your schema. However, it must be unset to test all examples with your schema.
On 10/02/2024 04:05, Saravana Kannan wrote: > The post-init-supplier property can be used to break a dependency cycle by > marking some supplier(s) as a post device initialization supplier(s). This > allows the kernel to do a better job at ordering initialization and > suspend/resume of the devices in a dependency cycle. ... > diff --git a/MAINTAINERS b/MAINTAINERS > index 3dfe7ea25320..40fd498543a5 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -6055,10 +6055,11 @@ S: Maintained > F: drivers/base/devcoredump.c > F: include/linux/devcoredump.h > > -DEVICE DEPENDENCY HELPER SCRIPT > +FIRMWARE DEVICE LINK (fw_devlink) This breaks ordering of MAINTAINERS... Best regards, Krzysztof
On Sat, Feb 10, 2024 at 3:26 AM Rob Herring <robh+dt@kernel.org> wrote: > > On Sat, Feb 10, 2024 at 3:06 AM Saravana Kannan <saravanak@google.com> wrote: > > > > The post-init-supplier property can be used to break a dependency cycle by > > marking some supplier(s) as a post device initialization supplier(s). This > > allows the kernel to do a better job at ordering initialization and > > s/the kernel/an OS/ > > > suspend/resume of the devices in a dependency cycle. > > > > Signed-off-by: Saravana Kannan <saravanak@google.com> > > --- > > .../bindings/post-init-supplier.yaml | 99 +++++++++++++++++++ > > This should probably go into dtschema instead, but fine here now for reviewing. How do I submit it for dtschema? > We have to consider if this property should be automatically allowed > on any node or nodes with specific suppliers, or if it should be > explicit for users. The former needs tool support. I'm leaning towards > the latter as I want to know when this is needed. I think I understand your question. This property can be relevant for any device -- it really depends on the device's consumers and suppliers and if they form a cycle. Ideally, we can check if a device is in a cycle and then allow it only for those, but that's going to be too complicated I think. So, I think this should be allowed for any device. Since this property is defined as a "post-init-supplier" and only applies to an existing direct supplier, there's not a lot of room for misuse. Actually if you can add a check "are all the phandles listed in this property also present in one of the other properties in this device node", that should be sufficient. I'll also edit the documentation to make it clear it's meaningful only for an existing supplier. > > > MAINTAINERS | 3 +- > > 2 files changed, 101 insertions(+), 1 deletion(-) > > create mode 100644 Documentation/devicetree/bindings/post-init-supplier.yaml > > > > diff --git a/Documentation/devicetree/bindings/post-init-supplier.yaml b/Documentation/devicetree/bindings/post-init-supplier.yaml > > new file mode 100644 > > index 000000000000..cf9071ecd06e > > --- /dev/null > > +++ b/Documentation/devicetree/bindings/post-init-supplier.yaml > > @@ -0,0 +1,99 @@ > > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) > > +# Copyright 2018 Linaro Ltd. > > ? Oops... didn't edit out all the lines in the example. I'll address all your other comments in the next patch. Btw, this is the command I used to validate this file (before I sent v1), but it didn't spit out anything meaningful that I could understand: make dt_binding_check DT_CHECKER_FLAGS=-m DT_SCHEMA_FILES=Documentation/devicetree/bindings/trivial-devices.yaml Is that the right way to run it? I'm using yamllint 1.32. So it's not terribly old either (can't install anything newer). -Saravana > > > +%YAML 1.2 > > +--- > > +$id: http://devicetree.org/schemas/post-init-supplier.yaml# > > +$schema: http://devicetree.org/meta-schemas/core.yaml# > > + > > +title: Post device initialization supplier > > + > > +maintainers: > > + - Saravana Kannan <saravanak@google.com> > > + > > +description: | > > + This property is used to indicate that the device(s) pointed to by the > > + property are not needed for the initialization of the device that lists this > > + property. > > + > > + A device can list its suppliers in devicetree using one or more of the > > + standard devicetree bindings. By default, it would be safe to assume the > > + supplier device can be initialized before the consumer device is initialized. > > + > > + However, that assumption cannot be made when there are cyclic dependecies > > typo > > > + between devices. Since each device is a supplier (directly or indirectly) of > > + the others in the cycle, there is no guaranteed safe order for initalizing > > typo > > > + the devices in a cycle. We can try to initialize them in an arbitrary order > > + and eventually successfully initialize all of them, but that doesn't always > > + work well. > > + > > + For example, say, > > + * The device tree has the following cyclic dependency X -> Y -> Z -> X (where > > + -> denotes "depends on"). > > + * But X is not needed to fully initialize Z (X might be needed only when a > > + specific functionality if requested post initialization). > > + > > + If all the other -> are mandatory initialization dependencies, then trying to > > + initialize the devices in a loop (or arbitrarily) will always eventually end > > + up with the devices being initialized in the order Z, Y and X. > > + > > + However, if Y is an optional supplier for X (where X provides limited > > + functionality when Y is not initialized and providing its services), then > > + trying to initialize the devices in a loop (or arbitrarily) could end up with > > + the devices being initialized in the following order: > > + > > + * Z, Y and X - All devices provide full functionality > > + * Z, X and Y - X provides partial functionality > > + * X, Z and Y - X provides partial functionality > > + > > + However, we always want to initialize the devices in the order Z, Y and X > > + since that provides the full functionality without interruptions. > > + > > + One alternate option that might be suggested is to have the driver for X > > + notice that Y became available at a later point and adjust the functionality > > + it provides. However, other userspace applications could have started using X > > + with the limited functionality before Y was available and it might not be > > + possible to transparently transition X or the users of X to full > > + functionality while X is in use. > > + > > + Similarly, when it comes to suspend (resume) ordering, it's unclear which > > + device in a dependency cycle needs to be suspended/resumed first and trying > > + arbitrary orders can result in system crashes or instability. > > + > > + Explicitly calling out which link in a cycle needs to be broken when > > + determining the order, simplifies things a lot, improves efficiency, makes > > + the behavior more deterministic and maximizes the functionality that can be > > + provided without interruption. > > + > > + This property is used to provide this additional information between devices > > + in a cycle by telling which supplier(s) is not needed for initializing the > > + device that lists this property. > > + > > + In the example above, Z would list X as a post-init-supplier and the > > + initialization dependency would become X -> Y -> Z -/-> X. So the best order > > + to initialize them become clear: Z, Y and then X. > > + > > select: true > > Otherwise, this is never applied. > > > +properties: > > + # A dictionary of DT properties for this binding schema > > Drop > > > + post-init-supplier: > > + # One or more suppliers can be marked as post initialization supplier > > + minItems: 1 > > That's the default. > > > + description: > > + List of phandles to suppliers that are not needed for initializing or > > + resuming this device. > > + $ref: /schemas/types.yaml#/definitions/phandle > > Should be phandle-array plus: > > items: > maxItems: 1 > > (as each entry is a single phandle) > > > + > > +examples: > > + - | > > + gcc: general-clock-controller@1000 { > > clock-controller@1000 > > > + compatible = "vendor,soc4-gcc", "vendor,soc1-gcc"; > > + reg = <0x1000 0x80>;> + clocks = <&dispcc 0x1> > > + #clock-cells = <1>; > > + post-init-supplier = <&dispcc>; > > + }; > > + dispcc: display-clock-controller@2000 { > > clock-controller@2000 > > > + compatible = "vendor,soc4-dispcc", "vendor,soc1-dispcc"; > > + reg = <0x2000 0x80>; > > + clocks = <&gcc 0xdd> > > + #clock-cells = <1>; > > + }; > > diff --git a/MAINTAINERS b/MAINTAINERS > > index 3dfe7ea25320..40fd498543a5 100644 > > --- a/MAINTAINERS > > +++ b/MAINTAINERS > > @@ -6055,10 +6055,11 @@ S: Maintained > > F: drivers/base/devcoredump.c > > F: include/linux/devcoredump.h > > > > -DEVICE DEPENDENCY HELPER SCRIPT > > +FIRMWARE DEVICE LINK (fw_devlink) > > M: Saravana Kannan <saravanak@google.com> > > L: linux-kernel@vger.kernel.org > > S: Maintained > > +F: Documentation/devicetree/bindings/post-init-supplier.yaml > > F: scripts/dev-needs.sh > > > > DEVICE DIRECT ACCESS (DAX) > > -- > > 2.43.0.687.g38aa6559b0-goog > >
On Sun, Feb 11, 2024 at 6:52 AM Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> wrote: > > On 10/02/2024 04:05, Saravana Kannan wrote: > > The post-init-supplier property can be used to break a dependency cycle by > > marking some supplier(s) as a post device initialization supplier(s). This > > allows the kernel to do a better job at ordering initialization and > > suspend/resume of the devices in a dependency cycle. > > ... > > > diff --git a/MAINTAINERS b/MAINTAINERS > > index 3dfe7ea25320..40fd498543a5 100644 > > --- a/MAINTAINERS > > +++ b/MAINTAINERS > > @@ -6055,10 +6055,11 @@ S: Maintained > > F: drivers/base/devcoredump.c > > F: include/linux/devcoredump.h > > > > -DEVICE DEPENDENCY HELPER SCRIPT > > +FIRMWARE DEVICE LINK (fw_devlink) > > This breaks ordering of MAINTAINERS... Oops. Thanks. Will fix it. -Saravana
diff --git a/Documentation/devicetree/bindings/post-init-supplier.yaml b/Documentation/devicetree/bindings/post-init-supplier.yaml new file mode 100644 index 000000000000..cf9071ecd06e --- /dev/null +++ b/Documentation/devicetree/bindings/post-init-supplier.yaml @@ -0,0 +1,99 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +# Copyright 2018 Linaro Ltd. +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/post-init-supplier.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Post device initialization supplier + +maintainers: + - Saravana Kannan <saravanak@google.com> + +description: | + This property is used to indicate that the device(s) pointed to by the + property are not needed for the initialization of the device that lists this + property. + + A device can list its suppliers in devicetree using one or more of the + standard devicetree bindings. By default, it would be safe to assume the + supplier device can be initialized before the consumer device is initialized. + + However, that assumption cannot be made when there are cyclic dependecies + between devices. Since each device is a supplier (directly or indirectly) of + the others in the cycle, there is no guaranteed safe order for initalizing + the devices in a cycle. We can try to initialize them in an arbitrary order + and eventually successfully initialize all of them, but that doesn't always + work well. + + For example, say, + * The device tree has the following cyclic dependency X -> Y -> Z -> X (where + -> denotes "depends on"). + * But X is not needed to fully initialize Z (X might be needed only when a + specific functionality if requested post initialization). + + If all the other -> are mandatory initialization dependencies, then trying to + initialize the devices in a loop (or arbitrarily) will always eventually end + up with the devices being initialized in the order Z, Y and X. + + However, if Y is an optional supplier for X (where X provides limited + functionality when Y is not initialized and providing its services), then + trying to initialize the devices in a loop (or arbitrarily) could end up with + the devices being initialized in the following order: + + * Z, Y and X - All devices provide full functionality + * Z, X and Y - X provides partial functionality + * X, Z and Y - X provides partial functionality + + However, we always want to initialize the devices in the order Z, Y and X + since that provides the full functionality without interruptions. + + One alternate option that might be suggested is to have the driver for X + notice that Y became available at a later point and adjust the functionality + it provides. However, other userspace applications could have started using X + with the limited functionality before Y was available and it might not be + possible to transparently transition X or the users of X to full + functionality while X is in use. + + Similarly, when it comes to suspend (resume) ordering, it's unclear which + device in a dependency cycle needs to be suspended/resumed first and trying + arbitrary orders can result in system crashes or instability. + + Explicitly calling out which link in a cycle needs to be broken when + determining the order, simplifies things a lot, improves efficiency, makes + the behavior more deterministic and maximizes the functionality that can be + provided without interruption. + + This property is used to provide this additional information between devices + in a cycle by telling which supplier(s) is not needed for initializing the + device that lists this property. + + In the example above, Z would list X as a post-init-supplier and the + initialization dependency would become X -> Y -> Z -/-> X. So the best order + to initialize them become clear: Z, Y and then X. + +properties: + # A dictionary of DT properties for this binding schema + post-init-supplier: + # One or more suppliers can be marked as post initialization supplier + minItems: 1 + description: + List of phandles to suppliers that are not needed for initializing or + resuming this device. + $ref: /schemas/types.yaml#/definitions/phandle + +examples: + - | + gcc: general-clock-controller@1000 { + compatible = "vendor,soc4-gcc", "vendor,soc1-gcc"; + reg = <0x1000 0x80>; + clocks = <&dispcc 0x1> + #clock-cells = <1>; + post-init-supplier = <&dispcc>; + }; + dispcc: display-clock-controller@2000 { + compatible = "vendor,soc4-dispcc", "vendor,soc1-dispcc"; + reg = <0x2000 0x80>; + clocks = <&gcc 0xdd> + #clock-cells = <1>; + }; diff --git a/MAINTAINERS b/MAINTAINERS index 3dfe7ea25320..40fd498543a5 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -6055,10 +6055,11 @@ S: Maintained F: drivers/base/devcoredump.c F: include/linux/devcoredump.h -DEVICE DEPENDENCY HELPER SCRIPT +FIRMWARE DEVICE LINK (fw_devlink) M: Saravana Kannan <saravanak@google.com> L: linux-kernel@vger.kernel.org S: Maintained +F: Documentation/devicetree/bindings/post-init-supplier.yaml F: scripts/dev-needs.sh DEVICE DIRECT ACCESS (DAX)
The post-init-supplier property can be used to break a dependency cycle by marking some supplier(s) as a post device initialization supplier(s). This allows the kernel to do a better job at ordering initialization and suspend/resume of the devices in a dependency cycle. Signed-off-by: Saravana Kannan <saravanak@google.com> --- .../bindings/post-init-supplier.yaml | 99 +++++++++++++++++++ MAINTAINERS | 3 +- 2 files changed, 101 insertions(+), 1 deletion(-) create mode 100644 Documentation/devicetree/bindings/post-init-supplier.yaml