| Message ID | 20240210030549.4048795-1-saravanak@google.com |
|---|---|
| Headers | show |
| Series | Add post-init-supplier binding to improve suspend/resume stability | expand |
On Sat, Feb 10, 2024 at 3:06 AM Saravana Kannan <saravanak@google.com> wrote: > > The post-init-supplier property can be used to break a dependency cycle by > marking some supplier(s) as a post device initialization supplier(s). This > allows the kernel to do a better job at ordering initialization and s/the kernel/an OS/ > suspend/resume of the devices in a dependency cycle. > > Signed-off-by: Saravana Kannan <saravanak@google.com> > --- > .../bindings/post-init-supplier.yaml | 99 +++++++++++++++++++ This should probably go into dtschema instead, but fine here now for reviewing. We have to consider if this property should be automatically allowed on any node or nodes with specific suppliers, or if it should be explicit for users. The former needs tool support. I'm leaning towards the latter as I want to know when this is needed. > MAINTAINERS | 3 +- > 2 files changed, 101 insertions(+), 1 deletion(-) > create mode 100644 Documentation/devicetree/bindings/post-init-supplier.yaml > > diff --git a/Documentation/devicetree/bindings/post-init-supplier.yaml b/Documentation/devicetree/bindings/post-init-supplier.yaml > new file mode 100644 > index 000000000000..cf9071ecd06e > --- /dev/null > +++ b/Documentation/devicetree/bindings/post-init-supplier.yaml > @@ -0,0 +1,99 @@ > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) > +# Copyright 2018 Linaro Ltd. ? > +%YAML 1.2 > +--- > +$id: http://devicetree.org/schemas/post-init-supplier.yaml# > +$schema: http://devicetree.org/meta-schemas/core.yaml# > + > +title: Post device initialization supplier > + > +maintainers: > + - Saravana Kannan <saravanak@google.com> > + > +description: | > + This property is used to indicate that the device(s) pointed to by the > + property are not needed for the initialization of the device that lists this > + property. > + > + A device can list its suppliers in devicetree using one or more of the > + standard devicetree bindings. By default, it would be safe to assume the > + supplier device can be initialized before the consumer device is initialized. > + > + However, that assumption cannot be made when there are cyclic dependecies typo > + between devices. Since each device is a supplier (directly or indirectly) of > + the others in the cycle, there is no guaranteed safe order for initalizing typo > + the devices in a cycle. We can try to initialize them in an arbitrary order > + and eventually successfully initialize all of them, but that doesn't always > + work well. > + > + For example, say, > + * The device tree has the following cyclic dependency X -> Y -> Z -> X (where > + -> denotes "depends on"). > + * But X is not needed to fully initialize Z (X might be needed only when a > + specific functionality if requested post initialization). > + > + If all the other -> are mandatory initialization dependencies, then trying to > + initialize the devices in a loop (or arbitrarily) will always eventually end > + up with the devices being initialized in the order Z, Y and X. > + > + However, if Y is an optional supplier for X (where X provides limited > + functionality when Y is not initialized and providing its services), then > + trying to initialize the devices in a loop (or arbitrarily) could end up with > + the devices being initialized in the following order: > + > + * Z, Y and X - All devices provide full functionality > + * Z, X and Y - X provides partial functionality > + * X, Z and Y - X provides partial functionality > + > + However, we always want to initialize the devices in the order Z, Y and X > + since that provides the full functionality without interruptions. > + > + One alternate option that might be suggested is to have the driver for X > + notice that Y became available at a later point and adjust the functionality > + it provides. However, other userspace applications could have started using X > + with the limited functionality before Y was available and it might not be > + possible to transparently transition X or the users of X to full > + functionality while X is in use. > + > + Similarly, when it comes to suspend (resume) ordering, it's unclear which > + device in a dependency cycle needs to be suspended/resumed first and trying > + arbitrary orders can result in system crashes or instability. > + > + Explicitly calling out which link in a cycle needs to be broken when > + determining the order, simplifies things a lot, improves efficiency, makes > + the behavior more deterministic and maximizes the functionality that can be > + provided without interruption. > + > + This property is used to provide this additional information between devices > + in a cycle by telling which supplier(s) is not needed for initializing the > + device that lists this property. > + > + In the example above, Z would list X as a post-init-supplier and the > + initialization dependency would become X -> Y -> Z -/-> X. So the best order > + to initialize them become clear: Z, Y and then X. > + select: true Otherwise, this is never applied. > +properties: > + # A dictionary of DT properties for this binding schema Drop > + post-init-supplier: > + # One or more suppliers can be marked as post initialization supplier > + minItems: 1 That's the default. > + description: > + List of phandles to suppliers that are not needed for initializing or > + resuming this device. > + $ref: /schemas/types.yaml#/definitions/phandle Should be phandle-array plus: items: maxItems: 1 (as each entry is a single phandle) > + > +examples: > + - | > + gcc: general-clock-controller@1000 { clock-controller@1000 > + compatible = "vendor,soc4-gcc", "vendor,soc1-gcc"; > + reg = <0x1000 0x80>;> + clocks = <&dispcc 0x1> > + #clock-cells = <1>; > + post-init-supplier = <&dispcc>; > + }; > + dispcc: display-clock-controller@2000 { clock-controller@2000 > + compatible = "vendor,soc4-dispcc", "vendor,soc1-dispcc"; > + reg = <0x2000 0x80>; > + clocks = <&gcc 0xdd> > + #clock-cells = <1>; > + }; > diff --git a/MAINTAINERS b/MAINTAINERS > index 3dfe7ea25320..40fd498543a5 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -6055,10 +6055,11 @@ S: Maintained > F: drivers/base/devcoredump.c > F: include/linux/devcoredump.h > > -DEVICE DEPENDENCY HELPER SCRIPT > +FIRMWARE DEVICE LINK (fw_devlink) > M: Saravana Kannan <saravanak@google.com> > L: linux-kernel@vger.kernel.org > S: Maintained > +F: Documentation/devicetree/bindings/post-init-supplier.yaml > F: scripts/dev-needs.sh > > DEVICE DIRECT ACCESS (DAX) > -- > 2.43.0.687.g38aa6559b0-goog >
On 10/02/2024 04:05, Saravana Kannan wrote: > The post-init-supplier property can be used to break a dependency cycle by > marking some supplier(s) as a post device initialization supplier(s). This > allows the kernel to do a better job at ordering initialization and > suspend/resume of the devices in a dependency cycle. ... > diff --git a/MAINTAINERS b/MAINTAINERS > index 3dfe7ea25320..40fd498543a5 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -6055,10 +6055,11 @@ S: Maintained > F: drivers/base/devcoredump.c > F: include/linux/devcoredump.h > > -DEVICE DEPENDENCY HELPER SCRIPT > +FIRMWARE DEVICE LINK (fw_devlink) This breaks ordering of MAINTAINERS... Best regards, Krzysztof
On Sat, Feb 10, 2024 at 3:26 AM Rob Herring <robh+dt@kernel.org> wrote: > > On Sat, Feb 10, 2024 at 3:06 AM Saravana Kannan <saravanak@google.com> wrote: > > > > The post-init-supplier property can be used to break a dependency cycle by > > marking some supplier(s) as a post device initialization supplier(s). This > > allows the kernel to do a better job at ordering initialization and > > s/the kernel/an OS/ > > > suspend/resume of the devices in a dependency cycle. > > > > Signed-off-by: Saravana Kannan <saravanak@google.com> > > --- > > .../bindings/post-init-supplier.yaml | 99 +++++++++++++++++++ > > This should probably go into dtschema instead, but fine here now for reviewing. How do I submit it for dtschema? > We have to consider if this property should be automatically allowed > on any node or nodes with specific suppliers, or if it should be > explicit for users. The former needs tool support. I'm leaning towards > the latter as I want to know when this is needed. I think I understand your question. This property can be relevant for any device -- it really depends on the device's consumers and suppliers and if they form a cycle. Ideally, we can check if a device is in a cycle and then allow it only for those, but that's going to be too complicated I think. So, I think this should be allowed for any device. Since this property is defined as a "post-init-supplier" and only applies to an existing direct supplier, there's not a lot of room for misuse. Actually if you can add a check "are all the phandles listed in this property also present in one of the other properties in this device node", that should be sufficient. I'll also edit the documentation to make it clear it's meaningful only for an existing supplier. > > > MAINTAINERS | 3 +- > > 2 files changed, 101 insertions(+), 1 deletion(-) > > create mode 100644 Documentation/devicetree/bindings/post-init-supplier.yaml > > > > diff --git a/Documentation/devicetree/bindings/post-init-supplier.yaml b/Documentation/devicetree/bindings/post-init-supplier.yaml > > new file mode 100644 > > index 000000000000..cf9071ecd06e > > --- /dev/null > > +++ b/Documentation/devicetree/bindings/post-init-supplier.yaml > > @@ -0,0 +1,99 @@ > > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) > > +# Copyright 2018 Linaro Ltd. > > ? Oops... didn't edit out all the lines in the example. I'll address all your other comments in the next patch. Btw, this is the command I used to validate this file (before I sent v1), but it didn't spit out anything meaningful that I could understand: make dt_binding_check DT_CHECKER_FLAGS=-m DT_SCHEMA_FILES=Documentation/devicetree/bindings/trivial-devices.yaml Is that the right way to run it? I'm using yamllint 1.32. So it's not terribly old either (can't install anything newer). -Saravana > > > +%YAML 1.2 > > +--- > > +$id: http://devicetree.org/schemas/post-init-supplier.yaml# > > +$schema: http://devicetree.org/meta-schemas/core.yaml# > > + > > +title: Post device initialization supplier > > + > > +maintainers: > > + - Saravana Kannan <saravanak@google.com> > > + > > +description: | > > + This property is used to indicate that the device(s) pointed to by the > > + property are not needed for the initialization of the device that lists this > > + property. > > + > > + A device can list its suppliers in devicetree using one or more of the > > + standard devicetree bindings. By default, it would be safe to assume the > > + supplier device can be initialized before the consumer device is initialized. > > + > > + However, that assumption cannot be made when there are cyclic dependecies > > typo > > > + between devices. Since each device is a supplier (directly or indirectly) of > > + the others in the cycle, there is no guaranteed safe order for initalizing > > typo > > > + the devices in a cycle. We can try to initialize them in an arbitrary order > > + and eventually successfully initialize all of them, but that doesn't always > > + work well. > > + > > + For example, say, > > + * The device tree has the following cyclic dependency X -> Y -> Z -> X (where > > + -> denotes "depends on"). > > + * But X is not needed to fully initialize Z (X might be needed only when a > > + specific functionality if requested post initialization). > > + > > + If all the other -> are mandatory initialization dependencies, then trying to > > + initialize the devices in a loop (or arbitrarily) will always eventually end > > + up with the devices being initialized in the order Z, Y and X. > > + > > + However, if Y is an optional supplier for X (where X provides limited > > + functionality when Y is not initialized and providing its services), then > > + trying to initialize the devices in a loop (or arbitrarily) could end up with > > + the devices being initialized in the following order: > > + > > + * Z, Y and X - All devices provide full functionality > > + * Z, X and Y - X provides partial functionality > > + * X, Z and Y - X provides partial functionality > > + > > + However, we always want to initialize the devices in the order Z, Y and X > > + since that provides the full functionality without interruptions. > > + > > + One alternate option that might be suggested is to have the driver for X > > + notice that Y became available at a later point and adjust the functionality > > + it provides. However, other userspace applications could have started using X > > + with the limited functionality before Y was available and it might not be > > + possible to transparently transition X or the users of X to full > > + functionality while X is in use. > > + > > + Similarly, when it comes to suspend (resume) ordering, it's unclear which > > + device in a dependency cycle needs to be suspended/resumed first and trying > > + arbitrary orders can result in system crashes or instability. > > + > > + Explicitly calling out which link in a cycle needs to be broken when > > + determining the order, simplifies things a lot, improves efficiency, makes > > + the behavior more deterministic and maximizes the functionality that can be > > + provided without interruption. > > + > > + This property is used to provide this additional information between devices > > + in a cycle by telling which supplier(s) is not needed for initializing the > > + device that lists this property. > > + > > + In the example above, Z would list X as a post-init-supplier and the > > + initialization dependency would become X -> Y -> Z -/-> X. So the best order > > + to initialize them become clear: Z, Y and then X. > > + > > select: true > > Otherwise, this is never applied. > > > +properties: > > + # A dictionary of DT properties for this binding schema > > Drop > > > + post-init-supplier: > > + # One or more suppliers can be marked as post initialization supplier > > + minItems: 1 > > That's the default. > > > + description: > > + List of phandles to suppliers that are not needed for initializing or > > + resuming this device. > > + $ref: /schemas/types.yaml#/definitions/phandle > > Should be phandle-array plus: > > items: > maxItems: 1 > > (as each entry is a single phandle) > > > + > > +examples: > > + - | > > + gcc: general-clock-controller@1000 { > > clock-controller@1000 > > > + compatible = "vendor,soc4-gcc", "vendor,soc1-gcc"; > > + reg = <0x1000 0x80>;> + clocks = <&dispcc 0x1> > > + #clock-cells = <1>; > > + post-init-supplier = <&dispcc>; > > + }; > > + dispcc: display-clock-controller@2000 { > > clock-controller@2000 > > > + compatible = "vendor,soc4-dispcc", "vendor,soc1-dispcc"; > > + reg = <0x2000 0x80>; > > + clocks = <&gcc 0xdd> > > + #clock-cells = <1>; > > + }; > > diff --git a/MAINTAINERS b/MAINTAINERS > > index 3dfe7ea25320..40fd498543a5 100644 > > --- a/MAINTAINERS > > +++ b/MAINTAINERS > > @@ -6055,10 +6055,11 @@ S: Maintained > > F: drivers/base/devcoredump.c > > F: include/linux/devcoredump.h > > > > -DEVICE DEPENDENCY HELPER SCRIPT > > +FIRMWARE DEVICE LINK (fw_devlink) > > M: Saravana Kannan <saravanak@google.com> > > L: linux-kernel@vger.kernel.org > > S: Maintained > > +F: Documentation/devicetree/bindings/post-init-supplier.yaml > > F: scripts/dev-needs.sh > > > > DEVICE DIRECT ACCESS (DAX) > > -- > > 2.43.0.687.g38aa6559b0-goog > >
On Sun, Feb 11, 2024 at 6:52 AM Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> wrote: > > On 10/02/2024 04:05, Saravana Kannan wrote: > > The post-init-supplier property can be used to break a dependency cycle by > > marking some supplier(s) as a post device initialization supplier(s). This > > allows the kernel to do a better job at ordering initialization and > > suspend/resume of the devices in a dependency cycle. > > ... > > > diff --git a/MAINTAINERS b/MAINTAINERS > > index 3dfe7ea25320..40fd498543a5 100644 > > --- a/MAINTAINERS > > +++ b/MAINTAINERS > > @@ -6055,10 +6055,11 @@ S: Maintained > > F: drivers/base/devcoredump.c > > F: include/linux/devcoredump.h > > > > -DEVICE DEPENDENCY HELPER SCRIPT > > +FIRMWARE DEVICE LINK (fw_devlink) > > This breaks ordering of MAINTAINERS... Oops. Thanks. Will fix it. -Saravana
This patch series adds a "post-init-supplier" device tree binding that can be used to break dependency cycles in device tree and enforce a more determinstic probe/suspend/resume order. This will also improve the stability of global async probing and async suspend/resume and allow us to enable them more easily. Yet another step away from playing initcall chicken with probing and step towards fully async probing and suspend/resume. Patch 3 (the binding docunentation) provide a lot more details and examples. Saravana Kannan (4): driver core: Adds flags param to fwnode_link_add() driver core: Add FWLINK_FLAG_IGNORE to completely ignore a fwnode link dt-bindings: Add post-init-supplier property of: property: fw_devlink: Add support for "post-init-supplier" property .../bindings/post-init-supplier.yaml | 99 +++++++++++++++++++ MAINTAINERS | 3 +- drivers/base/core.c | 14 ++- drivers/firmware/efi/sysfb_efi.c | 2 +- drivers/of/property.c | 17 +++- include/linux/fwnode.h | 5 +- 6 files changed, 131 insertions(+), 9 deletions(-) create mode 100644 Documentation/devicetree/bindings/post-init-supplier.yaml