Message ID | 20190723051446.20013-1-bjorn.andersson@linaro.org |
---|---|
State | New |
Headers | show |
Series | [RFC] clk: Remove cached cores in parent map during unregister | expand |
Quoting Bjorn Andersson (2019-07-22 22:14:46) > As clocks are registered their parents are resolved and the parent_map > is updated to cache the clk_core objects of each existing parent. > But in the event of a clock being unregistered this cache will carry > dangling pointers if not invalidated, so do this for all children of the > clock being unregistered. > > Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org> > --- > > This resolves the issue seen where the DSI PLL (and it's provided clocks) is > being registered and unregistered multiple times due to probe deferral. > > Marking it RFC because I don't fully understand the life of the clock yet. The concept sounds sane but the implementation is going to be not much fun. The problem is that a clk can be in many different parent caches, even ones for clks that aren't currently parented to it. We would need to walk the entire tree(s) and find anywhere that we've cached the clk_core pointer and invalidate it. Maybe we can speed that up a little bit by keeping a reference to the entry of each parent cache that is for the parent we're removing, essentially holding an inverse cache, but I'm not sure it will provide any benefit besides wasting space for this one operation that we shouldn't be doing very often if at all. It certainly sounds easier to iterate through the whole tree and just invalidate entries in all the caches under the prepare lock. We can optimize it later. > > drivers/clk/clk.c | 18 +++++++++++++++--- > 1 file changed, 15 insertions(+), 3 deletions(-) > > diff --git a/drivers/clk/clk.c b/drivers/clk/clk.c > index c0990703ce54..8cd1ad977c50 100644 > --- a/drivers/clk/clk.c > +++ b/drivers/clk/clk.c > @@ -2517,7 +2528,8 @@ int clk_set_parent(struct clk *clk, struct clk *parent) > clk_core_rate_unprotect(clk->core); > > ret = clk_core_set_parent_nolock(clk->core, > - parent ? parent->core : NULL); > + parent ? parent->core : NULL, > + false); Nitpick. I'd prefer another function to invalidate the cache of a clk given another clk_core pointer. > > if (clk->exclusive_count) > clk_core_rate_protect(clk->core);
Quoting Stephen Boyd (2019-07-29 15:46:51) > Quoting Bjorn Andersson (2019-07-22 22:14:46) > > As clocks are registered their parents are resolved and the parent_map > > is updated to cache the clk_core objects of each existing parent. > > But in the event of a clock being unregistered this cache will carry > > dangling pointers if not invalidated, so do this for all children of the > > clock being unregistered. > > > > Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org> > > --- > > > > This resolves the issue seen where the DSI PLL (and it's provided clocks) is > > being registered and unregistered multiple times due to probe deferral. > > > > Marking it RFC because I don't fully understand the life of the clock yet. > > The concept sounds sane but the implementation is going to be not much > fun. The problem is that a clk can be in many different parent caches, > even ones for clks that aren't currently parented to it. We would need > to walk the entire tree(s) and find anywhere that we've cached the > clk_core pointer and invalidate it. Maybe we can speed that up a little > bit by keeping a reference to the entry of each parent cache that is for > the parent we're removing, essentially holding an inverse cache, but I'm > not sure it will provide any benefit besides wasting space for this one > operation that we shouldn't be doing very often if at all. > > It certainly sounds easier to iterate through the whole tree and just > invalidate entries in all the caches under the prepare lock. We can > optimize it later. Here's an attempt at the simple approach. There's another problem where the cached 'hw' member of the parent data is held around when we don't know when the caller has destroyed it. Not much else we can do for that though. ---8<--- diff --git a/drivers/clk/clk.c b/drivers/clk/clk.c index c0990703ce54..f42a803fb11a 100644 --- a/drivers/clk/clk.c +++ b/drivers/clk/clk.c @@ -3737,6 +3737,37 @@ static const struct clk_ops clk_nodrv_ops = { .set_parent = clk_nodrv_set_parent, }; +static void clk_core_evict_parent_cache_subtree(struct clk_core *root, + struct clk_core *target) +{ + int i; + struct clk_core *child; + + if (!root) + return; + + for (i = 0; i < root->num_parents; i++) + if (root->parents[i].core == target) + root->parents[i].core = NULL; + + hlist_for_each_entry(child, &root->children, child_node) + clk_core_evict_parent_cache_subtree(child, target); +} + +/* Remove this clk from all parent caches */ +static void clk_core_evict_parent_cache(struct clk_core *core) +{ + struct hlist_head **lists; + struct clk_core *root; + + lockdep_assert_held(&prepare_lock); + + for (lists = all_lists; *lists; lists++) + hlist_for_each_entry(root, *lists, child_node) + clk_core_evict_parent_cache_subtree(root, core); + +} + /** * clk_unregister - unregister a currently registered clock * @clk: clock to unregister @@ -3775,6 +3806,8 @@ void clk_unregister(struct clk *clk) clk_core_set_parent_nolock(child, NULL); } + clk_core_evict_parent_cache(clk->core); + hlist_del_init(&clk->core->child_node); if (clk->core->prepare_count)
Hi Stephen, On 2019-08-21 18:10, Stephen Boyd wrote: > > Here's an attempt at the simple approach. There's another problem where > the cached 'hw' member of the parent data is held around when we don't > know when the caller has destroyed it. Not much else we can do for that > though. > > ---8<--- > diff --git a/drivers/clk/clk.c b/drivers/clk/clk.c > index c0990703ce54..f42a803fb11a 100644 > --- a/drivers/clk/clk.c > +++ b/drivers/clk/clk.c > @@ -3737,6 +3737,37 @@ static const struct clk_ops clk_nodrv_ops = { > .set_parent = clk_nodrv_set_parent, > }; > > +static void clk_core_evict_parent_cache_subtree(struct clk_core *root, > + struct clk_core *target) > +{ > + int i; > + struct clk_core *child; > + > + if (!root) > + return; > + > + for (i = 0; i < root->num_parents; i++) > + if (root->parents[i].core == target) > + root->parents[i].core = NULL; > + > + hlist_for_each_entry(child, &root->children, child_node) > + clk_core_evict_parent_cache_subtree(child, target); > +} > + > +/* Remove this clk from all parent caches */ > +static void clk_core_evict_parent_cache(struct clk_core *core) > +{ > + struct hlist_head **lists; > + struct clk_core *root; > + > + lockdep_assert_held(&prepare_lock); > + > + for (lists = all_lists; *lists; lists++) > + hlist_for_each_entry(root, *lists, child_node) > + clk_core_evict_parent_cache_subtree(root, core); > + > +} > + > /** > * clk_unregister - unregister a currently registered clock > * @clk: clock to unregister > @@ -3775,6 +3806,8 @@ void clk_unregister(struct clk *clk) > clk_core_set_parent_nolock(child, NULL); > } > > + clk_core_evict_parent_cache(clk->core); > + > hlist_del_init(&clk->core->child_node); > > if (clk->core->prepare_count) Thanks for the patch. This fixes the dispcc issue which I was observing with latest kernel on cheza. You can have the tested by when you post the actual patch. Tested-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org> For reference, below is the error log: [ 3.992243] msm ae00000.mdss: Adding to iommu group 1 [ 3.999877] msm ae00000.mdss: bound ae01000.mdp (ops 0xffffff8010a87b78) [ 4.023597] msm ae00000.mdss: failed to bind ae94000.dsi (ops 0xffffff8010a8b360): -517 [ 4.031898] msm ae00000.mdss: master bind failed: -517 [ 5.297509] msm ae00000.mdss: bound ae01000.mdp (ops 0xffffff8010a87b78) [ 5.320906] msm ae00000.mdss: failed to bind ae94000.dsi (ops 0xffffff8010a8b360): -517 [ 5.329261] msm ae00000.mdss: master bind failed: -517 [ 5.648667] msm ae00000.mdss: bound ae01000.mdp (ops 0xffffff8010a87b78) [ 5.672005] clk_core_set_parent_nolock: clk dsi0_phy_pll_out_byteclk can not be parent of clk disp_cc_mdss_byte0_clk_src p_index=-22 <--- error [ 5.699799] msm ae00000.mdss: failed to bind ae94000.dsi (ops 0xffffff8010a8b360): -22 [ 5.708086] msm ae00000.mdss: master bind failed: -22 [ 5.714052] msm: probe of ae00000.mdss failed with error -22 [ 6.033541] clk_core_set_parent_nolock: clk dsi0_phy_pll_out_dsiclk can not be parent of clk disp_cc_mdss_pclk0_clk_src p_index=-22 <--- error Thanks, Sai -- QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Quoting Stephen Boyd (2019-08-21 11:10:08) > Quoting Stephen Boyd (2019-07-29 15:46:51) > > Quoting Bjorn Andersson (2019-07-22 22:14:46) > > > As clocks are registered their parents are resolved and the parent_map > > > is updated to cache the clk_core objects of each existing parent. > > > But in the event of a clock being unregistered this cache will carry > > > dangling pointers if not invalidated, so do this for all children of the > > > clock being unregistered. > > > > > > Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org> > > > --- > > > > > > This resolves the issue seen where the DSI PLL (and it's provided clocks) is > > > being registered and unregistered multiple times due to probe deferral. > > > > > > Marking it RFC because I don't fully understand the life of the clock yet. > > > > The concept sounds sane but the implementation is going to be not much > > fun. The problem is that a clk can be in many different parent caches, > > even ones for clks that aren't currently parented to it. We would need > > to walk the entire tree(s) and find anywhere that we've cached the > > clk_core pointer and invalidate it. Maybe we can speed that up a little > > bit by keeping a reference to the entry of each parent cache that is for > > the parent we're removing, essentially holding an inverse cache, but I'm > > not sure it will provide any benefit besides wasting space for this one > > operation that we shouldn't be doing very often if at all. > > > > It certainly sounds easier to iterate through the whole tree and just > > invalidate entries in all the caches under the prepare lock. We can > > optimize it later. > > Here's an attempt at the simple approach. There's another problem where > the cached 'hw' member of the parent data is held around when we don't > know when the caller has destroyed it. Not much else we can do for that > though. > > ---8<--- > diff --git a/drivers/clk/clk.c b/drivers/clk/clk.c > index c0990703ce54..f42a803fb11a 100644 > --- a/drivers/clk/clk.c > +++ b/drivers/clk/clk.c > @@ -3737,6 +3737,37 @@ static const struct clk_ops clk_nodrv_ops = { > .set_parent = clk_nodrv_set_parent, > }; > > +static void clk_core_evict_parent_cache_subtree(struct clk_core *root, > + struct clk_core *target) > +{ > + int i; > + struct clk_core *child; > + > + if (!root) > + return; I don't think we need this part. Child is always a valid pointer.
On Mon, Aug 26, 2019 at 02:24:14PM -0700, Stephen Boyd wrote: > > > > ---8<--- > > diff --git a/drivers/clk/clk.c b/drivers/clk/clk.c > > index c0990703ce54..f42a803fb11a 100644 > > --- a/drivers/clk/clk.c > > +++ b/drivers/clk/clk.c > > @@ -3737,6 +3737,37 @@ static const struct clk_ops clk_nodrv_ops = { > > .set_parent = clk_nodrv_set_parent, > > }; > > > > +static void clk_core_evict_parent_cache_subtree(struct clk_core *root, > > + struct clk_core *target) > > +{ > > + int i; > > + struct clk_core *child; > > + > > + if (!root) > > + return; > > I don't think we need this part. Child is always a valid pointer. > Bjorn or Saiprakash Are there any plans to send out Stephen's proposed patch? Thanks
diff --git a/drivers/clk/clk.c b/drivers/clk/clk.c index c0990703ce54..8cd1ad977c50 100644 --- a/drivers/clk/clk.c +++ b/drivers/clk/clk.c @@ -2423,11 +2423,14 @@ bool clk_has_parent(struct clk *clk, struct clk *parent) EXPORT_SYMBOL_GPL(clk_has_parent); static int clk_core_set_parent_nolock(struct clk_core *core, - struct clk_core *parent) + struct clk_core *parent, + bool invalidate_parent) { + struct clk_core *old_parent = core->parent; int ret = 0; int p_index = 0; unsigned long p_rate = 0; + int i; lockdep_assert_held(&prepare_lock); @@ -2481,6 +2484,14 @@ static int clk_core_set_parent_nolock(struct clk_core *core, __clk_recalc_accuracies(core); } + /* invalidate the parent cache */ + if (!parent && invalidate_parent) { + for (i = 0; i < core->num_parents; i++) { + if (core->parents[i].core == old_parent) + core->parents[i].core = NULL; + } + } + runtime_put: clk_pm_runtime_put(core); @@ -2517,7 +2528,8 @@ int clk_set_parent(struct clk *clk, struct clk *parent) clk_core_rate_unprotect(clk->core); ret = clk_core_set_parent_nolock(clk->core, - parent ? parent->core : NULL); + parent ? parent->core : NULL, + false); if (clk->exclusive_count) clk_core_rate_protect(clk->core); @@ -3772,7 +3784,7 @@ void clk_unregister(struct clk *clk) /* Reparent all children to the orphan list. */ hlist_for_each_entry_safe(child, t, &clk->core->children, child_node) - clk_core_set_parent_nolock(child, NULL); + clk_core_set_parent_nolock(child, NULL, true); } hlist_del_init(&clk->core->child_node);
As clocks are registered their parents are resolved and the parent_map is updated to cache the clk_core objects of each existing parent. But in the event of a clock being unregistered this cache will carry dangling pointers if not invalidated, so do this for all children of the clock being unregistered. Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org> --- This resolves the issue seen where the DSI PLL (and it's provided clocks) is being registered and unregistered multiple times due to probe deferral. Marking it RFC because I don't fully understand the life of the clock yet. drivers/clk/clk.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) -- 2.18.0