| Message ID | b38dda39bf6cd18df5fab86f7c7cc86a6979786f.1397492345.git.viresh.kumar@linaro.org |
|---|---|
| State | New |
| Headers | show |
On Mon, Apr 14, 2014 at 09:53:51PM +0530, Viresh Kumar wrote: > __tick_nohz_task_switch() was called only from tick_nohz_task_switch() and there > is nothing much in tick_nohz_task_switch() as well. IOW, we don't need > unnecessary wrapper over __tick_nohz_task_switch() to be there. Merge all code > from __tick_nohz_task_switch() into tick_nohz_task_switch() and move it to > tick-sched.c. > > This also moves check for tick_nohz_tick_stopped() outside of irq_save() > context. No, the wrapper is there on purpose in order to optimize the full dynticks off case in the context switch path with the jump label'ed check on tick_nohz_full_enabled(). -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
On 15 April 2014 04:52, Frederic Weisbecker <fweisbec@gmail.com> wrote: > On Mon, Apr 14, 2014 at 09:53:51PM +0530, Viresh Kumar wrote: >> __tick_nohz_task_switch() was called only from tick_nohz_task_switch() and there >> is nothing much in tick_nohz_task_switch() as well. IOW, we don't need >> unnecessary wrapper over __tick_nohz_task_switch() to be there. Merge all code >> from __tick_nohz_task_switch() into tick_nohz_task_switch() and move it to >> tick-sched.c. >> >> This also moves check for tick_nohz_tick_stopped() outside of irq_save() >> context. > > No, the wrapper is there on purpose in order to optimize the full dynticks off case in > the context switch path with the jump label'ed check on tick_nohz_full_enabled(). Just to clarify, you are saying that: Wrapper was there to save an extra function call when tick_nohz_full_enabled() returns false, as tick_nohz_task_switch() will be inlined ? In this case probably we can move !can_stop_full_tick() as well to the wrapper ? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
On Tue, Apr 15, 2014 at 10:15:24AM +0530, Viresh Kumar wrote: > On 15 April 2014 04:52, Frederic Weisbecker <fweisbec@gmail.com> wrote: > > On Mon, Apr 14, 2014 at 09:53:51PM +0530, Viresh Kumar wrote: > >> __tick_nohz_task_switch() was called only from tick_nohz_task_switch() and there > >> is nothing much in tick_nohz_task_switch() as well. IOW, we don't need > >> unnecessary wrapper over __tick_nohz_task_switch() to be there. Merge all code > >> from __tick_nohz_task_switch() into tick_nohz_task_switch() and move it to > >> tick-sched.c. > >> > >> This also moves check for tick_nohz_tick_stopped() outside of irq_save() > >> context. > > > > No, the wrapper is there on purpose in order to optimize the full dynticks off case in > > the context switch path with the jump label'ed check on tick_nohz_full_enabled(). > > Just to clarify, you are saying that: > > Wrapper was there to save an extra function call when tick_nohz_full_enabled() > returns false, as tick_nohz_task_switch() will be inlined ? Yeah. But not just that. Using an inline saves a function call and reduce the offline case to a simple condition check. But there is also the jump label that reduce the condition check to an unconditional jump in the off case. To summarize, here's how calling tick_nohz_task_switch() maps to final C code: finish_task_switch() { //do things before calling tick_nohz_task_switch()... // call tick_nohz_task_switch goto offcase; if (tick_nohz_full_enabled()) __tick_nohz_task_switch(tsk); offcase: //end of call to tick_nohz_task_switch //do things before calling tick_nohz_task_switch()... } In the offcase, the code is like above. We don't even do the check, thanks to the jump label code we unconditionally jump to what's next in finish_task_switch() (there is actually nothing afterward but that's for the picture). Now if there is at least a CPU that is full dynticks on boot, it is enabled with context_tracking_cpu_set(). Then the jump label code patches the code in finish_task_switch() to turn the goto offcase into a nop. Then the condition is actually verified on every call to finish_task_switch(). So it goes beyond than just saving a function call. > > In this case probably we can move !can_stop_full_tick() as well to the wrapper ? Do you mean moving all the code of __tick_nohz_task_switch() to tick_nohz_task_switch()? I much prefer we don't do that. This is going to make can_stop_full_tick() a publicly visible nohz internal. And it may uglify tick.h as well. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
On 15 April 2014 14:43, Frederic Weisbecker <fweisbec@gmail.com> wrote: > Yeah. But not just that. > > Using an inline saves a function call and reduce the offline case to a simple > condition check. But there is also the jump label that reduce the condition check > to an unconditional jump in the off case. > > To summarize, here's how calling tick_nohz_task_switch() maps to final C code: > > finish_task_switch() > { > //do things before calling tick_nohz_task_switch()... > // call tick_nohz_task_switch > goto offcase; > if (tick_nohz_full_enabled()) > __tick_nohz_task_switch(tsk); > offcase: > //end of call to tick_nohz_task_switch > //do things before calling tick_nohz_task_switch()... > } > > In the offcase, the code is like above. We don't even do the check, thanks to > the jump label code we unconditionally jump to what's next in finish_task_switch() > (there is actually nothing afterward but that's for the picture). > > Now if there is at least a CPU that is full dynticks on boot, it is enabled > with context_tracking_cpu_set(). Then the jump label code patches the code in > finish_task_switch() to turn the goto offcase into a nop. Then the condition is > actually verified on every call to finish_task_switch(). > > So it goes beyond than just saving a function call. Sorry, but my poor mind still couldn't understand what you are trying to tell me :( So lets clarify things one by one :) - What do you mean by offcase? CONFIG_NO_HZ_FULL not configured into the kernel or it is configured but none of the CPUs is running in that mode? - Also what does it correspond to in code: goto offcase; ? There is no labels or goto statements in code that I can see.. This is how the code looks to me. > finish_task_switch() > { > //do things before calling tick_nohz_task_switch()... > // call tick_nohz_task_switch > if (tick_nohz_full_enabled()) > __tick_nohz_task_switch(tsk); > } __tick_nohz_task_switch() may or maynot be available at all depending on CONFIG_NO_HZ_FULL is enabled into the kernel or not. But that was the case with tick_nohz_task_switch() as well in my patch. So shouldn't make a difference.. Again, sorry for not understanding what you are trying to explain here. I want to understand this once and for all and probably add a comment here as well :) -- viresh -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
On Tue, Apr 15, 2014 at 03:23:37PM +0530, Viresh Kumar wrote: > On 15 April 2014 14:43, Frederic Weisbecker <fweisbec@gmail.com> wrote: > > Yeah. But not just that. > > > > Using an inline saves a function call and reduce the offline case to a simple > > condition check. But there is also the jump label that reduce the condition check > > to an unconditional jump in the off case. > > > > To summarize, here's how calling tick_nohz_task_switch() maps to final C code: > > > > finish_task_switch() > > { > > //do things before calling tick_nohz_task_switch()... > > // call tick_nohz_task_switch > > goto offcase; > > if (tick_nohz_full_enabled()) > > __tick_nohz_task_switch(tsk); > > offcase: > > //end of call to tick_nohz_task_switch > > //do things before calling tick_nohz_task_switch()... > > } > > > > In the offcase, the code is like above. We don't even do the check, thanks to > > the jump label code we unconditionally jump to what's next in finish_task_switch() > > (there is actually nothing afterward but that's for the picture). > > > > Now if there is at least a CPU that is full dynticks on boot, it is enabled > > with context_tracking_cpu_set(). Then the jump label code patches the code in > > finish_task_switch() to turn the goto offcase into a nop. Then the condition is > > actually verified on every call to finish_task_switch(). > > > > So it goes beyond than just saving a function call. > > Sorry, but my poor mind still couldn't understand what you are trying to > tell me :( Welcome to the club of the daily confused people. I'm happy to hear I'm not alone :) > > So lets clarify things one by one :) > > - What do you mean by offcase? CONFIG_NO_HZ_FULL not configured > into the kernel or it is configured but none of the CPUs is running in that > mode? So by offcase I mean CONFIG_NO_HZ_FULL=y but the nohz_full boot parameter is empty, or simply not passed at all. And of course CONFIG_NO_HZ_FULL_ALL=n This config is now likely on some distros because we want to make full dynticks available for users who want it. But if it's not used (which is 99.999% of the usecases), we want to minimize as much as possible its overhead. Lets call that dynamic off-case. > > - Also what does it correspond to in code: goto offcase; ? There is no labels > or goto statements in code that I can see.. This is how the code looks to me. > > > finish_task_switch() > > { > > //do things before calling tick_nohz_task_switch()... > > // call tick_nohz_task_switch > > if (tick_nohz_full_enabled()) > > __tick_nohz_task_switch(tsk); > > } Sure but check out the static_key_false() in the implementation of tick_nohz_full_enabled(). That's where the magic hides. > > __tick_nohz_task_switch() may or maynot be available at all depending > on CONFIG_NO_HZ_FULL is enabled into the kernel or not. But that > was the case with tick_nohz_task_switch() as well in my patch. So > shouldn't make a difference.. > > Again, sorry for not understanding what you are trying to explain here. > I want to understand this once and for all and probably add a comment > here as well :) No problem, the jump label/static key code is quite tricky. And its use can be easily missed, as in here. Also its unfamous API naming (static_key_true/static_key_true) that is anything but intuitive. > > -- > viresh -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
diff --git a/include/linux/tick.h b/include/linux/tick.h index 1065a51..585be84 100644 --- a/include/linux/tick.h +++ b/include/linux/tick.h @@ -201,7 +201,7 @@ extern void tick_nohz_init(void); extern void __tick_nohz_full_check(void); extern void tick_nohz_full_kick(void); extern void tick_nohz_full_kick_all(void); -extern void __tick_nohz_task_switch(void); +extern void tick_nohz_task_switch(void); #else static inline void tick_nohz_init(void) { } static inline bool tick_nohz_full_enabled(void) { return false; } @@ -209,7 +209,7 @@ static inline bool tick_nohz_full_cpu(int cpu) { return false; } static inline void __tick_nohz_full_check(void) { } static inline void tick_nohz_full_kick(void) { } static inline void tick_nohz_full_kick_all(void) { } -static inline void __tick_nohz_task_switch(void) { } +static inline void tick_nohz_task_switch(void) { } #endif static inline void tick_nohz_full_check(void) @@ -218,11 +218,4 @@ static inline void tick_nohz_full_check(void) __tick_nohz_full_check(); } -static inline void tick_nohz_task_switch(void) -{ - if (tick_nohz_full_enabled()) - __tick_nohz_task_switch(); -} - - #endif diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 5f7796d..d8b9a69 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -266,13 +266,16 @@ void tick_nohz_full_kick_all(void) * It might need the tick due to per task/process properties: * perf events, posix cpu timers, ... */ -void __tick_nohz_task_switch(void) +void tick_nohz_task_switch(void) { unsigned long flags; + if (!tick_nohz_full_enabled() || !tick_nohz_tick_stopped()) + return; + local_irq_save(flags); - if (tick_nohz_tick_stopped() && !can_stop_full_tick()) + if (!can_stop_full_tick()) tick_nohz_full_kick(); local_irq_restore(flags);
__tick_nohz_task_switch() was called only from tick_nohz_task_switch() and there is nothing much in tick_nohz_task_switch() as well. IOW, we don't need unnecessary wrapper over __tick_nohz_task_switch() to be there. Merge all code from __tick_nohz_task_switch() into tick_nohz_task_switch() and move it to tick-sched.c. This also moves check for tick_nohz_tick_stopped() outside of irq_save() context. Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org> --- include/linux/tick.h | 11 ++--------- kernel/time/tick-sched.c | 7 +++++-- 2 files changed, 7 insertions(+), 11 deletions(-)