From patchwork Mon May 13 08:29:33 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viresh Kumar X-Patchwork-Id: 16874 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-gg0-f197.google.com (mail-gg0-f197.google.com [209.85.161.197]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id E2DD123973 for ; Mon, 13 May 2013 08:30:08 +0000 (UTC) Received: by mail-gg0-f197.google.com with SMTP id w6sf6613150ggk.4 for ; Mon, 13 May 2013 01:29:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:x-beenthere:x-received:received-spf:x-received :x-forwarded-to:x-forwarded-for:delivered-to:x-received:received-spf :mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:x-gm-message-state:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :x-google-group-id:list-post:list-help:list-archive:list-unsubscribe :content-type; bh=g+o4C0AgGrpySErZBPHNLiMr/g1BSerMUMiGueZIQ9Y=; b=mFtFqpufEgx5O7cMWyPp1TmB0ppIyA6HCsJALrqnD6b1hrJMJemnmeGy+oubsLTlDG 0iXG9rUbjhaBMO9Oi/h6SmunuivTE2MvR9PoEMpmmnDW6xJw4IEaVANVkoHbJWG1Ex6r l6Z/9z1iyfEPdob0B5MDMY+GdS17WYxftheBGDpV3T+eWZDkE2EVf9zcxGHVn2aUTAGL xirdegbcFMhshwIBYDgz7I9IBJXG8ZJN21hfch7owmrW4a8GUJyIjm/0aebPPVirt6YB q8Mggw+rYOy2pF8m8chQuRI1F0J0JH1jCNlHPMetdEysek3W2CLYpNVKGHE5IczIyOsi uQbA== X-Received: by 10.224.59.205 with SMTP id m13mr18415398qah.7.1368433775777; Mon, 13 May 2013 01:29:35 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.49.49.200 with SMTP id w8ls2870692qen.1.gmail; Mon, 13 May 2013 01:29:35 -0700 (PDT) X-Received: by 10.58.37.165 with SMTP id z5mr17870000vej.12.1368433775496; Mon, 13 May 2013 01:29:35 -0700 (PDT) Received: from mail-vc0-f179.google.com (mail-vc0-f179.google.com [209.85.220.179]) by mx.google.com with ESMTPS id by3si7963689vdc.43.2013.05.13.01.29.35 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 13 May 2013 01:29:35 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.220.179 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.220.179; Received: by mail-vc0-f179.google.com with SMTP id hz10so5326300vcb.10 for ; Mon, 13 May 2013 01:29:35 -0700 (PDT) X-Received: by 10.58.48.166 with SMTP id m6mr17409643ven.59.1368433775333; Mon, 13 May 2013 01:29:35 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.220.217.15 with SMTP id hk15csp9607vcb; Mon, 13 May 2013 01:29:34 -0700 (PDT) X-Received: by 10.182.226.136 with SMTP id rs8mr12694802obc.50.1368433773755; Mon, 13 May 2013 01:29:33 -0700 (PDT) Received: from mail-oa0-f45.google.com (mail-oa0-f45.google.com [209.85.219.45]) by mx.google.com with ESMTPS id fi9si14388053obb.204.2013.05.13.01.29.33 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 13 May 2013 01:29:33 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.219.45 is neither permitted nor denied by best guess record for domain of viresh.kumar@linaro.org) client-ip=209.85.219.45; Received: by mail-oa0-f45.google.com with SMTP id j6so6042221oag.4 for ; Mon, 13 May 2013 01:29:33 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.182.246.227 with SMTP id xz3mr4392580obc.67.1368433773242; Mon, 13 May 2013 01:29:33 -0700 (PDT) Received: by 10.182.79.35 with HTTP; Mon, 13 May 2013 01:29:33 -0700 (PDT) In-Reply-To: <20130426191118.GD1433@mtj.dyndns.org> References: <480f5d6da920e2f2b868f311228c81635727799e.1366803121.git.viresh.kumar@linaro.org> <20130424161208.GA16991@mtj.dyndns.org> <20130426191118.GD1433@mtj.dyndns.org> Date: Mon, 13 May 2013 13:59:33 +0530 Message-ID: Subject: Re: [PATCH V5 1/5] workqueues: Introduce new flag WQ_POWER_EFFICIENT for power oriented workqueues From: Viresh Kumar To: Tejun Heo Cc: Amit Kucheria , davem@davemloft.net, linux-rt-users@vger.kernel.org, linux-kernel@vger.kernel.org, Robin Randhawa , Charles Garcia-Tobin , Steve Bannister , Peter Zijlstra , Steven Rostedt , Arvind Chauhan , Patch Tracking , airlied@redhat.com, mingo@redhat.com, Jens Axboe , Liviu Dudau , Lists linaro-kernel X-Gm-Message-State: ALoCoQnzQjlt3ltg1GW9uRn+VOBY86H1cR79wtL/zNnflbJWqMrCq6bMeH9U2hM4KF3icr3rSZND X-Original-Sender: viresh.kumar@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.220.179 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , On 27 April 2013 00:41, Tejun Heo wrote: > > It's already too late for the upcoming merge window, but things > generally look good to me and I'll apply the patchset once wq/for-3.11 > opens. One nitpick tho. > > On Thu, Apr 25, 2013 at 09:13:44AM +0530, Viresh Kumar wrote: >> + workqueue.power_efficient >> + Workqueues can be performance or power-oriented. >> + Currently, most workqueues are bound to the CPU they > ^^^^ > per-cpu would be better > >> + were created on. This gives good performance (due to >> + cache effects) at the cost of potentially waking up >> + otherwise idle cores just to process some work. To save >> + power, we can allow the work to be rescheduled on a core >> + that is already awake. > > The above description is confusing to me. As have been discussed > multiple times before, per-cpu workqueue in itself doesn't wake up the > CPU physically. The timer may but per-cpu workqueue doesn't. It was > confusing when this patchset was first posted and the above phrasing > is still confusing. What the patchset tries to do is preventing the > scheduler from perceiving the CPU as active due to the activated > worker thread pinned to that CPU, right? The knob doesn't really do > anything about waking up the processor in itself. It just avoids > feeding the scheduler with noisy activation events and allows it to > allocate work item execution according to the scheduler's view of CPU > active/idleness. As the scheduler has longer / larger scope of > overall CPU activities and means to regulate them, this leads to more > power-efficient allocation of work item executions, right? It'd be > really great if the descriptions and the comment above the flag makes > this abundantly clear because it's not something too apparent. Sorry for the long delay for such a small change. I went on long leaves.. I have added following to make things more clear at places: (Idle from scheduler's perspective. Which may or may not be physically idle).. Let me know if it is still unclear.. And this is the new patch: (Attached it too for applying cleanly) ---------x---------------x------------------- From: Viresh Kumar Date: Mon, 8 Apr 2013 16:45:40 +0530 Subject: [PATCH V5 resent 1/5] workqueues: Introduce new flag WQ_POWER_EFFICIENT for power oriented workqueues Workqueues can be performance or power-oriented. Currently, most workqueues are bound to the CPU they were created on. This gives good performance (due to cache effects) at the cost of potentially waking up otherwise idle cores (Idle from scheduler's perspective. Which may or may not be physically idle) just to process some work. To save power, we can allow the work to be rescheduled on a core that is already awake. Workqueues created with the WQ_UNBOUND flag will allow some power savings. However, we don't change the default behaviour of the system. To enable power-saving behaviour, a new config option CONFIG_WQ_POWER_EFFICIENT needs to be turned on. This option can also be overridden by the workqueue.power_efficient boot parameter. Signed-off-by: Viresh Kumar Reviewed-by: Amit Kucheria --- Documentation/kernel-parameters.txt | 18 ++++++++++++++++++ include/linux/workqueue.h | 3 +++ kernel/power/Kconfig | 21 +++++++++++++++++++++ kernel/workqueue.c | 11 +++++++++++ 4 files changed, 53 insertions(+) exclusion */ @@ -4085,6 +4093,9 @@ struct workqueue_struct *__alloc_workqueue_key(const char *fmt, struct workqueue_struct *wq; struct pool_workqueue *pwq; + if ((flags & WQ_POWER_EFFICIENT) && wq_power_efficient) + flags |= WQ_UNBOUND; + /* allocate wq and format name */ if (flags & WQ_UNBOUND) tbl_size = wq_numa_tbl_len * sizeof(wq->numa_pwq_tbl[0]); diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index c3bfacb..9a991b6 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3320,6 +3320,24 @@ bytes respectively. Such letter suffixes can also be entirely omitted. that this also can be controlled per-workqueue for workqueues visible under /sys/bus/workqueue/. + workqueue.power_efficient + Workqueues can be performance or power-oriented. + Currently, most workqueues are bound to the CPU they + were created on. This gives good performance (due to + cache effects) at the cost of potentially waking up + otherwise idle cores (Idle from scheduler's perspective. + Which may or may not be physically idle) just to process + some work. To save power, we can allow the work to be + rescheduled on a core that is already awake. + + Workqueues created with the WQ_UNBOUND flag will allow + some power savings. However, we don't change the + default behaviour of the system. To enable power-saving + behaviour, a new config option CONFIG_WQ_POWER_EFFICIENT + needs to be turned on. This option can also be + overridden by the workqueue.power_efficient boot + parameter. + x2apic_phys [X86-64,APIC] Use x2apic physical mode instead of default x2apic cluster mode on platforms supporting x2apic. diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h index 623488f..83fa570 100644 --- a/include/linux/workqueue.h +++ b/include/linux/workqueue.h @@ -302,6 +302,9 @@ enum { WQ_HIGHPRI = 1 << 4, /* high priority */ WQ_CPU_INTENSIVE = 1 << 5, /* cpu instensive workqueue */ WQ_SYSFS = 1 << 6, /* visible in sysfs, see wq_sysfs_register() */ + WQ_POWER_EFFICIENT = 1 << 7, /* WQ_UNBOUND, for power + * saving, if wq_power_efficient is + * enabled. Unused otherwise. */ __WQ_DRAINING = 1 << 16, /* internal: workqueue is draining */ __WQ_ORDERED = 1 << 17, /* internal: workqueue is ordered */ diff --git a/kernel/power/Kconfig b/kernel/power/Kconfig index 5dfdc9e..e8c2b77 100644 --- a/kernel/power/Kconfig +++ b/kernel/power/Kconfig @@ -263,6 +263,27 @@ config PM_GENERIC_DOMAINS bool depends on PM +config WQ_POWER_EFFICIENT + bool "Workqueue allocated as UNBOUND (by default) for power efficiency" + depends on PM + default n + help + Workqueues can be performance or power-oriented. Currently, most + workqueues are bound to the CPU they were created on. This gives good + performance (due to cache effects) at the cost of potentially waking + up otherwise idle cores (Idle from scheduler's perspective. Which may + or may not be physically idle) just to process some work. To save + power, we can allow the work to be rescheduled on a core that is + already awake. + + Workqueues created with the WQ_UNBOUND flag will allow some power + savings. However, we don't change the default behaviour of the + system. To enable power-saving behaviour, a new config option + CONFIG_WQ_POWER_EFFICIENT needs to be turned on. This option can also + be overridden by the workqueue.power_efficient boot parameter. + + If in doubt, say N. + config PM_GENERIC_DOMAINS_SLEEP def_bool y depends on PM_SLEEP && PM_GENERIC_DOMAINS diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 1ae6028..9d8753e 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -272,6 +272,14 @@ static cpumask_var_t *wq_numa_possible_cpumask; static bool wq_disable_numa; module_param_named(disable_numa, wq_disable_numa, bool, 0444); +#ifdef CONFIG_WQ_POWER_EFFICIENT +static bool wq_power_efficient = true; +#else +static bool wq_power_efficient; +#endif + +module_param_named(power_efficient, wq_power_efficient, bool, 0444); + static bool wq_numa_enabled; /* unbound NUMA affinity enabled */ /* buf for wq_update_unbound_numa_attrs(), protected by CPU hotplug