From patchwork Thu Aug 24 18:08:55 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Bellasi X-Patchwork-Id: 110942 Delivered-To: patch@linaro.org Received: by 10.140.95.78 with SMTP id h72csp6047964qge; Thu, 24 Aug 2017 11:09:43 -0700 (PDT) X-Received: by 10.84.178.164 with SMTP id z33mr7980079plb.419.1503598183634; Thu, 24 Aug 2017 11:09:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1503598183; cv=none; d=google.com; s=arc-20160816; b=AvA7V6X3mo7P2xAmLL9vC3nkp4Lk91AQfUpZnjB0hymG871d5tdcDUV+lHIM6p6zsF hixs2o7nSGKvZbtSaOhUnyVOwj7fyKigLxGRD68R4pjeoQm7N3NZw2lz2Jb3hYwTuurT NwaHPu3JJryz3NJ6jjctAAMViHKfyONczWqV0GqH04FlokovUz7WrEYl94xwd+w+QOcu rQ2sNTVkAZ4M+g2sPG3H5PPKW6QmZzLlSmAK2tQ50MFiuEXwVzwQWoLaEcrKZXLE6X+K HIif/6B1hOIhFfB1rBBj2VtMCNnSnyrAKvgVpHAfbytPnlQ11F0jvXhBg0J0spDKlbiI AmGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=RE3DqthpU4tMjM3ktTUqN2Vly4/1i45yn1NRb3juT14=; b=ALN1hyQGTsKjhYDuDS9YFVeneI07KO9Xo/KTPsfWiqrqh5NBv5SZ3f0paTpAcC8N9Z mDoM10XMJYeCseIthNxaCw6q/ECnI76Te8SrFRqYTwa/njeUFVSIgUpFv8YqNIyhfMCa 5y2/RFdPyj50mlrta58fKQbt6fX7qWcD7o+fu4p8AZJH5E6eF76quV7iGXscP9f4f8sw DJYbsikkSPtTCEN23z1Jc9UsnbVGmcWSrnwE2AgXV7W4koVicXqec50ccSVrvMTz1R5H QjorJJLiprERC3dHrjjPhU+JQS3Dx75jyErDClPdT+k0ICMjy1RQaDUQX3XssxnJHEDV qz1Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-pm-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u6si1739460pld.225.2017.08.24.11.09.42; Thu, 24 Aug 2017 11:09:43 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-pm-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-pm-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753566AbdHXSJl (ORCPT + 12 others); Thu, 24 Aug 2017 14:09:41 -0400 Received: from foss.arm.com ([217.140.101.70]:45486 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753555AbdHXSJg (ORCPT ); Thu, 24 Aug 2017 14:09:36 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 48355169E; Thu, 24 Aug 2017 11:09:36 -0700 (PDT) Received: from e110439-lin.cambridge.arm.com (e110439-lin.cambridge.arm.com [10.1.210.68]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 515653F3E1; Thu, 24 Aug 2017 11:09:33 -0700 (PDT) From: Patrick Bellasi To: linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org Cc: Ingo Molnar , Peter Zijlstra , Tejun Heo , "Rafael J . Wysocki" , Paul Turner , Vincent Guittot , John Stultz , Morten Rasmussen , Dietmar Eggemann , Juri Lelli , Tim Murray , Todd Kjos , Andres Oportus , Joel Fernandes , Viresh Kumar Subject: [RFCv4 4/6] sched/core: sync task_group's with CPU's clamp groups Date: Thu, 24 Aug 2017 19:08:55 +0100 Message-Id: <20170824180857.32103-5-patrick.bellasi@arm.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20170824180857.32103-1-patrick.bellasi@arm.com> References: <20170824180857.32103-1-patrick.bellasi@arm.com> Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The util_{min,max} clamp values for a task group are usually updated from a user-space process (slow-path) but they require a synchronization with the scheduler's (fast-path) maintained clamp group reference couters. Indeed, each time the clamp value of a task group is changed, the old and new clamp groups have to be updated for each CPU containing a RUNNABLE task belonging to this tasks group. Non RUNNABLE tasks are not updated since they will be enqueued with the proper clamp group index at their next activation. To properly update clamp group's reference counter of runnable tasks we use the same locking schema use by __set_cpus_allowed_ptr(). This might lock the (previous) RQ of a !RUNNABLE task, but that's the price to pay to safely serialize util_{min,max} updates with RQ's enqueues, dequeues and migration operations. Signed-off-by: Patrick Bellasi Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Tejun Heo Cc: linux-kernel@vger.kernel.org Cc: linux-pm@vger.kernel.org --- kernel/sched/core.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++ kernel/sched/sched.h | 21 +++++++++++++++++ 2 files changed, 87 insertions(+) -- 2.14.1 diff --git a/kernel/sched/core.c b/kernel/sched/core.c index ba31bb4e14c7..e4ce25dbad6f 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -754,6 +754,12 @@ static void set_load_weight(struct task_struct *p) #ifdef CONFIG_UTIL_CLAMP /** * uclamp_mutex: serialize updates of TG's utilization clamp values + * + * A task groups's utilization clamp value update is usually triggered from a + * user-space process (slow-path) but it requires a synchronization with the + * scheduler's (fast-path) enqueue/dequeue operations. + * While the fast-path synchronization is protected by RQs spinlock, this + * mutex ensure that we sequentially serve user-space requests. */ static DEFINE_MUTEX(uclamp_mutex); @@ -1022,6 +1028,52 @@ static inline void uclamp_cpu_put(struct task_struct *p, int cpu, int clamp_id) uclamp_cpu_update(cpu, clamp_id); } +/** + * uclamp_task_update_active: update the clamp group of a RUNNABLE task + * @p: the task which clamp groups must be updated + * @clamp_id: the clamp index to consider + * @group_id: the clamp group to update + * + * Each time the clamp value of a task group is changed, the old and new clamp + * groups have to be updated for each CPU containing a RUNNABLE task belonging + * to this tasks group. Sleeping tasks are not updated since they will be + * enqueued with the proper clamp group index at their next activation. + */ +static inline void +uclamp_task_update_active(struct task_struct *p, int clamp_id, int group_id) +{ + struct rq_flags rf; + struct rq *rq; + + /* + * Lock the task and the CPU where the task is (or was) queued. + * + * We might lock the (previous) RQ of a !RUNNABLE task, but that's the + * price to pay to safely serialize util_{min,max} updates with + * enqueues, dequeues and migration operations. + * This is the same locking schema used by __set_cpus_allowed_ptr(). + */ + rq = task_rq_lock(p, &rf); + + /* + * The setting of the clamp group is serialized by task_rq_lock(). + * Thus, if the task's task_struct is not referencing a valid group + * index, then that task is not yet RUNNABLE or it's going to be + * enqueued with the proper clamp group value. + */ + if (!uclamp_task_active(p)) + goto done; + + /* Release p's currently referenced clamp group */ + uclamp_cpu_put(p, task_cpu(p), clamp_id); + + /* Get p's new clamp group */ + uclamp_cpu_get(p, task_cpu(p), clamp_id); + +done: + task_rq_unlock(rq, p, &rf); +} + /** * uclamp_group_put: decrease the reference count for a clamp group * @clamp_id: the clamp index which was affected by a task group @@ -1070,6 +1122,8 @@ static inline int uclamp_group_get(struct cgroup_subsys_state *css, struct uclamp_map *uc_map = &uclamp_maps[clamp_id][0]; int prev_group_id = uc_tg->group_id; int next_group_id = UCLAMP_NONE; + struct css_task_iter it; + struct task_struct *p; unsigned long flags; /* Lookup for a usable utilization clamp group */ @@ -1091,6 +1145,18 @@ static inline int uclamp_group_get(struct cgroup_subsys_state *css, uc_map[next_group_id].tg_count += 1; raw_spin_unlock_irqrestore(&uc_map[next_group_id].tg_lock, flags); + /* Newly created TG don't have tasks assigned */ + if (!css) + goto release; + + /* Update clamp groups for RUNNABLE tasks in this TG */ + css_task_iter_start(css, &it); + while ((p = css_task_iter_next(&it))) + uclamp_task_update_active(p, clamp_id, next_group_id); + css_task_iter_end(&it); + +release: + /* Release the previous clamp group */ uclamp_group_put(clamp_id, prev_group_id); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index b0f17c19c0f6..164a8ac152b3 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -462,6 +462,27 @@ static inline bool uclamp_task_affects(struct task_struct *p, int clamp_id) return (task_group_id != UCLAMP_NONE); } +/** + * uclamp_task_active: check if a task is currently clamping a CPU + * @p: the task to check + * + * A task affects the utilization clamp of a CPU if it references a valid + * clamp group index for at least one clamp index. + * + * Return: true if p is currently clamping the utilization of its CPU. + */ +static inline bool uclamp_task_active(struct task_struct *p) +{ + int clamp_id; + + for (clamp_id = 0; clamp_id < UCLAMP_CNT; ++clamp_id) { + if (uclamp_task_affects(p, clamp_id)) + return true; + } + + return false; +} + /** * uclamp_group_active: check if a clamp group is active on a CPU * @uc_cpu: the array of clamp groups for a CPU