From patchwork Thu Aug 30 19:03:03 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 11067 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 75B7D23E52 for ; Thu, 30 Aug 2012 19:03:39 +0000 (UTC) Received: from mail-ie0-f180.google.com (mail-ie0-f180.google.com [209.85.223.180]) by fiordland.canonical.com (Postfix) with ESMTP id 04A5AA18A88 for ; Thu, 30 Aug 2012 19:03:02 +0000 (UTC) Received: by ieak11 with SMTP id k11so1030793iea.11 for ; Thu, 30 Aug 2012 12:03:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf:from:to:cc :subject:date:message-id:x-mailer:in-reply-to:references :x-content-scanned:x-cbid:x-gm-message-state; bh=wok5Ob21kje5Gw6rs85bmVC7wI2F9UVpRciXRZClLbM=; b=jgZIZU/FTZMRBVLTc7oGw1EQRX7MpYyV1qK41miHmuwpaAf2+kyT2TDD1pSUiwn0BQ 9G/oc2kD0pgbxeqQe/N4tdinZGwa/n6ZXXgG28YNZJgP53yzY7Z2MVPesYKUj4OCiBql 1Qp/duqnMpPM8T6ykJ/8Y/t9U+/QqBtwe5TPd5iDlo3f16wiuv8Fu9hFil4bChoog3KB CAwS7A9wRAX3NRc2Ke/8RP0o+KTD1WC3R+09Vhcdulgrf1pyxQY9JUVBhusSlRTgWqvg q5Hw9HuHsjOPt2xEgqk3ETaQ1xaxS4/g6J00mQoUfMfcpR8qFc72YlWFukyz3Q2TFJ+H XS3Q== Received: by 10.50.180.129 with SMTP id do1mr1997298igc.28.1346353418136; Thu, 30 Aug 2012 12:03:38 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.50.184.232 with SMTP id ex8csp26368igc; Thu, 30 Aug 2012 12:03:37 -0700 (PDT) Received: by 10.182.119.72 with SMTP id ks8mr5783795obb.10.1346353417569; Thu, 30 Aug 2012 12:03:37 -0700 (PDT) Received: from e35.co.us.ibm.com (e35.co.us.ibm.com. [32.97.110.153]) by mx.google.com with ESMTPS id tg7si3213705obb.60.2012.08.30.12.03.37 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 30 Aug 2012 12:03:37 -0700 (PDT) Received-SPF: pass (google.com: domain of paulmck@linux.vnet.ibm.com designates 32.97.110.153 as permitted sender) client-ip=32.97.110.153; Authentication-Results: mx.google.com; spf=pass (google.com: domain of paulmck@linux.vnet.ibm.com designates 32.97.110.153 as permitted sender) smtp.mail=paulmck@linux.vnet.ibm.com Received: from /spool/local by e35.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 30 Aug 2012 13:03:37 -0600 Received: from d03dlp02.boulder.ibm.com (9.17.202.178) by e35.co.us.ibm.com (192.168.1.135) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 30 Aug 2012 13:03:34 -0600 Received: from d03relay02.boulder.ibm.com (d03relay02.boulder.ibm.com [9.17.195.227]) by d03dlp02.boulder.ibm.com (Postfix) with ESMTP id 154743E4003F; Thu, 30 Aug 2012 13:03:33 -0600 (MDT) Received: from d03av01.boulder.ibm.com (d03av01.boulder.ibm.com [9.17.195.167]) by d03relay02.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q7UJ3PlV171950; Thu, 30 Aug 2012 13:03:28 -0600 Received: from d03av01.boulder.ibm.com (loopback [127.0.0.1]) by d03av01.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q7UJ38lk017321; Thu, 30 Aug 2012 13:03:15 -0600 Received: from paulmck-ThinkPad-W500 (sig-9-77-132-62.mts.ibm.com [9.77.132.62]) by d03av01.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q7UJ37aU017078; Thu, 30 Aug 2012 13:03:08 -0600 Received: by paulmck-ThinkPad-W500 (Postfix, from userid 1000) id 59E5AEA838; Thu, 30 Aug 2012 12:03:07 -0700 (PDT) From: "Paul E. McKenney" To: linux-kernel@vger.kernel.org Cc: mingo@elte.hu, laijs@cn.fujitsu.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@polymtl.ca, josh@joshtriplett.org, niv@us.ibm.com, tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org, Valdis.Kletnieks@vt.edu, dhowells@redhat.com, eric.dumazet@gmail.com, darren@dvhart.com, fweisbec@gmail.com, sbw@mit.edu, patches@linaro.org, Peter Zijlstra , "Paul E. McKenney" Subject: [PATCH tip/core/rcu 3/3] sched: Fix load avg vs cpu-hotplug Date: Thu, 30 Aug 2012 12:03:03 -0700 Message-Id: <1346353383-350-3-git-send-email-paulmck@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.8 In-Reply-To: <1346353383-350-1-git-send-email-paulmck@linux.vnet.ibm.com> References: <20120830190245.GA32625@linux.vnet.ibm.com> <1346353383-350-1-git-send-email-paulmck@linux.vnet.ibm.com> X-Content-Scanned: Fidelis XPS MAILER x-cbid: 12083019-6148-0000-0000-0000091682AC X-Gm-Message-State: ALoCoQlTTBiBGc8x7dhXoZeICJFRIY5wC2s2h2MQx1wHFJw3c43PFM0PwvqwFg3Rhdds96wJlumo From: Peter Zijlstra Rabik and Paul reported two different issues related to the same few lines of code. Rabik's issue is that the nr_uninterruptible migration code is wrong in that he sees artifacts due to this (Rabik please do expand in more detail). Paul's issue is that this code as it stands relies on us using stop_machine() for unplug, we all would like to remove this assumption so that eventually we can remove this stop_machine() usage altogether. The only reason we'd have to migrate nr_uninterruptible is so that we could use for_each_online_cpu() loops in favour of for_each_possible_cpu() loops, however since nr_uninterruptible() is the only such loop and its using possible lets not bother at all. The problem Rabik sees is (probably) caused by the fact that by migrating nr_uninterruptible we screw rq->calc_load_active for both rqs involved. So don't bother with fancy migration schemes (meaning we now have to keep using for_each_possible_cpu()) and instead fold any nr_active delta after we migrate all tasks away to make sure we don't have any skewed nr_active accounting. [ paulmck: Move call to calc_load_migration to CPU_DEAD to avoid miscounting noted by Rakib. ] Reported-by: Rakib Mullick Reported-by: Paul E. McKenney Signed-off-by: Peter Zijlstra Signed-off-by: Paul E. McKenney --- kernel/sched/core.c | 41 ++++++++++++++++++++--------------------- 1 files changed, 20 insertions(+), 21 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index d325c4b..06051e0 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5300,27 +5300,17 @@ void idle_task_exit(void) } /* - * While a dead CPU has no uninterruptible tasks queued at this point, - * it might still have a nonzero ->nr_uninterruptible counter, because - * for performance reasons the counter is not stricly tracking tasks to - * their home CPUs. So we just add the counter to another CPU's counter, - * to keep the global sum constant after CPU-down: - */ -static void migrate_nr_uninterruptible(struct rq *rq_src) -{ - struct rq *rq_dest = cpu_rq(cpumask_any(cpu_active_mask)); - - rq_dest->nr_uninterruptible += rq_src->nr_uninterruptible; - rq_src->nr_uninterruptible = 0; -} - -/* - * remove the tasks which were accounted by rq from calc_load_tasks. + * Since this CPU is going 'away' for a while, fold any nr_active delta + * we might have. Assumes we're called after migrate_tasks() so that the + * nr_active count is stable. + * + * Also see the comment "Global load-average calculations". */ -static void calc_global_load_remove(struct rq *rq) +static void calc_load_migrate(struct rq *rq) { - atomic_long_sub(rq->calc_load_active, &calc_load_tasks); - rq->calc_load_active = 0; + long delta = calc_load_fold_active(rq); + if (delta) + atomic_long_add(delta, &calc_load_tasks); } /* @@ -5613,9 +5603,18 @@ migration_call(struct notifier_block *nfb, unsigned long action, void *hcpu) migrate_tasks(cpu); BUG_ON(rq->nr_running != 1); /* the migration thread */ raw_spin_unlock_irqrestore(&rq->lock, flags); + break; - migrate_nr_uninterruptible(rq); - calc_global_load_remove(rq); + case CPU_DEAD: + { + struct rq *dest_rq; + + local_irq_save(flags); + dest_rq = cpu_rq(smp_processor_id()); + raw_spin_lock(&dest_rq->lock); + calc_load_migrate(rq); + raw_spin_unlock_irqrestore(&dest_rq->lock, flags); + } break; #endif }