From patchwork Thu Mar 26 19:23:22 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Stultz X-Patchwork-Id: 46375 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-lb0-f198.google.com (mail-lb0-f198.google.com [209.85.217.198]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 0646E21585 for ; Thu, 26 Mar 2015 19:23:38 +0000 (UTC) Received: by lbbro7 with SMTP id ro7sf6571940lbb.0 for ; Thu, 26 Mar 2015 12:23:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-unsubscribe; bh=smDKVoSYRbm7mpT8MPyNMBwa9oWM6tMAjqjfQi3M9k8=; b=WmJkGyLJZGe61E4YnM0Ecc1/JgLBzUZNatFE6xRYq0LtMk/StWNoWZqEqZ8KWkC5BE E3fIXfCX5n9QGAZO9+GKH0gHeg0ESxH73mD4LoPJ2QWyLN809pSVlPdDDGos14Rx4YUB d7N+P01OFP4iYTnAQ88/3gl6dly94wsQa+OY/ph4Kkcp3dDMNKDXa5C0CPg3Y6UbVmhg 9abKRMlJqo/OJbcyV3poXfjKnDkLzZmnMGkG3UZuLBMyhxmGbB/EGjgOnfngarIYwjMj mo1NZZhvLtCYaekqPXz4/B3ZJnxG1exZQWpAmaQ/b7zNTy9QywcUHnMwrzSCxTP3h4ru KHMw== X-Gm-Message-State: ALoCoQlBKTbzp18l+WxvJK/r5LqRhGu15KcKRlgeWwwr+D5RfjLdIf+D1x8iEnFSKVB6fxEznkgf X-Received: by 10.194.236.196 with SMTP id uw4mr3751707wjc.5.1427397817014; Thu, 26 Mar 2015 12:23:37 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.234.236 with SMTP id uh12ls294505lac.54.gmail; Thu, 26 Mar 2015 12:23:36 -0700 (PDT) X-Received: by 10.112.235.227 with SMTP id up3mr5269509lbc.86.1427397816853; Thu, 26 Mar 2015 12:23:36 -0700 (PDT) Received: from mail-la0-f50.google.com (mail-la0-f50.google.com. [209.85.215.50]) by mx.google.com with ESMTPS id mu1si5351451lbc.3.2015.03.26.12.23.36 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Mar 2015 12:23:36 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.50 as permitted sender) client-ip=209.85.215.50; Received: by lahp7 with SMTP id p7so34886709lah.2 for ; Thu, 26 Mar 2015 12:23:36 -0700 (PDT) X-Received: by 10.112.212.106 with SMTP id nj10mr14724649lbc.36.1427397816704; Thu, 26 Mar 2015 12:23:36 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.112.57.201 with SMTP id k9csp813200lbq; Thu, 26 Mar 2015 12:23:35 -0700 (PDT) X-Received: by 10.70.98.139 with SMTP id ei11mr29103629pdb.3.1427397814542; Thu, 26 Mar 2015 12:23:34 -0700 (PDT) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com. [209.85.220.54]) by mx.google.com with ESMTPS id cy3si9647721pdb.42.2015.03.26.12.23.33 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Mar 2015 12:23:34 -0700 (PDT) Received-SPF: pass (google.com: domain of john.stultz@linaro.org designates 209.85.220.54 as permitted sender) client-ip=209.85.220.54; Received: by pacwz10 with SMTP id wz10so20820887pac.2 for ; Thu, 26 Mar 2015 12:23:33 -0700 (PDT) X-Received: by 10.66.142.12 with SMTP id rs12mr28979843pab.12.1427397813514; Thu, 26 Mar 2015 12:23:33 -0700 (PDT) Received: from localhost.localdomain (c-67-170-153-23.hsd1.or.comcast.net. [67.170.153.23]) by mx.google.com with ESMTPSA id pv4sm6365753pbb.17.2015.03.26.12.23.32 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 26 Mar 2015 12:23:32 -0700 (PDT) From: John Stultz To: lkml Cc: Daniel Thompson , Russell King , Will Deacon , Catalin Marinas , Thomas Gleixner , Stephen Boyd , Peter Zijlstra , Ingo Molnar , John Stultz Subject: [PATCH 1/5] sched_clock: Match scope of read and write seqcounts Date: Thu, 26 Mar 2015 12:23:22 -0700 Message-Id: <1427397806-20889-2-git-send-email-john.stultz@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1427397806-20889-1-git-send-email-john.stultz@linaro.org> References: <1427397806-20889-1-git-send-email-john.stultz@linaro.org> X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: john.stultz@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.50 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , From: Daniel Thompson Currently the scope of the raw_write_seqcount_begin/end in sched_clock_register far exceeds the scope of the read section in sched_clock. This gives the impression of safety during cursory review but achieves little. Note that this is likely to be a latent issue at present because sched_clock_register() is typically called before we enable interrupts, however the issue does risk bugs being needlessly introduced as the code evolves. This patch fixes the problem by increasing the scope of the read locking performed by sched_clock() to cover all data modified by sched_clock_register. We also improve clarity by moving writes to struct clock_data that do not impact sched_clock() outside of the critical section. Cc: Russell King Cc: Will Deacon Cc: Catalin Marinas Cc: Daniel Thompson Cc: Thomas Gleixner Cc: Stephen Boyd Cc: Peter Zijlstra Cc: Ingo Molnar Reviewed-by: Stephen Boyd Acked-by: Peter Zijlstra (Intel) Signed-off-by: Daniel Thompson [jstultz: Slight rework to apply to tip/timers/core] Signed-off-by: John Stultz --- kernel/time/sched_clock.c | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/kernel/time/sched_clock.c b/kernel/time/sched_clock.c index ca3bc5c..1751e95 100644 --- a/kernel/time/sched_clock.c +++ b/kernel/time/sched_clock.c @@ -58,23 +58,21 @@ static inline u64 notrace cyc_to_ns(u64 cyc, u32 mult, u32 shift) unsigned long long notrace sched_clock(void) { - u64 epoch_ns; - u64 epoch_cyc; - u64 cyc; + u64 cyc, res; unsigned long seq; - if (cd.suspended) - return cd.epoch_ns; - do { seq = raw_read_seqcount_begin(&cd.seq); - epoch_cyc = cd.epoch_cyc; - epoch_ns = cd.epoch_ns; + + res = cd.epoch_ns; + if (!cd.suspended) { + cyc = read_sched_clock(); + cyc = (cyc - cd.epoch_cyc) & sched_clock_mask; + res += cyc_to_ns(cyc, cd.mult, cd.shift); + } } while (read_seqcount_retry(&cd.seq, seq)); - cyc = read_sched_clock(); - cyc = (cyc - epoch_cyc) & sched_clock_mask; - return epoch_ns + cyc_to_ns(cyc, cd.mult, cd.shift); + return res; } /* @@ -111,7 +109,6 @@ void __init sched_clock_register(u64 (*read)(void), int bits, { u64 res, wrap, new_mask, new_epoch, cyc, ns; u32 new_mult, new_shift; - ktime_t new_wrap_kt; unsigned long r; char r_unit; @@ -124,10 +121,11 @@ void __init sched_clock_register(u64 (*read)(void), int bits, clocks_calc_mult_shift(&new_mult, &new_shift, rate, NSEC_PER_SEC, 3600); new_mask = CLOCKSOURCE_MASK(bits); + cd.rate = rate; /* calculate how many nanosecs until we risk wrapping */ wrap = clocks_calc_max_nsecs(new_mult, new_shift, 0, new_mask, NULL); - new_wrap_kt = ns_to_ktime(wrap); + cd.wrap_kt = ns_to_ktime(wrap); /* update epoch for new counter and update epoch_ns from old counter*/ new_epoch = read(); @@ -138,8 +136,6 @@ void __init sched_clock_register(u64 (*read)(void), int bits, raw_write_seqcount_begin(&cd.seq); read_sched_clock = read; sched_clock_mask = new_mask; - cd.rate = rate; - cd.wrap_kt = new_wrap_kt; cd.mult = new_mult; cd.shift = new_shift; cd.epoch_cyc = new_epoch;