From patchwork Fri Jul 27 18:48:12 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Stultz X-Patchwork-Id: 10340 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 7EF3523F08 for ; Fri, 27 Jul 2012 18:49:08 +0000 (UTC) Received: from mail-gh0-f180.google.com (mail-gh0-f180.google.com [209.85.160.180]) by fiordland.canonical.com (Postfix) with ESMTP id 2B371A186ED for ; Fri, 27 Jul 2012 18:49:08 +0000 (UTC) Received: by ghbz12 with SMTP id z12so3532320ghb.11 for ; Fri, 27 Jul 2012 11:49:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf:from:to:cc :subject:date:message-id:x-mailer:in-reply-to:references :x-content-scanned:x-cbid:x-ibm-iss-spamdetectors :x-ibm-iss-detailinfo:x-gm-message-state; bh=v4LDZSpS3wZkeGV/+udC8QuyUCTntJr2e6Zg2aspplw=; b=H8Qhfz9fx32QnSZx4OTHTQGpX2yZybOTgg8JRNdjKi4qMTXzpkf5uRj4BZy1HH5TlG hCc5DqfMbWqB8SISeWKpo624JSyKuUG7JeFrhQdNCIKRztzkdCukcNBBHLMlo4mSstki JkSOleHphrb2hG+C3PmY7VFtlo6M7WzGQRJfiyvu1w8Hg0HWjqwNz0k37ALipllO8g85 VhD0dtgt6kwnhGYbm9+rUDVn3hMZ2y8rlLLu3QuuIgdAaB6fll4AvKtm7/KOYvChZFU+ Ojwn/Vrp9r9h60ModgcAIJ3BagqzgndLguumM5wh83OGRQsi+irzRXNXi0C+IbIac3TH F1VQ== Received: by 10.50.182.229 with SMTP id eh5mr5687207igc.38.1343414946942; Fri, 27 Jul 2012 11:49:06 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.43.93.3 with SMTP id bs3csp46601icc; Fri, 27 Jul 2012 11:49:06 -0700 (PDT) Received: by 10.68.242.7 with SMTP id wm7mr15993502pbc.98.1343414946156; Fri, 27 Jul 2012 11:49:06 -0700 (PDT) Received: from e34.co.us.ibm.com (e34.co.us.ibm.com. [32.97.110.152]) by mx.google.com with ESMTPS id tc8si5790929pbc.144.2012.07.27.11.49.05 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 27 Jul 2012 11:49:06 -0700 (PDT) Received-SPF: neutral (google.com: 32.97.110.152 is neither permitted nor denied by best guess record for domain of john.stultz@linaro.org) client-ip=32.97.110.152; Authentication-Results: mx.google.com; spf=neutral (google.com: 32.97.110.152 is neither permitted nor denied by best guess record for domain of john.stultz@linaro.org) smtp.mail=john.stultz@linaro.org Received: from /spool/local by e34.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 27 Jul 2012 12:49:05 -0600 Received: from d03dlp03.boulder.ibm.com (9.17.202.179) by e34.co.us.ibm.com (192.168.1.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Fri, 27 Jul 2012 12:49:03 -0600 Received: from d03relay01.boulder.ibm.com (d03relay01.boulder.ibm.com [9.17.195.226]) by d03dlp03.boulder.ibm.com (Postfix) with ESMTP id 021A019D8040; Fri, 27 Jul 2012 18:48:59 +0000 (WET) Received: from d03av06.boulder.ibm.com (d03av06.boulder.ibm.com [9.17.195.245]) by d03relay01.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q6RImhtl114324; Fri, 27 Jul 2012 12:48:43 -0600 Received: from d03av06.boulder.ibm.com (loopback [127.0.0.1]) by d03av06.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q6RInj2j011117; Fri, 27 Jul 2012 12:49:45 -0600 Received: from kernel-pok.stglabs.ibm.com (kernel.stglabs.ibm.com [9.114.214.19]) by d03av06.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q6RInUr0010520; Fri, 27 Jul 2012 12:49:44 -0600 From: John Stultz To: lkml Cc: John Stultz , Ingo Molnar , Peter Zijlstra , Richard Cochran , Prarit Bhargava , Thomas Gleixner Subject: [PATCH 4/5] time: Cleanup offs_real/wall_to_mono and offs_boot/total_sleep_time updates Date: Fri, 27 Jul 2012 14:48:12 -0400 Message-Id: <1343414893-45779-5-git-send-email-john.stultz@linaro.org> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1343414893-45779-1-git-send-email-john.stultz@linaro.org> References: <1343414893-45779-1-git-send-email-john.stultz@linaro.org> X-Content-Scanned: Fidelis XPS MAILER x-cbid: 12072718-1780-0000-0000-000007CDCA33 X-IBM-ISS-SpamDetectors: X-IBM-ISS-DetailInfo: BY=3.00000288; HX=3.00000193; KW=3.00000007; PH=3.00000001; SC=3.00000006; SDB=6.00160306; UDB=6.00036212; UTC=2012-07-27 18:49:04 X-Gm-Message-State: ALoCoQnlFVrMIeYdvFUFkPD6JMFd9C+luTU3zseMvDs3o/l5WE1nrFPR6K5+GradQjrB8ZZ/rp87 For performance reasons, we maintain ktime_t based duplicates of wall_to_monotonic (offs_real) and total_sleep_time (offs_boot). Since large problems could occur (such as the resume regression on 3.5-rc7, or the leapsecond hrtimer issue) if these value pairs were to be inconsistently updated, this patch this cleans up how we modify these value pairs to ensure we are always consistent. As a side-effect this is also more efficient as we only caulculate the duplicate values when they are changed, rather then every update_wall_time call. This also provides WARN_ONs to detect if future changes break the invariants. Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Richard Cochran Cc: Prarit Bhargava Cc: Thomas Gleixner Signed-off-by: John Stultz --- kernel/time/timekeeping.c | 91 +++++++++++++++++++++++++++------------------ 1 file changed, 55 insertions(+), 36 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 8d8636f..c4061f5 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -65,14 +65,14 @@ struct timekeeper { * used instead. */ struct timespec wall_to_monotonic; - /* time spent in suspend */ - struct timespec total_sleep_time; - /* The raw monotonic time for the CLOCK_MONOTONIC_RAW posix clock. */ - struct timespec raw_time; /* Offset clock monotonic -> clock realtime */ ktime_t offs_real; + /* time spent in suspend */ + struct timespec total_sleep_time; /* Offset clock monotonic -> clock boottime */ ktime_t offs_boot; + /* The raw monotonic time for the CLOCK_MONOTONIC_RAW posix clock. */ + struct timespec raw_time; /* Seqlock for all timekeeper values */ seqlock_t lock; }; @@ -117,6 +117,32 @@ static void tk_xtime_add(struct timekeeper *tk, const struct timespec *ts) tk->xtime_nsec += ts->tv_nsec << tk->shift; } +static void tk_set_wall_to_mono(struct timekeeper *tk, struct timespec wtm) +{ + struct timespec tmp; + + /* + * Verify consistency of: offset_real = -wall_to_monotonic + * before modifying anything + */ + set_normalized_timespec(&tmp, -tk->wall_to_monotonic.tv_sec, + -tk->wall_to_monotonic.tv_nsec); + WARN_ON_ONCE(tk->offs_real.tv64 != timespec_to_ktime(tmp).tv64); + tk->wall_to_monotonic = wtm; + set_normalized_timespec(&tmp, -wtm.tv_sec, -wtm.tv_nsec); + tk->offs_real = timespec_to_ktime(tmp); +} + +static void tk_set_sleep_time(struct timekeeper *tk, struct timespec t) +{ + /* Verify consistency before modifying */ + WARN_ON_ONCE(tk->offs_boot.tv64 != + timespec_to_ktime(tk->total_sleep_time).tv64); + + tk->total_sleep_time = t; + tk->offs_boot = timespec_to_ktime(t); +} + /** * timekeeper_setup_internals - Set up internals to use clocksource clock. * @@ -217,14 +243,6 @@ static inline s64 timekeeping_get_ns_raw(struct timekeeper *tk) return nsec + arch_gettimeoffset(); } -static void update_rt_offset(struct timekeeper *tk) -{ - struct timespec tmp, *wtm = &tk->wall_to_monotonic; - - set_normalized_timespec(&tmp, -wtm->tv_sec, -wtm->tv_nsec); - tk->offs_real = timespec_to_ktime(tmp); -} - /* must hold write on timekeeper.lock */ static void timekeeping_update(struct timekeeper *tk, bool clearntp) { @@ -234,7 +252,6 @@ static void timekeeping_update(struct timekeeper *tk, bool clearntp) tk->ntp_error = 0; ntp_clear(); } - update_rt_offset(tk); xt = tk_xtime(tk); update_vsyscall(&xt, &tk->wall_to_monotonic, tk->clock, tk->mult); } @@ -419,8 +436,8 @@ int do_settimeofday(const struct timespec *tv) ts_delta.tv_sec = tv->tv_sec - xt.tv_sec; ts_delta.tv_nsec = tv->tv_nsec - xt.tv_nsec; - timekeeper.wall_to_monotonic = - timespec_sub(timekeeper.wall_to_monotonic, ts_delta); + tk_set_wall_to_mono(&timekeeper, + timespec_sub(timekeeper.wall_to_monotonic, ts_delta)); tk_set_xtime(&timekeeper, tv); @@ -454,8 +471,8 @@ int timekeeping_inject_offset(struct timespec *ts) tk_xtime_add(&timekeeper, ts); - timekeeper.wall_to_monotonic = - timespec_sub(timekeeper.wall_to_monotonic, *ts); + tk_set_wall_to_mono(&timekeeper, + timespec_sub(timekeeper.wall_to_monotonic, *ts)); timekeeping_update(&timekeeper, true); @@ -621,7 +638,7 @@ void __init timekeeping_init(void) { struct clocksource *clock; unsigned long flags; - struct timespec now, boot; + struct timespec now, boot, tmp; read_persistent_clock(&now); read_boot_clock(&boot); @@ -642,23 +659,19 @@ void __init timekeeping_init(void) if (boot.tv_sec == 0 && boot.tv_nsec == 0) boot = tk_xtime(&timekeeper); - set_normalized_timespec(&timekeeper.wall_to_monotonic, - -boot.tv_sec, -boot.tv_nsec); - update_rt_offset(&timekeeper); - timekeeper.total_sleep_time.tv_sec = 0; - timekeeper.total_sleep_time.tv_nsec = 0; + set_normalized_timespec(&tmp, -boot.tv_sec, -boot.tv_nsec); + tk_set_wall_to_mono(&timekeeper, tmp); + + tmp.tv_sec = 0; + tmp.tv_nsec = 0; + tk_set_sleep_time(&timekeeper, tmp); + write_sequnlock_irqrestore(&timekeeper.lock, flags); } /* time in seconds when suspend began */ static struct timespec timekeeping_suspend_time; -static void update_sleep_time(struct timespec t) -{ - timekeeper.total_sleep_time = t; - timekeeper.offs_boot = timespec_to_ktime(t); -} - /** * __timekeeping_inject_sleeptime - Internal function to add sleep interval * @delta: pointer to a timespec delta value @@ -674,10 +687,9 @@ static void __timekeeping_inject_sleeptime(struct timekeeper *tk, "sleep delta value!\n"); return; } - tk_xtime_add(tk, delta); - tk->wall_to_monotonic = timespec_sub(tk->wall_to_monotonic, *delta); - update_sleep_time(timespec_add(tk->total_sleep_time, *delta)); + tk_set_wall_to_mono(tk, timespec_sub(tk->wall_to_monotonic, *delta)); + tk_set_sleep_time(tk, timespec_add(tk->total_sleep_time, *delta)); } /** @@ -1018,11 +1030,18 @@ static inline void accumulate_nsecs_to_secs(struct timekeeper *tk) /* Figure out if its a leap sec and apply if needed */ leap = second_overflow(tk->xtime_sec); - tk->xtime_sec += leap; - tk->wall_to_monotonic.tv_sec -= leap; - if (leap) - clock_was_set_delayed(); + if (unlikely(leap)) { + struct timespec ts; + + tk->xtime_sec += leap; + ts.tv_sec = leap; + ts.tv_nsec = 0; + tk_set_wall_to_mono(tk, + timespec_sub(tk->wall_to_monotonic, ts)); + + clock_was_set_delayed(); + } } }