From patchwork Sun Jul 1 09:36:11 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: john stultz X-Patchwork-Id: 9719 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 7A3A723E01 for ; Sun, 1 Jul 2012 09:37:20 +0000 (UTC) Received: from mail-ob0-f180.google.com (mail-ob0-f180.google.com [209.85.214.180]) by fiordland.canonical.com (Postfix) with ESMTP id 19D19A1831D for ; Sun, 1 Jul 2012 09:37:19 +0000 (UTC) Received: by obbuo19 with SMTP id uo19so7089506obb.11 for ; Sun, 01 Jul 2012 02:37:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf:from:to:cc :subject:date:message-id:x-mailer:x-cbid:x-gm-message-state; bh=0H0Dzef5hDBqPytqgVNB111Q9+0ZGODp+Q7Ce5WBXT4=; b=hoYGcjc1O88xwhgmhdjggYsL6rrG/iIUOqXDfp66ApRAS4L6vW5OkLe/NdfyuyTqMN WA1zrgyzeiBDDRl8N8miciQRakJdbjtMMFxitxu5vI5sGEPGd5TgURMydRR16uyioNRV AuUbWiiOLnMvuQ3yWq9Z6YdC5RxUICeNNOVe62jWKPEF/3bfDRujdGTaxU9PoRqU6n+h MUpqBiHC2GdFDzwT0Q90I8U4aYxYsxFVeAozM+Qebk4ISsSoj1z42HDzBJMaLWFymzFH 02KUm5Cc/Ya7jhQqqkDiT5IinQTVRYujwK10JYTTc4bT+dRGSm0oOn4VBcZuPU15cPm1 9vBA== Received: by 10.50.46.232 with SMTP id y8mr4799452igm.57.1341135439252; Sun, 01 Jul 2012 02:37:19 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.231.24.148 with SMTP id v20csp7185ibb; Sun, 1 Jul 2012 02:37:18 -0700 (PDT) Received: by 10.50.207.40 with SMTP id lt8mr4863103igc.16.1341135438179; Sun, 01 Jul 2012 02:37:18 -0700 (PDT) Received: from e7.ny.us.ibm.com (e7.ny.us.ibm.com. [32.97.182.137]) by mx.google.com with ESMTPS id wo6si8218441igc.57.2012.07.01.02.37.17 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 01 Jul 2012 02:37:18 -0700 (PDT) Received-SPF: pass (google.com: domain of johnstul@us.ibm.com designates 32.97.182.137 as permitted sender) client-ip=32.97.182.137; Authentication-Results: mx.google.com; spf=pass (google.com: domain of johnstul@us.ibm.com designates 32.97.182.137 as permitted sender) smtp.mail=johnstul@us.ibm.com Received: from /spool/local by e7.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Sun, 1 Jul 2012 05:37:17 -0400 Received: from d01relay04.pok.ibm.com (9.56.227.236) by e7.ny.us.ibm.com (192.168.1.107) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Sun, 1 Jul 2012 05:37:15 -0400 Received: from d03av01.boulder.ibm.com (d03av01.boulder.ibm.com [9.17.195.167]) by d01relay04.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q619bEdo414702 for ; Sun, 1 Jul 2012 05:37:14 -0400 Received: from d03av01.boulder.ibm.com (loopback [127.0.0.1]) by d03av01.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q619bDJk011884 for ; Sun, 1 Jul 2012 03:37:13 -0600 Received: from kernel.stglabs.ibm.com (kernel.stglabs.ibm.com [9.114.214.19]) by d03av01.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q619bDCB011875; Sun, 1 Jul 2012 03:37:13 -0600 From: John Stultz To: Linux Kernel Mailing List Cc: John Stultz , stable@vger.kernel.org, Thomas Gleixner Subject: [PATCH] [RFC] Potential fix for leapsecond caused futex related load spikes Date: Sun, 1 Jul 2012 05:36:11 -0400 Message-Id: <1341135371-45034-1-git-send-email-johnstul@us.ibm.com> X-Mailer: git-send-email 1.7.9.5 x-cbid: 12070109-5806-0000-0000-000016D8D1B9 X-Gm-Message-State: ALoCoQnswr5G0matkVzkOGt+VE2PRd/L4tnJxpvIjRYNlRdOpMYn4eolSkcJVoTBsmZd/qqYTpwA As widely reported on the internet today, some Linux systems after the leapsecond was inserted are experiencing futex related load spikes (usually connected to MySQL, Firefox, Thunderbird, Java, etc). An apparent for this issue workaround is running: $ date -s "`date`" Credit: http://www.sheeri.com/content/mysql-and-leap-second-high-cpu-and-fix I believe this issue is due to the leapsecond being added without calling clock_was_set() to notify the hrtimer subsystem of the change. (Although I've not yet chased all the way down to the hrtimer code to validate exactly what's going on there). The workaround functions as it forces a clock_was_set() call from settimeofday(). This fix adds some extra logic to track when a leapsecond is added from update_wall_time() and calls clock_was_set() once the timekeeper.lock is released. I've been able to reproduce the load spike using Thunderbird when triggering a leap second and with this patch the issue did not crop up. NOTE: Some reports have been of a hard hang right at or before the leapsecond. I've not been able to reproduce or diagnose this, so this fix does not likely address the reported hard hangs (unless they end up being connected to the futex/hrtimer issue). It had been a long day before I heard about this issue, so my brain is a little mushy right now. Reviews and extra testing would be greatly appreciated. CC: stable@vger.kernel.org CC: Thomas Gleixner Reported-by: Jan Engelhardt Signed-off-by: John Stultz --- kernel/time/timekeeping.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 6f46a00..e5da44f 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -942,7 +942,7 @@ static void timekeeping_adjust(s64 offset) * * Returns the unconsumed cycles. */ -static cycle_t logarithmic_accumulation(cycle_t offset, int shift) +static cycle_t logarithmic_accumulation(cycle_t offset, int shift, int* clockset) { u64 nsecps = (u64)NSEC_PER_SEC << timekeeper.shift; u64 raw_nsecs; @@ -963,6 +963,8 @@ static cycle_t logarithmic_accumulation(cycle_t offset, int shift) leap = second_overflow(timekeeper.xtime.tv_sec); timekeeper.xtime.tv_sec += leap; timekeeper.wall_to_monotonic.tv_sec -= leap; + if (leap) + *clockset = 1; } /* Accumulate raw time */ @@ -994,6 +996,7 @@ static void update_wall_time(void) struct clocksource *clock; cycle_t offset; int shift = 0, maxshift; + int clockset = 0; unsigned long flags; write_seqlock_irqsave(&timekeeper.lock, flags); @@ -1026,7 +1029,7 @@ static void update_wall_time(void) maxshift = (64 - (ilog2(ntp_tick_length())+1)) - 1; shift = min(shift, maxshift); while (offset >= timekeeper.cycle_interval) { - offset = logarithmic_accumulation(offset, shift); + offset = logarithmic_accumulation(offset, shift, &clockset); if(offset < timekeeper.cycle_interval<