From patchwork Wed Sep 11 23:50:56 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Stultz X-Patchwork-Id: 19983 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ve0-f199.google.com (mail-ve0-f199.google.com [209.85.128.199]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 60487248D9 for ; Wed, 11 Sep 2013 23:51:40 +0000 (UTC) Received: by mail-ve0-f199.google.com with SMTP id db12sf10359004veb.2 for ; Wed, 11 Sep 2013 16:51:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:x-original-sender:x-original-authentication-results :precedence:mailing-list:list-id:list-post:list-help:list-archive :list-unsubscribe; bh=Oc1HyOXx8bUk/WnnzFNwvuimdVdjljoYHI3A5scSxI0=; b=Z6y9t/krL6k7RrnYjLqhtQgybAJvodUGww9lvI8IcItUmzovm1FMrMl18bc7NBj4KX sTe+fA+bU00BQOZbl2aZtodCCwimpJp4I7TOBybY1hn2FMABNmqYXc3v1QK6dqY8PGWK JbGC3d5LVgychj8Yg1tI8+UyachBjdgCXynGd/lgIqWAhlPHnbOf/F2ekMxL2ugC9Oom 3l53GYdiPtySxxJDOYiXIup1Yw3DA9ap9cHJluQan6U2SUGsSoJRAqFnISt+N7jYOmR3 45YAdL/vVZdeZJGhFAgjBEKoRjvGqEG4Cs7CGyJd4dPmT4MusxZOp61aIgIMvLn9ru2D As/w== X-Gm-Message-State: ALoCoQk00Vi7PF3M701MUjCKJ1j7bCZr7cLaituNfh6rPMeHOtq2NScvYQF2cux6JMhOtzOejD5O X-Received: by 10.236.94.205 with SMTP id n53mr1688232yhf.44.1378943499921; Wed, 11 Sep 2013 16:51:39 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.49.47.82 with SMTP id b18ls520570qen.53.gmail; Wed, 11 Sep 2013 16:51:39 -0700 (PDT) X-Received: by 10.221.64.17 with SMTP id xg17mr3708446vcb.5.1378943499643; Wed, 11 Sep 2013 16:51:39 -0700 (PDT) Received: from mail-vc0-f169.google.com (mail-vc0-f169.google.com [209.85.220.169]) by mx.google.com with ESMTPS id om2si227379vec.113.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 11 Sep 2013 16:51:39 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.220.169 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.220.169; Received: by mail-vc0-f169.google.com with SMTP id ib11so6780555vcb.14 for ; Wed, 11 Sep 2013 16:51:39 -0700 (PDT) X-Received: by 10.58.246.8 with SMTP id xs8mr3768581vec.9.1378943499341; Wed, 11 Sep 2013 16:51:39 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.220.174.196 with SMTP id u4csp85648vcz; Wed, 11 Sep 2013 16:51:38 -0700 (PDT) X-Received: by 10.66.146.42 with SMTP id sz10mr6524980pab.100.1378943498440; Wed, 11 Sep 2013 16:51:38 -0700 (PDT) Received: from mail-pb0-f50.google.com (mail-pb0-f50.google.com [209.85.160.50]) by mx.google.com with ESMTPS id pz2si3778402pac.231.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 11 Sep 2013 16:51:38 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.160.50 is neither permitted nor denied by best guess record for domain of john.stultz@linaro.org) client-ip=209.85.160.50; Received: by mail-pb0-f50.google.com with SMTP id uo5so9714682pbc.23 for ; Wed, 11 Sep 2013 16:51:37 -0700 (PDT) X-Received: by 10.66.142.42 with SMTP id rt10mr6327645pab.1.1378943497497; Wed, 11 Sep 2013 16:51:37 -0700 (PDT) Received: from localhost.localdomain (c-67-170-153-23.hsd1.or.comcast.net. [67.170.153.23]) by mx.google.com with ESMTPSA id go4sm600939pbb.15.1969.12.31.16.00.00 (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 11 Sep 2013 16:51:36 -0700 (PDT) From: John Stultz To: LKML Cc: John Stultz , Ingo Molnar , Thomas Gleixner , Gerlando Falauto , Mathieu Desnoyers , stable Subject: [PATCH] timekeeping: Fix HRTICK related deadlock from ntp lock changes Date: Wed, 11 Sep 2013 16:50:56 -0700 Message-Id: <1378943457-27314-1-git-send-email-john.stultz@linaro.org> X-Mailer: git-send-email 1.8.1.2 X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: john.stultz@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.220.169 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , It was reported that when HRTICK is enabled, its possible to trigger system deadlocks. These were hard to reproduce, as HRTICK has been broken in the past, but seemed to be connected to the timekeeping_seq lock. Since seqlock/seqcount's aren't supported w/ lockdep, I added some extra spinlock based locking and triggered the following lockdep output: [ 15.849182] ntpd/4062 is trying to acquire lock: [ 15.849765] (&(&pool->lock)->rlock){..-...}, at: [] __queue_work+0x145/0x480 [ 15.850051] [ 15.850051] but task is already holding lock: [ 15.850051] (timekeeper_lock){-.-.-.}, at: [] do_adjtimex+0x7f/0x100 [ 15.850051] Chain exists of: &(&pool->lock)->rlock --> &p->pi_lock --> timekeeper_lock [ 15.850051] Possible unsafe locking scenario: [ 15.850051] [ 15.850051] CPU0 CPU1 [ 15.850051] ---- ---- [ 15.850051] lock(timekeeper_lock); [ 15.850051] lock(&p->pi_lock); [ 15.850051] lock(timekeeper_lock); [ 15.850051] lock(&(&pool->lock)->rlock); [ 15.850051] [ 15.850051] *** DEADLOCK *** The deadlock was introduced by 06c017fdd4dc48451a (timekeeping: Hold timekeepering locks in do_adjtimex and hardpps) in 3.10 This patch avoids this deadlock, by moving the call to schedule_delayed_work() outside of the timekeeper lock critical section. Cc: Ingo Molnar Cc: Thomas Gleixner Cc: Gerlando Falauto Cc: Mathieu Desnoyers Cc: stable #3.11, 3.10 Reported-by: Gerlando Falauto Tested-by: Lin Ming Signed-off-by: John Stultz --- include/linux/timex.h | 1 + kernel/time/ntp.c | 6 ++---- kernel/time/timekeeping.c | 2 ++ 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/include/linux/timex.h b/include/linux/timex.h index b3726e6..dd3edd7 100644 --- a/include/linux/timex.h +++ b/include/linux/timex.h @@ -141,6 +141,7 @@ extern int do_adjtimex(struct timex *); extern void hardpps(const struct timespec *, const struct timespec *); int read_current_timer(unsigned long *timer_val); +void ntp_notify_cmos_timer(void); /* The clock frequency of the i8253/i8254 PIT */ #define PIT_TICK_RATE 1193182ul diff --git a/kernel/time/ntp.c b/kernel/time/ntp.c index 8f5b3b9..bb22151 100644 --- a/kernel/time/ntp.c +++ b/kernel/time/ntp.c @@ -516,13 +516,13 @@ static void sync_cmos_clock(struct work_struct *work) schedule_delayed_work(&sync_cmos_work, timespec_to_jiffies(&next)); } -static void notify_cmos_timer(void) +void ntp_notify_cmos_timer(void) { schedule_delayed_work(&sync_cmos_work, 0); } #else -static inline void notify_cmos_timer(void) { } +void ntp_notify_cmos_timer(void) { } #endif @@ -687,8 +687,6 @@ int __do_adjtimex(struct timex *txc, struct timespec *ts, s32 *time_tai) if (!(time_status & STA_NANO)) txc->time.tv_usec /= NSEC_PER_USEC; - notify_cmos_timer(); - return result; } diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 48b9fff..947ba25 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -1703,6 +1703,8 @@ int do_adjtimex(struct timex *txc) write_seqcount_end(&timekeeper_seq); raw_spin_unlock_irqrestore(&timekeeper_lock, flags); + ntp_notify_cmos_timer(); + return ret; }