From patchwork Tue Jul 12 17:11:13 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viresh Kumar X-Patchwork-Id: 71877 Delivered-To: patch@linaro.org Received: by 10.140.29.52 with SMTP id a49csp581318qga; Tue, 12 Jul 2016 10:11:18 -0700 (PDT) X-Received: by 10.66.15.138 with SMTP id x10mr5651805pac.92.1468343478516; Tue, 12 Jul 2016 10:11:18 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v63si4470696pfv.160.2016.07.12.10.11.18; Tue, 12 Jul 2016 10:11:18 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-pm-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org; spf=pass (google.com: best guess record for domain of linux-pm-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org; dmarc=fail (p=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752596AbcGLRLQ (ORCPT + 14 others); Tue, 12 Jul 2016 13:11:16 -0400 Received: from mail-pa0-f45.google.com ([209.85.220.45]:33337 "EHLO mail-pa0-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752450AbcGLRLP (ORCPT ); Tue, 12 Jul 2016 13:11:15 -0400 Received: by mail-pa0-f45.google.com with SMTP id ks6so8453480pab.0 for ; Tue, 12 Jul 2016 10:11:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=eGIBzxU5GxsBksIMN3dVbgxncgcxTSgI4w+JyBYzyR8=; b=de8LxNoffR1xNjtlBTEVOdyyOjK+g12dE+XPRcbov5dPMOuuPjR8pH18lOfYWLP+JM 6/x4JeCWpMKyrkzRbe3STFapullRxEBZtLbgTj0NA9Rf9h+vXvzchjesZa6ww+T6NOxw iaNDGdxHYMd2om+ZC2+7p6L7MaIOZ5QdiLkA8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=eGIBzxU5GxsBksIMN3dVbgxncgcxTSgI4w+JyBYzyR8=; b=czzqbst0XFwb2h9F5NRc25JN9bzsQVz3gGfamkfAtH9LMylssiKwG42wxH+79jj7d5 /hDxpoWIIcLzsnS5YRA5b+KfxB3ivvPQ7xzKIvOAIW/0UBHcHhy9Ui6jwRX1kqmkLCRR DXrnWvFWNBBPQPBn2YQZBzi6/dO4P+Hnk2MCmlj6yGEyCx8oyMcqTj0Kyw2KySTcA9XX zg/KKin9+gupInPFwH2aNZsWwUCKsoxQey7jubgrLajCZHimfAkQA4E0mPjTeNaHVzlt KJu/RFNH3tfseTH6SSB00RscqcZPqq38QjbKs3FEik7qVLcrOWHbCZUQ4TeQ6klZhZaG +Rcw== X-Gm-Message-State: ALyK8tKYGCIu10cpefUhKeWlrVfsP0pvtSEdlGWOb4Wjqo8M38BbS879G9YWOT5HgRS2xiLj X-Received: by 10.66.123.77 with SMTP id ly13mr5865782pab.88.1468343474547; Tue, 12 Jul 2016 10:11:14 -0700 (PDT) Received: from localhost ([104.132.1.108]) by smtp.gmail.com with ESMTPSA id n80sm445962pfi.19.2016.07.12.10.11.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Jul 2016 10:11:13 -0700 (PDT) Date: Tue, 12 Jul 2016 10:11:13 -0700 From: Viresh Kumar To: Petr Mladek , rjw@rjwysocki.net Cc: Sergey Senozhatsky , Jan Kara , Sergey Senozhatsky , Tejun Heo , Greg Kroah-Hartman , Linux Kernel Mailing List , vlevenetz@mm-sol.com, vaibhav.hiremath@linaro.org, alex.elder@linaro.org, johan@kernel.org, akpm@linux-foundation.org, rostedt@goodmis.org, linux-pm@vger.kernel.org Subject: Re: [Query] Preemption (hogging) of the work handler Message-ID: <20160712171113.GD4695@ubuntu> References: <20160701165959.GR12473@ubuntu> <20160701172232.GD28719@htj.duckdns.org> <20160706182842.GS2671@ubuntu> <20160711102603.GI12410@quack2.suse.cz> <20160711154438.GA528@swordfish> <20160711223501.GI4695@ubuntu> <20160712093805.GA498@swordfish> <20160712125243.GA8597@pathway.suse.cz> <20160712131203.GN4695@ubuntu> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20160712131203.GN4695@ubuntu> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org On 12-07-16, 06:12, Viresh Kumar wrote: > Yeah, so I tried debugging this more and I am able to get printing > done to just before arch_suspend_disable_irqs() in suspend.c and then > it stops because of the async nature. > > I get to this point for both successful suspend/resume (where system > resumes back successfully) and in the bad case (where the system just > hangs/crashes). > > FWIW, I also tried commenting out following in suspend_enter(): > > error = suspend_ops->enter(state); > > so that the system doesn't go into suspend at all, and just resume > back immediately (similar to TEST_CORE) and I saw the hang/crash then > as well one of the times. So I tried it cleanly without any local hacks using: echo core > /sys/power/pm_test and I still see the problem, so whatever happens, happens before putting the system into complete suspend. FWIW, I also tried this hacky thing: i.e. I disabled async-printk after interrupts are disabled on the last running CPU (0) and enabled it again before enabling interrupts back. This FIXES the hangs for me :) I don't think its a crash but some sort of deadlock in async printk thread because of the state it was left in before we offlined all other CPUs and disabled interrupts on the local one. -- viresh -- To unsubscribe from this list: send the line "unsubscribe linux-pm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/kernel/power/suspend.c b/kernel/power/suspend.c index bc71478fac26..045ebc88fe08 100644 --- a/kernel/power/suspend.c +++ b/kernel/power/suspend.c @@ -170,6 +170,7 @@ void __attribute__ ((weak)) arch_suspend_enable_irqs(void) * * This function should be called after devices have been suspended. */ +extern bool printk_sync_suspended; static int suspend_enter(suspend_state_t state, bool *wakeup) { char suspend_abort[MAX_SUSPEND_ABORT_LEN]; @@ -218,6 +219,7 @@ static int suspend_enter(suspend_state_t state, bool *wakeup) } arch_suspend_disable_irqs(); + printk_sync_suspended = true; BUG_ON(!irqs_disabled()); error = syscore_suspend(); @@ -237,6 +239,7 @@ static int suspend_enter(suspend_state_t state, bool *wakeup) syscore_resume(); } + printk_sync_suspended = false; arch_suspend_enable_irqs(); BUG_ON(irqs_disabled()); diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index 46bb017ac2c9..187054074b96 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -293,6 +293,7 @@ static u32 log_buf_len = __LOG_BUF_LEN; /* Control whether printing to console must be synchronous. */ static bool __read_mostly printk_sync = false; +bool printk_sync_suspended = false; /* Printing kthread for async printk */ static struct task_struct *printk_kthread; /* When `true' printing thread has messages to print */ @@ -300,7 +301,7 @@ static bool printk_kthread_need_flush_console; static inline bool can_printk_async(void) { - return !printk_sync && printk_kthread; + return !printk_sync && !printk_sync_suspended && printk_kthread; } /* Return log buffer address */