From patchwork Tue Jul 7 15:05:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sumit Garg X-Patchwork-Id: 235014 Delivered-To: patches@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp997385ilg; Tue, 7 Jul 2020 08:05:47 -0700 (PDT) X-Received: by 2002:a17:902:ed13:: with SMTP id b19mr37196524pld.294.1594134347366; Tue, 07 Jul 2020 08:05:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594134347; cv=none; d=google.com; s=arc-20160816; b=B2+rQV7gkA4QKDO4SeAgaaaskYOlVptA8CC2XpTcYEdptgPhQ7cUbr6R2tO1RHbFR4 LCks07f8p9uktplZkMizf7C1sHdseSaTnkWi6+ALbDkD5eFU0XthBoUbYo89uR/dSwck Nj1W2zjl9IckWoDYGn4fibg4Ll9IiJIjOaP6VWShClfxviWD7xG8NEnaOy6CAQW3t/RD kAlepMGT6an7I+88dHhf0nzEcf8SHNaLAZn/Ko7zXU8O9ccs2UN+QWVe26X9dueZoqsB HUFxarfGASmdE4ZQf0NPd5ZW2zHUuJYwbLfCatXCAp/kMGYTfilUUPAbR+eTSyHddIOh K0Xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from:dkim-signature; bh=wxHIhBMETpDFFxgUbCHxQv41vIYY+Vv4oIYlRuwNjXQ=; b=rodsNlhi6IqbyVxnwY4yRjISKzvoFwyvy/gROh1EoMJLSI410Fl+VjY/DH1N4t4L6j E5aKJ2mOWe9JM/hiVeA2h/gKl+IjckEPvii+TKXqNBX24V2UdhvEko6thN2kMLhJncvm 4fbsTa3G0GWVXJu2+ny2LNpVJjGJuZGksibxGFy3kUce3Dj+IX1/lRl9GSIGz9oxuq4g bf/uB5wpzxGVpKfL150IklO/jnMSxSP6K6nhIXDfqKF3YMp+Y6O0KV+ubTCiXyoVl65R iVWoSys6opGA7aKedUKvkXV2jxUpwhkkCCATDblbPpgIg6CHURSRzJf+jdRfpN52J+Tg 8vpQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=N5XP1VDK; spf=pass (google.com: domain of sumit.garg@linaro.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=sumit.garg@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id u1sor3630678pjy.2.2020.07.07.08.05.47 for (Google Transport Security); Tue, 07 Jul 2020 08:05:47 -0700 (PDT) Received-SPF: pass (google.com: domain of sumit.garg@linaro.org designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=N5XP1VDK; spf=pass (google.com: domain of sumit.garg@linaro.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=sumit.garg@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=wxHIhBMETpDFFxgUbCHxQv41vIYY+Vv4oIYlRuwNjXQ=; b=N5XP1VDKrzTRRTQffSMCEiDl4kJa6K4DJFemEnlFJC/nh/Zikof+nCDhQtrGEXv1/n L8NzMDXU0Ni+b9fu6FQwANFVm1AAdbg7Sz1FMoHeW76vhUSw0KrsMpx25N+FA0MZnOCI kfw0cGJKi8Gs36wGpnCg1OTW3G0W/THROSqPxzt4Dnc1eOfIVWum9ZKUCGDZwoo90wMj V+Q4qYnsgJIsKHjISZlNEDJ3cbWzLKSYwbiI0yXuHViLOPTOg/pbKEItZCJb31hRpqE3 d7lLFv3v7BiUTvkmgV9oxFWNfXMqrTbSC/Jf6PlkYLzmfAa8bxDIV6j12ctgiFxyU23o KHqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=wxHIhBMETpDFFxgUbCHxQv41vIYY+Vv4oIYlRuwNjXQ=; b=OxBvKXiVUMvnvapxA9qBM90yH0uNlgnW+qdwWGPa0ZV7x3vS6DI4vrCTtUmFL5WMV6 c3qhp6sDaHAayAFMKwudUiGFahJMkYeSVHzXAkn6fn+xc0uLFF/UfOH4eBzz+TcVKZpa ncGDBxIkhky1qdhvJ+IV0lyUxjQ1vbqaiJch2BtJi5GKguE1FmMreSMfzU6+Jpw6OIvB zr1GC+DK8F5g52N3sC7gTCPPuaVzYrbF9D8ITGcoQ5c9sJgRWloX3eJQhXUQCOji38PE Q6Am+h4JL5W8hywNGRsjHcqz+5bJ2e+IYBcCnrD3ngk6jURFQQmuLRbwJAnm2Fm+K1mb GDsQ== X-Gm-Message-State: AOAM5309Bs7gsaHqXemLMd3v7muD0hc/+EuIFSTrdL2b048a+8VsOG// aHzXCxIIgc64zW1VeLJOn3YsuCyo X-Google-Smtp-Source: ABdhPJyW8kB21rCd0YAA3f6lbpfx/X8mmVWf7a3lkugjjCIFylJsJ6NalMZ+713zVoLeRkNpBSVPqA== X-Received: by 2002:a17:90b:120a:: with SMTP id gl10mr4615735pjb.44.1594134346917; Tue, 07 Jul 2020 08:05:46 -0700 (PDT) Return-Path: Received: from localhost.localdomain ([117.252.67.186]) by smtp.gmail.com with ESMTPSA id y18sm23400656pff.10.2020.07.07.08.05.44 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Jul 2020 08:05:46 -0700 (PDT) From: Sumit Garg To: daniel.thompson@linaro.org Cc: patches@linaro.org, Sumit Garg Subject: [RFC INTERNAL 1/2] tty/sysrq: Make sysrq handler NMI aware Date: Tue, 7 Jul 2020 20:35:24 +0530 Message-Id: <1594134325-7912-1-git-send-email-sumit.garg@linaro.org> X-Mailer: git-send-email 2.7.4 With the advent of pseudo NMIs on arm64 its been possible to raise serial device interrupt as an NMI in polling mode. With that its now possible to do a magic sysrq in NMI context rather than in normal IRQ context. So make sysrq handler NMI aware and allow system requests to be marked as NMI safe which can run directly in NMI context while categorized as not being NMI safe will be queued as irq_work for later processing in normal interrupt context. This feature is especially helpful in case the primary CPU is stuck in deadlock with interrupts disabled won't honour serial device interrupt. So with sysrq running in NMI context is helpful to debug such scenarios. Signed-off-by: Sumit Garg --- drivers/tty/sysrq.c | 36 +++++++++++++++++++++++++++++++++++- include/linux/sysrq.h | 1 + kernel/debug/debug_core.c | 1 + 3 files changed, 37 insertions(+), 1 deletion(-) -- 2.7.4 diff --git a/drivers/tty/sysrq.c b/drivers/tty/sysrq.c index 7c95afa9..97393c7 100644 --- a/drivers/tty/sysrq.c +++ b/drivers/tty/sysrq.c @@ -50,6 +50,8 @@ #include #include #include +#include +#include #include #include @@ -111,6 +113,7 @@ static const struct sysrq_key_op sysrq_loglevel_op = { .help_msg = "loglevel(0-9)", .action_msg = "Changing Loglevel", .enable_mask = SYSRQ_ENABLE_LOG, + .nmi_safe = true, }; #ifdef CONFIG_VT @@ -157,6 +160,7 @@ static const struct sysrq_key_op sysrq_crash_op = { .help_msg = "crash(c)", .action_msg = "Trigger a crash", .enable_mask = SYSRQ_ENABLE_DUMP, + .nmi_safe = true, }; static void sysrq_handle_reboot(int key) @@ -170,6 +174,7 @@ static const struct sysrq_key_op sysrq_reboot_op = { .help_msg = "reboot(b)", .action_msg = "Resetting", .enable_mask = SYSRQ_ENABLE_BOOT, + .nmi_safe = true, }; const struct sysrq_key_op *__sysrq_reboot_op = &sysrq_reboot_op; @@ -217,6 +222,7 @@ static const struct sysrq_key_op sysrq_showlocks_op = { .handler = sysrq_handle_showlocks, .help_msg = "show-all-locks(d)", .action_msg = "Show Locks Held", + .nmi_safe = true, }; #else #define sysrq_showlocks_op (*(const struct sysrq_key_op *)NULL) @@ -289,6 +295,7 @@ static const struct sysrq_key_op sysrq_showregs_op = { .help_msg = "show-registers(p)", .action_msg = "Show Regs", .enable_mask = SYSRQ_ENABLE_DUMP, + .nmi_safe = true, }; static void sysrq_handle_showstate(int key) @@ -326,6 +333,7 @@ static const struct sysrq_key_op sysrq_ftrace_dump_op = { .help_msg = "dump-ftrace-buffer(z)", .action_msg = "Dump ftrace buffer", .enable_mask = SYSRQ_ENABLE_DUMP, + .nmi_safe = true, }; #else #define sysrq_ftrace_dump_op (*(const struct sysrq_key_op *)NULL) @@ -538,6 +546,26 @@ static void __sysrq_put_key_op(int key, const struct sysrq_key_op *op_p) sysrq_key_table[i] = op_p; } +#define SYSRQ_NMI_FIFO_SIZE 64 +static DEFINE_KFIFO(sysrq_nmi_fifo, int, SYSRQ_NMI_FIFO_SIZE); + +static void sysrq_do_nmi_work(struct irq_work *work) +{ + struct sysrq_key_op *op_p; + int key; + + if (!kfifo_len(&sysrq_nmi_fifo)) + return; + + while (kfifo_out(&sysrq_nmi_fifo, &key, 1)) { + op_p = __sysrq_get_key_op(key); + if (op_p) + op_p->handler(key); + } +} + +static DEFINE_IRQ_WORK(sysrq_nmi_work, sysrq_do_nmi_work); + void __handle_sysrq(int key, bool check_mask) { const struct sysrq_key_op *op_p; @@ -568,7 +596,13 @@ void __handle_sysrq(int key, bool check_mask) if (!check_mask || sysrq_on_mask(op_p->enable_mask)) { pr_info("%s\n", op_p->action_msg); console_loglevel = orig_log_level; - op_p->handler(key); + + if (in_nmi() && !op_p->nmi_safe) { + kfifo_in(&sysrq_nmi_fifo, &key, 1); + irq_work_queue(&sysrq_nmi_work); + } else { + op_p->handler(key); + } } else { pr_info("This sysrq operation is disabled.\n"); console_loglevel = orig_log_level; diff --git a/include/linux/sysrq.h b/include/linux/sysrq.h index 3a582ec..630b5b9 100644 --- a/include/linux/sysrq.h +++ b/include/linux/sysrq.h @@ -34,6 +34,7 @@ struct sysrq_key_op { const char * const help_msg; const char * const action_msg; const int enable_mask; + const bool nmi_safe; }; #ifdef CONFIG_MAGIC_SYSRQ diff --git a/kernel/debug/debug_core.c b/kernel/debug/debug_core.c index 9e59347..2b51173 100644 --- a/kernel/debug/debug_core.c +++ b/kernel/debug/debug_core.c @@ -943,6 +943,7 @@ static const struct sysrq_key_op sysrq_dbg_op = { .handler = sysrq_handle_dbg, .help_msg = "debug(g)", .action_msg = "DEBUG", + .nmi_safe = true, }; #endif From patchwork Tue Jul 7 15:05:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sumit Garg X-Patchwork-Id: 235015 Delivered-To: patches@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp997437ilg; Tue, 7 Jul 2020 08:05:50 -0700 (PDT) X-Received: by 2002:aca:d60c:: with SMTP id n12mr3370554oig.167.1594134350516; Tue, 07 Jul 2020 08:05:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594134350; cv=none; d=google.com; s=arc-20160816; b=0i7zLRuOQCtOq0YvMUfdwkN6E+ELV1BVGX+N4LKEen8yZMBJSUVM4rbcPpA1Qv7Vit JLrFFE5nhC9E2UqmVLxNBN9CAd0q5/uHKDNyeaWnC0X6jpSwJTzfgtrbsNVGqgPAXkxy zs8+O2n9Q9CrP5RXddw/O6EJNoXnKbq9HPx1HsN6owV7m8FqaGh9VaIV6LF9AO0ZB/Dg HAJSqR6z0SNDRr+lbqIIPI9B+6AB+S2cDvrMF8ysLHhn6gPKaq+znAJTDSDZwDK12y+S MyxNqkPas7jkecz9+u2oou0Ee+rX69OFWTmXnZWMcH/yN7jm50r1kWlbGXVhUTwWtBtw qy3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=PF+W7HY3w0Eb0axwi89t/ACyKgPGheukJTEkjWnsErY=; b=wwdlRJ3Ulbqs6phfZbdbWDFmUjry3VhhSj0BkdSCkfyYNmHivaAYwQLQ1OMHkbBFuv 4YV0/jT+8XbIdmX5F4GfwNp+7DOd6jS17/1dQRWXRvTIif5zoDd1D3y8dI3jUyxVs/Kb Hr44W4sDT2waPC9O8A4Au0aLql8pkhP6ZyE3sZxTBhBxBErzvRXDdCUhi5WhO8w1ZipH R+9FabGuu8Mg0UCXKQLzg4mjvwMjpdGKEXBpJh6Kce6ighMfwYVE5q4ZbC9+A+UcSgx8 /bzEoogjkeV4HkOAREpQpBao8p5lhbaD6NXmLYY6OvTkS5MEMfoxndJ05PgWsfbh/MpF SDsA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Xu5Hkrc+; spf=pass (google.com: domain of sumit.garg@linaro.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=sumit.garg@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r1sor14094519ote.58.2020.07.07.08.05.50 for (Google Transport Security); Tue, 07 Jul 2020 08:05:50 -0700 (PDT) Received-SPF: pass (google.com: domain of sumit.garg@linaro.org designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Xu5Hkrc+; spf=pass (google.com: domain of sumit.garg@linaro.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=sumit.garg@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=PF+W7HY3w0Eb0axwi89t/ACyKgPGheukJTEkjWnsErY=; b=Xu5Hkrc+9ojHOONv31XWiLSniGIuYs6QuHXRXoawJJ5j4+/EMvsfHA180QORi5e4pH Q93gV2GiftaijagJl+FxLPaLK8UU1Hp84mTmjkZB6QUsbqeHXETMHd8Ic+0Bckh5c9dU lk2AHjqyguflxcPBg8XUC8VGivUGb2g41poTBLZwxgB9/pgEap6SxYMwqnfxCMOEfEyO 6oeVZ5a24Zfkl1rnJqxC7/dRXr6mWd+GGBK6c9WNj9gAh9o+5VXbiTREs1J/9mhx67KF HNS5jViHlqkPEhBqNAMmLzKz/jb0CApRmhpJcZerjj8fSBOJWtQfqxt1gy/vymneVVF/ vGaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=PF+W7HY3w0Eb0axwi89t/ACyKgPGheukJTEkjWnsErY=; b=N/kVaQN3WY12ulfsdjc2boXiFU/14fuAdg44z6D938A0H04G8VDWG2VhSWoxB3jU26 +LqZL+vkcYg+fmDaZ3rd/72m6zTjBZh/ZfhUxR+C6tooM7RoPIdK9DiGUti0zL82Fsr+ eFbYKeEFzOT5WmXCXC/KJd3K6iJcRwH4ZX/HG9r+PINK1BDqsqRLscX0I2DsBqWv35Vz cOvcKatSwYy5YPK40+otFZ3DmOHnDHesqkKBRVFLeWDbNmPul1bbMM795kaDI81mIms2 WX9sqyF5g+alOOMJqV9lnzfyW8a2bOKYTNu9WcPKcZ2oqNiCZ4QPqYyQX1dNkw5HKVYP OC6Q== X-Gm-Message-State: AOAM531EKl3Y64d5TYTjp0/JvBSBAhkzsPg5bkZ06wF3L9fzGktMbpYW /MOXbbmQthiulSumS5RaIo0Wh+ze68I81w== X-Google-Smtp-Source: ABdhPJzbQcy1TqYiN1liLuuWa2Hwfgf+g5ng9a+85OVndUnDNT+JLUGXlxSLv2UnTM4cCk5p4kxvnA== X-Received: by 2002:a17:90a:9a98:: with SMTP id e24mr4829032pjp.141.1594134350085; Tue, 07 Jul 2020 08:05:50 -0700 (PDT) Return-Path: Received: from localhost.localdomain ([117.252.67.186]) by smtp.gmail.com with ESMTPSA id y18sm23400656pff.10.2020.07.07.08.05.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Jul 2020 08:05:49 -0700 (PDT) From: Sumit Garg To: daniel.thompson@linaro.org Cc: patches@linaro.org, Sumit Garg Subject: [RFC INTERNAL 2/2] serial: amba-pl011: Allow uart interrupt to be NMI Date: Tue, 7 Jul 2020 20:35:25 +0530 Message-Id: <1594134325-7912-2-git-send-email-sumit.garg@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1594134325-7912-1-git-send-email-sumit.garg@linaro.org> References: <1594134325-7912-1-git-send-email-sumit.garg@linaro.org> With the advent on pseudo NMIs on arm64 its now possible to request serial interrupt as an NMI. So allow uart interrupt to be requested as an NMI in polling mode. This feaure is especially useful for NMI debugging where the serial device is operating in polling mode. Currently only RX interrupt has been enabled as NMI and TX operates in polling mode which has a performance overhead when compared with TX operating in interrupt mode. Signed-off-by: Sumit Garg --- drivers/tty/serial/amba-pl011.c | 177 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 175 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/drivers/tty/serial/amba-pl011.c b/drivers/tty/serial/amba-pl011.c index 8efd7c2..05bfe9f 100644 --- a/drivers/tty/serial/amba-pl011.c +++ b/drivers/tty/serial/amba-pl011.c @@ -41,6 +41,10 @@ #include #include #include +#include +#include +#include +#include #include "amba-pl011.h" @@ -273,6 +277,12 @@ struct uart_amba_port { struct pl011_dmatx_data dmatx; bool dma_probed; #endif +#ifdef CONFIG_CONSOLE_POLL + bool poll_mode_active; + struct irq_work nmi_work; + DECLARE_KFIFO(nmi_fifo, unsigned int, 256); + DECLARE_KFIFO(nmi_fifo_flag, unsigned int, 256); +#endif }; static unsigned int pl011_reg_to_offset(const struct uart_amba_port *uap, @@ -347,6 +357,13 @@ static int pl011_fifo_to_tty(struct uart_amba_port *uap) if (uart_handle_sysrq_char(&uap->port, ch & 255)) continue; +#ifdef CONFIG_CONSOLE_POLL + if (in_nmi()) { + kfifo_in(&uap->nmi_fifo, &ch, 1); + kfifo_in(&uap->nmi_fifo_flag, &flag, 1); + continue; + } +#endif uart_insert_char(&uap->port, ch, UART011_DR_OE, ch, flag); } @@ -1286,6 +1303,10 @@ static void pl011_stop_tx(struct uart_port *port) struct uart_amba_port *uap = container_of(port, struct uart_amba_port, port); +#ifdef CONFIG_CONSOLE_POLL + if (uap->poll_mode_active) + return; +#endif uap->im &= ~UART011_TXIM; pl011_write(uap->im, uap, REG_IMSC); pl011_dma_tx_stop(uap); @@ -1302,11 +1323,21 @@ static void pl011_start_tx_pio(struct uart_amba_port *uap) } } +#ifdef CONFIG_CONSOLE_POLL +static void pl011_poll_tx_chars(struct uart_amba_port *uap); +#endif + static void pl011_start_tx(struct uart_port *port) { struct uart_amba_port *uap = container_of(port, struct uart_amba_port, port); +#ifdef CONFIG_CONSOLE_POLL + if (uap->poll_mode_active) { + pl011_poll_tx_chars(uap); + return; + } +#endif if (!pl011_dma_tx_start(uap)) pl011_start_tx_pio(uap); } @@ -1316,6 +1347,10 @@ static void pl011_stop_rx(struct uart_port *port) struct uart_amba_port *uap = container_of(port, struct uart_amba_port, port); +#ifdef CONFIG_CONSOLE_POLL + if (uap->poll_mode_active) + return; +#endif uap->im &= ~(UART011_RXIM|UART011_RTIM|UART011_FEIM| UART011_PEIM|UART011_BEIM|UART011_OEIM); pl011_write(uap->im, uap, REG_IMSC); @@ -1637,6 +1672,128 @@ static void pl011_put_poll_char(struct uart_port *port, pl011_write(ch, uap, REG_DR); } +static void pl011_poll_tx_chars(struct uart_amba_port *uap) +{ + struct circ_buf *xmit = &uap->port.state->xmit; + + if (uap->port.x_char) { + pl011_put_poll_char(&uap->port, uap->port.x_char); + uap->port.x_char = 0; + } + + while (!uart_circ_empty(xmit)) { + pl011_put_poll_char(&uap->port, xmit->buf[xmit->tail]); + uap->port.icount.tx++; + + xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1); + } +} + +static void pl011_nmi_tty_work(struct irq_work *nmi_work) +{ + struct uart_amba_port *uap = + container_of(nmi_work, struct uart_amba_port, nmi_work); + unsigned int ch, flag; + + if (!uap->port.state || !tty_port_active(&uap->port.state->port)) { + kfifo_reset(&uap->nmi_fifo); + kfifo_reset(&uap->nmi_fifo_flag); + return; + } + + if (unlikely(!kfifo_len(&uap->nmi_fifo) || + !kfifo_len(&uap->nmi_fifo_flag))) + return; + + spin_lock(&uap->port.lock); + while (kfifo_out(&uap->nmi_fifo, &ch, 1) && + kfifo_out(&uap->nmi_fifo_flag, &flag, 1)) + uart_insert_char(&uap->port, ch, UART011_DR_OE, ch, flag); + spin_unlock(&uap->port.lock); + + tty_flip_buffer_push(&uap->port.state->port); +} + +static void pl011_nmi_rx_chars(struct uart_amba_port *uap) +{ + pl011_fifo_to_tty(uap); + irq_work_queue(&uap->nmi_work); +} + +static irqreturn_t pl011_nmi_int(int irq, void *dev_id) +{ + struct uart_amba_port *uap = dev_id; + unsigned int status, pass_counter = AMBA_ISR_PASS_LIMIT; + int handled = 0; + + status = pl011_read(uap, REG_MIS); + if (status) { + do { + check_apply_cts_event_workaround(uap); + + pl011_write(status, uap, REG_ICR); + + if (status) + pl011_nmi_rx_chars(uap); + + if (pass_counter-- == 0) + break; + + status = pl011_read(uap, REG_MIS); + } while (status != 0); + handled = 1; + } + + return IRQ_RETVAL(handled); +} + +static int pl011_allocate_nmi(struct uart_amba_port *uap) +{ + int ret; + + irq_set_status_flags(uap->port.irq, IRQ_NOAUTOEN); + ret = request_nmi(uap->port.irq, pl011_nmi_int, IRQF_PERCPU, + "uart-pl011", uap); + if (ret) { + irq_clear_status_flags(uap->port.irq, IRQ_NOAUTOEN); + return ret; + } + + enable_irq(uap->port.irq); + + return ret; +} + +static int pl011_hwinit(struct uart_port *port); + +static int pl011_poll_init(struct uart_port *port) +{ + struct uart_amba_port *uap = + container_of(port, struct uart_amba_port, port); + int retval; + + retval = pl011_hwinit(port); + if (retval) + goto clk_dis; + + /* In case NMI isn't supported, fallback to normal interrupt mode */ + retval = pl011_allocate_nmi(uap); + if (retval) + return 0; + + INIT_KFIFO(uap->nmi_fifo); + INIT_KFIFO(uap->nmi_fifo_flag); + init_irq_work(&uap->nmi_work, pl011_nmi_tty_work); + + uap->poll_mode_active = true; + + return 0; + + clk_dis: + clk_disable_unprepare(uap->clk); + return retval; +} + #endif /* CONFIG_CONSOLE_POLL */ static int pl011_hwinit(struct uart_port *port) @@ -1748,6 +1905,10 @@ static int pl011_startup(struct uart_port *port) unsigned int cr; int retval; +#ifdef CONFIG_CONSOLE_POLL + if (uap->poll_mode_active) + return 0; +#endif retval = pl011_hwinit(port); if (retval) goto clk_dis; @@ -1790,6 +1951,10 @@ static int sbsa_uart_startup(struct uart_port *port) container_of(port, struct uart_amba_port, port); int retval; +#ifdef CONFIG_CONSOLE_POLL + if (uap->poll_mode_active) + return 0; +#endif retval = pl011_hwinit(port); if (retval) return retval; @@ -1859,6 +2024,10 @@ static void pl011_shutdown(struct uart_port *port) struct uart_amba_port *uap = container_of(port, struct uart_amba_port, port); +#ifdef CONFIG_CONSOLE_POLL + if (uap->poll_mode_active) + return; +#endif pl011_disable_interrupts(uap); pl011_dma_shutdown(uap); @@ -1891,6 +2060,10 @@ static void sbsa_uart_shutdown(struct uart_port *port) struct uart_amba_port *uap = container_of(port, struct uart_amba_port, port); +#ifdef CONFIG_CONSOLE_POLL + if (uap->poll_mode_active) + return; +#endif pl011_disable_interrupts(uap); free_irq(uap->port.irq, uap); @@ -2142,7 +2315,7 @@ static const struct uart_ops amba_pl011_pops = { .config_port = pl011_config_port, .verify_port = pl011_verify_port, #ifdef CONFIG_CONSOLE_POLL - .poll_init = pl011_hwinit, + .poll_init = pl011_poll_init, .poll_get_char = pl011_get_poll_char, .poll_put_char = pl011_put_poll_char, #endif @@ -2173,7 +2346,7 @@ static const struct uart_ops sbsa_uart_pops = { .config_port = pl011_config_port, .verify_port = pl011_verify_port, #ifdef CONFIG_CONSOLE_POLL - .poll_init = pl011_hwinit, + .poll_init = pl011_poll_init, .poll_get_char = pl011_get_poll_char, .poll_put_char = pl011_put_poll_char, #endif