From patchwork Fri Feb 22 18:50:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 159064 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp2070531jaa; Fri, 22 Feb 2019 10:51:27 -0800 (PST) X-Google-Smtp-Source: AHgI3IY4XsemBOoSfrkYz3XLgfbJUE+U0F49z9yMdzJztS5n9hlSayUvvKp6/ZQTMMz7PNP1zU2n X-Received: by 2002:a62:6383:: with SMTP id x125mr5525813pfb.239.1550861487007; Fri, 22 Feb 2019 10:51:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550861486; cv=none; d=google.com; s=arc-20160816; b=XBqh98nUH9z1xXGciQnlpKTKvsYCY/fhAaxJHkcScQb8rOBlsvMwuBoMqOH9dKI8/z 3buyo/GZxE4tfqfE8h8C4kASSD/zjKvd2qAOu9U5HyBcOqg83lUucZRtNMncUm9PbA0u 7fgmCwh2734Ts9ihRHxz1ZjwWfyPt6X4F+9T7rFoCLtxb3midUeLev7kU4N2LHD16dpk b017H4PrIvEXnIqzu13kdOL/gaVhJJrc4xjiYIi1TGlou/unA4U/CXhxVI03GtGNWt7B OsZ74f0vWsglt992tyrhDcWAmhGugVrruCOQY+1IgipNqBRCql0ZR+HDGgUA1emSAbrQ mYWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=Q3bze3M5kkO37tP/+WKlaahrsw58CodI8z4s6oClH70=; b=sRrqwmMP29EDv/wJ2BRYukKcLyWPt6ikqw1IpztLHSE8N9bFJ0lujwn6/FTlxuQf8q /7X3RG50GuClAu4J2AG1NH4oCwKCIDJwRd1uiac4W2cgMVMqS3srL6gI/Qxmyz6GcKUE pb4oyolYdoW51iwdpTrYbKjYfZDtciyZM/ATfQ3Li/MIQ/zCCxw9VC9om919X9mcCCyY epJnmsLrOca6CSuOKw4/me1ATcEyvZ/xAqNsJDSkwa4M+HbdJB6KbSulTis1HgeH7Q9n 6TQVSQEdLrO+XBTwT4QHvUa5rpZtf/wHJvARpOgSquT0dT3TyJ8O1d+BHMVs8R02LYWx e/zA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id cn17si2047109plb.139.2019.02.22.10.51.26; Fri, 22 Feb 2019 10:51:26 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727643AbfBVSvZ (ORCPT + 32 others); Fri, 22 Feb 2019 13:51:25 -0500 Received: from foss.arm.com ([217.140.101.70]:39574 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727627AbfBVSvY (ORCPT ); Fri, 22 Feb 2019 13:51:24 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2B2F116A3; Fri, 22 Feb 2019 10:51:24 -0800 (PST) Received: from fuggles.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E4C3D3F5C1; Fri, 22 Feb 2019 10:51:20 -0800 (PST) From: Will Deacon To: linux-arch@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Will Deacon , "Paul E. McKenney" , Benjamin Herrenschmidt , Michael Ellerman , Arnd Bergmann , Peter Zijlstra , Andrea Parri , Palmer Dabbelt , Daniel Lustig , David Howells , Alan Stern , Linus Torvalds , "Maciej W. Rozycki" , Paul Burton , Ingo Molnar , Yoshinori Sato , Rich Felker , Tony Luck Subject: [RFC PATCH 14/20] Documentation: Kill all references to mmiowb() Date: Fri, 22 Feb 2019 18:50:20 +0000 Message-Id: <20190222185026.10973-15-will.deacon@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190222185026.10973-1-will.deacon@arm.com> References: <20190222185026.10973-1-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The guarantees provided by mmiowb() are now provided implicitly by spin_unlock(), so we can remove all references from this most confusing of barriers from our Documentation. Good riddance. Signed-off-by: Will Deacon --- Documentation/driver-api/device-io.rst | 45 -------------- Documentation/driver-api/pci/p2pdma.rst | 4 -- Documentation/memory-barriers.txt | 103 ++------------------------------ 3 files changed, 4 insertions(+), 148 deletions(-) -- 2.11.0 diff --git a/Documentation/driver-api/device-io.rst b/Documentation/driver-api/device-io.rst index b00b23903078..0e389378f71d 100644 --- a/Documentation/driver-api/device-io.rst +++ b/Documentation/driver-api/device-io.rst @@ -103,51 +103,6 @@ continuing execution:: ha->flags.ints_enabled = 0; } -In addition to write posting, on some large multiprocessing systems -(e.g. SGI Challenge, Origin and Altix machines) posted writes won't be -strongly ordered coming from different CPUs. Thus it's important to -properly protect parts of your driver that do memory-mapped writes with -locks and use the :c:func:`mmiowb()` to make sure they arrive in the -order intended. Issuing a regular readX() will also ensure write ordering, -but should only be used when the -driver has to be sure that the write has actually arrived at the device -(not that it's simply ordered with respect to other writes), since a -full readX() is a relatively expensive operation. - -Generally, one should use :c:func:`mmiowb()` prior to releasing a spinlock -that protects regions using :c:func:`writeb()` or similar functions that -aren't surrounded by readb() calls, which will ensure ordering -and flushing. The following pseudocode illustrates what might occur if -write ordering isn't guaranteed via :c:func:`mmiowb()` or one of the -readX() functions:: - - CPU A: spin_lock_irqsave(&dev_lock, flags) - CPU A: ... - CPU A: writel(newval, ring_ptr); - CPU A: spin_unlock_irqrestore(&dev_lock, flags) - ... - CPU B: spin_lock_irqsave(&dev_lock, flags) - CPU B: writel(newval2, ring_ptr); - CPU B: ... - CPU B: spin_unlock_irqrestore(&dev_lock, flags) - -In the case above, newval2 could be written to ring_ptr before newval. -Fixing it is easy though:: - - CPU A: spin_lock_irqsave(&dev_lock, flags) - CPU A: ... - CPU A: writel(newval, ring_ptr); - CPU A: mmiowb(); /* ensure no other writes beat us to the device */ - CPU A: spin_unlock_irqrestore(&dev_lock, flags) - ... - CPU B: spin_lock_irqsave(&dev_lock, flags) - CPU B: writel(newval2, ring_ptr); - CPU B: ... - CPU B: mmiowb(); - CPU B: spin_unlock_irqrestore(&dev_lock, flags) - -See tg3.c for a real world example of how to use :c:func:`mmiowb()` - PCI ordering rules also guarantee that PIO read responses arrive after any outstanding DMA writes from that bus, since for some devices the result of a readb() call may signal to the driver that a DMA transaction is diff --git a/Documentation/driver-api/pci/p2pdma.rst b/Documentation/driver-api/pci/p2pdma.rst index 6d85b5a2598d..44deb52beeb4 100644 --- a/Documentation/driver-api/pci/p2pdma.rst +++ b/Documentation/driver-api/pci/p2pdma.rst @@ -132,10 +132,6 @@ precludes passing these pages to userspace. P2P memory is also technically IO memory but should never have any side effects behind it. Thus, the order of loads and stores should not be important and ioreadX(), iowriteX() and friends should not be necessary. -However, as the memory is not cache coherent, if access ever needs to -be protected by a spinlock then :c:func:`mmiowb()` must be used before -unlocking the lock. (See ACQUIRES VS I/O ACCESSES in -Documentation/memory-barriers.txt) P2P DMA Support Library diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt index 158947ae78c2..6d6eff413462 100644 --- a/Documentation/memory-barriers.txt +++ b/Documentation/memory-barriers.txt @@ -1937,21 +1937,6 @@ There are some more advanced barrier functions: information on consistent memory. -MMIO WRITE BARRIER ------------------- - -The Linux kernel also has a special barrier for use with memory-mapped I/O -writes: - - mmiowb(); - -This is a variation on the mandatory write barrier that causes writes to weakly -ordered I/O regions to be partially ordered. Its effects may go beyond the -CPU->Hardware interface and actually affect the hardware at some level. - -See the subsection "Acquires vs I/O accesses" for more information. - - =============================== IMPLICIT KERNEL MEMORY BARRIERS =============================== @@ -2317,75 +2302,6 @@ But it won't see any of: *E, *F or *G following RELEASE Q - -ACQUIRES VS I/O ACCESSES ------------------------- - -Under certain circumstances (especially involving NUMA), I/O accesses within -two spinlocked sections on two different CPUs may be seen as interleaved by the -PCI bridge, because the PCI bridge does not necessarily participate in the -cache-coherence protocol, and is therefore incapable of issuing the required -read memory barriers. - -For example: - - CPU 1 CPU 2 - =============================== =============================== - spin_lock(Q) - writel(0, ADDR) - writel(1, DATA); - spin_unlock(Q); - spin_lock(Q); - writel(4, ADDR); - writel(5, DATA); - spin_unlock(Q); - -may be seen by the PCI bridge as follows: - - STORE *ADDR = 0, STORE *ADDR = 4, STORE *DATA = 1, STORE *DATA = 5 - -which would probably cause the hardware to malfunction. - - -What is necessary here is to intervene with an mmiowb() before dropping the -spinlock, for example: - - CPU 1 CPU 2 - =============================== =============================== - spin_lock(Q) - writel(0, ADDR) - writel(1, DATA); - mmiowb(); - spin_unlock(Q); - spin_lock(Q); - writel(4, ADDR); - writel(5, DATA); - mmiowb(); - spin_unlock(Q); - -this will ensure that the two stores issued on CPU 1 appear at the PCI bridge -before either of the stores issued on CPU 2. - - -Furthermore, following a store by a load from the same device obviates the need -for the mmiowb(), because the load forces the store to complete before the load -is performed: - - CPU 1 CPU 2 - =============================== =============================== - spin_lock(Q) - writel(0, ADDR) - a = readl(DATA); - spin_unlock(Q); - spin_lock(Q); - writel(4, ADDR); - b = readl(DATA); - spin_unlock(Q); - - -See Documentation/driver-api/device-io.rst for more information. - - ================================= WHERE ARE MEMORY BARRIERS NEEDED? ================================= @@ -2532,16 +2448,9 @@ the device to malfunction. Inside of the Linux kernel, I/O should be done through the appropriate accessor routines - such as inb() or writel() - which know how to make such accesses appropriately sequential. While this, for the most part, renders the explicit -use of memory barriers unnecessary, there are a couple of situations where they -might be needed: - - (1) On some systems, I/O stores are not strongly ordered across all CPUs, and - so for _all_ general drivers locks should be used and mmiowb() must be - issued prior to unlocking the critical section. - - (2) If the accessor functions are used to refer to an I/O memory window with - relaxed memory access properties, then _mandatory_ memory barriers are - required to enforce ordering. +use of memory barriers unnecessary, if the accessor functions are used to refer +to an I/O memory window with relaxed memory access properties, then _mandatory_ +memory barriers are required to enforce ordering. See Documentation/driver-api/device-io.rst for more information. @@ -2586,8 +2495,7 @@ explicit barriers are used. Normally this won't be a problem because the I/O accesses done inside such sections will include synchronous load operations on strictly ordered I/O -registers that form implicit I/O barriers. If this isn't sufficient then an -mmiowb() may need to be used explicitly. +registers that form implicit I/O barriers. A similar situation may occur between an interrupt routine and two routines @@ -2687,9 +2595,6 @@ guarantees: All of these accessors assume that the underlying peripheral is little-endian, and will therefore perform byte-swapping operations on big-endian architectures. -Composing I/O ordering barriers with SMP ordering barriers and LOCK/UNLOCK -operations is a dangerous sport which may require the use of mmiowb(). See the -subsection "Acquires vs I/O accesses" for more information. ======================================== ASSUMED MINIMUM EXECUTION ORDERING MODEL