From patchwork Fri Aug 24 15:52:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 145078 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp1410718ljw; Fri, 24 Aug 2018 08:53:06 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbbU2eo/4FDI9yAbJA7kHQtQBF7ZV/w4JH2UOScqvf+8yv602ztFLXzfQUM/PaRRAcMT3v5 X-Received: by 2002:a63:b19:: with SMTP id 25-v6mr2289461pgl.301.1535125985863; Fri, 24 Aug 2018 08:53:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535125985; cv=none; d=google.com; s=arc-20160816; b=a09is4g5pp+wtdB8CohddwDvgA9TZS87cm6bw9mqfsekpfAk+W6FF65Q1WhTavbkhP TdJ5bx/YB051ZBfgUInqQoYoVB//3ddr9o1xWj5rxf613YrsUx9TkNyuFxcdTkRlPyMQ q2BhQshBHXnRYIm68AzniRzrIfjiakJsRk7JfCX73gmmvoq++mko6Hdm6Ovx+VkD74Bq 7wBlRrYO9/WeMGHfatz8q6FuAUBVrE737BjZipN143jfpAqGsxRP+Tycm3MP8y6Twp2X WCWioKII7EWA2f/S3vh2ARt7szMMOMuwaO4L3wXVgpnxfkee6rQkgKOgAbWa2nnz0ijx IM2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=SuSTYfW56h5OPXzEN7E9R6K4q9ZQKOw8hFmFbAmIwyc=; b=RY+d9ai62fq2l5ezeAbx6t0YUM+09Av4MNY/A/VBZNW354wP/U30VMmGWMawo1Vuh0 Ty1GiODrPaDQWBW4ZgmHgOpwBGemX6braNfjUiR3ZmY/3TC8WbLChjTu4ffbDpSrSRH8 sLEOzwssUEfypZ7QIzewUaw2A4Sq/1ep5V671q4sgZEXEuBtfnxaReLkaEo3sh7KpbkO 4W65GvbaM3dgVtI8jMnpUx+FdKJ8FR2wZKuOemRNaTaRhEAbPZ9uQh0bh+YKeKL7Alhq +HWGgn2u38KCH33uRaHwDu6oJPb01dNVpIIojCLxbq8dJ4o0tIeESOqGIeWd9y0gGcZP l4PA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r206-v6si7404272pgr.17.2018.08.24.08.53.05; Fri, 24 Aug 2018 08:53:05 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728086AbeHXT2R (ORCPT + 32 others); Fri, 24 Aug 2018 15:28:17 -0400 Received: from foss.arm.com ([217.140.101.70]:32902 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727613AbeHXT1w (ORCPT ); Fri, 24 Aug 2018 15:27:52 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E8EC01AC1; Fri, 24 Aug 2018 08:52:38 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id BB12A3F994; Fri, 24 Aug 2018 08:52:38 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 50CF51AE33DD; Fri, 24 Aug 2018 16:52:48 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org Cc: peterz@infradead.org, benh@au1.ibm.com, torvalds@linux-foundation.org, npiggin@gmail.com, catalin.marinas@arm.com, linux-arm-kernel@lists.infradead.org, Will Deacon Subject: [RFC PATCH 10/11] arm64: tlb: Adjust stride and type of TLBI according to mmu_gather Date: Fri, 24 Aug 2018 16:52:45 +0100 Message-Id: <1535125966-7666-11-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1535125966-7666-1-git-send-email-will.deacon@arm.com> References: <1535125966-7666-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Now that the core mmu_gather code keeps track of both the levels of page table cleared and also whether or not these entries correspond to intermediate entries, we can use this in our tlb_flush() callback to reduce the number of invalidations we issue as well as their scope. Signed-off-by: Will Deacon --- arch/arm64/include/asm/tlb.h | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h index bd00017d529a..baca8dff6884 100644 --- a/arch/arm64/include/asm/tlb.h +++ b/arch/arm64/include/asm/tlb.h @@ -34,20 +34,21 @@ static void tlb_flush(struct mmu_gather *tlb); static inline void tlb_flush(struct mmu_gather *tlb) { struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0); + bool last_level = !tlb->freed_tables; + unsigned long stride = tlb_get_unmap_granule(tlb); /* - * The ASID allocator will either invalidate the ASID or mark - * it as used. + * If we're tearing down the address space then we only care about + * invalidating the walk-cache, since the ASID allocator won't + * reallocate our ASID without invalidating the entire TLB. */ - if (tlb->fullmm) + if (tlb->fullmm) { + if (!last_level) + flush_tlb_mm(tlb->mm); return; + } - /* - * The intermediate page table levels are already handled by - * the __(pte|pmd|pud)_free_tlb() functions, so last level - * TLBI is sufficient here. - */ - __flush_tlb_range(&vma, tlb->start, tlb->end, PAGE_SIZE, true); + __flush_tlb_range(&vma, tlb->start, tlb->end, stride, last_level); } static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte,