From patchwork Tue May 14 12:21:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 164167 Delivered-To: patch@linaro.org Received: by 2002:a92:9e1a:0:0:0:0:0 with SMTP id q26csp2552416ili; Tue, 14 May 2019 05:23:26 -0700 (PDT) X-Google-Smtp-Source: APXvYqxdcAArAONQmWhwEvVPz+gxhyelCci/WplqV2fE4VulzOTl7/aUJvSnDwhONrDGOqK8BbFn X-Received: by 2002:a24:2782:: with SMTP id g124mr527651ita.137.1557836606173; Tue, 14 May 2019 05:23:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557836606; cv=none; d=google.com; s=arc-20160816; b=aLIuCfCPTRjURsJNu21/9+OyjagyCjjNIaQ6hn3qjcul4plFwLgqWt1KqqwUKrLqg7 ABITg+ng7mc/GYh4y6ML4sYpHiPj9fvZwsVxX3J/TZ+iF3G0VzMc34tJ4+jYO0QARBIb 2wiAhVL/pWFJaXUPyhkiOekssoXDGyExVf6a0gxVNy0g6YF8zSYRhK8j4TiOPBKNxVPv qIonCyq7bw9nJyNIAnImX/UvlVG/JcEg3rgYXpFnSdqQmNWvYUo8Csx2DQK/e3hjVcfU C+zeuILlEkOQZBC46jCRBE7UVpJO+holvZdDzUEZtm3afXyWw99AVLJ/ZE+Sj9xNzrb+ gIBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=296nWOIdD8CG10ZnJrzh6KlRoWEZk/F2RrPHIPTfgf8=; b=xEpSwsoQDEth3CWZpDrjg2pDFNnWS2mMd9X8ZrH5ZvxK81I8QKYGjSbhlGUp+mqzT5 oeKY4I8mJMSaon8hQZ1KoWUMdqwYeyvPnnsQ1eioB+dQ5dENekymAAExDrTu15FYImiU ZOeLpHTq0O3DPz4/erNikaaqj26OjPDmBC/CHSr1yl/L5QULnGUEJTTsiOJjN0Qw/BWl NMOIy14EP9OWxcxgvEWzHERVHROVvDioj0kS2YfmryV461nghiV3909df1eh0IkJvhHS v6BkzYqOqrPIB3bxaCaZiTMHAjQuqMJa++u5uZ7N/fBf6EB4vGQfV+LRpS7NbRcI9JMi L0bQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id w138si1509728ita.21.2019.05.14.05.23.26 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 May 2019 05:23:26 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWRX-0008JM-4h; Tue, 14 May 2019 12:22:03 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWRV-0008H5-EV for xen-devel@lists.xenproject.org; Tue, 14 May 2019 12:22:01 +0000 X-Inumbo-ID: dfa1e180-7642-11e9-963d-47d53be77ade Received: from foss.arm.com (unknown [217.140.101.70]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id dfa1e180-7642-11e9-963d-47d53be77ade; Tue, 14 May 2019 12:22:00 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1CC79341; Tue, 14 May 2019 05:22:00 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DE0753F71E; Tue, 14 May 2019 05:21:58 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Tue, 14 May 2019 13:21:20 +0100 Message-Id: <20190514122136.28215-12-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190514122136.28215-1-julien.grall@arm.com> References: <20190514122136.28215-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH MM-PART1 v3 6/8] xen/arm: Gather all TLB flush helpers in tlbflush.h X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" At the moment, TLB helpers are scattered in 2 headers: page.h (for Xen TLB helpers) and tlbflush.h (for guest TLB helpers). This patch is gathering all of them in tlbflush. This will help to uniformize and update the logic of the helpers in follow-up patches. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov Acked-by: Stefano Stabellini --- Changes in v3: - Add Stefano's acked-by Changes in v2: - Add Andrii's reviewed-by --- xen/include/asm-arm/arm32/flushtlb.h | 22 +++++++++++++++++++++ xen/include/asm-arm/arm32/page.h | 22 --------------------- xen/include/asm-arm/arm64/flushtlb.h | 23 ++++++++++++++++++++++ xen/include/asm-arm/arm64/page.h | 23 ---------------------- xen/include/asm-arm/flushtlb.h | 38 ++++++++++++++++++++++++++++++++++++ xen/include/asm-arm/page.h | 38 ------------------------------------ 6 files changed, 83 insertions(+), 83 deletions(-) diff --git a/xen/include/asm-arm/arm32/flushtlb.h b/xen/include/asm-arm/arm32/flushtlb.h index 22e100eccf..b629db61cb 100644 --- a/xen/include/asm-arm/arm32/flushtlb.h +++ b/xen/include/asm-arm/arm32/flushtlb.h @@ -45,6 +45,28 @@ static inline void flush_all_guests_tlb(void) isb(); } +/* Flush all hypervisor mappings from the TLB of the local processor. */ +static inline void flush_xen_tlb_local(void) +{ + asm volatile("dsb;" /* Ensure preceding are visible */ + CMD_CP32(TLBIALLH) + "dsb;" /* Ensure completion of the TLB flush */ + "isb;" + : : : "memory"); +} + +/* Flush TLB of local processor for address va. */ +static inline void __flush_xen_tlb_one_local(vaddr_t va) +{ + asm volatile(STORE_CP32(0, TLBIMVAH) : : "r" (va) : "memory"); +} + +/* Flush TLB of all processors in the inner-shareable domain for address va. */ +static inline void __flush_xen_tlb_one(vaddr_t va) +{ + asm volatile(STORE_CP32(0, TLBIMVAHIS) : : "r" (va) : "memory"); +} + #endif /* __ASM_ARM_ARM32_FLUSHTLB_H__ */ /* * Local variables: diff --git a/xen/include/asm-arm/arm32/page.h b/xen/include/asm-arm/arm32/page.h index 0b41b9214b..715a9e4fef 100644 --- a/xen/include/asm-arm/arm32/page.h +++ b/xen/include/asm-arm/arm32/page.h @@ -61,28 +61,6 @@ static inline void invalidate_icache_local(void) isb(); /* Synchronize fetched instruction stream. */ } -/* Flush all hypervisor mappings from the TLB of the local processor. */ -static inline void flush_xen_tlb_local(void) -{ - asm volatile("dsb;" /* Ensure preceding are visible */ - CMD_CP32(TLBIALLH) - "dsb;" /* Ensure completion of the TLB flush */ - "isb;" - : : : "memory"); -} - -/* Flush TLB of local processor for address va. */ -static inline void __flush_xen_tlb_one_local(vaddr_t va) -{ - asm volatile(STORE_CP32(0, TLBIMVAH) : : "r" (va) : "memory"); -} - -/* Flush TLB of all processors in the inner-shareable domain for address va. */ -static inline void __flush_xen_tlb_one(vaddr_t va) -{ - asm volatile(STORE_CP32(0, TLBIMVAHIS) : : "r" (va) : "memory"); -} - /* Ask the MMU to translate a VA for us */ static inline uint64_t __va_to_par(vaddr_t va) { diff --git a/xen/include/asm-arm/arm64/flushtlb.h b/xen/include/asm-arm/arm64/flushtlb.h index adbbd5c522..2fed34b2ec 100644 --- a/xen/include/asm-arm/arm64/flushtlb.h +++ b/xen/include/asm-arm/arm64/flushtlb.h @@ -45,6 +45,29 @@ static inline void flush_all_guests_tlb(void) : : : "memory"); } +/* Flush all hypervisor mappings from the TLB of the local processor. */ +static inline void flush_xen_tlb_local(void) +{ + asm volatile ( + "dsb sy;" /* Ensure visibility of PTE writes */ + "tlbi alle2;" /* Flush hypervisor TLB */ + "dsb sy;" /* Ensure completion of TLB flush */ + "isb;" + : : : "memory"); +} + +/* Flush TLB of local processor for address va. */ +static inline void __flush_xen_tlb_one_local(vaddr_t va) +{ + asm volatile("tlbi vae2, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); +} + +/* Flush TLB of all processors in the inner-shareable domain for address va. */ +static inline void __flush_xen_tlb_one(vaddr_t va) +{ + asm volatile("tlbi vae2is, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); +} + #endif /* __ASM_ARM_ARM64_FLUSHTLB_H__ */ /* * Local variables: diff --git a/xen/include/asm-arm/arm64/page.h b/xen/include/asm-arm/arm64/page.h index 31d04ecf76..0cba266373 100644 --- a/xen/include/asm-arm/arm64/page.h +++ b/xen/include/asm-arm/arm64/page.h @@ -45,29 +45,6 @@ static inline void invalidate_icache_local(void) isb(); } -/* Flush all hypervisor mappings from the TLB of the local processor. */ -static inline void flush_xen_tlb_local(void) -{ - asm volatile ( - "dsb sy;" /* Ensure visibility of PTE writes */ - "tlbi alle2;" /* Flush hypervisor TLB */ - "dsb sy;" /* Ensure completion of TLB flush */ - "isb;" - : : : "memory"); -} - -/* Flush TLB of local processor for address va. */ -static inline void __flush_xen_tlb_one_local(vaddr_t va) -{ - asm volatile("tlbi vae2, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); -} - -/* Flush TLB of all processors in the inner-shareable domain for address va. */ -static inline void __flush_xen_tlb_one(vaddr_t va) -{ - asm volatile("tlbi vae2is, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); -} - /* Ask the MMU to translate a VA for us */ static inline uint64_t __va_to_par(vaddr_t va) { diff --git a/xen/include/asm-arm/flushtlb.h b/xen/include/asm-arm/flushtlb.h index 83ff9fa8b3..ab1aae5c90 100644 --- a/xen/include/asm-arm/flushtlb.h +++ b/xen/include/asm-arm/flushtlb.h @@ -28,6 +28,44 @@ static inline void page_set_tlbflush_timestamp(struct page_info *page) /* Flush specified CPUs' TLBs */ void flush_tlb_mask(const cpumask_t *mask); +/* + * Flush a range of VA's hypervisor mappings from the TLB of the local + * processor. + */ +static inline void flush_xen_tlb_range_va_local(vaddr_t va, + unsigned long size) +{ + vaddr_t end = va + size; + + dsb(sy); /* Ensure preceding are visible */ + while ( va < end ) + { + __flush_xen_tlb_one_local(va); + va += PAGE_SIZE; + } + dsb(sy); /* Ensure completion of the TLB flush */ + isb(); +} + +/* + * Flush a range of VA's hypervisor mappings from the TLB of all + * processors in the inner-shareable domain. + */ +static inline void flush_xen_tlb_range_va(vaddr_t va, + unsigned long size) +{ + vaddr_t end = va + size; + + dsb(sy); /* Ensure preceding are visible */ + while ( va < end ) + { + __flush_xen_tlb_one(va); + va += PAGE_SIZE; + } + dsb(sy); /* Ensure completion of the TLB flush */ + isb(); +} + #endif /* __ASM_ARM_FLUSHTLB_H__ */ /* * Local variables: diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h index 195345e24a..2bcdb0f1a5 100644 --- a/xen/include/asm-arm/page.h +++ b/xen/include/asm-arm/page.h @@ -233,44 +233,6 @@ static inline int clean_and_invalidate_dcache_va_range : : "r" (_p), "m" (*_p)); \ } while (0) -/* - * Flush a range of VA's hypervisor mappings from the TLB of the local - * processor. - */ -static inline void flush_xen_tlb_range_va_local(vaddr_t va, - unsigned long size) -{ - vaddr_t end = va + size; - - dsb(sy); /* Ensure preceding are visible */ - while ( va < end ) - { - __flush_xen_tlb_one_local(va); - va += PAGE_SIZE; - } - dsb(sy); /* Ensure completion of the TLB flush */ - isb(); -} - -/* - * Flush a range of VA's hypervisor mappings from the TLB of all - * processors in the inner-shareable domain. - */ -static inline void flush_xen_tlb_range_va(vaddr_t va, - unsigned long size) -{ - vaddr_t end = va + size; - - dsb(sy); /* Ensure preceding are visible */ - while ( va < end ) - { - __flush_xen_tlb_one(va); - va += PAGE_SIZE; - } - dsb(sy); /* Ensure completion of the TLB flush */ - isb(); -} - /* Flush the dcache for an entire page. */ void flush_page_to_ram(unsigned long mfn, bool sync_icache);