From patchwork Tue May 14 12:11:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 164137 Delivered-To: patch@linaro.org Received: by 2002:a92:9e1a:0:0:0:0:0 with SMTP id q26csp2540668ili; Tue, 14 May 2019 05:13:04 -0700 (PDT) X-Google-Smtp-Source: APXvYqy5WfrddjTH2bXK46K/vtMehd9svnZGwYGiFneJFnRg3Q2GIgbbS7eBfq4Nkg2EQq/L1DEb X-Received: by 2002:a24:4a81:: with SMTP id k123mr3426384itb.159.1557835984148; Tue, 14 May 2019 05:13:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557835984; cv=none; d=google.com; s=arc-20160816; b=XYDzZG+GS40aB++M8zrwqKTd1cZ0Sczh5ZpIwC46CcuN5J6hleh4GyQJ4i653/MoCj BaXiDPAE+0aOgQ6ZUhEkAfZEVWUevVqYmSrB7Na7/OkftqwpUNbRwzEnPxLrJ8OzyFrE IGA1J3KLmmdUK3smxfDgVLYQnCwo2Yix5UbIEcDTpFeAWsesShT1fqgyLsQzGelXm55X rnyCpoi9Ligl4f7RQZW2B2bkAoktt6tnC4Y0ftSQ2ILtQk+HrBQWg228U4RPngV3SKEM 33ekHKVjxvBdfL4Vj1mOSCFEE/XUxjTFraKX6POSflYlTJzShs1zNWX903yo+5UBzAKx MBCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=l7IMz6SmFkbIDcRqmOwsxEm6qLyqX/4jEzjnX1BC1qQ=; b=uOm0uTKap/aYJQHkdXuRLaMsGkzJq2araOqZ/o4AFggUzD0B3ZytBBrBMg2Q3YwR5H Ootda/DYy7V9OckzH/yihmN0l6xcIEKzrOFDY4GEGWXyfQOU/as1p8xNJjA+zJI+48AT 6ru0YzSpGZnOgoNZV6cU8rsLmNzjQ+y1FFKVgRj2PA/yopAzeWEeo1Za2TVbFet+1BNp W8K6bjJh1i4AbzeW0+3oH72YwSMEtVpYjV1DKSH7fbe2sjQ5ntpI21HkiP+2eAfpG8Y4 uTfsJv0+G9hxRz/LItBHG5CjXOjJihCIqFBQb2GVgdnEy20eg57qLPq7F+JR9MeEoL2S t/JA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id p20si11327314jam.1.2019.05.14.05.13.04 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 May 2019 05:13:04 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHX-0006EB-Od; Tue, 14 May 2019 12:11:43 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHV-0006DS-O4 for xen-devel@lists.xenproject.org; Tue, 14 May 2019 12:11:41 +0000 X-Inumbo-ID: 6e6b95e8-7641-11e9-8980-bc764e045a96 Received: from foss.arm.com (unknown [217.140.101.70]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTP id 6e6b95e8-7641-11e9-8980-bc764e045a96; Tue, 14 May 2019 12:11:40 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A10CC15AB; Tue, 14 May 2019 05:11:40 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 905D03F71E; Tue, 14 May 2019 05:11:39 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Tue, 14 May 2019 13:11:25 +0100 Message-Id: <20190514121132.26732-2-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190514121132.26732-1-julien.grall@arm.com> References: <20190514121132.26732-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH MM-PART1 v3 1/8] xen/arm: Don't boot Xen on platform using AIVIVT instruction caches X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii_Anisov@epam.com MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The AIVIVT is a type of instruction cache available on Armv7. This is the only cache not implementing the IVIPT extension and therefore requiring specific care. To simplify maintenance requirements, Xen will not boot on platform using AIVIVT cache. This should not be an issue because Xen Arm32 can only boot on a small number of processors (see arch/arm/arm32/proc-v7.S). All of them are not using AIVIVT cache. Signed-off-by: Julien Grall Acked-by: Stefano Stabellini --- Changes in v3: - Patch added --- xen/arch/arm/setup.c | 5 +++++ xen/include/asm-arm/processor.h | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index ccb0f181ea..faaf029b99 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -526,10 +526,15 @@ static void __init setup_mm(unsigned long dtb_paddr, size_t dtb_size) unsigned long boot_mfn_start, boot_mfn_end; int i; void *fdt; + const uint32_t ctr = READ_CP32(CTR); if ( !bootinfo.mem.nr_banks ) panic("No memory bank\n"); + /* We only supports instruction caches implementing the IVIPT extension. */ + if ( ((ctr >> CTR_L1Ip_SHIFT) & CTR_L1Ip_MASK) == CTR_L1Ip_AIVIVT ) + panic("AIVIVT instruction cache not supported\n"); + init_pdx(); ram_start = bootinfo.mem.bank[0].start; diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h index b5f515805d..04b05b3f39 100644 --- a/xen/include/asm-arm/processor.h +++ b/xen/include/asm-arm/processor.h @@ -6,6 +6,11 @@ #endif #include +/* CTR Cache Type Register */ +#define CTR_L1Ip_MASK 0x3 +#define CTR_L1Ip_SHIFT 14 +#define CTR_L1Ip_AIVIVT 0x1 + /* MIDR Main ID Register */ #define MIDR_REVISION_MASK 0xf #define MIDR_RESIVION(midr) ((midr) & MIDR_REVISION_MASK) From patchwork Tue May 14 12:11:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 164136 Delivered-To: patch@linaro.org Received: by 2002:a92:9e1a:0:0:0:0:0 with SMTP id q26csp2540608ili; Tue, 14 May 2019 05:13:01 -0700 (PDT) X-Google-Smtp-Source: APXvYqw38o2A2oQqzc2wYc0P7KAFFQAz4Kbe47HFmwu0YPpKdeQne7xekQswNHRLYb46D5QQDQvP X-Received: by 2002:a6b:6304:: with SMTP id p4mr3938240iog.211.1557835981304; Tue, 14 May 2019 05:13:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557835981; cv=none; d=google.com; s=arc-20160816; b=xGeIX8UruDIeb9wtvtsjn3a3iwlZ4XjFyK/fbpyhR7WeKH2JPnFqMTIHHzICV9qx/g LqJC5B7i6lC1Rev/FcVVh6Ye//PDVf1NDdimPruJjtsEYwOY06QyWnHwe3vfK564fp5D 3nbMELYqoUM2YnUCbWuNrCoF/uCqwWyAMoxO96jUPa7B8SJ3vKXIE9gX2nt2pCcm0PvG 8bf+RbMSb/911A+ABDmDxpWfeP4jPnGKfyn9nZpTE94omxw4QyJvat8bpBIOGqI2XVAh 1EFBfTy3YsSeUpOrDOS/IV1YpvjCVMAMMyuyVS7Nz4x1lgEhuyPfzDqv/X1yufGUeGd1 C7ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=BhTZ+0klaTehJOtVb+DOx1H9QOZvkNN7IXM574DSmRw=; b=e0NyZLtedawAOiLhLGDFiuZEQmJMRjhVs7rfGrdstoddqt4Bj/iFZZcln1DNEg1A4K 0a+2f/3Sd7x+A/DYuErUFTRvjqwOIj6SBV5ZjBY8abiCSrGgObumASV4HlBMF+L/atKW X9sFE2BLGmyYu7ajGZ8Lql3FmTPj3FHyo6emqLt0hEZhpJz5yRPZrCHxkD6Mo80PlUJD SF/Z92WZZAzUypHgLl1Jb1GCfxEersk+iP80xKriF0+2uz6O57Rjcq7MEh+ZlXMZ8jdG HTcxwrCcZBvGQb01Z6QvYT8oHJjtxczuoBLzhrNCciQRU8xRPR4ON++uC5vr5qbqy9Kd 0Sqg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id w188si1469105itg.45.2019.05.14.05.13.01 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 May 2019 05:13:01 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHZ-0006Ey-3a; Tue, 14 May 2019 12:11:45 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHY-0006EZ-6Q for xen-devel@lists.xenproject.org; Tue, 14 May 2019 12:11:44 +0000 X-Inumbo-ID: 6f4ff7aa-7641-11e9-8980-bc764e045a96 Received: from foss.arm.com (unknown [217.140.101.70]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTP id 6f4ff7aa-7641-11e9-8980-bc764e045a96; Tue, 14 May 2019 12:11:42 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1EA4C15AD; Tue, 14 May 2019 05:11:42 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DF6C83F71E; Tue, 14 May 2019 05:11:40 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Tue, 14 May 2019 13:11:26 +0100 Message-Id: <20190514121132.26732-3-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190514121132.26732-1-julien.grall@arm.com> References: <20190514121132.26732-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH MM-PART1 v3 2/8] xen/arm: mm: Consolidate setting SCTLR_EL2.WXN in a single place X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The logic to set SCTLR_EL2.WXN is the same for the boot CPU and non-boot CPU. So introduce a function to set the bit and clear TLBs. This new function will help us to document and update the logic in a single place. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov Reviewed-by: Stefano Stabellini --- Changes in v3: - Add Stefano's reviewed-by Changes in v2: - Fix typo in the commit message - Add Andrii's reviewed-by --- xen/arch/arm/mm.c | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index 01ae2cccc0..93ad118183 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -601,6 +601,19 @@ void __init remove_early_mappings(void) flush_xen_data_tlb_range_va(BOOT_FDT_VIRT_START, BOOT_FDT_SLOT_SIZE); } +/* + * After boot, Xen page-tables should not contain mapping that are both + * Writable and eXecutables. + * + * This should be called on each CPU to enforce the policy. + */ +static void xen_pt_enforce_wnx(void) +{ + WRITE_SYSREG32(READ_SYSREG32(SCTLR_EL2) | SCTLR_WXN, SCTLR_EL2); + /* Flush everything after setting WXN bit. */ + flush_xen_text_tlb_local(); +} + extern void switch_ttbr(uint64_t ttbr); /* Clear a translation table and clean & invalidate the cache */ @@ -702,10 +715,7 @@ void __init setup_pagetables(unsigned long boot_phys_offset) clear_table(boot_second); clear_table(boot_third); - /* From now on, no mapping may be both writable and executable. */ - WRITE_SYSREG32(READ_SYSREG32(SCTLR_EL2) | SCTLR_WXN, SCTLR_EL2); - /* Flush everything after setting WXN bit. */ - flush_xen_text_tlb_local(); + xen_pt_enforce_wnx(); #ifdef CONFIG_ARM_32 per_cpu(xen_pgtable, 0) = cpu0_pgtable; @@ -777,9 +787,7 @@ int init_secondary_pagetables(int cpu) /* MMU setup for secondary CPUS (which already have paging enabled) */ void mmu_init_secondary_cpu(void) { - /* From now on, no mapping may be both writable and executable. */ - WRITE_SYSREG32(READ_SYSREG32(SCTLR_EL2) | SCTLR_WXN, SCTLR_EL2); - flush_xen_text_tlb_local(); + xen_pt_enforce_wnx(); } #ifdef CONFIG_ARM_32 From patchwork Tue May 14 12:11:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 164138 Delivered-To: patch@linaro.org Received: by 2002:a92:9e1a:0:0:0:0:0 with SMTP id q26csp2540743ili; Tue, 14 May 2019 05:13:07 -0700 (PDT) X-Google-Smtp-Source: APXvYqxnbKLfiFDrkd06riE1h/I2fhHdx6O+BWUbnoxXXNmEh11WRPtVoHEdmNRkVnST1unF+PVe X-Received: by 2002:a24:7309:: with SMTP id y9mr3115399itb.162.1557835987434; Tue, 14 May 2019 05:13:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557835987; cv=none; d=google.com; s=arc-20160816; b=zR/YJfvcdxyNhDPKhyMCTuY0NHFN8DWUi5LHETMNjnz5f4O3a7wQNnDRf31z4FW5jP MBMADDPDH+QC3PumHqb4yIR1mUMyNYokfjh2paN6+EKGDFYDGdLhOB2n22QMuKFDN13B I2kXNtJqnIKJRKIGCUlIXj1uxcsg+3PQtum3cO/pS0KTMcP8MUSrLzfxBJZiHs7yBBp1 PSetaPDWOblTwRzqnvGe10+epTwAJHq1l6VewzE8B0DQpVCDUtL5RXSX2hYfuhX4fRCb UUOJdtJgBn6Am8GvXdwpVYwINiTVVPC3xBdCKQwpYzEDxVjjrhFs5WV8yjWJmG+k1MSr ePXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=iclZddht+DWiNIUSx4flRMaUt3yS0rXYIKQgnnxlCZk=; b=UnEK9elIRrd/F3kjyGVuvyEbVkRh6r8rLXAiQXrhI64ZR0/11JZ7EQxF/U1I6RwYxq UYfytQSLmX9ZqbMrQNfs+oG8kWR2k2kQbN/tZQ+cyopxATDotLDWsqhP8va5Gs7vrI2s iWK8VSFqpFc7zqMsVSbdFqU4Pa9bYs4aNUnrdL5hA2dguhl+nlCgl69qyQFGKMvQIMD7 oAzuaYTQBk9C+eE7yOGD2s6Ng0YBUfjSa9vtGkhujyNgiJ7wQC4cwUKnfFcSs8SWWj7Q R/IDYubjws3GhZsTVd6ggfOtFewLSBgSlcg9caZ+26w1xNfagLorroVvvd+VIW6P9WpM OaHA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id d129si1390085ita.94.2019.05.14.05.13.07 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 May 2019 05:13:07 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHa-0006GQ-NS; Tue, 14 May 2019 12:11:46 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHZ-0006FL-GP for xen-devel@lists.xenproject.org; Tue, 14 May 2019 12:11:45 +0000 X-Inumbo-ID: 7023b6cd-7641-11e9-8980-bc764e045a96 Received: from foss.arm.com (unknown [217.140.101.70]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTP id 7023b6cd-7641-11e9-8980-bc764e045a96; Tue, 14 May 2019 12:11:43 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 996F5341; Tue, 14 May 2019 05:11:43 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5F89E3F71E; Tue, 14 May 2019 05:11:42 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Tue, 14 May 2019 13:11:27 +0100 Message-Id: <20190514121132.26732-4-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190514121132.26732-1-julien.grall@arm.com> References: <20190514121132.26732-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH MM-PART1 v3 3/8] xen/arm: Remove flush_xen_text_tlb_local() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The function flush_xen_text_tlb_local() has been misused and will result to invalidate the instruction cache more than necessary. For instance, there is no need to invalidate the instruction cache if we are setting SCTLR_EL2.WXN. There is effectively only one caller (i.e free_init_memory() who would need to invalidate the instruction cache. So rather than keeping around the function flush_xen_text_tlb_local() replace it with call to flush_xen_tlb_local() and explicitely flush the cache when necessary. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov Reviewed-by: Stefano Stabellini --- Changes in v3: - Fix typoes Changes in v2: - Add Andrii's reviewed-by --- xen/arch/arm/mm.c | 17 ++++++++++++++--- xen/include/asm-arm/arm32/page.h | 23 +++++++++-------------- xen/include/asm-arm/arm64/page.h | 21 +++++---------------- 3 files changed, 28 insertions(+), 33 deletions(-) diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index 93ad118183..dfbe39c70a 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -610,8 +610,12 @@ void __init remove_early_mappings(void) static void xen_pt_enforce_wnx(void) { WRITE_SYSREG32(READ_SYSREG32(SCTLR_EL2) | SCTLR_WXN, SCTLR_EL2); - /* Flush everything after setting WXN bit. */ - flush_xen_text_tlb_local(); + /* + * The TLBs may cache SCTLR_EL2.WXN. So ensure it is synchronized + * before flushing the TLBs. + */ + isb(); + flush_xen_data_tlb_local(); } extern void switch_ttbr(uint64_t ttbr); @@ -1123,7 +1127,7 @@ static void set_pte_flags_on_range(const char *p, unsigned long l, enum mg mg) } write_pte(xen_xenmap + i, pte); } - flush_xen_text_tlb_local(); + flush_xen_data_tlb_local(); } /* Release all __init and __initdata ranges to be reused */ @@ -1136,6 +1140,13 @@ void free_init_memory(void) uint32_t *p; set_pte_flags_on_range(__init_begin, len, mg_rw); + + /* + * From now on, init will not be used for execution anymore, + * so nuke the instruction cache to remove entries related to init. + */ + invalidate_icache_local(); + #ifdef CONFIG_ARM_32 /* udf instruction i.e (see A8.8.247 in ARM DDI 0406C.c) */ insn = 0xe7f000f0; diff --git a/xen/include/asm-arm/arm32/page.h b/xen/include/asm-arm/arm32/page.h index ea4b312c70..40a77daa9d 100644 --- a/xen/include/asm-arm/arm32/page.h +++ b/xen/include/asm-arm/arm32/page.h @@ -46,24 +46,19 @@ static inline void invalidate_icache(void) } /* - * Flush all hypervisor mappings from the TLB and branch predictor of - * the local processor. - * - * This is needed after changing Xen code mappings. - * - * The caller needs to issue the necessary DSB and D-cache flushes - * before calling flush_xen_text_tlb. + * Invalidate all instruction caches on the local processor to PoU. + * We also need to flush the branch predictor for ARMv7 as it may be + * architecturally visible to the software (see B2.2.4 in ARM DDI 0406C.b). */ -static inline void flush_xen_text_tlb_local(void) +static inline void invalidate_icache_local(void) { asm volatile ( - "isb;" /* Ensure synchronization with previous changes to text */ - CMD_CP32(TLBIALLH) /* Flush hypervisor TLB */ - CMD_CP32(ICIALLU) /* Flush I-cache */ - CMD_CP32(BPIALL) /* Flush branch predictor */ - "dsb;" /* Ensure completion of TLB+BP flush */ - "isb;" + CMD_CP32(ICIALLU) /* Flush I-cache. */ + CMD_CP32(BPIALL) /* Flush branch predictor. */ : : : "memory"); + + dsb(nsh); /* Ensure completion of the flush I-cache */ + isb(); /* Synchronize fetched instruction stream. */ } /* diff --git a/xen/include/asm-arm/arm64/page.h b/xen/include/asm-arm/arm64/page.h index 23d778154d..6c36d0210f 100644 --- a/xen/include/asm-arm/arm64/page.h +++ b/xen/include/asm-arm/arm64/page.h @@ -37,23 +37,12 @@ static inline void invalidate_icache(void) isb(); } -/* - * Flush all hypervisor mappings from the TLB of the local processor. - * - * This is needed after changing Xen code mappings. - * - * The caller needs to issue the necessary DSB and D-cache flushes - * before calling flush_xen_text_tlb. - */ -static inline void flush_xen_text_tlb_local(void) +/* Invalidate all instruction caches on the local processor to PoU */ +static inline void invalidate_icache_local(void) { - asm volatile ( - "isb;" /* Ensure synchronization with previous changes to text */ - "tlbi alle2;" /* Flush hypervisor TLB */ - "ic iallu;" /* Flush I-cache */ - "dsb sy;" /* Ensure completion of TLB flush */ - "isb;" - : : : "memory"); + asm volatile ("ic iallu"); + dsb(nsh); /* Ensure completion of the I-cache flush */ + isb(); } /* From patchwork Tue May 14 12:11:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 164142 Delivered-To: patch@linaro.org Received: by 2002:a92:9e1a:0:0:0:0:0 with SMTP id q26csp2540910ili; Tue, 14 May 2019 05:13:16 -0700 (PDT) X-Google-Smtp-Source: APXvYqzSTLv3C+A3L0NBXX09Q5J4bvQDy0MX5MJTsHld/TLLWyTKx+5X4XQCYniW8h3X7sgSdYgs X-Received: by 2002:a05:660c:190:: with SMTP id v16mr3139159itj.0.1557835996761; Tue, 14 May 2019 05:13:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557835996; cv=none; d=google.com; s=arc-20160816; b=Je1pZNK3zYlddIs5+ZibgX9czPSzWqq2XfrLCIXtf6ncgDPX8h81x9ICfpUF4VTW+J a95M0fbFRBZHMxnnpzQTlA8wNNYQnsuceFPFjdWUzTSR2e+1CUDgM2KwYklFkmQBi4SB mHKHYTaNUJ/LMJtkfVqGwPCFr7TMrZmXLpivOr6SAJC8xbAPXQdhXn+tqQhIgJXCGPsP UpQVJYl/8qPgB+agpTfTa0CYTnLab7tG/M5Ad8lRzY7dg4evPuTFpc78vcA4GY7lB0K6 TWPBFn/BNVE5Z4cO9VLnGG7gFw6Xe8iGU6ZgXrdkNqzvMarkCqWcZKSdoWINmsY3qCM1 nwHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=lNMMQwgXRYnnZP1QZqbv1IplB6ZIReRMQwYRlH78GRI=; b=pY2j2w2rkcSoPIM93pofk7h3m5tdEwd+OliYdd6bd7aOWVRwA4jHrdXnZkNteAH7ie SVGj3DzGA9qQKJ50+G7khtkPghz1EPOAytdfICE+w12PLNE+GGzGqveRtkszQOtpkXF3 oi6EWVcxhay9S2HVzD6aKZz2tHLtXDYrvOs+b9jrNIeanfMWI9drMFknlIM2I4Eazj6+ 6gkI3QhIQYHGHmP4Hr3vvKQUydjxPyn5trZuRAzR7W/rAiR86QyISayZbThdGYuU3/bM k71hsRWYBZjbFEwy3TF+LSVslwdZt52yGuW4QxdORPw0IRHUb18myEwrdFe4F4dY+7jS eYzw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id s6si8676358iog.41.2019.05.14.05.13.16 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 May 2019 05:13:16 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHc-0006HU-2C; Tue, 14 May 2019 12:11:48 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHa-0006GD-HK for xen-devel@lists.xenproject.org; Tue, 14 May 2019 12:11:46 +0000 X-Inumbo-ID: 7124b5ee-7641-11e9-a4d6-4703e6a0277d Received: from foss.arm.com (unknown [217.140.101.70]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id 7124b5ee-7641-11e9-a4d6-4703e6a0277d; Tue, 14 May 2019 12:11:45 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1A5D815AB; Tue, 14 May 2019 05:11:45 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D7B1E3F71E; Tue, 14 May 2019 05:11:43 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Tue, 14 May 2019 13:11:28 +0100 Message-Id: <20190514121132.26732-5-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190514121132.26732-1-julien.grall@arm.com> References: <20190514121132.26732-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH MM-PART1 v3 4/8] xen/arm: tlbflush: Clarify the TLB helpers name X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" TLB helpers in the headers tlbflush.h are currently quite confusing to use the name may lead to think they are dealing with hypervisors TLBs while they actually deal with guest TLBs. Rename them to make it clearer that we are dealing with guest TLBs. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov Acked-by: Stefano Stabellini --- Changes in v3: - Add Stefano's acked-by Changes in v2: - Add Andrii's reviewed-by --- xen/arch/arm/p2m.c | 6 +++--- xen/arch/arm/smp.c | 2 +- xen/arch/arm/traps.c | 2 +- xen/include/asm-arm/arm32/flushtlb.h | 8 ++++---- xen/include/asm-arm/arm64/flushtlb.h | 8 ++++---- 5 files changed, 13 insertions(+), 13 deletions(-) diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c index c38bd7e16e..92c2413f20 100644 --- a/xen/arch/arm/p2m.c +++ b/xen/arch/arm/p2m.c @@ -151,7 +151,7 @@ void p2m_restore_state(struct vcpu *n) * when running multiple vCPU of the same domain on a single pCPU. */ if ( *last_vcpu_ran != INVALID_VCPU_ID && *last_vcpu_ran != n->vcpu_id ) - flush_tlb_local(); + flush_guest_tlb_local(); *last_vcpu_ran = n->vcpu_id; } @@ -196,7 +196,7 @@ static void p2m_force_tlb_flush_sync(struct p2m_domain *p2m) isb(); } - flush_tlb(); + flush_guest_tlb(); if ( ovttbr != READ_SYSREG64(VTTBR_EL2) ) { @@ -1969,7 +1969,7 @@ static void setup_virt_paging_one(void *data) WRITE_SYSREG(READ_SYSREG(HCR_EL2) | HCR_VM, HCR_EL2); isb(); - flush_tlb_all_local(); + flush_all_guests_tlb_local(); } } diff --git a/xen/arch/arm/smp.c b/xen/arch/arm/smp.c index 62f57f0ba2..ce1fcc8ef9 100644 --- a/xen/arch/arm/smp.c +++ b/xen/arch/arm/smp.c @@ -8,7 +8,7 @@ void flush_tlb_mask(const cpumask_t *mask) { /* No need to IPI other processors on ARM, the processor takes care of it. */ - flush_tlb_all(); + flush_all_guests_tlb(); } void smp_send_event_check_mask(const cpumask_t *mask) diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c index d8b9a8a0f0..1aba970415 100644 --- a/xen/arch/arm/traps.c +++ b/xen/arch/arm/traps.c @@ -1924,7 +1924,7 @@ static void do_trap_stage2_abort_guest(struct cpu_user_regs *regs, * still be inaccurate. */ if ( !is_data ) - flush_tlb_local(); + flush_guest_tlb_local(); rc = gva_to_ipa(gva, &gpa, GV2M_READ); /* diff --git a/xen/include/asm-arm/arm32/flushtlb.h b/xen/include/asm-arm/arm32/flushtlb.h index bbcc82f490..22e100eccf 100644 --- a/xen/include/asm-arm/arm32/flushtlb.h +++ b/xen/include/asm-arm/arm32/flushtlb.h @@ -2,7 +2,7 @@ #define __ASM_ARM_ARM32_FLUSHTLB_H__ /* Flush local TLBs, current VMID only */ -static inline void flush_tlb_local(void) +static inline void flush_guest_tlb_local(void) { dsb(sy); @@ -13,7 +13,7 @@ static inline void flush_tlb_local(void) } /* Flush inner shareable TLBs, current VMID only */ -static inline void flush_tlb(void) +static inline void flush_guest_tlb(void) { dsb(sy); @@ -24,7 +24,7 @@ static inline void flush_tlb(void) } /* Flush local TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_tlb_all_local(void) +static inline void flush_all_guests_tlb_local(void) { dsb(sy); @@ -35,7 +35,7 @@ static inline void flush_tlb_all_local(void) } /* Flush innershareable TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_tlb_all(void) +static inline void flush_all_guests_tlb(void) { dsb(sy); diff --git a/xen/include/asm-arm/arm64/flushtlb.h b/xen/include/asm-arm/arm64/flushtlb.h index 942f2d3992..adbbd5c522 100644 --- a/xen/include/asm-arm/arm64/flushtlb.h +++ b/xen/include/asm-arm/arm64/flushtlb.h @@ -2,7 +2,7 @@ #define __ASM_ARM_ARM64_FLUSHTLB_H__ /* Flush local TLBs, current VMID only */ -static inline void flush_tlb_local(void) +static inline void flush_guest_tlb_local(void) { asm volatile( "dsb sy;" @@ -13,7 +13,7 @@ static inline void flush_tlb_local(void) } /* Flush innershareable TLBs, current VMID only */ -static inline void flush_tlb(void) +static inline void flush_guest_tlb(void) { asm volatile( "dsb sy;" @@ -24,7 +24,7 @@ static inline void flush_tlb(void) } /* Flush local TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_tlb_all_local(void) +static inline void flush_all_guests_tlb_local(void) { asm volatile( "dsb sy;" @@ -35,7 +35,7 @@ static inline void flush_tlb_all_local(void) } /* Flush innershareable TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_tlb_all(void) +static inline void flush_all_guests_tlb(void) { asm volatile( "dsb sy;" From patchwork Tue May 14 12:11:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 164141 Delivered-To: patch@linaro.org Received: by 2002:a92:9e1a:0:0:0:0:0 with SMTP id q26csp2540814ili; Tue, 14 May 2019 05:13:11 -0700 (PDT) X-Google-Smtp-Source: APXvYqwfR2puiNcdOPXdclQQrAotd0sde01Ma8MFapp+alqrNkn8sOh7GL3O7mLkyN5lOU7Tp/BB X-Received: by 2002:a5e:c817:: with SMTP id y23mr20800972iol.290.1557835991201; Tue, 14 May 2019 05:13:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557835991; cv=none; d=google.com; s=arc-20160816; b=AyN19IchEPeRlA0TjbEcoG7yff1oegdta/gzFrkHFTvhcrmadrZg6JimjpllG8W66T +sOe45kR7/mt6vb49HzS9T9kqJrL9qUYTyF6zxYAObaLMCuwZYh5OGwEhCNTh6+ULUTy wV1SMYaVfIJae/fASSHqXNjdNXvDEYZc4aAFTqbBhSrhZv+iWxYQLriZ19UcOz1eB3tV X+3Y9N3duVfQTz/mpWMMsufltCKig6n+Krw983j4qaJkSZ+dzEG7QJduj1H3FFBKOcO7 CdVJm3vHI2SH4D0e5aGIMvjJ4vmUxPPTEpdXs+QLb9hOEaPhFtpnupdTfQIHwQ6IpwaX wuKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=PsY0/jy7yLQj+JfVQD18biixAqpMMy6miYM5Xm+urxg=; b=PXRXj+jffO3vlTX+hIgwown+5kw9Rsw6eIrACg/WHIPEVe+HHtVHd8pA4KsFv+zeUr kSJB9aukSTiumHSoEkP7XamJ2c0F0BfWCB/A0R1OEDG2s0MZm0/upwMKbln1nm1AaQc4 I4bIgOgnv2oewx/19ndSJKHLpCHlexI62MSxtnKV3d86Yg0KJ9NtCG3IJozT99um+CAw QHUuqW9i7scZiHiCWQpzJUSiE+HOQlSePP/vsdwMCMB+Iye/Kfkb3khOCzB/482TjNtZ TmQE35+lfjRSaYJbnHL7qJ5aztG7x4jtuSzUKwzr3a8cKClM02as7fXF5bP2d7bU7dI9 ynZQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id t136si1407520ita.48.2019.05.14.05.13.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 May 2019 05:13:11 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHc-0006Ho-Ed; Tue, 14 May 2019 12:11:48 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHb-0006HI-Q5 for xen-devel@lists.xenproject.org; Tue, 14 May 2019 12:11:47 +0000 X-Inumbo-ID: 71f86d3a-7641-11e9-a832-d3dd06e529ef Received: from foss.arm.com (unknown [217.140.101.70]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id 71f86d3a-7641-11e9-a832-d3dd06e529ef; Tue, 14 May 2019 12:11:47 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AF3FA15AD; Tue, 14 May 2019 05:11:46 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 58B5F3F71E; Tue, 14 May 2019 05:11:45 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Tue, 14 May 2019 13:11:29 +0100 Message-Id: <20190514121132.26732-6-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190514121132.26732-1-julien.grall@arm.com> References: <20190514121132.26732-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH MM-PART1 v3 5/8] xen/arm: page: Clarify the Xen TLBs helpers name X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Now that we dropped flush_xen_text_tlb_local(), we have only one set of helpers acting on Xen TLBs. There naming are quite confusing because the TLB instructions used will act on both Data and Instruction TLBs. Take the opportunity to rework the documentation which can be confusing to read as they don't match the implementation. Note the mention about the instruction cache maintenance has been removed because modifying mapping does not require instruction cache maintenance. Lastly, switch from unsigned long to vaddr_t as the function technically deal with virtual address. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov --- Changes in v3: - Update commit message - Fix typoes Changes in v2: - Add Andrii's reviewed-by --- xen/arch/arm/mm.c | 18 +++++++++--------- xen/include/asm-arm/arm32/page.h | 15 +++++---------- xen/include/asm-arm/arm64/page.h | 15 +++++---------- xen/include/asm-arm/page.h | 28 ++++++++++++++-------------- 4 files changed, 33 insertions(+), 43 deletions(-) diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index dfbe39c70a..8ee828d445 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -335,7 +335,7 @@ void set_fixmap(unsigned map, mfn_t mfn, unsigned int flags) pte.pt.table = 1; /* 4k mappings always have this bit set */ pte.pt.xn = 1; write_pte(xen_fixmap + third_table_offset(FIXMAP_ADDR(map)), pte); - flush_xen_data_tlb_range_va(FIXMAP_ADDR(map), PAGE_SIZE); + flush_xen_tlb_range_va(FIXMAP_ADDR(map), PAGE_SIZE); } /* Remove a mapping from a fixmap entry */ @@ -343,7 +343,7 @@ void clear_fixmap(unsigned map) { lpae_t pte = {0}; write_pte(xen_fixmap + third_table_offset(FIXMAP_ADDR(map)), pte); - flush_xen_data_tlb_range_va(FIXMAP_ADDR(map), PAGE_SIZE); + flush_xen_tlb_range_va(FIXMAP_ADDR(map), PAGE_SIZE); } /* Create Xen's mappings of memory. @@ -377,7 +377,7 @@ static void __init create_mappings(lpae_t *second, write_pte(p + i, pte); pte.pt.base += 1 << LPAE_SHIFT; } - flush_xen_data_tlb_local(); + flush_xen_tlb_local(); } #ifdef CONFIG_DOMAIN_PAGE @@ -455,7 +455,7 @@ void *map_domain_page(mfn_t mfn) * We may not have flushed this specific subpage at map time, * since we only flush the 4k page not the superpage */ - flush_xen_data_tlb_range_va_local(va, PAGE_SIZE); + flush_xen_tlb_range_va_local(va, PAGE_SIZE); return (void *)va; } @@ -598,7 +598,7 @@ void __init remove_early_mappings(void) write_pte(xen_second + second_table_offset(BOOT_FDT_VIRT_START), pte); write_pte(xen_second + second_table_offset(BOOT_FDT_VIRT_START + SZ_2M), pte); - flush_xen_data_tlb_range_va(BOOT_FDT_VIRT_START, BOOT_FDT_SLOT_SIZE); + flush_xen_tlb_range_va(BOOT_FDT_VIRT_START, BOOT_FDT_SLOT_SIZE); } /* @@ -615,7 +615,7 @@ static void xen_pt_enforce_wnx(void) * before flushing the TLBs. */ isb(); - flush_xen_data_tlb_local(); + flush_xen_tlb_local(); } extern void switch_ttbr(uint64_t ttbr); @@ -879,7 +879,7 @@ void __init setup_xenheap_mappings(unsigned long base_mfn, vaddr += FIRST_SIZE; } - flush_xen_data_tlb_local(); + flush_xen_tlb_local(); } #endif @@ -1052,7 +1052,7 @@ static int create_xen_entries(enum xenmap_operation op, BUG(); } } - flush_xen_data_tlb_range_va(virt, PAGE_SIZE * nr_mfns); + flush_xen_tlb_range_va(virt, PAGE_SIZE * nr_mfns); rc = 0; @@ -1127,7 +1127,7 @@ static void set_pte_flags_on_range(const char *p, unsigned long l, enum mg mg) } write_pte(xen_xenmap + i, pte); } - flush_xen_data_tlb_local(); + flush_xen_tlb_local(); } /* Release all __init and __initdata ranges to be reused */ diff --git a/xen/include/asm-arm/arm32/page.h b/xen/include/asm-arm/arm32/page.h index 40a77daa9d..0b41b9214b 100644 --- a/xen/include/asm-arm/arm32/page.h +++ b/xen/include/asm-arm/arm32/page.h @@ -61,12 +61,8 @@ static inline void invalidate_icache_local(void) isb(); /* Synchronize fetched instruction stream. */ } -/* - * Flush all hypervisor mappings from the data TLB of the local - * processor. This is not sufficient when changing code mappings or - * for self modifying code. - */ -static inline void flush_xen_data_tlb_local(void) +/* Flush all hypervisor mappings from the TLB of the local processor. */ +static inline void flush_xen_tlb_local(void) { asm volatile("dsb;" /* Ensure preceding are visible */ CMD_CP32(TLBIALLH) @@ -76,14 +72,13 @@ static inline void flush_xen_data_tlb_local(void) } /* Flush TLB of local processor for address va. */ -static inline void __flush_xen_data_tlb_one_local(vaddr_t va) +static inline void __flush_xen_tlb_one_local(vaddr_t va) { asm volatile(STORE_CP32(0, TLBIMVAH) : : "r" (va) : "memory"); } -/* Flush TLB of all processors in the inner-shareable domain for - * address va. */ -static inline void __flush_xen_data_tlb_one(vaddr_t va) +/* Flush TLB of all processors in the inner-shareable domain for address va. */ +static inline void __flush_xen_tlb_one(vaddr_t va) { asm volatile(STORE_CP32(0, TLBIMVAHIS) : : "r" (va) : "memory"); } diff --git a/xen/include/asm-arm/arm64/page.h b/xen/include/asm-arm/arm64/page.h index 6c36d0210f..31d04ecf76 100644 --- a/xen/include/asm-arm/arm64/page.h +++ b/xen/include/asm-arm/arm64/page.h @@ -45,12 +45,8 @@ static inline void invalidate_icache_local(void) isb(); } -/* - * Flush all hypervisor mappings from the data TLB of the local - * processor. This is not sufficient when changing code mappings or - * for self modifying code. - */ -static inline void flush_xen_data_tlb_local(void) +/* Flush all hypervisor mappings from the TLB of the local processor. */ +static inline void flush_xen_tlb_local(void) { asm volatile ( "dsb sy;" /* Ensure visibility of PTE writes */ @@ -61,14 +57,13 @@ static inline void flush_xen_data_tlb_local(void) } /* Flush TLB of local processor for address va. */ -static inline void __flush_xen_data_tlb_one_local(vaddr_t va) +static inline void __flush_xen_tlb_one_local(vaddr_t va) { asm volatile("tlbi vae2, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); } -/* Flush TLB of all processors in the inner-shareable domain for - * address va. */ -static inline void __flush_xen_data_tlb_one(vaddr_t va) +/* Flush TLB of all processors in the inner-shareable domain for address va. */ +static inline void __flush_xen_tlb_one(vaddr_t va) { asm volatile("tlbi vae2is, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); } diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h index 1a1713ce02..195345e24a 100644 --- a/xen/include/asm-arm/page.h +++ b/xen/include/asm-arm/page.h @@ -234,18 +234,18 @@ static inline int clean_and_invalidate_dcache_va_range } while (0) /* - * Flush a range of VA's hypervisor mappings from the data TLB of the - * local processor. This is not sufficient when changing code mappings - * or for self modifying code. + * Flush a range of VA's hypervisor mappings from the TLB of the local + * processor. */ -static inline void flush_xen_data_tlb_range_va_local(unsigned long va, - unsigned long size) +static inline void flush_xen_tlb_range_va_local(vaddr_t va, + unsigned long size) { - unsigned long end = va + size; + vaddr_t end = va + size; + dsb(sy); /* Ensure preceding are visible */ while ( va < end ) { - __flush_xen_data_tlb_one_local(va); + __flush_xen_tlb_one_local(va); va += PAGE_SIZE; } dsb(sy); /* Ensure completion of the TLB flush */ @@ -253,18 +253,18 @@ static inline void flush_xen_data_tlb_range_va_local(unsigned long va, } /* - * Flush a range of VA's hypervisor mappings from the data TLB of all - * processors in the inner-shareable domain. This is not sufficient - * when changing code mappings or for self modifying code. + * Flush a range of VA's hypervisor mappings from the TLB of all + * processors in the inner-shareable domain. */ -static inline void flush_xen_data_tlb_range_va(unsigned long va, - unsigned long size) +static inline void flush_xen_tlb_range_va(vaddr_t va, + unsigned long size) { - unsigned long end = va + size; + vaddr_t end = va + size; + dsb(sy); /* Ensure preceding are visible */ while ( va < end ) { - __flush_xen_data_tlb_one(va); + __flush_xen_tlb_one(va); va += PAGE_SIZE; } dsb(sy); /* Ensure completion of the TLB flush */ From patchwork Tue May 14 12:11:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 164140 Delivered-To: patch@linaro.org Received: by 2002:a92:9e1a:0:0:0:0:0 with SMTP id q26csp2540791ili; Tue, 14 May 2019 05:13:10 -0700 (PDT) X-Google-Smtp-Source: APXvYqyM5ZRyyfJKUCi6Y9IcFIVHK1am+EKISh9i3gN+YZjYdldverZvxtjUwy0OCH3IzCE9jcmQ X-Received: by 2002:a05:660c:b02:: with SMTP id f2mr3259909itk.36.1557835990280; Tue, 14 May 2019 05:13:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557835990; cv=none; d=google.com; s=arc-20160816; b=YQXoXfv9i5AoVOJ5tkJJqVBUTduRXF7byyrgejLIutBTSHv9MeRC1mZD7g/se82W7o QIiKRwWGXG/8rhzymn9u9LDGdwitVmfbs8dJCgAx5L7i5tgPJ8h3/NsixgkXJqmE52BN Nx+51XX25PFjSc+8EzAxBcXErC7A0rHzvTOmT2YCrCdPy7sZnQ961MxRt8YDovNneAv6 DFzFfTpk/xBOtDIBvFaa/ocvYEES6HjIdNby+Q8KKPvvM7h0JDbzTYR2kXMLdegifmv0 7+U/V3mNcli32+iAD4N14rblYRC0QvNS8lsJGUzXSs/GgTxwp38ZH+0vKk0qvLrfm0OA CvUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=296nWOIdD8CG10ZnJrzh6KlRoWEZk/F2RrPHIPTfgf8=; b=x6ZJJBYF3AZpG/Z0wkB8figJ/flh1SA9/3gerTNJ0yWTMv6VjKYf+nzNA7sN/0nJWU ebtWaYRYIWrgJRK1ATkNwgYijaTnlL19HkTROqAmmkdcHZx+0wwaNsIl6Sz4k28y5zW6 Cpd2oVztwhI6uB2Rue9Xm1j2p35OXXGy//T5gqYLOLnV5nX6BmDRXUIjUC1tczWB8J3J vgpoMIfr3fCLSiBXhjD4Ns0VS4gOIS0Are4kA0TKjc68Y/ocMD3x9f/hB4+auXiwqyhf +mmlK8dtWj7zBl1DiQWzUKQ/TaXLy+X5jgRTxZ+M/4dQqTcVf9BSNvawZlViw2xq974E beGw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id c17si1396899itc.40.2019.05.14.05.13.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 May 2019 05:13:10 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHe-0006Jx-OQ; Tue, 14 May 2019 12:11:50 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHd-0006If-18 for xen-devel@lists.xenproject.org; Tue, 14 May 2019 12:11:49 +0000 X-Inumbo-ID: 72e168aa-7641-11e9-919d-6349c0b3b1ca Received: from foss.arm.com (unknown [217.140.101.70]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id 72e168aa-7641-11e9-919d-6349c0b3b1ca; Tue, 14 May 2019 12:11:48 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2C8F0341; Tue, 14 May 2019 05:11:48 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id ED6F33F71E; Tue, 14 May 2019 05:11:46 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Tue, 14 May 2019 13:11:30 +0100 Message-Id: <20190514121132.26732-7-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190514121132.26732-1-julien.grall@arm.com> References: <20190514121132.26732-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH MM-PART1 v3 6/8] xen/arm: Gather all TLB flush helpers in tlbflush.h X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" At the moment, TLB helpers are scattered in 2 headers: page.h (for Xen TLB helpers) and tlbflush.h (for guest TLB helpers). This patch is gathering all of them in tlbflush. This will help to uniformize and update the logic of the helpers in follow-up patches. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov Acked-by: Stefano Stabellini --- Changes in v3: - Add Stefano's acked-by Changes in v2: - Add Andrii's reviewed-by --- xen/include/asm-arm/arm32/flushtlb.h | 22 +++++++++++++++++++++ xen/include/asm-arm/arm32/page.h | 22 --------------------- xen/include/asm-arm/arm64/flushtlb.h | 23 ++++++++++++++++++++++ xen/include/asm-arm/arm64/page.h | 23 ---------------------- xen/include/asm-arm/flushtlb.h | 38 ++++++++++++++++++++++++++++++++++++ xen/include/asm-arm/page.h | 38 ------------------------------------ 6 files changed, 83 insertions(+), 83 deletions(-) diff --git a/xen/include/asm-arm/arm32/flushtlb.h b/xen/include/asm-arm/arm32/flushtlb.h index 22e100eccf..b629db61cb 100644 --- a/xen/include/asm-arm/arm32/flushtlb.h +++ b/xen/include/asm-arm/arm32/flushtlb.h @@ -45,6 +45,28 @@ static inline void flush_all_guests_tlb(void) isb(); } +/* Flush all hypervisor mappings from the TLB of the local processor. */ +static inline void flush_xen_tlb_local(void) +{ + asm volatile("dsb;" /* Ensure preceding are visible */ + CMD_CP32(TLBIALLH) + "dsb;" /* Ensure completion of the TLB flush */ + "isb;" + : : : "memory"); +} + +/* Flush TLB of local processor for address va. */ +static inline void __flush_xen_tlb_one_local(vaddr_t va) +{ + asm volatile(STORE_CP32(0, TLBIMVAH) : : "r" (va) : "memory"); +} + +/* Flush TLB of all processors in the inner-shareable domain for address va. */ +static inline void __flush_xen_tlb_one(vaddr_t va) +{ + asm volatile(STORE_CP32(0, TLBIMVAHIS) : : "r" (va) : "memory"); +} + #endif /* __ASM_ARM_ARM32_FLUSHTLB_H__ */ /* * Local variables: diff --git a/xen/include/asm-arm/arm32/page.h b/xen/include/asm-arm/arm32/page.h index 0b41b9214b..715a9e4fef 100644 --- a/xen/include/asm-arm/arm32/page.h +++ b/xen/include/asm-arm/arm32/page.h @@ -61,28 +61,6 @@ static inline void invalidate_icache_local(void) isb(); /* Synchronize fetched instruction stream. */ } -/* Flush all hypervisor mappings from the TLB of the local processor. */ -static inline void flush_xen_tlb_local(void) -{ - asm volatile("dsb;" /* Ensure preceding are visible */ - CMD_CP32(TLBIALLH) - "dsb;" /* Ensure completion of the TLB flush */ - "isb;" - : : : "memory"); -} - -/* Flush TLB of local processor for address va. */ -static inline void __flush_xen_tlb_one_local(vaddr_t va) -{ - asm volatile(STORE_CP32(0, TLBIMVAH) : : "r" (va) : "memory"); -} - -/* Flush TLB of all processors in the inner-shareable domain for address va. */ -static inline void __flush_xen_tlb_one(vaddr_t va) -{ - asm volatile(STORE_CP32(0, TLBIMVAHIS) : : "r" (va) : "memory"); -} - /* Ask the MMU to translate a VA for us */ static inline uint64_t __va_to_par(vaddr_t va) { diff --git a/xen/include/asm-arm/arm64/flushtlb.h b/xen/include/asm-arm/arm64/flushtlb.h index adbbd5c522..2fed34b2ec 100644 --- a/xen/include/asm-arm/arm64/flushtlb.h +++ b/xen/include/asm-arm/arm64/flushtlb.h @@ -45,6 +45,29 @@ static inline void flush_all_guests_tlb(void) : : : "memory"); } +/* Flush all hypervisor mappings from the TLB of the local processor. */ +static inline void flush_xen_tlb_local(void) +{ + asm volatile ( + "dsb sy;" /* Ensure visibility of PTE writes */ + "tlbi alle2;" /* Flush hypervisor TLB */ + "dsb sy;" /* Ensure completion of TLB flush */ + "isb;" + : : : "memory"); +} + +/* Flush TLB of local processor for address va. */ +static inline void __flush_xen_tlb_one_local(vaddr_t va) +{ + asm volatile("tlbi vae2, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); +} + +/* Flush TLB of all processors in the inner-shareable domain for address va. */ +static inline void __flush_xen_tlb_one(vaddr_t va) +{ + asm volatile("tlbi vae2is, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); +} + #endif /* __ASM_ARM_ARM64_FLUSHTLB_H__ */ /* * Local variables: diff --git a/xen/include/asm-arm/arm64/page.h b/xen/include/asm-arm/arm64/page.h index 31d04ecf76..0cba266373 100644 --- a/xen/include/asm-arm/arm64/page.h +++ b/xen/include/asm-arm/arm64/page.h @@ -45,29 +45,6 @@ static inline void invalidate_icache_local(void) isb(); } -/* Flush all hypervisor mappings from the TLB of the local processor. */ -static inline void flush_xen_tlb_local(void) -{ - asm volatile ( - "dsb sy;" /* Ensure visibility of PTE writes */ - "tlbi alle2;" /* Flush hypervisor TLB */ - "dsb sy;" /* Ensure completion of TLB flush */ - "isb;" - : : : "memory"); -} - -/* Flush TLB of local processor for address va. */ -static inline void __flush_xen_tlb_one_local(vaddr_t va) -{ - asm volatile("tlbi vae2, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); -} - -/* Flush TLB of all processors in the inner-shareable domain for address va. */ -static inline void __flush_xen_tlb_one(vaddr_t va) -{ - asm volatile("tlbi vae2is, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); -} - /* Ask the MMU to translate a VA for us */ static inline uint64_t __va_to_par(vaddr_t va) { diff --git a/xen/include/asm-arm/flushtlb.h b/xen/include/asm-arm/flushtlb.h index 83ff9fa8b3..ab1aae5c90 100644 --- a/xen/include/asm-arm/flushtlb.h +++ b/xen/include/asm-arm/flushtlb.h @@ -28,6 +28,44 @@ static inline void page_set_tlbflush_timestamp(struct page_info *page) /* Flush specified CPUs' TLBs */ void flush_tlb_mask(const cpumask_t *mask); +/* + * Flush a range of VA's hypervisor mappings from the TLB of the local + * processor. + */ +static inline void flush_xen_tlb_range_va_local(vaddr_t va, + unsigned long size) +{ + vaddr_t end = va + size; + + dsb(sy); /* Ensure preceding are visible */ + while ( va < end ) + { + __flush_xen_tlb_one_local(va); + va += PAGE_SIZE; + } + dsb(sy); /* Ensure completion of the TLB flush */ + isb(); +} + +/* + * Flush a range of VA's hypervisor mappings from the TLB of all + * processors in the inner-shareable domain. + */ +static inline void flush_xen_tlb_range_va(vaddr_t va, + unsigned long size) +{ + vaddr_t end = va + size; + + dsb(sy); /* Ensure preceding are visible */ + while ( va < end ) + { + __flush_xen_tlb_one(va); + va += PAGE_SIZE; + } + dsb(sy); /* Ensure completion of the TLB flush */ + isb(); +} + #endif /* __ASM_ARM_FLUSHTLB_H__ */ /* * Local variables: diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h index 195345e24a..2bcdb0f1a5 100644 --- a/xen/include/asm-arm/page.h +++ b/xen/include/asm-arm/page.h @@ -233,44 +233,6 @@ static inline int clean_and_invalidate_dcache_va_range : : "r" (_p), "m" (*_p)); \ } while (0) -/* - * Flush a range of VA's hypervisor mappings from the TLB of the local - * processor. - */ -static inline void flush_xen_tlb_range_va_local(vaddr_t va, - unsigned long size) -{ - vaddr_t end = va + size; - - dsb(sy); /* Ensure preceding are visible */ - while ( va < end ) - { - __flush_xen_tlb_one_local(va); - va += PAGE_SIZE; - } - dsb(sy); /* Ensure completion of the TLB flush */ - isb(); -} - -/* - * Flush a range of VA's hypervisor mappings from the TLB of all - * processors in the inner-shareable domain. - */ -static inline void flush_xen_tlb_range_va(vaddr_t va, - unsigned long size) -{ - vaddr_t end = va + size; - - dsb(sy); /* Ensure preceding are visible */ - while ( va < end ) - { - __flush_xen_tlb_one(va); - va += PAGE_SIZE; - } - dsb(sy); /* Ensure completion of the TLB flush */ - isb(); -} - /* Flush the dcache for an entire page. */ void flush_page_to_ram(unsigned long mfn, bool sync_icache); From patchwork Tue May 14 12:11:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 164139 Delivered-To: patch@linaro.org Received: by 2002:a92:9e1a:0:0:0:0:0 with SMTP id q26csp2540774ili; Tue, 14 May 2019 05:13:09 -0700 (PDT) X-Google-Smtp-Source: APXvYqzVPq62xDgSjMBmu3HvX23LhITEvYn2BVPQuTCuUzys2QcJqkI6S3i1e7AUBJHxIfVjiIKk X-Received: by 2002:a24:194c:: with SMTP id b73mr3147630itb.157.1557835989668; Tue, 14 May 2019 05:13:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557835989; cv=none; d=google.com; s=arc-20160816; b=OmJDaOX7gjpXIEbsARWb/hZ077AH+fnpYNnkgFKlZk7YUxVJpePBX0kIOazKUjHgl3 mfxKkbV2vB7/IF19pY86u3SiYu4OPFaid6Vaw4R7TYdtYE3CFOrejLuMUakImq39bR6s 9gwxyaKpTqKmHGvy9dcM9KNakI5qZ+8ZgWyk7rFuTw8siJyC/fR2gN8p6wD4U1FLOly7 iQKvEmh2ffFreZvcNwSEjZHcKA4MapOKb7R99GQnofHBGxJlbbQWbQ5xPW660Oa8KXHu QPdcm4CsjbhiU0sH5u5EQD/rNA+NbzZejMtcLwsHobJHUvFaCIN9K7ezUnAfEAVcsTf6 K9zA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=6s0wbddSKHZBuwpPx9Dgxd11ATe8+3bn80carkeP2c4=; b=BVXJfgUgSNW9T/ij9mDrCNvI6T+nFh2colMsas9yLWedxlLOpjGjCx6F6cYhRH3//m M58aYmH17y9cbZL+PvCULpp4Zt2tNEfeSQHbL2eYJ/YLZExUcwNxYDyCMO7b3qfEw0Vr bHZ4Z79OH7sIgoMS4li6opxWM0pee+s6Bk4TQjaQnWM49PbT9NdvDvdtAKUGRBkUdYXj K79khkiIO8LRvrP211IY3S96aB8spYNGdApK/YrvPC7cizNS8vKSeZJk1BJhfa0Pd5sW n2PGt2SooikugfowwZDbleDZkvpwkRETGhtLoYQfVBMbk0kKY523acjaB0rQTVTYj94f TuYg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id a16si10316654jac.77.2019.05.14.05.13.09 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 May 2019 05:13:09 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHh-0006Ma-9u; Tue, 14 May 2019 12:11:53 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHf-0006Kf-8q for xen-devel@lists.xenproject.org; Tue, 14 May 2019 12:11:51 +0000 X-Inumbo-ID: 73c1c2ed-7641-11e9-8980-bc764e045a96 Received: from foss.arm.com (unknown [217.140.101.70]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTP id 73c1c2ed-7641-11e9-8980-bc764e045a96; Tue, 14 May 2019 12:11:49 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9E1C215AB; Tue, 14 May 2019 05:11:49 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6AFDB3F71E; Tue, 14 May 2019 05:11:48 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Tue, 14 May 2019 13:11:31 +0100 Message-Id: <20190514121132.26732-8-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190514121132.26732-1-julien.grall@arm.com> References: <20190514121132.26732-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH MM-PART1 v3 7/8] xen/arm: tlbflush: Rework TLB helpers X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" All the TLBs helpers invalidate all the TLB entries are using the same pattern: DSB SY TLBI ... DSB SY ISB This pattern is following pattern recommended by the Arm Arm to ensure visibility of updates to translation tables (see K11.5.2 in ARM DDI 0487D.b). We have been a bit too eager in Xen and use system-wide DSBs when this can be limited to the inner-shareable domain. Furthermore, the first DSB can be restrict further to only store in the inner-shareable domain. This is because the DSB is here to ensure visibility of the update to translation table walks. Lastly, there are a lack of documentation in most of the TLBs helper. Rather than trying to update the helpers one by one, this patch introduce a per-arch macro to generate the TLB helpers. This will be easier to update the TLBs helper in the future and the documentation. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov Reviewed-by: Stefano Stabellini --- Changes in v3: - Add Stefano's reviewed-by Changes in v2: - Update the reference to the Arm Arm to the latest spec - Add Andrii's reviewed-by --- xen/include/asm-arm/arm32/flushtlb.h | 73 ++++++++++++++-------------------- xen/include/asm-arm/arm64/flushtlb.h | 76 +++++++++++++++--------------------- 2 files changed, 60 insertions(+), 89 deletions(-) diff --git a/xen/include/asm-arm/arm32/flushtlb.h b/xen/include/asm-arm/arm32/flushtlb.h index b629db61cb..9085e65011 100644 --- a/xen/include/asm-arm/arm32/flushtlb.h +++ b/xen/include/asm-arm/arm32/flushtlb.h @@ -1,59 +1,44 @@ #ifndef __ASM_ARM_ARM32_FLUSHTLB_H__ #define __ASM_ARM_ARM32_FLUSHTLB_H__ -/* Flush local TLBs, current VMID only */ -static inline void flush_guest_tlb_local(void) -{ - dsb(sy); - - WRITE_CP32((uint32_t) 0, TLBIALL); - - dsb(sy); - isb(); +/* + * Every invalidation operation use the following patterns: + * + * DSB ISHST // Ensure prior page-tables updates have completed + * TLBI... // Invalidate the TLB + * DSB ISH // Ensure the TLB invalidation has completed + * ISB // See explanation below + * + * For Xen page-tables the ISB will discard any instructions fetched + * from the old mappings. + * + * For the Stage-2 page-tables the ISB ensures the completion of the DSB + * (and therefore the TLB invalidation) before continuing. So we know + * the TLBs cannot contain an entry for a mapping we may have removed. + */ +#define TLB_HELPER(name, tlbop) \ +static inline void name(void) \ +{ \ + dsb(ishst); \ + WRITE_CP32(0, tlbop); \ + dsb(ish); \ + isb(); \ } -/* Flush inner shareable TLBs, current VMID only */ -static inline void flush_guest_tlb(void) -{ - dsb(sy); - - WRITE_CP32((uint32_t) 0, TLBIALLIS); +/* Flush local TLBs, current VMID only */ +TLB_HELPER(flush_guest_tlb_local, TLBIALL); - dsb(sy); - isb(); -} +/* Flush inner shareable TLBs, current VMID only */ +TLB_HELPER(flush_guest_tlb, TLBIALLIS); /* Flush local TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_all_guests_tlb_local(void) -{ - dsb(sy); - - WRITE_CP32((uint32_t) 0, TLBIALLNSNH); - - dsb(sy); - isb(); -} +TLB_HELPER(flush_all_guests_tlb_local, TLBIALLNSNH); /* Flush innershareable TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_all_guests_tlb(void) -{ - dsb(sy); - - WRITE_CP32((uint32_t) 0, TLBIALLNSNHIS); - - dsb(sy); - isb(); -} +TLB_HELPER(flush_all_guests_tlb, TLBIALLNSNHIS); /* Flush all hypervisor mappings from the TLB of the local processor. */ -static inline void flush_xen_tlb_local(void) -{ - asm volatile("dsb;" /* Ensure preceding are visible */ - CMD_CP32(TLBIALLH) - "dsb;" /* Ensure completion of the TLB flush */ - "isb;" - : : : "memory"); -} +TLB_HELPER(flush_xen_tlb_local, TLBIALLH); /* Flush TLB of local processor for address va. */ static inline void __flush_xen_tlb_one_local(vaddr_t va) diff --git a/xen/include/asm-arm/arm64/flushtlb.h b/xen/include/asm-arm/arm64/flushtlb.h index 2fed34b2ec..ceec59542e 100644 --- a/xen/include/asm-arm/arm64/flushtlb.h +++ b/xen/include/asm-arm/arm64/flushtlb.h @@ -1,60 +1,46 @@ #ifndef __ASM_ARM_ARM64_FLUSHTLB_H__ #define __ASM_ARM_ARM64_FLUSHTLB_H__ -/* Flush local TLBs, current VMID only */ -static inline void flush_guest_tlb_local(void) -{ - asm volatile( - "dsb sy;" - "tlbi vmalls12e1;" - "dsb sy;" - "isb;" - : : : "memory"); +/* + * Every invalidation operation use the following patterns: + * + * DSB ISHST // Ensure prior page-tables updates have completed + * TLBI... // Invalidate the TLB + * DSB ISH // Ensure the TLB invalidation has completed + * ISB // See explanation below + * + * For Xen page-tables the ISB will discard any instructions fetched + * from the old mappings. + * + * For the Stage-2 page-tables the ISB ensures the completion of the DSB + * (and therefore the TLB invalidation) before continuing. So we know + * the TLBs cannot contain an entry for a mapping we may have removed. + */ +#define TLB_HELPER(name, tlbop) \ +static inline void name(void) \ +{ \ + asm volatile( \ + "dsb ishst;" \ + "tlbi " # tlbop ";" \ + "dsb ish;" \ + "isb;" \ + : : : "memory"); \ } +/* Flush local TLBs, current VMID only. */ +TLB_HELPER(flush_guest_tlb_local, vmalls12e1); + /* Flush innershareable TLBs, current VMID only */ -static inline void flush_guest_tlb(void) -{ - asm volatile( - "dsb sy;" - "tlbi vmalls12e1is;" - "dsb sy;" - "isb;" - : : : "memory"); -} +TLB_HELPER(flush_guest_tlb, vmalls12e1is); /* Flush local TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_all_guests_tlb_local(void) -{ - asm volatile( - "dsb sy;" - "tlbi alle1;" - "dsb sy;" - "isb;" - : : : "memory"); -} +TLB_HELPER(flush_all_guests_tlb_local, alle1); /* Flush innershareable TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_all_guests_tlb(void) -{ - asm volatile( - "dsb sy;" - "tlbi alle1is;" - "dsb sy;" - "isb;" - : : : "memory"); -} +TLB_HELPER(flush_all_guests_tlb, alle1is); /* Flush all hypervisor mappings from the TLB of the local processor. */ -static inline void flush_xen_tlb_local(void) -{ - asm volatile ( - "dsb sy;" /* Ensure visibility of PTE writes */ - "tlbi alle2;" /* Flush hypervisor TLB */ - "dsb sy;" /* Ensure completion of TLB flush */ - "isb;" - : : : "memory"); -} +TLB_HELPER(flush_xen_tlb_local, alle2); /* Flush TLB of local processor for address va. */ static inline void __flush_xen_tlb_one_local(vaddr_t va) From patchwork Tue May 14 12:11:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 164143 Delivered-To: patch@linaro.org Received: by 2002:a92:9e1a:0:0:0:0:0 with SMTP id q26csp2541049ili; Tue, 14 May 2019 05:13:23 -0700 (PDT) X-Google-Smtp-Source: APXvYqxn9kvt9f8i3ivZGEPaDnXKnVZ9EU8YAB1HaCR4/caeKpsodxVBISvIRotC43X9PbfulwVz X-Received: by 2002:a02:9685:: with SMTP id w5mr23516147jai.131.1557836003916; Tue, 14 May 2019 05:13:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557836003; cv=none; d=google.com; s=arc-20160816; b=Jj2gdBGk2uP/gbO4foYIAUPF2K6Pd0wfPbLaSneKaEogZjbeeOjtdgfZVkZhtsD4Dl zsAc8b4imiN6VtzNSTLS7Gcm7JBSUz5E4fZsVkHM5eLnxpyLnWgoKGTV4vLY/yynI+Sq Yd38/z05/tgu8Mei9nliDv3Vjk0mHsSTtcMvt7YnRQFvHBXBeBhBOkx/N5onR1JGRUzb aKKJ6IFPz3p9Lk0mNEm4PAdBLOBwaXBLtSX4yMD1UszUG6ND/bzlTTgUUmWVDzxGG3uN DmpK5k4JDQDI3Vp1WizYswSCw/aWAf3fdNE5FFl+OXTrONfKPX71+P7OUoV6CymsZ5mb BtBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=g3Z4pVaRU9xc9T/B8NCdJWDiIuJKE3zj4IDLXRCZPkI=; b=gEYewxJeXZz6ct2bgaYXko3m92omf3+vmPKb86HVJZ0aIoajl+y6zCYgErEPDrsz4R iPOtD8JdbvlBGEjRYEfPAHAg6CZYWVy78048Cc6PZnMet6t0VjSyrygYS6j1e4VKSff1 c1r/epWLVoer99wAjCi0bmePvkyOH4YXyTaSnZ5i17gMewX1Mz5tyaawq2dwmLHBBslr qjwyIusI6hEQGYQ56AMp+avIivALiQGwGvJVVOFi9YmG8dmsWBqv7bCcea+IQzcYXV7p vYGLsdMmE5BJSenLxu3EXFJGIcO6+lsLy9Bn6HfmfwajHHVorEVbzjQ4H8PXYoxjiCUB EnQw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id m73si1489296itm.73.2019.05.14.05.13.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 May 2019 05:13:23 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHh-0006N0-MM; Tue, 14 May 2019 12:11:53 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hQWHg-0006Lr-8Q for xen-devel@lists.xenproject.org; Tue, 14 May 2019 12:11:52 +0000 X-Inumbo-ID: 74a179be-7641-11e9-99f4-1f59df7d2d55 Received: from foss.arm.com (unknown [217.140.101.70]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id 74a179be-7641-11e9-99f4-1f59df7d2d55; Tue, 14 May 2019 12:11:51 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1B93415AD; Tue, 14 May 2019 05:11:51 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DC6B13F71E; Tue, 14 May 2019 05:11:49 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Tue, 14 May 2019 13:11:32 +0100 Message-Id: <20190514121132.26732-9-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190514121132.26732-1-julien.grall@arm.com> References: <20190514121132.26732-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH MM-PART1 v3 8/8] xen/arm: mm: Flush the TLBs even if a mapping failed in create_xen_entries X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" At the moment, create_xen_entries will only flush the TLBs if the full range has successfully been updated. This may lead to leave unwanted entries in the TLBs if we fail to update some entries. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov Reviewed-by: Stefano Stabellini --- Changes in v2: - Add Andrii's reviewed-by - Add Stefano's reviewed-by --- xen/arch/arm/mm.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index 8ee828d445..9d584e4cbf 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -984,7 +984,7 @@ static int create_xen_entries(enum xenmap_operation op, unsigned long nr_mfns, unsigned int flags) { - int rc; + int rc = 0; unsigned long addr = virt, addr_end = addr + nr_mfns * PAGE_SIZE; lpae_t pte, *entry; lpae_t *third = NULL; @@ -1013,7 +1013,8 @@ static int create_xen_entries(enum xenmap_operation op, { printk("%s: trying to replace an existing mapping addr=%lx mfn=%"PRI_mfn"\n", __func__, addr, mfn_x(mfn)); - return -EINVAL; + rc = -EINVAL; + goto out; } if ( op == RESERVE ) break; @@ -1030,7 +1031,8 @@ static int create_xen_entries(enum xenmap_operation op, { printk("%s: trying to %s a non-existing mapping addr=%lx\n", __func__, op == REMOVE ? "remove" : "modify", addr); - return -EINVAL; + rc = -EINVAL; + goto out; } if ( op == REMOVE ) pte.bits = 0; @@ -1043,7 +1045,8 @@ static int create_xen_entries(enum xenmap_operation op, { printk("%s: Incorrect combination for addr=%lx\n", __func__, addr); - return -EINVAL; + rc = -EINVAL; + goto out; } } write_pte(entry, pte); @@ -1052,11 +1055,14 @@ static int create_xen_entries(enum xenmap_operation op, BUG(); } } +out: + /* + * Flush the TLBs even in case of failure because we may have + * partially modified the PT. This will prevent any unexpected + * behavior afterwards. + */ flush_xen_tlb_range_va(virt, PAGE_SIZE * nr_mfns); - rc = 0; - -out: return rc; }