From patchwork Wed May 8 16:15:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 163634 Delivered-To: patch@linaro.org Received: by 2002:a05:6e02:81:0:0:0:0 with SMTP id l1csp2885541ilm; Wed, 8 May 2019 09:17:40 -0700 (PDT) X-Google-Smtp-Source: APXvYqyOsLWqrdpW7AV6c5/1+EV1/E1knXTS9n/PjJWUePGPzRHt8ElSoBIC67a/vqZGmoN+heR5 X-Received: by 2002:a24:4315:: with SMTP id s21mr4276217itb.84.1557332260722; Wed, 08 May 2019 09:17:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557332260; cv=none; d=google.com; s=arc-20160816; b=yacAGjeif4+9GP27+iD74ABe/qG3ZELgj7NMfAPstUTQF23PuzbLf/RLlkv1LTx1ho oVqB08kUgCqPzNaEps6/qJuuNSc88Mi0WdALLBzbK5rkG4ESdsapr2DVqO/yfScWBuLs vUpzgPW9IsbuNAL1mAPzAJ0c4Ud5gd7rVudNSwR/50nme/3Y9f/Dvik3+gPLT+gZLea8 deQ76UcU4ROiTPyMHpXPO2H5wkZf+QySi7ZHsgwbUd17Zjkcs6RiZKjPqinU8SlUIx29 Py7Z8UFRsFWvmavWFmQIpwuDnGzD71hU9Ws18Nhpq/JsFi/MP0uoQTRYGNEd+//e2grd DLQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=3Zh803serRdnZphRTkShksmmdUCxVeuY+ncr1SbSnwQ=; b=opJVeSSFaVniW3+KLi0ROB/ctgl1hyVo7c06S3WVivdt+OB7KhrEWzlxfyQcOCHiQO ScW1sXzladshMcD0wZSYrZs9vB5cskAUrV4/lGvT95bjumHRbUHhwlk6XLBDURBSh15j bw2T07s5Oq2Fk8uD86+CCuth6WwFBQ9OxO1pGgH70Z5t60NA00hh5zAgqgiQc0gJRXuA v34Jkqhh7FHDLwQkSxTVdSeRF/DhpmS7z57a3oHHql4vjfhyeqdLePK+p/DZy8+u/YFv LkUXh5kXLa5QzlBYZ6B1DkWBPMtYneR6eUBT4Unhn9iacZrdhICRXSXQTEjdoSdFJxME TL8w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id k195si2281162itb.11.2019.05.08.09.17.40 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 08 May 2019 09:17:40 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hOPEq-0008EW-3P; Wed, 08 May 2019 16:16:12 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hOPEo-0008EE-T5 for xen-devel@lists.xenproject.org; Wed, 08 May 2019 16:16:10 +0000 X-Inumbo-ID: 976a1332-71ac-11e9-90eb-eb305858670c Received: from foss.arm.com (unknown [217.140.101.70]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id 976a1332-71ac-11e9-90eb-eb305858670c; Wed, 08 May 2019 16:16:10 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B64EC80D; Wed, 8 May 2019 09:16:09 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 830353F238; Wed, 8 May 2019 09:16:08 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Wed, 8 May 2019 17:15:57 +0100 Message-Id: <20190508161603.21964-2-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190508161603.21964-1-julien.grall@arm.com> References: <20190508161603.21964-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH v2 1/7] xen/arm: mm: Consolidate setting SCTLR_EL2.WXN in a single place X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The logic to set SCTLR_EL2.WXN is the same for the boot CPU and non-boot CPU. So introduce a function to set the bit and clear TLBs. This new function will help us to document and update the logic in a single place. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov Reviewed-by: Stefano Stabellini --- Changes in v2: - Fix typo in the commit message - Add Andrii's reviewed-by --- xen/arch/arm/mm.c | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index 01ae2cccc0..93ad118183 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -601,6 +601,19 @@ void __init remove_early_mappings(void) flush_xen_data_tlb_range_va(BOOT_FDT_VIRT_START, BOOT_FDT_SLOT_SIZE); } +/* + * After boot, Xen page-tables should not contain mapping that are both + * Writable and eXecutables. + * + * This should be called on each CPU to enforce the policy. + */ +static void xen_pt_enforce_wnx(void) +{ + WRITE_SYSREG32(READ_SYSREG32(SCTLR_EL2) | SCTLR_WXN, SCTLR_EL2); + /* Flush everything after setting WXN bit. */ + flush_xen_text_tlb_local(); +} + extern void switch_ttbr(uint64_t ttbr); /* Clear a translation table and clean & invalidate the cache */ @@ -702,10 +715,7 @@ void __init setup_pagetables(unsigned long boot_phys_offset) clear_table(boot_second); clear_table(boot_third); - /* From now on, no mapping may be both writable and executable. */ - WRITE_SYSREG32(READ_SYSREG32(SCTLR_EL2) | SCTLR_WXN, SCTLR_EL2); - /* Flush everything after setting WXN bit. */ - flush_xen_text_tlb_local(); + xen_pt_enforce_wnx(); #ifdef CONFIG_ARM_32 per_cpu(xen_pgtable, 0) = cpu0_pgtable; @@ -777,9 +787,7 @@ int init_secondary_pagetables(int cpu) /* MMU setup for secondary CPUS (which already have paging enabled) */ void mmu_init_secondary_cpu(void) { - /* From now on, no mapping may be both writable and executable. */ - WRITE_SYSREG32(READ_SYSREG32(SCTLR_EL2) | SCTLR_WXN, SCTLR_EL2); - flush_xen_text_tlb_local(); + xen_pt_enforce_wnx(); } #ifdef CONFIG_ARM_32 From patchwork Wed May 8 16:15:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 163632 Delivered-To: patch@linaro.org Received: by 2002:a05:6e02:81:0:0:0:0 with SMTP id l1csp2885501ilm; Wed, 8 May 2019 09:17:39 -0700 (PDT) X-Google-Smtp-Source: APXvYqxC1WzW/UV1E7/0qPl3cbmd5PHVMow90KJFiHO8GaZm1OfdF4pyF1iCgjs7tOF+tgnzFRVF X-Received: by 2002:a6b:400a:: with SMTP id k10mr2891659ioa.291.1557332259087; Wed, 08 May 2019 09:17:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557332259; cv=none; d=google.com; s=arc-20160816; b=VEt5emt/ticioXBzhzqN2nKa2DJYwX2dwpYDG6Z4NTLpD629+1b2BF44JWu3jyRXe7 TarqGsk54vIebTAkLtxWovnBXEfcJvZCrjpxXOinavJc2AWChf3rs3uAEeLzuofYG24G Uk2bRp3LUn+VvYbtFpjLUzBfFE+8esvL+VxOKML32xcjCRNa4BLE0x7dsv3/5q2s4W0P yQjk8bLtWkLpak1twQLYSynN3Jc9TQ4QrI4HCrrMq7DLQbvcMkuQxno4oEjVVtzjbLUc 4fAq6FXIf4cfvk3izKKxLyUA8Z9s695f/5dJ1QCcOLxvKu3H3mxvlKRCtODzmjVBePVX krAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=NoZMvxkSkprMe5xAdHfiM6TdqLi/731CfK3ogVRKyJs=; b=A320iKEPr5ri6v06zFl30Bo0iVznbk9XkPB0FFTcO0Dij6B1NJ22YhQyXBjfu3PAXl FfF11urSws4X5HFL1S2xEz6YV8XAwv6UwHAekuHH754oJfrwf/G0Iu83y0l+/7erIDxB qP1yt/yv2+RkZ3/A2M6Fz8ufN8D0mZPdwktcEJ7d1spqu4sexLz3M3rTF7vZFQm56fRt YbQCg4NiGDww4kY3UiXo0iuUObYx80YE1iNpdqYFmSYtwpDshOFyZv3SRJ8FAP3AmySL 5GtW3TDU/JHgFI+Fa0g8Vj3nLi9I6m8NuT7hmsG2R66BUJo1CLYvEQ5Ddk2MKuAPJjUV CTtA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id t11si2228234itb.34.2019.05.08.09.17.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 08 May 2019 09:17:39 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hOPEs-0008Ez-Cf; Wed, 08 May 2019 16:16:14 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hOPEq-0008Eg-JT for xen-devel@lists.xenproject.org; Wed, 08 May 2019 16:16:12 +0000 X-Inumbo-ID: 9839b0ba-71ac-11e9-843c-bc764e045a96 Received: from foss.arm.com (unknown [217.140.101.70]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTP id 9839b0ba-71ac-11e9-843c-bc764e045a96; Wed, 08 May 2019 16:16:11 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 32B5815AB; Wed, 8 May 2019 09:16:11 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F3A763F238; Wed, 8 May 2019 09:16:09 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Wed, 8 May 2019 17:15:58 +0100 Message-Id: <20190508161603.21964-3-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190508161603.21964-1-julien.grall@arm.com> References: <20190508161603.21964-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH v2 2/7] xen/arm: Remove flush_xen_text_tlb_local() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The function flush_xen_text_tlb_local() has been misused and will result to invalidate the instruction cache more than necessary. For instance, there are no need to invalidate the instruction cache if we are setting SCTLR_EL2.WXN. There are effectively only one caller (i.e free_init_memory() would who need to invalidate the instruction cache. So rather than keeping around the function flush_xen_text_tlb_local() around, replace it with call to flush_xen_tlb_local() and explicitely flush the cache when necessary. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov --- Changes in v2: - Add Andrii's reviewed-by --- xen/arch/arm/mm.c | 17 ++++++++++++++--- xen/include/asm-arm/arm32/page.h | 23 +++++++++-------------- xen/include/asm-arm/arm64/page.h | 21 +++++---------------- 3 files changed, 28 insertions(+), 33 deletions(-) diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index 93ad118183..dfbe39c70a 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -610,8 +610,12 @@ void __init remove_early_mappings(void) static void xen_pt_enforce_wnx(void) { WRITE_SYSREG32(READ_SYSREG32(SCTLR_EL2) | SCTLR_WXN, SCTLR_EL2); - /* Flush everything after setting WXN bit. */ - flush_xen_text_tlb_local(); + /* + * The TLBs may cache SCTLR_EL2.WXN. So ensure it is synchronized + * before flushing the TLBs. + */ + isb(); + flush_xen_data_tlb_local(); } extern void switch_ttbr(uint64_t ttbr); @@ -1123,7 +1127,7 @@ static void set_pte_flags_on_range(const char *p, unsigned long l, enum mg mg) } write_pte(xen_xenmap + i, pte); } - flush_xen_text_tlb_local(); + flush_xen_data_tlb_local(); } /* Release all __init and __initdata ranges to be reused */ @@ -1136,6 +1140,13 @@ void free_init_memory(void) uint32_t *p; set_pte_flags_on_range(__init_begin, len, mg_rw); + + /* + * From now on, init will not be used for execution anymore, + * so nuke the instruction cache to remove entries related to init. + */ + invalidate_icache_local(); + #ifdef CONFIG_ARM_32 /* udf instruction i.e (see A8.8.247 in ARM DDI 0406C.c) */ insn = 0xe7f000f0; diff --git a/xen/include/asm-arm/arm32/page.h b/xen/include/asm-arm/arm32/page.h index ea4b312c70..40a77daa9d 100644 --- a/xen/include/asm-arm/arm32/page.h +++ b/xen/include/asm-arm/arm32/page.h @@ -46,24 +46,19 @@ static inline void invalidate_icache(void) } /* - * Flush all hypervisor mappings from the TLB and branch predictor of - * the local processor. - * - * This is needed after changing Xen code mappings. - * - * The caller needs to issue the necessary DSB and D-cache flushes - * before calling flush_xen_text_tlb. + * Invalidate all instruction caches on the local processor to PoU. + * We also need to flush the branch predictor for ARMv7 as it may be + * architecturally visible to the software (see B2.2.4 in ARM DDI 0406C.b). */ -static inline void flush_xen_text_tlb_local(void) +static inline void invalidate_icache_local(void) { asm volatile ( - "isb;" /* Ensure synchronization with previous changes to text */ - CMD_CP32(TLBIALLH) /* Flush hypervisor TLB */ - CMD_CP32(ICIALLU) /* Flush I-cache */ - CMD_CP32(BPIALL) /* Flush branch predictor */ - "dsb;" /* Ensure completion of TLB+BP flush */ - "isb;" + CMD_CP32(ICIALLU) /* Flush I-cache. */ + CMD_CP32(BPIALL) /* Flush branch predictor. */ : : : "memory"); + + dsb(nsh); /* Ensure completion of the flush I-cache */ + isb(); /* Synchronize fetched instruction stream. */ } /* diff --git a/xen/include/asm-arm/arm64/page.h b/xen/include/asm-arm/arm64/page.h index 23d778154d..6c36d0210f 100644 --- a/xen/include/asm-arm/arm64/page.h +++ b/xen/include/asm-arm/arm64/page.h @@ -37,23 +37,12 @@ static inline void invalidate_icache(void) isb(); } -/* - * Flush all hypervisor mappings from the TLB of the local processor. - * - * This is needed after changing Xen code mappings. - * - * The caller needs to issue the necessary DSB and D-cache flushes - * before calling flush_xen_text_tlb. - */ -static inline void flush_xen_text_tlb_local(void) +/* Invalidate all instruction caches on the local processor to PoU */ +static inline void invalidate_icache_local(void) { - asm volatile ( - "isb;" /* Ensure synchronization with previous changes to text */ - "tlbi alle2;" /* Flush hypervisor TLB */ - "ic iallu;" /* Flush I-cache */ - "dsb sy;" /* Ensure completion of TLB flush */ - "isb;" - : : : "memory"); + asm volatile ("ic iallu"); + dsb(nsh); /* Ensure completion of the I-cache flush */ + isb(); } /* From patchwork Wed May 8 16:15:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 163633 Delivered-To: patch@linaro.org Received: by 2002:a05:6e02:81:0:0:0:0 with SMTP id l1csp2885505ilm; Wed, 8 May 2019 09:17:39 -0700 (PDT) X-Google-Smtp-Source: APXvYqzCh4TG9/CjMm/WP9iLJLUJ9VH0+DSZ0QcF2PVWCe1VtHUwPSt1VTS91d4sM7jDZAoP0EcQ X-Received: by 2002:a05:6638:154:: with SMTP id y20mr29297518jao.100.1557332259188; Wed, 08 May 2019 09:17:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557332259; cv=none; d=google.com; s=arc-20160816; b=bRtA1vZVM2QaDi410Yd4ZgfwQERGK9UReSjeAowMHV/nWGMxlK3YW6xuqQsJsFKSmt KXNf5aXC7i6BmAxsQdL9VvTM1Tjy0tY9js6EHR7vrOoIN1y+xbN0ov13IaQDaCYu0e7G F87c0P4R3FdDXlcPMpUIGvWTfy5JaGqs/Qse/eGlT2RdtC+KteAvOSmyObzzohjDujTR +Ql1lQHDW34OnOfLrfOO4c/wW+PcACFhvShCWstdMWxAujN6aillGlLUi1GBVnSTAb2q r1/BQStiaFyYk+5fKHOv0MZLNeSFE19KQef+MCOe2PQqoKo7DtH7d0Xgyq/Wx7oUyhHE Uakw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=6IwjXIO+Z9rc1qGHwzbDc3zlzNl0PZv63r008KGRc40=; b=mAY7n6y5fVXlwAASmbvSFMq3oabLLQWhsVXGIeew70R5Jny9DZZJB8PsDVQCgcP7Cv 9te3xwJAMPV0HM9rFnKwNhHLAbDAEMQa2Iu0Nk+c69a0TWIPWYwr8M13IRGY24sz81Ej J2VmOuYUqOXAR8048SdMJKWqYt8jcPHPCzKwK/KKsm0CtD/K5NT30gasF4Kd9aDoHr+3 QufkCMd4fQb/euyTFnnz3Se0XeDHjLi7keytpjh1e6A8atrF9GAPZgVtlmIN11w5U6DY KlcaXvmzf9LOf3Lx5d1krGqsH7wXdPVEzJgfiJf+I0oSPuKg1E1uX22L//FKPI/pmNI8 t9Ug== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id k8si2236942itk.98.2019.05.08.09.17.39 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 08 May 2019 09:17:39 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hOPEs-0008F7-Lg; Wed, 08 May 2019 16:16:14 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hOPEr-0008Es-RE for xen-devel@lists.xenproject.org; Wed, 08 May 2019 16:16:13 +0000 X-Inumbo-ID: 991d23c9-71ac-11e9-843c-bc764e045a96 Received: from foss.arm.com (unknown [217.140.101.70]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTP id 991d23c9-71ac-11e9-843c-bc764e045a96; Wed, 08 May 2019 16:16:12 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A3FE180D; Wed, 8 May 2019 09:16:12 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 70C3B3F238; Wed, 8 May 2019 09:16:11 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Wed, 8 May 2019 17:15:59 +0100 Message-Id: <20190508161603.21964-4-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190508161603.21964-1-julien.grall@arm.com> References: <20190508161603.21964-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH v2 3/7] xen/arm: tlbflush: Clarify the TLB helpers name X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" TLB helpers in the headers tlbflush.h are currently quite confusing to use the name may lead to think they are dealing with hypervisors TLBs while they actually deal with guest TLBs. Rename them to make it clearer that we are dealing with guest TLBs. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov Acked-by: Stefano Stabellini --- Changes in v2: - Add Andrii's reviewed-by --- xen/arch/arm/p2m.c | 6 +++--- xen/arch/arm/smp.c | 2 +- xen/arch/arm/traps.c | 2 +- xen/include/asm-arm/arm32/flushtlb.h | 8 ++++---- xen/include/asm-arm/arm64/flushtlb.h | 8 ++++---- 5 files changed, 13 insertions(+), 13 deletions(-) diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c index c38bd7e16e..92c2413f20 100644 --- a/xen/arch/arm/p2m.c +++ b/xen/arch/arm/p2m.c @@ -151,7 +151,7 @@ void p2m_restore_state(struct vcpu *n) * when running multiple vCPU of the same domain on a single pCPU. */ if ( *last_vcpu_ran != INVALID_VCPU_ID && *last_vcpu_ran != n->vcpu_id ) - flush_tlb_local(); + flush_guest_tlb_local(); *last_vcpu_ran = n->vcpu_id; } @@ -196,7 +196,7 @@ static void p2m_force_tlb_flush_sync(struct p2m_domain *p2m) isb(); } - flush_tlb(); + flush_guest_tlb(); if ( ovttbr != READ_SYSREG64(VTTBR_EL2) ) { @@ -1969,7 +1969,7 @@ static void setup_virt_paging_one(void *data) WRITE_SYSREG(READ_SYSREG(HCR_EL2) | HCR_VM, HCR_EL2); isb(); - flush_tlb_all_local(); + flush_all_guests_tlb_local(); } } diff --git a/xen/arch/arm/smp.c b/xen/arch/arm/smp.c index 62f57f0ba2..ce1fcc8ef9 100644 --- a/xen/arch/arm/smp.c +++ b/xen/arch/arm/smp.c @@ -8,7 +8,7 @@ void flush_tlb_mask(const cpumask_t *mask) { /* No need to IPI other processors on ARM, the processor takes care of it. */ - flush_tlb_all(); + flush_all_guests_tlb(); } void smp_send_event_check_mask(const cpumask_t *mask) diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c index d8b9a8a0f0..1aba970415 100644 --- a/xen/arch/arm/traps.c +++ b/xen/arch/arm/traps.c @@ -1924,7 +1924,7 @@ static void do_trap_stage2_abort_guest(struct cpu_user_regs *regs, * still be inaccurate. */ if ( !is_data ) - flush_tlb_local(); + flush_guest_tlb_local(); rc = gva_to_ipa(gva, &gpa, GV2M_READ); /* diff --git a/xen/include/asm-arm/arm32/flushtlb.h b/xen/include/asm-arm/arm32/flushtlb.h index bbcc82f490..22e100eccf 100644 --- a/xen/include/asm-arm/arm32/flushtlb.h +++ b/xen/include/asm-arm/arm32/flushtlb.h @@ -2,7 +2,7 @@ #define __ASM_ARM_ARM32_FLUSHTLB_H__ /* Flush local TLBs, current VMID only */ -static inline void flush_tlb_local(void) +static inline void flush_guest_tlb_local(void) { dsb(sy); @@ -13,7 +13,7 @@ static inline void flush_tlb_local(void) } /* Flush inner shareable TLBs, current VMID only */ -static inline void flush_tlb(void) +static inline void flush_guest_tlb(void) { dsb(sy); @@ -24,7 +24,7 @@ static inline void flush_tlb(void) } /* Flush local TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_tlb_all_local(void) +static inline void flush_all_guests_tlb_local(void) { dsb(sy); @@ -35,7 +35,7 @@ static inline void flush_tlb_all_local(void) } /* Flush innershareable TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_tlb_all(void) +static inline void flush_all_guests_tlb(void) { dsb(sy); diff --git a/xen/include/asm-arm/arm64/flushtlb.h b/xen/include/asm-arm/arm64/flushtlb.h index 942f2d3992..adbbd5c522 100644 --- a/xen/include/asm-arm/arm64/flushtlb.h +++ b/xen/include/asm-arm/arm64/flushtlb.h @@ -2,7 +2,7 @@ #define __ASM_ARM_ARM64_FLUSHTLB_H__ /* Flush local TLBs, current VMID only */ -static inline void flush_tlb_local(void) +static inline void flush_guest_tlb_local(void) { asm volatile( "dsb sy;" @@ -13,7 +13,7 @@ static inline void flush_tlb_local(void) } /* Flush innershareable TLBs, current VMID only */ -static inline void flush_tlb(void) +static inline void flush_guest_tlb(void) { asm volatile( "dsb sy;" @@ -24,7 +24,7 @@ static inline void flush_tlb(void) } /* Flush local TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_tlb_all_local(void) +static inline void flush_all_guests_tlb_local(void) { asm volatile( "dsb sy;" @@ -35,7 +35,7 @@ static inline void flush_tlb_all_local(void) } /* Flush innershareable TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_tlb_all(void) +static inline void flush_all_guests_tlb(void) { asm volatile( "dsb sy;" From patchwork Wed May 8 16:16:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 163638 Delivered-To: patch@linaro.org Received: by 2002:a05:6e02:81:0:0:0:0 with SMTP id l1csp2885583ilm; Wed, 8 May 2019 09:17:43 -0700 (PDT) X-Google-Smtp-Source: APXvYqx2o3bXpPGi9xO1oV5Zr9Dt8H0V9zzeBI5BC5aa1iIn0vTmuuZes9MAM7ntPS4THguaQ5qb X-Received: by 2002:a02:1d07:: with SMTP id 7mr28698031jaj.122.1557332263017; Wed, 08 May 2019 09:17:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557332263; cv=none; d=google.com; s=arc-20160816; b=Yp/L8bhd78twbokCYS8t2HWR/XWXiDA4mvff44Fn5SSVR2wwCwzHFosm0xskCpY7if Y2VhGrHGcP5cIduwq8bCrviv15BD+1TPobcvTU5vkYc4u0kca2CpEH0LgjQC55dx6txU OU3hVKLsQcMR6qMIj5/IC+6L3yF6AbduofzFb/Gqv8uVZZwnbUIZDRvszsVJu/ZnmRAk G1rRyri8V4ZP1YQ+6KPlRZtAgXHhoKROuLUMthg6OvjjIaH5MAldhhF0uHBMNTw6lh2E YpSudoGRJ8MNJImkTPxKUdEETIM0kmdvuNDeg00VJj9Nc/BzL36XmpMDp8WB4H2wP2BR E94g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=SkM3vuhEkpdW5toqPQi/R0bIf7nR2bwm4tKhMrt31t0=; b=goeqvf3d1PzeekOMIlAyid9C+Wn9yzE838Rp8IIQpnKc6buda6/um5mdRli9nqb4uc zd77Ofi4tL29L4kNKv17CrJxBSjemsrbkirTYuBz2jxwYI0IsApa9WEwJSFKVMekmuCX cbC+GyDwBNQBqzAgcBKPlWjNh8cJ1rUkzp1QtTqgMoIfsZmNA9ZpNPJ1hq8ZfMuRcudC saUXgT12WVXchbIDD4n1VbYzLCJRTsuJBdz+ub3kiVsnu30WZN2MX/1OpGdWiRHqZ6mj 0rb9bM3JjcZbuui58kUW3WUW4VEQy1++S/USu48Kv7J/S7YE/nHI37F6+exbR8VrEOZQ RAhA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id w82si2253209itb.0.2019.05.08.09.17.42 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 08 May 2019 09:17:43 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hOPEv-0008GI-0O; Wed, 08 May 2019 16:16:17 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hOPEt-0008FN-NB for xen-devel@lists.xenproject.org; Wed, 08 May 2019 16:16:15 +0000 X-Inumbo-ID: 9a089764-71ac-11e9-843c-bc764e045a96 Received: from foss.arm.com (unknown [217.140.101.70]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTP id 9a089764-71ac-11e9-843c-bc764e045a96; Wed, 08 May 2019 16:16:14 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4473B374; Wed, 8 May 2019 09:16:14 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E1F963F238; Wed, 8 May 2019 09:16:12 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Wed, 8 May 2019 17:16:00 +0100 Message-Id: <20190508161603.21964-5-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190508161603.21964-1-julien.grall@arm.com> References: <20190508161603.21964-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH v2 4/7] xen/arm: page: Clarify the Xen TLBs helpers name X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Now that we dropped flush_xen_text_tlb_local(), we have only one set of helpers acting on Xen TLBs. There naming are quite confusing because the TLB instructions used will act on both Data and Instruction TLBs. Take the opportunity to rework the documentation that can be confusing to read as they don't match the implementation. Lastly, switch from unsigned lont to vaddr_t as the function technically deal with virtual address. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov Acked-by: Stefano Stabellini --- Changes in v2: - Add Andrii's reviewed-by --- xen/arch/arm/mm.c | 18 +++++++++--------- xen/include/asm-arm/arm32/page.h | 15 +++++---------- xen/include/asm-arm/arm64/page.h | 15 +++++---------- xen/include/asm-arm/page.h | 28 ++++++++++++++-------------- 4 files changed, 33 insertions(+), 43 deletions(-) diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index dfbe39c70a..8ee828d445 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -335,7 +335,7 @@ void set_fixmap(unsigned map, mfn_t mfn, unsigned int flags) pte.pt.table = 1; /* 4k mappings always have this bit set */ pte.pt.xn = 1; write_pte(xen_fixmap + third_table_offset(FIXMAP_ADDR(map)), pte); - flush_xen_data_tlb_range_va(FIXMAP_ADDR(map), PAGE_SIZE); + flush_xen_tlb_range_va(FIXMAP_ADDR(map), PAGE_SIZE); } /* Remove a mapping from a fixmap entry */ @@ -343,7 +343,7 @@ void clear_fixmap(unsigned map) { lpae_t pte = {0}; write_pte(xen_fixmap + third_table_offset(FIXMAP_ADDR(map)), pte); - flush_xen_data_tlb_range_va(FIXMAP_ADDR(map), PAGE_SIZE); + flush_xen_tlb_range_va(FIXMAP_ADDR(map), PAGE_SIZE); } /* Create Xen's mappings of memory. @@ -377,7 +377,7 @@ static void __init create_mappings(lpae_t *second, write_pte(p + i, pte); pte.pt.base += 1 << LPAE_SHIFT; } - flush_xen_data_tlb_local(); + flush_xen_tlb_local(); } #ifdef CONFIG_DOMAIN_PAGE @@ -455,7 +455,7 @@ void *map_domain_page(mfn_t mfn) * We may not have flushed this specific subpage at map time, * since we only flush the 4k page not the superpage */ - flush_xen_data_tlb_range_va_local(va, PAGE_SIZE); + flush_xen_tlb_range_va_local(va, PAGE_SIZE); return (void *)va; } @@ -598,7 +598,7 @@ void __init remove_early_mappings(void) write_pte(xen_second + second_table_offset(BOOT_FDT_VIRT_START), pte); write_pte(xen_second + second_table_offset(BOOT_FDT_VIRT_START + SZ_2M), pte); - flush_xen_data_tlb_range_va(BOOT_FDT_VIRT_START, BOOT_FDT_SLOT_SIZE); + flush_xen_tlb_range_va(BOOT_FDT_VIRT_START, BOOT_FDT_SLOT_SIZE); } /* @@ -615,7 +615,7 @@ static void xen_pt_enforce_wnx(void) * before flushing the TLBs. */ isb(); - flush_xen_data_tlb_local(); + flush_xen_tlb_local(); } extern void switch_ttbr(uint64_t ttbr); @@ -879,7 +879,7 @@ void __init setup_xenheap_mappings(unsigned long base_mfn, vaddr += FIRST_SIZE; } - flush_xen_data_tlb_local(); + flush_xen_tlb_local(); } #endif @@ -1052,7 +1052,7 @@ static int create_xen_entries(enum xenmap_operation op, BUG(); } } - flush_xen_data_tlb_range_va(virt, PAGE_SIZE * nr_mfns); + flush_xen_tlb_range_va(virt, PAGE_SIZE * nr_mfns); rc = 0; @@ -1127,7 +1127,7 @@ static void set_pte_flags_on_range(const char *p, unsigned long l, enum mg mg) } write_pte(xen_xenmap + i, pte); } - flush_xen_data_tlb_local(); + flush_xen_tlb_local(); } /* Release all __init and __initdata ranges to be reused */ diff --git a/xen/include/asm-arm/arm32/page.h b/xen/include/asm-arm/arm32/page.h index 40a77daa9d..0b41b9214b 100644 --- a/xen/include/asm-arm/arm32/page.h +++ b/xen/include/asm-arm/arm32/page.h @@ -61,12 +61,8 @@ static inline void invalidate_icache_local(void) isb(); /* Synchronize fetched instruction stream. */ } -/* - * Flush all hypervisor mappings from the data TLB of the local - * processor. This is not sufficient when changing code mappings or - * for self modifying code. - */ -static inline void flush_xen_data_tlb_local(void) +/* Flush all hypervisor mappings from the TLB of the local processor. */ +static inline void flush_xen_tlb_local(void) { asm volatile("dsb;" /* Ensure preceding are visible */ CMD_CP32(TLBIALLH) @@ -76,14 +72,13 @@ static inline void flush_xen_data_tlb_local(void) } /* Flush TLB of local processor for address va. */ -static inline void __flush_xen_data_tlb_one_local(vaddr_t va) +static inline void __flush_xen_tlb_one_local(vaddr_t va) { asm volatile(STORE_CP32(0, TLBIMVAH) : : "r" (va) : "memory"); } -/* Flush TLB of all processors in the inner-shareable domain for - * address va. */ -static inline void __flush_xen_data_tlb_one(vaddr_t va) +/* Flush TLB of all processors in the inner-shareable domain for address va. */ +static inline void __flush_xen_tlb_one(vaddr_t va) { asm volatile(STORE_CP32(0, TLBIMVAHIS) : : "r" (va) : "memory"); } diff --git a/xen/include/asm-arm/arm64/page.h b/xen/include/asm-arm/arm64/page.h index 6c36d0210f..31d04ecf76 100644 --- a/xen/include/asm-arm/arm64/page.h +++ b/xen/include/asm-arm/arm64/page.h @@ -45,12 +45,8 @@ static inline void invalidate_icache_local(void) isb(); } -/* - * Flush all hypervisor mappings from the data TLB of the local - * processor. This is not sufficient when changing code mappings or - * for self modifying code. - */ -static inline void flush_xen_data_tlb_local(void) +/* Flush all hypervisor mappings from the TLB of the local processor. */ +static inline void flush_xen_tlb_local(void) { asm volatile ( "dsb sy;" /* Ensure visibility of PTE writes */ @@ -61,14 +57,13 @@ static inline void flush_xen_data_tlb_local(void) } /* Flush TLB of local processor for address va. */ -static inline void __flush_xen_data_tlb_one_local(vaddr_t va) +static inline void __flush_xen_tlb_one_local(vaddr_t va) { asm volatile("tlbi vae2, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); } -/* Flush TLB of all processors in the inner-shareable domain for - * address va. */ -static inline void __flush_xen_data_tlb_one(vaddr_t va) +/* Flush TLB of all processors in the inner-shareable domain for address va. */ +static inline void __flush_xen_tlb_one(vaddr_t va) { asm volatile("tlbi vae2is, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); } diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h index 1a1713ce02..195345e24a 100644 --- a/xen/include/asm-arm/page.h +++ b/xen/include/asm-arm/page.h @@ -234,18 +234,18 @@ static inline int clean_and_invalidate_dcache_va_range } while (0) /* - * Flush a range of VA's hypervisor mappings from the data TLB of the - * local processor. This is not sufficient when changing code mappings - * or for self modifying code. + * Flush a range of VA's hypervisor mappings from the TLB of the local + * processor. */ -static inline void flush_xen_data_tlb_range_va_local(unsigned long va, - unsigned long size) +static inline void flush_xen_tlb_range_va_local(vaddr_t va, + unsigned long size) { - unsigned long end = va + size; + vaddr_t end = va + size; + dsb(sy); /* Ensure preceding are visible */ while ( va < end ) { - __flush_xen_data_tlb_one_local(va); + __flush_xen_tlb_one_local(va); va += PAGE_SIZE; } dsb(sy); /* Ensure completion of the TLB flush */ @@ -253,18 +253,18 @@ static inline void flush_xen_data_tlb_range_va_local(unsigned long va, } /* - * Flush a range of VA's hypervisor mappings from the data TLB of all - * processors in the inner-shareable domain. This is not sufficient - * when changing code mappings or for self modifying code. + * Flush a range of VA's hypervisor mappings from the TLB of all + * processors in the inner-shareable domain. */ -static inline void flush_xen_data_tlb_range_va(unsigned long va, - unsigned long size) +static inline void flush_xen_tlb_range_va(vaddr_t va, + unsigned long size) { - unsigned long end = va + size; + vaddr_t end = va + size; + dsb(sy); /* Ensure preceding are visible */ while ( va < end ) { - __flush_xen_data_tlb_one(va); + __flush_xen_tlb_one(va); va += PAGE_SIZE; } dsb(sy); /* Ensure completion of the TLB flush */ From patchwork Wed May 8 16:16:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 163637 Delivered-To: patch@linaro.org Received: by 2002:a05:6e02:81:0:0:0:0 with SMTP id l1csp2885572ilm; Wed, 8 May 2019 09:17:42 -0700 (PDT) X-Google-Smtp-Source: APXvYqyq4409DEmB8f7AEywXr5J1qsAqYUDvdFq1aRNTvzF6VCn6y86CNZ/YNfUeCRrHZdFDT3mz X-Received: by 2002:a5d:83c2:: with SMTP id u2mr16445199ior.288.1557332262667; Wed, 08 May 2019 09:17:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557332262; cv=none; d=google.com; s=arc-20160816; b=N345VdbllB1N6SO2gM+JyqEs1YED+gjhbU0X9wA+IiyRcGamaYkJM5hOsTPVu2SZRw eQ2Tp0fZtXM17QsYDZv11ovSDA03Ozz9R8F3n/UqJQbaYHDhPwRI3rL0zgBJd1ASgvSb HIjfa2qQcl47KtxtY3u26ZRQdBTmBSUQBtQTKiKRCn/d+G46tdIYYeZIlDmp8DG0K6gr tP8VqOqRksS9pT5Dt4N614USgWGM3P5Q815LR6LowQcTS5wGbLN9s8/vyAp1yJ+CkODt GKqISN96Pj6EiJDr3RkUpCD2PVF9VEpJ1VkYma/+v6Ve97trRfG2svXZLjuBlWBJ+tV1 /NZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=arNG0/87z11H4RXOw6Heq48vNP5FmDxNFvyWtoSD1C0=; b=UoPahpQVIaDaaudnlPnHd3aA+A0aUxMoJDkXgSQSxq492Q1zoWqWuz+D+eXztPnxTO XUVI8gmQNPsJ41W/JviwTrRhKot1uDiQp835XP/K1c8K63DPcUWctdoDKbAvGTjknGte zJp3fiaj5AESfomxgdFfyZyQh37n0aIy9aZjhP3gVDNjYPFK9fcSPM1+9xccSHL/9pcR VYLKpOj2HSI06JxsA2VksEsRv3Z/7ItMgehc1AXDr4/BgOCPtl9c5yoJ0WTY4xLHTz0i yQfhDMI5APFvICGk20IRKvbZ/iQS2nr/1HLpzi1TfKOsVPXYAlhpBWqdXXx68HN17Snn Yfhg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id n186si12373972iof.2.2019.05.08.09.17.42 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 08 May 2019 09:17:42 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hOPEx-0008HZ-Aq; Wed, 08 May 2019 16:16:19 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hOPEv-0008Gf-Fh for xen-devel@lists.xenproject.org; Wed, 08 May 2019 16:16:17 +0000 X-Inumbo-ID: 9af648d6-71ac-11e9-bc36-afa48e3f8fba Received: from foss.arm.com (unknown [217.140.101.70]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id 9af648d6-71ac-11e9-bc36-afa48e3f8fba; Wed, 08 May 2019 16:16:16 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B5A0815AB; Wed, 8 May 2019 09:16:15 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 827B63F238; Wed, 8 May 2019 09:16:14 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Wed, 8 May 2019 17:16:01 +0100 Message-Id: <20190508161603.21964-6-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190508161603.21964-1-julien.grall@arm.com> References: <20190508161603.21964-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH v2 5/7] xen/arm: Gather all TLB flush helpers in tlbflush.h X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" At the moment, TLB helpers are scattered in 2 headers: page.h (for Xen TLB helpers) and tlbflush.h (for guest TLB helpers). This patch is gathering all of them in tlbflush. This will help to uniformize and update the logic of the helpers in follow-up patches. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov Acked-by: Stefano Stabellini --- Changes in v2: - Add Andrii's reviewed-by --- xen/include/asm-arm/arm32/flushtlb.h | 22 +++++++++++++++++++++ xen/include/asm-arm/arm32/page.h | 22 --------------------- xen/include/asm-arm/arm64/flushtlb.h | 23 ++++++++++++++++++++++ xen/include/asm-arm/arm64/page.h | 23 ---------------------- xen/include/asm-arm/flushtlb.h | 38 ++++++++++++++++++++++++++++++++++++ xen/include/asm-arm/page.h | 38 ------------------------------------ 6 files changed, 83 insertions(+), 83 deletions(-) diff --git a/xen/include/asm-arm/arm32/flushtlb.h b/xen/include/asm-arm/arm32/flushtlb.h index 22e100eccf..b629db61cb 100644 --- a/xen/include/asm-arm/arm32/flushtlb.h +++ b/xen/include/asm-arm/arm32/flushtlb.h @@ -45,6 +45,28 @@ static inline void flush_all_guests_tlb(void) isb(); } +/* Flush all hypervisor mappings from the TLB of the local processor. */ +static inline void flush_xen_tlb_local(void) +{ + asm volatile("dsb;" /* Ensure preceding are visible */ + CMD_CP32(TLBIALLH) + "dsb;" /* Ensure completion of the TLB flush */ + "isb;" + : : : "memory"); +} + +/* Flush TLB of local processor for address va. */ +static inline void __flush_xen_tlb_one_local(vaddr_t va) +{ + asm volatile(STORE_CP32(0, TLBIMVAH) : : "r" (va) : "memory"); +} + +/* Flush TLB of all processors in the inner-shareable domain for address va. */ +static inline void __flush_xen_tlb_one(vaddr_t va) +{ + asm volatile(STORE_CP32(0, TLBIMVAHIS) : : "r" (va) : "memory"); +} + #endif /* __ASM_ARM_ARM32_FLUSHTLB_H__ */ /* * Local variables: diff --git a/xen/include/asm-arm/arm32/page.h b/xen/include/asm-arm/arm32/page.h index 0b41b9214b..715a9e4fef 100644 --- a/xen/include/asm-arm/arm32/page.h +++ b/xen/include/asm-arm/arm32/page.h @@ -61,28 +61,6 @@ static inline void invalidate_icache_local(void) isb(); /* Synchronize fetched instruction stream. */ } -/* Flush all hypervisor mappings from the TLB of the local processor. */ -static inline void flush_xen_tlb_local(void) -{ - asm volatile("dsb;" /* Ensure preceding are visible */ - CMD_CP32(TLBIALLH) - "dsb;" /* Ensure completion of the TLB flush */ - "isb;" - : : : "memory"); -} - -/* Flush TLB of local processor for address va. */ -static inline void __flush_xen_tlb_one_local(vaddr_t va) -{ - asm volatile(STORE_CP32(0, TLBIMVAH) : : "r" (va) : "memory"); -} - -/* Flush TLB of all processors in the inner-shareable domain for address va. */ -static inline void __flush_xen_tlb_one(vaddr_t va) -{ - asm volatile(STORE_CP32(0, TLBIMVAHIS) : : "r" (va) : "memory"); -} - /* Ask the MMU to translate a VA for us */ static inline uint64_t __va_to_par(vaddr_t va) { diff --git a/xen/include/asm-arm/arm64/flushtlb.h b/xen/include/asm-arm/arm64/flushtlb.h index adbbd5c522..2fed34b2ec 100644 --- a/xen/include/asm-arm/arm64/flushtlb.h +++ b/xen/include/asm-arm/arm64/flushtlb.h @@ -45,6 +45,29 @@ static inline void flush_all_guests_tlb(void) : : : "memory"); } +/* Flush all hypervisor mappings from the TLB of the local processor. */ +static inline void flush_xen_tlb_local(void) +{ + asm volatile ( + "dsb sy;" /* Ensure visibility of PTE writes */ + "tlbi alle2;" /* Flush hypervisor TLB */ + "dsb sy;" /* Ensure completion of TLB flush */ + "isb;" + : : : "memory"); +} + +/* Flush TLB of local processor for address va. */ +static inline void __flush_xen_tlb_one_local(vaddr_t va) +{ + asm volatile("tlbi vae2, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); +} + +/* Flush TLB of all processors in the inner-shareable domain for address va. */ +static inline void __flush_xen_tlb_one(vaddr_t va) +{ + asm volatile("tlbi vae2is, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); +} + #endif /* __ASM_ARM_ARM64_FLUSHTLB_H__ */ /* * Local variables: diff --git a/xen/include/asm-arm/arm64/page.h b/xen/include/asm-arm/arm64/page.h index 31d04ecf76..0cba266373 100644 --- a/xen/include/asm-arm/arm64/page.h +++ b/xen/include/asm-arm/arm64/page.h @@ -45,29 +45,6 @@ static inline void invalidate_icache_local(void) isb(); } -/* Flush all hypervisor mappings from the TLB of the local processor. */ -static inline void flush_xen_tlb_local(void) -{ - asm volatile ( - "dsb sy;" /* Ensure visibility of PTE writes */ - "tlbi alle2;" /* Flush hypervisor TLB */ - "dsb sy;" /* Ensure completion of TLB flush */ - "isb;" - : : : "memory"); -} - -/* Flush TLB of local processor for address va. */ -static inline void __flush_xen_tlb_one_local(vaddr_t va) -{ - asm volatile("tlbi vae2, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); -} - -/* Flush TLB of all processors in the inner-shareable domain for address va. */ -static inline void __flush_xen_tlb_one(vaddr_t va) -{ - asm volatile("tlbi vae2is, %0;" : : "r" (va>>PAGE_SHIFT) : "memory"); -} - /* Ask the MMU to translate a VA for us */ static inline uint64_t __va_to_par(vaddr_t va) { diff --git a/xen/include/asm-arm/flushtlb.h b/xen/include/asm-arm/flushtlb.h index 83ff9fa8b3..ab1aae5c90 100644 --- a/xen/include/asm-arm/flushtlb.h +++ b/xen/include/asm-arm/flushtlb.h @@ -28,6 +28,44 @@ static inline void page_set_tlbflush_timestamp(struct page_info *page) /* Flush specified CPUs' TLBs */ void flush_tlb_mask(const cpumask_t *mask); +/* + * Flush a range of VA's hypervisor mappings from the TLB of the local + * processor. + */ +static inline void flush_xen_tlb_range_va_local(vaddr_t va, + unsigned long size) +{ + vaddr_t end = va + size; + + dsb(sy); /* Ensure preceding are visible */ + while ( va < end ) + { + __flush_xen_tlb_one_local(va); + va += PAGE_SIZE; + } + dsb(sy); /* Ensure completion of the TLB flush */ + isb(); +} + +/* + * Flush a range of VA's hypervisor mappings from the TLB of all + * processors in the inner-shareable domain. + */ +static inline void flush_xen_tlb_range_va(vaddr_t va, + unsigned long size) +{ + vaddr_t end = va + size; + + dsb(sy); /* Ensure preceding are visible */ + while ( va < end ) + { + __flush_xen_tlb_one(va); + va += PAGE_SIZE; + } + dsb(sy); /* Ensure completion of the TLB flush */ + isb(); +} + #endif /* __ASM_ARM_FLUSHTLB_H__ */ /* * Local variables: diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h index 195345e24a..2bcdb0f1a5 100644 --- a/xen/include/asm-arm/page.h +++ b/xen/include/asm-arm/page.h @@ -233,44 +233,6 @@ static inline int clean_and_invalidate_dcache_va_range : : "r" (_p), "m" (*_p)); \ } while (0) -/* - * Flush a range of VA's hypervisor mappings from the TLB of the local - * processor. - */ -static inline void flush_xen_tlb_range_va_local(vaddr_t va, - unsigned long size) -{ - vaddr_t end = va + size; - - dsb(sy); /* Ensure preceding are visible */ - while ( va < end ) - { - __flush_xen_tlb_one_local(va); - va += PAGE_SIZE; - } - dsb(sy); /* Ensure completion of the TLB flush */ - isb(); -} - -/* - * Flush a range of VA's hypervisor mappings from the TLB of all - * processors in the inner-shareable domain. - */ -static inline void flush_xen_tlb_range_va(vaddr_t va, - unsigned long size) -{ - vaddr_t end = va + size; - - dsb(sy); /* Ensure preceding are visible */ - while ( va < end ) - { - __flush_xen_tlb_one(va); - va += PAGE_SIZE; - } - dsb(sy); /* Ensure completion of the TLB flush */ - isb(); -} - /* Flush the dcache for an entire page. */ void flush_page_to_ram(unsigned long mfn, bool sync_icache); From patchwork Wed May 8 16:16:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 163639 Delivered-To: patch@linaro.org Received: by 2002:a05:6e02:81:0:0:0:0 with SMTP id l1csp2885577ilm; Wed, 8 May 2019 09:17:42 -0700 (PDT) X-Google-Smtp-Source: APXvYqwF9Jx262zPO6FVIiX24sczHFh1Q4TCtJha32yZQmWpHnAWTxWydsnf3wjHsSbZ/GpcFlGg X-Received: by 2002:a05:660c:202:: with SMTP id y2mr4195758itj.0.1557332262767; Wed, 08 May 2019 09:17:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557332262; cv=none; d=google.com; s=arc-20160816; b=L7PUPQREJmfEBbcolPci9M1e1XKVtkxSjq5slkH+1qgJHZenK6ub6AQuLDBQz3zcVb JuzmCc9XLuoPUXwJf+rTRwJ5XXWi97EwE8s1K7Ski04XC8hwLJ2H3cGpyx+2jEHbZ3Z6 43VQAI5brWHoSLrC/wcI0Sx6ijSwasQEg5I3L6v6RCiDvNN0jX+m+7mguSKw/lCTrP5Q Ple9O2MS+taKjlQyn6nKCgCqP59vY1dOJon2XDXFicqbLd88TPIM1zzGnIOWD7uT1Nue Bb8XyS0ui1kX/Ds9kw1CioQrPn0QKYtOcb2Oy+mQWFihues1JIQezLCh26NTgcUb1ZQ1 WJzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=A9GjsSVdYPduU6iHQ7mOax0Lvn/MWxzSi2ZS6OjlU38=; b=ACECtUVotuZP4KX0mVjj0ZFVImIk7Z2YlcNrPZeXkrnTciITnu5oHun6rYgnD7u7Ih MNnVrWYZAto+p6XUIFDNT9xMJvFkwbbk4LPdNgbkWi1OysJh/ZFRsEK7mJFTBEG1iE4F nIBo1Oj4QAWeBqXESYpR60W3HB8k3pkxciAgJmc9pS9djUEsAly2jZx991QENiJJk9W0 7YCWUyuKmbsxDAnWnz5Jm4SGY0ncfauGGRi61iTr1LVxJ2grpLVv0aezu77tHkcrSs3A Kxg/Q88K5AsZqdoyWbnFUGTlNcDVsQbb3ZSZelHyxgYmQuMIYQF/ivObEJlb7Rv1PHYY 7Tyw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id 10si2210373ity.47.2019.05.08.09.17.42 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 08 May 2019 09:17:42 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hOPEx-0008IE-TG; Wed, 08 May 2019 16:16:19 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hOPEw-0008HL-PW for xen-devel@lists.xenproject.org; Wed, 08 May 2019 16:16:18 +0000 X-Inumbo-ID: 9bd14cba-71ac-11e9-a7ef-0ffc496098b6 Received: from foss.arm.com (unknown [217.140.101.70]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id 9bd14cba-71ac-11e9-a7ef-0ffc496098b6; Wed, 08 May 2019 16:16:17 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3226A80D; Wed, 8 May 2019 09:16:17 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F2F2C3F238; Wed, 8 May 2019 09:16:15 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Wed, 8 May 2019 17:16:02 +0100 Message-Id: <20190508161603.21964-7-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190508161603.21964-1-julien.grall@arm.com> References: <20190508161603.21964-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH v2 6/7] xen/arm: tlbflush: Rework TLB helpers X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" All the TLBs helpers invalidate all the TLB entries are using the same pattern: DSB SY TLBI ... DSB SY ISB This pattern is following pattern recommended by the Arm Arm to ensure visibility of updates to translation tables (see K11.5.2 in ARM DDI 0487D.b). We have been a bit too eager in Xen and use system-wide DSBs when this can be limited to the inner-shareable domain. Furthermore, the first DSB can be restrict further to only store in the inner-shareable domain. This is because the DSB is here to ensure visibility of the update to translation table walks. Lastly, there are a lack of documentation in most of the TLBs helper. Rather than trying to update the helpers one by one, this patch introduce a per-arch macro to generate the TLB helpers. This will be easier to update the TLBs helper in the future and the documentation. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov Reviewed-by: Stefano Stabellini --- Changes in v2: - Update the reference to the Arm Arm to the latest spec - Add Andrii's reviewed-by --- xen/include/asm-arm/arm32/flushtlb.h | 73 ++++++++++++++-------------------- xen/include/asm-arm/arm64/flushtlb.h | 76 +++++++++++++++--------------------- 2 files changed, 60 insertions(+), 89 deletions(-) diff --git a/xen/include/asm-arm/arm32/flushtlb.h b/xen/include/asm-arm/arm32/flushtlb.h index b629db61cb..9085e65011 100644 --- a/xen/include/asm-arm/arm32/flushtlb.h +++ b/xen/include/asm-arm/arm32/flushtlb.h @@ -1,59 +1,44 @@ #ifndef __ASM_ARM_ARM32_FLUSHTLB_H__ #define __ASM_ARM_ARM32_FLUSHTLB_H__ -/* Flush local TLBs, current VMID only */ -static inline void flush_guest_tlb_local(void) -{ - dsb(sy); - - WRITE_CP32((uint32_t) 0, TLBIALL); - - dsb(sy); - isb(); +/* + * Every invalidation operation use the following patterns: + * + * DSB ISHST // Ensure prior page-tables updates have completed + * TLBI... // Invalidate the TLB + * DSB ISH // Ensure the TLB invalidation has completed + * ISB // See explanation below + * + * For Xen page-tables the ISB will discard any instructions fetched + * from the old mappings. + * + * For the Stage-2 page-tables the ISB ensures the completion of the DSB + * (and therefore the TLB invalidation) before continuing. So we know + * the TLBs cannot contain an entry for a mapping we may have removed. + */ +#define TLB_HELPER(name, tlbop) \ +static inline void name(void) \ +{ \ + dsb(ishst); \ + WRITE_CP32(0, tlbop); \ + dsb(ish); \ + isb(); \ } -/* Flush inner shareable TLBs, current VMID only */ -static inline void flush_guest_tlb(void) -{ - dsb(sy); - - WRITE_CP32((uint32_t) 0, TLBIALLIS); +/* Flush local TLBs, current VMID only */ +TLB_HELPER(flush_guest_tlb_local, TLBIALL); - dsb(sy); - isb(); -} +/* Flush inner shareable TLBs, current VMID only */ +TLB_HELPER(flush_guest_tlb, TLBIALLIS); /* Flush local TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_all_guests_tlb_local(void) -{ - dsb(sy); - - WRITE_CP32((uint32_t) 0, TLBIALLNSNH); - - dsb(sy); - isb(); -} +TLB_HELPER(flush_all_guests_tlb_local, TLBIALLNSNH); /* Flush innershareable TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_all_guests_tlb(void) -{ - dsb(sy); - - WRITE_CP32((uint32_t) 0, TLBIALLNSNHIS); - - dsb(sy); - isb(); -} +TLB_HELPER(flush_all_guests_tlb, TLBIALLNSNHIS); /* Flush all hypervisor mappings from the TLB of the local processor. */ -static inline void flush_xen_tlb_local(void) -{ - asm volatile("dsb;" /* Ensure preceding are visible */ - CMD_CP32(TLBIALLH) - "dsb;" /* Ensure completion of the TLB flush */ - "isb;" - : : : "memory"); -} +TLB_HELPER(flush_xen_tlb_local, TLBIALLH); /* Flush TLB of local processor for address va. */ static inline void __flush_xen_tlb_one_local(vaddr_t va) diff --git a/xen/include/asm-arm/arm64/flushtlb.h b/xen/include/asm-arm/arm64/flushtlb.h index 2fed34b2ec..ceec59542e 100644 --- a/xen/include/asm-arm/arm64/flushtlb.h +++ b/xen/include/asm-arm/arm64/flushtlb.h @@ -1,60 +1,46 @@ #ifndef __ASM_ARM_ARM64_FLUSHTLB_H__ #define __ASM_ARM_ARM64_FLUSHTLB_H__ -/* Flush local TLBs, current VMID only */ -static inline void flush_guest_tlb_local(void) -{ - asm volatile( - "dsb sy;" - "tlbi vmalls12e1;" - "dsb sy;" - "isb;" - : : : "memory"); +/* + * Every invalidation operation use the following patterns: + * + * DSB ISHST // Ensure prior page-tables updates have completed + * TLBI... // Invalidate the TLB + * DSB ISH // Ensure the TLB invalidation has completed + * ISB // See explanation below + * + * For Xen page-tables the ISB will discard any instructions fetched + * from the old mappings. + * + * For the Stage-2 page-tables the ISB ensures the completion of the DSB + * (and therefore the TLB invalidation) before continuing. So we know + * the TLBs cannot contain an entry for a mapping we may have removed. + */ +#define TLB_HELPER(name, tlbop) \ +static inline void name(void) \ +{ \ + asm volatile( \ + "dsb ishst;" \ + "tlbi " # tlbop ";" \ + "dsb ish;" \ + "isb;" \ + : : : "memory"); \ } +/* Flush local TLBs, current VMID only. */ +TLB_HELPER(flush_guest_tlb_local, vmalls12e1); + /* Flush innershareable TLBs, current VMID only */ -static inline void flush_guest_tlb(void) -{ - asm volatile( - "dsb sy;" - "tlbi vmalls12e1is;" - "dsb sy;" - "isb;" - : : : "memory"); -} +TLB_HELPER(flush_guest_tlb, vmalls12e1is); /* Flush local TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_all_guests_tlb_local(void) -{ - asm volatile( - "dsb sy;" - "tlbi alle1;" - "dsb sy;" - "isb;" - : : : "memory"); -} +TLB_HELPER(flush_all_guests_tlb_local, alle1); /* Flush innershareable TLBs, all VMIDs, non-hypervisor mode */ -static inline void flush_all_guests_tlb(void) -{ - asm volatile( - "dsb sy;" - "tlbi alle1is;" - "dsb sy;" - "isb;" - : : : "memory"); -} +TLB_HELPER(flush_all_guests_tlb, alle1is); /* Flush all hypervisor mappings from the TLB of the local processor. */ -static inline void flush_xen_tlb_local(void) -{ - asm volatile ( - "dsb sy;" /* Ensure visibility of PTE writes */ - "tlbi alle2;" /* Flush hypervisor TLB */ - "dsb sy;" /* Ensure completion of TLB flush */ - "isb;" - : : : "memory"); -} +TLB_HELPER(flush_xen_tlb_local, alle2); /* Flush TLB of local processor for address va. */ static inline void __flush_xen_tlb_one_local(vaddr_t va) From patchwork Wed May 8 16:16:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 163635 Delivered-To: patch@linaro.org Received: by 2002:a05:6e02:81:0:0:0:0 with SMTP id l1csp2885552ilm; Wed, 8 May 2019 09:17:41 -0700 (PDT) X-Google-Smtp-Source: APXvYqyfef49YgmSCDt0C1aPzo/A0Phj61ht0X5LpCWvSI6Nc4Ft+UQdZzhxmRSO+8o1O76N6aYV X-Received: by 2002:a24:2fcb:: with SMTP id j194mr4438557itj.14.1557332261647; Wed, 08 May 2019 09:17:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557332261; cv=none; d=google.com; s=arc-20160816; b=i3BH870/MtZaOSHlMTGh8Mkjzj0T6DI14PCsmD2T23VNz4ZDGFvkkDxAUd0xEULVPS V7gs0a0JX+YQCxSzvjfyPuD63xioH2zCO9aF7ezYvotI2JCIwya+m0Di0qawuevh5nii HOczakYMMdUdDdQhWSVhm8bBzlKNFosEvuV+ZaVNDwm0G1DxFEL8NEWA+3h4/7xuSFs2 euiDwFx6T3B5ExW7TNf6QeAJkI+m6G3xklE4e5YVljHlLcaCdqXQqpswUqVNPjm3I8tM PrjGSaZLa971Mm5RnEEnGhTZPkynB4b+PG5vFsM2krmAP7NWwCcbt9jiqjZeudOfjQgy O5OQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=h4qucULm9utmZfFmFq+FElQErh6gNmI8fsaX/GtbH94=; b=hWC4rEmUrC3xV+6bE4ADjfu1CoAPNtIamGiykRk4f3ZsPtWJ6V0VWZFYTPMTTVziJe SjrGV00JYgrtjwbNHdotCsCnZaac/XHTE4IXzE3nEusgqVwMDtdhwurrVzfvsNbRZqYm TG2i0ejoMGzSXMvcoAscGECdDBc7JHRpeGMoSq2e3ycheOT190Tz8Ufu6p+HQmJiw5nx H5ZxCCOr9HdNJHhEHljEpNaevS6xkKSYherl1X3+jjP/IX0xd3DQK5NoYBegmsKJqA1k NuKBxoZ8VOhhKrPaddGOZ0Q8tLdTSHCdLdfOP5NH2MBgPuxeaM1uKeyLeV+FbpyYK+iF w9eQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id m4si2424775ioq.50.2019.05.08.09.17.41 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 08 May 2019 09:17:41 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hOPEz-0008JD-7F; Wed, 08 May 2019 16:16:21 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hOPEy-0008IK-0W for xen-devel@lists.xenproject.org; Wed, 08 May 2019 16:16:20 +0000 X-Inumbo-ID: 9cb39e44-71ac-11e9-b79a-5f8566900e7a Received: from foss.arm.com (unknown [217.140.101.70]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id 9cb39e44-71ac-11e9-b79a-5f8566900e7a; Wed, 08 May 2019 16:16:19 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A319E374; Wed, 8 May 2019 09:16:18 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6FE603F238; Wed, 8 May 2019 09:16:17 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Wed, 8 May 2019 17:16:03 +0100 Message-Id: <20190508161603.21964-8-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190508161603.21964-1-julien.grall@arm.com> References: <20190508161603.21964-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH v2 7/7] xen/arm: mm: Flush the TLBs even if a mapping failed in create_xen_entries X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr_Tyshchenko@epam.com, Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" At the moment, create_xen_entries will only flush the TLBs if the full range has successfully been updated. This may lead to leave unwanted entries in the TLBs if we fail to update some entries. Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov Reviewed-by: Stefano Stabellini --- Changes in v2: - Add Andrii's reviewed-by --- xen/arch/arm/mm.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index 8ee828d445..9d584e4cbf 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -984,7 +984,7 @@ static int create_xen_entries(enum xenmap_operation op, unsigned long nr_mfns, unsigned int flags) { - int rc; + int rc = 0; unsigned long addr = virt, addr_end = addr + nr_mfns * PAGE_SIZE; lpae_t pte, *entry; lpae_t *third = NULL; @@ -1013,7 +1013,8 @@ static int create_xen_entries(enum xenmap_operation op, { printk("%s: trying to replace an existing mapping addr=%lx mfn=%"PRI_mfn"\n", __func__, addr, mfn_x(mfn)); - return -EINVAL; + rc = -EINVAL; + goto out; } if ( op == RESERVE ) break; @@ -1030,7 +1031,8 @@ static int create_xen_entries(enum xenmap_operation op, { printk("%s: trying to %s a non-existing mapping addr=%lx\n", __func__, op == REMOVE ? "remove" : "modify", addr); - return -EINVAL; + rc = -EINVAL; + goto out; } if ( op == REMOVE ) pte.bits = 0; @@ -1043,7 +1045,8 @@ static int create_xen_entries(enum xenmap_operation op, { printk("%s: Incorrect combination for addr=%lx\n", __func__, addr); - return -EINVAL; + rc = -EINVAL; + goto out; } } write_pte(entry, pte); @@ -1052,11 +1055,14 @@ static int create_xen_entries(enum xenmap_operation op, BUG(); } } +out: + /* + * Flush the TLBs even in case of failure because we may have + * partially modified the PT. This will prevent any unexpected + * behavior afterwards. + */ flush_xen_tlb_range_va(virt, PAGE_SIZE * nr_mfns); - rc = 0; - -out: return rc; }