From patchwork Mon Oct 8 18:33:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 148420 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp3981904lji; Mon, 8 Oct 2018 11:36:07 -0700 (PDT) X-Google-Smtp-Source: ACcGV63tCojUjljaodd1vhH0jCFW9bzUERTMfRC9CMxrJVnPPJ9PgPyIbuKwYiuztoUOpyYRliHr X-Received: by 2002:a5e:c90c:: with SMTP id z12-v6mr16687209iol.72.1539023767237; Mon, 08 Oct 2018 11:36:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539023767; cv=none; d=google.com; s=arc-20160816; b=kOdssT/EaYZYq9aFwUvUmghvuei83k3guMvW5XjFLxPGqCSrMmHn8lVAN3T0eSXBp7 0/5l9C10Y3cPoMltoA30BalmAT4pLPitw4qE9HIgw0Y84NOLX8IsLuaAFIcQd0OW4nPx FTgmWknNx2TfoqE6S0JbgUWH8xVDaB5MAw6ARjYUxGtYfeDgxSavE3Qk1qnfp3prLM9Q ICZs5OS73N4Ccx831Z731Y6/I9tNza1sIPLVSfQgVrBKGngQOkNpddXjiaInwfwkcX2n 1AjQC8Pj3QjPRuxkGCG7Kh+wX//9p/eaOBC/PUD4dQdKkPVvzVaAdE/WR2gRhioTXhG+ eEPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=nxwPz/qt3PuJRfWnse2hjNSmv3bI4Yvq9wfshTMLlBc=; b=u2Tp+hiqQjn3ZmcAL9HqEGtIPXzzux3/009oQVAsamqVa2arsKlLvt6WMeNzAsZhCQ SAg0eWqK6cFvu55hX8roFytpefSPyG6nxiP3JetYNxW3zkTEBfRup64SUP7IK9I1XdXq hQnEHVjTZeW6PdYDicqBf7cQ7l3Uj46RmzDYfbJSiGxeuPQUN6bzh8L08R8jI7RTUG9P T4WkRgCtwzYuQd4MWxhoP2av2/9RRRxczK5oVBEXHQmTqRIR0sr+bKF8jMQAK1KoQ+zk iMK07+zRzyrSymWFOT59mIIhQ/pWT/y/vnCiNBzoJy3e7ElHhb3tSPGKyUyZPtfLH4GI jO6w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id o53-v6si8350654jaj.66.2018.10.08.11.36.06 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 08 Oct 2018 11:36:07 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1g9aMH-0005so-KK; Mon, 08 Oct 2018 18:34:21 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1g9aMF-0005qA-W6 for xen-devel@lists.xen.org; Mon, 08 Oct 2018 18:34:20 +0000 X-Inumbo-ID: 69ef97ec-cb28-11e8-a8a5-bc764e045a96 Received: from foss.arm.com (unknown [217.140.101.70]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTP id 69ef97ec-cb28-11e8-a8a5-bc764e045a96; Mon, 08 Oct 2018 20:31:47 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7DC9615B2; Mon, 8 Oct 2018 11:34:18 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 93D953F5B3; Mon, 8 Oct 2018 11:34:17 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xen.org Date: Mon, 8 Oct 2018 19:33:51 +0100 Message-Id: <20181008183352.16291-16-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181008183352.16291-1-julien.grall@arm.com> References: <20181008183352.16291-1-julien.grall@arm.com> Subject: [Xen-devel] [RFC 15/16] xen/arm: Implement Set/Way operations X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Julien Grall , sstabellini@kernel.org, andre.przywara@linaro.org MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Set/Way operations are used to perform maintenance on a given cache. At the moment, Set/Way operations are not trapped and therefore a guest OS will directly act on the local cache. However, a vCPU may migrate to another pCPU in the middle of the processor. This will result to have cache with stall data (Set/Way are not propagated) potentially causing crash. This may be the cause of heisenbug noticed in Osstest [1]. Furthermore, Set/Way operations are not available on system cache. This means that OS, such as Linux 32-bit, relying on those operations to fully clean the cache before disabling MMU may break because data may sits in system caches and not in RAM. For more details about Set/Way, see the talk "The Art of Virtualizing Cache Maintenance" given at Xen Summit 2018 [2]. In the context of Xen, we need to trap Set/Way operations and emulate them. From the Arm Arm (B1.14.4 in DDI 046C.c), Set/Way operations are difficult to virtualized. So we can assume that a guest OS using them will suffer the consequence (i.e slowness) until developer removes all the usage of Set/Way. As the software is not allowed to infer the Set/Way to Physical Address mapping, Xen will need to go through the guest P2M and clean & invalidate all the entries mapped. Because Set/Way happen in batch (a loop on all Set/Way of a cache), Xen would need to go through the P2M for every instructions. This is quite expensive and would severely impact the guest OS. The implementation is re-using the KVM policy to limit the number of flush: - If we trap a Set/Way operations, we enable VM trapping (i.e HVC_EL2.TVM) to detect cache being turned on/off, and do a full clean. - We clean the caches when turning on and off - Once the caches are enabled, we stop trapping VM instructions [1] https://lists.xenproject.org/archives/html/xen-devel/2017-09/msg03191.html [2] https://fr.slideshare.net/xen_com_mgr/virtualizing-cache Signed-off-by: Julien Grall --- xen/arch/arm/arm64/vsysreg.c | 27 +++++++++++++++++- xen/arch/arm/p2m.c | 68 ++++++++++++++++++++++++++++++++++++++++++++ xen/arch/arm/traps.c | 2 +- xen/arch/arm/vcpreg.c | 23 +++++++++++++++ xen/include/asm-arm/p2m.h | 16 +++++++++++ 5 files changed, 134 insertions(+), 2 deletions(-) diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c index 1517879697..43c6c3e30d 100644 --- a/xen/arch/arm/arm64/vsysreg.c +++ b/xen/arch/arm/arm64/vsysreg.c @@ -40,7 +40,20 @@ static bool vreg_emulate_##reg(struct cpu_user_regs *regs, \ } /* Defining helpers for emulating sysreg registers. */ -TVM_REG(SCTLR_EL1) +static bool vreg_emulate_SCTLR_EL1(struct cpu_user_regs *regs, uint64_t *r, + bool read) +{ + struct vcpu *v = current; + bool cache_enabled = vcpu_has_cache_enabled(v); + + GUEST_BUG_ON(read); + WRITE_SYSREG(*r, SCTLR_EL1); + + p2m_toggle_cache(v, cache_enabled); + + return true; +} + TVM_REG(TTBR0_EL1) TVM_REG(TTBR1_EL1) TVM_REG(TCR_EL1) @@ -84,6 +97,18 @@ void do_sysreg(struct cpu_user_regs *regs, break; /* + * HCR_EL2.TSW + * + * ARMv8 (DDI 0487B.b): Table D1-42 + */ + case HSR_SYSREG_DCISW: + case HSR_SYSREG_DCCSW: + case HSR_SYSREG_DCCISW: + if ( hsr.sysreg.read ) + p2m_set_way_flush(current); + break; + + /* * HCR_EL2.TVM * * ARMv8 (DDI 0487B.b): Table D1-37 diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c index df6b48d73b..a3d4c563b1 100644 --- a/xen/arch/arm/p2m.c +++ b/xen/arch/arm/p2m.c @@ -1564,6 +1564,74 @@ int p2m_cache_flush_range(struct domain *d, gfn_t start, gfn_t end) return 0; } +static void p2m_flush_vm(struct vcpu *v) +{ + struct p2m_domain *p2m = p2m_get_hostp2m(v->domain); + + /* XXX: Handle preemption */ + p2m_cache_flush_range(v->domain, p2m->lowest_mapped_gfn, + p2m->max_mapped_gfn); +} + +/* + * See note at ARMv7 ARM B1.14.4 (DDI 0406C.c) (TL;DR: S/W ops are not + * easily virtualized). + * + * Main problems: + * - S/W ops are local to a CPU (not broadcast) + * - We have line migration behind our back (speculation) + * - System caches don't support S/W at all (damn!) + * + * In the face of the above, the best we can do is to try and convert + * S/W ops to VA ops. Because the guest is not allowed to infer the S/W + * to PA mapping, it can only use S/W to nuke the whole cache, which is + * rather a good thing for us. + * + * Also, it is only used when turning caches on/off ("The expected + * usage of the cache maintenance instructions that operate by set/way + * is associated with the powerdown and powerup of caches, if this is + * required by the implementation."). + * + * We use the following policy: + * - If we trap a S/W operation, we enabled VM trapping to detect + * caches being turned on/off, and do a full clean. + * + * - We flush the caches on both caches being turned on and off. + * + * - Once the caches are enabled, we stop trapping VM ops. + */ +void p2m_set_way_flush(struct vcpu *v) +{ + /* This function can only work with the current vCPU. */ + ASSERT(v == current); + + if ( !(v->arch.hcr_el2 & HCR_TVM) ) + { + p2m_flush_vm(v); + vcpu_hcr_set_flags(v, HCR_TVM); + } +} + +void p2m_toggle_cache(struct vcpu *v, bool was_enabled) +{ + bool now_enabled = vcpu_has_cache_enabled(v); + + /* This function can only work with the current vCPU. */ + ASSERT(v == current); + + /* + * If switching the MMU+caches on, need to invalidate the caches. + * If switching it off, need to clean the caches. + * Clean + invalidate does the trick always. + */ + if ( was_enabled != now_enabled ) + p2m_flush_vm(v); + + /* Caches are now on, stop trapping VM ops (until a S/W op) */ + if ( now_enabled ) + vcpu_hcr_clear_flags(v, HCR_TVM); +} + mfn_t gfn_to_mfn(struct domain *d, gfn_t gfn) { return p2m_lookup(d, gfn, NULL); diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c index 169b57cb6b..cdc10eee5a 100644 --- a/xen/arch/arm/traps.c +++ b/xen/arch/arm/traps.c @@ -98,7 +98,7 @@ register_t get_default_hcr_flags(void) { return (HCR_PTW|HCR_BSU_INNER|HCR_AMO|HCR_IMO|HCR_FMO|HCR_VM| (vwfi != NATIVE ? (HCR_TWI|HCR_TWE) : 0) | - HCR_TSC|HCR_TAC|HCR_SWIO|HCR_TIDCP|HCR_FB); + HCR_TSC|HCR_TAC|HCR_SWIO|HCR_TIDCP|HCR_FB|HCR_TSW); } static enum { diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c index 49529b97cd..dc46d9d0d7 100644 --- a/xen/arch/arm/vcpreg.c +++ b/xen/arch/arm/vcpreg.c @@ -45,9 +45,14 @@ #define TVM_REG(sz, func, reg...) \ static bool func(struct cpu_user_regs *regs, uint##sz##_t *r, bool read) \ { \ + struct vcpu *v = current; \ + bool cache_enabled = vcpu_has_cache_enabled(v); \ + \ GUEST_BUG_ON(read); \ WRITE_SYSREG##sz(*r, reg); \ \ + p2m_toggle_cache(v, cache_enabled); \ + \ return true; \ } @@ -65,6 +70,8 @@ static bool func(struct cpu_user_regs *regs, uint##sz##_t *r, bool read) \ static bool vreg_emulate_##xreg(struct cpu_user_regs *regs, uint32_t *r, \ bool read, bool hi) \ { \ + struct vcpu *v = current; \ + bool cache_enabled = vcpu_has_cache_enabled(v); \ register_t reg = READ_SYSREG(xreg); \ \ GUEST_BUG_ON(read); \ @@ -80,6 +87,8 @@ static bool vreg_emulate_##xreg(struct cpu_user_regs *regs, uint32_t *r, \ } \ WRITE_SYSREG(reg, xreg); \ \ + p2m_toggle_cache(v, cache_enabled); \ + \ return true; \ } \ \ @@ -98,6 +107,7 @@ static bool vreg_emulate_##hireg(struct cpu_user_regs *regs, uint32_t *r, \ /* Defining helpers for emulating co-processor registers. */ TVM_REG32(SCTLR, SCTLR_EL1) + /* * AArch32 provides two way to access TTBR* depending on the access * size, whilst AArch64 provides one way. @@ -180,6 +190,19 @@ void do_cp15_32(struct cpu_user_regs *regs, const union hsr hsr) break; /* + * HCR_EL2.TSW + * + * ARMv7 (DDI 0406C.b): B1.14.6 + * ARMv8 (DDI 0487B.b): Table D1-42 + */ + case HSR_CPREG32(DCISW): + case HSR_CPREG32(DCCSW): + case HSR_CPREG32(DCCISW): + if ( !cp32.read ) + p2m_set_way_flush(current); + break; + + /* * HCR_EL2.TVM * * ARMv8 (DDI 0487B.b): Table D1-37 diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h index 92213dd1ab..c470f062db 100644 --- a/xen/include/asm-arm/p2m.h +++ b/xen/include/asm-arm/p2m.h @@ -231,6 +231,10 @@ int p2m_set_entry(struct p2m_domain *p2m, */ int p2m_cache_flush_range(struct domain *d, gfn_t start, gfn_t end); +void p2m_set_way_flush(struct vcpu *v); + +void p2m_toggle_cache(struct vcpu *v, bool was_enabled); + /* * Map a region in the guest p2m with a specific p2m type. * The memory attributes will be derived from the p2m type. @@ -358,6 +362,18 @@ static inline int set_foreign_p2m_entry(struct domain *d, unsigned long gfn, return -EOPNOTSUPP; } +/* + * A vCPU has cache enabled only when the MMU is enabled and data cache + * is enabled. + */ +static inline bool vcpu_has_cache_enabled(struct vcpu *v) +{ + /* Only works with the current vCPU */ + ASSERT(current == v); + + return (READ_SYSREG32(SCTLR_EL1) & (SCTLR_C|SCTLR_M)) == (SCTLR_C|SCTLR_M); +} + #endif /* _XEN_P2M_H */ /*