From patchwork Mon Jan 9 06:24:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jintack Lim X-Patchwork-Id: 90344 Delivered-To: patch@linaro.org Received: by 10.140.20.101 with SMTP id 92csp707374qgi; Sun, 8 Jan 2017 22:29:13 -0800 (PST) X-Received: by 10.84.238.131 with SMTP id v3mr16975977plk.143.1483943353032; Sun, 08 Jan 2017 22:29:13 -0800 (PST) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 3si87779920pln.317.2017.01.08.22.29.12; Sun, 08 Jan 2017 22:29:13 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760786AbdAIG2r (ORCPT + 25 others); Mon, 9 Jan 2017 01:28:47 -0500 Received: from outprodmail01.cc.columbia.edu ([128.59.72.39]:38671 "EHLO outprodmail01.cc.columbia.edu" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S939849AbdAIG0m (ORCPT ); Mon, 9 Jan 2017 01:26:42 -0500 Received: from hazelnut (hazelnut.cc.columbia.edu [128.59.213.250]) by outprodmail01.cc.columbia.edu (8.14.4/8.14.4) with ESMTP id v096PXwK017320 for ; Mon, 9 Jan 2017 01:26:34 -0500 Received: from hazelnut (localhost.localdomain [127.0.0.1]) by hazelnut (Postfix) with ESMTP id 2C2AC83 for ; Mon, 9 Jan 2017 01:26:34 -0500 (EST) Received: from sendprodmail02.cc.columbia.edu (sendprodmail02.cc.columbia.edu [128.59.72.14]) by hazelnut (Postfix) with ESMTP id 128AF81 for ; Mon, 9 Jan 2017 01:26:34 -0500 (EST) Received: from mail-qk0-f198.google.com (mail-qk0-f198.google.com [209.85.220.198]) by sendprodmail02.cc.columbia.edu (8.14.4/8.14.4) with ESMTP id v096QXTM043074 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 9 Jan 2017 01:26:33 -0500 Received: by mail-qk0-f198.google.com with SMTP id 11so31135971qkl.4 for ; Sun, 08 Jan 2017 22:26:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=pBoYn7AGcV6yIgQVRGmhf7m8PNt401Ix7tyOglmAo2o=; b=Twh/faVFPcYaZf93DkwQ52uZyCxvfN9w0t9T9475hrf14/V5XbRhL6vDU7BxfwOxNg h9oEJt0HVjPcER2M5H/ot0X/h6AukgsRAE/0QHpPGYkIy5/hOSOrBUcf73YGs1DoMUwa 50Lrvqz6q6b3+ItGi5d0Vw13i5on3juDMChq3p87n76rQItdhDhhi7oQqwCmSiZSDr2h 2/WDC7oo6G+koxwCqqFFy+H4ggn2RqcjMxhKYHZG7GhaUS9myGx+Fhun+whob2DJqoab zKBkY7WNsfBsj42sRkLZNG2Xsjef2NQKcnnsNR3KVmaPK82j+pvPbGgyTAvNzvZwjrNG zK6w== X-Gm-Message-State: AIkVDXKCyXBRH0A0P/dIUDfYj/KjKFlKyP01GxeXaik/2eLXzpw6ptxeR6sjdzrbMiqPD9GSOxSRX3xtUoCT8GHrLGMTqtP0OTk4KyeYEpY2C9cXNFmreDVdZzvhVuQCRypCLaToncEGWeRcrX3ENu9sH7I= X-Received: by 10.55.122.197 with SMTP id v188mr79430397qkc.120.1483943193323; Sun, 08 Jan 2017 22:26:33 -0800 (PST) X-Received: by 10.55.122.197 with SMTP id v188mr79430375qkc.120.1483943193148; Sun, 08 Jan 2017 22:26:33 -0800 (PST) Received: from jintack.cs.columbia.edu ([2001:18d8:ffff:16:21a:4aff:feaa:f900]) by smtp.gmail.com with ESMTPSA id h3sm8623257qtc.6.2017.01.08.22.26.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 08 Jan 2017 22:26:32 -0800 (PST) From: Jintack Lim To: christoffer.dall@linaro.org, marc.zyngier@arm.com, pbonzini@redhat.com, rkrcmar@redhat.com, linux@armlinux.org.uk, catalin.marinas@arm.com, will.deacon@arm.com, vladimir.murzin@arm.com, suzuki.poulose@arm.com, mark.rutland@arm.com, james.morse@arm.com, lorenzo.pieralisi@arm.com, kevin.brodsky@arm.com, wcohen@redhat.com, shankerd@codeaurora.org, geoff@infradead.org, andre.przywara@arm.com, eric.auger@redhat.com, anna-maria@linutronix.de, shihwei@cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: jintack@cs.columbia.edu Subject: [RFC 49/55] KVM: arm64: Fixes to toggle_cache for nesting Date: Mon, 9 Jan 2017 01:24:45 -0500 Message-Id: <1483943091-1364-50-git-send-email-jintack@cs.columbia.edu> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1483943091-1364-1-git-send-email-jintack@cs.columbia.edu> References: <1483943091-1364-1-git-send-email-jintack@cs.columbia.edu> X-No-Spam-Score: Local X-Scanned-By: MIMEDefang 2.78 on 128.59.72.14 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Christoffer Dall So far we were flushing almost the entire universe whenever a VM would load/unload the SCTLR_EL1 and the two versions of that register had different MMU enabled settings. This turned out to be so slow that it prevented forward progress for a nested VM, because a scheduler timer tick interrupt would always be pending when we reached the nested VM. To avoid this problem, we consider the SCTLR_EL2 when evaluating if caches are on or off when entering virtual EL2 (because this is the value that we end up shadowing onto the hardware EL1 register). We also reduce the scope of the flush operation to only flush shadow stage 2 page table state of the particular VCPU toggling the caches instead of the shadow stage 2 state of all possible VCPUs. Signed-off-by: Christoffer Dall Signed-off-by: Jintack Lim --- arch/arm/kvm/mmu.c | 31 ++++++++++++++++++++++++++++++- arch/arm64/include/asm/kvm_mmu.h | 7 ++++++- 2 files changed, 36 insertions(+), 2 deletions(-) -- 1.9.1 diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c index 68fc8e8..344bc01 100644 --- a/arch/arm/kvm/mmu.c +++ b/arch/arm/kvm/mmu.c @@ -422,6 +422,35 @@ static void stage2_flush_vm(struct kvm *kvm) srcu_read_unlock(&kvm->srcu, idx); } +/** + * Same as above but only flushed shadow state for specific vcpu + */ +static void stage2_flush_vcpu(struct kvm_vcpu *vcpu) +{ + struct kvm *kvm = vcpu->kvm; + struct kvm_memslots *slots; + struct kvm_memory_slot *memslot; + int idx; + struct kvm_nested_s2_mmu __maybe_unused *nested_mmu; + + idx = srcu_read_lock(&kvm->srcu); + spin_lock(&kvm->mmu_lock); + + slots = kvm_memslots(kvm); + kvm_for_each_memslot(memslot, slots) + stage2_flush_memslot(&kvm->arch.mmu, memslot); + +#ifdef CONFIG_KVM_ARM_NESTED_HYP + list_for_each_entry_rcu(nested_mmu, &vcpu->kvm->arch.nested_mmu_list, + list) { + kvm_stage2_flush_range(&nested_mmu->mmu, 0, KVM_PHYS_SIZE); + } +#endif + + spin_unlock(&kvm->mmu_lock); + srcu_read_unlock(&kvm->srcu, idx); +} + static void clear_hyp_pgd_entry(pgd_t *pgd) { pud_t *pud_table __maybe_unused = pud_offset(pgd, 0UL); @@ -2074,7 +2103,7 @@ void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool was_enabled) * Clean + invalidate does the trick always. */ if (now_enabled != was_enabled) - stage2_flush_vm(vcpu->kvm); + stage2_flush_vcpu(vcpu); /* Caches are now on, stop trapping VM ops (until a S/W op) */ if (now_enabled) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 2086296..7754f3e 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -241,7 +241,12 @@ static inline bool kvm_page_empty(void *ptr) static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu) { - return (vcpu_sys_reg(vcpu, SCTLR_EL1) & 0b101) == 0b101; + u32 mode = vcpu->arch.ctxt.gp_regs.regs.pstate & PSR_MODE_MASK; + + if (mode != PSR_MODE_EL2h && mode != PSR_MODE_EL2t) + return (vcpu_sys_reg(vcpu, SCTLR_EL1) & 0b101) == 0b101; + else + return (vcpu_el2_reg(vcpu, SCTLR_EL2) & 0b101) == 0b101; } static inline void __coherent_cache_guest_page(struct kvm_vcpu *vcpu,