From patchwork Tue Mar 16 18:33:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suzuki K Poulose X-Patchwork-Id: 401719 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp4553566jai; Tue, 16 Mar 2021 11:35:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxU2n5WZiurmw7H9u48iNQovpPyJECFDVfj/HSlt7qNBAMopXriW4TRRHkHb0QRtWjBFz/9 X-Received: by 2002:a50:d753:: with SMTP id i19mr37748145edj.43.1615919703115; Tue, 16 Mar 2021 11:35:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1615919703; cv=none; d=google.com; s=arc-20160816; b=fHrWJV/mgHW42jbQpXPegOJhkiFgjK1sxXa23Cl0ww72I96rATeROl5DkSbPH2a+kx gY2E0w7WaE+lr/D9mX9YwugIIu7Znlg4EKMTo3f39Ri/MD8Earcea8ryoQ5KzOScw+nt VGQWjZzRM/GYTP6ZtOyHqdSCWNJ3lWX1oa+I0q69ctXrrDw0E8055qmr1CYdpBWLqjgA /HuXU7WmddkAgeXP0LAxHr89NUazxhgXfUCyGjLnzL8NdMRNSwrZs5IEexLEe3C6BeSO +39xpZRUSkOBAnU8ubt/VhUpl1mNXjO+7m+r9YkoezjMx69e1nDu2xH7yYwCjnX8/wBf 45Yw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=HIj01XgemQbQZifQoxDEsBP1jlJpFbfyWBO4Odh6orE=; b=kfV/28grwgxogR5D+Rl+k3Eiy6jI+MyBOr13CRIK9W8fbLm0DZ7zyd2ol5HwAkRuFQ 9eGJqsnK0w38sIrpShE2VsAjnkFmgF6UqsC8aKbJhb1UgpkV14Rtjw5Sz3SrQVg+u+b0 P17dbvKjsp1gXtPreq3TLciJyyR1/X9XJdv3yjSUJdf3ve/zdeSel1bNqGZiPugO27YI 7NfpNNEsWkvD+UGIJXq5mifDj/BBof/N//ykogNAvpwZOz2xie1MhhtyQf0gTfR9/GXO wc+njxAiogdzN2cC0ccPyggTTMOt/oBYG2VszwKVlSI1CQdPWpWCJ7yBo35u1KmeZ70g JF3Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id bt17si13838739edb.424.2021.03.16.11.35.02; Tue, 16 Mar 2021 11:35:03 -0700 (PDT) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236941AbhCPSea (ORCPT + 12 others); Tue, 16 Mar 2021 14:34:30 -0400 Received: from foss.arm.com ([217.140.110.172]:55918 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240031AbhCPSeH (ORCPT ); Tue, 16 Mar 2021 14:34:07 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 79A2931B; Tue, 16 Mar 2021 11:34:06 -0700 (PDT) Received: from ewhatever.cambridge.arm.com (ewhatever.cambridge.arm.com [10.1.197.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 772B03F70D; Tue, 16 Mar 2021 11:34:05 -0700 (PDT) From: Suzuki K Poulose To: stable@vger.kernel.org Cc: suzuki.poulose@arm.com, maz@kernel.org, catalin.marinas@arm.com, will@kernel.org, alexandru.elisei@arm.com, christoffer.dall@arm.com Subject: [PATCH] KVM: arm64: nvhe: Save the SPE context early Date: Tue, 16 Mar 2021 18:33:53 +0000 Message-Id: <20210316183353.4081445-1-suzuki.poulose@arm.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <16157981451454@kroah.com> References: <16157981451454@kroah.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org commit b96b0c5de685df82019e16826a282d53d86d112c upstream The nVHE KVM hyp drains and disables the SPE buffer, before entering the guest, as the EL1&0 translation regime is going to be loaded with that of the guest. But this operation is performed way too late, because : - The owning translation regime of the SPE buffer is transferred to EL2. (MDCR_EL2_E2PB == 0) - The guest Stage1 is loaded. Thus the flush could use the host EL1 virtual address, but use the EL2 translations instead of host EL1, for writing out any cached data. Fix this by moving the SPE buffer handling early enough. The restore path is doing the right thing. Cc: stable@vger.kernel.org # v5.4- Cc: Christoffer Dall Cc: Marc Zyngier Cc: Will Deacon Cc: Catalin Marinas Cc: Mark Rutland Cc: Alexandru Elisei Signed-off-by: Suzuki K Poulose --- arch/arm64/include/asm/kvm_hyp.h | 3 +++ arch/arm64/kvm/hyp/debug-sr.c | 24 +++++++++++++++--------- arch/arm64/kvm/hyp/switch.c | 13 ++++++++++++- 3 files changed, 30 insertions(+), 10 deletions(-) -- 2.24.1 Acked-by: Marc Zyngier diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 97f21cc66657..7f7fdb16bb96 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -71,6 +71,9 @@ void __sysreg32_restore_state(struct kvm_vcpu *vcpu); void __debug_switch_to_guest(struct kvm_vcpu *vcpu); void __debug_switch_to_host(struct kvm_vcpu *vcpu); +void __debug_save_host_buffers_nvhe(struct kvm_vcpu *vcpu); +void __debug_restore_host_buffers_nvhe(struct kvm_vcpu *vcpu); + void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c index 0fc9872a1467..aead8a5fbe91 100644 --- a/arch/arm64/kvm/hyp/debug-sr.c +++ b/arch/arm64/kvm/hyp/debug-sr.c @@ -168,6 +168,21 @@ static void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu, write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1); } +void __hyp_text __debug_save_host_buffers_nvhe(struct kvm_vcpu *vcpu) +{ + /* + * Non-VHE: Disable and flush SPE data generation + * VHE: The vcpu can run, but it can't hide. + */ + __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1); + +} + +void __hyp_text __debug_restore_host_buffers_nvhe(struct kvm_vcpu *vcpu) +{ + __debug_restore_spe_nvhe(vcpu->arch.host_debug_state.pmscr_el1); +} + void __hyp_text __debug_switch_to_guest(struct kvm_vcpu *vcpu) { struct kvm_cpu_context *host_ctxt; @@ -175,13 +190,6 @@ void __hyp_text __debug_switch_to_guest(struct kvm_vcpu *vcpu) struct kvm_guest_debug_arch *host_dbg; struct kvm_guest_debug_arch *guest_dbg; - /* - * Non-VHE: Disable and flush SPE data generation - * VHE: The vcpu can run, but it can't hide. - */ - if (!has_vhe()) - __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1); - if (!(vcpu->arch.flags & KVM_ARM64_DEBUG_DIRTY)) return; @@ -201,8 +209,6 @@ void __hyp_text __debug_switch_to_host(struct kvm_vcpu *vcpu) struct kvm_guest_debug_arch *host_dbg; struct kvm_guest_debug_arch *guest_dbg; - if (!has_vhe()) - __debug_restore_spe_nvhe(vcpu->arch.host_debug_state.pmscr_el1); if (!(vcpu->arch.flags & KVM_ARM64_DEBUG_DIRTY)) return; diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index 84964983198e..14607fac7ca3 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -682,6 +682,15 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu) __sysreg_save_state_nvhe(host_ctxt); + /* + * We must flush and disable the SPE buffer for nVHE, as + * the translation regime(EL1&0) is going to be loaded with + * that of the guest. And we must do this before we change the + * translation regime to EL2 (via MDCR_EL2_EPB == 0) and + * before we load guest Stage1. + */ + __debug_save_host_buffers_nvhe(vcpu); + __activate_vm(kern_hyp_va(vcpu->kvm)); __activate_traps(vcpu); @@ -720,11 +729,13 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu) if (vcpu->arch.flags & KVM_ARM64_FP_ENABLED) __fpsimd_save_fpexc32(vcpu); + __debug_switch_to_host(vcpu); + /* * This must come after restoring the host sysregs, since a non-VHE * system may enable SPE here and make use of the TTBRs. */ - __debug_switch_to_host(vcpu); + __debug_restore_host_buffers_nvhe(vcpu); if (pmu_switch_needed) __pmu_switch_to_host(host_ctxt);