From patchwork Wed Apr 14 14:49:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 421578 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8B51C4363E for ; Wed, 14 Apr 2021 14:50:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AC1A4611B0 for ; Wed, 14 Apr 2021 14:50:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352072AbhDNOvG (ORCPT ); Wed, 14 Apr 2021 10:51:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38904 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350003AbhDNOut (ORCPT ); Wed, 14 Apr 2021 10:50:49 -0400 Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CB474C061574; Wed, 14 Apr 2021 07:50:26 -0700 (PDT) Received: by mail-pj1-x102c.google.com with SMTP id f2-20020a17090a4a82b02900c67bf8dc69so12659772pjh.1; Wed, 14 Apr 2021 07:50:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Iz6kQKmaAStJz2j939/tUEj2NIuTRDYIEtDill+C7IA=; b=D2taYVTe0z905CX87qYISBKtJJpcQncx/+jlzFtm+dqJ1FQrZXjYZ180oWEO28Y5y7 hlj6+yRPwjy4zDBc0qTSTOI4n3R9lkebe1U3i+D2ckp00qt3mqEM+7XERpL1KMDNXJn4 7wjSfLucr+uk3vZczBhcDhkKTijEPUzFZEFYwB9fuw000bOFSdkhpR5X452OAoNmDZSB 6R6DV6d3gKGUbhWxGXR8WhVAEQ7hP4x7wYAWlA87Le8gNxzh+cepaPjZjUSKkvMtMjhr s43ZTcJK8Kv06ThNFTuC2wlW97YulfZnT+41LKU+KQPRORDoo8Bg8JSixilt/nL1ihKW mQ+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Iz6kQKmaAStJz2j939/tUEj2NIuTRDYIEtDill+C7IA=; b=AATid6iZ/MaYVrgV5BoG0WktjMso1ISqO5MoNS+tQbKPR+vO7ctXEfwOs62LltuWec EhkeF8jvT1QO+Giuxp5RBX5GEG0AEjH+1MpxQno6D2Vf+HGhRoKGr4lpfZ1PSif2PxFg BQaofx9SUuHql05ukc9qeLsJxJJLvhQfeJd5hb9ep0grRimZ94hjFRZbiKymm2xxLl+N HyMhB7yHjsNl07PTAAA+AqZ3ykYXnYMSNRcauxgYa/751zGRYa/7R/R/kyP7vF7EpWtM qsXt+ngr0+PJmO7DPsfBgHdsPF+vJk0gjV1DRJJqG7VSy/GwESvtuJbU4IXLokdHVRvh PTMQ== X-Gm-Message-State: AOAM530VU/uagLS7AtplvH2xozMVyB67rp3ZxaaYCq48Pors8+v57ojh 0gL6w4WNg5UiH6hRni06zDo= X-Google-Smtp-Source: ABdhPJwoWQUtM+Kmo2QdePpZgHK8I2K8arUG2KPZ/TVvHugCh22W4iDEwoDeLZUiglDRWNvFY5wZBA== X-Received: by 2002:a17:90a:bf17:: with SMTP id c23mr4103212pjs.12.1618411826429; Wed, 14 Apr 2021 07:50:26 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:35:ebad:12c1:f579:e332]) by smtp.gmail.com with ESMTPSA id w67sm17732522pgb.87.2021.04.14.07.50.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Apr 2021 07:50:26 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, wei.liu@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, akpm@linux-foundation.org, gregkh@linuxfoundation.org, konrad.wilk@oracle.com, hch@lst.de, m.szyprowski@samsung.com, robin.murphy@arm.com, joro@8bytes.org, will@kernel.org, davem@davemloft.net, kuba@kernel.org, jejb@linux.ibm.com, martin.petersen@oracle.com Cc: Tianyu Lan , iommu@lists.linux-foundation.org, linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-scsi@vger.kernel.org, netdev@vger.kernel.org, vkuznets@redhat.com, thomas.lendacky@amd.com, brijesh.singh@amd.com, sunilmut@microsoft.com Subject: [Resend RFC PATCH V2 02/12] x86/HV: Initialize shared memory boundary in Isolation VM Date: Wed, 14 Apr 2021 10:49:35 -0400 Message-Id: <20210414144945.3460554-3-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210414144945.3460554-1-ltykernel@gmail.com> References: <20210414144945.3460554-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tianyu Lan Hyper-V exposes shared memory boundary via cpuid HYPERV_ CPUID_ISOLATION_CONFIG and store it in the shared_gpa_ boundary of ms_hyperv struct. This prepares to share memory with host for AMD SEV SNP guest. Signed-off-by: Tianyu Lan --- arch/x86/kernel/cpu/mshyperv.c | 2 ++ include/asm-generic/mshyperv.h | 13 ++++++++++++- 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index e88bc296afca..aeafd4017c89 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -328,6 +328,8 @@ static void __init ms_hyperv_init_platform(void) if (ms_hyperv.features_b & HV_ISOLATION) { ms_hyperv.isolation_config_a = cpuid_eax(HYPERV_CPUID_ISOLATION_CONFIG); ms_hyperv.isolation_config_b = cpuid_ebx(HYPERV_CPUID_ISOLATION_CONFIG); + ms_hyperv.shared_gpa_boundary = + (u64)1 << ms_hyperv.shared_gpa_boundary_bits; pr_info("Hyper-V: Isolation Config: Group A 0x%x, Group B 0x%x\n", ms_hyperv.isolation_config_a, ms_hyperv.isolation_config_b); diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h index c6f4c5c20fb8..b73e201abc70 100644 --- a/include/asm-generic/mshyperv.h +++ b/include/asm-generic/mshyperv.h @@ -34,8 +34,19 @@ struct ms_hyperv_info { u32 max_vp_index; u32 max_lp_index; u32 isolation_config_a; - u32 isolation_config_b; + union + { + u32 isolation_config_b; + struct { + u32 cvm_type : 4; + u32 Reserved11 : 1; + u32 shared_gpa_boundary_active : 1; + u32 shared_gpa_boundary_bits : 6; + u32 Reserved12 : 20; + }; + }; void __percpu **ghcb_base; + u64 shared_gpa_boundary; }; extern struct ms_hyperv_info ms_hyperv; From patchwork Wed Apr 14 14:49:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 421577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FBDCC433B4 for ; Wed, 14 Apr 2021 14:50:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5A5D5611EE for ; Wed, 14 Apr 2021 14:50:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352096AbhDNOvS (ORCPT ); Wed, 14 Apr 2021 10:51:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38918 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351955AbhDNOuv (ORCPT ); Wed, 14 Apr 2021 10:50:51 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E48EBC06138C; Wed, 14 Apr 2021 07:50:28 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id w8so7905465plg.9; Wed, 14 Apr 2021 07:50:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4uHLDXPTV0NPaJIPR0JcU4jnzokfZP+lvBQuVxSWxeo=; b=nz98yTtMltCvc8Lxfq4xMqWxGfuacMv6r1QxoLnhLqwEJVXhV14zOJuZJk+Qpe6Eh2 LQBFybrZrRrAPeV5c9jwoCQhjUxuCzEm2HyA8UpdL1ZyGqunpDdxSoB0bpWpXHgydWm0 NP4ZYbte8Cj2Mm9o9A9JalIHNGFbQjKvuhzRjxaq+5eoYjMr5ZkXSrl8kP7EPsWrZXv4 fbIcXezTUHFNuiEl/leuLsb4nTDN3g/iZi6bXSfF/uijSv9wY/FGr+K5QwCHGra8knj3 5NUoOm6VqALCqxQJMX9YpAL3igLtgbiNMoNPGHEOz4kiZ0AW4E3sQ4LJgr4FIkNSG9Xh EBpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4uHLDXPTV0NPaJIPR0JcU4jnzokfZP+lvBQuVxSWxeo=; b=gz9uFWl6bW1rH4r/mwXE6hp9M477Lj7Odakmu6CcvmUezM4tqs+pICNfZ/gXxQo0yq Ltzm6O2Fs0ltsKpI4Es/l3BnDFCrjhVuBUazcDocKOHi/wA1+G9NEFRwdPQ8XRReF/Hh N8lhejf8iWQuh7vMFEO4VOFAirPCsp0/TpkCfHiYGq5bjPh557NCaJtk5nPpNxbL5l+Y kAB+GzxK4+4Cgayx7rqM8l9TH/iyUQibS9o84Ag1CGI9MAlN4wmkD0+KZ+DzlRxt62yx zmqFlByXEyFGiVWR6QvHgBeey6YuZef1lOy07HNc4AaGE7ZIygl69OGcsEbnOCFFVQN1 WqmA== X-Gm-Message-State: AOAM533FI855UKGZEmAmkunjDXn9Q1P0Q/ktak7q+9UzCU/JcE4KDfYr bYMcYXEmED+1Wo2pQjH2+sI= X-Google-Smtp-Source: ABdhPJxNEHJsldB9/l9DrcCPvmaCi5un8TIOv2g6d7WPnLPy6fp1j7ZtGlaVow6yyJCqbdyXmxus9A== X-Received: by 2002:a17:90a:5907:: with SMTP id k7mr4219377pji.197.1618411828425; Wed, 14 Apr 2021 07:50:28 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:35:ebad:12c1:f579:e332]) by smtp.gmail.com with ESMTPSA id w67sm17732522pgb.87.2021.04.14.07.50.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Apr 2021 07:50:28 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, wei.liu@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, akpm@linux-foundation.org, gregkh@linuxfoundation.org, konrad.wilk@oracle.com, hch@lst.de, m.szyprowski@samsung.com, robin.murphy@arm.com, joro@8bytes.org, will@kernel.org, davem@davemloft.net, kuba@kernel.org, jejb@linux.ibm.com, martin.petersen@oracle.com Cc: Tianyu Lan , iommu@lists.linux-foundation.org, linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-scsi@vger.kernel.org, netdev@vger.kernel.org, vkuznets@redhat.com, thomas.lendacky@amd.com, brijesh.singh@amd.com, sunilmut@microsoft.com Subject: [Resend RFC PATCH V2 04/12] HV: Add Write/Read MSR registers via ghcb Date: Wed, 14 Apr 2021 10:49:37 -0400 Message-Id: <20210414144945.3460554-5-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210414144945.3460554-1-ltykernel@gmail.com> References: <20210414144945.3460554-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tianyu Lan Hyper-V provides GHCB protocol to write Synthetic Interrupt Controller MSR registers and these registers are emulated by Hypervisor rather than paravisor. Hyper-V requests to write SINTx MSR registers twice(once via GHCB and once via wrmsr instruction including the proxy bit 21) Guest OS ID MSR also needs to be set via GHCB. Signed-off-by: Tianyu Lan --- arch/x86/hyperv/hv_init.c | 18 +---- arch/x86/hyperv/ivm.c | 130 ++++++++++++++++++++++++++++++++ arch/x86/include/asm/mshyperv.h | 87 +++++++++++++++++---- arch/x86/kernel/cpu/mshyperv.c | 3 + drivers/hv/hv.c | 65 +++++++++++----- include/asm-generic/mshyperv.h | 4 +- 6 files changed, 261 insertions(+), 46 deletions(-) diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c index 90e65fbf4c58..87b1dd9c84d6 100644 --- a/arch/x86/hyperv/hv_init.c +++ b/arch/x86/hyperv/hv_init.c @@ -475,6 +475,9 @@ void __init hyperv_init(void) ghcb_base = (void **)this_cpu_ptr(ms_hyperv.ghcb_base); *ghcb_base = ghcb_va; + + /* Hyper-V requires to write guest os id via ghcb in SNP IVM. */ + hv_ghcb_msr_write(HV_X64_MSR_GUEST_OS_ID, guest_id); } rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64); @@ -561,6 +564,7 @@ void hyperv_cleanup(void) /* Reset our OS id */ wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0); + hv_ghcb_msr_write(HV_X64_MSR_GUEST_OS_ID, 0); /* * Reset hypercall page reference before reset the page, @@ -668,17 +672,3 @@ bool hv_is_hibernation_supported(void) return !hv_root_partition && acpi_sleep_state_supported(ACPI_STATE_S4); } EXPORT_SYMBOL_GPL(hv_is_hibernation_supported); - -enum hv_isolation_type hv_get_isolation_type(void) -{ - if (!(ms_hyperv.features_b & HV_ISOLATION)) - return HV_ISOLATION_TYPE_NONE; - return FIELD_GET(HV_ISOLATION_TYPE, ms_hyperv.isolation_config_b); -} -EXPORT_SYMBOL_GPL(hv_get_isolation_type); - -bool hv_is_isolation_supported(void) -{ - return hv_get_isolation_type() != HV_ISOLATION_TYPE_NONE; -} -EXPORT_SYMBOL_GPL(hv_is_isolation_supported); diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index a5950b7a9214..2ec64b367aaf 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -6,12 +6,139 @@ * Tianyu Lan */ +#include +#include #include #include #include #include +#include +#include #include +union hv_ghcb { + struct ghcb ghcb; +} __packed __aligned(PAGE_SIZE); + +void hv_ghcb_msr_write(u64 msr, u64 value) +{ + union hv_ghcb *hv_ghcb; + void **ghcb_base; + unsigned long flags; + + if (!ms_hyperv.ghcb_base) + return; + + local_irq_save(flags); + ghcb_base = (void **)this_cpu_ptr(ms_hyperv.ghcb_base); + hv_ghcb = (union hv_ghcb *)*ghcb_base; + if (!hv_ghcb) { + local_irq_restore(flags); + return; + } + + memset(hv_ghcb, 0x00, HV_HYP_PAGE_SIZE); + + hv_ghcb->ghcb.protocol_version = 1; + hv_ghcb->ghcb.ghcb_usage = 0; + + ghcb_set_sw_exit_code(&hv_ghcb->ghcb, SVM_EXIT_MSR); + ghcb_set_rcx(&hv_ghcb->ghcb, msr); + ghcb_set_rax(&hv_ghcb->ghcb, lower_32_bits(value)); + ghcb_set_rdx(&hv_ghcb->ghcb, value >> 32); + ghcb_set_sw_exit_info_1(&hv_ghcb->ghcb, 1); + ghcb_set_sw_exit_info_2(&hv_ghcb->ghcb, 0); + + VMGEXIT(); + + if ((hv_ghcb->ghcb.save.sw_exit_info_1 & 0xffffffff) == 1) + pr_warn("Fail to write msr via ghcb.\n."); + + local_irq_restore(flags); +} +EXPORT_SYMBOL_GPL(hv_ghcb_msr_write); + +void hv_ghcb_msr_read(u64 msr, u64 *value) +{ + union hv_ghcb *hv_ghcb; + void **ghcb_base; + unsigned long flags; + + if (!ms_hyperv.ghcb_base) + return; + + local_irq_save(flags); + ghcb_base = (void **)this_cpu_ptr(ms_hyperv.ghcb_base); + hv_ghcb = (union hv_ghcb *)*ghcb_base; + if (!hv_ghcb) { + local_irq_restore(flags); + return; + } + + memset(hv_ghcb, 0x00, PAGE_SIZE); + hv_ghcb->ghcb.protocol_version = 1; + hv_ghcb->ghcb.ghcb_usage = 0; + + ghcb_set_sw_exit_code(&hv_ghcb->ghcb, SVM_EXIT_MSR); + ghcb_set_rcx(&hv_ghcb->ghcb, msr); + ghcb_set_sw_exit_info_1(&hv_ghcb->ghcb, 0); + ghcb_set_sw_exit_info_2(&hv_ghcb->ghcb, 0); + + VMGEXIT(); + + if ((hv_ghcb->ghcb.save.sw_exit_info_1 & 0xffffffff) == 1) + pr_warn("Fail to write msr via ghcb.\n."); + else + *value = (u64)lower_32_bits(hv_ghcb->ghcb.save.rax) + | ((u64)lower_32_bits(hv_ghcb->ghcb.save.rdx) << 32); + local_irq_restore(flags); +} +EXPORT_SYMBOL_GPL(hv_ghcb_msr_read); + +void hv_sint_rdmsrl_ghcb(u64 msr, u64 *value) +{ + hv_ghcb_msr_read(msr, value); +} +EXPORT_SYMBOL_GPL(hv_sint_rdmsrl_ghcb); + +void hv_sint_wrmsrl_ghcb(u64 msr, u64 value) +{ + hv_ghcb_msr_write(msr, value); + + /* Write proxy bit vua wrmsrl instruction. */ + if (msr >= HV_X64_MSR_SINT0 && msr <= HV_X64_MSR_SINT15) + wrmsrl(msr, value | 1 << 20); +} +EXPORT_SYMBOL_GPL(hv_sint_wrmsrl_ghcb); + +inline void hv_signal_eom_ghcb(void) +{ + hv_sint_wrmsrl_ghcb(HV_X64_MSR_EOM, 0); +} +EXPORT_SYMBOL_GPL(hv_signal_eom_ghcb); + +enum hv_isolation_type hv_get_isolation_type(void) +{ + if (!(ms_hyperv.features_b & HV_ISOLATION)) + return HV_ISOLATION_TYPE_NONE; + return FIELD_GET(HV_ISOLATION_TYPE, ms_hyperv.isolation_config_b); +} +EXPORT_SYMBOL_GPL(hv_get_isolation_type); + +bool hv_is_isolation_supported(void) +{ + return hv_get_isolation_type() != HV_ISOLATION_TYPE_NONE; +} +EXPORT_SYMBOL_GPL(hv_is_isolation_supported); + +DEFINE_STATIC_KEY_FALSE(isolation_type_snp); + +bool hv_isolation_type_snp(void) +{ + return static_branch_unlikely(&isolation_type_snp); +} +EXPORT_SYMBOL_GPL(hv_isolation_type_snp); + /* * hv_set_mem_host_visibility - Set host visibility for specified memory. */ @@ -22,6 +149,9 @@ int hv_set_mem_host_visibility(void *kbuffer, u32 size, u32 visibility) u64 *pfn_array; int ret = 0; + if (!hv_is_isolation_supported()) + return 0; + pfn_array = vzalloc(HV_HYP_PAGE_SIZE); if (!pfn_array) return -ENOMEM; diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index d9437f096ce5..73501dbbc240 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -10,6 +10,8 @@ #include #include +DECLARE_STATIC_KEY_FALSE(isolation_type_snp); + typedef int (*hyperv_fill_flush_list_func)( struct hv_guest_mapping_flush_list *flush, void *data); @@ -19,24 +21,64 @@ typedef int (*hyperv_fill_flush_list_func)( #define hv_init_timer_config(timer, val) \ wrmsrl(HV_X64_MSR_STIMER0_CONFIG + (2*timer), val) -#define hv_get_simp(val) rdmsrl(HV_X64_MSR_SIMP, val) -#define hv_set_simp(val) wrmsrl(HV_X64_MSR_SIMP, val) +#define hv_get_sint_reg(val, reg) { \ + if (hv_isolation_type_snp()) \ + hv_get_##reg##_ghcb(&val); \ + else \ + rdmsrl(HV_X64_MSR_##reg, val); \ + } + +#define hv_set_sint_reg(val, reg) { \ + if (hv_isolation_type_snp()) \ + hv_set_##reg##_ghcb(val); \ + else \ + wrmsrl(HV_X64_MSR_##reg, val); \ + } + -#define hv_get_siefp(val) rdmsrl(HV_X64_MSR_SIEFP, val) -#define hv_set_siefp(val) wrmsrl(HV_X64_MSR_SIEFP, val) +#define hv_get_simp(val) hv_get_sint_reg(val, SIMP) +#define hv_get_siefp(val) hv_get_sint_reg(val, SIEFP) -#define hv_get_synic_state(val) rdmsrl(HV_X64_MSR_SCONTROL, val) -#define hv_set_synic_state(val) wrmsrl(HV_X64_MSR_SCONTROL, val) +#define hv_set_simp(val) hv_set_sint_reg(val, SIMP) +#define hv_set_siefp(val) hv_set_sint_reg(val, SIEFP) + +#define hv_get_synic_state(val) { \ + if (hv_isolation_type_snp()) \ + hv_get_synic_state_ghcb(&val); \ + else \ + rdmsrl(HV_X64_MSR_SCONTROL, val); \ + } +#define hv_set_synic_state(val) { \ + if (hv_isolation_type_snp()) \ + hv_set_synic_state_ghcb(val); \ + else \ + wrmsrl(HV_X64_MSR_SCONTROL, val); \ + } #define hv_get_vp_index(index) rdmsrl(HV_X64_MSR_VP_INDEX, index) -#define hv_signal_eom() wrmsrl(HV_X64_MSR_EOM, 0) +#define hv_signal_eom() { \ + if (hv_isolation_type_snp() && \ + old_msg_type != HVMSG_TIMER_EXPIRED) \ + hv_signal_eom_ghcb(); \ + else \ + wrmsrl(HV_X64_MSR_EOM, 0); \ + } -#define hv_get_synint_state(int_num, val) \ - rdmsrl(HV_X64_MSR_SINT0 + int_num, val) -#define hv_set_synint_state(int_num, val) \ - wrmsrl(HV_X64_MSR_SINT0 + int_num, val) -#define hv_recommend_using_aeoi() \ +#define hv_get_synint_state(int_num, val) { \ + if (hv_isolation_type_snp()) \ + hv_get_synint_state_ghcb(int_num, &val);\ + else \ + rdmsrl(HV_X64_MSR_SINT0 + int_num, val);\ + } +#define hv_set_synint_state(int_num, val) { \ + if (hv_isolation_type_snp()) \ + hv_set_synint_state_ghcb(int_num, val); \ + else \ + wrmsrl(HV_X64_MSR_SINT0 + int_num, val);\ + } + +#define hv_recommend_using_aeoi() \ (!(ms_hyperv.hints & HV_DEPRECATING_AEOI_RECOMMENDED)) #define hv_get_crash_ctl(val) \ @@ -271,6 +313,25 @@ int hv_unmap_ioapic_interrupt(int ioapic_id, struct hv_interrupt_entry *entry); int hv_set_mem_host_visibility(void *kbuffer, u32 size, u32 visibility); int hv_mark_gpa_visibility(u16 count, const u64 pfn[], u32 visibility); +void hv_sint_wrmsrl_ghcb(u64 msr, u64 value); +void hv_sint_rdmsrl_ghcb(u64 msr, u64 *value); +void hv_signal_eom_ghcb(void); +void hv_ghcb_msr_write(u64 msr, u64 value); +void hv_ghcb_msr_read(u64 msr, u64 *value); + +#define hv_get_synint_state_ghcb(int_num, val) \ + hv_sint_rdmsrl_ghcb(HV_X64_MSR_SINT0 + int_num, val) +#define hv_set_synint_state_ghcb(int_num, val) \ + hv_sint_wrmsrl_ghcb(HV_X64_MSR_SINT0 + int_num, val) + +#define hv_get_SIMP_ghcb(val) hv_sint_rdmsrl_ghcb(HV_X64_MSR_SIMP, val) +#define hv_set_SIMP_ghcb(val) hv_sint_wrmsrl_ghcb(HV_X64_MSR_SIMP, val) + +#define hv_get_SIEFP_ghcb(val) hv_sint_rdmsrl_ghcb(HV_X64_MSR_SIEFP, val) +#define hv_set_SIEFP_ghcb(val) hv_sint_wrmsrl_ghcb(HV_X64_MSR_SIEFP, val) + +#define hv_get_synic_state_ghcb(val) hv_sint_rdmsrl_ghcb(HV_X64_MSR_SCONTROL, val) +#define hv_set_synic_state_ghcb(val) hv_sint_wrmsrl_ghcb(HV_X64_MSR_SCONTROL, val) #else /* CONFIG_HYPERV */ static inline void hyperv_init(void) {} static inline void hyperv_setup_mmu_ops(void) {} @@ -289,9 +350,9 @@ static inline int hyperv_flush_guest_mapping_range(u64 as, { return -1; } +static inline void hv_signal_eom_ghcb(void) { }; #endif /* CONFIG_HYPERV */ - #include #endif diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index aeafd4017c89..6eaa0891c0f7 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -333,6 +333,9 @@ static void __init ms_hyperv_init_platform(void) pr_info("Hyper-V: Isolation Config: Group A 0x%x, Group B 0x%x\n", ms_hyperv.isolation_config_a, ms_hyperv.isolation_config_b); + + if (hv_get_isolation_type() == HV_ISOLATION_TYPE_SNP) + static_branch_enable(&isolation_type_snp); } if (ms_hyperv.hints & HV_X64_ENLIGHTENED_VMCS_RECOMMENDED) { diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c index f202ac7f4b3d..069530eeb7c6 100644 --- a/drivers/hv/hv.c +++ b/drivers/hv/hv.c @@ -99,17 +99,24 @@ int hv_synic_alloc(void) tasklet_init(&hv_cpu->msg_dpc, vmbus_on_msg_dpc, (unsigned long) hv_cpu); - hv_cpu->synic_message_page = - (void *)get_zeroed_page(GFP_ATOMIC); - if (hv_cpu->synic_message_page == NULL) { - pr_err("Unable to allocate SYNIC message page\n"); - goto err; - } + /* + * Synic message and event pages are allocated by paravisor. + * Skip these pages allocation here. + */ + if (!hv_isolation_type_snp()) { + hv_cpu->synic_message_page = + (void *)get_zeroed_page(GFP_ATOMIC); + if (hv_cpu->synic_message_page == NULL) { + pr_err("Unable to allocate SYNIC message page\n"); + goto err; + } - hv_cpu->synic_event_page = (void *)get_zeroed_page(GFP_ATOMIC); - if (hv_cpu->synic_event_page == NULL) { - pr_err("Unable to allocate SYNIC event page\n"); - goto err; + hv_cpu->synic_event_page = + (void *)get_zeroed_page(GFP_ATOMIC); + if (hv_cpu->synic_event_page == NULL) { + pr_err("Unable to allocate SYNIC event page\n"); + goto err; + } } hv_cpu->post_msg_page = (void *)get_zeroed_page(GFP_ATOMIC); @@ -136,10 +143,17 @@ void hv_synic_free(void) for_each_present_cpu(cpu) { struct hv_per_cpu_context *hv_cpu = per_cpu_ptr(hv_context.cpu_context, cpu); + free_page((unsigned long)hv_cpu->post_msg_page); + + /* + * Synic message and event pages are allocated by paravisor. + * Skip free these pages here. + */ + if (hv_isolation_type_snp()) + continue; free_page((unsigned long)hv_cpu->synic_event_page); free_page((unsigned long)hv_cpu->synic_message_page); - free_page((unsigned long)hv_cpu->post_msg_page); } kfree(hv_context.hv_numa_map); @@ -161,20 +175,36 @@ void hv_synic_enable_regs(unsigned int cpu) union hv_synic_sint shared_sint; union hv_synic_scontrol sctrl; - /* Setup the Synic's message page */ + + /* Setup the Synic's message. */ hv_get_simp(simp.as_uint64); simp.simp_enabled = 1; - simp.base_simp_gpa = virt_to_phys(hv_cpu->synic_message_page) - >> HV_HYP_PAGE_SHIFT; + + if (hv_isolation_type_snp()) { + hv_cpu->synic_message_page + = ioremap_cache(simp.base_simp_gpa << HV_HYP_PAGE_SHIFT, + HV_HYP_PAGE_SIZE); + if (!hv_cpu->synic_message_page) + pr_err("Fail to map syinc message page.\n"); + } else { + simp.base_simp_gpa = virt_to_phys(hv_cpu->synic_message_page) + >> HV_HYP_PAGE_SHIFT; + } hv_set_simp(simp.as_uint64); /* Setup the Synic's event page */ hv_get_siefp(siefp.as_uint64); siefp.siefp_enabled = 1; - siefp.base_siefp_gpa = virt_to_phys(hv_cpu->synic_event_page) - >> HV_HYP_PAGE_SHIFT; - + if (hv_isolation_type_snp()) { + hv_cpu->synic_event_page = ioremap_cache( + siefp.base_siefp_gpa << HV_HYP_PAGE_SHIFT, HV_HYP_PAGE_SIZE); + if (!hv_cpu->synic_event_page) + pr_err("Fail to map syinc event page.\n"); + } else { + siefp.base_siefp_gpa = virt_to_phys(hv_cpu->synic_event_page) + >> HV_HYP_PAGE_SHIFT; + } hv_set_siefp(siefp.as_uint64); /* Setup the shared SINT. */ @@ -188,7 +218,6 @@ void hv_synic_enable_regs(unsigned int cpu) /* Enable the global synic bit */ hv_get_synic_state(sctrl.as_uint64); sctrl.enable = 1; - hv_set_synic_state(sctrl.as_uint64); } diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h index b73e201abc70..59777377e641 100644 --- a/include/asm-generic/mshyperv.h +++ b/include/asm-generic/mshyperv.h @@ -23,6 +23,7 @@ #include #include #include +#include #include struct ms_hyperv_info { @@ -52,7 +53,7 @@ extern struct ms_hyperv_info ms_hyperv; extern u64 hv_do_hypercall(u64 control, void *inputaddr, void *outputaddr); extern u64 hv_do_fast_hypercall8(u16 control, u64 input8); - +extern bool hv_isolation_type_snp(void); /* Generate the guest OS identifier as described in the Hyper-V TLFS */ static inline __u64 generate_guest_id(__u64 d_info1, __u64 kernel_version, @@ -186,6 +187,7 @@ bool hv_is_hyperv_initialized(void); bool hv_is_hibernation_supported(void); enum hv_isolation_type hv_get_isolation_type(void); bool hv_is_isolation_supported(void); +bool hv_isolation_type_snp(void); void hyperv_cleanup(void); #else /* CONFIG_HYPERV */ static inline bool hv_is_hyperv_initialized(void) { return false; } From patchwork Wed Apr 14 14:49:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 421576 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC3C3C43470 for ; Wed, 14 Apr 2021 14:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C6C7B6117A for ; Wed, 14 Apr 2021 14:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352113AbhDNOvX (ORCPT ); Wed, 14 Apr 2021 10:51:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38904 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351963AbhDNOuw (ORCPT ); Wed, 14 Apr 2021 10:50:52 -0400 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB1F2C06138D; Wed, 14 Apr 2021 07:50:30 -0700 (PDT) Received: by mail-pl1-x636.google.com with SMTP id u15so1688487plf.10; Wed, 14 Apr 2021 07:50:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CewzcGN2queQHJd+J82WZLFtEsY18cNfiGHIrQj3GfM=; b=nOk4uGV2612icoE6KiiMnihVY5JHClv+KaiNAxHeX6msxOQj9xeFgeyXgzuV9Sd7Ij 3EGstiuG7E5ZaMqZazQ8zFoKTzoT62iMzgmBH+tZh+iYpEEsK600XCCcKCclZh0Y0x/u lPI4sDQZdZAMWqD+sOUOmLSztUBF0emqbSbT/bkw4pOBBCkruQo/ok2VzdifznwsKJMs ocsgxtRZuZhwa/4H3uMNQQH/MJK+ZGYBL5Une8lbE2Hi2dVG9kDPFVwgbdYojKKilNwO hZojnsV4Uidmr/W8cT1MCoZwuTcUd6+620Huo4X0ZKIUTvMpEIF6giQ2Vs7uJP0RlbHX V1IA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CewzcGN2queQHJd+J82WZLFtEsY18cNfiGHIrQj3GfM=; b=A2zSNmAY/eNh5iTFCqMjc0M/4kLDDRmqA3/NStBj5XrJAmzjjRSqHDnLv4z1u5OikC ZziqdwZTqEM1Nhfq5+ogcI+tnzhPFInfkSrKi4LiTgPE8LT7rxqKylt8GS5zZNDWA7Ac bG4s8GjJiXwuYREhwknBxWvks0LxqvnAPIqLtEbrlF0XCsm/85uhVW7588aWZ9PJbFih r+5CFLd05zckaqfqYhFPrOx8pP7I3fSVFH1bR4N/R+4/jX3T972kHGWFzHBWgeuy3Top iDhq1zrPx8snO8fl7hJxuiX6cbd0F2uCuCQinE12Zjfd35F0mVtnYtnuRNdjjXQ0guPI +RyQ== X-Gm-Message-State: AOAM5320WSVKha8CREFKYGwvPHXJSx8RlmNzdB4gp5i3eNeOneaLgjsg fFWY45IQBzE0lOM+VcgNrnk= X-Google-Smtp-Source: ABdhPJwbB1PldwXbeyiSuCQkyOEtifm++hxPqnW8Qh9Ekcj5s31E29ArK58tbba9T8p/t1IP5sOX7w== X-Received: by 2002:a17:90a:5306:: with SMTP id x6mr4089960pjh.39.1618411830278; Wed, 14 Apr 2021 07:50:30 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:35:ebad:12c1:f579:e332]) by smtp.gmail.com with ESMTPSA id w67sm17732522pgb.87.2021.04.14.07.50.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Apr 2021 07:50:29 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, wei.liu@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, akpm@linux-foundation.org, gregkh@linuxfoundation.org, konrad.wilk@oracle.com, hch@lst.de, m.szyprowski@samsung.com, robin.murphy@arm.com, joro@8bytes.org, will@kernel.org, davem@davemloft.net, kuba@kernel.org, jejb@linux.ibm.com, martin.petersen@oracle.com Cc: Tianyu Lan , iommu@lists.linux-foundation.org, linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-scsi@vger.kernel.org, netdev@vger.kernel.org, vkuznets@redhat.com, thomas.lendacky@amd.com, brijesh.singh@amd.com, sunilmut@microsoft.com Subject: [Resend RFC PATCH V2 06/12] HV/Vmbus: Add SNP support for VMbus channel initiate message Date: Wed, 14 Apr 2021 10:49:39 -0400 Message-Id: <20210414144945.3460554-7-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210414144945.3460554-1-ltykernel@gmail.com> References: <20210414144945.3460554-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tianyu Lan The physical address of monitor pages in the CHANNELMSG_INITIATE_CONTACT msg should be in the extra address space for SNP support and these pages also should be accessed via the extra address space inside Linux guest and remap the extra address by ioremap function. Signed-off-by: Tianyu Lan --- drivers/hv/connection.c | 62 +++++++++++++++++++++++++++++++++++++++ drivers/hv/hyperv_vmbus.h | 1 + 2 files changed, 63 insertions(+) diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c index 79bca653dce9..a0be9c11d737 100644 --- a/drivers/hv/connection.c +++ b/drivers/hv/connection.c @@ -101,6 +101,12 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version) msg->monitor_page1 = virt_to_phys(vmbus_connection.monitor_pages[0]); msg->monitor_page2 = virt_to_phys(vmbus_connection.monitor_pages[1]); + + if (hv_isolation_type_snp()) { + msg->monitor_page1 += ms_hyperv.shared_gpa_boundary; + msg->monitor_page2 += ms_hyperv.shared_gpa_boundary; + } + msg->target_vcpu = hv_cpu_number_to_vp_number(VMBUS_CONNECT_CPU); /* @@ -145,6 +151,29 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version) return -ECONNREFUSED; } + if (hv_isolation_type_snp()) { + vmbus_connection.monitor_pages_va[0] + = vmbus_connection.monitor_pages[0]; + vmbus_connection.monitor_pages[0] + = ioremap_cache(msg->monitor_page1, HV_HYP_PAGE_SIZE); + if (!vmbus_connection.monitor_pages[0]) + return -ENOMEM; + + vmbus_connection.monitor_pages_va[1] + = vmbus_connection.monitor_pages[1]; + vmbus_connection.monitor_pages[1] + = ioremap_cache(msg->monitor_page2, HV_HYP_PAGE_SIZE); + if (!vmbus_connection.monitor_pages[1]) { + vunmap(vmbus_connection.monitor_pages[0]); + return -ENOMEM; + } + + memset(vmbus_connection.monitor_pages[0], 0x00, + HV_HYP_PAGE_SIZE); + memset(vmbus_connection.monitor_pages[1], 0x00, + HV_HYP_PAGE_SIZE); + } + return ret; } @@ -156,6 +185,7 @@ int vmbus_connect(void) struct vmbus_channel_msginfo *msginfo = NULL; int i, ret = 0; __u32 version; + u64 pfn[2]; /* Initialize the vmbus connection */ vmbus_connection.conn_state = CONNECTING; @@ -213,6 +243,16 @@ int vmbus_connect(void) goto cleanup; } + if (hv_isolation_type_snp()) { + pfn[0] = virt_to_hvpfn(vmbus_connection.monitor_pages[0]); + pfn[1] = virt_to_hvpfn(vmbus_connection.monitor_pages[1]); + if (hv_mark_gpa_visibility(2, pfn, + VMBUS_PAGE_VISIBLE_READ_WRITE)) { + ret = -EFAULT; + goto cleanup; + } + } + msginfo = kzalloc(sizeof(*msginfo) + sizeof(struct vmbus_channel_initiate_contact), GFP_KERNEL); @@ -279,6 +319,8 @@ int vmbus_connect(void) void vmbus_disconnect(void) { + u64 pfn[2]; + /* * First send the unload request to the host. */ @@ -298,6 +340,26 @@ void vmbus_disconnect(void) vmbus_connection.int_page = NULL; } + if (hv_isolation_type_snp()) { + if (vmbus_connection.monitor_pages_va[0]) { + vunmap(vmbus_connection.monitor_pages[0]); + vmbus_connection.monitor_pages[0] + = vmbus_connection.monitor_pages_va[0]; + vmbus_connection.monitor_pages_va[0] = NULL; + } + + if (vmbus_connection.monitor_pages_va[1]) { + vunmap(vmbus_connection.monitor_pages[1]); + vmbus_connection.monitor_pages[1] + = vmbus_connection.monitor_pages_va[1]; + vmbus_connection.monitor_pages_va[1] = NULL; + } + + pfn[0] = virt_to_hvpfn(vmbus_connection.monitor_pages[0]); + pfn[1] = virt_to_hvpfn(vmbus_connection.monitor_pages[1]); + hv_mark_gpa_visibility(2, pfn, VMBUS_PAGE_NOT_VISIBLE); + } + hv_free_hyperv_page((unsigned long)vmbus_connection.monitor_pages[0]); hv_free_hyperv_page((unsigned long)vmbus_connection.monitor_pages[1]); vmbus_connection.monitor_pages[0] = NULL; diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h index 9416e09ebd58..0778add21a9c 100644 --- a/drivers/hv/hyperv_vmbus.h +++ b/drivers/hv/hyperv_vmbus.h @@ -240,6 +240,7 @@ struct vmbus_connection { * is child->parent notification */ struct hv_monitor_page *monitor_pages[2]; + void *monitor_pages_va[2]; struct list_head chn_msg_list; spinlock_t channelmsg_lock; From patchwork Wed Apr 14 14:49:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 421575 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E404C43611 for ; Wed, 14 Apr 2021 14:51:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 410AF611F0 for ; Wed, 14 Apr 2021 14:51:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352129AbhDNOv2 (ORCPT ); Wed, 14 Apr 2021 10:51:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351969AbhDNOux (ORCPT ); Wed, 14 Apr 2021 10:50:53 -0400 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5D4D1C061756; Wed, 14 Apr 2021 07:50:32 -0700 (PDT) Received: by mail-pg1-x532.google.com with SMTP id f29so14577103pgm.8; Wed, 14 Apr 2021 07:50:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bVkRTqUPBzQHoWP1iz19JnEvLpWiwpbLKKeUDMKlliw=; b=E+2cIG4luSiauZCXt8F0KkKTk61dfk7whIIE/VHBZpNK6ARjyQdSQbpdQClQ4sSrp/ dGr6hJu/fS4Ac4GvGFZRUctlN3UoGYzjwv/J4NIfKYbBrLXDX0DII3Kqd0z13GyzT7N9 3K8eWfVsyRnUmawlgrvkAxuuTExFaWpsJRH42W4ZrmtKgAL9Ko81HY7WIxnfJaEZ8aiH Re9gsUe5b3chBVWCDNntU9qAIDzO3tUL22enF3c2ium4ZddwWgTLyh5yVJnZkUplcHhN 5Di+lzE1Zg2G82MYEnz9s+UIXemJpVVS+BIdIcbJOLkBiyzwWmhzx/daa53C594lB6Z4 MjZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bVkRTqUPBzQHoWP1iz19JnEvLpWiwpbLKKeUDMKlliw=; b=aDPftXD0vZ+cp0ekM59SlWDwFIKr348YQdK0x7vNhL7sfG9ZdWzWqLRz0Brktf47mJ Bly5vE8eVVMhJO8uBxSvHMDHMfF5gdA4sP5OXh80fLDPcLVMK37Tgi4FOwlE1JUCgW7W 7/BCzuexFCZH25V6k+9B9l94LThQ4se/SX7PjRo0k4CVcGmxyniDCQqny87Gig3T2uVd IxBPGtIB9+Gut0Y9phvxFBpQ3VIOx/quuvyxUOiP+GnmWbXe2SbFphnq+PG/ASsUlMqw WagdgNM1JW6n/ch7Pw2lubs388W3wHUetPYY7gZiUO0IG1N3wT7tlVgx8+mPRCCGQ4AZ 1u1w== X-Gm-Message-State: AOAM533s0XiZ93jVavkCA5rkn8tONKfik7MoLMZaT77DVytldPZ/vh85 r/e7Tzun/rBPp0qn7eg7SHk= X-Google-Smtp-Source: ABdhPJyJQo3QYxkwZd2R/2jbCS/20DZeDnEWp76po+53FmLx6JhMmfhewCcVF2Q2CCy9ma7VhRrp6w== X-Received: by 2002:a63:4e47:: with SMTP id o7mr37301695pgl.286.1618411832013; Wed, 14 Apr 2021 07:50:32 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:35:ebad:12c1:f579:e332]) by smtp.gmail.com with ESMTPSA id w67sm17732522pgb.87.2021.04.14.07.50.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Apr 2021 07:50:31 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, wei.liu@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, akpm@linux-foundation.org, gregkh@linuxfoundation.org, konrad.wilk@oracle.com, hch@lst.de, m.szyprowski@samsung.com, robin.murphy@arm.com, joro@8bytes.org, will@kernel.org, davem@davemloft.net, kuba@kernel.org, jejb@linux.ibm.com, martin.petersen@oracle.com Cc: Tianyu Lan , iommu@lists.linux-foundation.org, linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-scsi@vger.kernel.org, netdev@vger.kernel.org, vkuznets@redhat.com, thomas.lendacky@amd.com, brijesh.singh@amd.com, sunilmut@microsoft.com Subject: [Resend RFC PATCH V2 08/12] UIO/Hyper-V: Not load UIO HV driver in the isolation VM. Date: Wed, 14 Apr 2021 10:49:41 -0400 Message-Id: <20210414144945.3460554-9-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210414144945.3460554-1-ltykernel@gmail.com> References: <20210414144945.3460554-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tianyu Lan UIO HV driver should not load in the isolation VM for security reason. Return ENOTSUPP in the hv_uio_probe() in the isolation VM. Signed-off-by: Tianyu Lan --- drivers/uio/uio_hv_generic.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/uio/uio_hv_generic.c b/drivers/uio/uio_hv_generic.c index 0330ba99730e..678b021d66f8 100644 --- a/drivers/uio/uio_hv_generic.c +++ b/drivers/uio/uio_hv_generic.c @@ -29,6 +29,7 @@ #include #include #include +#include #include "../hv/hyperv_vmbus.h" @@ -241,6 +242,10 @@ hv_uio_probe(struct hv_device *dev, void *ring_buffer; int ret; + /* UIO driver should not be loaded in the isolation VM.*/ + if (hv_is_isolation_supported()) + return -ENOTSUPP; + /* Communicating with host has to be via shared memory not hypercall */ if (!channel->offermsg.monitor_allocated) { dev_err(&dev->device, "vmbus channel requires hypercall\n"); From patchwork Wed Apr 14 14:49:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 421574 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 444F3C433ED for ; Wed, 14 Apr 2021 14:51:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 188F1611B0 for ; Wed, 14 Apr 2021 14:51:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350080AbhDNOvf (ORCPT ); Wed, 14 Apr 2021 10:51:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38918 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351974AbhDNOuy (ORCPT ); Wed, 14 Apr 2021 10:50:54 -0400 Received: from mail-pg1-x52e.google.com (mail-pg1-x52e.google.com [IPv6:2607:f8b0:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F89DC06138C; Wed, 14 Apr 2021 07:50:33 -0700 (PDT) Received: by mail-pg1-x52e.google.com with SMTP id l76so14569079pga.6; Wed, 14 Apr 2021 07:50:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SNYshAOIhIfYWplCoKI3GKQhqu3xrQdJgtDgLoPsrVY=; b=gibuXI9Xv7PARINSU43+2rKSBQQrekXZwe4JwQ/fSldoGkdBytnFZKjdh1uWMusUxb BgQY9KiYPHHhszJbeOVJvpB162ge6nEZtamVklk/8VdVGikS2QFEHa3d9VbIgsDYtERj U1OSC2tKMrST6MfaGv0Wb6lLasokI084S8I+7JWirTDKYvAwF3mlQNItQCfKFGXibXmC 7VRGkERZJV1hiFl4tiD76EFkxRAk795i6uHAAHh8lY4jJ7szOhMfztnOyTsBNuY9Oyph VepV73vkdDvoHqf5lLhBskr9fBJmbsLOV+hZuDDGaRYdMEzhtsR6nl0UPEAno8awUema iuOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SNYshAOIhIfYWplCoKI3GKQhqu3xrQdJgtDgLoPsrVY=; b=WyFm8hdhyNdU97kPaXyUYnTAyEeTNBbPcnO+TJ8Fg8FokWcMho+yuus+ZxLFrKzOOf mXm2ziEPweDCnvXSTpKGc+mdGCu4Ytde9/pcm1LQ1A+SzuIpFwgQ55yAmvp4IVXT1tEV 02sjV4+dEbIVhfGMujaDxwox+prMTvBwGaefNL5KiXOoYNtRkWGylRZMNIWg1qQzBb33 G130oX2DHCbfSCZy+GNIsiolIT3wPMtUoTpfhVAVNaeUZX+YU+7bp1GF9hxSp6r/bhES gZXKsH1OsCB3b5cHuBxdwWR0+vSdUPk+n1hB9jNUxegmlN+rC8D/xSi0SPVnfgKGIt8G imJw== X-Gm-Message-State: AOAM532ap4JHEYp+4mHyATGPmV0XBUm0s2ZgwOHEU00PTQux3r/P/1bQ c2lQV4VuY6eT37CF79xnZsA= X-Google-Smtp-Source: ABdhPJwnbnl089PfsrvpvokDoexGwlHUvaiFWcLBnccsg7u15LM3+for4Cf+c0I1NoLoEZ/CfdV9tg== X-Received: by 2002:a63:c741:: with SMTP id v1mr37348049pgg.207.1618411832810; Wed, 14 Apr 2021 07:50:32 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:35:ebad:12c1:f579:e332]) by smtp.gmail.com with ESMTPSA id w67sm17732522pgb.87.2021.04.14.07.50.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Apr 2021 07:50:32 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, wei.liu@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, akpm@linux-foundation.org, gregkh@linuxfoundation.org, konrad.wilk@oracle.com, hch@lst.de, m.szyprowski@samsung.com, robin.murphy@arm.com, joro@8bytes.org, will@kernel.org, davem@davemloft.net, kuba@kernel.org, jejb@linux.ibm.com, martin.petersen@oracle.com Cc: Tianyu Lan , iommu@lists.linux-foundation.org, linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-scsi@vger.kernel.org, netdev@vger.kernel.org, vkuznets@redhat.com, thomas.lendacky@amd.com, brijesh.singh@amd.com, sunilmut@microsoft.com Subject: [Resend RFC PATCH V2 09/12] swiotlb: Add bounce buffer remap address setting function Date: Wed, 14 Apr 2021 10:49:42 -0400 Message-Id: <20210414144945.3460554-10-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210414144945.3460554-1-ltykernel@gmail.com> References: <20210414144945.3460554-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tianyu Lan For Hyper-V isolation VM with AMD SEV SNP, the bounce buffer(shared memory) needs to be accessed via extra address space(e.g address above bit39). Hyper-V code may remap extra address space outside of swiotlb. swiotlb_bounce() needs to use remap virtual address to copy data from/to bounce buffer. Add new interface swiotlb_set_bounce_remap() to do that. Signed-off-by: Tianyu Lan --- include/linux/swiotlb.h | 5 +++++ kernel/dma/swiotlb.c | 13 ++++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h index d9c9fc9ca5d2..3ccd08116683 100644 --- a/include/linux/swiotlb.h +++ b/include/linux/swiotlb.h @@ -82,8 +82,13 @@ unsigned int swiotlb_max_segment(void); size_t swiotlb_max_mapping_size(struct device *dev); bool is_swiotlb_active(void); void __init swiotlb_adjust_size(unsigned long new_size); +void swiotlb_set_bounce_remap(unsigned char *vaddr); #else #define swiotlb_force SWIOTLB_NO_FORCE +static inline void swiotlb_set_bounce_remap(unsigned char *vaddr) +{ +} + static inline bool is_swiotlb_buffer(phys_addr_t paddr) { return false; diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index 7c42df6e6100..5fd2db6aa149 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -94,6 +94,7 @@ static unsigned int io_tlb_index; * not be bounced (unless SWIOTLB_FORCE is set). */ static unsigned int max_segment; +static unsigned char *swiotlb_bounce_remap_addr; /* * We need to save away the original address corresponding to a mapped entry @@ -421,6 +422,11 @@ void __init swiotlb_exit(void) swiotlb_cleanup(); } +void swiotlb_set_bounce_remap(unsigned char *vaddr) +{ + swiotlb_bounce_remap_addr = vaddr; +} + /* * Bounce: copy the swiotlb buffer from or back to the original dma location */ @@ -428,7 +434,12 @@ static void swiotlb_bounce(phys_addr_t orig_addr, phys_addr_t tlb_addr, size_t size, enum dma_data_direction dir) { unsigned long pfn = PFN_DOWN(orig_addr); - unsigned char *vaddr = phys_to_virt(tlb_addr); + unsigned char *vaddr; + + if (swiotlb_bounce_remap_addr) + vaddr = swiotlb_bounce_remap_addr + tlb_addr - io_tlb_start; + else + vaddr = phys_to_virt(tlb_addr); if (PageHighMem(pfn_to_page(pfn))) { /* The buffer does not have a mapping. Map it in and copy */ From patchwork Wed Apr 14 14:49:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 421573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4805AC433B4 for ; Wed, 14 Apr 2021 14:51:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 26DCF61074 for ; Wed, 14 Apr 2021 14:51:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352141AbhDNOwA (ORCPT ); Wed, 14 Apr 2021 10:52:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38918 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352003AbhDNOu5 (ORCPT ); Wed, 14 Apr 2021 10:50:57 -0400 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E7A99C061756; Wed, 14 Apr 2021 07:50:35 -0700 (PDT) Received: by mail-pg1-x532.google.com with SMTP id d10so14564546pgf.12; Wed, 14 Apr 2021 07:50:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FJbcZ8KnexDAAQ5C4Sah9FWkqLY6Pv3HeJbolDJY4aA=; b=pJqRunkA+uvhxmvikCt8JBi6ru5mMAHBSouNbpNF9rJ0/UZBeBZbNJBCihskZckNdz VF19al63x7JcKIwSaI+KgRMKfSCSAY1Rw9bm3SeZTFumtAQ1F6F2t95WHbTa7fd9i7Tn oYWyOXXwTN9+AehaDQOGu3IYPhNL0uJPAF/Y5xrfhILPXupgDCaiPCDkfxwVRVWqt8q+ v1FJRcKkKffDj4UMUcXDwJEin7yz4TBxBoW5ErGESAAdJKNwJ1ukgEYbn+s0z/MPExFU EV7Gn16e8uqLMsjL1/Ua5qlsNxwABA+5vNnnX0j44xHWZEyyVt04dkbZk7p6QtVwr46f qTdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FJbcZ8KnexDAAQ5C4Sah9FWkqLY6Pv3HeJbolDJY4aA=; b=GvNpKmeYZCewxMj0gRCDXVdOYeh3MQ+MeNV8pKu3jhgw/kVZ8J2zvbxrBbvyksOm0S 3vOKTdwaaNX6Ik2Sg4yR2auW25wWkQplZqArGOtHuaCTL5+D/tZV9K9ZGBpLNZcWL90P ymxJ6i7rczWFuvBk9gGbKNTBx5OuHEpSlFHU+C/hLqP122iLaMDoIyLxI6z+YlU2Zk7C u19mOOuaSQqqzp0brUhBt2bviw7ZoMg2X4gaotqHDKrtrKA1fcCGeE8FjW9D4YPgTi7O nym2F5zriPEz0mJC8ODj7X1sd4dl2yyc8/eivA+HembqYyUBNs7nzGTGMEakh0C0wLIY 3YkA== X-Gm-Message-State: AOAM533X0H9AFVC1D6IT0Dl2YDG0eK7eSMQCSeObnhKoiAIrecBnPlgF VXjh1zdDMqm1xfd8bghS/j0= X-Google-Smtp-Source: ABdhPJxHAv83JdLDizjV20z7u62dRC/ZGAyfJ/XHZQUwINjnLav8NcwqNU6XQrZbUuAS+bM3Dz+Utw== X-Received: by 2002:a63:1646:: with SMTP id 6mr39049996pgw.321.1618411835543; Wed, 14 Apr 2021 07:50:35 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:35:ebad:12c1:f579:e332]) by smtp.gmail.com with ESMTPSA id w67sm17732522pgb.87.2021.04.14.07.50.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Apr 2021 07:50:35 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, wei.liu@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, akpm@linux-foundation.org, gregkh@linuxfoundation.org, konrad.wilk@oracle.com, hch@lst.de, m.szyprowski@samsung.com, robin.murphy@arm.com, joro@8bytes.org, will@kernel.org, davem@davemloft.net, kuba@kernel.org, jejb@linux.ibm.com, martin.petersen@oracle.com Cc: Tianyu Lan , iommu@lists.linux-foundation.org, linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-scsi@vger.kernel.org, netdev@vger.kernel.org, vkuznets@redhat.com, thomas.lendacky@amd.com, brijesh.singh@amd.com, sunilmut@microsoft.com Subject: [Resend RFC PATCH V2 12/12] HV/Storvsc: Add Isolation VM support for storvsc driver Date: Wed, 14 Apr 2021 10:49:45 -0400 Message-Id: <20210414144945.3460554-13-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210414144945.3460554-1-ltykernel@gmail.com> References: <20210414144945.3460554-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tianyu Lan In Isolation VM, all shared memory with host needs to mark visible to host via hvcall. vmbus_establish_gpadl() has already done it for netvsc rx/tx ring buffer. The page buffer used by vmbus_sendpacket_ mpb_desc() still need to handle. Use DMA API to map/umap these memory during sending/receiving packet and Hyper-V DMA ops callback will use swiotlb fucntion to allocate bounce buffer and copy data from/to bounce buffer. Signed-off-by: Tianyu Lan --- drivers/scsi/storvsc_drv.c | 67 +++++++++++++++++++++++++++++++++++++- 1 file changed, 66 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c index 2e4fa77445fd..d271578b1811 100644 --- a/drivers/scsi/storvsc_drv.c +++ b/drivers/scsi/storvsc_drv.c @@ -21,6 +21,8 @@ #include #include #include +#include +#include #include #include #include @@ -414,6 +416,11 @@ static void storvsc_on_channel_callback(void *context); #define STORVSC_IDE_MAX_TARGETS 1 #define STORVSC_IDE_MAX_CHANNELS 1 +struct dma_range { + dma_addr_t dma; + u32 mapping_size; +}; + struct storvsc_cmd_request { struct scsi_cmnd *cmd; @@ -427,6 +434,8 @@ struct storvsc_cmd_request { u32 payload_sz; struct vstor_packet vstor_packet; + u32 hvpg_count; + struct dma_range *dma_range; }; @@ -1236,6 +1245,7 @@ static void storvsc_on_channel_callback(void *context) const struct vmpacket_descriptor *desc; struct hv_device *device; struct storvsc_device *stor_device; + int i; if (channel->primary_channel != NULL) device = channel->primary_channel->device_obj; @@ -1249,6 +1259,8 @@ static void storvsc_on_channel_callback(void *context) foreach_vmbus_pkt(desc, channel) { void *packet = hv_pkt_data(desc); struct storvsc_cmd_request *request; + enum dma_data_direction dir; + u32 attrs; u64 cmd_rqst; cmd_rqst = vmbus_request_addr(&channel->requestor, @@ -1261,6 +1273,22 @@ static void storvsc_on_channel_callback(void *context) request = (struct storvsc_cmd_request *)(unsigned long)cmd_rqst; + if (request->vstor_packet.vm_srb.data_in == READ_TYPE) + dir = DMA_FROM_DEVICE; + else + dir = DMA_TO_DEVICE; + + if (request->dma_range) { + for (i = 0; i < request->hvpg_count; i++) + dma_unmap_page_attrs(&device->device, + request->dma_range[i].dma, + request->dma_range[i].mapping_size, + request->vstor_packet.vm_srb.data_in + == READ_TYPE ? + DMA_FROM_DEVICE : DMA_TO_DEVICE, attrs); + kfree(request->dma_range); + } + if (request == &stor_device->init_request || request == &stor_device->reset_request) { memcpy(&request->vstor_packet, packet, @@ -1682,8 +1710,10 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd) struct vmscsi_request *vm_srb; struct scatterlist *cur_sgl; struct vmbus_packet_mpb_array *payload; + enum dma_data_direction dir; u32 payload_sz; u32 length; + u32 attrs; if (vmstor_proto_version <= VMSTOR_PROTO_VERSION_WIN8) { /* @@ -1722,14 +1752,17 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd) case DMA_TO_DEVICE: vm_srb->data_in = WRITE_TYPE; vm_srb->win8_extension.srb_flags |= SRB_FLAGS_DATA_OUT; + dir = DMA_TO_DEVICE; break; case DMA_FROM_DEVICE: vm_srb->data_in = READ_TYPE; vm_srb->win8_extension.srb_flags |= SRB_FLAGS_DATA_IN; + dir = DMA_FROM_DEVICE; break; case DMA_NONE: vm_srb->data_in = UNKNOWN_TYPE; vm_srb->win8_extension.srb_flags |= SRB_FLAGS_NO_DATA_TRANSFER; + dir = DMA_NONE; break; default: /* @@ -1786,6 +1819,12 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd) hvpgoff = sgl->offset >> HV_HYP_PAGE_SHIFT; cur_sgl = sgl; + + cmd_request->dma_range = kzalloc(sizeof(struct dma_range) * hvpg_count, + GFP_ATOMIC); + if (!cmd_request->dma_range) + return -ENOMEM; + for (i = 0; i < hvpg_count; i++) { /* * 'i' is the index of hv pages in the payload and @@ -1805,6 +1844,8 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd) */ unsigned int hvpgoff_in_page = (i + hvpgoff) % NR_HV_HYP_PAGES_IN_PAGE; + dma_addr_t dma; + u32 size; /* * Two cases that we need to fetch a page: @@ -1817,8 +1858,28 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd) cur_sgl = sg_next(cur_sgl); } - payload->range.pfn_array[i] = hvpfn + hvpgoff_in_page; + size = min(HV_HYP_PAGE_SIZE - offset_in_hvpg, (unsigned long)length); + dma = dma_map_page_attrs(&dev->device, + pfn_to_page(hvpfn), + offset_in_hvpg, size, + scmnd->sc_data_direction, attrs); + if (dma_mapping_error(&dev->device, dma)) { + pr_warn("dma map error.\n"); + ret = -ENOMEM; + goto free_dma_range; + } + + if (offset_in_hvpg) { + payload->range.offset = dma & ~HV_HYP_PAGE_MASK; + offset_in_hvpg = 0; + } + + cmd_request->dma_range[i].dma = dma; + cmd_request->dma_range[i].mapping_size = size; + payload->range.pfn_array[i] = dma >> HV_HYP_PAGE_SHIFT; + length -= size; } + cmd_request->hvpg_count = hvpg_count; } cmd_request->payload = payload; @@ -1836,6 +1897,10 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd) } return 0; + +free_dma_range: + kfree(cmd_request->dma_range); + return ret; } static struct scsi_host_template scsi_driver = {