| Message ID | 1620841067-46606-3-git-send-email-mikelley@microsoft.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Enable Linux guests on Hyper-V on ARM64 | expand |
On Wed, May 12, 2021 at 10:37:42AM -0700, Michael Kelley wrote: > hyperv-tlfs.h defines Hyper-V interfaces from the Hyper-V Top Level > Functional Spec (TLFS), and #includes the architecture-independent > part of hyperv-tlfs.h in include/asm-generic. The published TLFS > is distinctly oriented to x86/x64, so the ARM64-specific > hyperv-tlfs.h includes information for ARM64 that is not yet formally > published. The TLFS is available here: > > docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs > > mshyperv.h defines Linux-specific structures and routines for > interacting with Hyper-V on ARM64, and #includes the architecture- > independent part of mshyperv.h in include/asm-generic. > > Use these definitions to provide utility functions to make > Hyper-V hypercalls and to get and set Hyper-V provided > registers associated with a virtual processor. > > Signed-off-by: Michael Kelley <mikelley@microsoft.com> > Reviewed-by: Sunil Muthuswamy <sunilmut@microsoft.com> > --- > MAINTAINERS | 3 + > arch/arm64/Kbuild | 1 + > arch/arm64/hyperv/Makefile | 2 + > arch/arm64/hyperv/hv_core.c | 130 +++++++++++++++++++++++++++++++++++ > arch/arm64/include/asm/hyperv-tlfs.h | 69 +++++++++++++++++++ > arch/arm64/include/asm/mshyperv.h | 54 +++++++++++++++ > 6 files changed, 259 insertions(+) > create mode 100644 arch/arm64/hyperv/Makefile > create mode 100644 arch/arm64/hyperv/hv_core.c > create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h > create mode 100644 arch/arm64/include/asm/mshyperv.h > +/* > + * hv_do_hypercall- Invoke the specified hypercall > + */ > +u64 hv_do_hypercall(u64 control, void *input, void *output) > +{ > + struct arm_smccc_res res; > + u64 input_address; > + u64 output_address; > + > + input_address = input ? virt_to_phys(input) : 0; > + output_address = output ? virt_to_phys(output) : 0; I may have asked this before, but are `input` and `output` always linear map pointers, or can they ever be vmalloc pointers? Otherwise, this looks fine to me. Mark.
From: Mark Rutland <mark.rutland@arm.com> Sent: Friday, May 14, 2021 5:53 AM > > On Wed, May 12, 2021 at 10:37:42AM -0700, Michael Kelley wrote: > > hyperv-tlfs.h defines Hyper-V interfaces from the Hyper-V Top Level > > Functional Spec (TLFS), and #includes the architecture-independent > > part of hyperv-tlfs.h in include/asm-generic. The published TLFS > > is distinctly oriented to x86/x64, so the ARM64-specific > > hyperv-tlfs.h includes information for ARM64 that is not yet formally > > published. The TLFS is available here: > > > > docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs > > > > mshyperv.h defines Linux-specific structures and routines for > > interacting with Hyper-V on ARM64, and #includes the architecture- > > independent part of mshyperv.h in include/asm-generic. > > > > Use these definitions to provide utility functions to make > > Hyper-V hypercalls and to get and set Hyper-V provided > > registers associated with a virtual processor. > > > > Signed-off-by: Michael Kelley <mikelley@microsoft.com> > > Reviewed-by: Sunil Muthuswamy <sunilmut@microsoft.com> > > --- > > MAINTAINERS | 3 + > > arch/arm64/Kbuild | 1 + > > arch/arm64/hyperv/Makefile | 2 + > > arch/arm64/hyperv/hv_core.c | 130 +++++++++++++++++++++++++++++++++++ > > arch/arm64/include/asm/hyperv-tlfs.h | 69 +++++++++++++++++++ > > arch/arm64/include/asm/mshyperv.h | 54 +++++++++++++++ > > 6 files changed, 259 insertions(+) > > create mode 100644 arch/arm64/hyperv/Makefile > > create mode 100644 arch/arm64/hyperv/hv_core.c > > create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h > > create mode 100644 arch/arm64/include/asm/mshyperv.h > > > +/* > > + * hv_do_hypercall- Invoke the specified hypercall > > + */ > > +u64 hv_do_hypercall(u64 control, void *input, void *output) > > +{ > > + struct arm_smccc_res res; > > + u64 input_address; > > + u64 output_address; > > + > > + input_address = input ? virt_to_phys(input) : 0; > > + output_address = output ? virt_to_phys(output) : 0; > > I may have asked this before, but are `input` and `output` always linear > map pointers, or can they ever be vmalloc pointers? > > Otherwise, this looks fine to me. > The caller must ensure that hypercall arguments are aligned to 4 Kbytes, and no larger than 4 Kbytes, since that's the page size used by Hyper-V regardless of the guest page size. A per-CPU 4 Kbyte memory area (hyperv_pcpu_input_arg) meeting these requirements is pre-allocated that callers can use for this purpose. On the x86 side there's also a similar hyperv_pcpu_output_arg, but there are no use cases for it on ARM64. Michael
On Fri, May 14, 2021 at 03:14:41PM +0000, Michael Kelley wrote: > From: Mark Rutland <mark.rutland@arm.com> Sent: Friday, May 14, 2021 5:53 AM > > > > On Wed, May 12, 2021 at 10:37:42AM -0700, Michael Kelley wrote: > > > hyperv-tlfs.h defines Hyper-V interfaces from the Hyper-V Top Level > > > Functional Spec (TLFS), and #includes the architecture-independent > > > part of hyperv-tlfs.h in include/asm-generic. The published TLFS > > > is distinctly oriented to x86/x64, so the ARM64-specific > > > hyperv-tlfs.h includes information for ARM64 that is not yet formally > > > published. The TLFS is available here: > > > > > > docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs > > > > > > mshyperv.h defines Linux-specific structures and routines for > > > interacting with Hyper-V on ARM64, and #includes the architecture- > > > independent part of mshyperv.h in include/asm-generic. > > > > > > Use these definitions to provide utility functions to make > > > Hyper-V hypercalls and to get and set Hyper-V provided > > > registers associated with a virtual processor. > > > > > > Signed-off-by: Michael Kelley <mikelley@microsoft.com> > > > Reviewed-by: Sunil Muthuswamy <sunilmut@microsoft.com> > > > --- > > > MAINTAINERS | 3 + > > > arch/arm64/Kbuild | 1 + > > > arch/arm64/hyperv/Makefile | 2 + > > > arch/arm64/hyperv/hv_core.c | 130 +++++++++++++++++++++++++++++++++++ > > > arch/arm64/include/asm/hyperv-tlfs.h | 69 +++++++++++++++++++ > > > arch/arm64/include/asm/mshyperv.h | 54 +++++++++++++++ > > > 6 files changed, 259 insertions(+) > > > create mode 100644 arch/arm64/hyperv/Makefile > > > create mode 100644 arch/arm64/hyperv/hv_core.c > > > create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h > > > create mode 100644 arch/arm64/include/asm/mshyperv.h > > > > > +/* > > > + * hv_do_hypercall- Invoke the specified hypercall > > > + */ > > > +u64 hv_do_hypercall(u64 control, void *input, void *output) > > > +{ > > > + struct arm_smccc_res res; > > > + u64 input_address; > > > + u64 output_address; > > > + > > > + input_address = input ? virt_to_phys(input) : 0; > > > + output_address = output ? virt_to_phys(output) : 0; > > > > I may have asked this before, but are `input` and `output` always linear > > map pointers, or can they ever be vmalloc pointers? > > > > Otherwise, this looks fine to me. > > The caller must ensure that hypercall arguments are aligned to > 4 Kbytes, and no larger than 4 Kbytes, since that's the page size > used by Hyper-V regardless of the guest page size. A per-CPU > 4 Kbyte memory area (hyperv_pcpu_input_arg) meeting these > requirements is pre-allocated that callers can use for this purpose. What I was trying to find out was how that was allocated, as vmalloc()'d pointers aren't legitimate to pass to virt_to_phys(). From scanning ahead to patch 5, I see that memory comes from kmalloc(), and so it is legitimate to use virt_to_phys(). I see; and from patch 5 I see that memory come from kmalloc(), and will therefore be part of the linear map, and so virt_to_phys() is legitimate. What I was asking here was how that memory was allocated. So long as those are the only buffers used, this looks fine to me. Thanks, Mark.
From: Mark Rutland <mark.rutland@arm.com> Sent: Monday, May 17, 2021 4:45 AM > > On Fri, May 14, 2021 at 03:14:41PM +0000, Michael Kelley wrote: > > From: Mark Rutland <mark.rutland@arm.com> Sent: Friday, May 14, 2021 5:53 AM > > > > > > On Wed, May 12, 2021 at 10:37:42AM -0700, Michael Kelley wrote: > > > > hyperv-tlfs.h defines Hyper-V interfaces from the Hyper-V Top Level > > > > Functional Spec (TLFS), and #includes the architecture-independent > > > > part of hyperv-tlfs.h in include/asm-generic. The published TLFS > > > > is distinctly oriented to x86/x64, so the ARM64-specific > > > > hyperv-tlfs.h includes information for ARM64 that is not yet formally > > > > published. The TLFS is available here: > > > > > > > > docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs > > > > > > > > mshyperv.h defines Linux-specific structures and routines for > > > > interacting with Hyper-V on ARM64, and #includes the architecture- > > > > independent part of mshyperv.h in include/asm-generic. > > > > > > > > Use these definitions to provide utility functions to make > > > > Hyper-V hypercalls and to get and set Hyper-V provided > > > > registers associated with a virtual processor. > > > > > > > > Signed-off-by: Michael Kelley <mikelley@microsoft.com> > > > > Reviewed-by: Sunil Muthuswamy <sunilmut@microsoft.com> > > > > --- > > > > MAINTAINERS | 3 + > > > > arch/arm64/Kbuild | 1 + > > > > arch/arm64/hyperv/Makefile | 2 + > > > > arch/arm64/hyperv/hv_core.c | 130 > +++++++++++++++++++++++++++++++++++ > > > > arch/arm64/include/asm/hyperv-tlfs.h | 69 +++++++++++++++++++ > > > > arch/arm64/include/asm/mshyperv.h | 54 +++++++++++++++ > > > > 6 files changed, 259 insertions(+) > > > > create mode 100644 arch/arm64/hyperv/Makefile > > > > create mode 100644 arch/arm64/hyperv/hv_core.c > > > > create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h > > > > create mode 100644 arch/arm64/include/asm/mshyperv.h > > > > > > > +/* > > > > + * hv_do_hypercall- Invoke the specified hypercall > > > > + */ > > > > +u64 hv_do_hypercall(u64 control, void *input, void *output) > > > > +{ > > > > + struct arm_smccc_res res; > > > > + u64 input_address; > > > > + u64 output_address; > > > > + > > > > + input_address = input ? virt_to_phys(input) : 0; > > > > + output_address = output ? virt_to_phys(output) : 0; > > > > > > I may have asked this before, but are `input` and `output` always linear > > > map pointers, or can they ever be vmalloc pointers? > > > > > > Otherwise, this looks fine to me. > > > > The caller must ensure that hypercall arguments are aligned to > > 4 Kbytes, and no larger than 4 Kbytes, since that's the page size > > used by Hyper-V regardless of the guest page size. A per-CPU > > 4 Kbyte memory area (hyperv_pcpu_input_arg) meeting these > > requirements is pre-allocated that callers can use for this purpose. > > What I was trying to find out was how that was allocated, as vmalloc()'d > pointers aren't legitimate to pass to virt_to_phys(). > > From scanning ahead to patch 5, I see that memory comes from kmalloc(), > and so it is legitimate to use virt_to_phys(). > > > I see; and from patch 5 I see that memory come from kmalloc(), and will > therefore be part of the linear map, and so virt_to_phys() is > legitimate. > > What I was asking here was how that memory was allocated. So long as > those are the only buffers used, this looks fine to me. > There is no vmalloc() or stack local memory used as arguments to a hypercall. Call sites know about this requirement along with the requirement of being aligned to 4 Kbytes and no larger than 4 Kbytes. Michael
diff --git a/MAINTAINERS b/MAINTAINERS index bd7aff0c..6c650cb 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8437,6 +8437,9 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git F: Documentation/ABI/stable/sysfs-bus-vmbus F: Documentation/ABI/testing/debugfs-hyperv F: Documentation/networking/device_drivers/ethernet/microsoft/netvsc.rst +F: arch/arm64/hyperv +F: arch/arm64/include/asm/hyperv-tlfs.h +F: arch/arm64/include/asm/mshyperv.h F: arch/x86/hyperv F: arch/x86/include/asm/hyperv-tlfs.h F: arch/x86/include/asm/mshyperv.h diff --git a/arch/arm64/Kbuild b/arch/arm64/Kbuild index d646582..7a37608 100644 --- a/arch/arm64/Kbuild +++ b/arch/arm64/Kbuild @@ -3,4 +3,5 @@ obj-y += kernel/ mm/ obj-$(CONFIG_NET) += net/ obj-$(CONFIG_KVM) += kvm/ obj-$(CONFIG_XEN) += xen/ +obj-$(subst m,y,$(CONFIG_HYPERV)) += hyperv/ obj-$(CONFIG_CRYPTO) += crypto/ diff --git a/arch/arm64/hyperv/Makefile b/arch/arm64/hyperv/Makefile new file mode 100644 index 0000000..1697d30 --- /dev/null +++ b/arch/arm64/hyperv/Makefile @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-y := hv_core.o diff --git a/arch/arm64/hyperv/hv_core.c b/arch/arm64/hyperv/hv_core.c new file mode 100644 index 0000000..34004a5 --- /dev/null +++ b/arch/arm64/hyperv/hv_core.c @@ -0,0 +1,130 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Low level utility routines for interacting with Hyper-V. + * + * Copyright (C) 2021, Microsoft, Inc. + * + * Author : Michael Kelley <mikelley@microsoft.com> + */ + +#include <linux/types.h> +#include <linux/export.h> +#include <linux/mm.h> +#include <linux/hyperv.h> +#include <linux/arm-smccc.h> +#include <linux/module.h> +#include <asm-generic/bug.h> +#include <asm/hyperv-tlfs.h> +#include <asm/mshyperv.h> + +/* + * hv_do_hypercall- Invoke the specified hypercall + */ +u64 hv_do_hypercall(u64 control, void *input, void *output) +{ + struct arm_smccc_res res; + u64 input_address; + u64 output_address; + + input_address = input ? virt_to_phys(input) : 0; + output_address = output ? virt_to_phys(output) : 0; + + arm_smccc_1_1_hvc(HV_FUNC_ID, control, + input_address, output_address, &res); + return res.a0; +} +EXPORT_SYMBOL_GPL(hv_do_hypercall); + +/* + * hv_do_fast_hypercall8 -- Invoke the specified hypercall + * with arguments in registers instead of physical memory. + * Avoids the overhead of virt_to_phys for simple hypercalls. + */ + +u64 hv_do_fast_hypercall8(u16 code, u64 input) +{ + struct arm_smccc_res res; + u64 control; + + control = (u64)code | HV_HYPERCALL_FAST_BIT; + + arm_smccc_1_1_hvc(HV_FUNC_ID, control, input, &res); + return res.a0; +} +EXPORT_SYMBOL_GPL(hv_do_fast_hypercall8); + +/* + * Set a single VP register to a 64-bit value. + */ +void hv_set_vpreg(u32 msr, u64 value) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc( + HV_FUNC_ID, + HVCALL_SET_VP_REGISTERS | HV_HYPERCALL_FAST_BIT | + HV_HYPERCALL_REP_COMP_1, + HV_PARTITION_ID_SELF, + HV_VP_INDEX_SELF, + msr, + 0, + value, + 0, + &res); + + /* + * Something is fundamentally broken in the hypervisor if + * setting a VP register fails. There's really no way to + * continue as a guest VM, so panic. + */ + BUG_ON(!hv_result_success(res.a0)); +} +EXPORT_SYMBOL_GPL(hv_set_vpreg); + +/* + * Get the value of a single VP register. One version + * returns just 64 bits and another returns the full 128 bits. + * The two versions are separate to avoid complicating the + * calling sequence for the more frequently used 64 bit version. + */ + +void hv_get_vpreg_128(u32 msr, struct hv_get_vp_registers_output *result) +{ + struct arm_smccc_1_2_regs args; + struct arm_smccc_1_2_regs res; + + args.a0 = HV_FUNC_ID; + args.a1 = HVCALL_GET_VP_REGISTERS | HV_HYPERCALL_FAST_BIT | + HV_HYPERCALL_REP_COMP_1; + args.a2 = HV_PARTITION_ID_SELF; + args.a3 = HV_VP_INDEX_SELF; + args.a4 = msr; + + /* + * Use the SMCCC 1.2 interface because the results are in registers + * beyond X0-X3. + */ + arm_smccc_1_2_hvc(&args, &res); + + /* + * Something is fundamentally broken in the hypervisor if + * getting a VP register fails. There's really no way to + * continue as a guest VM, so panic. + */ + BUG_ON(!hv_result_success(res.a0)); + + result->as64.low = res.a6; + result->as64.high = res.a7; +} +EXPORT_SYMBOL_GPL(hv_get_vpreg_128); + +u64 hv_get_vpreg(u32 msr) +{ + struct hv_get_vp_registers_output output; + + hv_get_vpreg_128(msr, &output); + + return output.as64.low; +} +EXPORT_SYMBOL_GPL(hv_get_vpreg); diff --git a/arch/arm64/include/asm/hyperv-tlfs.h b/arch/arm64/include/asm/hyperv-tlfs.h new file mode 100644 index 0000000..4d964a7 --- /dev/null +++ b/arch/arm64/include/asm/hyperv-tlfs.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * This file contains definitions from the Hyper-V Hypervisor Top-Level + * Functional Specification (TLFS): + * https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs + * + * Copyright (C) 2021, Microsoft, Inc. + * + * Author : Michael Kelley <mikelley@microsoft.com> + */ + +#ifndef _ASM_HYPERV_TLFS_H +#define _ASM_HYPERV_TLFS_H + +#include <linux/types.h> + +/* + * All data structures defined in the TLFS that are shared between Hyper-V + * and a guest VM use Little Endian byte ordering. This matches the default + * byte ordering of Linux running on ARM64, so no special handling is required. + */ + +/* + * These Hyper-V registers provide information equivalent to the CPUID + * instruction on x86/x64. + */ +#define HV_REGISTER_HYPERVISOR_VERSION 0x00000100 /*CPUID 0x40000002 */ +#define HV_REGISTER_FEATURES 0x00000200 /*CPUID 0x40000003 */ +#define HV_REGISTER_ENLIGHTENMENTS 0x00000201 /*CPUID 0x40000004 */ + +/* + * Group C Features. See the asm-generic version of hyperv-tlfs.h + * for a description of Feature Groups. + */ + +/* Crash MSRs available */ +#define HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE BIT(8) + +/* STIMER direct mode is available */ +#define HV_STIMER_DIRECT_MODE_AVAILABLE BIT(13) + +/* + * Synthetic register definitions equivalent to MSRs on x86/x64 + */ +#define HV_REGISTER_CRASH_P0 0x00000210 +#define HV_REGISTER_CRASH_P1 0x00000211 +#define HV_REGISTER_CRASH_P2 0x00000212 +#define HV_REGISTER_CRASH_P3 0x00000213 +#define HV_REGISTER_CRASH_P4 0x00000214 +#define HV_REGISTER_CRASH_CTL 0x00000215 + +#define HV_REGISTER_GUEST_OSID 0x00090002 +#define HV_REGISTER_VP_INDEX 0x00090003 +#define HV_REGISTER_TIME_REF_COUNT 0x00090004 +#define HV_REGISTER_REFERENCE_TSC 0x00090017 + +#define HV_REGISTER_SINT0 0x000A0000 +#define HV_REGISTER_SCONTROL 0x000A0010 +#define HV_REGISTER_SIEFP 0x000A0012 +#define HV_REGISTER_SIMP 0x000A0013 +#define HV_REGISTER_EOM 0x000A0014 + +#define HV_REGISTER_STIMER0_CONFIG 0x000B0000 +#define HV_REGISTER_STIMER0_COUNT 0x000B0001 + +#include <asm-generic/hyperv-tlfs.h> + +#endif diff --git a/arch/arm64/include/asm/mshyperv.h b/arch/arm64/include/asm/mshyperv.h new file mode 100644 index 0000000..c448704 --- /dev/null +++ b/arch/arm64/include/asm/mshyperv.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * Linux-specific definitions for managing interactions with Microsoft's + * Hyper-V hypervisor. The definitions in this file are specific to + * the ARM64 architecture. See include/asm-generic/mshyperv.h for + * definitions are that architecture independent. + * + * Definitions that are specified in the Hyper-V Top Level Functional + * Spec (TLFS) should not go in this file, but should instead go in + * hyperv-tlfs.h. + * + * Copyright (C) 2021, Microsoft, Inc. + * + * Author : Michael Kelley <mikelley@microsoft.com> + */ + +#ifndef _ASM_MSHYPERV_H +#define _ASM_MSHYPERV_H + +#include <linux/types.h> +#include <linux/arm-smccc.h> +#include <asm/hyperv-tlfs.h> + +/* + * Declare calls to get and set Hyper-V VP register values on ARM64, which + * requires a hypercall. + */ + +extern void hv_set_vpreg(u32 reg, u64 value); +extern u64 hv_get_vpreg(u32 reg); +extern void hv_get_vpreg_128(u32 reg, struct hv_get_vp_registers_output *result); + +static inline void hv_set_register(unsigned int reg, u64 value) +{ + hv_set_vpreg(reg, value); +} + +static inline u64 hv_get_register(unsigned int reg) +{ + return hv_get_vpreg(reg); +} + +/* SMCCC hypercall parameters */ +#define HV_SMCCC_FUNC_NUMBER 1 +#define HV_FUNC_ID ARM_SMCCC_CALL_VAL( \ + ARM_SMCCC_STD_CALL, \ + ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_VENDOR_HYP, \ + HV_SMCCC_FUNC_NUMBER) + +#include <asm-generic/mshyperv.h> + +#endif