| Message ID | 1615233439-23346-1-git-send-email-mikelley@microsoft.com |
|---|---|
| Headers | show |
| Series | Enable Linux guests on Hyper-V on ARM64 | expand |
From: Michael Kelley <mikelley@microsoft.com> Sent: Monday, March 8, 2021 11:57 AM > > This series enables Linux guests running on Hyper-V on ARM64 > hardware. New ARM64-specific code in arch/arm64/hyperv initializes > Hyper-V, including its interrupts and hypercall mechanism. > Existing architecture independent drivers for Hyper-V's VMbus and > synthetic devices just work when built for ARM64. Hyper-V code is > built and included in the image and modules only if CONFIG_HYPERV > is enabled. ARM64 maintainers -- What are the prospects for getting your review and Ack on this patch set? We're wanting to get the Hyper-V support on ARM64 finally accepted upstream. Previous comments should be addressed in this revision, with perhaps a remaining discussion point around the alternate SMCCC hypercall interface in Patch 1 that makes use of changes in v1.2 of the SMCCC spec. There are several viable approaches that I've noted in the patch, depending on your preferences. Michael > > The seven patches are organized as follows: > > 1) Update include/linux/arm-smccc.h to provide an HVC wrapper > variant that returns results in other than X0 thru X3 > > 2) Add definitions and functions for making Hyper-V hypercalls > and getting/setting virtual processor registers provided by > Hyper-V > > 3) Add architecture specific definitions needed by the > architecture independent Hyper-V clocksource driver in > drivers/clocksource/hyperv_timer.c. Update the clocksource > driver to be initialized on ARM64. > > 4) Add functions needed by the arch independent VMbus driver > for reporting a panic to Hyper-V and as stubs for the kexec > and crash handlers. > > 5) Add Hyper-V initialization code and utility functions that > report Hyper-v status. > > 6) Export screen_info so it may be used by the Hyper-V frame buffer > driver built as a module. It is already exported for x86, > powerpc, and alpha architectures. > > 7) Make CONFIG_HYPERV selectable on ARM64 in addition to x86/x64. > > Hyper-V on ARM64 runs with a 4 Kbyte page size, but allows guests > with 4K/16K/64K page size. Linux guests with this ARM64 enablement > code work with all three supported ARM64 page sizes. > > The Hyper-V vPCI driver at drivers/pci/host/pci-hyperv.c has > x86/x64-specific code and is not being built for ARM64. Fixing > this driver to enable vPCI devices on ARM64 will be done later. > > In a few cases, terminology from the x86/x64 world has been carried > over into the ARM64 code ("MSR", "TSC"). Hyper-V still uses the > x86/x64 terminology and has not replaced it with something more > generic, so the code uses the Hyper-V terminology. This will be > fixed when Hyper-V updates the usage in the TLFS. > > This patch set is based on the hyperv-next branch of the code tree > https://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git/ > > Changes in v9: > * Added Patch 1 to enable making an SMCCC compliant hypercall > that returns results in other than registers X0 thru X3, per > version 1.2 and later of the SMCCC spec. > * Using the ability to return results in registers X6 and X7, > converted hv_get_vpreg_128() to use a "fast" hypercall that > passes inputs and outputs in registers, and in doing so eliminated > a lot of memory allocation complexity. > * Cleaned up some extra blank lines and use of spaces in aligning > local variables. [Sunil Muthuswamy] > * Based on discussion about future directions, reverted the > population of hv_vp_index array to use a cpuhp state instead > of a hypercall, which is like it was in v7 and earlier. > > Changes in v8: > * Removed a lot of code based on refactoring the boundary between > arch independent and arch dependent code for Hyper-V, per comments > from Arnd Bergmann. The removed code was either duplicated on > the x86 side, or has been folded into architecture independent > code as not really being architecture dependent. > * Added config dependency on !CONFIG_CPU_BIG_ENDIAN [Arnd Bergmann] > * Reworked the approach to Hyper-V initialization. The functionality > is the same, but is now structured like the Xen code with an early > init function called in setup_arch() and an early initcall to > finish the initialization. [Arnd Bergmann] > > Changes in v7: > * Separately upstreamed split of hyperv-tlfs.h into arch dependent > and independent versions. In this patch set, update the ARM64 > hyperv-tlfs.h to include architecture independent definitions. > This approach eliminates a lot of lines of otherwise duplicated > code on the ARM64 side. > * Break ARM64 mshyperv.h into smaller pieces. Have an initial > baseline, and add code along with patches for a particular > functional area. [Marc Zyngier] > * In mshyperv.h, use static inline functions instead of #defines > where possible. [Arnd Bergmann] > * Use VMbus INTID obtained from ACPI DSDT instead of hardcoding. > The STIMER INTID is still hardcoded because it is needed > before Linux has initialized the ACPI subsystem, so it can't > be obtained from the DSDT. Wedging it into the GTDT seems > dubious, so was not done. [Marc Zyngier] > * Update Hyper-V page size allocation functions to use > alloc_page() if PAGE_SIZE == HV_HYP_PAGE_SIZE [Arnd > Bergmann] > * Various other minor changes based on feedback and to rebase > to latest linux-next [Marc Zyngier and Arnd Bergmann] > > Changes in v6: > * Use SMCCC hypercall interface instead of direct invocation > of HVC instruction and the Hyper-V hypercall interface > [Marc Zyngier] > * Reimplemented functions to alloc/free Hyper-V size pages > using kmalloc/kfree since kmalloc now guarantees alignment of > power of 2 size allocations [Marc Zyngier] > * Export screen_info in arm64 architecture so it can be used > by the Hyper-V buffer driver built as a module > * Renamed source file arch/arm64/hyperv/hv_init.c to hv_core.c > to better reflect its content > * Fixed the bit position of certain feature flags presented by > Hyper-V to the guest. The bit positions on ARM64 don't match > the position on x86 like originally thought. > * Minor fixups to rebase to 5.6-rc5 linux-next > > Changes in v5: > * Minor fixups to rebase to 5.4-rc1 linux-next > > Changes in v4: > * Moved clock-related code into an architecture independent > Hyper-V clocksource driver that is already upstream. Clock > related code is removed from this patch set except for the > ARM64 specific interrupt handler. [Marc Zyngier] > * Separately upstreamed the split of mshyperv.h into arch independent > and arch dependent portions. The arch independent portion has been > removed from this patch set. > * Divided patch #2 of the series into multiple smaller patches > [Marc Zyngier] > * Changed a dozen or so smaller things based on feedback > [Marc Zyngier, Will Deacon] > * Added functions to alloc/free Hyper-V size pages for use by > drivers for Hyper-V synthetic devices when updated to not assume > guest page size and Hyper-v page size are the same > > Changes in v3: > * Added initialization of hv_vp_index array like was recently > added on x86 branch [KY Srinivasan] > * Changed Hyper-V ARM64 register symbols to be all uppercase > instead of mixed case [KY Srinivasan] > * Separated mshyperv.h into two files, one architecture > independent and one architecture dependent. After this code > is upstream, will make changes to the x86 code to use the > architecture independent file and remove duplication. And > once we have a multi-architecture Hyper-V TLFS, will do a > separate patch to split hyperv-tlfs.h in the same way. > [KY Srinivasan] > * Minor tweaks to rebase to latest linux-next code > > Changes in v2: > * Removed patch to implement slow_virt_to_phys() on ARM64. > Use of slow_virt_to_phys() in arch independent Hyper-V > drivers has been eliminated by commit 6ba34171bcbd > ("Drivers: hv: vmbus: Remove use of slow_virt_to_phys()") > * Minor tweaks to rebase to latest linux-next code > *** BLURB HERE *** > > Michael Kelley (7): > smccc: Add HVC call variant with result registers other than 0 thru 3 > arm64: hyperv: Add Hyper-V hypercall and register access utilities > arm64: hyperv: Add Hyper-V clocksource/clockevent support > arm64: hyperv: Add kexec and panic handlers > arm64: hyperv: Initialize hypervisor on boot > arm64: efi: Export screen_info > Drivers: hv: Enable Hyper-V code to be built on ARM64 > > MAINTAINERS | 3 + > arch/arm64/Kbuild | 1 + > arch/arm64/hyperv/Makefile | 2 + > arch/arm64/hyperv/hv_core.c | 178 +++++++++++++++++++++++++++++++++++ > arch/arm64/hyperv/mshyperv.c | 173 ++++++++++++++++++++++++++++++++++ > arch/arm64/include/asm/hyperv-tlfs.h | 69 ++++++++++++++ > arch/arm64/include/asm/mshyperv.h | 72 ++++++++++++++ > arch/arm64/kernel/efi.c | 1 + > arch/arm64/kernel/setup.c | 4 + > drivers/clocksource/hyperv_timer.c | 14 +++ > drivers/hv/Kconfig | 3 +- > include/linux/arm-smccc.h | 29 ++++-- > 12 files changed, 542 insertions(+), 7 deletions(-) > create mode 100644 arch/arm64/hyperv/Makefile > create mode 100644 arch/arm64/hyperv/hv_core.c > create mode 100644 arch/arm64/hyperv/mshyperv.c > create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h > create mode 100644 arch/arm64/include/asm/mshyperv.h > > -- > 1.8.3.1
From: Michael Kelley <mikelley@microsoft.com> > Sent: Wednesday, March 24, 2021 8:55 AM > To: will@kernel.org; catalin.marinas@arm.com; Mark Rutland <Mark.Rutland@arm.com>; > lorenzo.pieralisi@arm.com; sudeep.holla@arm.com > Cc: linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; linux- > hyperv@vger.kernel.org; linux-efi@vger.kernel.org; arnd@arndb.de; wei.liu@kernel.org; > ardb@kernel.org; daniel.lezcano@linaro.org; KY Srinivasan <kys@microsoft.com> > Subject: RE: [PATCH v9 0/7] Enable Linux guests on Hyper-V on ARM64 > > From: Michael Kelley <mikelley@microsoft.com> Sent: Monday, March 8, 2021 11:57 AM > > > > This series enables Linux guests running on Hyper-V on ARM64 > > hardware. New ARM64-specific code in arch/arm64/hyperv initializes > > Hyper-V, including its interrupts and hypercall mechanism. > > Existing architecture independent drivers for Hyper-V's VMbus and > > synthetic devices just work when built for ARM64. Hyper-V code is > > built and included in the image and modules only if CONFIG_HYPERV > > is enabled. > > ARM64 maintainers -- > > What are the prospects for getting your review and Ack on this patch set? > We're wanting to get the Hyper-V support on ARM64 finally accepted upstream. > Previous comments should be addressed in this revision, with perhaps a > remaining discussion point around the alternate SMCCC hypercall interface > in Patch 1 that makes use of changes in v1.2 of the SMCCC spec. There are > several viable approaches that I've noted in the patch, depending on > your preferences. > > Michael Thanks, Mark, for jumping in on the SMCCC hypercall interface. But I'm still looking for feedback or ACKs on the other patches in the series. There's only one place in Patch 2 of the series that needs the SMCCC v1.2 interface, and I'd like to be able to respond to any remaining issues with the other patches while the SMCCC details are finished up. Michael
This series enables Linux guests running on Hyper-V on ARM64 hardware. New ARM64-specific code in arch/arm64/hyperv initializes Hyper-V, including its interrupts and hypercall mechanism. Existing architecture independent drivers for Hyper-V's VMbus and synthetic devices just work when built for ARM64. Hyper-V code is built and included in the image and modules only if CONFIG_HYPERV is enabled. The seven patches are organized as follows: 1) Update include/linux/arm-smccc.h to provide an HVC wrapper variant that returns results in other than X0 thru X3 2) Add definitions and functions for making Hyper-V hypercalls and getting/setting virtual processor registers provided by Hyper-V 3) Add architecture specific definitions needed by the architecture independent Hyper-V clocksource driver in drivers/clocksource/hyperv_timer.c. Update the clocksource driver to be initialized on ARM64. 4) Add functions needed by the arch independent VMbus driver for reporting a panic to Hyper-V and as stubs for the kexec and crash handlers. 5) Add Hyper-V initialization code and utility functions that report Hyper-v status. 6) Export screen_info so it may be used by the Hyper-V frame buffer driver built as a module. It is already exported for x86, powerpc, and alpha architectures. 7) Make CONFIG_HYPERV selectable on ARM64 in addition to x86/x64. Hyper-V on ARM64 runs with a 4 Kbyte page size, but allows guests with 4K/16K/64K page size. Linux guests with this ARM64 enablement code work with all three supported ARM64 page sizes. The Hyper-V vPCI driver at drivers/pci/host/pci-hyperv.c has x86/x64-specific code and is not being built for ARM64. Fixing this driver to enable vPCI devices on ARM64 will be done later. In a few cases, terminology from the x86/x64 world has been carried over into the ARM64 code ("MSR", "TSC"). Hyper-V still uses the x86/x64 terminology and has not replaced it with something more generic, so the code uses the Hyper-V terminology. This will be fixed when Hyper-V updates the usage in the TLFS. This patch set is based on the hyperv-next branch of the code tree https://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git/ Changes in v9: * Added Patch 1 to enable making an SMCCC compliant hypercall that returns results in other than registers X0 thru X3, per version 1.2 and later of the SMCCC spec. * Using the ability to return results in registers X6 and X7, converted hv_get_vpreg_128() to use a "fast" hypercall that passes inputs and outputs in registers, and in doing so eliminated a lot of memory allocation complexity. * Cleaned up some extra blank lines and use of spaces in aligning local variables. [Sunil Muthuswamy] * Based on discussion about future directions, reverted the population of hv_vp_index array to use a cpuhp state instead of a hypercall, which is like it was in v7 and earlier. Changes in v8: * Removed a lot of code based on refactoring the boundary between arch independent and arch dependent code for Hyper-V, per comments from Arnd Bergmann. The removed code was either duplicated on the x86 side, or has been folded into architecture independent code as not really being architecture dependent. * Added config dependency on !CONFIG_CPU_BIG_ENDIAN [Arnd Bergmann] * Reworked the approach to Hyper-V initialization. The functionality is the same, but is now structured like the Xen code with an early init function called in setup_arch() and an early initcall to finish the initialization. [Arnd Bergmann] Changes in v7: * Separately upstreamed split of hyperv-tlfs.h into arch dependent and independent versions. In this patch set, update the ARM64 hyperv-tlfs.h to include architecture independent definitions. This approach eliminates a lot of lines of otherwise duplicated code on the ARM64 side. * Break ARM64 mshyperv.h into smaller pieces. Have an initial baseline, and add code along with patches for a particular functional area. [Marc Zyngier] * In mshyperv.h, use static inline functions instead of #defines where possible. [Arnd Bergmann] * Use VMbus INTID obtained from ACPI DSDT instead of hardcoding. The STIMER INTID is still hardcoded because it is needed before Linux has initialized the ACPI subsystem, so it can't be obtained from the DSDT. Wedging it into the GTDT seems dubious, so was not done. [Marc Zyngier] * Update Hyper-V page size allocation functions to use alloc_page() if PAGE_SIZE == HV_HYP_PAGE_SIZE [Arnd Bergmann] * Various other minor changes based on feedback and to rebase to latest linux-next [Marc Zyngier and Arnd Bergmann] Changes in v6: * Use SMCCC hypercall interface instead of direct invocation of HVC instruction and the Hyper-V hypercall interface [Marc Zyngier] * Reimplemented functions to alloc/free Hyper-V size pages using kmalloc/kfree since kmalloc now guarantees alignment of power of 2 size allocations [Marc Zyngier] * Export screen_info in arm64 architecture so it can be used by the Hyper-V buffer driver built as a module * Renamed source file arch/arm64/hyperv/hv_init.c to hv_core.c to better reflect its content * Fixed the bit position of certain feature flags presented by Hyper-V to the guest. The bit positions on ARM64 don't match the position on x86 like originally thought. * Minor fixups to rebase to 5.6-rc5 linux-next Changes in v5: * Minor fixups to rebase to 5.4-rc1 linux-next Changes in v4: * Moved clock-related code into an architecture independent Hyper-V clocksource driver that is already upstream. Clock related code is removed from this patch set except for the ARM64 specific interrupt handler. [Marc Zyngier] * Separately upstreamed the split of mshyperv.h into arch independent and arch dependent portions. The arch independent portion has been removed from this patch set. * Divided patch #2 of the series into multiple smaller patches [Marc Zyngier] * Changed a dozen or so smaller things based on feedback [Marc Zyngier, Will Deacon] * Added functions to alloc/free Hyper-V size pages for use by drivers for Hyper-V synthetic devices when updated to not assume guest page size and Hyper-v page size are the same Changes in v3: * Added initialization of hv_vp_index array like was recently added on x86 branch [KY Srinivasan] * Changed Hyper-V ARM64 register symbols to be all uppercase instead of mixed case [KY Srinivasan] * Separated mshyperv.h into two files, one architecture independent and one architecture dependent. After this code is upstream, will make changes to the x86 code to use the architecture independent file and remove duplication. And once we have a multi-architecture Hyper-V TLFS, will do a separate patch to split hyperv-tlfs.h in the same way. [KY Srinivasan] * Minor tweaks to rebase to latest linux-next code Changes in v2: * Removed patch to implement slow_virt_to_phys() on ARM64. Use of slow_virt_to_phys() in arch independent Hyper-V drivers has been eliminated by commit 6ba34171bcbd ("Drivers: hv: vmbus: Remove use of slow_virt_to_phys()") * Minor tweaks to rebase to latest linux-next code *** BLURB HERE *** Michael Kelley (7): smccc: Add HVC call variant with result registers other than 0 thru 3 arm64: hyperv: Add Hyper-V hypercall and register access utilities arm64: hyperv: Add Hyper-V clocksource/clockevent support arm64: hyperv: Add kexec and panic handlers arm64: hyperv: Initialize hypervisor on boot arm64: efi: Export screen_info Drivers: hv: Enable Hyper-V code to be built on ARM64 MAINTAINERS | 3 + arch/arm64/Kbuild | 1 + arch/arm64/hyperv/Makefile | 2 + arch/arm64/hyperv/hv_core.c | 178 +++++++++++++++++++++++++++++++++++ arch/arm64/hyperv/mshyperv.c | 173 ++++++++++++++++++++++++++++++++++ arch/arm64/include/asm/hyperv-tlfs.h | 69 ++++++++++++++ arch/arm64/include/asm/mshyperv.h | 72 ++++++++++++++ arch/arm64/kernel/efi.c | 1 + arch/arm64/kernel/setup.c | 4 + drivers/clocksource/hyperv_timer.c | 14 +++ drivers/hv/Kconfig | 3 +- include/linux/arm-smccc.h | 29 ++++-- 12 files changed, 542 insertions(+), 7 deletions(-) create mode 100644 arch/arm64/hyperv/Makefile create mode 100644 arch/arm64/hyperv/hv_core.c create mode 100644 arch/arm64/hyperv/mshyperv.c create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h create mode 100644 arch/arm64/include/asm/mshyperv.h