| Message ID | 20231127052024.435743-1-gustavo.romero@linaro.org |
|---|---|
| Headers | show |
| Series | Add ivshmem-flat device | expand |
Gustavo Romero <gustavo.romero@linaro.org> writes: > This patchset introduces a new device, ivshmem-flat, which is similar to the > current ivshmem device but does not require a PCI bus. It implements the ivshmem > status and control registers as MMRs and the shared memory as a directly > accessible memory region in the VM memory layout. It's meant to be used on > machines like those with Cortex-M MCUs, which usually lack a PCI bus, e.g., > lm3s6965evb and mps2-an385. Additionally, it has the benefit of requiring a tiny > 'device driver,' which is helpful on some RTOSes, like Zephyr, that run on > memory-constrained resource targets. > > The patchset includes a QTest for the ivshmem-flat device, however, it's also > possible to experiment with it in two ways: > > (a) using two Cortex-M VMs running Zephyr; or > (b) using one aarch64 VM running Linux with the ivshmem PCI device and another > arm (Cortex-M) VM running Zephyr with the new ivshmem-flat device. > > Please note that for running the ivshmem-flat QTests the following patch, which > is not committed to the tree yet, must be applied: > > https://lists.nongnu.org/archive/html/qemu-devel/2023-11/msg03176.html > > -- > > To experiment with (a), clone this Zephyr repo [0], set the Zephyr build > environment [1], and follow the instructions in the 'ivshmem' sample main.c [2]. > > [0] https://github.com/gromero/zephyr/tree/ivshmem > [1] https://docs.zephyrproject.org/latest/develop/getting_started/index.html > [2] https://github.com/gromero/zephyr/commit/73fbd481e352b25ae5483ba5048a2182b90b7f00#diff-16fa1f481a49b995d0d1a62da37b9f33033f5ee477035e73465e7208521ddbe0R9-R70 > > To experiment with (b): If you could put these blobs up on fileserver.linaro.org you could add an avocado test to compliment the qtest.
This patchset introduces a new device, ivshmem-flat, which is similar to the current ivshmem device but does not require a PCI bus. It implements the ivshmem status and control registers as MMRs and the shared memory as a directly accessible memory region in the VM memory layout. It's meant to be used on machines like those with Cortex-M MCUs, which usually lack a PCI bus, e.g., lm3s6965evb and mps2-an385. Additionally, it has the benefit of requiring a tiny 'device driver,' which is helpful on some RTOSes, like Zephyr, that run on memory-constrained resource targets. The patchset includes a QTest for the ivshmem-flat device, however, it's also possible to experiment with it in two ways: (a) using two Cortex-M VMs running Zephyr; or (b) using one aarch64 VM running Linux with the ivshmem PCI device and another arm (Cortex-M) VM running Zephyr with the new ivshmem-flat device. Please note that for running the ivshmem-flat QTests the following patch, which is not committed to the tree yet, must be applied: https://lists.nongnu.org/archive/html/qemu-devel/2023-11/msg03176.html -- To experiment with (a), clone this Zephyr repo [0], set the Zephyr build environment [1], and follow the instructions in the 'ivshmem' sample main.c [2]. [0] https://github.com/gromero/zephyr/tree/ivshmem [1] https://docs.zephyrproject.org/latest/develop/getting_started/index.html [2] https://github.com/gromero/zephyr/commit/73fbd481e352b25ae5483ba5048a2182b90b7f00#diff-16fa1f481a49b995d0d1a62da37b9f33033f5ee477035e73465e7208521ddbe0R9-R70 To experiment with (b): $ git clone -b uio_ivshmem --single-branch https://github.com/gromero/linux.git $ cd linux $ wget https://people.linaro.org/~gustavo.romero/ivshmem/arm64_uio_ivshmem.config -O .config If in an x86_64 machine, cross compile the kernel, for instance: $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j 36 Install image in some directory, let's say, in ~/linux: $ mkdir ~/linux $ export INSTALL_PATH=~/linux $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j 36 install or, if you prefer, download the compiled image from: $ wget https://people.linaro.org/~gustavo.romero/ivshmem/vmlinuz-6.6.0-rc1-g28f3f88ee261 ... and then the rootfs: $ wget https://people.linaro.org/~gustavo.romero/ivshmem/rootfs.qcow2 Now, build QEMU with this patchset applied: $ mkdir build && cd build $ ../configure --target-list=arm-softmmu,aarch64-softmmu $ make -j 36 Start the ivshmem server: $ contrib/ivshmem-server/ivshmem-server -F Start the aarch64 VM + Linux + ivshmem PCI device: $ ./qemu-system-aarch64 -kernel ~/linux/vmlinuz-6.6.0-rc1-g28f3f88ee261 -append "root=/dev/vda initrd=/bin/bash console=ttyAMA0,115200" -drive file=~/linux/rootfs.qcow2,media=disk,if=virtio -machine virt-6.2 -nographic -accel tcg -cpu cortex-a57 -m 8192 -netdev bridge,id=hostnet0,br=virbr0,helper=/usr/lib/qemu/qemu-bridge-helper -device pcie-root-port,port=8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:d9:d1:12,bus=pci.1,addr=0x0 -device ivshmem-doorbell,vectors=2,chardev=ivshmem -chardev socket,path=/tmp/ivshmem_socket,id=ivshmem Log into the VM with user/pass: root/abc123 # dmesg | grep uio should show: [ 2.656367] uio_ivshmem 0000:00:02.0: ivshmem-mmr at 0x0000000010203000, size 0x0000000000001000 [ 2.656931] uio_ivshmem 0000:00:02.0: ivshmem-shmem at 0x0000008000000000, size 0x0000000000400000 [ 2.662554] uio_ivshmem 0000:00:02.0: module successfully loaded In another console, clone and build Zephyr image from 'uio_ivhsmem' branch: $ git clone -b uio_ivshmem --single-branch https://github.com/gromero/zephyr $ west -v --verbose build -p always -b qemu_cortex_m3 ./samples/uio_ivshmem/ ... and then start the arm VM + Zephyr image + ivshmem-flat device: $ ./qemu-system-arm -machine lm3s6965evb -nographic -net none -chardev socket,path=/tmp/ivshmem_socket,id=ivshmem_flat -device ivshmem-flat,chardev=ivshmem_flat,x-irq-qompath='/machine/unattached/device[1]/nvic/unnamed-gpio-in[0]',x-bus-qompath='/sysbus' -kernel ~/zephyrproject/zephyr/build/qemu_cortex_m3/uio_ivshmem/zephyr/zephyr.elf You should see something like: *** Booting Zephyr OS build zephyr-v3.3.0-8350-gfb003e583600 *** *** Board: qemu_cortex_m3 *** Installing direct IRQ handler for external IRQ0 (Exception #16)... *** Enabling IRQ0 in the NVIC logic... *** Received IVSHMEM PEER ID: 7 *** Waiting notification from peers to start... Now, from the Linux terminal, notify the arm VM (use the "IVSHMEM PEER ID" reported by Zephyr as the third arg, in this example: 7): # ./zephyr_ivshmem_test /dev/uio0 7 MMRs mapped at 0xffff8fb28000 in VMA. shmem mapped at 0xffff8f728000 in VMA. mmr0: 0 0 mmr1: 0 0 mmr2: 6 6 mmr3: 0 0 Data ok. 4194304 byte(s) checked. # The arm VM should report something like: *** Got interrupt at vector 0! *** Writting constant 0xb5b5b5b5 to shmem... done! *** Notifying back peer ID 6 at vector 0... Cheers, Gustavo Gustavo Romero (4): Add ivshmem-flat device tests/qtest: Reorganize common code in ivshmem-test tests/qtest: Add ivshmem-flat test hw/misc/ivshmem: Rename ivshmem to ivshmem-pci docs/system/devices/ivshmem-flat.rst | 89 +++++ hw/arm/mps2.c | 2 + hw/arm/stellaris.c | 5 +- hw/arm/virt.c | 2 + hw/core/sysbus-fdt.c | 1 + hw/misc/Kconfig | 5 + hw/misc/ivshmem-flat.c | 477 +++++++++++++++++++++++++++ hw/misc/{ivshmem.c => ivshmem-pci.c} | 0 hw/misc/meson.build | 4 +- hw/misc/trace-events | 18 + include/hw/misc/ivshmem-flat.h | 72 ++++ tests/qtest/ivshmem-flat-test.c | 319 ++++++++++++++++++ tests/qtest/ivshmem-test.c | 113 +------ tests/qtest/ivshmem-utils.c | 155 +++++++++ tests/qtest/ivshmem-utils.h | 56 ++++ tests/qtest/meson.build | 8 +- 16 files changed, 1215 insertions(+), 111 deletions(-) create mode 100644 docs/system/devices/ivshmem-flat.rst create mode 100644 hw/misc/ivshmem-flat.c rename hw/misc/{ivshmem.c => ivshmem-pci.c} (100%) create mode 100644 include/hw/misc/ivshmem-flat.h create mode 100644 tests/qtest/ivshmem-flat-test.c create mode 100644 tests/qtest/ivshmem-utils.c create mode 100644 tests/qtest/ivshmem-utils.h