[Xen-devel,v4,31/33] libxl: Add support for non-PCI passthrough

Message ID 1426793399-6283-32-git-send-email-julien.grall@linaro.org
State New
Headers show

Commit Message

Julien Grall March 19, 2015, 7:29 p.m.
On ARM, every non-PCI device are described in the device tree. Each of
them can be found via a path.

This patch introduces a very basic support, only the IOMMU will be set
up correctly. The user will have to:
    - Describe the device in the partial device tree
    - Map manually MMIO/IRQ

This is a first approach, that will allow to have a basic non-PCI
passthrough support in Xen. This could be improved later.

Furthermore add LIBXL_HAVE_DEVICETREE_PASSTHROUGH to indicate we
support non-PCI passthrough and partial device tree (introduced by a
previous patch).

Signed-off-by: Julien Grall <julien.grall@linaro.org>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>

---
    Changes in v4:
        - Add LIBXL_HAVE_DEVICTREE_PASSTHROUGH to indicate we support
        non-PCI passthrough. This is also used in order to indicate
        partial device tree is supported
        - Remove libxl_dtdev.c as it contains only a 2 lines functions
        and call directly xc_* from libxl_create.c
        - Introduce domcreate_attach_dtdev

    Changes in v3:
        - Dynamic allocation has been dropped
        - Rework the commit message in accordance with the previous
        item

    Changes in v2:
        - Get DT infos earlier
        - Allocate/map IRQ in libxl__arch_domain_create rather than in
        libxl__device_dt_add
        - Use VIRQ rather than the PIRQ to construct the interrupts
        properties of the device tree
        - Correct cpumask in make_dtdev_node. We allow the interrupt to
        be used on the 8 CPUs
        - Fix LOGE when we map the MMIO region in the guest in
        libxl__device_dt_add. The domain and the IRQ were inverted
        - Calculate the number of SPIs to configure the VGIC
        - xc_physdev_dtdev_* helpers has been renamed to xc_dtdev_*
        - Rename libxl_device_dt to libxl_device_dtdev
---
 tools/libxl/libxl.h          |  6 ++++++
 tools/libxl/libxl_create.c   | 32 ++++++++++++++++++++++++++++++++
 tools/libxl/libxl_internal.h |  5 +++++
 tools/libxl/libxl_types.idl  |  5 +++++
 4 files changed, 48 insertions(+)

Comments

Julien Grall March 31, 2015, 1 p.m. | #1
Hi Ian,

On 31/03/15 12:49, Ian Campbell wrote:
> On Thu, 2015-03-19 at 19:29 +0000, Julien Grall wrote:
>> On ARM, every non-PCI device are described in the device tree. Each of
>> them can be found via a path.
>>
>> This patch introduces a very basic support, only the IOMMU will be set
>> up correctly. The user will have to:
>>     - Describe the device in the partial device tree
>>     - Map manually MMIO/IRQ
>>
>> This is a first approach, that will allow to have a basic non-PCI
>> passthrough support in Xen. This could be improved later.
>>
>> Furthermore add LIBXL_HAVE_DEVICETREE_PASSTHROUGH to indicate we
>> support non-PCI passthrough and partial device tree (introduced by a
>> previous patch).
> 
> Ah, you can ignore my comment on the previous patch then.
> 
> The comment with the #define might be an OK place for the
> big-scary-warning I mentioned in the previous patch too?

I think this IDL would be a more suitable place.

>>
>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
>> Cc: Wei Liu <wei.liu2@citrix.com>
>>
>> ---
>>     Changes in v4:
>>         - Add LIBXL_HAVE_DEVICTREE_PASSTHROUGH to indicate we support
>>         non-PCI passthrough. This is also used in order to indicate
>>         partial device tree is supported
>>         - Remove libxl_dtdev.c as it contains only a 2 lines functions
>>         and call directly xc_* from libxl_create.c
>>         - Introduce domcreate_attach_dtdev
>>
>>     Changes in v3:
>>         - Dynamic allocation has been dropped
>>         - Rework the commit message in accordance with the previous
>>         item
>>
>>     Changes in v2:
>>         - Get DT infos earlier
>>         - Allocate/map IRQ in libxl__arch_domain_create rather than in
>>         libxl__device_dt_add
>>         - Use VIRQ rather than the PIRQ to construct the interrupts
>>         properties of the device tree
>>         - Correct cpumask in make_dtdev_node. We allow the interrupt to
>>         be used on the 8 CPUs
>>         - Fix LOGE when we map the MMIO region in the guest in
>>         libxl__device_dt_add. The domain and the IRQ were inverted
>>         - Calculate the number of SPIs to configure the VGIC
>>         - xc_physdev_dtdev_* helpers has been renamed to xc_dtdev_*
>>         - Rename libxl_device_dt to libxl_device_dtdev
>> ---
>>  tools/libxl/libxl.h          |  6 ++++++
>>  tools/libxl/libxl_create.c   | 32 ++++++++++++++++++++++++++++++++
>>  tools/libxl/libxl_internal.h |  5 +++++
>>  tools/libxl/libxl_types.idl  |  5 +++++
>>  4 files changed, 48 insertions(+)
>>
>> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
>> index 6bc75c5..baaf06b 100644
>> --- a/tools/libxl/libxl.h
>> +++ b/tools/libxl/libxl.h
>> @@ -179,6 +179,12 @@
>>  #define LIBXL_HAVE_BUILDINFO_HVM_MMIO_HOLE_MEMKB 1
>>  
>>  /*
>> + * libxl_domain_build_info has device_tree and libxl_device_dtdev
>> + * exists. This mean non-PCI passthrough is supported for ARM
> 
> Rather than non-PCI I'd say device tree here, since I'm sure you aren't
> supporting the new flargle-bus I've just invented...

Ok

> 
>> + *
>> +#define LIBXL_HAVE_DEVICETREE_PASSTHROUGH 1
>> +
>> +/*
>>   * libxl ABI compatibility
>>   *
>>   * The only guarantee which libxl makes regarding ABI compatibility
>> diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
>> index 15b464e..39c828b 100644
>> --- a/tools/libxl/libxl_create.c
>> +++ b/tools/libxl/libxl_create.c
>> @@ -751,6 +751,8 @@ static void domcreate_attach_vtpms(libxl__egc *egc, libxl__multidev *multidev,
>>                                     int ret);
>>  static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *aodevs,
>>                                   int ret);
>> +static void domcreate_attach_dtdev(libxl__egc *egc,
>> +                                   libxl__domain_create_state *dcs);
>>  
>>  static void domcreate_console_available(libxl__egc *egc,
>>                                          libxl__domain_create_state *dcs);
>> @@ -1444,6 +1446,36 @@ static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *multidev,
>>          }
>>      }
>>  
>> +    domcreate_attach_dtdev(egc, dcs);
>> +    return;
>> +
>> +error_out:
>> +    assert(ret);
>> +    domcreate_complete(egc, dcs, ret);
>> +}
>> +
>> +static void domcreate_attach_dtdev(libxl__egc *egc,
>> +                                   libxl__domain_create_state *dcs)
> 
> I know it isn't strictly needed, but I think for consistency this
> function should take a ret and check it + propagate it via
> domcreate_complete on error, like the other ones in the call chain do.

The other caller have the ret because they use multidev. When it's not
the case (such as domcreate_console_available), the ret parameter is not
present.

So I'm not keen to add a new unused parameter.

Regards,

Patch

diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 6bc75c5..baaf06b 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -179,6 +179,12 @@ 
 #define LIBXL_HAVE_BUILDINFO_HVM_MMIO_HOLE_MEMKB 1
 
 /*
+ * libxl_domain_build_info has device_tree and libxl_device_dtdev
+ * exists. This mean non-PCI passthrough is supported for ARM
+ */
+#define LIBXL_HAVE_DEVICETREE_PASSTHROUGH 1
+
+/*
  * libxl ABI compatibility
  *
  * The only guarantee which libxl makes regarding ABI compatibility
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 15b464e..39c828b 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -751,6 +751,8 @@  static void domcreate_attach_vtpms(libxl__egc *egc, libxl__multidev *multidev,
                                    int ret);
 static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *aodevs,
                                  int ret);
+static void domcreate_attach_dtdev(libxl__egc *egc,
+                                   libxl__domain_create_state *dcs);
 
 static void domcreate_console_available(libxl__egc *egc,
                                         libxl__domain_create_state *dcs);
@@ -1444,6 +1446,36 @@  static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *multidev,
         }
     }
 
+    domcreate_attach_dtdev(egc, dcs);
+    return;
+
+error_out:
+    assert(ret);
+    domcreate_complete(egc, dcs, ret);
+}
+
+static void domcreate_attach_dtdev(libxl__egc *egc,
+                                   libxl__domain_create_state *dcs)
+{
+    STATE_AO_GC(dcs->ao);
+    int i;
+    int ret;
+    int domid = dcs->guest_domid;
+
+    /* convenience aliases */
+    libxl_domain_config *const d_config = dcs->guest_config;
+
+    for (i = 0; i < d_config->num_dtdevs; i++) {
+        const libxl_device_dtdev *dtdev = &d_config->dtdevs[i];
+
+        LOG(DEBUG, "Assign device \"%s\" to dom%u", dtdev->path, domid);
+        ret = xc_assign_dt_device(CTX->xch, domid, dtdev->path);
+        if (ret < 0) {
+            LOG(ERROR, "xc_assign_dtdevice failed: %d\n", ret);
+            goto error_out;
+        }
+    }
+
     domcreate_console_available(egc, dcs);
 
     domcreate_complete(egc, dcs, 0);
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 46fa624..1191098 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1198,6 +1198,11 @@  _hidden int libxl__create_pci_backend(libxl__gc *gc, uint32_t domid,
                                       libxl_device_pci *pcidev, int num);
 _hidden int libxl__device_pci_destroy_all(libxl__gc *gc, uint32_t domid);
 
+/* from libxl_dtdev */
+
+_hidden int libxl__device_dt_add(libxl__gc *gc, uint32_t domid,
+                                 const libxl_device_dtdev *dtdev);
+
 /*----- xswait: wait for a xenstore node to be suitable -----*/
 
 typedef struct libxl__xswait_state libxl__xswait_state;
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index c97e6ed..2ab8587 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -534,6 +534,10 @@  libxl_device_pci = Struct("device_pci", [
     ("seize", bool),
     ])
 
+libxl_device_dtdev = Struct("device_dtdev", [
+    ("path", string),
+    ])
+
 libxl_device_vtpm = Struct("device_vtpm", [
     ("backend_domid",    libxl_domid),
     ("backend_domname",  string),
@@ -560,6 +564,7 @@  libxl_domain_config = Struct("domain_config", [
     ("disks", Array(libxl_device_disk, "num_disks")),
     ("nics", Array(libxl_device_nic, "num_nics")),
     ("pcidevs", Array(libxl_device_pci, "num_pcidevs")),
+    ("dtdevs", Array(libxl_device_dtdev, "num_dtdevs")),
     ("vfbs", Array(libxl_device_vfb, "num_vfbs")),
     ("vkbs", Array(libxl_device_vkb, "num_vkbs")),
     ("vtpms", Array(libxl_device_vtpm, "num_vtpms")),