mbox series

[v2,0/3] Apple M1 DART IOMMU driver

Message ID 20210328074009.95932-1-sven@svenpeter.dev
Headers show
Series Apple M1 DART IOMMU driver | expand

Message

Sven Peter March 28, 2021, 7:40 a.m. UTC
Hi,

Here's v2 of my Apple M1 DART IOMMU driver series as a follow up to the original
version [1].

Short summary: this series adds support for the iommu found in Apple's new M1
SoC which is required to use DMA on most peripherals. So far this code has been
tested with dwc3 in host and device mode on a M1 Mac Mini on top of the latest
version of Hector's bringup series [2,3] together with my m1n1 bootloader
branch to bring up USB [4]. It will also apply (but not be very useful) on
top of iommu/next and v5.12-rc3.

Thanks everyone for the suggestions and discussions so far. I believe they
have already significantly improved the state of this driver and our
understanding of the DART iommu!

The part I'm most unsure about is the way I keep track of the multiple
iommu nodes attached to a device. I would love to especially get some
feedback there.


Changes for v2:
 - fixed devicetree binding linting issues pointed out by Rob Herring and
   reworked that file.
 - made DART-specific code in io-pgtable.c unconditional and removed flag from
   Kconfig as proposed by Robin Murphy.
 - allowed multiple DART nodes in the "iommus" property as proposed by
   Rob Herring and Robin Murphy. this resulted in significant changes
   to apple-iommu-dart.c.
 - the domain aperture is now forced to 32bit if translation is enabled after
   the original suggestion to limit the aperture by Mark Kettenis and the
   follow-up discussion and investigation with Mark Kettenis, Arnd Bergmann,
   Robin Murphy and Rob Herring. This change also simplified the code
   in io-pgtable.c and made some of the improvements suggested during review
   not apply anymore.
 - added support for bypassed and isolated domain modes.
 - reject IOMMU_MMIO and IOMMU_NOEXEC since it's unknown how to set these up
   for now or if the hardware even supports these flags.
 - renamed some registers to be less confusing (mainly s/DOMAIN/STREAM/ to
   prevent confusion with linux's iommu domain concept).

I have also fixed my email provider so this time the series should actually
be a single thread and not contain any HTML by accident anymore...

Best,


Sven


[1] https://lore.kernel.org/linux-iommu/20210320151903.60759-1-sven@svenpeter.dev/
[2] https://lore.kernel.org/linux-arch/20210304213902.83903-1-marcan@marcan.st/
[3] https://github.com/AsahiLinux/linux/tree/upstream-bringup-v4
[4] https://github.com/svenpeter42/m1n1/tree/usb-dwc3-serial-wip

Sven Peter (3):
  iommu: io-pgtable: add DART pagetable format
  dt-bindings: iommu: add DART iommu bindings
  iommu: dart: Add DART iommu driver

 .../devicetree/bindings/iommu/apple,dart.yaml |  81 ++
 MAINTAINERS                                   |   7 +
 drivers/iommu/Kconfig                         |  14 +
 drivers/iommu/Makefile                        |   1 +
 drivers/iommu/apple-dart-iommu.c              | 858 ++++++++++++++++++
 drivers/iommu/io-pgtable-arm.c                |  59 ++
 drivers/iommu/io-pgtable.c                    |   1 +
 include/linux/io-pgtable.h                    |   6 +
 8 files changed, 1027 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/iommu/apple,dart.yaml
 create mode 100644 drivers/iommu/apple-dart-iommu.c

Comments

Arnd Bergmann March 28, 2021, 8:16 a.m. UTC | #1
On Sun, Mar 28, 2021 at 9:40 AM Sven Peter <sven@svenpeter.dev> wrote:

I noticed only one detail here:

> +  - |+
> +    dart2a: dart2a@82f00000 {
> +      compatible = "apple,t8103-dart";
> +      reg = <0x82f00000 0x4000>;
> +      interrupts = <1 781 4>;
> +      #iommu-cells = <1>;
> +    };

The name of the iommu should be iommu@82f00000, not dart2a@82f00000.

       Arnd
Will Deacon April 7, 2021, 10:42 a.m. UTC | #2
On Sun, Mar 28, 2021 at 09:40:09AM +0200, Sven Peter wrote:
> Apple's new SoCs use iommus for almost all peripherals. These Device

> Address Resolution Tables must be setup before these peripherals can

> act as DMA masters.

> 

> Signed-off-by: Sven Peter <sven@svenpeter.dev>

> ---

>  MAINTAINERS                      |   1 +

>  drivers/iommu/Kconfig            |  14 +

>  drivers/iommu/Makefile           |   1 +

>  drivers/iommu/apple-dart-iommu.c | 858 +++++++++++++++++++++++++++++++

>  4 files changed, 874 insertions(+)

>  create mode 100644 drivers/iommu/apple-dart-iommu.c


[...]

> +/* must be called with held domain->lock */

> +static int apple_dart_attach_stream(struct apple_dart_domain *domain,

> +				    struct apple_dart *dart, u32 sid)

> +{

> +	unsigned long flags;

> +	struct apple_dart_stream *stream;

> +	struct io_pgtable_cfg *pgtbl_cfg;

> +	int ret;

> +

> +	list_for_each_entry(stream, &domain->streams, stream_head) {

> +		if (stream->dart == dart && stream->sid == sid) {

> +			stream->num_devices++;

> +			return 0;

> +		}

> +	}

> +

> +	spin_lock_irqsave(&dart->lock, flags);

> +

> +	if (WARN_ON(dart->used_sids & BIT(sid))) {

> +		ret = -EINVAL;

> +		goto error;

> +	}

> +

> +	stream = kzalloc(sizeof(*stream), GFP_KERNEL);

> +	if (!stream) {

> +		ret = -ENOMEM;

> +		goto error;

> +	}


Just in case you missed it, a cocci bot noticed that you're using GFP_KERNEL
to allocate while holding a spinlock here:

https://lore.kernel.org/r/alpine.DEB.2.22.394.2104041724340.2958@hadrien

Will
Sven Peter April 9, 2021, 4:50 p.m. UTC | #3
On Wed, Apr 7, 2021, at 12:42, Will Deacon wrote:
> On Sun, Mar 28, 2021 at 09:40:09AM +0200, Sven Peter wrote:

> > Apple's new SoCs use iommus for almost all peripherals. These Device

> > Address Resolution Tables must be setup before these peripherals can

> > act as DMA masters.

> > 

> > Signed-off-by: Sven Peter <sven@svenpeter.dev>

> > ---

> >  MAINTAINERS                      |   1 +

> >  drivers/iommu/Kconfig            |  14 +

> >  drivers/iommu/Makefile           |   1 +

> >  drivers/iommu/apple-dart-iommu.c | 858 +++++++++++++++++++++++++++++++

> >  4 files changed, 874 insertions(+)

> >  create mode 100644 drivers/iommu/apple-dart-iommu.c

> 

> [...]

> 

> > +/* must be called with held domain->lock */

> > +static int apple_dart_attach_stream(struct apple_dart_domain *domain,

> > +				    struct apple_dart *dart, u32 sid)

> > +{

> > +	unsigned long flags;

> > +	struct apple_dart_stream *stream;

> > +	struct io_pgtable_cfg *pgtbl_cfg;

> > +	int ret;

> > +

> > +	list_for_each_entry(stream, &domain->streams, stream_head) {

> > +		if (stream->dart == dart && stream->sid == sid) {

> > +			stream->num_devices++;

> > +			return 0;

> > +		}

> > +	}

> > +

> > +	spin_lock_irqsave(&dart->lock, flags);

> > +

> > +	if (WARN_ON(dart->used_sids & BIT(sid))) {

> > +		ret = -EINVAL;

> > +		goto error;

> > +	}

> > +

> > +	stream = kzalloc(sizeof(*stream), GFP_KERNEL);

> > +	if (!stream) {

> > +		ret = -ENOMEM;

> > +		goto error;

> > +	}

> 

> Just in case you missed it, a cocci bot noticed that you're using GFP_KERNEL

> to allocate while holding a spinlock here:

> 

> https://lore.kernel.org/r/alpine.DEB.2.22.394.2104041724340.2958@hadrien

> 


Thanks for the reminder!
I haven't replied yet because that one was found later when the bot picked up
a (slightly earlier) version that Marc was using to bring up pcie I believe.
I'll fix it for the next version.


Sven