mbox series

[v6,0/5] iommu/arm-smmu: Split pagetable support for arm-smmu-v2

Message ID 20200409233350.6343-1-jcrouse@codeaurora.org
Headers show
Series iommu/arm-smmu: Split pagetable support for arm-smmu-v2 | expand

Message

Jordan Crouse April 9, 2020, 11:33 p.m. UTC
This is another iteration for the split pagetable support based on the
suggestions from Robin and Will [1].

Background: In order to support per-context pagetables the GPU needs to enable
split tables so that we can store global buffers in the TTBR1 space leaving the
GPU free to program the TTBR0 register with the address of a context specific
pagetable.

If the DOMAIN_ATTR_SPLIT_TABLES attribute is set on the domain before attaching,
the context bank assigned to the domain will be programmed to allow translations
in the TTBR1 space. Translations in the TTBR0 region will be disallowed because,
as Robin pointe out, having a un-programmed TTBR0 register is dangerous.

The driver can determine if TTBR1 was successfully programmed by querying
DOMAIN_ATTR_SPLIT_TABLES after attaching. The domain geometry will also be
updated to reflect the virtual address space for the TTBR1 range.

Upcoming changes will allow auxiliary domains to be attached to the device which
will enable and program TTBR0.

This patchset is based on top of linux-next-20200409

Change log:

v6: Cleanups for the arm-smmu TTBR1 patch from Will Deacon
v4: Only program TTBR1 when split pagetables are requested. TTBR0 will be
enabled later when an auxiliary domain is attached
v3: Remove the implementation specific and make split pagetable support
part of the generic configuration

[1] https://lists.linuxfoundation.org/pipermail/iommu/2020-January/041373.html


Jordan Crouse (5):
  iommu: Add DOMAIN_ATTR_SPLIT_TABLES
  iommu/arm-smmu: Add support for TTBR1
  drm/msm: Attach the IOMMU device during initialization
  drm/msm: Refactor address space initialization
  drm/msm/a6xx: Support split pagetables

 drivers/gpu/drm/msm/adreno/a2xx_gpu.c    | 16 ++++++++
 drivers/gpu/drm/msm/adreno/a3xx_gpu.c    |  1 +
 drivers/gpu/drm/msm/adreno/a4xx_gpu.c    |  1 +
 drivers/gpu/drm/msm/adreno/a5xx_gpu.c    |  1 +
 drivers/gpu/drm/msm/adreno/a6xx_gpu.c    | 51 ++++++++++++++++++++++++
 drivers/gpu/drm/msm/adreno/adreno_gpu.c  | 23 ++++++++---
 drivers/gpu/drm/msm/adreno/adreno_gpu.h  |  8 ++++
 drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c  | 18 +++------
 drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c | 18 ++++-----
 drivers/gpu/drm/msm/disp/mdp5/mdp5_cfg.c |  4 --
 drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c | 18 ++++-----
 drivers/gpu/drm/msm/msm_drv.h            |  8 +---
 drivers/gpu/drm/msm/msm_gem_vma.c        | 36 +++--------------
 drivers/gpu/drm/msm/msm_gpu.c            | 49 +----------------------
 drivers/gpu/drm/msm/msm_gpu.h            |  4 +-
 drivers/gpu/drm/msm/msm_gpummu.c         |  6 ---
 drivers/gpu/drm/msm/msm_iommu.c          | 18 +++++----
 drivers/gpu/drm/msm/msm_mmu.h            |  1 -
 drivers/iommu/arm-smmu.c                 | 48 ++++++++++++++++++----
 drivers/iommu/arm-smmu.h                 | 24 ++++++++---
 include/linux/iommu.h                    |  2 +
 21 files changed, 200 insertions(+), 155 deletions(-)

Comments

Eric Anholt June 17, 2020, 8:16 p.m. UTC | #1
On Thu, Apr 9, 2020 at 4:34 PM Jordan Crouse <jcrouse@codeaurora.org> wrote:
>
> Refactor how address space initialization works. Instead of having the
> address space function create the MMU object (and thus require separate but
> equal functions for gpummu and iommu) use a single function and pass the
> MMU struct in. Make the generic code cleaner by using target specific
> functions to create the address space so a2xx can do its own thing in its
> own space.  For all the other targets use a generic helper to initialize
> IOMMU but leave the door open for newer targets to use customization
> if they need it.

I'm seeing regressions in dEQP-VK.memory.allocation.random.* on cheza
after this commit.   The symptom is that large allocations fail with
-ENOSPC from MSM_GEM_INFO(IOVA).

Possibly relevant change from having stuffed some debug info in:

before:
[    3.791436] [drm:msm_gem_address_space_create] *ERROR* msmgem
address space create: 0x1000000 + 0xfeffffff
[    3.801672] platform 506a000.gmu: Adding to iommu group 6
[    3.807359] [drm:msm_gem_address_space_create] *ERROR* msmgem
address space create: 0x0 + 0x7fffffff
[    3.817140] msm ae00000.mdss: bound 5000000.gpu (ops a3xx_ops)
[    3.823212] msm_dpu ae01000.mdp: [drm:msm_ioremap] *ERROR* failed
to get memory resource: vbif_nrt
[    3.832429] msm_dpu ae01000.mdp: [drm:msm_ioremap] *ERROR* failed
to get memory resource: regdma
[    3.841478] [drm:dpu_kms_hw_init:878] dpu hardware revision:0x40000000
[    3.848193] [drm:msm_gem_address_space_create] *ERROR* msmgem
address space create: 0x1000 + 0xffffefff

after:

[    3.798707] [drm:msm_gem_address_space_create] *ERROR* msmgem
address space create: 0x1000000 + 0xfffffff
[    3.808731] platform 506a000.gmu: Adding to iommu group 6
[    3.814440] [drm:msm_gem_address_space_create] *ERROR* msmgem
address space create: 0x0 + 0x7fffffff
[    3.820494] hub 2-1:1.0: USB hub found
[    3.824108] msm ae00000.mdss: bound 5000000.gpu (ops a3xx_ops)
[    3.828554] hub 2-1:1.0: 4 ports detected
[    3.833756] msm_dpu ae01000.mdp: [drm:msm_ioremap] *ERROR* failed
to get memory resource: vbif_nrt
[    3.847038] msm_dpu ae01000.mdp: [drm:msm_ioremap] *ERROR* failed
to get memory resource: regdma
[    3.856095] [drm:dpu_kms_hw_init:878] dpu hardware revision:0x40000000
[    3.862840] [drm:msm_gem_address_space_create] *ERROR* msmgem
address space create: 0x1000 + 0xfffffff

256MB for GMU address space?
Eric Anholt June 17, 2020, 8:37 p.m. UTC | #2
On Wed, Jun 17, 2020 at 1:16 PM Eric Anholt <eric@anholt.net> wrote:
>

> On Thu, Apr 9, 2020 at 4:34 PM Jordan Crouse <jcrouse@codeaurora.org> wrote:

> >

> > Refactor how address space initialization works. Instead of having the

> > address space function create the MMU object (and thus require separate but

> > equal functions for gpummu and iommu) use a single function and pass the

> > MMU struct in. Make the generic code cleaner by using target specific

> > functions to create the address space so a2xx can do its own thing in its

> > own space.  For all the other targets use a generic helper to initialize

> > IOMMU but leave the door open for newer targets to use customization

> > if they need it.

>

> I'm seeing regressions in dEQP-VK.memory.allocation.random.* on cheza

> after this commit.   The symptom is that large allocations fail with

> -ENOSPC from MSM_GEM_INFO(IOVA).

>

> Possibly relevant change from having stuffed some debug info in:

>

> before:

> [    3.791436] [drm:msm_gem_address_space_create] *ERROR* msmgem

> address space create: 0x1000000 + 0xfeffffff

> [    3.801672] platform 506a000.gmu: Adding to iommu group 6

> [    3.807359] [drm:msm_gem_address_space_create] *ERROR* msmgem

> address space create: 0x0 + 0x7fffffff

> [    3.817140] msm ae00000.mdss: bound 5000000.gpu (ops a3xx_ops)

> [    3.823212] msm_dpu ae01000.mdp: [drm:msm_ioremap] *ERROR* failed

> to get memory resource: vbif_nrt

> [    3.832429] msm_dpu ae01000.mdp: [drm:msm_ioremap] *ERROR* failed

> to get memory resource: regdma

> [    3.841478] [drm:dpu_kms_hw_init:878] dpu hardware revision:0x40000000

> [    3.848193] [drm:msm_gem_address_space_create] *ERROR* msmgem

> address space create: 0x1000 + 0xffffefff

>

> after:

>

> [    3.798707] [drm:msm_gem_address_space_create] *ERROR* msmgem

> address space create: 0x1000000 + 0xfffffff

> [    3.808731] platform 506a000.gmu: Adding to iommu group 6

> [    3.814440] [drm:msm_gem_address_space_create] *ERROR* msmgem

> address space create: 0x0 + 0x7fffffff

> [    3.820494] hub 2-1:1.0: USB hub found

> [    3.824108] msm ae00000.mdss: bound 5000000.gpu (ops a3xx_ops)

> [    3.828554] hub 2-1:1.0: 4 ports detected

> [    3.833756] msm_dpu ae01000.mdp: [drm:msm_ioremap] *ERROR* failed

> to get memory resource: vbif_nrt

> [    3.847038] msm_dpu ae01000.mdp: [drm:msm_ioremap] *ERROR* failed

> to get memory resource: regdma

> [    3.856095] [drm:dpu_kms_hw_init:878] dpu hardware revision:0x40000000

> [    3.862840] [drm:msm_gem_address_space_create] *ERROR* msmgem

> address space create: 0x1000 + 0xfffffff

>

> 256MB for GMU address space?


Found the bug, fixes at the last 2 commits of
https://github.com/anholt/linux/tree/drm-msm-address-space

I'm going to try having another go at convincing gmail to let git
send-email through, but all the howtos in the world didn't work last
time (gsuite has different behavior from normal gmail).