mbox

[GIT,PULL] CMA and ARM DMA-mapping updates for v3.5

Message ID 1337672417-10065-1-git-send-email-m.szyprowski@samsung.com
State Accepted
Headers show

Pull-request

git://git.linaro.org/people/mszyprowski/linux-dma-mapping.git for-linus

Message

Marek Szyprowski May 22, 2012, 7:40 a.m. UTC
Hi Linus,

I would like to ask for pulling Contiguous Memory Allocator (CMA) and
ARM DMA-mapping framework updates for v3.5.

The following changes since commit 76e10d158efb6d4516018846f60c2ab5501900bc:

  Linux 3.4 (2012-05-20 15:29:13 -0700)

with the top-most commit 0f51596bd39a5c928307ffcffc9ba07f90f42a8b

  Merge branch 'for-next-arm-dma' into for-linus

are available in the git repository at:
  git://git.linaro.org/people/mszyprowski/linux-dma-mapping.git for-linus


These patches contains 2 major updates for DMA mapping subsystem (mainly
for ARM architecture). First one is Contiguous Memory Allocator (CMA)
which makes it possible for device drivers to allocate big contiguous
chunks of memory after the system has booted. 

The main difference from the similar frameworks is the fact that CMA
allows to transparently reuse memory region reserved for the big chunk
allocation as a system memory, so no memory is wasted when no big chunk
is allocated. Once the alloc request is issued, the framework migrates
system pages to create a space for the required big chunk of physically
contiguous memory.

For more information one can refer to nice LWN articles:
'A reworked contiguous memory allocator': http://lwn.net/Articles/447405/
'CMA and ARM': http://lwn.net/Articles/450286/
'A deep dive into CMA': http://lwn.net/Articles/486301/
and the following thread with the patches and links to all previous versions:
https://lkml.org/lkml/2012/4/3/204

The main client for this new framework is ARM DMA-mapping subsystem.

The second part provides a complete redesign in ARM DMA-mapping
subsystem. The core implementation has been changed to use common struct
dma_map_ops based infrastructure with the recent updates for new dma
attributes merged in v3.4-rc2. This allows to use more than one
implementation of dma-mapping calls and change/select them on the struct
device basis. The first client of this new infractructure is dmabounce
implementation which has been completely cut out of the core, common
code.

The last patch of this redesign update introduces a new, experimental
implementation of dma-mapping calls on top of generic IOMMU framework.
This lets ARM sub-platform to transparently use IOMMU for DMA-mapping
calls if one provides required IOMMU hardware.

For more information please refer to the following thread:
http://www.spinics.net/lists/arm-kernel/msg175729.html

The last patch merges changes from both updates and provides a
resolution for the conflicts which cannot be avoided when patches have
been applied on the same files (mainly arch/arm/mm/dma-mapping.c).

Thanks!

Best regards
Marek Szyprowski
Samsung Poland R&D Center


Patch summary:

Marek Szyprowski (17):
      common: add dma_mmap_from_coherent() function
      ARM: dma-mapping: use dma_mmap_from_coherent()
      ARM: dma-mapping: use pr_* instread of printk
      ARM: dma-mapping: introduce DMA_ERROR_CODE constant
      ARM: dma-mapping: remove offset parameter to prepare for generic dma_ops
      ARM: dma-mapping: use asm-generic/dma-mapping-common.h
      ARM: dma-mapping: implement dma sg methods on top of any generic dma ops
      ARM: dma-mapping: move all dma bounce code to separate dma ops structure
      ARM: dma-mapping: remove redundant code and do the cleanup
      ARM: dma-mapping: use alloc, mmap, free from dma_ops
      ARM: dma-mapping: add support for IOMMU mapper
      mm: extract reclaim code from __alloc_pages_direct_reclaim()
      mm: trigger page reclaim in alloc_contig_range() to stabilise watermarks
      drivers: add Contiguous Memory Allocator
      X86: integrate CMA with DMA-mapping subsystem
      ARM: integrate CMA with DMA-mapping subsystem
      Merge branch 'for-next-arm-dma' into for-linus

Mel Gorman (1):
      mm: Serialize access to min_free_kbytes

Michal Nazarewicz (9):
      mm: page_alloc: remove trailing whitespace
      mm: compaction: introduce isolate_migratepages_range()
      mm: compaction: introduce map_pages()
      mm: compaction: introduce isolate_freepages_range()
      mm: compaction: export some of the functions
      mm: page_alloc: introduce alloc_contig_range()
      mm: page_alloc: change fallbacks array handling
      mm: mmzone: MIGRATE_CMA migration type added
      mm: page_isolation: MIGRATE_CMA isolation functions added

Minchan Kim (1):
      cma: fix migration mode

Vitaly Andrianov (1):
      ARM: dma-mapping: use PMD size for section unmap

 Documentation/kernel-parameters.txt   |    9 +
 arch/Kconfig                          |    3 +
 arch/arm/Kconfig                      |   11 +
 arch/arm/common/dmabounce.c           |   84 ++-
 arch/arm/include/asm/device.h         |    4 +
 arch/arm/include/asm/dma-contiguous.h |   15 +
 arch/arm/include/asm/dma-iommu.h      |   34 +
 arch/arm/include/asm/dma-mapping.h    |  407 +++--------
 arch/arm/include/asm/mach/map.h       |    1 +
 arch/arm/kernel/setup.c               |    9 +-
 arch/arm/mm/dma-mapping.c             | 1348 ++++++++++++++++++++++++++++-----
 arch/arm/mm/init.c                    |   23 +-
 arch/arm/mm/mm.h                      |    3 +
 arch/arm/mm/mmu.c                     |   31 +-
 arch/arm/mm/vmregion.h                |    2 +-
 arch/x86/Kconfig                      |    1 +
 arch/x86/include/asm/dma-contiguous.h |   13 +
 arch/x86/include/asm/dma-mapping.h    |    5 +
 arch/x86/kernel/pci-dma.c             |   18 +-
 arch/x86/kernel/pci-nommu.c           |    8 +-
 arch/x86/kernel/setup.c               |    2 +
 drivers/base/Kconfig                  |   89 +++
 drivers/base/Makefile                 |    1 +
 drivers/base/dma-coherent.c           |   42 +
 drivers/base/dma-contiguous.c         |  401 ++++++++++
 include/asm-generic/dma-coherent.h    |    4 +-
 include/asm-generic/dma-contiguous.h  |   28 +
 include/linux/device.h                |    4 +
 include/linux/dma-contiguous.h        |  110 +++
 include/linux/gfp.h                   |   12 +
 include/linux/mmzone.h                |   47 +-
 include/linux/page-isolation.h        |   18 +-
 mm/Kconfig                            |    2 +-
 mm/Makefile                           |    3 +-
 mm/compaction.c                       |  418 +++++++----
 mm/internal.h                         |   33 +
 mm/memory-failure.c                   |    2 +-
 mm/memory_hotplug.c                   |    6 +-
 mm/page_alloc.c                       |  409 +++++++++--
 mm/page_isolation.c                   |   15 +-
 mm/vmstat.c                           |    3 +
 41 files changed, 2898 insertions(+), 780 deletions(-)
 create mode 100644 arch/arm/include/asm/dma-contiguous.h
 create mode 100644 arch/arm/include/asm/dma-iommu.h
 create mode 100644 arch/x86/include/asm/dma-contiguous.h
 create mode 100644 drivers/base/dma-contiguous.c
 create mode 100644 include/asm-generic/dma-contiguous.h
 create mode 100644 include/linux/dma-contiguous.h

Comments

Barry Song May 22, 2012, 8:27 a.m. UTC | #1
Hi Marek,

2012/5/22 Marek Szyprowski <m.szyprowski@samsung.com>:
> Hi Linus,
>
> I would like to ask for pulling Contiguous Memory Allocator (CMA) and
> ARM DMA-mapping framework updates for v3.5.
>
> The following changes since commit 76e10d158efb6d4516018846f60c2ab5501900bc:
>
>  Linux 3.4 (2012-05-20 15:29:13 -0700)
>
> with the top-most commit 0f51596bd39a5c928307ffcffc9ba07f90f42a8b
>
>  Merge branch 'for-next-arm-dma' into for-linus
>
> are available in the git repository at:
>  git://git.linaro.org/people/mszyprowski/linux-dma-mapping.git for-linus
>
>
> Patch summary:
>
> Marek Szyprowski (17):
>      common: add dma_mmap_from_coherent() function
>      ARM: dma-mapping: use dma_mmap_from_coherent()
>      ARM: dma-mapping: use pr_* instread of printk
>      ARM: dma-mapping: introduce DMA_ERROR_CODE constant
>      ARM: dma-mapping: remove offset parameter to prepare for generic dma_ops
>      ARM: dma-mapping: use asm-generic/dma-mapping-common.h
>      ARM: dma-mapping: implement dma sg methods on top of any generic dma ops
>      ARM: dma-mapping: move all dma bounce code to separate dma ops structure
>      ARM: dma-mapping: remove redundant code and do the cleanup
>      ARM: dma-mapping: use alloc, mmap, free from dma_ops
>      ARM: dma-mapping: add support for IOMMU mapper
>      mm: extract reclaim code from __alloc_pages_direct_reclaim()
>      mm: trigger page reclaim in alloc_contig_range() to stabilise watermarks
>      drivers: add Contiguous Memory Allocator
>      X86: integrate CMA with DMA-mapping subsystem
>      ARM: integrate CMA with DMA-mapping subsystem
>      Merge branch 'for-next-arm-dma' into for-linus
>
> Mel Gorman (1):
>      mm: Serialize access to min_free_kbytes
>
> Michal Nazarewicz (9):
>      mm: page_alloc: remove trailing whitespace
>      mm: compaction: introduce isolate_migratepages_range()
>      mm: compaction: introduce map_pages()
>      mm: compaction: introduce isolate_freepages_range()
>      mm: compaction: export some of the functions
>      mm: page_alloc: introduce alloc_contig_range()
>      mm: page_alloc: change fallbacks array handling
>      mm: mmzone: MIGRATE_CMA migration type added
>      mm: page_isolation: MIGRATE_CMA isolation functions added
>
> Minchan Kim (1):
>      cma: fix migration mode
>
> Vitaly Andrianov (1):
>      ARM: dma-mapping: use PMD size for section unmap

would you pls add CMA test module in this patchset too?

http://lists.infradead.org/pipermail/linux-arm-kernel/2012-March/088412.html

mm: cma: add a simple kernel module as the helper to test CMA

Signed-off-by: Barry Song <Baohua.Song@csr.com>
Reviewed-by: Michal Nazarewicz <mina86@mina86.com>

Thanks
barry
Russell King - ARM Linux May 29, 2012, 12:29 p.m. UTC | #2
I notice we have new warnings as a result of CMA being merged, though
thankfully they're just in Kconfig:

warning: (ARM) selects CMA which has unmet direct dependencies (HAVE_DMA_CONTIGUOUS && HAVE_MEMBLOCK && EXPERIMENTAL)

This seems totally weird: you're mandating that ARM must have CMA
selected, but it's an experimental feature?  So you're implying that
the entire ARM kernel becomes totally experimental for the next
release cycle?

I think this needs fixing.
Marek Szyprowski May 29, 2012, 2:50 p.m. UTC | #3
Hello,

On Tuesday, May 29, 2012 2:30 PM Russell King - ARM Linux wrote:

> I notice we have new warnings as a result of CMA being merged, though
> thankfully they're just in Kconfig:
> 
> warning: (ARM) selects CMA which has unmet direct dependencies (HAVE_DMA_CONTIGUOUS &&
> HAVE_MEMBLOCK && EXPERIMENTAL)
> 
> This seems totally weird: you're mandating that ARM must have CMA
> selected, but it's an experimental feature?  So you're implying that
> the entire ARM kernel becomes totally experimental for the next
> release cycle?
> 
> I think this needs fixing.

No, that wasn't my intention. I will provide a patch which removes unconditional dependency
on CMA - it will let one to disable CMA and use old allocation method if needed, but this 
requires a few more changes in the dma-mapping implementation. I wasn't aware of the 
consequences and no one has complained about this since v15 of CMA patches (Aug 2011).

Best regards
Russell King - ARM Linux June 1, 2012, 3:06 p.m. UTC | #4
On Tue, May 29, 2012 at 04:50:45PM +0200, Marek Szyprowski wrote:
> Hello,
> 
> On Tuesday, May 29, 2012 2:30 PM Russell King - ARM Linux wrote:
> 
> > I notice we have new warnings as a result of CMA being merged, though
> > thankfully they're just in Kconfig:
> > 
> > warning: (ARM) selects CMA which has unmet direct dependencies (HAVE_DMA_CONTIGUOUS &&
> > HAVE_MEMBLOCK && EXPERIMENTAL)
> > 
> > This seems totally weird: you're mandating that ARM must have CMA
> > selected, but it's an experimental feature?  So you're implying that
> > the entire ARM kernel becomes totally experimental for the next
> > release cycle?
> > 
> > I think this needs fixing.
> 
> No, that wasn't my intention. I will provide a patch which removes unconditional dependency
> on CMA - it will let one to disable CMA and use old allocation method if needed, but this 
> requires a few more changes in the dma-mapping implementation. I wasn't aware of the 
> consequences and no one has complained about this since v15 of CMA patches (Aug 2011).

I've just been looking at the automatic build and boot logs, and CMA looks
like it has the potential to cause regressions:

Linux version 3.4.0+ (rmk@rmk-PC.arm.linux.org.uk) (gcc version 4.3.5 (GCC) ) #1 PREEMPT Fri Jun 1 02:01:58 BST 2012
CPU: ARMv7 Processor [411fc082] revision 2 (ARMv7), cr=10c53c7d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
Machine: OMAP LDP board
vmalloc area is too big, limiting to 976MB
Reserving 2097152 bytes SDRAM for VRAM
cma: CMA: failed to reserve 16 MiB
...
Kernel command line: console=ttyO2,115200n8 noinitrd vmalloc=1G mem=128M root=/dev/mmcblk0p2 rw ip=none rootdelay=2 video=omap24xxfb:rotation=270
...
NET: Registered protocol family 16
initlevel:2=postcore, 16 registered initcalls
DMA: failed to allocate 256 KiB pool for atomic coherent allocation

I don't see any other failures, but that doesn't look good...

Note that that command line has worked perfectly well up until now.