From patchwork Thu Mar 22 10:02:23 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 7404 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 889AC23DEA for ; Thu, 22 Mar 2012 10:02:31 +0000 (UTC) Received: from mail-iy0-f180.google.com (mail-iy0-f180.google.com [209.85.210.180]) by fiordland.canonical.com (Postfix) with ESMTP id 377B1A183EF for ; Thu, 22 Mar 2012 10:02:31 +0000 (UTC) Received: by mail-iy0-f180.google.com with SMTP id e36so3827723iag.11 for ; Thu, 22 Mar 2012 03:02:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf:from:to :date:message-id:x-mailer:in-reply-to:references:cc:subject :x-beenthere:x-mailman-version:precedence:list-id:list-unsubscribe :list-archive:list-post:list-help:list-subscribe:mime-version :content-type:content-transfer-encoding:sender:errors-to :x-gm-message-state; bh=v06TCzhxjTvCuSnMvsw9VeFQyFex63cI0YBJb6gY9yc=; b=F0S3QWNzM4DXI+MXY7b/d3AUm+fXxKBEvSKjZ5JrDtCkrXjTPeB8qw9jvbwsMQ6jQV V8r+COqlgw12VaPiQYHBfcNWYK9V0pSptlpAeCAZzqR9iIhzn6LFUIamnLtFRDXKQza9 AwBh42HWCxJcO9dV7tUrdn+XzuzZ9ZBjLezpyF/zGXWuyTUVHY2hQqe2w7p9SJAIltFI QdYHtJnOnOnbR6/z8KcPNdtvcWYtuBzYcxKPLDQ8AoQ5GCsCJmxqEFI1DbTFSOqX/liR IJ1UehWraz0vnuMl6NaAqwwo6USX8p4BkNRbrn76ZDOVtqkUgmbuPviQUAfFJ/Ir3mPQ HAKw== Received: by 10.50.183.163 with SMTP id en3mr964590igc.12.1332410550852; Thu, 22 Mar 2012 03:02:30 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.231.203.79 with SMTP id fh15csp43655ibb; Thu, 22 Mar 2012 03:02:30 -0700 (PDT) Received: by 10.180.101.136 with SMTP id fg8mr3547815wib.4.1332410544956; Thu, 22 Mar 2012 03:02:24 -0700 (PDT) Received: from mombin.canonical.com (mombin.canonical.com. [91.189.95.16]) by mx.google.com with ESMTP id v2si4573427weq.39.2012.03.22.03.02.23; Thu, 22 Mar 2012 03:02:24 -0700 (PDT) Received-SPF: neutral (google.com: 91.189.95.16 is neither permitted nor denied by best guess record for domain of linaro-mm-sig-bounces@lists.linaro.org) client-ip=91.189.95.16; Authentication-Results: mx.google.com; spf=neutral (google.com: 91.189.95.16 is neither permitted nor denied by best guess record for domain of linaro-mm-sig-bounces@lists.linaro.org) smtp.mail=linaro-mm-sig-bounces@lists.linaro.org Received: from localhost ([127.0.0.1] helo=mombin.canonical.com) by mombin.canonical.com with esmtp (Exim 4.71) (envelope-from ) id 1SAeqd-0002S5-7N; Thu, 22 Mar 2012 10:02:23 +0000 Received: from perceval.ideasonboard.com ([95.142.166.194]) by mombin.canonical.com with esmtp (Exim 4.71) (envelope-from ) id 1SAeqb-0001vX-Nl for linaro-mm-sig@lists.linaro.org; Thu, 22 Mar 2012 10:02:21 +0000 Received: from localhost.localdomain (ptr-91-87-217-68.mobistar.be [91.87.217.68]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 198EE7B0B; Thu, 22 Mar 2012 11:02:16 +0100 (CET) From: Laurent Pinchart To: Tomasz Stanislawski Date: Thu, 22 Mar 2012 11:02:23 +0100 Message-Id: <1332410543-14433-4-git-send-email-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 1.7.3.4 In-Reply-To: <5187420.ORugDzE4Gp@avalon> References: <5187420.ORugDzE4Gp@avalon> Cc: sumit.semwal@ti.com, robdclark@gmail.com, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, kyungmin.park@samsung.com, sakari.ailus@iki.fi, Andrzej Pietrasiewicz , airlied@redhat.com, Kamil Debski Subject: [Linaro-mm-sig] [RFCv2 PATCH 2/9 - 4/4] v4l: vb2-dma-contig: update and code refactoring X-BeenThere: linaro-mm-sig@lists.linaro.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Unified memory management interest group." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: linaro-mm-sig-bounces@lists.linaro.org Errors-To: linaro-mm-sig-bounces@lists.linaro.org X-Gm-Message-State: ALoCoQkXKXcL1z08JqL+1h+p/wTWNW/c8yZo4ZsGxsB5Cv4Qf95q2U29rw41WKQJA2AvuHA5k85F From: Tomasz Stanislawski This patch combines updates and fixes to dma-contig allocator. Moreover the allocator code was refactored. The most important changes are: - functions were reordered - move compression of scatterlist to separete function - add support for multichunk but contiguous scatterlists - simplified implementation of vb2-dma-contig context structure - let mmap method to use dma_mmap_writecombine - add support for scatterlist in userptr mode Signed-off-by: Marek Szyprowski [mmap method] Signed-off-by: Andrzej Pietrasiewicz [scatterlist in userptr mode] Signed-off-by: Kamil Debski [bugfixing] Signed-off-by: Tomasz Stanislawski [core refactoring, helper functions] Signed-off-by: Kyungmin Park --- drivers/media/video/videobuf2-dma-contig.c | 400 +++++++++++++++++++++++++--- 1 files changed, 365 insertions(+), 35 deletions(-) diff --git a/drivers/media/video/videobuf2-dma-contig.c b/drivers/media/video/videobuf2-dma-contig.c index c898e6f..9965465 100644 --- a/drivers/media/video/videobuf2-dma-contig.c +++ b/drivers/media/video/videobuf2-dma-contig.c @@ -10,9 +10,12 @@ * the Free Software Foundation. */ +#include +#include #include +#include +#include #include -#include #include #include @@ -22,16 +25,115 @@ struct vb2_dc_buf { void *vaddr; unsigned long size; dma_addr_t dma_addr; + struct sg_table *dma_sgt; + enum dma_data_direction dma_dir; /* MMAP related */ struct vb2_vmarea_handler handler; atomic_t refcount; + struct sg_table *sgt_base; /* USERPTR related */ struct vm_area_struct *vma; }; /*********************************************/ +/* scatterlist table functions */ +/*********************************************/ + +static struct sg_table *vb2_dc_pages_to_sgt(struct page **pages, + unsigned long n_pages, size_t offset, size_t offset2) +{ + struct sg_table *sgt; + int i, j; /* loop counters */ + int cur_page, chunks; + int ret; + struct scatterlist *s; + + sgt = kzalloc(sizeof *sgt, GFP_KERNEL); + if (!sgt) + return ERR_PTR(-ENOMEM); + + /* compute number of chunks */ + chunks = 1; + for (i = 1; i < n_pages; ++i) + if (pages[i] != pages[i - 1] + 1) + ++chunks; + + ret = sg_alloc_table(sgt, chunks, GFP_KERNEL); + if (ret) { + kfree(sgt); + return ERR_PTR(-ENOMEM); + } + + /* merging chunks and putting them into the scatterlist */ + cur_page = 0; + for_each_sg(sgt->sgl, s, sgt->orig_nents, i) { + size_t size = PAGE_SIZE; + + for (j = cur_page + 1; j < n_pages; ++j) { + if (pages[j] != pages[j - 1] + 1) + break; + size += PAGE_SIZE; + } + + /* cut offset if chunk starts at the first page */ + if (cur_page == 0) + size -= offset; + /* cut offset2 if chunk ends at the last page */ + if (j == n_pages) + size -= offset2; + + sg_set_page(s, pages[cur_page], size, offset); + offset = 0; + cur_page = j; + } + + return sgt; +} + +static void vb2_dc_release_sgtable(struct sg_table *sgt) +{ + sg_free_table(sgt); + kfree(sgt); +} + +static void vb2_dc_put_sgtable(struct sg_table *sgt, int dirty) +{ + struct scatterlist *s; + int i, j; + + for_each_sg(sgt->sgl, s, sgt->nents, i) { + struct page *page = sg_page(s); + int n_pages = PAGE_ALIGN(s->offset + s->length) >> PAGE_SHIFT; + + for (j = 0; j < n_pages; ++j, ++page) { + if (dirty) + set_page_dirty_lock(page); + put_page(page); + } + } + + vb2_dc_release_sgtable(sgt); +} + +static unsigned long vb2_dc_get_contiguous_size(struct sg_table *sgt) +{ + struct scatterlist *s; + dma_addr_t expected = sg_dma_address(sgt->sgl); + int i; + unsigned long size = 0; + + for_each_sg(sgt->sgl, s, sgt->nents, i) { + if (sg_dma_address(s) != expected) + break; + expected = sg_dma_address(s) + sg_dma_len(s); + size += sg_dma_len(s); + } + return size; +} + +/*********************************************/ /* callbacks for all buffers */ /*********************************************/ @@ -45,8 +147,6 @@ static void *vb2_dc_cookie(void *buf_priv) static void *vb2_dc_vaddr(void *buf_priv) { struct vb2_dc_buf *buf = buf_priv; - if (!buf) - return 0; return buf->vaddr; } @@ -58,6 +158,28 @@ static unsigned int vb2_dc_num_users(void *buf_priv) return atomic_read(&buf->refcount); } +static void vb2_dc_prepare(void *buf_priv) +{ + struct vb2_dc_buf *buf = buf_priv; + struct sg_table *sgt = buf->dma_sgt; + + if (!sgt) + return; + + dma_sync_sg_for_device(buf->dev, sgt->sgl, sgt->nents, buf->dma_dir); +} + +static void vb2_dc_finish(void *buf_priv) +{ + struct vb2_dc_buf *buf = buf_priv; + struct sg_table *sgt = buf->dma_sgt; + + if (!sgt) + return; + + dma_sync_sg_for_cpu(buf->dev, sgt->sgl, sgt->nents, buf->dma_dir); +} + /*********************************************/ /* callbacks for MMAP buffers */ /*********************************************/ @@ -66,31 +188,70 @@ static void vb2_dc_put(void *buf_priv) { struct vb2_dc_buf *buf = buf_priv; - if (atomic_dec_and_test(&buf->refcount)) { - dma_free_coherent(buf->dev, buf->size, buf->vaddr, - buf->dma_addr); - kfree(buf); - } + if (!atomic_dec_and_test(&buf->refcount)) + return; + + vb2_dc_release_sgtable(buf->sgt_base); + dma_free_coherent(buf->dev, buf->size, buf->vaddr, + buf->dma_addr); + kfree(buf); } static void *vb2_dc_alloc(void *alloc_ctx, unsigned long size) { struct device *dev = alloc_ctx; struct vb2_dc_buf *buf; + int ret; + int n_pages; + struct page **pages = NULL; buf = kzalloc(sizeof *buf, GFP_KERNEL); if (!buf) return ERR_PTR(-ENOMEM); - buf->vaddr = dma_alloc_coherent(dev, size, &buf->dma_addr, GFP_KERNEL); + buf->dev = dev; + buf->size = size; + buf->vaddr = dma_alloc_coherent(buf->dev, buf->size, &buf->dma_addr, + GFP_KERNEL); + + ret = -ENOMEM; if (!buf->vaddr) { - dev_err(dev, "dma_alloc_coherent of size %ld failed\n", size); - kfree(buf); - return ERR_PTR(-ENOMEM); + dev_err(dev, "dma_alloc_coherent of size %ld failed\n", + size); + goto fail_buf; } - buf->dev = dev; - buf->size = size; + WARN_ON((unsigned long)buf->vaddr & ~PAGE_MASK); + WARN_ON(buf->dma_addr & ~PAGE_MASK); + + n_pages = PAGE_ALIGN(size) >> PAGE_SHIFT; + + pages = kmalloc(n_pages * sizeof pages[0], GFP_KERNEL); + if (!pages) { + printk(KERN_ERR "failed to alloc page table\n"); + goto fail_dma; + } + + ret = dma_get_pages(dev, buf->vaddr, buf->dma_addr, pages, n_pages); + if (ret < 0) { + printk(KERN_ERR "failed to get buffer pages from DMA API\n"); + goto fail_pages; + } + if (ret != n_pages) { + ret = -EFAULT; + printk(KERN_ERR "failed to get all pages from DMA API\n"); + goto fail_pages; + } + + buf->sgt_base = vb2_dc_pages_to_sgt(pages, n_pages, 0, 0); + if (IS_ERR(buf->sgt_base)) { + ret = PTR_ERR(buf->sgt_base); + printk(KERN_ERR "failed to prepare sg table\n"); + goto fail_pages; + } + + /* pages are no longer needed */ + kfree(pages); buf->handler.refcount = &buf->refcount; buf->handler.put = vb2_dc_put; @@ -99,59 +260,226 @@ static void *vb2_dc_alloc(void *alloc_ctx, unsigned long size) atomic_inc(&buf->refcount); return buf; + +fail_pages: + kfree(pages); + +fail_dma: + dma_free_coherent(buf->dev, buf->size, buf->vaddr, buf->dma_addr); + +fail_buf: + kfree(buf); + + return ERR_PTR(ret); } static int vb2_dc_mmap(void *buf_priv, struct vm_area_struct *vma) { struct vb2_dc_buf *buf = buf_priv; + int ret; + + /* + * dma_mmap_* uses vm_pgoff as in-buffer offset, but we want to + * map whole buffer + */ + vma->vm_pgoff = 0; + + ret = dma_mmap_writecombine(buf->dev, vma, buf->vaddr, + buf->dma_addr, buf->size); - if (!buf) { - printk(KERN_ERR "No buffer to map\n"); - return -EINVAL; + if (ret) { + printk(KERN_ERR "Remapping memory failed, error: %d\n", ret); + return ret; } - return vb2_mmap_pfn_range(vma, buf->dma_addr, buf->size, - &vb2_common_vm_ops, &buf->handler); + vma->vm_flags |= VM_DONTEXPAND | VM_RESERVED; + vma->vm_private_data = &buf->handler; + vma->vm_ops = &vb2_common_vm_ops; + + vma->vm_ops->open(vma); + + printk(KERN_DEBUG "%s: mapped dma addr 0x%08lx at 0x%08lx, size %ld\n", + __func__, (unsigned long)buf->dma_addr, vma->vm_start, + buf->size); + + return 0; } /*********************************************/ /* callbacks for USERPTR buffers */ /*********************************************/ +static inline int vma_is_io(struct vm_area_struct *vma) +{ + return !!(vma->vm_flags & (VM_IO | VM_PFNMAP)); +} + +static int vb2_dc_get_pages(unsigned long start, struct page **pages, + int n_pages, struct vm_area_struct **copy_vma, int write) +{ + struct vm_area_struct *vma; + int n = 0; /* number of get pages */ + int ret = -EFAULT; + + /* entering critical section for mm access */ + down_read(¤t->mm->mmap_sem); + + vma = find_vma(current->mm, start); + if (!vma) { + printk(KERN_ERR "no vma for address %lu\n", start); + goto cleanup; + } + + if (vma_is_io(vma)) { + unsigned long pfn; + + if (vma->vm_end - start < n_pages * PAGE_SIZE) { + printk(KERN_ERR "vma is too small\n"); + goto cleanup; + } + + for (n = 0; n < n_pages; ++n, start += PAGE_SIZE) { + ret = follow_pfn(vma, start, &pfn); + if (ret) { + printk(KERN_ERR "no page for address %lu\n", + start); + goto cleanup; + } + pages[n] = pfn_to_page(pfn); + get_page(pages[n]); + } + } else { + n = get_user_pages(current, current->mm, start & PAGE_MASK, + n_pages, write, 1, pages, NULL); + if (n != n_pages) { + printk(KERN_ERR "got only %d of %d user pages\n", + n, n_pages); + goto cleanup; + } + } + + *copy_vma = vb2_get_vma(vma); + if (!*copy_vma) { + printk(KERN_ERR "failed to copy vma\n"); + ret = -ENOMEM; + goto cleanup; + } + + /* leaving critical section for mm access */ + up_read(¤t->mm->mmap_sem); + + return 0; + +cleanup: + up_read(¤t->mm->mmap_sem); + + /* putting user pages if used, can be done wothout the lock */ + while (n) + put_page(pages[--n]); + + return ret; +} + static void *vb2_dc_get_userptr(void *alloc_ctx, unsigned long vaddr, - unsigned long size, int write) + unsigned long size, int write) { struct vb2_dc_buf *buf; - struct vm_area_struct *vma; - dma_addr_t dma_addr = 0; - int ret; + unsigned long start, end, offset, offset2; + struct page **pages; + int n_pages; + int ret = 0; + struct sg_table *sgt; + unsigned long contig_size; buf = kzalloc(sizeof *buf, GFP_KERNEL); if (!buf) return ERR_PTR(-ENOMEM); - ret = vb2_get_contig_userptr(vaddr, size, &vma, &dma_addr); + buf->dev = alloc_ctx; + buf->dma_dir = write ? DMA_FROM_DEVICE : DMA_TO_DEVICE; + + start = (unsigned long)vaddr & PAGE_MASK; + offset = (unsigned long)vaddr & ~PAGE_MASK; + end = PAGE_ALIGN((unsigned long)vaddr + size); + offset2 = end - (unsigned long)vaddr - size; + n_pages = (end - start) >> PAGE_SHIFT; + + pages = kmalloc(n_pages * sizeof pages[0], GFP_KERNEL); + if (!pages) { + ret = -ENOMEM; + printk(KERN_ERR "failed to allocate pages table\n"); + goto fail_buf; + } + + /* extract page list from userspace mapping */ + ret = vb2_dc_get_pages(start, pages, n_pages, &buf->vma, write); if (ret) { - printk(KERN_ERR "Failed acquiring VMA for vaddr 0x%08lx\n", - vaddr); - kfree(buf); - return ERR_PTR(ret); + printk(KERN_ERR "failed to get user pages\n"); + goto fail_pages; + } + + sgt = vb2_dc_pages_to_sgt(pages, n_pages, offset, offset2); + if (!sgt) { + printk(KERN_ERR "failed to create scatterlist table\n"); + ret = -ENOMEM; + goto fail_get_pages; } + /* pages are no longer needed */ + kfree(pages); + pages = NULL; + + sgt->nents = dma_map_sg(buf->dev, sgt->sgl, sgt->orig_nents, + buf->dma_dir); + if (sgt->nents <= 0) { + printk(KERN_ERR "failed to map scatterlist\n"); + ret = -EIO; + goto fail_sgt; + } + + contig_size = vb2_dc_get_contiguous_size(sgt); + if (contig_size < size) { + printk(KERN_ERR "contiguous mapping is too small %lu/%lu\n", + contig_size, size); + ret = -EFAULT; + goto fail_map_sg; + } + + buf->dma_addr = sg_dma_address(sgt->sgl); buf->size = size; - buf->dma_addr = dma_addr; - buf->vma = vma; + buf->dma_sgt = sgt; + + atomic_inc(&buf->refcount); return buf; + +fail_map_sg: + dma_unmap_sg(buf->dev, sgt->sgl, sgt->nents, buf->dma_dir); + +fail_sgt: + vb2_dc_put_sgtable(sgt, 0); + +fail_get_pages: + while (pages && n_pages) + put_page(pages[--n_pages]); + vb2_put_vma(buf->vma); + +fail_pages: + kfree(pages); /* kfree is NULL-proof */ + +fail_buf: + kfree(buf); + + return ERR_PTR(ret); } -static void vb2_dc_put_userptr(void *mem_priv) +static void vb2_dc_put_userptr(void *buf_priv) { - struct vb2_dc_buf *buf = mem_priv; - - if (!buf) - return; + struct vb2_dc_buf *buf = buf_priv; + struct sg_table *sgt = buf->dma_sgt; + dma_unmap_sg(buf->dev, sgt->sgl, sgt->orig_nents, buf->dma_dir); + vb2_dc_put_sgtable(sgt, !vma_is_io(buf->vma)); vb2_put_vma(buf->vma); kfree(buf); } @@ -168,6 +496,8 @@ const struct vb2_mem_ops vb2_dma_contig_memops = { .mmap = vb2_dc_mmap, .get_userptr = vb2_dc_get_userptr, .put_userptr = vb2_dc_put_userptr, + .prepare = vb2_dc_prepare, + .finish = vb2_dc_finish, .num_users = vb2_dc_num_users, }; EXPORT_SYMBOL_GPL(vb2_dma_contig_memops);