From patchwork Tue Mar 5 20:54:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Stultz X-Patchwork-Id: 159712 Delivered-To: patches@linaro.org Received: by 2002:a02:5cc1:0:0:0:0:0 with SMTP id w62csp5410368jad; Tue, 5 Mar 2019 12:54:45 -0800 (PST) X-Received: by 2002:a17:902:8f81:: with SMTP id z1mr3254331plo.265.1551819285313; Tue, 05 Mar 2019 12:54:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1551819285; cv=none; d=google.com; s=arc-20160816; b=VMHveXfxxq+0DyJNzvh2pbOErJ/zeZ4Vo9Q11eQwwYIHr2JHOabZPF6qOf0+6S3yp7 6Brr17BTfdqN1JLjwTo+/gVV94Xfa0sqSqjazPjrKm1AelVfYICcQZyJ+G24VkQIAIde rinZiFypVSFT9zwxMUe8Mhe20zZ+epp8TxXuX1EozNtajy1MiuioxtJHUY3UboE1FRFu ybmBER80VJtuOmDiQomatiTxDTgAAyek2gLA+RWEswlY8dN4sdeLnJGRm7TIdMbEZOhI UqIiz4VXAwDHzpedE/3h1gIMFiE9RxRt5EXocCaTkgqX3alDSwF8A8aX2o1G7cG4FNPK WtDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=V4gbFKLyjI3IwmyANMx9RTxgAjbN5gJRajPVnQsoKow=; b=WSpnzZ5UTK0XD30UiuzT92sljvpyivrjI96BBrLsKBwjpmqej0cb77MAmUWtprVupy YMn5f2ul7/jrzaZ13CdBPbR6o50yur1/Er8ljUgJve6L7qTvXgpEzRVoAwB00OK0DP0t vzXiW4Qwq/5Q4t1Ygk9T7n8LEgUAdnF9eZhwfFAGIqXT2h+lWyZVwxRCiwwI2oODltdp xcRuoO/gMw7yIAC4PSGn2LHaZC6mTmIQ9SExIkw8Ap6Y7q4A8LQUgvVQoA1gZaIZ+fX8 orgI2wytoTquZeSC34Tshc28yVt8fCop9yUns2NzK8gH1Ga7vdrSEmAeORT4fLHdSp2J vTuQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tQ3o7bDE; spf=pass (google.com: domain of john.stultz@linaro.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=john.stultz@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id j187sor15079460pge.20.2019.03.05.12.54.45 for (Google Transport Security); Tue, 05 Mar 2019 12:54:45 -0800 (PST) Received-SPF: pass (google.com: domain of john.stultz@linaro.org designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tQ3o7bDE; spf=pass (google.com: domain of john.stultz@linaro.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=john.stultz@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=V4gbFKLyjI3IwmyANMx9RTxgAjbN5gJRajPVnQsoKow=; b=tQ3o7bDEfeFHNNx9lxcsffx/ug2FlH6XBA80njihZ82Qubnsm9gFWpTOt3ssLsgmke NVoHUmU82mFiPJkh9x+SrBQOwCRNNdbmJ5F47VB2OKwuWtoFa8Z1ykd9aryppldlnhcV RTgJS41k57yo0St04Wff+xyrcDmVD8sLa10X1fxerw9NcgfUjOdne2Q4kiztqENL80wW dRxtZtc2vxLK2D7ym5tjbbkF//mBdrWXByMIz8bu1oLhUsNKfh0z1aAhpABTE0cAu3Tw EpIgqIT5HXIqN35clE+c4O4XmtdPoqy3sVU5WWVZVdFeEiBueZbmohXH82F9LEMt/c9x e1Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=V4gbFKLyjI3IwmyANMx9RTxgAjbN5gJRajPVnQsoKow=; b=Ggy4aDYKdFT9f0zBJIY7wIPm5kXlkQzgWz4rUJxKw1EuCzPLPzO1spqrue2seENTJ5 lXTdpDpndgSlHjFIqcgvexEpZoVP6HvR2Jm/LQLzzuUDOo8ZlRfnPcXbfs0RhcBZWUGa yg0MSJhDPBFeCTuDLxt+hKoCuacO4i4UoTxn1OwnmO4JjDdlPypC3H1fDMr2jV5K3iK7 iu7NCWk+qQciwT4D6X5jNgIrWEqefWelp8dzPiOxFaH8wcpM4QhyqQiUgQ32Nh7K/the DBL5FrCumBIbbPIDZgNKrAaYeYxDT4O4obyASfmBVyZGjGnikEgaSS/ModoxnuFcU9xC qKTg== X-Gm-Message-State: APjAAAX/rcIVBB5p+cwDgt0hZMjXmMLj3NaqNUMC5gLc39hyaO7sXH9n jVAGGzNEGyycDGLhjNSsLf4CToi+ X-Google-Smtp-Source: APXvYqwjoDN0TV9qV8dX52oCEtTS7B1X2+9fRWiLb/GYUrddBubzSbk3CGQdr1xikKUKpm7vwp+aIg== X-Received: by 2002:a65:6241:: with SMTP id q1mr3146039pgv.340.1551819284810; Tue, 05 Mar 2019 12:54:44 -0800 (PST) Return-Path: Received: from localhost.localdomain ([2601:1c2:680:1319:4e72:b9ff:fe99:466a]) by smtp.gmail.com with ESMTPSA id i4sm13411788pfo.158.2019.03.05.12.54.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 05 Mar 2019 12:54:43 -0800 (PST) From: John Stultz To: lkml Cc: John Stultz , Laura Abbott , Benjamin Gaignard , Greg KH , Sumit Semwal , Liam Mark , Brian Starkey , "Andrew F . Davis" , Chenbo Feng , Alistair Strachan , dri-devel@lists.freedesktop.org Subject: [RFC][PATCH 2/5 v2] dma-buf: heaps: Add heap helpers Date: Tue, 5 Mar 2019 12:54:30 -0800 Message-Id: <1551819273-640-3-git-send-email-john.stultz@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1551819273-640-1-git-send-email-john.stultz@linaro.org> References: <1551819273-640-1-git-send-email-john.stultz@linaro.org> Add generic helper dmabuf ops for dma heaps, so we can reduce the amount of duplicative code for the exported dmabufs. This code is an evolution of the Android ION implementation, so thanks to its original authors and maintainters: Rebecca Schultz Zavin, Colin Cross, Laura Abbott, and others! Cc: Laura Abbott Cc: Benjamin Gaignard Cc: Greg KH Cc: Sumit Semwal Cc: Liam Mark Cc: Brian Starkey Cc: Andrew F. Davis Cc: Chenbo Feng Cc: Alistair Strachan Cc: dri-devel@lists.freedesktop.org Signed-off-by: John Stultz --- v2: * Removed cache management performance hack that I had accidentally folded in. * Removed stats code that was in helpers * Lots of checkpatch cleanups --- drivers/dma-buf/Makefile | 1 + drivers/dma-buf/heaps/Makefile | 2 + drivers/dma-buf/heaps/heap-helpers.c | 335 +++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/heap-helpers.h | 48 +++++ 4 files changed, 386 insertions(+) create mode 100644 drivers/dma-buf/heaps/Makefile create mode 100644 drivers/dma-buf/heaps/heap-helpers.c create mode 100644 drivers/dma-buf/heaps/heap-helpers.h -- 2.7.4 diff --git a/drivers/dma-buf/Makefile b/drivers/dma-buf/Makefile index b0332f1..09c2f2d 100644 --- a/drivers/dma-buf/Makefile +++ b/drivers/dma-buf/Makefile @@ -1,4 +1,5 @@ obj-y := dma-buf.o dma-fence.o dma-fence-array.o reservation.o seqno-fence.o +obj-$(CONFIG_DMABUF_HEAPS) += heaps/ obj-$(CONFIG_DMABUF_HEAPS) += dma-heap.o obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.o diff --git a/drivers/dma-buf/heaps/Makefile b/drivers/dma-buf/heaps/Makefile new file mode 100644 index 0000000..de49898 --- /dev/null +++ b/drivers/dma-buf/heaps/Makefile @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-y += heap-helpers.o diff --git a/drivers/dma-buf/heaps/heap-helpers.c b/drivers/dma-buf/heaps/heap-helpers.c new file mode 100644 index 0000000..ae5e9d0 --- /dev/null +++ b/drivers/dma-buf/heaps/heap-helpers.c @@ -0,0 +1,335 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include +#include +#include +#include + +#include "heap-helpers.h" + + +static void *dma_heap_map_kernel(struct heap_helper_buffer *buffer) +{ + struct scatterlist *sg; + int i, j; + void *vaddr; + pgprot_t pgprot; + struct sg_table *table = buffer->sg_table; + int npages = PAGE_ALIGN(buffer->heap_buffer.size) / PAGE_SIZE; + struct page **pages = vmalloc(array_size(npages, + sizeof(struct page *))); + struct page **tmp = pages; + + if (!pages) + return ERR_PTR(-ENOMEM); + + pgprot = PAGE_KERNEL; + + for_each_sg(table->sgl, sg, table->nents, i) { + int npages_this_entry = PAGE_ALIGN(sg->length) / PAGE_SIZE; + struct page *page = sg_page(sg); + + WARN_ON(i >= npages); + for (j = 0; j < npages_this_entry; j++) + *(tmp++) = page++; + } + vaddr = vmap(pages, npages, VM_MAP, pgprot); + vfree(pages); + + if (!vaddr) + return ERR_PTR(-ENOMEM); + + return vaddr; +} + +static int dma_heap_map_user(struct heap_helper_buffer *buffer, + struct vm_area_struct *vma) +{ + struct sg_table *table = buffer->sg_table; + unsigned long addr = vma->vm_start; + unsigned long offset = vma->vm_pgoff * PAGE_SIZE; + struct scatterlist *sg; + int i; + int ret; + + for_each_sg(table->sgl, sg, table->nents, i) { + struct page *page = sg_page(sg); + unsigned long remainder = vma->vm_end - addr; + unsigned long len = sg->length; + + if (offset >= sg->length) { + offset -= sg->length; + continue; + } else if (offset) { + page += offset / PAGE_SIZE; + len = sg->length - offset; + offset = 0; + } + len = min(len, remainder); + ret = remap_pfn_range(vma, addr, page_to_pfn(page), len, + vma->vm_page_prot); + if (ret) + return ret; + addr += len; + if (addr >= vma->vm_end) + return 0; + } + + return 0; +} + + +void dma_heap_buffer_destroy(struct dma_heap_buffer *heap_buffer) +{ + struct heap_helper_buffer *buffer = to_helper_buffer(heap_buffer); + + if (buffer->kmap_cnt > 0) { + pr_warn_once("%s: buffer still mapped in the kernel\n", + __func__); + vunmap(buffer->vaddr); + } + + buffer->free(buffer); +} + +static void *dma_heap_buffer_kmap_get(struct dma_heap_buffer *heap_buffer) +{ + struct heap_helper_buffer *buffer = to_helper_buffer(heap_buffer); + void *vaddr; + + if (buffer->kmap_cnt) { + buffer->kmap_cnt++; + return buffer->vaddr; + } + vaddr = dma_heap_map_kernel(buffer); + if (WARN_ONCE(!vaddr, + "heap->ops->map_kernel should return ERR_PTR on error")) + return ERR_PTR(-EINVAL); + if (IS_ERR(vaddr)) + return vaddr; + buffer->vaddr = vaddr; + buffer->kmap_cnt++; + return vaddr; +} + +static void dma_heap_buffer_kmap_put(struct dma_heap_buffer *heap_buffer) +{ + struct heap_helper_buffer *buffer = to_helper_buffer(heap_buffer); + + buffer->kmap_cnt--; + if (!buffer->kmap_cnt) { + vunmap(buffer->vaddr); + buffer->vaddr = NULL; + } +} + +static struct sg_table *dup_sg_table(struct sg_table *table) +{ + struct sg_table *new_table; + int ret, i; + struct scatterlist *sg, *new_sg; + + new_table = kzalloc(sizeof(*new_table), GFP_KERNEL); + if (!new_table) + return ERR_PTR(-ENOMEM); + + ret = sg_alloc_table(new_table, table->nents, GFP_KERNEL); + if (ret) { + kfree(new_table); + return ERR_PTR(-ENOMEM); + } + + new_sg = new_table->sgl; + for_each_sg(table->sgl, sg, table->nents, i) { + memcpy(new_sg, sg, sizeof(*sg)); + new_sg->dma_address = 0; + new_sg = sg_next(new_sg); + } + + return new_table; +} + +static void free_duped_table(struct sg_table *table) +{ + sg_free_table(table); + kfree(table); +} + +struct dma_heaps_attachment { + struct device *dev; + struct sg_table *table; + struct list_head list; +}; + +static int dma_heap_attach(struct dma_buf *dmabuf, + struct dma_buf_attachment *attachment) +{ + struct dma_heaps_attachment *a; + struct sg_table *table; + struct dma_heap_buffer *heap_buffer = dmabuf->priv; + struct heap_helper_buffer *buffer = to_helper_buffer(heap_buffer); + + a = kzalloc(sizeof(*a), GFP_KERNEL); + if (!a) + return -ENOMEM; + + table = dup_sg_table(buffer->sg_table); + if (IS_ERR(table)) { + kfree(a); + return -ENOMEM; + } + + a->table = table; + a->dev = attachment->dev; + INIT_LIST_HEAD(&a->list); + + attachment->priv = a; + + mutex_lock(&buffer->lock); + list_add(&a->list, &buffer->attachments); + mutex_unlock(&buffer->lock); + + return 0; +} + +static void dma_heap_detatch(struct dma_buf *dmabuf, + struct dma_buf_attachment *attachment) +{ + struct dma_heaps_attachment *a = attachment->priv; + struct dma_heap_buffer *heap_buffer = dmabuf->priv; + struct heap_helper_buffer *buffer = to_helper_buffer(heap_buffer); + + mutex_lock(&buffer->lock); + list_del(&a->list); + mutex_unlock(&buffer->lock); + free_duped_table(a->table); + + kfree(a); +} + +static struct sg_table *dma_heap_map_dma_buf( + struct dma_buf_attachment *attachment, + enum dma_data_direction direction) +{ + struct dma_heaps_attachment *a = attachment->priv; + struct sg_table *table; + + table = a->table; + + if (!dma_map_sg(attachment->dev, table->sgl, table->nents, + direction)) + table = ERR_PTR(-ENOMEM); + return table; +} + +static void dma_heap_unmap_dma_buf(struct dma_buf_attachment *attachment, + struct sg_table *table, + enum dma_data_direction direction) +{ + dma_unmap_sg(attachment->dev, table->sgl, table->nents, direction); +} + +static int dma_heap_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma) +{ + struct dma_heap_buffer *heap_buffer = dmabuf->priv; + struct heap_helper_buffer *buffer = to_helper_buffer(heap_buffer); + int ret = 0; + + mutex_lock(&buffer->lock); + /* now map it to userspace */ + ret = dma_heap_map_user(buffer, vma); + mutex_unlock(&buffer->lock); + + if (ret) + pr_err("%s: failure mapping buffer to userspace\n", + __func__); + + return ret; +} + +static void dma_heap_dma_buf_release(struct dma_buf *dmabuf) +{ + struct dma_heap_buffer *buffer = dmabuf->priv; + + dma_heap_buffer_destroy(buffer); +} + +static void *dma_heap_dma_buf_kmap(struct dma_buf *dmabuf, + unsigned long offset) +{ + struct dma_heap_buffer *heap_buffer = dmabuf->priv; + struct heap_helper_buffer *buffer = to_helper_buffer(heap_buffer); + + return buffer->vaddr + offset * PAGE_SIZE; +} + +static void dma_heap_dma_buf_kunmap(struct dma_buf *dmabuf, + unsigned long offset, + void *ptr) +{ +} + +static int dma_heap_dma_buf_begin_cpu_access(struct dma_buf *dmabuf, + enum dma_data_direction direction) +{ + struct dma_heap_buffer *heap_buffer = dmabuf->priv; + struct heap_helper_buffer *buffer = to_helper_buffer(heap_buffer); + void *vaddr; + struct dma_heaps_attachment *a; + int ret = 0; + + mutex_lock(&buffer->lock); + vaddr = dma_heap_buffer_kmap_get(heap_buffer); + if (IS_ERR(vaddr)) { + ret = PTR_ERR(vaddr); + goto unlock; + } + mutex_unlock(&buffer->lock); + + mutex_lock(&buffer->lock); + list_for_each_entry(a, &buffer->attachments, list) { + dma_sync_sg_for_cpu(a->dev, a->table->sgl, a->table->nents, + direction); + } + +unlock: + mutex_unlock(&buffer->lock); + return ret; +} + +static int dma_heap_dma_buf_end_cpu_access(struct dma_buf *dmabuf, + enum dma_data_direction direction) +{ + struct dma_heap_buffer *heap_buffer = dmabuf->priv; + struct heap_helper_buffer *buffer = to_helper_buffer(heap_buffer); + struct dma_heaps_attachment *a; + + mutex_lock(&buffer->lock); + dma_heap_buffer_kmap_put(heap_buffer); + mutex_unlock(&buffer->lock); + + mutex_lock(&buffer->lock); + list_for_each_entry(a, &buffer->attachments, list) { + dma_sync_sg_for_device(a->dev, a->table->sgl, a->table->nents, + direction); + } + mutex_unlock(&buffer->lock); + + return 0; +} + +const struct dma_buf_ops heap_helper_ops = { + .map_dma_buf = dma_heap_map_dma_buf, + .unmap_dma_buf = dma_heap_unmap_dma_buf, + .mmap = dma_heap_mmap, + .release = dma_heap_dma_buf_release, + .attach = dma_heap_attach, + .detach = dma_heap_detatch, + .begin_cpu_access = dma_heap_dma_buf_begin_cpu_access, + .end_cpu_access = dma_heap_dma_buf_end_cpu_access, + .map = dma_heap_dma_buf_kmap, + .unmap = dma_heap_dma_buf_kunmap, +}; diff --git a/drivers/dma-buf/heaps/heap-helpers.h b/drivers/dma-buf/heaps/heap-helpers.h new file mode 100644 index 0000000..0bd8643 --- /dev/null +++ b/drivers/dma-buf/heaps/heap-helpers.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * DMABUF Heaps helper code + * + * Copyright (C) 2011 Google, Inc. + * Copyright (C) 2019 Linaro Ltd. + */ + +#ifndef _HEAP_HELPERS_H +#define _HEAP_HELPERS_H + +#include +#include + +struct heap_helper_buffer { + struct dma_heap_buffer heap_buffer; + + unsigned long private_flags; + void *priv_virt; + struct mutex lock; + int kmap_cnt; + void *vaddr; + struct sg_table *sg_table; + struct list_head attachments; + + void (*free)(struct heap_helper_buffer *buffer); + +}; + +#define to_helper_buffer(x) \ + container_of(x, struct heap_helper_buffer, heap_buffer) + +static inline void INIT_HEAP_HELPER_BUFFER(struct heap_helper_buffer *buffer, + void (*free)(struct heap_helper_buffer *)) +{ + buffer->private_flags = 0; + buffer->priv_virt = NULL; + mutex_init(&buffer->lock); + buffer->kmap_cnt = 0; + buffer->vaddr = NULL; + buffer->sg_table = NULL; + INIT_LIST_HEAD(&buffer->attachments); + buffer->free = free; +} + +extern const struct dma_buf_ops heap_helper_ops; + +#endif /* _HEAP_HELPERS_H */