[v2,2/2] dma-buf: heaps: Give default CMA heap a fixed name

The CMA heap's name in devtmpfs can vary depending on how the heap is
defined. Its name defaults to "reserved", but if a CMA area is defined
in the devicetree, the heap takes on the devicetree node's name, such as
"default-pool" or "linux,cma". To simplify naming, just name it
"default_cma", and keep a legacy node in place backed by the same
underlying structure for backwards compatibility.

Signed-off-by: Jared Kangas <jkangas@redhat.com>
---
 Documentation/userspace-api/dma-buf-heaps.rst | 11 +++++++----
 drivers/dma-buf/heaps/Kconfig                 | 10 ++++++++++
 drivers/dma-buf/heaps/cma_heap.c              | 14 +++++++++++++-
 3 files changed, 30 insertions(+), 5 deletions(-)

On Thu, Apr 24, 2025 at 11:58 PM Maxime Ripard <mripard@kernel.org> wrote:
> On Thu, Apr 24, 2025 at 05:13:47PM -0700, John Stultz wrote:
> > On Thu, Apr 24, 2025 at 1:34 AM Maxime Ripard <mripard@kernel.org> wrote:
> > > I appreciate this is kind of bikeshed-color territory, but I think "cma"
> > > would be a better option here. There's nothing "default" about it.
> >
> > I disagree.  It very much is "default" as it's returning the
> > dma_contiguous_default_area.
>
> My main concern here is that it's "default" as opposed to what, exactly?
> We have a single CMA allocator. We could have multiple buffer
> attributes, but then "cached_cma" would make more sense to me if we
> expect to have uncached CMA allocations at some point.

Well, there may be one CMA allocator, but there can be multiple CMA regions.

So in the kernel, cma_alloc() always takes the cma area as an
argument.  And dma_alloc_contiguous() lets you do allocations against
a device, which may reference a specific cma area. Or if the device
doesn't specify a region it will utilize the default region.

> > There can be multiple CMA areas, and out of tree, vendors do reserve
> > separate areas for specific purposes, exposing multiple CMA dmabuf
> > heaps.
>
> By "CMA areas", I guess you mean carved-out memory regions? If so, how
> is it relevant to userspace if we use CMA or any other implementation to
> expose a carved-out region, and thus that we carry that implemenattion
> detail in the name?

So, no, I don't mean carve-out regions.  It's more about dealing with
competition between multiple CMA users. In some cases, where there are
known fixed buffer sizes, say camera buffers, it's much easier to
reserve a separate specific sized region to allocate from so that you
know it will always succeed and you don't need to waste much on safety
margins. Having this added as a separate CMA region makes it a lot
easier to account or reason about, and the kernel can still make
(limited) use of the CMA space when it's idle. Then you don't have to
worry about some other device having a short term cma allocation that
pushes back the alignment for your large allocation, possibly
impacting some other devices larger allocations.

And unlike with just using a carveout, you don't end up just wasting
all that space when it is unused.

So userland may want to allocate contiguous memory, but it may also be
relevant to userland to be able to allocate contiguous memory from a
purpose specific pool.

And while not used in Android, you could imagine having separate
purpose reserved cma heaps with different permissions on the heap
devnodes, allowing less trusted applications to allocate cma from a
small pool without having the potential to DoS the system.

> > There have been patches to expose multiple CMA heaps, but with no
> > upstream drivers using those purpose specific regions, we haven't
> > taken them yet.
> > I do hope as the drivers that utilize these purpose focused heaps go
> > upstream, we can add that logic, so I think being specific that this
> > is default CMA is a good idea.
>
> If heaps names are supposed to carry the region it exposes, then it
> should be default_cma_region/area. If heap names are supposed to expose
> the allocator (but I don't think it's a good idea), it should be cma. If
> they are meant to carry all that plus some policy,
> cached_default_cma_region should be used.
>
> Either way, default_cma seems to me either too specific or not specific
> enough. And we should really document what the policy for those heaps
> are supposed to be.

I don't see it as such a problem. It is clear it is cma, it also is
clear conceptually that it is the "default" region that the kernel
uses when devices aren't specific.
But I wouldn't object to cma_default_region/area as a name either, but
I don't see it as particularly improved over cma_default.

To your larger point about policy, I do get the tension that you want
to be able to programmatically derive or evaluate heap names, so that
applications can consistently derive a pathname to get what they want.
But I also think that there is so much variety in both the devices and
uses that there is no way that all use cases and all devices can be
satisfied with such a static or even programmatic mapping. From my
perspective, there just is going to have to be some device specific
glue logic that maps use->heap name. Same reason we have fstab and the
passwd file.  That said, I think advocating for naming conventions is
definitely useful, but I'm wary of trying to enforce too specific a
schema on the names as the incompleteness theorem will bite us.

thanks
-john

Hi John,

On Fri, Apr 25, 2025 at 12:39:40PM -0700, John Stultz wrote:
> On Thu, Apr 24, 2025 at 11:58 PM Maxime Ripard <mripard@kernel.org> wrote:
> > On Thu, Apr 24, 2025 at 05:13:47PM -0700, John Stultz wrote:
> > > On Thu, Apr 24, 2025 at 1:34 AM Maxime Ripard <mripard@kernel.org> wrote:
> > > > I appreciate this is kind of bikeshed-color territory, but I think "cma"
> > > > would be a better option here. There's nothing "default" about it.
> > >
> > > I disagree.  It very much is "default" as it's returning the
> > > dma_contiguous_default_area.
> >
> > My main concern here is that it's "default" as opposed to what, exactly?
> > We have a single CMA allocator. We could have multiple buffer
> > attributes, but then "cached_cma" would make more sense to me if we
> > expect to have uncached CMA allocations at some point.
> 
> Well, there may be one CMA allocator, but there can be multiple CMA regions.
> 
> So in the kernel, cma_alloc() always takes the cma area as an
> argument.  And dma_alloc_contiguous() lets you do allocations against
> a device, which may reference a specific cma area. Or if the device
> doesn't specify a region it will utilize the default region.
> 
> > > There can be multiple CMA areas, and out of tree, vendors do reserve
> > > separate areas for specific purposes, exposing multiple CMA dmabuf
> > > heaps.
> >
> > By "CMA areas", I guess you mean carved-out memory regions? If so, how
> > is it relevant to userspace if we use CMA or any other implementation to
> > expose a carved-out region, and thus that we carry that implemenattion
> > detail in the name?
> 
> So, no, I don't mean carve-out regions.  It's more about dealing with
> competition between multiple CMA users. In some cases, where there are
> known fixed buffer sizes, say camera buffers, it's much easier to
> reserve a separate specific sized region to allocate from so that you
> know it will always succeed and you don't need to waste much on safety
> margins. Having this added as a separate CMA region makes it a lot
> easier to account or reason about, and the kernel can still make
> (limited) use of the CMA space when it's idle. Then you don't have to
> worry about some other device having a short term cma allocation that
> pushes back the alignment for your large allocation, possibly
> impacting some other devices larger allocations.
> 
> And unlike with just using a carveout, you don't end up just wasting
> all that space when it is unused.

The way I see it, it's an implementation detail and is abstracted away
from userspace. That's what I meant by carved-out I guess: a region
dedicated to a (set of) devices(s) that the rest of the system won't
use from userspace point of view.

> So userland may want to allocate contiguous memory, but it may also be
> relevant to userland to be able to allocate contiguous memory from a
> purpose specific pool.
> 
> And while not used in Android, you could imagine having separate
> purpose reserved cma heaps with different permissions on the heap
> devnodes, allowing less trusted applications to allocate cma from a
> small pool without having the potential to DoS the system.

Yeah... I don't think it's the right approach for that. If Android
doesn't use it, and if it's the only Linux distro with 1 app / 1 user
policy, then the only permissions we'll effectively have is one for the
whole use, trusted and untrusted apps alike.

cgroups look like a much better path forward, and wouldn't require
multiple heaps.

Anyway... It's not really important at this point I guess.

> > > There have been patches to expose multiple CMA heaps, but with no
> > > upstream drivers using those purpose specific regions, we haven't
> > > taken them yet.
> > > I do hope as the drivers that utilize these purpose focused heaps go
> > > upstream, we can add that logic, so I think being specific that this
> > > is default CMA is a good idea.
> >
> > If heaps names are supposed to carry the region it exposes, then it
> > should be default_cma_region/area. If heap names are supposed to expose
> > the allocator (but I don't think it's a good idea), it should be cma. If
> > they are meant to carry all that plus some policy,
> > cached_default_cma_region should be used.
> >
> > Either way, default_cma seems to me either too specific or not specific
> > enough. And we should really document what the policy for those heaps
> > are supposed to be.
> 
> I don't see it as such a problem. It is clear it is cma, it also is
> clear conceptually that it is the "default" region that the kernel
> uses when devices aren't specific.
> But I wouldn't object to cma_default_region/area as a name either, but
> I don't see it as particularly improved over cma_default.
> 
> To your larger point about policy, I do get the tension that you want
> to be able to programmatically derive or evaluate heap names, so that
> applications can consistently derive a pathname to get what they want.

We've discussed it in the past, I don't really want to. But it was clear
from the last discussion that you (plural) wanted to infer heap
semantics from the names. I'm ok with that, but then if we want to make
it work we need to have well defined names.

And it's actually what I really want to discuss here: we've discussed at
length how bad the heaps name are (and not only here), but I don't think
we have any documented policy on what makes a good name?

For example, I'm not sure exposing the allocator name is a good idea:
it's an implementation detail and for all userspace cares about, we
could change it every release if it provided the same kind of buffers.

Taking your camera buffers example before, then we could also expose a
memory region id, and let the platform figure it out, or use the usecase
as the name.

But if we don't document that, how can we possibly expect everyone
including downstream to come up with perfect names every time. And FTR,
I'm willing to write that doc down once the discussion settles.

> But I also think that there is so much variety in both the devices and
> uses that there is no way that all use cases and all devices can be
> satisfied with such a static or even programmatic mapping. From my
> perspective, there just is going to have to be some device specific
> glue logic that maps use->heap name. Same reason we have fstab and the
> passwd file.

fstab and passwd can be generated at (first) boot time / install. fstab
is also being somewhat less important with the auto-partition discovery.
How would you generate that configuration file at boot?

I'm not really asking this as a theoretical question. Being able to
auto-discover which heap a driver/device would allocate from is central
for the cgroup work I mentioned earlier.

And I'm really not sure how distros or applications developpers are
supposed to keep up with the raw volume of devices that go out every
year, each and every one of them having different heap names, etc.
Possibly different from one version of the firmware to another.

> That said, I think advocating for naming conventions is definitely
> useful, but I'm wary of trying to enforce too specific a schema on the
> names as the incompleteness theorem will bite us.

On Mon, Apr 28, 2025 at 7:52 AM Maxime Ripard <mripard@kernel.org> wrote:
> On Fri, Apr 25, 2025 at 12:39:40PM -0700, John Stultz wrote:
> > To your larger point about policy, I do get the tension that you want
> > to be able to programmatically derive or evaluate heap names, so that
> > applications can consistently derive a pathname to get what they want.
>
> We've discussed it in the past, I don't really want to. But it was clear
> from the last discussion that you (plural) wanted to infer heap
> semantics from the names. I'm ok with that, but then if we want to make
> it work we need to have well defined names.

So my name keeps on getting attached to that, but I don't think I was
involved in the LPC conversation when that got decided.

> And it's actually what I really want to discuss here: we've discussed at
> length how bad the heaps name are (and not only here), but I don't think
> we have any documented policy on what makes a good name?

I very much think having a policy/guidance for better names is a good goal.

I just want to make sure it doesn't become a strict policy that lead
folks to make mistaken assumptions about a static solution being
viable in userland (like folks nostalgicly using "eth0" or a fixed
network device name in scripts expecting it to work on a different
system)

> For example, I'm not sure exposing the allocator name is a good idea:
> it's an implementation detail and for all userspace cares about, we
> could change it every release if it provided the same kind of buffers.

That is a fair point.

> Taking your camera buffers example before, then we could also expose a
> memory region id, and let the platform figure it out, or use the usecase
> as the name.
>
> But if we don't document that, how can we possibly expect everyone
> including downstream to come up with perfect names every time. And FTR,
> I'm willing to write that doc down once the discussion settles.

So again, yeah, I very much support having better guidance on the names.

I think the number of device constraints and device combinations makes
a raw enumeration of things difficult.

This is why the per-device use->heap mapping "glue" seems necessary to me.

And, I do get that this runs into a similar problem with enumerating
and defining "uses" (which boil down to a combination of
devices-in-a-pipeline and access use patterns), but for Andorid it has
so far been manageable.

Personally, I think the best idea I've heard so far to resolve this
from userland was Christian's suggestion that devices expose links to
compatible heaps, and then userland without a use->heap mapping could
for the set of devices they plan to use in a pipeline, figure out the
common heap name and use that to allocate.

However, that pushes the problem down a bit, requiring drivers
(instead of userland) to know what heaps they can work with and what
the names might be (which again, your goal for standardizing the heap
names isn't a bad thing!). Though, this approach also runs into
trouble as it opens a question of: should it only encode strict
constraint satisfaction, or something more subtle, as while something
might work with multiple heaps, its possible it won't be performant
enough unless it picks a specific one on device A or a different one
on device B.  And getting that sort of device-specific details
embedded into a driver isn't great either.

> > But I also think that there is so much variety in both the devices and
> > uses that there is no way that all use cases and all devices can be
> > satisfied with such a static or even programmatic mapping. From my
> > perspective, there just is going to have to be some device specific
> > glue logic that maps use->heap name. Same reason we have fstab and the
> > passwd file.
>
> fstab and passwd can be generated at (first) boot time / install. fstab
> is also being somewhat less important with the auto-partition discovery.
> How would you generate that configuration file at boot?
>
> I'm not really asking this as a theoretical question. Being able to
> auto-discover which heap a driver/device would allocate from is central
> for the cgroup work I mentioned earlier.
>
> And I'm really not sure how distros or applications developpers are
> supposed to keep up with the raw volume of devices that go out every
> year, each and every one of them having different heap names, etc.
> Possibly different from one version of the firmware to another.

For generic distros, I don't have a good answer here. Historically the
focus has always been on single device usage, so having the driver do
the allocation was fine, and if you were using multiple devices you
could just copy the memory between the driver allocated buffers.  But
as we've moved to disaggregated IP blocks and device pipelines, all
those potential copies wreck performance and power.   I'm not sure
generic distros have the concept of a device pipeline very well
abstracted (obviously mesa and the wayland/X have had to deal with it,
and the video and camera side is dealing with it more and more).
Maybe a more established notion of use -> pipeline/device collections,
is needed as a starting point? Then using Christian's suggestion, one
could at least enumerate  use -> heap that would be functional. And
maybe device makers could then supplement explicit optimized mapping
overrides for their device?

I just think leaving individual applications (or even individual
frameworks like mesa) to embed assumptions about heap names ->
functionality is going to be a problematic approach.

thanks
-john