diff mbox series

[v2,1/2] arm64: dts: broadcom: clear the warnings caused by empty dma-ranges

Message ID 20201016090833.1892-2-thunder.leizhen@huawei.com
State Accepted
Commit 2013a4b684b6eb614ee5c9a3c07b0ae6f5ca96d9
Headers show
Series eliminate two common errors reported by any yaml on arm64 | expand

Commit Message

Leizhen (ThunderTown) Oct. 16, 2020, 9:08 a.m. UTC
The scripts/dtc/checks.c requires that the node have empty "dma-ranges"
property must have the same "#address-cells" and "#size-cells" values as
the parent node. Otherwise, the following warnings is reported:

arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning \
(dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but \
its #address-cells (1) differs from / (2)
arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning \
(dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but \
its #size-cells (1) differs from / (2)

Arnd Bergmann figured out why it's necessary:
Also note that the #address-cells=<1> means that any device under
this bus is assumed to only support 32-bit addressing, and DMA will
have to go through a slow swiotlb in the absence of an IOMMU.

Suggested-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
---
 .../boot/dts/broadcom/stingray/stingray-usb.dtsi     | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

Comments

Leizhen (ThunderTown) Oct. 18, 2020, 2:10 a.m. UTC | #1
On 2020/10/17 3:27, Florian Fainelli wrote:
> On 10/16/20 11:23 AM, Arnd Bergmann wrote:

>> On Fri, Oct 16, 2020 at 6:48 PM Florian Fainelli <f.fainelli@gmail.com> wrote:

>>> On 10/16/20 4:01 AM, Arnd Bergmann wrote:

>>>> On Fri, Oct 16, 2020 at 11:09 AM Zhen Lei <thunder.leizhen@huawei.com> wrote:

>>>>>

>>>>> Suggested-by: Arnd Bergmann <arnd@arndb.de>

>>>>> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>

>>>>

>>>> Acked-by: Arnd Bergmann <arnd@arndb.de>

>>>>

>>>> I see that at least the 'bcd' and 'xhci' devices in fact try to

>>>> use 64-bit DMA. It would be good to test this on actual

>>>> hardware to ensure that it works correctly when this is enabled.

>>>>

>>>> Ideally avoiding the swiotlb bounce buffering should only

>>>> make it faster here, but there are many chips on which

>>>> 64-bit DMA is broken in some form.

>>>

>>> Is this change really an improvement though? This 'usb' pseudo bus node

>>> could just keep being defined with #address-cells = <1> and #size-cells

>>> = <1> so as to satisfy the 'reg' definition however we could just adjust

>>> dma-ranges to indicate full 64-bit addressing capability. Would not that

>>> work?

>>

>> When #address-cells is '1', you cannot specify dma-ranges that

>> go beyond a 32-bit address range.

> 

> Would not it be enough to remove the 'dma-ranges' property though? Sorry

> for being slow here.


Remove the 'dma-ranges' property should also work. After all, it is equivalent
to the original empty dma-ranges scheme. In addition, since the IOMMU nodes are
defined, it should be enabled. Therefore, Arnd's concern about the scenario where
the IOMMU is disabled may not be triggered. All roads lead to Rome. Which solution
should be chose depends on individual preferences. For me, either solution is fine.

The third solution is to define a non-empty dma-ranges property. However, because
stingray-usb.dtsi is included in multiple files, it may not be appropriate.


>
Leizhen (ThunderTown) Nov. 9, 2020, 6:18 a.m. UTC | #2
Hi, everybody:
  How do we deal with this problem? I updated the kernel to the latest and the problem still persists.

  make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j24 dtbs 2>err.txt
  vim err.txt

arch/arm64/boot/dts/qcom/ipq6018.dtsi:185.3-14: Warning (dma_ranges_format): /soc:dma-ranges: empty "dma-ranges" property but its #address-cells (1) differs from / (2)
arch/arm64/boot/dts/qcom/ipq6018.dtsi:185.3-14: Warning (dma_ranges_format): /soc:dma-ranges: empty "dma-ranges" property but its #size-cells (1) differs from / (2)
arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #address-cells (1) differs from / (2)
arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #size-cells (1) differs from / (2)
arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #address-cells (1) differs from / (2)
arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #size-cells (1) differs from / (2)
arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #address-cells (1) differs from / (2)
arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #size-cells (1) differs from / (2)




On 2020/10/26 10:21, Leizhen (ThunderTown) wrote:
> 
> 
> On 2020/10/23 15:17, Arnd Bergmann wrote:
>> On Sun, Oct 18, 2020 at 4:10 AM Leizhen (ThunderTown)
>> <thunder.leizhen@huawei.com> wrote:
>>> On 2020/10/17 3:27, Florian Fainelli wrote:
>>>> On 10/16/20 11:23 AM, Arnd Bergmann wrote:
>>>>> On Fri, Oct 16, 2020 at 6:48 PM Florian Fainelli <f.fainelli@gmail.com> wrote:
>>>>>> On 10/16/20 4:01 AM, Arnd Bergmann wrote:
>>>>>>> On Fri, Oct 16, 2020 at 11:09 AM Zhen Lei <thunder.leizhen@huawei.com> wrote:
>>>>>>>>
>>>>>>>> Suggested-by: Arnd Bergmann <arnd@arndb.de>
>>>>>>>> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
>>>>>>>
>>>>>>> Acked-by: Arnd Bergmann <arnd@arndb.de>
>>>>>>>
>>>>>>> I see that at least the 'bcd' and 'xhci' devices in fact try to
>>>>>>> use 64-bit DMA. It would be good to test this on actual
>>>>>>> hardware to ensure that it works correctly when this is enabled.
>>>>>>>
>>>>>>> Ideally avoiding the swiotlb bounce buffering should only
>>>>>>> make it faster here, but there are many chips on which
>>>>>>> 64-bit DMA is broken in some form.
>>>>>>
>>>>>> Is this change really an improvement though? This 'usb' pseudo bus node
>>>>>> could just keep being defined with #address-cells = <1> and #size-cells
>>>>>> = <1> so as to satisfy the 'reg' definition however we could just adjust
>>>>>> dma-ranges to indicate full 64-bit addressing capability. Would not that
>>>>>> work?
>>>>>
>>>>> When #address-cells is '1', you cannot specify dma-ranges that
>>>>> go beyond a 32-bit address range.
>>>>
>>>> Would not it be enough to remove the 'dma-ranges' property though? Sorry
>>>> for being slow here.
>>>
>>> Remove the 'dma-ranges' property should also work. After all, it is equivalent
>>> to the original empty dma-ranges scheme. In addition, since the IOMMU nodes are
>>> defined, it should be enabled.
>>
>> Are you sure? I was expecting the IOMMU not to get used here since
>> the devices do contain list an 'iommus' property.
> 
> OK,If the SMMU maybe disabled, then your proposal is necessary.
> 
>>
>>       Arnd
>>
>> .
>>
Florian Fainelli Nov. 9, 2020, 5:28 p.m. UTC | #3
On 11/8/20 10:18 PM, Leizhen (ThunderTown) wrote:
> Hi, everybody:

>   How do we deal with this problem? I updated the kernel to the latest and the problem still persists.

> 

>   make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j24 dtbs 2>err.txt

>   vim err.txt

> 

> arch/arm64/boot/dts/qcom/ipq6018.dtsi:185.3-14: Warning (dma_ranges_format): /soc:dma-ranges: empty "dma-ranges" property but its #address-cells (1) differs from / (2)

> arch/arm64/boot/dts/qcom/ipq6018.dtsi:185.3-14: Warning (dma_ranges_format): /soc:dma-ranges: empty "dma-ranges" property but its #size-cells (1) differs from / (2)

> arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #address-cells (1) differs from / (2)

> arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #size-cells (1) differs from / (2)

> arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #address-cells (1) differs from / (2)

> arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #size-cells (1) differs from / (2)

> arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #address-cells (1) differs from / (2)

> arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #size-cells (1) differs from / (2)


I was hoping that Ray or Scott could give this a try, and TBH, I am
still not a big fan of having to mangle the 'reg' property to be
compatible with #address-cells = <2> and #size-cells = <2>, I would have
preferred omitting the 'dma-ranges' property entirely.

We have plenty of time to get this patch applied for v5.11.
-- 
Florian
Arnd Bergmann Nov. 9, 2020, 5:56 p.m. UTC | #4
On Mon, Nov 9, 2020 at 6:30 PM Florian Fainelli <f.fainelli@gmail.com> wrote:
>
> On 11/8/20 10:18 PM, Leizhen (ThunderTown) wrote:
> > Hi, everybody:
> >   How do we deal with this problem? I updated the kernel to the latest and the problem still persists.
> >
> >   make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j24 dtbs 2>err.txt
> >   vim err.txt
> >
> > arch/arm64/boot/dts/qcom/ipq6018.dtsi:185.3-14: Warning (dma_ranges_format): /soc:dma-ranges: empty "dma-ranges" property but its #address-cells (1) differs from / (2)
> > arch/arm64/boot/dts/qcom/ipq6018.dtsi:185.3-14: Warning (dma_ranges_format): /soc:dma-ranges: empty "dma-ranges" property but its #size-cells (1) differs from / (2)
> > arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #address-cells (1) differs from / (2)
> > arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #size-cells (1) differs from / (2)
> > arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #address-cells (1) differs from / (2)
> > arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #size-cells (1) differs from / (2)
> > arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #address-cells (1) differs from / (2)
> > arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but its #size-cells (1) differs from / (2)
>
> I was hoping that Ray or Scott could give this a try, and TBH, I am
> still not a big fan of having to mangle the 'reg' property to be
> compatible with #address-cells = <2> and #size-cells = <2>, I would have
> preferred omitting the 'dma-ranges' property entirely.
>
> We have plenty of time to get this patch applied for v5.11.

I would much prefer to get the warning fixed for v5.10, once we know what the
hardware can or cannot actually do, as it is one of the warnings that shows
up in every kernelci build.

      Arnd
Florian Fainelli Nov. 28, 2020, 4:53 a.m. UTC | #5
On Fri, 16 Oct 2020 17:08:32 +0800, Zhen Lei <thunder.leizhen@huawei.com> wrote:
> The scripts/dtc/checks.c requires that the node have empty "dma-ranges"

> property must have the same "#address-cells" and "#size-cells" values as

> the parent node. Otherwise, the following warnings is reported:

> 

> arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning \

> (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but \

> its #address-cells (1) differs from / (2)

> arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning \

> (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but \

> its #size-cells (1) differs from / (2)

> 

> Arnd Bergmann figured out why it's necessary:

> Also note that the #address-cells=<1> means that any device under

> this bus is assumed to only support 32-bit addressing, and DMA will

> have to go through a slow swiotlb in the absence of an IOMMU.

> 

> Suggested-by: Arnd Bergmann <arnd@arndb.de>

> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>

> ---


Applied to devicetree-arm64/next, thanks!
--
Florian
Arnd Bergmann Nov. 28, 2020, 9:58 a.m. UTC | #6
On Sat, Nov 28, 2020 at 5:53 AM Florian Fainelli <f.fainelli@gmail.com> wrote:
>

> On Fri, 16 Oct 2020 17:08:32 +0800, Zhen Lei <thunder.leizhen@huawei.com> wrote:

> > The scripts/dtc/checks.c requires that the node have empty "dma-ranges"

> > property must have the same "#address-cells" and "#size-cells" values as

> > the parent node. Otherwise, the following warnings is reported:

> >

> > arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning \

> > (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but \

> > its #address-cells (1) differs from / (2)

> > arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning \

> > (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but \

> > its #size-cells (1) differs from / (2)

> >

> > Arnd Bergmann figured out why it's necessary:

> > Also note that the #address-cells=<1> means that any device under

> > this bus is assumed to only support 32-bit addressing, and DMA will

> > have to go through a slow swiotlb in the absence of an IOMMU.

> >

> > Suggested-by: Arnd Bergmann <arnd@arndb.de>

> > Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>

> > ---

>

> Applied to devicetree-arm64/next, thanks!


The notification may have gone missing, but I had merged it into v5.10-fixes
already, and as of today, it's in mainline, so you can drop it from your
next branch, or just leave it in if you want to avoid taking things out of
your tree.

      Arnd
Ray Jui Dec. 14, 2020, 7:09 p.m. UTC | #7
+ Bharat

On 11/28/2020 1:58 AM, Arnd Bergmann wrote:
> On Sat, Nov 28, 2020 at 5:53 AM Florian Fainelli <f.fainelli@gmail.com> wrote:

>>

>> On Fri, 16 Oct 2020 17:08:32 +0800, Zhen Lei <thunder.leizhen@huawei.com> wrote:

>>> The scripts/dtc/checks.c requires that the node have empty "dma-ranges"

>>> property must have the same "#address-cells" and "#size-cells" values as

>>> the parent node. Otherwise, the following warnings is reported:

>>>

>>> arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning \

>>> (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but \

>>> its #address-cells (1) differs from / (2)

>>> arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning \

>>> (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but \

>>> its #size-cells (1) differs from / (2)

>>>

>>> Arnd Bergmann figured out why it's necessary:

>>> Also note that the #address-cells=<1> means that any device under

>>> this bus is assumed to only support 32-bit addressing, and DMA will

>>> have to go through a slow swiotlb in the absence of an IOMMU.

>>>

>>> Suggested-by: Arnd Bergmann <arnd@arndb.de>

>>> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>

>>> ---

>>

>> Applied to devicetree-arm64/next, thanks!

> 

> The notification may have gone missing, but I had merged it into v5.10-fixes

> already, and as of today, it's in mainline, so you can drop it from your

> next branch, or just leave it in if you want to avoid taking things out of

> your tree.

> 

>       Arnd

> 


It looks like this patch might have caused a regression on Stingray USB.
Bharat, could you please confirm?

The fix would be to properly define the dma-ranges to be 32-bit (0x0 ~
0xffffffff) since IOMMU is disabled on this device and the device's DMA
engine is on a 32-bit bus.

Thanks,

Ray





-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.
Arnd Bergmann Dec. 14, 2020, 7:46 p.m. UTC | #8
On Mon, Dec 14, 2020 at 8:09 PM Ray Jui <ray.jui@broadcom.com> wrote:
> On 11/28/2020 1:58 AM, Arnd Bergmann wrote:
> > On Sat, Nov 28, 2020 at 5:53 AM Florian Fainelli <f.fainelli@gmail.com> wrote:
> >>
> >> On Fri, 16 Oct 2020 17:08:32 +0800, Zhen Lei <thunder.leizhen@huawei.com> wrote:
> >>> The scripts/dtc/checks.c requires that the node have empty "dma-ranges"
> >>> property must have the same "#address-cells" and "#size-cells" values as
> >>> the parent node. Otherwise, the following warnings is reported:
> >>>
> >>> arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning \
> >>> (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but \
> >>> its #address-cells (1) differs from / (2)
> >>> arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning \
> >>> (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but \
> >>> its #size-cells (1) differs from / (2)
> >>>
> >>> Arnd Bergmann figured out why it's necessary:
> >>> Also note that the #address-cells=<1> means that any device under
> >>> this bus is assumed to only support 32-bit addressing, and DMA will
> >>> have to go through a slow swiotlb in the absence of an IOMMU.
> >>>
> >>> Suggested-by: Arnd Bergmann <arnd@arndb.de>
> >>> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
> >>> ---
> >>
> >> Applied to devicetree-arm64/next, thanks!
> >
> > The notification may have gone missing, but I had merged it into v5.10-fixes
> > already, and as of today, it's in mainline, so you can drop it from your
> > next branch, or just leave it in if you want to avoid taking things out of
> > your tree.
>
> It looks like this patch might have caused a regression on Stingray USB.
> Bharat, could you please confirm?

Well, this is what I had asked about originally, I assumed that
Florian had asked someone with access to the datasheet.

> The fix would be to properly define the dma-ranges to be 32-bit (0x0 ~
> 0xffffffff) since IOMMU is disabled on this device and the device's DMA
> engine is on a 32-bit bus.

That's not how dma-ranges work, they tell you what the capabilities
of the bus are, while the capabilities of the device are identified by
the properties of that device.

The device claims to be compatible with "generic-xhci", so the
driver asks for a 64-bit mask to be set according to the xhci
specification. If this device is not xhci compliant, then it should
not ask for a 64-bit mask.

However, if this is a 64-bit capable bus master on a 32-bit bus,
then the dma-ranges property should list the capabilities of the
bus, so the kernel can force the driver to fall back to 32-bit addressing.

      Arnd
Florian Fainelli Dec. 15, 2020, 3:40 p.m. UTC | #9
On 12/14/2020 11:46 AM, Arnd Bergmann wrote:
> On Mon, Dec 14, 2020 at 8:09 PM Ray Jui <ray.jui@broadcom.com> wrote:
>> On 11/28/2020 1:58 AM, Arnd Bergmann wrote:
>>> On Sat, Nov 28, 2020 at 5:53 AM Florian Fainelli <f.fainelli@gmail.com> wrote:
>>>>
>>>> On Fri, 16 Oct 2020 17:08:32 +0800, Zhen Lei <thunder.leizhen@huawei.com> wrote:
>>>>> The scripts/dtc/checks.c requires that the node have empty "dma-ranges"
>>>>> property must have the same "#address-cells" and "#size-cells" values as
>>>>> the parent node. Otherwise, the following warnings is reported:
>>>>>
>>>>> arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning \
>>>>> (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but \
>>>>> its #address-cells (1) differs from / (2)
>>>>> arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi:7.3-14: Warning \
>>>>> (dma_ranges_format): /usb:dma-ranges: empty "dma-ranges" property but \
>>>>> its #size-cells (1) differs from / (2)
>>>>>
>>>>> Arnd Bergmann figured out why it's necessary:
>>>>> Also note that the #address-cells=<1> means that any device under
>>>>> this bus is assumed to only support 32-bit addressing, and DMA will
>>>>> have to go through a slow swiotlb in the absence of an IOMMU.
>>>>>
>>>>> Suggested-by: Arnd Bergmann <arnd@arndb.de>
>>>>> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
>>>>> ---
>>>>
>>>> Applied to devicetree-arm64/next, thanks!
>>>
>>> The notification may have gone missing, but I had merged it into v5.10-fixes
>>> already, and as of today, it's in mainline, so you can drop it from your
>>> next branch, or just leave it in if you want to avoid taking things out of
>>> your tree.
>>
>> It looks like this patch might have caused a regression on Stingray USB.
>> Bharat, could you please confirm?
> 
> Well, this is what I had asked about originally, I assumed that
> Florian had asked someone with access to the datasheet.

It looks like we had some bad communication here, the notification email
indicating that you had applied the patch did not make it through, and I
failed to make you aware that I was waiting for some testing from Scott
and Ray.
Florian Fainelli Dec. 15, 2020, 3:41 p.m. UTC | #10
On 12/15/2020 5:19 AM, Bharat Gooty wrote:
> Since the IOMMU is disabled and DMA engine is on 32-bit bus, We can not

> give the complete DDR for the USB DMA.

> So restricting the usable DAM size to 4GB.


Thanks, can you make this a proper patch submission along with a Fixes:
tag that is:

Fixes: 2013a4b684b6 ("arm64: dts: broadcom: clear the warnings caused by
empty dma-ranges")
-- 
Florian
Arnd Bergmann Dec. 15, 2020, 3:49 p.m. UTC | #11
On Tue, Dec 15, 2020 at 4:41 PM Florian Fainelli <f.fainelli@gmail.com> wrote:
>
> On 12/15/2020 5:19 AM, Bharat Gooty wrote:
> > Since the IOMMU is disabled and DMA engine is on 32-bit bus, We can not
> > give the complete DDR for the USB DMA.
> > So restricting the usable DAM size to 4GB.
>
> Thanks, can you make this a proper patch submission along with a Fixes:
> tag that is:
>
> Fixes: 2013a4b684b6 ("arm64: dts: broadcom: clear the warnings caused by
> empty dma-ranges")

Yes, that would be helpful, though I would appreciate a better description
that explains what is actually going on: is it the device or the bus that
has the 32-bit limitation, and if it is indeed a bug in the device, why do
you pretend that this is a 64-bit device on a 32-bit bus instead (this
could also use a comment in the dts file)?

       Arnd
Ray Jui Jan. 12, 2021, 6:28 p.m. UTC | #12
Hi Arnd,

On 2020-12-15 7:49 a.m., Arnd Bergmann wrote:
> On Tue, Dec 15, 2020 at 4:41 PM Florian Fainelli <f.fainelli@gmail.com> wrote:

>>

>> On 12/15/2020 5:19 AM, Bharat Gooty wrote:

>>> Since the IOMMU is disabled and DMA engine is on 32-bit bus, We can not

>>> give the complete DDR for the USB DMA.

>>> So restricting the usable DAM size to 4GB.

>>

>> Thanks, can you make this a proper patch submission along with a Fixes:

>> tag that is:

>>

>> Fixes: 2013a4b684b6 ("arm64: dts: broadcom: clear the warnings caused by

>> empty dma-ranges")

> 

> Yes, that would be helpful, though I would appreciate a better description

> that explains what is actually going on: is it the device or the bus that

> has the 32-bit limitation, and if it is indeed a bug in the device, why do

> you pretend that this is a 64-bit device on a 32-bit bus instead (this

> could also use a comment in the dts file)?

> 

>         Arnd

> 


Sorry for the delay in reply. Bharat finally got time to do some 
investigation to confirm the following:

These USB controllers indeed can address 64-bit. However, on the bus 
internally, only 40-bits are connected to the interconnect of CCN. As a 
result, the 'dma-ranges' should be modified to address 40-bit in size.

We also have a somewhat related question, is it true that since v5.10, 
defining of 'dma-ranges' on the bus node where its child device node has 
implication of IOMMU usage (through 'iommus' or 'iommu-map') is now 
mandatory? My understanding is that the 'dma-ranges' in this scheme will 
define the IOVA address to system address mapping required by all 
devices on that bus. Please help to confirm if my understanding is correct.

Thanks,

Ray
Arnd Bergmann Jan. 12, 2021, 8:40 p.m. UTC | #13
On Tue, Jan 12, 2021 at 7:28 PM Ray Jui <ray.jui@broadcom.com> wrote:
> On 2020-12-15 7:49 a.m., Arnd Bergmann wrote:
> > On Tue, Dec 15, 2020 at 4:41 PM Florian Fainelli <f.fainelli@gmail.com> wrote:
> >>
> >> On 12/15/2020 5:19 AM, Bharat Gooty wrote:
> >>> Since the IOMMU is disabled and DMA engine is on 32-bit bus, We can not
> >>> give the complete DDR for the USB DMA.
> >>> So restricting the usable DAM size to 4GB.
> >>
> >> Thanks, can you make this a proper patch submission along with a Fixes:
> >> tag that is:
> >>
> >> Fixes: 2013a4b684b6 ("arm64: dts: broadcom: clear the warnings caused by
> >> empty dma-ranges")
> >
> > Yes, that would be helpful, though I would appreciate a better description
> > that explains what is actually going on: is it the device or the bus that
> > has the 32-bit limitation, and if it is indeed a bug in the device, why do
> > you pretend that this is a 64-bit device on a 32-bit bus instead (this
> > could also use a comment in the dts file)?
> >
> >         Arnd
> >
>
> Sorry for the delay in reply. Bharat finally got time to do some
> investigation to confirm the following:
>
> These USB controllers indeed can address 64-bit. However, on the bus
> internally, only 40-bits are connected to the interconnect of CCN. As a
> result, the 'dma-ranges' should be modified to address 40-bit in size.

a 40-bit range is effectively the same as a 64-bit range though, so I
think you'll still need a driver quirk if the device cannot use the whole 40
bit addressing but is limited to 32 bits because of a bug in its dma engine.

> We also have a somewhat related question, is it true that since v5.10,
> defining of 'dma-ranges' on the bus node where its child device node has
> implication of IOMMU usage (through 'iommus' or 'iommu-map') is now
> mandatory? My understanding is that the 'dma-ranges' in this scheme will
> define the IOVA address to system address mapping required by all
> devices on that bus. Please help to confirm if my understanding is correct

I have not actually seen that change. In general, you should always
have a dma-ranges property in the parent of a DMA master, I think
we just never enforced that and fall back to a 32-bit mask by default.

      Arnd
Ray Jui Jan. 12, 2021, 8:57 p.m. UTC | #14
On 1/12/2021 12:40 PM, Arnd Bergmann wrote:
> On Tue, Jan 12, 2021 at 7:28 PM Ray Jui <ray.jui@broadcom.com> wrote:
>> On 2020-12-15 7:49 a.m., Arnd Bergmann wrote:
>>> On Tue, Dec 15, 2020 at 4:41 PM Florian Fainelli <f.fainelli@gmail.com> wrote:
>>>>
>>>> On 12/15/2020 5:19 AM, Bharat Gooty wrote:
>>>>> Since the IOMMU is disabled and DMA engine is on 32-bit bus, We can not
>>>>> give the complete DDR for the USB DMA.
>>>>> So restricting the usable DAM size to 4GB.
>>>>
>>>> Thanks, can you make this a proper patch submission along with a Fixes:
>>>> tag that is:
>>>>
>>>> Fixes: 2013a4b684b6 ("arm64: dts: broadcom: clear the warnings caused by
>>>> empty dma-ranges")
>>>
>>> Yes, that would be helpful, though I would appreciate a better description
>>> that explains what is actually going on: is it the device or the bus that
>>> has the 32-bit limitation, and if it is indeed a bug in the device, why do
>>> you pretend that this is a 64-bit device on a 32-bit bus instead (this
>>> could also use a comment in the dts file)?
>>>
>>>         Arnd
>>>
>>
>> Sorry for the delay in reply. Bharat finally got time to do some
>> investigation to confirm the following:
>>
>> These USB controllers indeed can address 64-bit. However, on the bus
>> internally, only 40-bits are connected to the interconnect of CCN. As a
>> result, the 'dma-ranges' should be modified to address 40-bit in size.
> 
> a 40-bit range is effectively the same as a 64-bit range though, so I
> think you'll still need a driver quirk if the device cannot use the whole 40
> bit addressing but is limited to 32 bits because of a bug in its dma engine.
> 

Bharat can correct me if I'm wrong, but I don't think we have a bug in
the USB DMA engine that causes it can only address 32-bit. I believe we
can set dma-ranges size to 40-bit here.

The dma-range property is though required to be specified, instead of
leaving it as empty, with the use of IOMMU. That seems to be a v5.10
specific behavior as I described below.

Thanks,

Ray

>> We also have a somewhat related question, is it true that since v5.10,
>> defining of 'dma-ranges' on the bus node where its child device node has
>> implication of IOMMU usage (through 'iommus' or 'iommu-map') is now
>> mandatory? My understanding is that the 'dma-ranges' in this scheme will
>> define the IOVA address to system address mapping required by all
>> devices on that bus. Please help to confirm if my understanding is correct
> 
> I have not actually seen that change. In general, you should always
> have a dma-ranges property in the parent of a DMA master, I think
> we just never enforced that and fall back to a 32-bit mask by default.
> 
>       Arnd
>
Bharat Gooty Jan. 13, 2021, 3:42 a.m. UTC | #15
Hello Ray,

I had cross checked with Design and integration team.
Yes we can set the "dma-rages" to 40 bit DMA ranges. Tested, it is working.

Thanks,
-Bharat

-----Original Message-----
From: Ray Jui <ray.jui@broadcom.com>
Sent: Wednesday, January 13, 2021 2:28 AM
To: Arnd Bergmann <arnd@kernel.org>
Cc: Florian Fainelli <f.fainelli@gmail.com>; Bharat Gooty
<bharat.gooty@broadcom.com>; devicetree <devicetree@vger.kernel.org>;
linux-kernel <linux-kernel@vger.kernel.org>; Arnd Bergmann <arnd@arndb.de>;
Scott Branden <sbranden@broadcom.com>; Ray Jui <rjui@broadcom.com>;
linux-arm-msm <linux-arm-msm@vger.kernel.org>; Andy Gross
<agross@kernel.org>; Bjorn Andersson <bjorn.andersson@linaro.org>; Rob
Herring <robh+dt@kernel.org>; bcm-kernel-feedback-list
<bcm-kernel-feedback-list@broadcom.com>; Zhen Lei
<thunder.leizhen@huawei.com>; linux-arm-kernel
<linux-arm-kernel@lists.infradead.org>
Subject: Re: [PATCH v2 1/2] arm64: dts: broadcom: clear the warnings caused
by empty dma-ranges



On 1/12/2021 12:40 PM, Arnd Bergmann wrote:
> On Tue, Jan 12, 2021 at 7:28 PM Ray Jui <ray.jui@broadcom.com> wrote:
>> On 2020-12-15 7:49 a.m., Arnd Bergmann wrote:
>>> On Tue, Dec 15, 2020 at 4:41 PM Florian Fainelli <f.fainelli@gmail.com>
>>> wrote:
>>>>
>>>> On 12/15/2020 5:19 AM, Bharat Gooty wrote:
>>>>> Since the IOMMU is disabled and DMA engine is on 32-bit bus, We can
>>>>> not
>>>>> give the complete DDR for the USB DMA.
>>>>> So restricting the usable DAM size to 4GB.
>>>>
>>>> Thanks, can you make this a proper patch submission along with a Fixes:
>>>> tag that is:
>>>>
>>>> Fixes: 2013a4b684b6 ("arm64: dts: broadcom: clear the warnings caused
>>>> by
>>>> empty dma-ranges")
>>>
>>> Yes, that would be helpful, though I would appreciate a better
>>> description
>>> that explains what is actually going on: is it the device or the bus
>>> that
>>> has the 32-bit limitation, and if it is indeed a bug in the device, why
>>> do
>>> you pretend that this is a 64-bit device on a 32-bit bus instead (this
>>> could also use a comment in the dts file)?
>>>
>>>         Arnd
>>>
>>
>> Sorry for the delay in reply. Bharat finally got time to do some
>> investigation to confirm the following:
>>
>> These USB controllers indeed can address 64-bit. However, on the bus
>> internally, only 40-bits are connected to the interconnect of CCN. As a
>> result, the 'dma-ranges' should be modified to address 40-bit in size.
>
> a 40-bit range is effectively the same as a 64-bit range though, so I
> think you'll still need a driver quirk if the device cannot use the whole
> 40
> bit addressing but is limited to 32 bits because of a bug in its dma
> engine.
>

Bharat can correct me if I'm wrong, but I don't think we have a bug in
the USB DMA engine that causes it can only address 32-bit. I believe we
can set dma-ranges size to 40-bit here.

The dma-range property is though required to be specified, instead of
leaving it as empty, with the use of IOMMU. That seems to be a v5.10
specific behavior as I described below.

Thanks,

Ray

>> We also have a somewhat related question, is it true that since v5.10,
>> defining of 'dma-ranges' on the bus node where its child device node has
>> implication of IOMMU usage (through 'iommus' or 'iommu-map') is now
>> mandatory? My understanding is that the 'dma-ranges' in this scheme will
>> define the IOVA address to system address mapping required by all
>> devices on that bus. Please help to confirm if my understanding is
>> correct
>
> I have not actually seen that change. In general, you should always
> have a dma-ranges property in the parent of a DMA master, I think
> we just never enforced that and fall back to a 32-bit mask by default.
>
>       Arnd
>
Arnd Bergmann Jan. 13, 2021, 8:05 a.m. UTC | #16
On Wed, Jan 13, 2021 at 4:42 AM Bharat Gooty <bharat.gooty@broadcom.com> wrote:
>

> Hello Ray,

>

> I had cross checked with Design and integration team.

> Yes we can set the "dma-rages" to 40 bit DMA ranges. Tested, it is working.

>

> -----Original Message-----

> From: Ray Jui <ray.jui@broadcom.com>

>

> Bharat can correct me if I'm wrong, but I don't think we have a bug in

> the USB DMA engine that causes it can only address 32-bit. I believe we

> can set dma-ranges size to 40-bit here.

>

> The dma-range property is though required to be specified, instead of

> leaving it as empty, with the use of IOMMU. That seems to be a v5.10

> specific behavior as I described below.


Ok, thanks for double-checking. I had misremembered the version
that actually went into the as the one that used 64-bit dma-ranges
and thought that was what broke, rather than the version without
dma-ranges.

If any of you want to send me that bugfix directly, or have Florian
pick it up through his fixes branch, I'll make sure we get it into v5.11.

       Arnd
Florian Fainelli Jan. 13, 2021, 4:55 p.m. UTC | #17
On 1/13/21 12:05 AM, Arnd Bergmann wrote:
> On Wed, Jan 13, 2021 at 4:42 AM Bharat Gooty <bharat.gooty@broadcom.com> wrote:

>>

>> Hello Ray,

>>

>> I had cross checked with Design and integration team.

>> Yes we can set the "dma-rages" to 40 bit DMA ranges. Tested, it is working.

>>

>> -----Original Message-----

>> From: Ray Jui <ray.jui@broadcom.com>

>>

>> Bharat can correct me if I'm wrong, but I don't think we have a bug in

>> the USB DMA engine that causes it can only address 32-bit. I believe we

>> can set dma-ranges size to 40-bit here.

>>

>> The dma-range property is though required to be specified, instead of

>> leaving it as empty, with the use of IOMMU. That seems to be a v5.10

>> specific behavior as I described below.

> 

> Ok, thanks for double-checking. I had misremembered the version

> that actually went into the as the one that used 64-bit dma-ranges

> and thought that was what broke, rather than the version without

> dma-ranges.

> 

> If any of you want to send me that bugfix directly, or have Florian

> pick it up through his fixes branch, I'll make sure we get it into v5.11.


I have another change for v5.11 that I would like to send, so please do
send a bugfix when you get a chance and we can lump those two changes
together, say, by the end of the week?
-- 
Florian
Ray Jui Jan. 13, 2021, 5:45 p.m. UTC | #18
Hi Bharat,

On 1/13/2021 8:55 AM, Florian Fainelli wrote:
> On 1/13/21 12:05 AM, Arnd Bergmann wrote:

>> On Wed, Jan 13, 2021 at 4:42 AM Bharat Gooty <bharat.gooty@broadcom.com> wrote:

>>>

>>> Hello Ray,

>>>

>>> I had cross checked with Design and integration team.

>>> Yes we can set the "dma-rages" to 40 bit DMA ranges. Tested, it is working.

>>>

>>> -----Original Message-----

>>> From: Ray Jui <ray.jui@broadcom.com>

>>>

>>> Bharat can correct me if I'm wrong, but I don't think we have a bug in

>>> the USB DMA engine that causes it can only address 32-bit. I believe we

>>> can set dma-ranges size to 40-bit here.

>>>

>>> The dma-range property is though required to be specified, instead of

>>> leaving it as empty, with the use of IOMMU. That seems to be a v5.10

>>> specific behavior as I described below.

>>

>> Ok, thanks for double-checking. I had misremembered the version

>> that actually went into the as the one that used 64-bit dma-ranges

>> and thought that was what broke, rather than the version without

>> dma-ranges.

>>

>> If any of you want to send me that bugfix directly, or have Florian

>> pick it up through his fixes branch, I'll make sure we get it into v5.11.

> 

> I have another change for v5.11 that I would like to send, so please do

> send a bugfix when you get a chance and we can lump those two changes

> together, say, by the end of the week?

> 


Would you be able to help to send this out by the end of this week as
Florian advised? Your previous patch with the extension to 40-bit in
size plus a fixes-by tag should do!

Thanks!

Ray
diff mbox series

Patch

diff --git a/arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi b/arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi
index 55259f973b5a9e4..aef8f2b00778d71 100644
--- a/arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi
+++ b/arch/arm64/boot/dts/broadcom/stingray/stingray-usb.dtsi
@@ -5,20 +5,20 @@ 
 	usb {
 		compatible = "simple-bus";
 		dma-ranges;
-		#address-cells = <1>;
-		#size-cells = <1>;
-		ranges = <0x0 0x0 0x68500000 0x00400000>;
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges = <0x0 0x0 0x0 0x68500000 0x0 0x00400000>;
 
 		usbphy0: usb-phy@0 {
 			compatible = "brcm,sr-usb-combo-phy";
-			reg = <0x00000000 0x100>;
+			reg = <0x0 0x00000000 0x0 0x100>;
 			#phy-cells = <1>;
 			status = "disabled";
 		};
 
 		xhci0: usb@1000 {
 			compatible = "generic-xhci";
-			reg = <0x00001000 0x1000>;
+			reg = <0x0 0x00001000 0x0 0x1000>;
 			interrupts = <GIC_SPI 256 IRQ_TYPE_LEVEL_HIGH>;
 			phys = <&usbphy0 1>, <&usbphy0 0>;
 			phy-names = "phy0", "phy1";
@@ -28,7 +28,7 @@ 
 
 		bdc0: usb@2000 {
 			compatible = "brcm,bdc-v0.16";
-			reg = <0x00002000 0x1000>;
+			reg = <0x0 0x00002000 0x0 0x1000>;
 			interrupts = <GIC_SPI 259 IRQ_TYPE_LEVEL_HIGH>;
 			phys = <&usbphy0 0>, <&usbphy0 1>;
 			phy-names = "phy0", "phy1";
@@ -38,21 +38,21 @@ 
 
 		usbphy1: usb-phy@10000 {
 			compatible = "brcm,sr-usb-combo-phy";
-			reg = <0x00010000 0x100>;
+			reg = <0x0 0x00010000 0x0 0x100>;
 			#phy-cells = <1>;
 			status = "disabled";
 		};
 
 		usbphy2: usb-phy@20000 {
 			compatible = "brcm,sr-usb-hs-phy";
-			reg = <0x00020000 0x100>;
+			reg = <0x0 0x00020000 0x0 0x100>;
 			#phy-cells = <0>;
 			status = "disabled";
 		};
 
 		xhci1: usb@11000 {
 			compatible = "generic-xhci";
-			reg = <0x00011000 0x1000>;
+			reg = <0x0 0x00011000 0x0 0x1000>;
 			interrupts = <GIC_SPI 263 IRQ_TYPE_LEVEL_HIGH>;
 			phys = <&usbphy1 1>, <&usbphy2>, <&usbphy1 0>;
 			phy-names = "phy0", "phy1", "phy2";
@@ -62,7 +62,7 @@ 
 
 		bdc1: usb@21000 {
 			compatible = "brcm,bdc-v0.16";
-			reg = <0x00021000 0x1000>;
+			reg = <0x0 0x00021000 0x0 0x1000>;
 			interrupts = <GIC_SPI 266 IRQ_TYPE_LEVEL_HIGH>;
 			phys = <&usbphy2>;
 			phy-names = "phy0";