ata: sata_mv: fix sg_tbl_pool alignment

Message ID 20130116155045.GI25500@titan.lakedaemon.net
State New
Headers show

Commit Message

Jason Cooper Jan. 16, 2013, 3:50 p.m.
On Wed, Jan 16, 2013 at 09:55:55AM +0100, Soeren Moch wrote:
> On 16.01.2013 04:24, Soeren Moch wrote:
> >On 16.01.2013 03:40, Jason Cooper wrote:
> >>On Wed, Jan 16, 2013 at 01:17:59AM +0100, Soeren Moch wrote:
> >>>On 15.01.2013 22:56, Jason Cooper wrote:
> >>>>On Tue, Jan 15, 2013 at 03:16:17PM -0500, Jason Cooper wrote:

> OK, I could trigger the error
>   ERROR: 1024 KiB atomic DMA coherent pool is too small!
>   Please increase it with coherent_pool= kernel parameter!
> only with em28xx sticks and sata, dib0700 sticks removed.

Did you test the reverse scenario?  ie dib0700 with sata_mv and no
em28xx.

What kind of throughput are you pushing to the sata disk?

> >>What would be most helpful is if you could do a git bisect between
> >>v3.5.x (working) and the oldest version where you know it started
> >>failing (v3.7.1 or earlier if you know it).
> >>
> >I did not bisect it, but Marek mentioned earlier that commit
> >e9da6e9905e639b0f842a244bc770b48ad0523e9 in Linux v3.6-rc1 introduced
> >new code for dma allocations. This is probably the root cause for the
> >new (mis-)behavior (due to my tests 3.6.0 is not working anymore).
> 
> I don't want to say that Mareks patch is wrong, probably it triggers a
> bug somewhere else! (in em28xx?)

Of the four drivers you listed, none are using dma.  sata_mv is the only
one.

If one is to believe the comments in sata_mv.c:~151, then the alignment
is wrong for the sg_tbl_pool.

Could you please try the following patch?

thx,

Jason.

---8<----------
From 566c7e30285e4c31d76724ea4811b016b753f24f Mon Sep 17 00:00:00 2001
From: Jason Cooper <jason@lakedaemon.net>
Date: Wed, 16 Jan 2013 15:43:37 +0000
Subject: [PATCH] ata: sata_mv: fix sg_tbl_pool alignment

If the comment is to be believed, the alignment should be 16B, and the
size 4K.  The current code sets both to 4K.  On some arm boards
(kirkwood), this causes:

   ERROR: 1024 KiB atomic DMA coherent pool is too small!
   Please increase it with coherent_pool= kernel parameter!

Set alignment to 16B to prevent exhausting the atomic_pool.

Signed-off-by: Jason Cooper <jason@lakedaemon.net>
---
 drivers/ata/sata_mv.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

Comments

Soeren Moch Jan. 16, 2013, 5:05 p.m. | #1
On 16.01.2013 16:50, Jason Cooper wrote:
> On Wed, Jan 16, 2013 at 09:55:55AM +0100, Soeren Moch wrote:
>> On 16.01.2013 04:24, Soeren Moch wrote:
>>> On 16.01.2013 03:40, Jason Cooper wrote:
>>>> On Wed, Jan 16, 2013 at 01:17:59AM +0100, Soeren Moch wrote:
>>>>> On 15.01.2013 22:56, Jason Cooper wrote:
>>>>>> On Tue, Jan 15, 2013 at 03:16:17PM -0500, Jason Cooper wrote:
>
>> OK, I could trigger the error
>>    ERROR: 1024 KiB atomic DMA coherent pool is too small!
>>    Please increase it with coherent_pool= kernel parameter!
>> only with em28xx sticks and sata, dib0700 sticks removed.
>
> Did you test the reverse scenario?  ie dib0700 with sata_mv and no
> em28xx.

Maybe I can test this next night.

> What kind of throughput are you pushing to the sata disk?

Close to nothing. In the last test I had the root filesystem running
on the sata disk plus a few 10 megabytes per hour.

>>>> What would be most helpful is if you could do a git bisect between
>>>> v3.5.x (working) and the oldest version where you know it started
>>>> failing (v3.7.1 or earlier if you know it).
>>>>
>>> I did not bisect it, but Marek mentioned earlier that commit
>>> e9da6e9905e639b0f842a244bc770b48ad0523e9 in Linux v3.6-rc1 introduced
>>> new code for dma allocations. This is probably the root cause for the
>>> new (mis-)behavior (due to my tests 3.6.0 is not working anymore).
>>
>> I don't want to say that Mareks patch is wrong, probably it triggers a
>> bug somewhere else! (in em28xx?)
>
> Of the four drivers you listed, none are using dma.  sata_mv is the only
> one.

usb_core is doing the actual DMA for the usb bridge drivers, I think.

> If one is to believe the comments in sata_mv.c:~151, then the alignment
> is wrong for the sg_tbl_pool.
>
> Could you please try the following patch?

OK, what should I test first, the setup from last night (em28xx, no
dib0700) plus your patch, or the reverse setup (dib0700, no em28xx)
without your patch, or my normal setting (all dvb sticks) plus your
patch?

Regards,
Soeren
Jason Cooper Jan. 16, 2013, 5:52 p.m. | #2
On Wed, Jan 16, 2013 at 06:05:59PM +0100, Soeren Moch wrote:
> On 16.01.2013 16:50, Jason Cooper wrote:
> >On Wed, Jan 16, 2013 at 09:55:55AM +0100, Soeren Moch wrote:
> >>On 16.01.2013 04:24, Soeren Moch wrote:
> >>>On 16.01.2013 03:40, Jason Cooper wrote:
> >>>>On Wed, Jan 16, 2013 at 01:17:59AM +0100, Soeren Moch wrote:
> >>>>>On 15.01.2013 22:56, Jason Cooper wrote:
> >>>>>>On Tue, Jan 15, 2013 at 03:16:17PM -0500, Jason Cooper wrote:
> >
> >>OK, I could trigger the error
> >>   ERROR: 1024 KiB atomic DMA coherent pool is too small!
> >>   Please increase it with coherent_pool= kernel parameter!
> >>only with em28xx sticks and sata, dib0700 sticks removed.
> >
> >Did you test the reverse scenario?  ie dib0700 with sata_mv and no
> >em28xx.
> 
> Maybe I can test this next night.

Please do, this will tell us if it is in the USB drivers or lower
(something in common).

> >>>>What would be most helpful is if you could do a git bisect between
> >>>>v3.5.x (working) and the oldest version where you know it started
> >>>>failing (v3.7.1 or earlier if you know it).
> >>>>
> >>>I did not bisect it, but Marek mentioned earlier that commit
> >>>e9da6e9905e639b0f842a244bc770b48ad0523e9 in Linux v3.6-rc1 introduced
> >>>new code for dma allocations. This is probably the root cause for the
> >>>new (mis-)behavior (due to my tests 3.6.0 is not working anymore).
> >>
> >>I don't want to say that Mareks patch is wrong, probably it triggers a
> >>bug somewhere else! (in em28xx?)
> >
> >Of the four drivers you listed, none are using dma.  sata_mv is the only
> >one.
> 
> usb_core is doing the actual DMA for the usb bridge drivers, I think.

Yes, my mistake.  I'd like to attribute that statement to pre-coffee
rambling. :-)

> >If one is to believe the comments in sata_mv.c:~151, then the alignment
> >is wrong for the sg_tbl_pool.
> >
> >Could you please try the following patch?
> 
> OK, what should I test first, the setup from last night (em28xx, no
> dib0700) plus your patch, or the reverse setup (dib0700, no em28xx)
> without your patch, or my normal setting (all dvb sticks) plus your
> patch?

if testing time is limited, please do the test I outlined at the top of
this email.  I've been digging more into the dma code and while I think
the patch is correct, I don't see where it would fix your problem (yet).

thx,

Jason.
Jason Cooper Jan. 16, 2013, 6:35 p.m. | #3
> > >On Wed, Jan 16, 2013 at 09:55:55AM +0100, Soeren Moch wrote:
> > >>I don't want to say that Mareks patch is wrong, probably it triggers a
> > >>bug somewhere else! (in em28xx?)

Could you send the output of:

lsusb -v -d VEND:PROD

for the em28xx?

thx,

Jason.
Soeren Moch Jan. 16, 2013, 10:26 p.m. | #4
On 16.01.2013 19:35, Jason Cooper wrote:
>>>> On Wed, Jan 16, 2013 at 09:55:55AM +0100, Soeren Moch wrote:
>>>>> I don't want to say that Mareks patch is wrong, probably it triggers a
>>>>> bug somewhere else! (in em28xx?)
>
> Could you send the output of:
>
> lsusb -v -d VEND:PROD
>
> for the em28xx?
>
> thx,
>
> Jason.
>

Here is the lsusb output for both of my em28xx sticks.

Regards,
Soeren


Bus 001 Device 005: ID 0ccd:00b2 TerraTec Electronic GmbH
Device Descriptor:
   bLength                18
   bDescriptorType         1
   bcdUSB               2.00
   bDeviceClass            0 (Defined at Interface level)
   bDeviceSubClass         0
   bDeviceProtocol         0
   bMaxPacketSize0        64
   idVendor           0x0ccd TerraTec Electronic GmbH
   idProduct          0x00b2
   bcdDevice            1.00
   iManufacturer           3 TERRATEC
   iProduct                1 Cinergy HTC Stick
   iSerial                 2 123456789ABCD
   bNumConfigurations      1
   Configuration Descriptor:
     bLength                 9
     bDescriptorType         2
     wTotalLength          305
     bNumInterfaces          1
     bConfigurationValue     1
     iConfiguration          0
     bmAttributes         0x80
       (Bus Powered)
     MaxPower              500mA
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       0
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0000  1x 0 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0000  1x 0 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0000  1x 0 bytes
         bInterval               1
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       1
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0000  1x 0 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x00c4  1x 196 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x03ac  1x 940 bytes
         bInterval               1
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       2
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0ad0  2x 720 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x00c4  1x 196 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x03ac  1x 940 bytes
         bInterval               1
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       3
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0c00  2x 1024 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x00c4  1x 196 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x03ac  1x 940 bytes
         bInterval               1
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       4
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x1300  3x 768 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x00c4  1x 196 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x03ac  1x 940 bytes
         bInterval               1
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       5
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x1380  3x 896 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x00c4  1x 196 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x03ac  1x 940 bytes
         bInterval               1
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       6
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x13c0  3x 960 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x00c4  1x 196 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x03ac  1x 940 bytes
         bInterval               1
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       7
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x1400  3x 1024 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x00c4  1x 196 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x03ac  1x 940 bytes
         bInterval               1
Device Qualifier (for other device speed):
   bLength                10
   bDescriptorType         6
   bcdUSB               2.00
   bDeviceClass            0 (Defined at Interface level)
   bDeviceSubClass         0
   bDeviceProtocol         0
   bMaxPacketSize0        64
   bNumConfigurations      1
Device Status:     0x0000
   (Bus Powered)





Bus 001 Device 009: ID 2304:0242 Pinnacle Systems, Inc.
Device Descriptor:
   bLength                18
   bDescriptorType         1
   bcdUSB               2.00
   bDeviceClass            0 (Defined at Interface level)
   bDeviceSubClass         0
   bDeviceProtocol         0
   bMaxPacketSize0        64
   idVendor           0x2304 Pinnacle Systems, Inc.
   idProduct          0x0242
   bcdDevice            1.00
   iManufacturer           1 Pinnacle Systems
   iProduct                2 PCTV 510e
   iSerial                 3 123456789012
   bNumConfigurations      1
   Configuration Descriptor:
     bLength                 9
     bDescriptorType         2
     wTotalLength          305
     bNumInterfaces          1
     bConfigurationValue     1
     iConfiguration          0
     bmAttributes         0x80
       (Bus Powered)
     MaxPower              500mA
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       0
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0000  1x 0 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0000  1x 0 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0000  1x 0 bytes
         bInterval               1
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       1
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0000  1x 0 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x00c4  1x 196 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x03ac  1x 940 bytes
         bInterval               1
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       2
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0ad0  2x 720 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x00c4  1x 196 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x03ac  1x 940 bytes
         bInterval               1
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       3
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0c00  2x 1024 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x00c4  1x 196 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x03ac  1x 940 bytes
         bInterval               1
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       4
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x1300  3x 768 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x00c4  1x 196 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x03ac  1x 940 bytes
         bInterval               1
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       5
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x1380  3x 896 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x00c4  1x 196 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x03ac  1x 940 bytes
         bInterval               1
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       6
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x13c0  3x 960 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x00c4  1x 196 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x03ac  1x 940 bytes
         bInterval               1
     Interface Descriptor:
       bLength                 9
       bDescriptorType         4
       bInterfaceNumber        0
       bAlternateSetting       7
       bNumEndpoints           4
       bInterfaceClass       255 Vendor Specific Class
       bInterfaceSubClass      0
       bInterfaceProtocol    255
       iInterface              0
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x81  EP 1 IN
         bmAttributes            3
           Transfer Type            Interrupt
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x0001  1x 1 bytes
         bInterval              11
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x82  EP 2 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x1400  3x 1024 bytes
         bInterval               1
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x83  EP 3 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x00c4  1x 196 bytes
         bInterval               4
       Endpoint Descriptor:
         bLength                 7
         bDescriptorType         5
         bEndpointAddress     0x84  EP 4 IN
         bmAttributes            1
           Transfer Type            Isochronous
           Synch Type               None
           Usage Type               Data
         wMaxPacketSize     0x03ac  1x 940 bytes
         bInterval               1
Device Qualifier (for other device speed):
   bLength                10
   bDescriptorType         6
   bcdUSB               2.00
   bDeviceClass            0 (Defined at Interface level)
   bDeviceSubClass         0
   bDeviceProtocol         0
   bMaxPacketSize0        64
   bNumConfigurations      1
Device Status:     0x0000
   (Bus Powered)
Soeren Moch Jan. 16, 2013, 11:10 p.m. | #5
On 16.01.2013 18:52, Jason Cooper wrote:
> On Wed, Jan 16, 2013 at 06:05:59PM +0100, Soeren Moch wrote:
>> On 16.01.2013 16:50, Jason Cooper wrote:
>>> On Wed, Jan 16, 2013 at 09:55:55AM +0100, Soeren Moch wrote:
>>>> On 16.01.2013 04:24, Soeren Moch wrote:
>>>>> On 16.01.2013 03:40, Jason Cooper wrote:
>>>>>> On Wed, Jan 16, 2013 at 01:17:59AM +0100, Soeren Moch wrote:
>>>>>>> On 15.01.2013 22:56, Jason Cooper wrote:
>>>>>>>> On Tue, Jan 15, 2013 at 03:16:17PM -0500, Jason Cooper wrote:
>>>
>>>> OK, I could trigger the error
>>>>    ERROR: 1024 KiB atomic DMA coherent pool is too small!
>>>>    Please increase it with coherent_pool= kernel parameter!
>>>> only with em28xx sticks and sata, dib0700 sticks removed.
>>>
>>> Did you test the reverse scenario?  ie dib0700 with sata_mv and no
>>> em28xx.
>>
>> Maybe I can test this next night.
>
> Please do, this will tell us if it is in the USB drivers or lower
> (something in common).
>
>>>>>> What would be most helpful is if you could do a git bisect between
>>>>>> v3.5.x (working) and the oldest version where you know it started
>>>>>> failing (v3.7.1 or earlier if you know it).
>>>>>>
>>>>> I did not bisect it, but Marek mentioned earlier that commit
>>>>> e9da6e9905e639b0f842a244bc770b48ad0523e9 in Linux v3.6-rc1 introduced
>>>>> new code for dma allocations. This is probably the root cause for the
>>>>> new (mis-)behavior (due to my tests 3.6.0 is not working anymore).
>>>>
>>>> I don't want to say that Mareks patch is wrong, probably it triggers a
>>>> bug somewhere else! (in em28xx?)
>>>
>>> Of the four drivers you listed, none are using dma.  sata_mv is the only
>>> one.
>>
>> usb_core is doing the actual DMA for the usb bridge drivers, I think.
>
> Yes, my mistake.  I'd like to attribute that statement to pre-coffee
> rambling. :-)
>
>>> If one is to believe the comments in sata_mv.c:~151, then the alignment
>>> is wrong for the sg_tbl_pool.
>>>
>>> Could you please try the following patch?
>>
>> OK, what should I test first, the setup from last night (em28xx, no
>> dib0700) plus your patch, or the reverse setup (dib0700, no em28xx)
>> without your patch, or my normal setting (all dvb sticks) plus your
>> patch?
>
> if testing time is limited, please do the test I outlined at the top of
> this email.  I've been digging more into the dma code and while I think
> the patch is correct, I don't see where it would fix your problem (yet).

Unfortunately test time is limited, and the test has to run about
10 hours to trigger the error.

I also think that sata is not causing the problem.
Maybe your patch even goes in the wrong direction. Perhaps the dma
memory pool is not too small, there might be enough memory available,
but it is too much fragmented to satisfy larger block allocations. With 
different drivers allocating totally different block sizes aligned to
bytes or words, perhaps we end up with lots of very small free blocks
in the dma pool after several hours of runtime?
So maybe it would help to align all allocations in the dma pool to 256B?

Regards,
Soeren
Soeren Moch Jan. 17, 2013, 9:11 a.m. | #6
On 16.01.2013 18:52, Jason Cooper wrote:
> On Wed, Jan 16, 2013 at 06:05:59PM +0100, Soeren Moch wrote:
>> On 16.01.2013 16:50, Jason Cooper wrote:
>>> On Wed, Jan 16, 2013 at 09:55:55AM +0100, Soeren Moch wrote:
>>>> On 16.01.2013 04:24, Soeren Moch wrote:
>>>>> On 16.01.2013 03:40, Jason Cooper wrote:
>>>>>> On Wed, Jan 16, 2013 at 01:17:59AM +0100, Soeren Moch wrote:
>>>>>>> On 15.01.2013 22:56, Jason Cooper wrote:
>>>>>>>> On Tue, Jan 15, 2013 at 03:16:17PM -0500, Jason Cooper wrote:
>>>
>>>> OK, I could trigger the error
>>>>    ERROR: 1024 KiB atomic DMA coherent pool is too small!
>>>>    Please increase it with coherent_pool= kernel parameter!
>>>> only with em28xx sticks and sata, dib0700 sticks removed.
>>>
>>> Did you test the reverse scenario?  ie dib0700 with sata_mv and no
>>> em28xx.
>>
>> Maybe I can test this next night.
>
> Please do, this will tell us if it is in the USB drivers or lower
> (something in common).

Until now there is no error with dib0700 + sata, without em28xx.

But to be sure that there is absolutely no problem with this setting
we probably need additional testing hours.
BTW, these dib0700 sticks use usb bulk transfers (and maybe smaller
dma buffers?).

Regards,
Soeren

Patch hide | download patch | download mbox

diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c
index 68f4fb5..e2e5a8a 100644
--- a/drivers/ata/sata_mv.c
+++ b/drivers/ata/sata_mv.c
@@ -148,6 +148,9 @@  enum {
 	 * CRPB needs alignment on a 256B boundary. Size == 256B
 	 * ePRD (SG) entries need alignment on a 16B boundary. Size == 16B
 	 */
+	MV_CRQB_Q_ALIGN		= 1024,
+	MV_CRPB_Q_ALIGN		= 256,
+	MV_SG_TBL_ALIGN		= 16,
 	MV_CRQB_Q_SZ		= (32 * MV_MAX_Q_DEPTH),
 	MV_CRPB_Q_SZ		= (8 * MV_MAX_Q_DEPTH),
 	MV_MAX_SG_CT		= 256,
@@ -3975,17 +3978,17 @@  done:
 static int mv_create_dma_pools(struct mv_host_priv *hpriv, struct device *dev)
 {
 	hpriv->crqb_pool   = dmam_pool_create("crqb_q", dev, MV_CRQB_Q_SZ,
-							     MV_CRQB_Q_SZ, 0);
+							MV_CRQB_Q_ALIGN, 0);
 	if (!hpriv->crqb_pool)
 		return -ENOMEM;
 
 	hpriv->crpb_pool   = dmam_pool_create("crpb_q", dev, MV_CRPB_Q_SZ,
-							     MV_CRPB_Q_SZ, 0);
+							MV_CRPB_Q_ALIGN, 0);
 	if (!hpriv->crpb_pool)
 		return -ENOMEM;
 
 	hpriv->sg_tbl_pool = dmam_pool_create("sg_tbl", dev, MV_SG_TBL_SZ,
-							     MV_SG_TBL_SZ, 0);
+							MV_SG_TBL_ALIGN, 0);
 	if (!hpriv->sg_tbl_pool)
 		return -ENOMEM;