mbox series

[v4,0/4] i2c: imx: prevent rescheduling in non-dma mode

Message ID 20241002112020.23913-1-eichest@gmail.com
Headers show
Series i2c: imx: prevent rescheduling in non-dma mode | expand

Message

Stefan Eichenberger Oct. 2, 2024, 11:19 a.m. UTC
While running tests on an i.MX8M Mini connected to a TI ADS1015 ADC, we
found that the ADC would stop responding to i2c requests because it
would timeout after the bus was idle for 25ms. This timeout could be
traced back to the rescheduling events in the i2c-imx driver. The
problem is that if the system is under heavy load, the schedule call and
the wait_event_timeout may be rescheduled too late to reach the 25ms
timeout. The same problem may occur with other SMBus devices. Therefore,
this patchset removes the scheduling calls for non-DMA mode by handling
the interrupt events directly in the ISR instead of scheduling a task to
handle the events.

This patch will introduce some bigger changes because the logic for
handling events in the ISR had to be rewritten. Therefore we have tested
the following combinations:
- i.MX8M Mini with dma
- i.MX8M Mini without dma
- i.MX8M Plus with dma
- i.MX8M Plus without dma
- i.MX7D with dma
- i.MX7D without dma
- i.MX7D atomic mode

Because we do not have any devices that use the SMBus block transfer
mode, we were not able to test it. 

The ideas are based on the RFC:
https://lore.kernel.org/all/20240531142437.74831-1-eichest@gmail.com/
However, the handling of events in the ISR is new, because further
testing showed that it was not enough to simply remove the schedule
call.

Changes since v3:
- Fixed style issues with checkpatch.pl --strict (Andi)
- Add comments to explain the code (Andi)

Changes since v2:
- Add Acked-by tags from Oleksij
- Renamed i2c_imx_start_read to i2c_imx_prepare_read
- I did not add a Fixes tag because the issues from Flavio have a
  different root cause and are not fixed by this patchset

Changes since v1:
- Add Reviewed-by tags from Frank
- Add new patch to use readb_relaxed and writeb_relaxed (Frank)
- Update commit message for patch 1 with some clarifications (Frank)

Stefan Eichenberger (4):
  i2c: imx: only poll for bus busy in multi master mode
  i2c: imx: separate atomic, dma and non-dma use case
  i2c: imx: use readb_relaxed and writeb_relaxed
  i2c: imx: prevent rescheduling in non dma mode

 drivers/i2c/busses/i2c-imx.c | 369 ++++++++++++++++++++++++++++++-----
 1 file changed, 317 insertions(+), 52 deletions(-)

Comments

Arnd Bergmann Oct. 2, 2024, 11:51 a.m. UTC | #1
On Wed, Oct 2, 2024, at 11:19, Stefan Eichenberger wrote:
> From: Stefan Eichenberger <stefan.eichenberger@toradex.com>
>
> Use the relaxed version of readb and writeb to reduce overhead. It is
> safe to use the relaxed version because we either do not rely on dma
> completion, or we use a dma callback to ensure that the dma transfer is
> complete before we continue.

I would still consider this a bug in general, you should
never default to the unsafe variants.

If there is a codepath that needs the barrierless version,
please add imx_i2c_write_reg_relaxed()/imx_i2c_read_reg_relaxed()
helpers that use those only in the places where it makes
a measurable difference, with a comment that explains
the usage.

     Arnd
Arnd Bergmann Oct. 2, 2024, 1:36 p.m. UTC | #2
On Wed, Oct 2, 2024, at 13:08, Stefan Eichenberger wrote:
> On Wed, Oct 02, 2024 at 11:51:22AM +0000, Arnd Bergmann wrote:
>> On Wed, Oct 2, 2024, at 11:19, Stefan Eichenberger wrote:
>> > From: Stefan Eichenberger <stefan.eichenberger@toradex.com>
>> >
>> > Use the relaxed version of readb and writeb to reduce overhead. It is
>> > safe to use the relaxed version because we either do not rely on dma
>> > completion, or we use a dma callback to ensure that the dma transfer is
>> > complete before we continue.
>> 
>> I would still consider this a bug in general, you should
>> never default to the unsafe variants.
>> 
>> If there is a codepath that needs the barrierless version,
>> please add imx_i2c_write_reg_relaxed()/imx_i2c_read_reg_relaxed()
>> helpers that use those only in the places where it makes
>> a measurable difference, with a comment that explains
>> the usage.
>
> I added the patch because of the following dicussion:
> https://lore.kernel.org/linux-i2c/ZpVWXlR6j2i0ZtVQ@lizhi-Precision-Tower-5810/
>
> I can't determine if the relaxed version improves performance. The
> 'normal' version worked well for our use case too. Therefore, dropping
> the change would be acceptable for us. Another potential solution could
> be to use the relaxed version only inside the ISR. Would that be an
> acceptable solution? What is your impression, Frank Li
> <Frank.Li@nxp.com>?

I'm pretty sure that Frank meant to use readb_relaxed()/writeb_relaxed()
inside of the FIFO access loop, not for everything else. This
makes a lot of sense, since the FIFO read in particular is
clearly performance sensitive and already serialized by the
implied control dependency.

If you can read multiple bytes, the best interface to use
would in fact be readsb() or possibly readsl() to read
four bytes with each access.

It appears that you did not implement the suggestion to
read the entire FIFO though, so you can probably just skip
the _relaxed() change entirely.

     Arnd
Stefan Eichenberger Oct. 2, 2024, 2:36 p.m. UTC | #3
On Wed, Oct 02, 2024 at 01:36:04PM +0000, Arnd Bergmann wrote:
> On Wed, Oct 2, 2024, at 13:08, Stefan Eichenberger wrote:
> > On Wed, Oct 02, 2024 at 11:51:22AM +0000, Arnd Bergmann wrote:
> >> On Wed, Oct 2, 2024, at 11:19, Stefan Eichenberger wrote:
> >> > From: Stefan Eichenberger <stefan.eichenberger@toradex.com>
> >> >
> >> > Use the relaxed version of readb and writeb to reduce overhead. It is
> >> > safe to use the relaxed version because we either do not rely on dma
> >> > completion, or we use a dma callback to ensure that the dma transfer is
> >> > complete before we continue.
> >> 
> >> I would still consider this a bug in general, you should
> >> never default to the unsafe variants.
> >> 
> >> If there is a codepath that needs the barrierless version,
> >> please add imx_i2c_write_reg_relaxed()/imx_i2c_read_reg_relaxed()
> >> helpers that use those only in the places where it makes
> >> a measurable difference, with a comment that explains
> >> the usage.
> >
> > I added the patch because of the following dicussion:
> > https://lore.kernel.org/linux-i2c/ZpVWXlR6j2i0ZtVQ@lizhi-Precision-Tower-5810/
> >
> > I can't determine if the relaxed version improves performance. The
> > 'normal' version worked well for our use case too. Therefore, dropping
> > the change would be acceptable for us. Another potential solution could
> > be to use the relaxed version only inside the ISR. Would that be an
> > acceptable solution? What is your impression, Frank Li
> > <Frank.Li@nxp.com>?
> 
> I'm pretty sure that Frank meant to use readb_relaxed()/writeb_relaxed()
> inside of the FIFO access loop, not for everything else. This
> makes a lot of sense, since the FIFO read in particular is
> clearly performance sensitive and already serialized by the
> implied control dependency.
> 
> If you can read multiple bytes, the best interface to use
> would in fact be readsb() or possibly readsl() to read
> four bytes with each access.
> 
> It appears that you did not implement the suggestion to
> read the entire FIFO though, so you can probably just skip
> the _relaxed() change entirely.

This makes sense, it appears this was a misunderstanding. If no one
objects, I will drop the patch in the next version. Thank you for the
clarification.

Regards,
Stefan
Stefan Eichenberger Oct. 2, 2024, 2:56 p.m. UTC | #4
On Wed, Oct 02, 2024 at 04:40:32PM +0200, Lucas Stach wrote:
> Am Mittwoch, dem 02.10.2024 um 13:19 +0200 schrieb Stefan Eichenberger:
> > From: Stefan Eichenberger <stefan.eichenberger@toradex.com>
> > 
> > According to the i.MX8M Mini reference manual chapter "16.1.4.2
> > Generation of Start" it is only necessary to poll for bus busy and
> > arbitration lost in multi master mode. This helps to avoid rescheduling
> > while the i2c bus is busy and avoids SMBus devices to timeout.
> > 
> This is a backward incompatible change, as far as I can see. Until now
> the driver would properly handle a multi-mastered bus, without any
> specific configuration. Now it requires the new multi-master DT
> property to be set, which isn't even documented in the binding to be
> understood by this driver.
> 
> Are you sure that every single instance of a i.MX i2c bus is only
> single mastered?
> 
> If this is a worthwhile performance improvement I guess you need to
> flip the logic around by adding a new single-master DT property (or
> something along those lines), which should go through proper DT binding
> review. You can then use this property for boards/busses to opt into
> skipping the arbitration lost check.

According to the discussion here the property documentation should not
be added:
https://lore.kernel.org/linux-i2c/2bbddaxyjkxfmlgmq3yqcbzo7dsb2pq5bvdatk2y4ig4iintkt@35btqkdv7sy3/

However, the point regarding single-master and multi-master is correct.
We also discussed this internally and assumed the single-master use case
is more likely to be the default and that this patch series would fix
issues for other devices out there. However, your point is valid and if
preferred I can change it to single-master with the next version.

Francesco Dolcini <francesco.dolcini@toradex.com> I think you once had a
discussion regarding multi master mode for i2c on i.MX devices? Maybe
you can remember the details?

Regards,
Stefan