mbox series

[v3,0/5] Fix mpt3sas driver sparse warnings

Message ID 20220307234854.148145-1-damien.lemoal@opensource.wdc.com
Headers show
Series Fix mpt3sas driver sparse warnings | expand

Message

Damien Le Moal March 7, 2022, 11:48 p.m. UTC 
This series fix (remove) all sparse warnings generated when compiling
the mpt3sas driver. All warnings are related to __iomem access and
endianness.

The series was tested on top of Martin's 5.18/scsi-staging branch with a
9400-8i HBA with direct attached iSAS and SATA drives. The fixes need
careful review by the maintainers as there is no documentation clearly
explaning the proper endianness of the values touched.

Changes from v2:
* Reworked patch 5 to keep writel() calls. Sparse warnings are
  suppressed with a declaration fix.

Changes from v1:
* Reworked patch 1 to remove the TaskMID field type change and simplify
  _ctl_set_task_mid() code.

Damien Le Moal (5):
  scsi: mpt3sas: fix _ctl_set_task_mid() TaskMID check
  scsi: mpt3sas: Fix writel() use
  scsi: mpt3sas: fix ioc->base_readl() use
  scsi: mpt3sas: fix event callback log_code value handling
  scsi: mpt3sas: fix adapter replyPostRegisterIndex declaration

 drivers/scsi/mpt3sas/mpt3sas_base.c  | 32 ++++++++++++++--------------
 drivers/scsi/mpt3sas/mpt3sas_base.h  |  2 +-
 drivers/scsi/mpt3sas/mpt3sas_ctl.c   | 11 +++++-----
 drivers/scsi/mpt3sas/mpt3sas_scsih.c |  6 +++---
 4 files changed, 26 insertions(+), 25 deletions(-)

Comments

Martin K. Petersen March 9, 2022, 3:56 a.m. UTC | #1 
Sreekanth,

> This series fix (remove) all sparse warnings generated when compiling
> the mpt3sas driver. All warnings are related to __iomem access and
> endianness.

Please review this series and validate the patch 5 modification.

Thanks!
Sreekanth Reddy March 9, 2022, 6:35 a.m. UTC | #2 
On Wed, Mar 9, 2022 at 9:26 AM Martin K. Petersen
<martin.petersen@oracle.com> wrote:
>
>
> Sreekanth,
>
> > This series fix (remove) all sparse warnings generated when compiling
> > the mpt3sas driver. All warnings are related to __iomem access and
> > endianness.
>
> Please review this series and validate the patch 5 modification.

Martin,
This patch set looks good, but before acknowledging this patch set I
just wanted to do some basic testing on a big endian machine.
Currently I don't have a big endian machine, internally I am checking
to get access to big endian machines. Meanwhile if anyone does a basic
testing on any big endian machine then please let me know. I will add
the acknowledgement signature.

Thanks,
Sreekanth

>
> Thanks!
>
> --
> Martin K. Petersen      Oracle Linux Engineering
Damien Le Moal April 26, 2022, 4:25 a.m. UTC | #3 
On 3/8/22 08:48, Damien Le Moal wrote:
> This series fix (remove) all sparse warnings generated when compiling
> the mpt3sas driver. All warnings are related to __iomem access and
> endianness.
> 
> The series was tested on top of Martin's 5.18/scsi-staging branch with a
> 9400-8i HBA with direct attached iSAS and SATA drives. The fixes need
> careful review by the maintainers as there is no documentation clearly
> explaning the proper endianness of the values touched.

Martin,

Can we get this one queued for 5.19 ?

> 
> Changes from v2:
> * Reworked patch 5 to keep writel() calls. Sparse warnings are
>   suppressed with a declaration fix.
> 
> Changes from v1:
> * Reworked patch 1 to remove the TaskMID field type change and simplify
>   _ctl_set_task_mid() code.
> 
> Damien Le Moal (5):
>   scsi: mpt3sas: fix _ctl_set_task_mid() TaskMID check
>   scsi: mpt3sas: Fix writel() use
>   scsi: mpt3sas: fix ioc->base_readl() use
>   scsi: mpt3sas: fix event callback log_code value handling
>   scsi: mpt3sas: fix adapter replyPostRegisterIndex declaration
> 
>  drivers/scsi/mpt3sas/mpt3sas_base.c  | 32 ++++++++++++++--------------
>  drivers/scsi/mpt3sas/mpt3sas_base.h  |  2 +-
>  drivers/scsi/mpt3sas/mpt3sas_ctl.c   | 11 +++++-----
>  drivers/scsi/mpt3sas/mpt3sas_scsih.c |  6 +++---
>  4 files changed, 26 insertions(+), 25 deletions(-)
>
Martin K. Petersen April 26, 2022, 10:37 a.m. UTC | #4 
Damien,

> Can we get this one queued for 5.19 ?

It's already in scsi-staging. Had some mail problems so not all merge
notifications went out.
Damien Le Moal April 26, 2022, 10:44 a.m. UTC | #5 
On 4/26/22 19:37, Martin K. Petersen wrote:
> 
> Damien,
> 
>> Can we get this one queued for 5.19 ?
> 
> It's already in scsi-staging. Had some mail problems so not all merge
> notifications went out.
> 

OK. Thanks !
Martin K. Petersen May 3, 2022, 12:51 a.m. UTC | #6 
On Tue, 8 Mar 2022 08:48:49 +0900, Damien Le Moal wrote:

> This series fix (remove) all sparse warnings generated when compiling
> the mpt3sas driver. All warnings are related to __iomem access and
> endianness.
> 
> The series was tested on top of Martin's 5.18/scsi-staging branch with a
> 9400-8i HBA with direct attached iSAS and SATA drives. The fixes need
> careful review by the maintainers as there is no documentation clearly
> explaning the proper endianness of the values touched.
> 
> [...]

Applied to 5.19/scsi-queue, thanks!

[1/5] scsi: mpt3sas: fix _ctl_set_task_mid() TaskMID check
      https://git.kernel.org/mkp/scsi/c/dceaef94a475
[2/5] scsi: mpt3sas: Fix writel() use
      https://git.kernel.org/mkp/scsi/c/b4efbec4c2a7
[3/5] scsi: mpt3sas: fix ioc->base_readl() use
      https://git.kernel.org/mkp/scsi/c/7ab4d2441b95
[4/5] scsi: mpt3sas: fix event callback log_code value handling
      https://git.kernel.org/mkp/scsi/c/82b4420c288c
[5/5] scsi: mpt3sas: fix adapter replyPostRegisterIndex declaration
      https://git.kernel.org/mkp/scsi/c/fe413ab32b24
matoro Aug. 2, 2022, 8:27 p.m. UTC | #7 
Hi folks, sorry for the lateness, unfortunately this is in fact broken 
on BE.  I use mpt3sas on sparc and my drives fail to come up on 5.19, 
bisected to this patchset.  Reverting both of the endian-related 
commits, b4efbec4c2a75b619fae4e8768be379e88c78687 and 
7ab4d2441b952977556672c2fe3f4c2a698cbb37, allows it to boot.  However, 
after booting, I can't load any modules - everything errors with 
"disagrees about version of symbol module_layout".  I have completely 
wiped out kernel sources, the module tree, and the kernel image, 
rebuilding both from scratch with ONLY the revert patch applied, but I 
still can't load any modules.  Presumably it would work with 
CONFIG_MODVERSIONS=n, but these CRC checks are there for a reason and I 
can't tell if it has something to do with the revert or not.

$ modprobe --dump-modversions 
/lib/modules/5.19.0-gentoo-sparc64/kernel/fs/openpromfs/openpromfs.ko  | 
grep "module_layout"
0xa6c23707      module_layout
$ grep "module_layout" /usr/src/linux/Module.symvers
0xa6c23707      module_layout   vmlinux EXPORT_SYMBOL

If you need real hardware access and do not have any on hand, please 
reach out and I can provide temporary access!

Here is the full error on vanilla 5.19:

mpt3sas version 42.100.00.00 loaded
mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem 
(32650280 kB)
mpt2sas_cm0: _base_wait_for_doorbell_not_used: failed due to timeout 
count(5000), doorbell_reg(18000000)!
mpt2sas_cm0: CurrentHostPageSize is 0: Setting default host page size to 
4k
mpt2sas_cm0: MSI-X vectors supported: 1
          no of cores: 64, max_msix_vectors: -1
mpt2sas_cm0:  0 1 1
mpt2sas_cm0: pci_alloc_irq_vectors failed (r=-22) !!!
mpt2sas_cm0: High IOPs queues : disabled
mpt2sas0: IO-APIC enabled: IRQ 4
mpt2sas_cm0: iomem(0x0000084100000000), mapped(0x(____ptrval____)), 
size(16384)
mpt2sas_cm0: ioport(0x0000085100000000), size(256)
mpt2sas_cm0: doorbell is in use (line=6869)
mpt2sas_cm0: _base_get_ioc_facts: handshake failed (r=-14)
mpt2sas_cm0: failure at 
drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
mpt2sas_cm1: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem 
(32650280 kB)
mpt2sas_cm1: _base_wait_for_doorbell_not_used: failed due to timeout 
count(5000), doorbell_reg(18000000)!
mpt2sas_cm1: CurrentHostPageSize is 0: Setting default host page size to 
4k
mpt2sas_cm1: MSI-X vectors supported: 1
          no of cores: 64, max_msix_vectors: -1
mpt2sas_cm1:  0 1 1
mpt2sas_cm1: pci_alloc_irq_vectors failed (r=-22) !!!
mpt2sas_cm1: High IOPs queues : disabled
mpt2sas1: IO-APIC enabled: IRQ 5
mpt2sas_cm1: iomem(0x0000084120000000), mapped(0x(____ptrval____)), 
size(16384)
mpt2sas_cm1: ioport(0x0000085100002000), size(256)
mpt2sas_cm1: doorbell is in use (line=6869)
mpt2sas_cm1: _base_get_ioc_facts: handshake failed (r=-14)
mpt2sas_cm1: failure at 
drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!

-------- Original Message --------
Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
Date: 2022-03-09 01:35
 From: Sreekanth Reddy <sreekanth.reddy@broadcom.com>
To: "Martin K. Petersen" <martin.petersen@oracle.com>

On Wed, Mar 9, 2022 at 9:26 AM Martin K. Petersen
<martin.petersen@oracle.com> wrote:
> 
> 
> Sreekanth,
> 
> > This series fix (remove) all sparse warnings generated when compiling
> > the mpt3sas driver. All warnings are related to __iomem access and
> > endianness.
> 
> Please review this series and validate the patch 5 modification.

Martin,
This patch set looks good, but before acknowledging this patch set I
just wanted to do some basic testing on a big endian machine.
Currently I don't have a big endian machine, internally I am checking
to get access to big endian machines. Meanwhile if anyone does a basic
testing on any big endian machine then please let me know. I will add
the acknowledgement signature.

Thanks,
Sreekanth

> 
> Thanks!
> 
> --
> Martin K. Petersen      Oracle Linux Engineering
Damien Le Moal Aug. 2, 2022, 11:36 p.m. UTC | #8 
On 8/3/22 05:27, matoro wrote:
> Hi folks, sorry for the lateness, unfortunately this is in fact broken 
> on BE.  I use mpt3sas on sparc and my drives fail to come up on 5.19, 
> bisected to this patchset.  Reverting both of the endian-related 
> commits, b4efbec4c2a75b619fae4e8768be379e88c78687 and 
> 7ab4d2441b952977556672c2fe3f4c2a698cbb37, allows it to boot.  However, 
> after booting, I can't load any modules - everything errors with 
> "disagrees about version of symbol module_layout".  I have completely 
> wiped out kernel sources, the module tree, and the kernel image, 
> rebuilding both from scratch with ONLY the revert patch applied, but I 
> still can't load any modules.  Presumably it would work with 
> CONFIG_MODVERSIONS=n, but these CRC checks are there for a reason and I 
> can't tell if it has something to do with the revert or not.

For b4efbec4c2a75b619fae4e8768be379e88c78687, removing the cpu_to_le32()
call results in the bytes actually being reversed by writel()/readl() for
your BE machine. So it looks like the values that need to be written to
the HBA have to be in CPU endian, not le32. Should be easy to fix.
And for 7ab4d2441b952977556672c2fe3f4c2a698cbb37, this looks like the same
problem.

I will be traveling and busy this week, but I can have a look at a fix
next Monday. If the Broadcom folks can send a fix faster than that, that
is of course welcome :)

> 
> $ modprobe --dump-modversions 
> /lib/modules/5.19.0-gentoo-sparc64/kernel/fs/openpromfs/openpromfs.ko  | 
> grep "module_layout"
> 0xa6c23707      module_layout
> $ grep "module_layout" /usr/src/linux/Module.symvers
> 0xa6c23707      module_layout   vmlinux EXPORT_SYMBOL
> 
> If you need real hardware access and do not have any on hand, please 
> reach out and I can provide temporary access!
> 
> Here is the full error on vanilla 5.19:
> 
> mpt3sas version 42.100.00.00 loaded
> mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem 
> (32650280 kB)
> mpt2sas_cm0: _base_wait_for_doorbell_not_used: failed due to timeout 
> count(5000), doorbell_reg(18000000)!
> mpt2sas_cm0: CurrentHostPageSize is 0: Setting default host page size to 
> 4k
> mpt2sas_cm0: MSI-X vectors supported: 1
>           no of cores: 64, max_msix_vectors: -1
> mpt2sas_cm0:  0 1 1
> mpt2sas_cm0: pci_alloc_irq_vectors failed (r=-22) !!!
> mpt2sas_cm0: High IOPs queues : disabled
> mpt2sas0: IO-APIC enabled: IRQ 4
> mpt2sas_cm0: iomem(0x0000084100000000), mapped(0x(____ptrval____)), 
> size(16384)
> mpt2sas_cm0: ioport(0x0000085100000000), size(256)
> mpt2sas_cm0: doorbell is in use (line=6869)
> mpt2sas_cm0: _base_get_ioc_facts: handshake failed (r=-14)
> mpt2sas_cm0: failure at 
> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
> mpt2sas_cm1: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem 
> (32650280 kB)
> mpt2sas_cm1: _base_wait_for_doorbell_not_used: failed due to timeout 
> count(5000), doorbell_reg(18000000)!
> mpt2sas_cm1: CurrentHostPageSize is 0: Setting default host page size to 
> 4k
> mpt2sas_cm1: MSI-X vectors supported: 1
>           no of cores: 64, max_msix_vectors: -1
> mpt2sas_cm1:  0 1 1
> mpt2sas_cm1: pci_alloc_irq_vectors failed (r=-22) !!!
> mpt2sas_cm1: High IOPs queues : disabled
> mpt2sas1: IO-APIC enabled: IRQ 5
> mpt2sas_cm1: iomem(0x0000084120000000), mapped(0x(____ptrval____)), 
> size(16384)
> mpt2sas_cm1: ioport(0x0000085100002000), size(256)
> mpt2sas_cm1: doorbell is in use (line=6869)
> mpt2sas_cm1: _base_get_ioc_facts: handshake failed (r=-14)
> mpt2sas_cm1: failure at 
> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
> 
> -------- Original Message --------
> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
> Date: 2022-03-09 01:35
>  From: Sreekanth Reddy <sreekanth.reddy@broadcom.com>
> To: "Martin K. Petersen" <martin.petersen@oracle.com>
> 
> On Wed, Mar 9, 2022 at 9:26 AM Martin K. Petersen
> <martin.petersen@oracle.com> wrote:
>>
>>
>> Sreekanth,
>>
>>> This series fix (remove) all sparse warnings generated when compiling
>>> the mpt3sas driver. All warnings are related to __iomem access and
>>> endianness.
>>
>> Please review this series and validate the patch 5 modification.
> 
> Martin,
> This patch set looks good, but before acknowledging this patch set I
> just wanted to do some basic testing on a big endian machine.
> Currently I don't have a big endian machine, internally I am checking
> to get access to big endian machines. Meanwhile if anyone does a basic
> testing on any big endian machine then please let me know. I will add
> the acknowledgement signature.
> 
> Thanks,
> Sreekanth
> 
>>
>> Thanks!
>>
>> --
>> Martin K. Petersen      Oracle Linux Engineering
matoro Aug. 11, 2022, 7:05 p.m. UTC | #9 
Just a small update, the module stuff turned out to be a separate, 
unrelated regression.  I bisected that one also (applying these reverts 
each time to allow me to boot) and reported it to Masahiro, who put in a 
fix for it here:  
https://lore.kernel.org/all/20220809141117.641543-1-masahiroy@kernel.org/ 
.  So you can ignore that stuff.  Are these two commits still planned to 
be reverted?

-------- Original Message --------
Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
Date: 2022-08-02 19:36
 From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
To: matoro <matoro_mailinglist_kernel@matoro.tk>, Sreekanth Reddy 
<sreekanth.reddy@broadcom.com>

On 8/3/22 05:27, matoro wrote:
> Hi folks, sorry for the lateness, unfortunately this is in fact broken
> on BE.  I use mpt3sas on sparc and my drives fail to come up on 5.19,
> bisected to this patchset.  Reverting both of the endian-related
> commits, b4efbec4c2a75b619fae4e8768be379e88c78687 and
> 7ab4d2441b952977556672c2fe3f4c2a698cbb37, allows it to boot.  However,
> after booting, I can't load any modules - everything errors with
> "disagrees about version of symbol module_layout".  I have completely
> wiped out kernel sources, the module tree, and the kernel image,
> rebuilding both from scratch with ONLY the revert patch applied, but I
> still can't load any modules.  Presumably it would work with
> CONFIG_MODVERSIONS=n, but these CRC checks are there for a reason and I
> can't tell if it has something to do with the revert or not.

For b4efbec4c2a75b619fae4e8768be379e88c78687, removing the cpu_to_le32()
call results in the bytes actually being reversed by writel()/readl() 
for
your BE machine. So it looks like the values that need to be written to
the HBA have to be in CPU endian, not le32. Should be easy to fix.
And for 7ab4d2441b952977556672c2fe3f4c2a698cbb37, this looks like the 
same
problem.

I will be traveling and busy this week, but I can have a look at a fix
next Monday. If the Broadcom folks can send a fix faster than that, that
is of course welcome :)

> 
> $ modprobe --dump-modversions
> /lib/modules/5.19.0-gentoo-sparc64/kernel/fs/openpromfs/openpromfs.ko  
> |
> grep "module_layout"
> 0xa6c23707      module_layout
> $ grep "module_layout" /usr/src/linux/Module.symvers
> 0xa6c23707      module_layout   vmlinux EXPORT_SYMBOL
> 
> If you need real hardware access and do not have any on hand, please
> reach out and I can provide temporary access!
> 
> Here is the full error on vanilla 5.19:
> 
> mpt3sas version 42.100.00.00 loaded
> mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem
> (32650280 kB)
> mpt2sas_cm0: _base_wait_for_doorbell_not_used: failed due to timeout
> count(5000), doorbell_reg(18000000)!
> mpt2sas_cm0: CurrentHostPageSize is 0: Setting default host page size 
> to
> 4k
> mpt2sas_cm0: MSI-X vectors supported: 1
>           no of cores: 64, max_msix_vectors: -1
> mpt2sas_cm0:  0 1 1
> mpt2sas_cm0: pci_alloc_irq_vectors failed (r=-22) !!!
> mpt2sas_cm0: High IOPs queues : disabled
> mpt2sas0: IO-APIC enabled: IRQ 4
> mpt2sas_cm0: iomem(0x0000084100000000), mapped(0x(____ptrval____)),
> size(16384)
> mpt2sas_cm0: ioport(0x0000085100000000), size(256)
> mpt2sas_cm0: doorbell is in use (line=6869)
> mpt2sas_cm0: _base_get_ioc_facts: handshake failed (r=-14)
> mpt2sas_cm0: failure at
> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
> mpt2sas_cm1: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem
> (32650280 kB)
> mpt2sas_cm1: _base_wait_for_doorbell_not_used: failed due to timeout
> count(5000), doorbell_reg(18000000)!
> mpt2sas_cm1: CurrentHostPageSize is 0: Setting default host page size 
> to
> 4k
> mpt2sas_cm1: MSI-X vectors supported: 1
>           no of cores: 64, max_msix_vectors: -1
> mpt2sas_cm1:  0 1 1
> mpt2sas_cm1: pci_alloc_irq_vectors failed (r=-22) !!!
> mpt2sas_cm1: High IOPs queues : disabled
> mpt2sas1: IO-APIC enabled: IRQ 5
> mpt2sas_cm1: iomem(0x0000084120000000), mapped(0x(____ptrval____)),
> size(16384)
> mpt2sas_cm1: ioport(0x0000085100002000), size(256)
> mpt2sas_cm1: doorbell is in use (line=6869)
> mpt2sas_cm1: _base_get_ioc_facts: handshake failed (r=-14)
> mpt2sas_cm1: failure at
> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
> 
> -------- Original Message --------
> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
> Date: 2022-03-09 01:35
>  From: Sreekanth Reddy <sreekanth.reddy@broadcom.com>
> To: "Martin K. Petersen" <martin.petersen@oracle.com>
> 
> On Wed, Mar 9, 2022 at 9:26 AM Martin K. Petersen
> <martin.petersen@oracle.com> wrote:
>> 
>> 
>> Sreekanth,
>> 
>>> This series fix (remove) all sparse warnings generated when compiling
>>> the mpt3sas driver. All warnings are related to __iomem access and
>>> endianness.
>> 
>> Please review this series and validate the patch 5 modification.
> 
> Martin,
> This patch set looks good, but before acknowledging this patch set I
> just wanted to do some basic testing on a big endian machine.
> Currently I don't have a big endian machine, internally I am checking
> to get access to big endian machines. Meanwhile if anyone does a basic
> testing on any big endian machine then please let me know. I will add
> the acknowledgement signature.
> 
> Thanks,
> Sreekanth
> 
>> 
>> Thanks!
>> 
>> --
>> Martin K. Petersen      Oracle Linux Engineering
Damien Le Moal Aug. 11, 2022, 7:17 p.m. UTC | #10 
On 2022/08/11 12:05, matoro wrote:
> Just a small update, the module stuff turned out to be a separate, 
> unrelated regression.  I bisected that one also (applying these reverts 
> each time to allow me to boot) and reported it to Masahiro, who put in a 
> fix for it here:  
> https://lore.kernel.org/all/20220809141117.641543-1-masahiroy@kernel.org/ 
> .  So you can ignore that stuff.  Are these two commits still planned to 
> be reverted?

Revert should be a last resort (I really want to get rid of all these sparse
warnings !). Let me first try to generate a fix for you to test.

> 
> -------- Original Message --------
> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
> Date: 2022-08-02 19:36
>  From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
> To: matoro <matoro_mailinglist_kernel@matoro.tk>, Sreekanth Reddy 
> <sreekanth.reddy@broadcom.com>
> 
> On 8/3/22 05:27, matoro wrote:
>> Hi folks, sorry for the lateness, unfortunately this is in fact broken
>> on BE.  I use mpt3sas on sparc and my drives fail to come up on 5.19,
>> bisected to this patchset.  Reverting both of the endian-related
>> commits, b4efbec4c2a75b619fae4e8768be379e88c78687 and
>> 7ab4d2441b952977556672c2fe3f4c2a698cbb37, allows it to boot.  However,
>> after booting, I can't load any modules - everything errors with
>> "disagrees about version of symbol module_layout".  I have completely
>> wiped out kernel sources, the module tree, and the kernel image,
>> rebuilding both from scratch with ONLY the revert patch applied, but I
>> still can't load any modules.  Presumably it would work with
>> CONFIG_MODVERSIONS=n, but these CRC checks are there for a reason and I
>> can't tell if it has something to do with the revert or not.
> 
> For b4efbec4c2a75b619fae4e8768be379e88c78687, removing the cpu_to_le32()
> call results in the bytes actually being reversed by writel()/readl() 
> for
> your BE machine. So it looks like the values that need to be written to
> the HBA have to be in CPU endian, not le32. Should be easy to fix.
> And for 7ab4d2441b952977556672c2fe3f4c2a698cbb37, this looks like the 
> same
> problem.
> 
> I will be traveling and busy this week, but I can have a look at a fix
> next Monday. If the Broadcom folks can send a fix faster than that, that
> is of course welcome :)
> 
>>
>> $ modprobe --dump-modversions
>> /lib/modules/5.19.0-gentoo-sparc64/kernel/fs/openpromfs/openpromfs.ko  
>> |
>> grep "module_layout"
>> 0xa6c23707      module_layout
>> $ grep "module_layout" /usr/src/linux/Module.symvers
>> 0xa6c23707      module_layout   vmlinux EXPORT_SYMBOL
>>
>> If you need real hardware access and do not have any on hand, please
>> reach out and I can provide temporary access!
>>
>> Here is the full error on vanilla 5.19:
>>
>> mpt3sas version 42.100.00.00 loaded
>> mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem
>> (32650280 kB)
>> mpt2sas_cm0: _base_wait_for_doorbell_not_used: failed due to timeout
>> count(5000), doorbell_reg(18000000)!
>> mpt2sas_cm0: CurrentHostPageSize is 0: Setting default host page size 
>> to
>> 4k
>> mpt2sas_cm0: MSI-X vectors supported: 1
>>           no of cores: 64, max_msix_vectors: -1
>> mpt2sas_cm0:  0 1 1
>> mpt2sas_cm0: pci_alloc_irq_vectors failed (r=-22) !!!
>> mpt2sas_cm0: High IOPs queues : disabled
>> mpt2sas0: IO-APIC enabled: IRQ 4
>> mpt2sas_cm0: iomem(0x0000084100000000), mapped(0x(____ptrval____)),
>> size(16384)
>> mpt2sas_cm0: ioport(0x0000085100000000), size(256)
>> mpt2sas_cm0: doorbell is in use (line=6869)
>> mpt2sas_cm0: _base_get_ioc_facts: handshake failed (r=-14)
>> mpt2sas_cm0: failure at
>> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
>> mpt2sas_cm1: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem
>> (32650280 kB)
>> mpt2sas_cm1: _base_wait_for_doorbell_not_used: failed due to timeout
>> count(5000), doorbell_reg(18000000)!
>> mpt2sas_cm1: CurrentHostPageSize is 0: Setting default host page size 
>> to
>> 4k
>> mpt2sas_cm1: MSI-X vectors supported: 1
>>           no of cores: 64, max_msix_vectors: -1
>> mpt2sas_cm1:  0 1 1
>> mpt2sas_cm1: pci_alloc_irq_vectors failed (r=-22) !!!
>> mpt2sas_cm1: High IOPs queues : disabled
>> mpt2sas1: IO-APIC enabled: IRQ 5
>> mpt2sas_cm1: iomem(0x0000084120000000), mapped(0x(____ptrval____)),
>> size(16384)
>> mpt2sas_cm1: ioport(0x0000085100002000), size(256)
>> mpt2sas_cm1: doorbell is in use (line=6869)
>> mpt2sas_cm1: _base_get_ioc_facts: handshake failed (r=-14)
>> mpt2sas_cm1: failure at
>> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
>>
>> -------- Original Message --------
>> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
>> Date: 2022-03-09 01:35
>>  From: Sreekanth Reddy <sreekanth.reddy@broadcom.com>
>> To: "Martin K. Petersen" <martin.petersen@oracle.com>
>>
>> On Wed, Mar 9, 2022 at 9:26 AM Martin K. Petersen
>> <martin.petersen@oracle.com> wrote:
>>>
>>>
>>> Sreekanth,
>>>
>>>> This series fix (remove) all sparse warnings generated when compiling
>>>> the mpt3sas driver. All warnings are related to __iomem access and
>>>> endianness.
>>>
>>> Please review this series and validate the patch 5 modification.
>>
>> Martin,
>> This patch set looks good, but before acknowledging this patch set I
>> just wanted to do some basic testing on a big endian machine.
>> Currently I don't have a big endian machine, internally I am checking
>> to get access to big endian machines. Meanwhile if anyone does a basic
>> testing on any big endian machine then please let me know. I will add
>> the acknowledgement signature.
>>
>> Thanks,
>> Sreekanth
>>
>>>
>>> Thanks!
>>>
>>> --
>>> Martin K. Petersen      Oracle Linux Engineering
>
matoro Aug. 22, 2022, 5:51 p.m. UTC | #11 
Hi Damien, were you able to put together a fix to test?  We're up to 
5.19.3 now and 5.18 was just marked EOL, so I want to make sure this 
doesn't drop off the radar.

-------- Original Message --------
Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
Date: 2022-08-11 15:17
 From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
To: matoro <matoro_mailinglist_kernel@matoro.tk>

On 2022/08/11 12:05, matoro wrote:
> Just a small update, the module stuff turned out to be a separate,
> unrelated regression.  I bisected that one also (applying these reverts
> each time to allow me to boot) and reported it to Masahiro, who put in 
> a
> fix for it here:
> https://lore.kernel.org/all/20220809141117.641543-1-masahiroy@kernel.org/
> .  So you can ignore that stuff.  Are these two commits still planned 
> to
> be reverted?

Revert should be a last resort (I really want to get rid of all these 
sparse
warnings !). Let me first try to generate a fix for you to test.

> 
> -------- Original Message --------
> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
> Date: 2022-08-02 19:36
>  From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
> To: matoro <matoro_mailinglist_kernel@matoro.tk>, Sreekanth Reddy
> <sreekanth.reddy@broadcom.com>
> 
> On 8/3/22 05:27, matoro wrote:
>> Hi folks, sorry for the lateness, unfortunately this is in fact broken
>> on BE.  I use mpt3sas on sparc and my drives fail to come up on 5.19,
>> bisected to this patchset.  Reverting both of the endian-related
>> commits, b4efbec4c2a75b619fae4e8768be379e88c78687 and
>> 7ab4d2441b952977556672c2fe3f4c2a698cbb37, allows it to boot.  However,
>> after booting, I can't load any modules - everything errors with
>> "disagrees about version of symbol module_layout".  I have completely
>> wiped out kernel sources, the module tree, and the kernel image,
>> rebuilding both from scratch with ONLY the revert patch applied, but I
>> still can't load any modules.  Presumably it would work with
>> CONFIG_MODVERSIONS=n, but these CRC checks are there for a reason and 
>> I
>> can't tell if it has something to do with the revert or not.
> 
> For b4efbec4c2a75b619fae4e8768be379e88c78687, removing the 
> cpu_to_le32()
> call results in the bytes actually being reversed by writel()/readl()
> for
> your BE machine. So it looks like the values that need to be written to
> the HBA have to be in CPU endian, not le32. Should be easy to fix.
> And for 7ab4d2441b952977556672c2fe3f4c2a698cbb37, this looks like the
> same
> problem.
> 
> I will be traveling and busy this week, but I can have a look at a fix
> next Monday. If the Broadcom folks can send a fix faster than that, 
> that
> is of course welcome :)
> 
>> 
>> $ modprobe --dump-modversions
>> /lib/modules/5.19.0-gentoo-sparc64/kernel/fs/openpromfs/openpromfs.ko
>> |
>> grep "module_layout"
>> 0xa6c23707      module_layout
>> $ grep "module_layout" /usr/src/linux/Module.symvers
>> 0xa6c23707      module_layout   vmlinux EXPORT_SYMBOL
>> 
>> If you need real hardware access and do not have any on hand, please
>> reach out and I can provide temporary access!
>> 
>> Here is the full error on vanilla 5.19:
>> 
>> mpt3sas version 42.100.00.00 loaded
>> mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem
>> (32650280 kB)
>> mpt2sas_cm0: _base_wait_for_doorbell_not_used: failed due to timeout
>> count(5000), doorbell_reg(18000000)!
>> mpt2sas_cm0: CurrentHostPageSize is 0: Setting default host page size
>> to
>> 4k
>> mpt2sas_cm0: MSI-X vectors supported: 1
>>           no of cores: 64, max_msix_vectors: -1
>> mpt2sas_cm0:  0 1 1
>> mpt2sas_cm0: pci_alloc_irq_vectors failed (r=-22) !!!
>> mpt2sas_cm0: High IOPs queues : disabled
>> mpt2sas0: IO-APIC enabled: IRQ 4
>> mpt2sas_cm0: iomem(0x0000084100000000), mapped(0x(____ptrval____)),
>> size(16384)
>> mpt2sas_cm0: ioport(0x0000085100000000), size(256)
>> mpt2sas_cm0: doorbell is in use (line=6869)
>> mpt2sas_cm0: _base_get_ioc_facts: handshake failed (r=-14)
>> mpt2sas_cm0: failure at
>> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
>> mpt2sas_cm1: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem
>> (32650280 kB)
>> mpt2sas_cm1: _base_wait_for_doorbell_not_used: failed due to timeout
>> count(5000), doorbell_reg(18000000)!
>> mpt2sas_cm1: CurrentHostPageSize is 0: Setting default host page size
>> to
>> 4k
>> mpt2sas_cm1: MSI-X vectors supported: 1
>>           no of cores: 64, max_msix_vectors: -1
>> mpt2sas_cm1:  0 1 1
>> mpt2sas_cm1: pci_alloc_irq_vectors failed (r=-22) !!!
>> mpt2sas_cm1: High IOPs queues : disabled
>> mpt2sas1: IO-APIC enabled: IRQ 5
>> mpt2sas_cm1: iomem(0x0000084120000000), mapped(0x(____ptrval____)),
>> size(16384)
>> mpt2sas_cm1: ioport(0x0000085100002000), size(256)
>> mpt2sas_cm1: doorbell is in use (line=6869)
>> mpt2sas_cm1: _base_get_ioc_facts: handshake failed (r=-14)
>> mpt2sas_cm1: failure at
>> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
>> 
>> -------- Original Message --------
>> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
>> Date: 2022-03-09 01:35
>>  From: Sreekanth Reddy <sreekanth.reddy@broadcom.com>
>> To: "Martin K. Petersen" <martin.petersen@oracle.com>
>> 
>> On Wed, Mar 9, 2022 at 9:26 AM Martin K. Petersen
>> <martin.petersen@oracle.com> wrote:
>>> 
>>> 
>>> Sreekanth,
>>> 
>>>> This series fix (remove) all sparse warnings generated when 
>>>> compiling
>>>> the mpt3sas driver. All warnings are related to __iomem access and
>>>> endianness.
>>> 
>>> Please review this series and validate the patch 5 modification.
>> 
>> Martin,
>> This patch set looks good, but before acknowledging this patch set I
>> just wanted to do some basic testing on a big endian machine.
>> Currently I don't have a big endian machine, internally I am checking
>> to get access to big endian machines. Meanwhile if anyone does a basic
>> testing on any big endian machine then please let me know. I will add
>> the acknowledgement signature.
>> 
>> Thanks,
>> Sreekanth
>> 
>>> 
>>> Thanks!
>>> 
>>> --
>>> Martin K. Petersen      Oracle Linux Engineering
>
Damien Le Moal Aug. 22, 2022, 5:53 p.m. UTC | #12 
On 2022/08/22 10:51, matoro wrote:
> Hi Damien, were you able to put together a fix to test?  We're up to 
> 5.19.3 now and 5.18 was just marked EOL, so I want to make sure this 
> doesn't drop off the radar.

No, sorry, I have been busy as I am traveling. Flying back home this week, so
likely will have more time next week.

> 
> -------- Original Message --------
> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
> Date: 2022-08-11 15:17
>  From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
> To: matoro <matoro_mailinglist_kernel@matoro.tk>
> 
> On 2022/08/11 12:05, matoro wrote:
>> Just a small update, the module stuff turned out to be a separate,
>> unrelated regression.  I bisected that one also (applying these reverts
>> each time to allow me to boot) and reported it to Masahiro, who put in 
>> a
>> fix for it here:
>> https://lore.kernel.org/all/20220809141117.641543-1-masahiroy@kernel.org/
>> .  So you can ignore that stuff.  Are these two commits still planned 
>> to
>> be reverted?
> 
> Revert should be a last resort (I really want to get rid of all these 
> sparse
> warnings !). Let me first try to generate a fix for you to test.
> 
>>
>> -------- Original Message --------
>> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
>> Date: 2022-08-02 19:36
>>  From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
>> To: matoro <matoro_mailinglist_kernel@matoro.tk>, Sreekanth Reddy
>> <sreekanth.reddy@broadcom.com>
>>
>> On 8/3/22 05:27, matoro wrote:
>>> Hi folks, sorry for the lateness, unfortunately this is in fact broken
>>> on BE.  I use mpt3sas on sparc and my drives fail to come up on 5.19,
>>> bisected to this patchset.  Reverting both of the endian-related
>>> commits, b4efbec4c2a75b619fae4e8768be379e88c78687 and
>>> 7ab4d2441b952977556672c2fe3f4c2a698cbb37, allows it to boot.  However,
>>> after booting, I can't load any modules - everything errors with
>>> "disagrees about version of symbol module_layout".  I have completely
>>> wiped out kernel sources, the module tree, and the kernel image,
>>> rebuilding both from scratch with ONLY the revert patch applied, but I
>>> still can't load any modules.  Presumably it would work with
>>> CONFIG_MODVERSIONS=n, but these CRC checks are there for a reason and 
>>> I
>>> can't tell if it has something to do with the revert or not.
>>
>> For b4efbec4c2a75b619fae4e8768be379e88c78687, removing the 
>> cpu_to_le32()
>> call results in the bytes actually being reversed by writel()/readl()
>> for
>> your BE machine. So it looks like the values that need to be written to
>> the HBA have to be in CPU endian, not le32. Should be easy to fix.
>> And for 7ab4d2441b952977556672c2fe3f4c2a698cbb37, this looks like the
>> same
>> problem.
>>
>> I will be traveling and busy this week, but I can have a look at a fix
>> next Monday. If the Broadcom folks can send a fix faster than that, 
>> that
>> is of course welcome :)
>>
>>>
>>> $ modprobe --dump-modversions
>>> /lib/modules/5.19.0-gentoo-sparc64/kernel/fs/openpromfs/openpromfs.ko
>>> |
>>> grep "module_layout"
>>> 0xa6c23707      module_layout
>>> $ grep "module_layout" /usr/src/linux/Module.symvers
>>> 0xa6c23707      module_layout   vmlinux EXPORT_SYMBOL
>>>
>>> If you need real hardware access and do not have any on hand, please
>>> reach out and I can provide temporary access!
>>>
>>> Here is the full error on vanilla 5.19:
>>>
>>> mpt3sas version 42.100.00.00 loaded
>>> mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem
>>> (32650280 kB)
>>> mpt2sas_cm0: _base_wait_for_doorbell_not_used: failed due to timeout
>>> count(5000), doorbell_reg(18000000)!
>>> mpt2sas_cm0: CurrentHostPageSize is 0: Setting default host page size
>>> to
>>> 4k
>>> mpt2sas_cm0: MSI-X vectors supported: 1
>>>           no of cores: 64, max_msix_vectors: -1
>>> mpt2sas_cm0:  0 1 1
>>> mpt2sas_cm0: pci_alloc_irq_vectors failed (r=-22) !!!
>>> mpt2sas_cm0: High IOPs queues : disabled
>>> mpt2sas0: IO-APIC enabled: IRQ 4
>>> mpt2sas_cm0: iomem(0x0000084100000000), mapped(0x(____ptrval____)),
>>> size(16384)
>>> mpt2sas_cm0: ioport(0x0000085100000000), size(256)
>>> mpt2sas_cm0: doorbell is in use (line=6869)
>>> mpt2sas_cm0: _base_get_ioc_facts: handshake failed (r=-14)
>>> mpt2sas_cm0: failure at
>>> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
>>> mpt2sas_cm1: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem
>>> (32650280 kB)
>>> mpt2sas_cm1: _base_wait_for_doorbell_not_used: failed due to timeout
>>> count(5000), doorbell_reg(18000000)!
>>> mpt2sas_cm1: CurrentHostPageSize is 0: Setting default host page size
>>> to
>>> 4k
>>> mpt2sas_cm1: MSI-X vectors supported: 1
>>>           no of cores: 64, max_msix_vectors: -1
>>> mpt2sas_cm1:  0 1 1
>>> mpt2sas_cm1: pci_alloc_irq_vectors failed (r=-22) !!!
>>> mpt2sas_cm1: High IOPs queues : disabled
>>> mpt2sas1: IO-APIC enabled: IRQ 5
>>> mpt2sas_cm1: iomem(0x0000084120000000), mapped(0x(____ptrval____)),
>>> size(16384)
>>> mpt2sas_cm1: ioport(0x0000085100002000), size(256)
>>> mpt2sas_cm1: doorbell is in use (line=6869)
>>> mpt2sas_cm1: _base_get_ioc_facts: handshake failed (r=-14)
>>> mpt2sas_cm1: failure at
>>> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
>>>
>>> -------- Original Message --------
>>> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
>>> Date: 2022-03-09 01:35
>>>  From: Sreekanth Reddy <sreekanth.reddy@broadcom.com>
>>> To: "Martin K. Petersen" <martin.petersen@oracle.com>
>>>
>>> On Wed, Mar 9, 2022 at 9:26 AM Martin K. Petersen
>>> <martin.petersen@oracle.com> wrote:
>>>>
>>>>
>>>> Sreekanth,
>>>>
>>>>> This series fix (remove) all sparse warnings generated when 
>>>>> compiling
>>>>> the mpt3sas driver. All warnings are related to __iomem access and
>>>>> endianness.
>>>>
>>>> Please review this series and validate the patch 5 modification.
>>>
>>> Martin,
>>> This patch set looks good, but before acknowledging this patch set I
>>> just wanted to do some basic testing on a big endian machine.
>>> Currently I don't have a big endian machine, internally I am checking
>>> to get access to big endian machines. Meanwhile if anyone does a basic
>>> testing on any big endian machine then please let me know. I will add
>>> the acknowledgement signature.
>>>
>>> Thanks,
>>> Sreekanth
>>>
>>>>
>>>> Thanks!
>>>>
>>>> --
>>>> Martin K. Petersen      Oracle Linux Engineering
>>
>
matoro Sept. 15, 2022, 3:55 p.m. UTC | #13 
Hi Damien, apologies for continuing to bother.  Did you get a chance to 
put together a quick fix for this to test?

-------- Original Message --------
Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
Date: 2022-08-22 13:53
 From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
To: matoro <matoro_mailinglist_kernel@matoro.tk>

On 2022/08/22 10:51, matoro wrote:
> Hi Damien, were you able to put together a fix to test?  We're up to
> 5.19.3 now and 5.18 was just marked EOL, so I want to make sure this
> doesn't drop off the radar.

No, sorry, I have been busy as I am traveling. Flying back home this 
week, so
likely will have more time next week.

> 
> -------- Original Message --------
> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
> Date: 2022-08-11 15:17
>  From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
> To: matoro <matoro_mailinglist_kernel@matoro.tk>
> 
> On 2022/08/11 12:05, matoro wrote:
>> Just a small update, the module stuff turned out to be a separate,
>> unrelated regression.  I bisected that one also (applying these 
>> reverts
>> each time to allow me to boot) and reported it to Masahiro, who put in
>> a
>> fix for it here:
>> https://lore.kernel.org/all/20220809141117.641543-1-masahiroy@kernel.org/
>> .  So you can ignore that stuff.  Are these two commits still planned
>> to
>> be reverted?
> 
> Revert should be a last resort (I really want to get rid of all these
> sparse
> warnings !). Let me first try to generate a fix for you to test.
> 
>> 
>> -------- Original Message --------
>> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
>> Date: 2022-08-02 19:36
>>  From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
>> To: matoro <matoro_mailinglist_kernel@matoro.tk>, Sreekanth Reddy
>> <sreekanth.reddy@broadcom.com>
>> 
>> On 8/3/22 05:27, matoro wrote:
>>> Hi folks, sorry for the lateness, unfortunately this is in fact 
>>> broken
>>> on BE.  I use mpt3sas on sparc and my drives fail to come up on 5.19,
>>> bisected to this patchset.  Reverting both of the endian-related
>>> commits, b4efbec4c2a75b619fae4e8768be379e88c78687 and
>>> 7ab4d2441b952977556672c2fe3f4c2a698cbb37, allows it to boot.  
>>> However,
>>> after booting, I can't load any modules - everything errors with
>>> "disagrees about version of symbol module_layout".  I have completely
>>> wiped out kernel sources, the module tree, and the kernel image,
>>> rebuilding both from scratch with ONLY the revert patch applied, but 
>>> I
>>> still can't load any modules.  Presumably it would work with
>>> CONFIG_MODVERSIONS=n, but these CRC checks are there for a reason and
>>> I
>>> can't tell if it has something to do with the revert or not.
>> 
>> For b4efbec4c2a75b619fae4e8768be379e88c78687, removing the
>> cpu_to_le32()
>> call results in the bytes actually being reversed by writel()/readl()
>> for
>> your BE machine. So it looks like the values that need to be written 
>> to
>> the HBA have to be in CPU endian, not le32. Should be easy to fix.
>> And for 7ab4d2441b952977556672c2fe3f4c2a698cbb37, this looks like the
>> same
>> problem.
>> 
>> I will be traveling and busy this week, but I can have a look at a fix
>> next Monday. If the Broadcom folks can send a fix faster than that,
>> that
>> is of course welcome :)
>> 
>>> 
>>> $ modprobe --dump-modversions
>>> /lib/modules/5.19.0-gentoo-sparc64/kernel/fs/openpromfs/openpromfs.ko
>>> |
>>> grep "module_layout"
>>> 0xa6c23707      module_layout
>>> $ grep "module_layout" /usr/src/linux/Module.symvers
>>> 0xa6c23707      module_layout   vmlinux EXPORT_SYMBOL
>>> 
>>> If you need real hardware access and do not have any on hand, please
>>> reach out and I can provide temporary access!
>>> 
>>> Here is the full error on vanilla 5.19:
>>> 
>>> mpt3sas version 42.100.00.00 loaded
>>> mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem
>>> (32650280 kB)
>>> mpt2sas_cm0: _base_wait_for_doorbell_not_used: failed due to timeout
>>> count(5000), doorbell_reg(18000000)!
>>> mpt2sas_cm0: CurrentHostPageSize is 0: Setting default host page size
>>> to
>>> 4k
>>> mpt2sas_cm0: MSI-X vectors supported: 1
>>>           no of cores: 64, max_msix_vectors: -1
>>> mpt2sas_cm0:  0 1 1
>>> mpt2sas_cm0: pci_alloc_irq_vectors failed (r=-22) !!!
>>> mpt2sas_cm0: High IOPs queues : disabled
>>> mpt2sas0: IO-APIC enabled: IRQ 4
>>> mpt2sas_cm0: iomem(0x0000084100000000), mapped(0x(____ptrval____)),
>>> size(16384)
>>> mpt2sas_cm0: ioport(0x0000085100000000), size(256)
>>> mpt2sas_cm0: doorbell is in use (line=6869)
>>> mpt2sas_cm0: _base_get_ioc_facts: handshake failed (r=-14)
>>> mpt2sas_cm0: failure at
>>> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
>>> mpt2sas_cm1: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem
>>> (32650280 kB)
>>> mpt2sas_cm1: _base_wait_for_doorbell_not_used: failed due to timeout
>>> count(5000), doorbell_reg(18000000)!
>>> mpt2sas_cm1: CurrentHostPageSize is 0: Setting default host page size
>>> to
>>> 4k
>>> mpt2sas_cm1: MSI-X vectors supported: 1
>>>           no of cores: 64, max_msix_vectors: -1
>>> mpt2sas_cm1:  0 1 1
>>> mpt2sas_cm1: pci_alloc_irq_vectors failed (r=-22) !!!
>>> mpt2sas_cm1: High IOPs queues : disabled
>>> mpt2sas1: IO-APIC enabled: IRQ 5
>>> mpt2sas_cm1: iomem(0x0000084120000000), mapped(0x(____ptrval____)),
>>> size(16384)
>>> mpt2sas_cm1: ioport(0x0000085100002000), size(256)
>>> mpt2sas_cm1: doorbell is in use (line=6869)
>>> mpt2sas_cm1: _base_get_ioc_facts: handshake failed (r=-14)
>>> mpt2sas_cm1: failure at
>>> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
>>> 
>>> -------- Original Message --------
>>> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
>>> Date: 2022-03-09 01:35
>>>  From: Sreekanth Reddy <sreekanth.reddy@broadcom.com>
>>> To: "Martin K. Petersen" <martin.petersen@oracle.com>
>>> 
>>> On Wed, Mar 9, 2022 at 9:26 AM Martin K. Petersen
>>> <martin.petersen@oracle.com> wrote:
>>>> 
>>>> 
>>>> Sreekanth,
>>>> 
>>>>> This series fix (remove) all sparse warnings generated when
>>>>> compiling
>>>>> the mpt3sas driver. All warnings are related to __iomem access and
>>>>> endianness.
>>>> 
>>>> Please review this series and validate the patch 5 modification.
>>> 
>>> Martin,
>>> This patch set looks good, but before acknowledging this patch set I
>>> just wanted to do some basic testing on a big endian machine.
>>> Currently I don't have a big endian machine, internally I am checking
>>> to get access to big endian machines. Meanwhile if anyone does a 
>>> basic
>>> testing on any big endian machine then please let me know. I will add
>>> the acknowledgement signature.
>>> 
>>> Thanks,
>>> Sreekanth
>>> 
>>>> 
>>>> Thanks!
>>>> 
>>>> --
>>>> Martin K. Petersen      Oracle Linux Engineering
>> 
>
Damien Le Moal Sept. 16, 2022, 8:39 a.m. UTC | #14 
On 2022/09/15 16:55, matoro wrote:
> Hi Damien, apologies for continuing to bother.  Did you get a chance to 
> put together a quick fix for this to test?

Not yet. Have been super busy with Plumbers & travel and other stuff. This is
still at the top of my to do list. Will try to send you something to test today.

> 
> -------- Original Message --------
> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
> Date: 2022-08-22 13:53
>  From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
> To: matoro <matoro_mailinglist_kernel@matoro.tk>
> 
> On 2022/08/22 10:51, matoro wrote:
>> Hi Damien, were you able to put together a fix to test?  We're up to
>> 5.19.3 now and 5.18 was just marked EOL, so I want to make sure this
>> doesn't drop off the radar.
> 
> No, sorry, I have been busy as I am traveling. Flying back home this 
> week, so
> likely will have more time next week.
> 
>>
>> -------- Original Message --------
>> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
>> Date: 2022-08-11 15:17
>>  From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
>> To: matoro <matoro_mailinglist_kernel@matoro.tk>
>>
>> On 2022/08/11 12:05, matoro wrote:
>>> Just a small update, the module stuff turned out to be a separate,
>>> unrelated regression.  I bisected that one also (applying these 
>>> reverts
>>> each time to allow me to boot) and reported it to Masahiro, who put in
>>> a
>>> fix for it here:
>>> https://lore.kernel.org/all/20220809141117.641543-1-masahiroy@kernel.org/
>>> .  So you can ignore that stuff.  Are these two commits still planned
>>> to
>>> be reverted?
>>
>> Revert should be a last resort (I really want to get rid of all these
>> sparse
>> warnings !). Let me first try to generate a fix for you to test.
>>
>>>
>>> -------- Original Message --------
>>> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
>>> Date: 2022-08-02 19:36
>>>  From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
>>> To: matoro <matoro_mailinglist_kernel@matoro.tk>, Sreekanth Reddy
>>> <sreekanth.reddy@broadcom.com>
>>>
>>> On 8/3/22 05:27, matoro wrote:
>>>> Hi folks, sorry for the lateness, unfortunately this is in fact 
>>>> broken
>>>> on BE.  I use mpt3sas on sparc and my drives fail to come up on 5.19,
>>>> bisected to this patchset.  Reverting both of the endian-related
>>>> commits, b4efbec4c2a75b619fae4e8768be379e88c78687 and
>>>> 7ab4d2441b952977556672c2fe3f4c2a698cbb37, allows it to boot.  
>>>> However,
>>>> after booting, I can't load any modules - everything errors with
>>>> "disagrees about version of symbol module_layout".  I have completely
>>>> wiped out kernel sources, the module tree, and the kernel image,
>>>> rebuilding both from scratch with ONLY the revert patch applied, but 
>>>> I
>>>> still can't load any modules.  Presumably it would work with
>>>> CONFIG_MODVERSIONS=n, but these CRC checks are there for a reason and
>>>> I
>>>> can't tell if it has something to do with the revert or not.
>>>
>>> For b4efbec4c2a75b619fae4e8768be379e88c78687, removing the
>>> cpu_to_le32()
>>> call results in the bytes actually being reversed by writel()/readl()
>>> for
>>> your BE machine. So it looks like the values that need to be written 
>>> to
>>> the HBA have to be in CPU endian, not le32. Should be easy to fix.
>>> And for 7ab4d2441b952977556672c2fe3f4c2a698cbb37, this looks like the
>>> same
>>> problem.
>>>
>>> I will be traveling and busy this week, but I can have a look at a fix
>>> next Monday. If the Broadcom folks can send a fix faster than that,
>>> that
>>> is of course welcome :)
>>>
>>>>
>>>> $ modprobe --dump-modversions
>>>> /lib/modules/5.19.0-gentoo-sparc64/kernel/fs/openpromfs/openpromfs.ko
>>>> |
>>>> grep "module_layout"
>>>> 0xa6c23707      module_layout
>>>> $ grep "module_layout" /usr/src/linux/Module.symvers
>>>> 0xa6c23707      module_layout   vmlinux EXPORT_SYMBOL
>>>>
>>>> If you need real hardware access and do not have any on hand, please
>>>> reach out and I can provide temporary access!
>>>>
>>>> Here is the full error on vanilla 5.19:
>>>>
>>>> mpt3sas version 42.100.00.00 loaded
>>>> mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem
>>>> (32650280 kB)
>>>> mpt2sas_cm0: _base_wait_for_doorbell_not_used: failed due to timeout
>>>> count(5000), doorbell_reg(18000000)!
>>>> mpt2sas_cm0: CurrentHostPageSize is 0: Setting default host page size
>>>> to
>>>> 4k
>>>> mpt2sas_cm0: MSI-X vectors supported: 1
>>>>           no of cores: 64, max_msix_vectors: -1
>>>> mpt2sas_cm0:  0 1 1
>>>> mpt2sas_cm0: pci_alloc_irq_vectors failed (r=-22) !!!
>>>> mpt2sas_cm0: High IOPs queues : disabled
>>>> mpt2sas0: IO-APIC enabled: IRQ 4
>>>> mpt2sas_cm0: iomem(0x0000084100000000), mapped(0x(____ptrval____)),
>>>> size(16384)
>>>> mpt2sas_cm0: ioport(0x0000085100000000), size(256)
>>>> mpt2sas_cm0: doorbell is in use (line=6869)
>>>> mpt2sas_cm0: _base_get_ioc_facts: handshake failed (r=-14)
>>>> mpt2sas_cm0: failure at
>>>> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
>>>> mpt2sas_cm1: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem
>>>> (32650280 kB)
>>>> mpt2sas_cm1: _base_wait_for_doorbell_not_used: failed due to timeout
>>>> count(5000), doorbell_reg(18000000)!
>>>> mpt2sas_cm1: CurrentHostPageSize is 0: Setting default host page size
>>>> to
>>>> 4k
>>>> mpt2sas_cm1: MSI-X vectors supported: 1
>>>>           no of cores: 64, max_msix_vectors: -1
>>>> mpt2sas_cm1:  0 1 1
>>>> mpt2sas_cm1: pci_alloc_irq_vectors failed (r=-22) !!!
>>>> mpt2sas_cm1: High IOPs queues : disabled
>>>> mpt2sas1: IO-APIC enabled: IRQ 5
>>>> mpt2sas_cm1: iomem(0x0000084120000000), mapped(0x(____ptrval____)),
>>>> size(16384)
>>>> mpt2sas_cm1: ioport(0x0000085100002000), size(256)
>>>> mpt2sas_cm1: doorbell is in use (line=6869)
>>>> mpt2sas_cm1: _base_get_ioc_facts: handshake failed (r=-14)
>>>> mpt2sas_cm1: failure at
>>>> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
>>>>
>>>> -------- Original Message --------
>>>> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
>>>> Date: 2022-03-09 01:35
>>>>  From: Sreekanth Reddy <sreekanth.reddy@broadcom.com>
>>>> To: "Martin K. Petersen" <martin.petersen@oracle.com>
>>>>
>>>> On Wed, Mar 9, 2022 at 9:26 AM Martin K. Petersen
>>>> <martin.petersen@oracle.com> wrote:
>>>>>
>>>>>
>>>>> Sreekanth,
>>>>>
>>>>>> This series fix (remove) all sparse warnings generated when
>>>>>> compiling
>>>>>> the mpt3sas driver. All warnings are related to __iomem access and
>>>>>> endianness.
>>>>>
>>>>> Please review this series and validate the patch 5 modification.
>>>>
>>>> Martin,
>>>> This patch set looks good, but before acknowledging this patch set I
>>>> just wanted to do some basic testing on a big endian machine.
>>>> Currently I don't have a big endian machine, internally I am checking
>>>> to get access to big endian machines. Meanwhile if anyone does a 
>>>> basic
>>>> testing on any big endian machine then please let me know. I will add
>>>> the acknowledgement signature.
>>>>
>>>> Thanks,
>>>> Sreekanth
>>>>
>>>>>
>>>>> Thanks!
>>>>>
>>>>> --
>>>>> Martin K. Petersen      Oracle Linux Engineering
>>>
>>
>
Damien Le Moal Sept. 16, 2022, 1:04 p.m. UTC | #15 
On 2022/09/15 16:55, matoro wrote:
> Hi Damien, apologies for continuing to bother.  Did you get a chance to 
> put together a quick fix for this to test?

I posted a revert for the 2 patches. I did try to fix this without reintroducing
the sparse warnings, but I do not have any specification for the Broadcom HBA
controller to know the exact endianness of the controller registers. So for now,
instead of blindly trying fixing this, I reverted.

> 
> -------- Original Message --------
> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
> Date: 2022-08-22 13:53
>  From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
> To: matoro <matoro_mailinglist_kernel@matoro.tk>
> 
> On 2022/08/22 10:51, matoro wrote:
>> Hi Damien, were you able to put together a fix to test?  We're up to
>> 5.19.3 now and 5.18 was just marked EOL, so I want to make sure this
>> doesn't drop off the radar.
> 
> No, sorry, I have been busy as I am traveling. Flying back home this 
> week, so
> likely will have more time next week.
> 
>>
>> -------- Original Message --------
>> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
>> Date: 2022-08-11 15:17
>>  From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
>> To: matoro <matoro_mailinglist_kernel@matoro.tk>
>>
>> On 2022/08/11 12:05, matoro wrote:
>>> Just a small update, the module stuff turned out to be a separate,
>>> unrelated regression.  I bisected that one also (applying these 
>>> reverts
>>> each time to allow me to boot) and reported it to Masahiro, who put in
>>> a
>>> fix for it here:
>>> https://lore.kernel.org/all/20220809141117.641543-1-masahiroy@kernel.org/
>>> .  So you can ignore that stuff.  Are these two commits still planned
>>> to
>>> be reverted?
>>
>> Revert should be a last resort (I really want to get rid of all these
>> sparse
>> warnings !). Let me first try to generate a fix for you to test.
>>
>>>
>>> -------- Original Message --------
>>> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
>>> Date: 2022-08-02 19:36
>>>  From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
>>> To: matoro <matoro_mailinglist_kernel@matoro.tk>, Sreekanth Reddy
>>> <sreekanth.reddy@broadcom.com>
>>>
>>> On 8/3/22 05:27, matoro wrote:
>>>> Hi folks, sorry for the lateness, unfortunately this is in fact 
>>>> broken
>>>> on BE.  I use mpt3sas on sparc and my drives fail to come up on 5.19,
>>>> bisected to this patchset.  Reverting both of the endian-related
>>>> commits, b4efbec4c2a75b619fae4e8768be379e88c78687 and
>>>> 7ab4d2441b952977556672c2fe3f4c2a698cbb37, allows it to boot.  
>>>> However,
>>>> after booting, I can't load any modules - everything errors with
>>>> "disagrees about version of symbol module_layout".  I have completely
>>>> wiped out kernel sources, the module tree, and the kernel image,
>>>> rebuilding both from scratch with ONLY the revert patch applied, but 
>>>> I
>>>> still can't load any modules.  Presumably it would work with
>>>> CONFIG_MODVERSIONS=n, but these CRC checks are there for a reason and
>>>> I
>>>> can't tell if it has something to do with the revert or not.
>>>
>>> For b4efbec4c2a75b619fae4e8768be379e88c78687, removing the
>>> cpu_to_le32()
>>> call results in the bytes actually being reversed by writel()/readl()
>>> for
>>> your BE machine. So it looks like the values that need to be written 
>>> to
>>> the HBA have to be in CPU endian, not le32. Should be easy to fix.
>>> And for 7ab4d2441b952977556672c2fe3f4c2a698cbb37, this looks like the
>>> same
>>> problem.
>>>
>>> I will be traveling and busy this week, but I can have a look at a fix
>>> next Monday. If the Broadcom folks can send a fix faster than that,
>>> that
>>> is of course welcome :)
>>>
>>>>
>>>> $ modprobe --dump-modversions
>>>> /lib/modules/5.19.0-gentoo-sparc64/kernel/fs/openpromfs/openpromfs.ko
>>>> |
>>>> grep "module_layout"
>>>> 0xa6c23707      module_layout
>>>> $ grep "module_layout" /usr/src/linux/Module.symvers
>>>> 0xa6c23707      module_layout   vmlinux EXPORT_SYMBOL
>>>>
>>>> If you need real hardware access and do not have any on hand, please
>>>> reach out and I can provide temporary access!
>>>>
>>>> Here is the full error on vanilla 5.19:
>>>>
>>>> mpt3sas version 42.100.00.00 loaded
>>>> mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem
>>>> (32650280 kB)
>>>> mpt2sas_cm0: _base_wait_for_doorbell_not_used: failed due to timeout
>>>> count(5000), doorbell_reg(18000000)!
>>>> mpt2sas_cm0: CurrentHostPageSize is 0: Setting default host page size
>>>> to
>>>> 4k
>>>> mpt2sas_cm0: MSI-X vectors supported: 1
>>>>           no of cores: 64, max_msix_vectors: -1
>>>> mpt2sas_cm0:  0 1 1
>>>> mpt2sas_cm0: pci_alloc_irq_vectors failed (r=-22) !!!
>>>> mpt2sas_cm0: High IOPs queues : disabled
>>>> mpt2sas0: IO-APIC enabled: IRQ 4
>>>> mpt2sas_cm0: iomem(0x0000084100000000), mapped(0x(____ptrval____)),
>>>> size(16384)
>>>> mpt2sas_cm0: ioport(0x0000085100000000), size(256)
>>>> mpt2sas_cm0: doorbell is in use (line=6869)
>>>> mpt2sas_cm0: _base_get_ioc_facts: handshake failed (r=-14)
>>>> mpt2sas_cm0: failure at
>>>> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
>>>> mpt2sas_cm1: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem
>>>> (32650280 kB)
>>>> mpt2sas_cm1: _base_wait_for_doorbell_not_used: failed due to timeout
>>>> count(5000), doorbell_reg(18000000)!
>>>> mpt2sas_cm1: CurrentHostPageSize is 0: Setting default host page size
>>>> to
>>>> 4k
>>>> mpt2sas_cm1: MSI-X vectors supported: 1
>>>>           no of cores: 64, max_msix_vectors: -1
>>>> mpt2sas_cm1:  0 1 1
>>>> mpt2sas_cm1: pci_alloc_irq_vectors failed (r=-22) !!!
>>>> mpt2sas_cm1: High IOPs queues : disabled
>>>> mpt2sas1: IO-APIC enabled: IRQ 5
>>>> mpt2sas_cm1: iomem(0x0000084120000000), mapped(0x(____ptrval____)),
>>>> size(16384)
>>>> mpt2sas_cm1: ioport(0x0000085100002000), size(256)
>>>> mpt2sas_cm1: doorbell is in use (line=6869)
>>>> mpt2sas_cm1: _base_get_ioc_facts: handshake failed (r=-14)
>>>> mpt2sas_cm1: failure at
>>>> drivers/scsi/mpt3sas/mpt3sas_scsih.c:12336/_scsih_probe()!
>>>>
>>>> -------- Original Message --------
>>>> Subject: Re: [PATCH v3 0/5] Fix mpt3sas driver sparse warnings
>>>> Date: 2022-03-09 01:35
>>>>  From: Sreekanth Reddy <sreekanth.reddy@broadcom.com>
>>>> To: "Martin K. Petersen" <martin.petersen@oracle.com>
>>>>
>>>> On Wed, Mar 9, 2022 at 9:26 AM Martin K. Petersen
>>>> <martin.petersen@oracle.com> wrote:
>>>>>
>>>>>
>>>>> Sreekanth,
>>>>>
>>>>>> This series fix (remove) all sparse warnings generated when
>>>>>> compiling
>>>>>> the mpt3sas driver. All warnings are related to __iomem access and
>>>>>> endianness.
>>>>>
>>>>> Please review this series and validate the patch 5 modification.
>>>>
>>>> Martin,
>>>> This patch set looks good, but before acknowledging this patch set I
>>>> just wanted to do some basic testing on a big endian machine.
>>>> Currently I don't have a big endian machine, internally I am checking
>>>> to get access to big endian machines. Meanwhile if anyone does a 
>>>> basic
>>>> testing on any big endian machine then please let me know. I will add
>>>> the acknowledgement signature.
>>>>
>>>> Thanks,
>>>> Sreekanth
>>>>
>>>>>
>>>>> Thanks!
>>>>>
>>>>> --
>>>>> Martin K. Petersen      Oracle Linux Engineering
>>>
>>
>