mbox series

[v6,0/9] Tegra234 Memory interconnect support

Message ID 20230411110002.19824-1-sumitg@nvidia.com
Headers show
Series Tegra234 Memory interconnect support | expand

Message

Sumit Gupta April 11, 2023, 10:59 a.m. UTC
Hi All,

Thank you for the review. Have incorporated the changes suggested in
v5. Requesting for ACK on the remaining patches and please consider
for merging.

The patches are divided into below groups:

- Patch [7-8]: Memory Interconnect support in PCI (MC client)
  -- Need ACK.

- Patch [3-6, 9]: Memory Interconnect support in CPUFREQ (MC client)
  -- 'Patch 3, 4 & 5': have ACK from Krzysztof.
  -- 'Patch 6 & 9': Need ACK.

- Patch [1-2]: Memory Interconnect base support
  -- 'Patch 2': has ACK from Krzysztof.
  -- 'Patch 1': Need ACK.

Both the MC client patches are dependent on the 'Patch [1-2]'.

Thank you,
Sumit Gupta
============

This patch series adds memory interconnect support for Tegra234 SoC.
It is used to dynamically scale DRAM Frequency as per the bandwidth
requests from different Memory Controller (MC) clients.
MC Clients use ICC Framework's icc_set_bw() api to dynamically request
for the DRAM bandwidth (BW). As per path, the request will be routed
from MC to the EMC driver. MC driver passes the request info like the
Client ID, type, and frequency request info to the BPMP-FW which will
set the final DRAM freq considering all exisiting requests.

MC and EMC are the ICC providers. Nodes in path for a request will be:
     Client[1-n] -> MC -> EMC -> EMEM/DRAM

The patch series also adds interconnect support in below clients:
1) CPUFREQ driver for scaling bandwidth with CPU frequency. For that,
   add per cluster OPP table which will be used in the CPUFREQ driver
   by requesting the minimum BW respective to the given CPU frequency
   in the OPP table of it's cluster.
2) PCIE driver to request BW required for different modes.

---
v5[5] -> v6:
- split PCI patch into two 'patch 7 & 8' as suggested by 'Lorenzo'.
- added more info into the commit description of PCI patches.

v4[4] -> v5:
- dropped 'patch 1 & 2' from v4 which added "nvidia,bpmp" in MC node.
- save BPMP ref from EMC node into 'mc->bpmp' and use it in MC driver.
- added check for array bounds violation in pci suggested by Bjorn.
- moved DT patch having OPP table to the last 'Patch 8'.
- did multiple small changes and cleanup suggested by Krzysztof in v4.

v3[3] -> v4:
- dropped 'patch 1' from v3 which returns 'struct tegra_bpmp *'.
- added 'patch 1 & 2' to get bpmp ref using 'nvidia,bpmp' property.
- dropped 'patch 10 & 11' from v3 and added those changes in 'patch 3'.
- added static to prototype of 'tegra_cpufreq_init_cpufreq_table()' to
  fix the warning reported by 'kernel test robot'.

v2[2] -> v3:
- in 'patch 7', set 'icc_dram_bw_scaling' to false if set_opp call failed
  to avoid flooding of uart with 'Failed to set bw' messages.
- added 'patch 10' to handle if the bpmp-fw is old and not support bwmgr mrq.
- added 'patch 11' to fix interconnect registration in tegra186-emc.
  ref patch link in linux next:
  [https://lore.kernel.org/all/20230306075651.2449-21-johan+linaro@kernel.org/]

v1[1] -> v2:
- moved BW setting to tegra234_mc_icc_set() from EMC driver.
- moved sw clients to the 'tegra_mc_clients' table.
- point 'node->data' to the entry within 'tegra_mc_clients'.
- removed 'struct tegra_icc_node' and get client info using 'node->data'.
- changed error handling in and around tegra_emc_interconnect_init().
- moved 'tegra-icc.h' from 'include/soc/tegra' to 'include/linux'.
- added interconnect support to PCIE driver in 'Patch 9'.
- merged 'Patch 9 & 10' from [1] to get num_channels and use.
- merged 'Patch 2 & 3' from [1] to add ISO and NISO clients.
- added 'Acked-by' of Krzysztof from 'Patch 05/10' of [1].
- Removed 'Patch 7' from [1] as that is merged now.

Sumit Gupta (9):
  memory: tegra: add interconnect support for DRAM scaling in Tegra234
  memory: tegra: add mc clients for Tegra234
  memory: tegra: add software mc clients in Tegra234
  dt-bindings: tegra: add icc ids for dummy MC clients
  memory: tegra: make cpu cluster bw request a multiple of mc channels
  cpufreq: tegra194: add OPP support and set bandwidth
  PCI: tegra194: Fix possible array out of bounds access
  PCI: tegra194: Add interconnect support in Tegra234
  arm64: tegra: Add cpu OPP tables and interconnects property

 arch/arm64/boot/dts/nvidia/tegra234.dtsi   | 276 ++++++++++
 drivers/cpufreq/tegra194-cpufreq.c         | 156 +++++-
 drivers/memory/tegra/mc.c                  |  24 +
 drivers/memory/tegra/mc.h                  |   1 +
 drivers/memory/tegra/tegra186-emc.c        | 130 +++++
 drivers/memory/tegra/tegra234.c            | 595 ++++++++++++++++++++-
 drivers/pci/controller/dwc/pcie-tegra194.c |  44 +-
 include/dt-bindings/memory/tegra234-mc.h   |   5 +
 include/linux/tegra-icc.h                  |  65 +++
 include/soc/tegra/mc.h                     |   8 +
 10 files changed, 1282 insertions(+), 22 deletions(-)
 create mode 100644 include/linux/tegra-icc.h

[1] https://lore.kernel.org/lkml/20221220160240.27494-1-sumitg@nvidia.com/
[2] https://lore.kernel.org/lkml/20230220140559.28289-1-sumitg@nvidia.com/
[3] https://lore.kernel.org/lkml/20230320182441.11904-1-sumitg@nvidia.com/
[4] https://lore.kernel.org/lkml/20230327161426.32639-1-sumitg@nvidia.com/
[5] https://lore.kernel.org/lkml/20230330133354.714-1-sumitg@nvidia.com/

Comments

Lorenzo Pieralisi April 21, 2023, 1:12 p.m. UTC | #1
On Tue, Apr 11, 2023 at 04:30:00PM +0530, Sumit Gupta wrote:
> Add check to fix the possible array out of bounds violation by
> making speed equal to GEN1_CORE_CLK_FREQ when its value is more
> than the size of "pcie_gen_freq" array. This array has size of
> four but possible speed (CLS) values are from "0 to 0xF". So,
> "speed - 1" values are "-1 to 0xE". This change was suggested by
> "Bjorn Helgaas" in the below link.

There is a Suggested-by tag and a Link: tag remove the last
sentence, that's duplicate information.

> Suggested-by: Bjorn Helgaas <helgaas@kernel.org>
> Signed-off-by: Sumit Gupta <sumitg@nvidia.com>
> Link: https://lore.kernel.org/lkml/72b9168b-d4d6-4312-32ea-69358df2f2d0@nvidia.com/
> ---
>  drivers/pci/controller/dwc/pcie-tegra194.c | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)

Acked-by: Lorenzo Pieralisi <lpieralisi@kernel.org>

> diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
> index 09825b4a075e..e6eec85480ca 100644
> --- a/drivers/pci/controller/dwc/pcie-tegra194.c
> +++ b/drivers/pci/controller/dwc/pcie-tegra194.c
> @@ -223,6 +223,7 @@
>  #define EP_STATE_ENABLED	1
>  
>  static const unsigned int pcie_gen_freq[] = {
> +	GEN1_CORE_CLK_FREQ,	/* PCI_EXP_LNKSTA_CLS == 0; undefined */
>  	GEN1_CORE_CLK_FREQ,
>  	GEN2_CORE_CLK_FREQ,
>  	GEN3_CORE_CLK_FREQ,
> @@ -459,7 +460,11 @@ static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
>  
>  	speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) &
>  		PCI_EXP_LNKSTA_CLS;
> -	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]);
> +
> +	if (speed >= ARRAY_SIZE(pcie_gen_freq))
> +		speed = 0;
> +
> +	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed]);
>  
>  	if (pcie->of_data->has_ltr_req_fix)
>  		return IRQ_HANDLED;
> @@ -1020,7 +1025,11 @@ static int tegra_pcie_dw_start_link(struct dw_pcie *pci)
>  
>  	speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) &
>  		PCI_EXP_LNKSTA_CLS;
> -	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]);
> +
> +	if (speed >= ARRAY_SIZE(pcie_gen_freq))
> +		speed = 0;
> +
> +	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed]);
>  
>  	tegra_pcie_enable_interrupts(pp);
>  
> -- 
> 2.17.1
>
Sumit Gupta April 24, 2023, 1:03 p.m. UTC | #2
On 21/04/23 18:42, Lorenzo Pieralisi wrote:
> External email: Use caution opening links or attachments
> 
> 
> On Tue, Apr 11, 2023 at 04:30:00PM +0530, Sumit Gupta wrote:
>> Add check to fix the possible array out of bounds violation by
>> making speed equal to GEN1_CORE_CLK_FREQ when its value is more
>> than the size of "pcie_gen_freq" array. This array has size of
>> four but possible speed (CLS) values are from "0 to 0xF". So,
>> "speed - 1" values are "-1 to 0xE". This change was suggested by
>> "Bjorn Helgaas" in the below link.
> 
> There is a Suggested-by tag and a Link: tag remove the last
> sentence, that's duplicate information.
> 
Removed in v7.

Thank you,
Sumit Gupta

>> Suggested-by: Bjorn Helgaas <helgaas@kernel.org>
>> Signed-off-by: Sumit Gupta <sumitg@nvidia.com>
>> Link: https://lore.kernel.org/lkml/72b9168b-d4d6-4312-32ea-69358df2f2d0@nvidia.com/
>> ---
>>   drivers/pci/controller/dwc/pcie-tegra194.c | 13 +++++++++++--
>>   1 file changed, 11 insertions(+), 2 deletions(-)
> 
> Acked-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
> 
>> diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
>> index 09825b4a075e..e6eec85480ca 100644
>> --- a/drivers/pci/controller/dwc/pcie-tegra194.c
>> +++ b/drivers/pci/controller/dwc/pcie-tegra194.c
>> @@ -223,6 +223,7 @@
>>   #define EP_STATE_ENABLED     1
>>
>>   static const unsigned int pcie_gen_freq[] = {
>> +     GEN1_CORE_CLK_FREQ,     /* PCI_EXP_LNKSTA_CLS == 0; undefined */
>>        GEN1_CORE_CLK_FREQ,
>>        GEN2_CORE_CLK_FREQ,
>>        GEN3_CORE_CLK_FREQ,
>> @@ -459,7 +460,11 @@ static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
>>
>>        speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) &
>>                PCI_EXP_LNKSTA_CLS;
>> -     clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]);
>> +
>> +     if (speed >= ARRAY_SIZE(pcie_gen_freq))
>> +             speed = 0;
>> +
>> +     clk_set_rate(pcie->core_clk, pcie_gen_freq[speed]);
>>
>>        if (pcie->of_data->has_ltr_req_fix)
>>                return IRQ_HANDLED;
>> @@ -1020,7 +1025,11 @@ static int tegra_pcie_dw_start_link(struct dw_pcie *pci)
>>
>>        speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) &
>>                PCI_EXP_LNKSTA_CLS;
>> -     clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]);
>> +
>> +     if (speed >= ARRAY_SIZE(pcie_gen_freq))
>> +             speed = 0;
>> +
>> +     clk_set_rate(pcie->core_clk, pcie_gen_freq[speed]);
>>
>>        tegra_pcie_enable_interrupts(pp);
>>
>> --
>> 2.17.1
>>