From patchwork Mon Dec 11 07:19:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srujana Challa X-Patchwork-Id: 752630 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="Z9PprcFs" Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 27B76D1; Sun, 10 Dec 2023 23:19:39 -0800 (PST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BAMh88K026392; Sun, 10 Dec 2023 23:19:28 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=UReJLMLrvpJIa4FY1T75cC81B7xyCL36z25xAsooeqc=; b=Z9P prcFsS3/nZon+cRMNsgGvCcUE0xU88c3NHF5Co4wPgUhJ/fpqg0NYO6SOZlE32GJ Ys6aJrVwJOXaNjyPWby64k8f5AY5WXHSDvzr6P2FJ1AC34Ha1SF2iTP1ke36USJF QExRSd7XQ8ziKBqZp2T1ucupCVt3JGAtubJ1W3ov6EjEe120JqqPnWXwF4wx+FzX mSKdADAmOJsU4FmRWGELVddxmrAIChPnshGlPCyAEb4Z5IKJHq6S0Yjm/WGnOwlG PU5KewsFtO5Wh0/vOIR2QxgljvzkZhhoHM//AR9yVCOrtjxC/ReTMSEZr27LRqem QeZCg9SxmR7auuULFLg== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3uvrmjkmgw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 10 Dec 2023 23:19:28 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Sun, 10 Dec 2023 23:19:26 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Sun, 10 Dec 2023 23:19:26 -0800 Received: from localhost.localdomain (unknown [10.28.36.175]) by maili.marvell.com (Postfix) with ESMTP id EF6BE3F7097; Sun, 10 Dec 2023 23:19:20 -0800 (PST) From: Srujana Challa To: , , CC: , , , , , , , , , , , , , , , , Subject: [PATCH net-next v1 01/10] crypto: octeontx2: remove CPT block reset Date: Mon, 11 Dec 2023 12:49:04 +0530 Message-ID: <20231211071913.151225-2-schalla@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231211071913.151225-1-schalla@marvell.com> References: <20231211071913.151225-1-schalla@marvell.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: XQHYazVQQA_2qmfv5Iqnp_6fpL-qNh-O X-Proofpoint-ORIG-GUID: XQHYazVQQA_2qmfv5Iqnp_6fpL-qNh-O X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 CPT block reset in CPT PF erase all the CPT configuration which is done in AF driver init. So, remove CPT block reset from CPT PF as it is also being done in AF init and not required in PF. Signed-off-by: Srujana Challa --- .../marvell/octeontx2/otx2_cptpf_main.c | 43 ------------------- 1 file changed, 43 deletions(-) diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c index e34223daa327..5436b0d3685c 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c @@ -587,45 +587,6 @@ static int cpt_is_pf_usable(struct otx2_cptpf_dev *cptpf) return 0; } -static int cptx_device_reset(struct otx2_cptpf_dev *cptpf, int blkaddr) -{ - int timeout = 10, ret; - u64 reg = 0; - - ret = otx2_cpt_write_af_reg(&cptpf->afpf_mbox, cptpf->pdev, - CPT_AF_BLK_RST, 0x1, blkaddr); - if (ret) - return ret; - - do { - ret = otx2_cpt_read_af_reg(&cptpf->afpf_mbox, cptpf->pdev, - CPT_AF_BLK_RST, ®, blkaddr); - if (ret) - return ret; - - if (!((reg >> 63) & 0x1)) - break; - - usleep_range(10000, 20000); - if (timeout-- < 0) - return -EBUSY; - } while (1); - - return ret; -} - -static int cptpf_device_reset(struct otx2_cptpf_dev *cptpf) -{ - int ret = 0; - - if (cptpf->has_cpt1) { - ret = cptx_device_reset(cptpf, BLKADDR_CPT1); - if (ret) - return ret; - } - return cptx_device_reset(cptpf, BLKADDR_CPT0); -} - static void cptpf_check_block_implemented(struct otx2_cptpf_dev *cptpf) { u64 cfg; @@ -643,10 +604,6 @@ static int cptpf_device_init(struct otx2_cptpf_dev *cptpf) /* check if 'implemented' bit is set for block BLKADDR_CPT1 */ cptpf_check_block_implemented(cptpf); - /* Reset the CPT PF device */ - ret = cptpf_device_reset(cptpf); - if (ret) - return ret; /* Get number of SE, IE and AE engines */ ret = otx2_cpt_read_af_reg(&cptpf->afpf_mbox, cptpf->pdev, From patchwork Mon Dec 11 07:19:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srujana Challa X-Patchwork-Id: 753212 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="g/Lusr50" Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80E5FD7; Sun, 10 Dec 2023 23:19:46 -0800 (PST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BB0PurD027071; Sun, 10 Dec 2023 23:19:35 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=n1b3/UR66oK1akursuPaJEmu8pUakuWHY2Got9hXtqk=; b=g/L usr50Nd4WlgaNLaox++xFhKrfNUtEUoHXOMiQ1JZAAsKdrrH5+wquD++p9lLKrfp 9O/Eqcqm10jaXomELJe4aI8fpyHTKT7LDW6Rnz1p/joipEZxduVVI7Sm6QCCyM19 OfNuewaAowMtgE+g3N9ZbjOZkv+VoRCsgNwrXs6QrWUVp0oTPfhjFXTbDGAX/IIB Nmy7YSb2W1ZUkIdHCvttRhKZnzcwdAKtKCBB1oGtOYaAwkbYaG7oD/SEe6FdM7E4 pnMmtiviv6UpT7TKxrOOAM8hcWKuq+EoGXRTbcoghT5x35zFwdpkHFAXV3z5kz1Z OkiGcs5vWFvyu7LmOBw== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3uw8xqa9d7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 10 Dec 2023 23:19:34 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Sun, 10 Dec 2023 23:19:32 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Sun, 10 Dec 2023 23:19:32 -0800 Received: from localhost.localdomain (unknown [10.28.36.175]) by maili.marvell.com (Postfix) with ESMTP id 113B33F7098; Sun, 10 Dec 2023 23:19:26 -0800 (PST) From: Srujana Challa To: , , CC: , , , , , , , , , , , , , , , , Subject: [PATCH net-next v1 02/10] :crypto: octeontx2: add SGv2 support for CN10KB or CN10KA B0 Date: Mon, 11 Dec 2023 12:49:05 +0530 Message-ID: <20231211071913.151225-3-schalla@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231211071913.151225-1-schalla@marvell.com> References: <20231211071913.151225-1-schalla@marvell.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: rNTZ21hNZP-zR-iNIkYr4GG018kUWalU X-Proofpoint-ORIG-GUID: rNTZ21hNZP-zR-iNIkYr4GG018kUWalU X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 Scatter Gather input format for CPT has changed on CN10KB/CN10KA B0 HW to make it comapatible with NIX Scatter Gather format to support SG mode for inline IPsec. This patch modifies the code to make the driver works for the same. This patch also enables CPT firmware load for these chips. Signed-off-by: Srujana Challa --- drivers/crypto/marvell/octeontx2/cn10k_cpt.c | 19 +- drivers/crypto/marvell/octeontx2/cn10k_cpt.h | 1 + .../marvell/octeontx2/otx2_cpt_common.h | 41 ++- .../marvell/octeontx2/otx2_cpt_hw_types.h | 3 + .../marvell/octeontx2/otx2_cpt_reqmgr.h | 293 ++++++++++++++++++ drivers/crypto/marvell/octeontx2/otx2_cptlf.h | 2 + .../marvell/octeontx2/otx2_cptpf_main.c | 26 +- .../marvell/octeontx2/otx2_cptpf_mbox.c | 2 +- .../marvell/octeontx2/otx2_cptpf_ucode.c | 33 +- .../marvell/octeontx2/otx2_cptpf_ucode.h | 3 +- drivers/crypto/marvell/octeontx2/otx2_cptvf.h | 2 + .../marvell/octeontx2/otx2_cptvf_main.c | 13 + .../marvell/octeontx2/otx2_cptvf_mbox.c | 26 ++ .../marvell/octeontx2/otx2_cptvf_reqmgr.c | 160 +--------- 14 files changed, 447 insertions(+), 177 deletions(-) diff --git a/drivers/crypto/marvell/octeontx2/cn10k_cpt.c b/drivers/crypto/marvell/octeontx2/cn10k_cpt.c index 93d22b328991..b23ae3a020e0 100644 --- a/drivers/crypto/marvell/octeontx2/cn10k_cpt.c +++ b/drivers/crypto/marvell/octeontx2/cn10k_cpt.c @@ -14,12 +14,14 @@ static struct cpt_hw_ops otx2_hw_ops = { .send_cmd = otx2_cpt_send_cmd, .cpt_get_compcode = otx2_cpt_get_compcode, .cpt_get_uc_compcode = otx2_cpt_get_uc_compcode, + .cpt_sg_info_create = otx2_sg_info_create, }; static struct cpt_hw_ops cn10k_hw_ops = { .send_cmd = cn10k_cpt_send_cmd, .cpt_get_compcode = cn10k_cpt_get_compcode, .cpt_get_uc_compcode = cn10k_cpt_get_uc_compcode, + .cpt_sg_info_create = otx2_sg_info_create, }; static void cn10k_cpt_send_cmd(union otx2_cpt_inst_s *cptinst, u32 insts_num, @@ -78,12 +80,9 @@ int cn10k_cptvf_lmtst_init(struct otx2_cptvf_dev *cptvf) struct pci_dev *pdev = cptvf->pdev; resource_size_t offset, size; - if (!test_bit(CN10K_LMTST, &cptvf->cap_flag)) { - cptvf->lfs.ops = &otx2_hw_ops; + if (!test_bit(CN10K_LMTST, &cptvf->cap_flag)) return 0; - } - cptvf->lfs.ops = &cn10k_hw_ops; offset = pci_resource_start(pdev, PCI_MBOX_BAR_NUM); size = pci_resource_len(pdev, PCI_MBOX_BAR_NUM); /* Map VF LMILINE region */ @@ -96,3 +95,15 @@ int cn10k_cptvf_lmtst_init(struct otx2_cptvf_dev *cptvf) return 0; } EXPORT_SYMBOL_NS_GPL(cn10k_cptvf_lmtst_init, CRYPTO_DEV_OCTEONTX2_CPT); + +int cptvf_hw_ops_get(struct otx2_cptvf_dev *cptvf) +{ + if (!test_bit(CN10K_LMTST, &cptvf->cap_flag)) { + cptvf->lfs.ops = &otx2_hw_ops; + return 0; + } + cptvf->lfs.ops = &cn10k_hw_ops; + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cptvf_hw_ops_get, CRYPTO_DEV_OCTEONTX2_CPT); diff --git a/drivers/crypto/marvell/octeontx2/cn10k_cpt.h b/drivers/crypto/marvell/octeontx2/cn10k_cpt.h index aaefc7e38e06..0f714ee564f5 100644 --- a/drivers/crypto/marvell/octeontx2/cn10k_cpt.h +++ b/drivers/crypto/marvell/octeontx2/cn10k_cpt.h @@ -30,5 +30,6 @@ static inline u8 otx2_cpt_get_uc_compcode(union otx2_cpt_res_s *result) int cn10k_cptpf_lmtst_init(struct otx2_cptpf_dev *cptpf); int cn10k_cptvf_lmtst_init(struct otx2_cptvf_dev *cptvf); +int cptvf_hw_ops_get(struct otx2_cptvf_dev *cptvf); #endif /* __CN10K_CPTLF_H */ diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h index 46b778bbbee4..2a7544252444 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h @@ -102,7 +102,10 @@ union otx2_cpt_eng_caps { u64 kasumi:1; u64 des:1; u64 crc:1; - u64 reserved_14_63:50; + u64 mmul:1; + u64 reserved_15_33:19; + u64 pdcp_chain:1; + u64 reserved_35_63:29; }; }; @@ -145,6 +148,41 @@ static inline bool is_dev_otx2(struct pci_dev *pdev) return false; } +static inline bool is_dev_cn10ka(struct pci_dev *pdev) +{ + if (pdev->subsystem_device == CPT_PCI_SUBSYS_DEVID_CN10K_A) + return true; + + return false; +} + +static inline bool is_dev_cn10ka_ax(struct pci_dev *pdev) +{ + if (pdev->subsystem_device == CPT_PCI_SUBSYS_DEVID_CN10K_A && + ((pdev->revision & 0xFF) == 4 || (pdev->revision & 0xFF) == 0x50 || + (pdev->revision & 0xff) == 0x51)) + return true; + + return false; +} + +static inline bool is_dev_cn10kb(struct pci_dev *pdev) +{ + if (pdev->subsystem_device == CPT_PCI_SUBSYS_DEVID_CN10K_B) + return true; + + return false; +} + +static inline bool is_dev_cn10ka_b0(struct pci_dev *pdev) +{ + if (pdev->subsystem_device == CPT_PCI_SUBSYS_DEVID_CN10K_A && + (pdev->revision & 0xFF) == 0x54) + return true; + + return false; +} + static inline void otx2_cpt_set_hw_caps(struct pci_dev *pdev, unsigned long *cap_flag) { @@ -154,7 +192,6 @@ static inline void otx2_cpt_set_hw_caps(struct pci_dev *pdev, } } - int otx2_cpt_send_ready_msg(struct otx2_mbox *mbox, struct pci_dev *pdev); int otx2_cpt_send_mbox_msg(struct otx2_mbox *mbox, struct pci_dev *pdev); diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_hw_types.h b/drivers/crypto/marvell/octeontx2/otx2_cpt_hw_types.h index 6f947978e4e8..756aee0c2b05 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_hw_types.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_hw_types.h @@ -13,6 +13,9 @@ #define CN10K_CPT_PCI_PF_DEVICE_ID 0xA0F2 #define CN10K_CPT_PCI_VF_DEVICE_ID 0xA0F3 +#define CPT_PCI_SUBSYS_DEVID_CN10K_A 0xB900 +#define CPT_PCI_SUBSYS_DEVID_CN10K_B 0xBD00 + /* Mailbox interrupts offset */ #define OTX2_CPT_PF_MBOX_INT 6 #define OTX2_CPT_PF_INT_VEC_E_MBOXX(x, a) ((x) + (a)) diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_reqmgr.h b/drivers/crypto/marvell/octeontx2/otx2_cpt_reqmgr.h index dbb1ee746f4c..b0f426531a42 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_reqmgr.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_reqmgr.h @@ -143,6 +143,8 @@ struct otx2_cpt_inst_info { unsigned long time_in; u32 dlen; u32 dma_len; + u64 gthr_sz; + u64 sctr_sz; u8 extra_time; }; @@ -157,6 +159,16 @@ struct otx2_cpt_sglist_component { __be64 ptr3; }; +struct cn10kb_cpt_sglist_component { + u16 len0; + u16 len1; + u16 len2; + u16 valid_segs; + u64 ptr0; + u64 ptr1; + u64 ptr2; +}; + static inline void otx2_cpt_info_destroy(struct pci_dev *pdev, struct otx2_cpt_inst_info *info) { @@ -188,6 +200,287 @@ static inline void otx2_cpt_info_destroy(struct pci_dev *pdev, kfree(info); } +static inline int setup_sgio_components(struct pci_dev *pdev, + struct otx2_cpt_buf_ptr *list, + int buf_count, u8 *buffer) +{ + struct otx2_cpt_sglist_component *sg_ptr = NULL; + int ret = 0, i, j; + int components; + + if (unlikely(!list)) { + dev_err(&pdev->dev, "Input list pointer is NULL\n"); + return -EFAULT; + } + + for (i = 0; i < buf_count; i++) { + if (unlikely(!list[i].vptr)) + continue; + list[i].dma_addr = dma_map_single(&pdev->dev, list[i].vptr, + list[i].size, + DMA_BIDIRECTIONAL); + if (unlikely(dma_mapping_error(&pdev->dev, list[i].dma_addr))) { + dev_err(&pdev->dev, "Dma mapping failed\n"); + ret = -EIO; + goto sg_cleanup; + } + } + components = buf_count / 4; + sg_ptr = (struct otx2_cpt_sglist_component *)buffer; + for (i = 0; i < components; i++) { + sg_ptr->len0 = cpu_to_be16(list[i * 4 + 0].size); + sg_ptr->len1 = cpu_to_be16(list[i * 4 + 1].size); + sg_ptr->len2 = cpu_to_be16(list[i * 4 + 2].size); + sg_ptr->len3 = cpu_to_be16(list[i * 4 + 3].size); + sg_ptr->ptr0 = cpu_to_be64(list[i * 4 + 0].dma_addr); + sg_ptr->ptr1 = cpu_to_be64(list[i * 4 + 1].dma_addr); + sg_ptr->ptr2 = cpu_to_be64(list[i * 4 + 2].dma_addr); + sg_ptr->ptr3 = cpu_to_be64(list[i * 4 + 3].dma_addr); + sg_ptr++; + } + components = buf_count % 4; + + switch (components) { + case 3: + sg_ptr->len2 = cpu_to_be16(list[i * 4 + 2].size); + sg_ptr->ptr2 = cpu_to_be64(list[i * 4 + 2].dma_addr); + fallthrough; + case 2: + sg_ptr->len1 = cpu_to_be16(list[i * 4 + 1].size); + sg_ptr->ptr1 = cpu_to_be64(list[i * 4 + 1].dma_addr); + fallthrough; + case 1: + sg_ptr->len0 = cpu_to_be16(list[i * 4 + 0].size); + sg_ptr->ptr0 = cpu_to_be64(list[i * 4 + 0].dma_addr); + break; + default: + break; + } + return ret; + +sg_cleanup: + for (j = 0; j < i; j++) { + if (list[j].dma_addr) { + dma_unmap_single(&pdev->dev, list[j].dma_addr, + list[j].size, DMA_BIDIRECTIONAL); + } + + list[j].dma_addr = 0; + } + return ret; +} + +static inline int sgv2io_components_setup(struct pci_dev *pdev, + struct otx2_cpt_buf_ptr *list, + int buf_count, u8 *buffer) +{ + struct cn10kb_cpt_sglist_component *sg_ptr = NULL; + int ret = 0, i, j; + int components; + + if (unlikely(!list)) { + dev_err(&pdev->dev, "Input list pointer is NULL\n"); + return -EFAULT; + } + + for (i = 0; i < buf_count; i++) { + if (unlikely(!list[i].vptr)) + continue; + list[i].dma_addr = dma_map_single(&pdev->dev, list[i].vptr, + list[i].size, + DMA_BIDIRECTIONAL); + if (unlikely(dma_mapping_error(&pdev->dev, list[i].dma_addr))) { + dev_err(&pdev->dev, "Dma mapping failed\n"); + ret = -EIO; + goto sg_cleanup; + } + } + components = buf_count / 3; + sg_ptr = (struct cn10kb_cpt_sglist_component *)buffer; + for (i = 0; i < components; i++) { + sg_ptr->len0 = list[i * 3 + 0].size; + sg_ptr->len1 = list[i * 3 + 1].size; + sg_ptr->len2 = list[i * 3 + 2].size; + sg_ptr->ptr0 = list[i * 3 + 0].dma_addr; + sg_ptr->ptr1 = list[i * 3 + 1].dma_addr; + sg_ptr->ptr2 = list[i * 3 + 2].dma_addr; + sg_ptr->valid_segs = 3; + sg_ptr++; + } + components = buf_count % 3; + + sg_ptr->valid_segs = components; + switch (components) { + case 2: + sg_ptr->len1 = list[i * 3 + 1].size; + sg_ptr->ptr1 = list[i * 3 + 1].dma_addr; + fallthrough; + case 1: + sg_ptr->len0 = list[i * 3 + 0].size; + sg_ptr->ptr0 = list[i * 3 + 0].dma_addr; + break; + default: + break; + } + return ret; + +sg_cleanup: + for (j = 0; j < i; j++) { + if (list[j].dma_addr) { + dma_unmap_single(&pdev->dev, list[j].dma_addr, + list[j].size, DMA_BIDIRECTIONAL); + } + + list[j].dma_addr = 0; + } + return ret; +} + +static inline struct otx2_cpt_inst_info *cn10k_sgv2_info_create( + struct pci_dev *pdev, + struct otx2_cpt_req_info *req, + gfp_t gfp) +{ + u32 dlen = 0, g_len, sg_len, info_len; + int align = OTX2_CPT_DMA_MINALIGN; + struct otx2_cpt_inst_info *info; + u16 g_sz_bytes, s_sz_bytes; + u32 total_mem_len; + int i; + + g_sz_bytes = ((req->in_cnt + 2) / 3) * + sizeof(struct cn10kb_cpt_sglist_component); + s_sz_bytes = ((req->out_cnt + 2) / 3) * + sizeof(struct cn10kb_cpt_sglist_component); + + g_len = ALIGN(g_sz_bytes, align); + sg_len = ALIGN(g_len + s_sz_bytes, align); + info_len = ALIGN(sizeof(*info), align); + total_mem_len = sg_len + info_len + sizeof(union otx2_cpt_res_s); + + info = kzalloc(total_mem_len, gfp); + if (unlikely(!info)) + return NULL; + + for (i = 0; i < req->in_cnt; i++) + dlen += req->in[i].size; + + info->dlen = dlen; + info->in_buffer = (u8 *)info + info_len; + info->gthr_sz = req->in_cnt; + info->sctr_sz = req->out_cnt; + + /* Setup gather (input) components */ + if (sgv2io_components_setup(pdev, req->in, req->in_cnt, + info->in_buffer)) { + dev_err(&pdev->dev, "Failed to setup gather list\n"); + goto destroy_info; + } + + if (sgv2io_components_setup(pdev, req->out, req->out_cnt, + &info->in_buffer[g_len])) { + dev_err(&pdev->dev, "Failed to setup scatter list\n"); + goto destroy_info; + } + + info->dma_len = total_mem_len - info_len; + info->dptr_baddr = dma_map_single(&pdev->dev, info->in_buffer, + info->dma_len, DMA_BIDIRECTIONAL); + if (unlikely(dma_mapping_error(&pdev->dev, info->dptr_baddr))) { + dev_err(&pdev->dev, "DMA Mapping failed for cpt req\n"); + goto destroy_info; + } + info->rptr_baddr = info->dptr_baddr + g_len; + /* + * Get buffer for union otx2_cpt_res_s response + * structure and its physical address + */ + info->completion_addr = info->in_buffer + sg_len; + info->comp_baddr = info->dptr_baddr + sg_len; + + return info; + +destroy_info: + otx2_cpt_info_destroy(pdev, info); + return NULL; +} + +/* SG list header size in bytes */ +#define SG_LIST_HDR_SIZE 8 +static inline struct otx2_cpt_inst_info *otx2_sg_info_create( + struct pci_dev *pdev, + struct otx2_cpt_req_info *req, + gfp_t gfp) +{ + int align = OTX2_CPT_DMA_MINALIGN; + struct otx2_cpt_inst_info *info; + u32 dlen, align_dlen, info_len; + u16 g_sz_bytes, s_sz_bytes; + u32 total_mem_len; + + if (unlikely(req->in_cnt > OTX2_CPT_MAX_SG_IN_CNT || + req->out_cnt > OTX2_CPT_MAX_SG_OUT_CNT)) { + dev_err(&pdev->dev, "Error too many sg components\n"); + return NULL; + } + + g_sz_bytes = ((req->in_cnt + 3) / 4) * + sizeof(struct otx2_cpt_sglist_component); + s_sz_bytes = ((req->out_cnt + 3) / 4) * + sizeof(struct otx2_cpt_sglist_component); + + dlen = g_sz_bytes + s_sz_bytes + SG_LIST_HDR_SIZE; + align_dlen = ALIGN(dlen, align); + info_len = ALIGN(sizeof(*info), align); + total_mem_len = align_dlen + info_len + sizeof(union otx2_cpt_res_s); + + info = kzalloc(total_mem_len, gfp); + if (unlikely(!info)) + return NULL; + + info->dlen = dlen; + info->in_buffer = (u8 *)info + info_len; + + ((u16 *)info->in_buffer)[0] = req->out_cnt; + ((u16 *)info->in_buffer)[1] = req->in_cnt; + ((u16 *)info->in_buffer)[2] = 0; + ((u16 *)info->in_buffer)[3] = 0; + cpu_to_be64s((u64 *)info->in_buffer); + + /* Setup gather (input) components */ + if (setup_sgio_components(pdev, req->in, req->in_cnt, + &info->in_buffer[8])) { + dev_err(&pdev->dev, "Failed to setup gather list\n"); + goto destroy_info; + } + + if (setup_sgio_components(pdev, req->out, req->out_cnt, + &info->in_buffer[8 + g_sz_bytes])) { + dev_err(&pdev->dev, "Failed to setup scatter list\n"); + goto destroy_info; + } + + info->dma_len = total_mem_len - info_len; + info->dptr_baddr = dma_map_single(&pdev->dev, info->in_buffer, + info->dma_len, DMA_BIDIRECTIONAL); + if (unlikely(dma_mapping_error(&pdev->dev, info->dptr_baddr))) { + dev_err(&pdev->dev, "DMA Mapping failed for cpt req\n"); + goto destroy_info; + } + /* + * Get buffer for union otx2_cpt_res_s response + * structure and its physical address + */ + info->completion_addr = info->in_buffer + align_dlen; + info->comp_baddr = info->dptr_baddr + align_dlen; + + return info; + +destroy_info: + otx2_cpt_info_destroy(pdev, info); + return NULL; +} + struct otx2_cptlf_wqe; int otx2_cpt_do_request(struct pci_dev *pdev, struct otx2_cpt_req_info *req, int cpu_num); diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptlf.h b/drivers/crypto/marvell/octeontx2/otx2_cptlf.h index 5302fe3d0e6f..70138d65ba9e 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptlf.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cptlf.h @@ -99,6 +99,8 @@ struct cpt_hw_ops { struct otx2_cptlf_info *lf); u8 (*cpt_get_compcode)(union otx2_cpt_res_s *result); u8 (*cpt_get_uc_compcode)(union otx2_cpt_res_s *result); + struct otx2_cpt_inst_info * (*cpt_sg_info_create)(struct pci_dev *pdev, + struct otx2_cpt_req_info *req, gfp_t gfp); }; struct otx2_cptlfs_info { diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c index 5436b0d3685c..c64c50a964ed 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c @@ -14,6 +14,8 @@ #define OTX2_CPT_DRV_STRING "Marvell RVU CPT Physical Function Driver" #define CPT_UC_RID_CN9K_B0 1 +#define CPT_UC_RID_CN10K_A 4 +#define CPT_UC_RID_CN10K_B 5 static void cptpf_enable_vfpf_mbox_intr(struct otx2_cptpf_dev *cptpf, int num_vfs) @@ -587,6 +589,26 @@ static int cpt_is_pf_usable(struct otx2_cptpf_dev *cptpf) return 0; } +static int cptpf_get_rid(struct pci_dev *pdev, struct otx2_cptpf_dev *cptpf) +{ + struct otx2_cpt_eng_grps *eng_grps = &cptpf->eng_grps; + u64 reg_val = 0x0; + + if (is_dev_otx2(pdev)) { + eng_grps->rid = pdev->revision; + return 0; + } + otx2_cpt_read_af_reg(&cptpf->afpf_mbox, pdev, CPT_AF_CTL, ®_val, + BLKADDR_CPT0); + if ((is_dev_cn10ka_b0(pdev) && (reg_val & BIT_ULL(18))) || + is_dev_cn10ka_ax(pdev)) + eng_grps->rid = CPT_UC_RID_CN10K_A; + else if (is_dev_cn10kb(pdev) || is_dev_cn10ka_b0(pdev)) + eng_grps->rid = CPT_UC_RID_CN10K_B; + + return 0; +} + static void cptpf_check_block_implemented(struct otx2_cptpf_dev *cptpf) { u64 cfg; @@ -657,7 +679,9 @@ static int cptpf_sriov_enable(struct pci_dev *pdev, int num_vfs) ret = cptpf_register_vfpf_intr(cptpf, num_vfs); if (ret) goto destroy_flr; - + ret = cptpf_get_rid(pdev, cptpf); + if (ret) + goto disable_intr; /* Get CPT HW capabilities using LOAD_FVC operation. */ ret = otx2_cpt_discover_eng_capabilities(cptpf); if (ret) diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c index 480b3720f15a..390ed146d309 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c @@ -78,7 +78,7 @@ static int handle_msg_get_caps(struct otx2_cptpf_dev *cptpf, rsp->hdr.sig = OTX2_MBOX_RSP_SIG; rsp->hdr.pcifunc = req->pcifunc; rsp->cpt_pf_drv_version = OTX2_CPT_PF_DRV_VERSION; - rsp->cpt_revision = cptpf->pdev->revision; + rsp->cpt_revision = cptpf->eng_grps.rid; memcpy(&rsp->eng_caps, &cptpf->eng_caps, sizeof(rsp->eng_caps)); return 0; diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c index 1958b797a421..7fccc348f66e 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c @@ -117,12 +117,10 @@ static char *get_ucode_type_str(int ucode_type) static int get_ucode_type(struct device *dev, struct otx2_cpt_ucode_hdr *ucode_hdr, - int *ucode_type) + int *ucode_type, u16 rid) { - struct otx2_cptpf_dev *cptpf = dev_get_drvdata(dev); char ver_str_prefix[OTX2_CPT_UCODE_VER_STR_SZ]; char tmp_ver_str[OTX2_CPT_UCODE_VER_STR_SZ]; - struct pci_dev *pdev = cptpf->pdev; int i, val = 0; u8 nn; @@ -130,7 +128,7 @@ static int get_ucode_type(struct device *dev, for (i = 0; i < strlen(tmp_ver_str); i++) tmp_ver_str[i] = tolower(tmp_ver_str[i]); - sprintf(ver_str_prefix, "ocpt-%02d", pdev->revision); + sprintf(ver_str_prefix, "ocpt-%02d", rid); if (!strnstr(tmp_ver_str, ver_str_prefix, OTX2_CPT_UCODE_VER_STR_SZ)) return -EINVAL; @@ -359,7 +357,7 @@ static int cpt_attach_and_enable_cores(struct otx2_cpt_eng_grp_info *eng_grp, } static int load_fw(struct device *dev, struct fw_info_t *fw_info, - char *filename) + char *filename, u16 rid) { struct otx2_cpt_ucode_hdr *ucode_hdr; struct otx2_cpt_uc_info_t *uc_info; @@ -375,7 +373,7 @@ static int load_fw(struct device *dev, struct fw_info_t *fw_info, goto free_uc_info; ucode_hdr = (struct otx2_cpt_ucode_hdr *)uc_info->fw->data; - ret = get_ucode_type(dev, ucode_hdr, &ucode_type); + ret = get_ucode_type(dev, ucode_hdr, &ucode_type, rid); if (ret) goto release_fw; @@ -389,6 +387,7 @@ static int load_fw(struct device *dev, struct fw_info_t *fw_info, set_ucode_filename(&uc_info->ucode, filename); memcpy(uc_info->ucode.ver_str, ucode_hdr->ver_str, OTX2_CPT_UCODE_VER_STR_SZ); + uc_info->ucode.ver_str[OTX2_CPT_UCODE_VER_STR_SZ] = 0; uc_info->ucode.ver_num = ucode_hdr->ver_num; uc_info->ucode.type = ucode_type; uc_info->ucode.size = ucode_size; @@ -448,7 +447,8 @@ static void print_uc_info(struct fw_info_t *fw_info) } } -static int cpt_ucode_load_fw(struct pci_dev *pdev, struct fw_info_t *fw_info) +static int cpt_ucode_load_fw(struct pci_dev *pdev, struct fw_info_t *fw_info, + u16 rid) { char filename[OTX2_CPT_NAME_LENGTH]; char eng_type[8] = {0}; @@ -462,9 +462,9 @@ static int cpt_ucode_load_fw(struct pci_dev *pdev, struct fw_info_t *fw_info) eng_type[i] = tolower(eng_type[i]); snprintf(filename, sizeof(filename), "mrvl/cpt%02d/%s.out", - pdev->revision, eng_type); + rid, eng_type); /* Request firmware for each engine type */ - ret = load_fw(&pdev->dev, fw_info, filename); + ret = load_fw(&pdev->dev, fw_info, filename, rid); if (ret) goto release_fw; } @@ -1155,7 +1155,7 @@ int otx2_cpt_create_eng_grps(struct otx2_cptpf_dev *cptpf, if (eng_grps->is_grps_created) goto unlock; - ret = cpt_ucode_load_fw(pdev, &fw_info); + ret = cpt_ucode_load_fw(pdev, &fw_info, eng_grps->rid); if (ret) goto unlock; @@ -1230,14 +1230,16 @@ int otx2_cpt_create_eng_grps(struct otx2_cptpf_dev *cptpf, */ rnm_to_cpt_errata_fixup(&pdev->dev); + otx2_cpt_read_af_reg(&cptpf->afpf_mbox, pdev, CPT_AF_CTL, ®_val, + BLKADDR_CPT0); /* * Configure engine group mask to allow context prefetching * for the groups and enable random number request, to enable * CPT to request random numbers from RNM. */ + reg_val |= OTX2_CPT_ALL_ENG_GRPS_MASK << 3 | BIT_ULL(16); otx2_cpt_write_af_reg(&cptpf->afpf_mbox, pdev, CPT_AF_CTL, - OTX2_CPT_ALL_ENG_GRPS_MASK << 3 | BIT_ULL(16), - BLKADDR_CPT0); + reg_val, BLKADDR_CPT0); /* * Set interval to periodically flush dirty data for the next * CTX cache entry. Set the interval count to maximum supported @@ -1412,7 +1414,7 @@ static int create_eng_caps_discovery_grps(struct pci_dev *pdev, int ret; mutex_lock(&eng_grps->lock); - ret = cpt_ucode_load_fw(pdev, &fw_info); + ret = cpt_ucode_load_fw(pdev, &fw_info, eng_grps->rid); if (ret) { mutex_unlock(&eng_grps->lock); return ret; @@ -1686,13 +1688,14 @@ int otx2_cpt_dl_custom_egrp_create(struct otx2_cptpf_dev *cptpf, goto err_unlock; } INIT_LIST_HEAD(&fw_info.ucodes); - ret = load_fw(dev, &fw_info, ucode_filename[0]); + + ret = load_fw(dev, &fw_info, ucode_filename[0], eng_grps->rid); if (ret) { dev_err(dev, "Unable to load firmware %s\n", ucode_filename[0]); goto err_unlock; } if (ucode_idx > 1) { - ret = load_fw(dev, &fw_info, ucode_filename[1]); + ret = load_fw(dev, &fw_info, ucode_filename[1], eng_grps->rid); if (ret) { dev_err(dev, "Unable to load firmware %s\n", ucode_filename[1]); diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.h b/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.h index e69320a54b5d..365fe8943bd9 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.h @@ -73,7 +73,7 @@ struct otx2_cpt_ucode_hdr { }; struct otx2_cpt_ucode { - u8 ver_str[OTX2_CPT_UCODE_VER_STR_SZ];/* + u8 ver_str[OTX2_CPT_UCODE_VER_STR_SZ + 1];/* * ucode version in readable * format */ @@ -150,6 +150,7 @@ struct otx2_cpt_eng_grps { int engs_num; /* total number of engines supported */ u8 eng_ref_cnt[OTX2_CPT_MAX_ENGINES];/* engines reference count */ bool is_grps_created; /* Is the engine groups are already created */ + u16 rid; }; struct otx2_cptpf_dev; int otx2_cpt_init_eng_grps(struct pci_dev *pdev, diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptvf.h b/drivers/crypto/marvell/octeontx2/otx2_cptvf.h index 994291e90da1..11ab9af1df15 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptvf.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cptvf.h @@ -22,6 +22,7 @@ struct otx2_cptvf_dev { int blkaddr; void *bbuf_base; unsigned long cap_flag; + u64 eng_caps[OTX2_CPT_MAX_ENG_TYPES]; }; irqreturn_t otx2_cptvf_pfvf_mbox_intr(int irq, void *arg); @@ -29,5 +30,6 @@ void otx2_cptvf_pfvf_mbox_handler(struct work_struct *work); int otx2_cptvf_send_eng_grp_num_msg(struct otx2_cptvf_dev *cptvf, int eng_type); int otx2_cptvf_send_kvf_limits_msg(struct otx2_cptvf_dev *cptvf); int otx2_cpt_mbox_bbuf_init(struct otx2_cptvf_dev *cptvf, struct pci_dev *pdev); +int otx2_cptvf_send_caps_msg(struct otx2_cptvf_dev *cptvf); #endif /* __OTX2_CPTVF_H */ diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptvf_main.c b/drivers/crypto/marvell/octeontx2/otx2_cptvf_main.c index bac729c885f9..5d1e11135c17 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptvf_main.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptvf_main.c @@ -380,6 +380,19 @@ static int otx2_cptvf_probe(struct pci_dev *pdev, goto destroy_pfvf_mbox; cptvf->blkaddr = BLKADDR_CPT0; + + ret = cptvf_hw_ops_get(cptvf); + if (ret) + goto unregister_interrupts; + + ret = otx2_cptvf_send_caps_msg(cptvf); + if (ret) { + dev_err(&pdev->dev, "Couldn't get CPT engine capabilities.\n"); + goto unregister_interrupts; + } + if (cptvf->eng_caps[OTX2_CPT_SE_TYPES] & BIT_ULL(35)) + cptvf->lfs.ops->cpt_sg_info_create = cn10k_sgv2_info_create; + /* Initialize CPT LFs */ ret = cptvf_lf_init(cptvf); if (ret) diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptvf_mbox.c b/drivers/crypto/marvell/octeontx2/otx2_cptvf_mbox.c index 75c403f2b1d9..f68da1d08fdf 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptvf_mbox.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptvf_mbox.c @@ -72,6 +72,7 @@ static void process_pfvf_mbox_mbox_msg(struct otx2_cptvf_dev *cptvf, struct otx2_cptlfs_info *lfs = &cptvf->lfs; struct otx2_cpt_kvf_limits_rsp *rsp_limits; struct otx2_cpt_egrp_num_rsp *rsp_grp; + struct otx2_cpt_caps_rsp *eng_caps; struct cpt_rd_wr_reg_msg *rsp_reg; struct msix_offset_rsp *rsp_msix; int i; @@ -127,6 +128,11 @@ static void process_pfvf_mbox_mbox_msg(struct otx2_cptvf_dev *cptvf, rsp_limits = (struct otx2_cpt_kvf_limits_rsp *) msg; cptvf->lfs.kvf_limits = rsp_limits->kvf_limits; break; + case MBOX_MSG_GET_CAPS: + eng_caps = (struct otx2_cpt_caps_rsp *)msg; + memcpy(cptvf->eng_caps, eng_caps->eng_caps, + sizeof(cptvf->eng_caps)); + break; default: dev_err(&cptvf->pdev->dev, "Unsupported msg %d received.\n", msg->id); @@ -205,3 +211,23 @@ int otx2_cptvf_send_kvf_limits_msg(struct otx2_cptvf_dev *cptvf) return otx2_cpt_send_mbox_msg(mbox, pdev); } + +int otx2_cptvf_send_caps_msg(struct otx2_cptvf_dev *cptvf) +{ + struct otx2_mbox *mbox = &cptvf->pfvf_mbox; + struct pci_dev *pdev = cptvf->pdev; + struct mbox_msghdr *req; + + req = (struct mbox_msghdr *) + otx2_mbox_alloc_msg_rsp(mbox, 0, sizeof(*req), + sizeof(struct otx2_cpt_caps_rsp)); + if (!req) { + dev_err(&pdev->dev, "RVU MBOX failed to get message.\n"); + return -EFAULT; + } + req->id = MBOX_MSG_GET_CAPS; + req->sig = OTX2_MBOX_REQ_SIG; + req->pcifunc = OTX2_CPT_RVU_PFFUNC(cptvf->vf_id, 0); + + return otx2_cpt_send_mbox_msg(mbox, pdev); +} diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptvf_reqmgr.c b/drivers/crypto/marvell/octeontx2/otx2_cptvf_reqmgr.c index 811ded72ce5f..997a2eb60c66 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptvf_reqmgr.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptvf_reqmgr.c @@ -4,9 +4,6 @@ #include "otx2_cptvf.h" #include "otx2_cpt_common.h" -/* SG list header size in bytes */ -#define SG_LIST_HDR_SIZE 8 - /* Default timeout when waiting for free pending entry in us */ #define CPT_PENTRY_TIMEOUT 1000 #define CPT_PENTRY_STEP 50 @@ -26,9 +23,9 @@ static void otx2_cpt_dump_sg_list(struct pci_dev *pdev, pr_debug("Gather list size %d\n", req->in_cnt); for (i = 0; i < req->in_cnt; i++) { - pr_debug("Buffer %d size %d, vptr 0x%p, dmaptr 0x%p\n", i, + pr_debug("Buffer %d size %d, vptr 0x%p, dmaptr 0x%llx\n", i, req->in[i].size, req->in[i].vptr, - (void *) req->in[i].dma_addr); + req->in[i].dma_addr); pr_debug("Buffer hexdump (%d bytes)\n", req->in[i].size); print_hex_dump_debug("", DUMP_PREFIX_NONE, 16, 1, @@ -36,9 +33,9 @@ static void otx2_cpt_dump_sg_list(struct pci_dev *pdev, } pr_debug("Scatter list size %d\n", req->out_cnt); for (i = 0; i < req->out_cnt; i++) { - pr_debug("Buffer %d size %d, vptr 0x%p, dmaptr 0x%p\n", i, + pr_debug("Buffer %d size %d, vptr 0x%p, dmaptr 0x%llx\n", i, req->out[i].size, req->out[i].vptr, - (void *) req->out[i].dma_addr); + req->out[i].dma_addr); pr_debug("Buffer hexdump (%d bytes)\n", req->out[i].size); print_hex_dump_debug("", DUMP_PREFIX_NONE, 16, 1, req->out[i].vptr, req->out[i].size, false); @@ -84,149 +81,6 @@ static inline void free_pentry(struct otx2_cpt_pending_entry *pentry) pentry->busy = false; } -static inline int setup_sgio_components(struct pci_dev *pdev, - struct otx2_cpt_buf_ptr *list, - int buf_count, u8 *buffer) -{ - struct otx2_cpt_sglist_component *sg_ptr = NULL; - int ret = 0, i, j; - int components; - - if (unlikely(!list)) { - dev_err(&pdev->dev, "Input list pointer is NULL\n"); - return -EFAULT; - } - - for (i = 0; i < buf_count; i++) { - if (unlikely(!list[i].vptr)) - continue; - list[i].dma_addr = dma_map_single(&pdev->dev, list[i].vptr, - list[i].size, - DMA_BIDIRECTIONAL); - if (unlikely(dma_mapping_error(&pdev->dev, list[i].dma_addr))) { - dev_err(&pdev->dev, "Dma mapping failed\n"); - ret = -EIO; - goto sg_cleanup; - } - } - components = buf_count / 4; - sg_ptr = (struct otx2_cpt_sglist_component *)buffer; - for (i = 0; i < components; i++) { - sg_ptr->len0 = cpu_to_be16(list[i * 4 + 0].size); - sg_ptr->len1 = cpu_to_be16(list[i * 4 + 1].size); - sg_ptr->len2 = cpu_to_be16(list[i * 4 + 2].size); - sg_ptr->len3 = cpu_to_be16(list[i * 4 + 3].size); - sg_ptr->ptr0 = cpu_to_be64(list[i * 4 + 0].dma_addr); - sg_ptr->ptr1 = cpu_to_be64(list[i * 4 + 1].dma_addr); - sg_ptr->ptr2 = cpu_to_be64(list[i * 4 + 2].dma_addr); - sg_ptr->ptr3 = cpu_to_be64(list[i * 4 + 3].dma_addr); - sg_ptr++; - } - components = buf_count % 4; - - switch (components) { - case 3: - sg_ptr->len2 = cpu_to_be16(list[i * 4 + 2].size); - sg_ptr->ptr2 = cpu_to_be64(list[i * 4 + 2].dma_addr); - fallthrough; - case 2: - sg_ptr->len1 = cpu_to_be16(list[i * 4 + 1].size); - sg_ptr->ptr1 = cpu_to_be64(list[i * 4 + 1].dma_addr); - fallthrough; - case 1: - sg_ptr->len0 = cpu_to_be16(list[i * 4 + 0].size); - sg_ptr->ptr0 = cpu_to_be64(list[i * 4 + 0].dma_addr); - break; - default: - break; - } - return ret; - -sg_cleanup: - for (j = 0; j < i; j++) { - if (list[j].dma_addr) { - dma_unmap_single(&pdev->dev, list[j].dma_addr, - list[j].size, DMA_BIDIRECTIONAL); - } - - list[j].dma_addr = 0; - } - return ret; -} - -static inline struct otx2_cpt_inst_info *info_create(struct pci_dev *pdev, - struct otx2_cpt_req_info *req, - gfp_t gfp) -{ - int align = OTX2_CPT_DMA_MINALIGN; - struct otx2_cpt_inst_info *info; - u32 dlen, align_dlen, info_len; - u16 g_sz_bytes, s_sz_bytes; - u32 total_mem_len; - - if (unlikely(req->in_cnt > OTX2_CPT_MAX_SG_IN_CNT || - req->out_cnt > OTX2_CPT_MAX_SG_OUT_CNT)) { - dev_err(&pdev->dev, "Error too many sg components\n"); - return NULL; - } - - g_sz_bytes = ((req->in_cnt + 3) / 4) * - sizeof(struct otx2_cpt_sglist_component); - s_sz_bytes = ((req->out_cnt + 3) / 4) * - sizeof(struct otx2_cpt_sglist_component); - - dlen = g_sz_bytes + s_sz_bytes + SG_LIST_HDR_SIZE; - align_dlen = ALIGN(dlen, align); - info_len = ALIGN(sizeof(*info), align); - total_mem_len = align_dlen + info_len + sizeof(union otx2_cpt_res_s); - - info = kzalloc(total_mem_len, gfp); - if (unlikely(!info)) - return NULL; - - info->dlen = dlen; - info->in_buffer = (u8 *)info + info_len; - - ((u16 *)info->in_buffer)[0] = req->out_cnt; - ((u16 *)info->in_buffer)[1] = req->in_cnt; - ((u16 *)info->in_buffer)[2] = 0; - ((u16 *)info->in_buffer)[3] = 0; - cpu_to_be64s((u64 *)info->in_buffer); - - /* Setup gather (input) components */ - if (setup_sgio_components(pdev, req->in, req->in_cnt, - &info->in_buffer[8])) { - dev_err(&pdev->dev, "Failed to setup gather list\n"); - goto destroy_info; - } - - if (setup_sgio_components(pdev, req->out, req->out_cnt, - &info->in_buffer[8 + g_sz_bytes])) { - dev_err(&pdev->dev, "Failed to setup scatter list\n"); - goto destroy_info; - } - - info->dma_len = total_mem_len - info_len; - info->dptr_baddr = dma_map_single(&pdev->dev, info->in_buffer, - info->dma_len, DMA_BIDIRECTIONAL); - if (unlikely(dma_mapping_error(&pdev->dev, info->dptr_baddr))) { - dev_err(&pdev->dev, "DMA Mapping failed for cpt req\n"); - goto destroy_info; - } - /* - * Get buffer for union otx2_cpt_res_s response - * structure and its physical address - */ - info->completion_addr = info->in_buffer + align_dlen; - info->comp_baddr = info->dptr_baddr + align_dlen; - - return info; - -destroy_info: - otx2_cpt_info_destroy(pdev, info); - return NULL; -} - static int process_request(struct pci_dev *pdev, struct otx2_cpt_req_info *req, struct otx2_cpt_pending_queue *pqueue, struct otx2_cptlf_info *lf) @@ -247,7 +101,7 @@ static int process_request(struct pci_dev *pdev, struct otx2_cpt_req_info *req, if (unlikely(!otx2_cptlf_started(lf->lfs))) return -ENODEV; - info = info_create(pdev, req, gfp); + info = lf->lfs->ops->cpt_sg_info_create(pdev, req, gfp); if (unlikely(!info)) { dev_err(&pdev->dev, "Setting up cpt inst info failed"); return -ENOMEM; @@ -303,8 +157,8 @@ static int process_request(struct pci_dev *pdev, struct otx2_cpt_req_info *req, /* 64-bit swap for microcode data reads, not needed for addresses*/ cpu_to_be64s(&iq_cmd.cmd.u); - iq_cmd.dptr = info->dptr_baddr; - iq_cmd.rptr = 0; + iq_cmd.dptr = info->dptr_baddr | info->gthr_sz << 60; + iq_cmd.rptr = info->rptr_baddr | info->sctr_sz << 60; iq_cmd.cptr.u = 0; iq_cmd.cptr.s.grp = ctrl->s.grp; From patchwork Mon Dec 11 07:19:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srujana Challa X-Patchwork-Id: 752629 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="Q5ZlSUMV" Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1D4AA8E; Sun, 10 Dec 2023 23:19:53 -0800 (PST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BAN0AkD024983; Sun, 10 Dec 2023 23:19:41 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=mzIRxzofxm69ARqKSSYIysXYLB1wXPi+5zsq3Gkvtc8=; b=Q5Z lSUMVjpzRqVGwJXChA8+SZ2K+XUUVLdaOpGMQJWGFgRx8Ldxx4aMXx+wDm05YWYm xoxJxE2yKEWEWWdjotxEL+tKw4lnB9VDJRjtMgyucLfyF9zAS7T6ppEcsQ5rqBjc nX5pi8uhTqMfJDeyXaZg+6qCPSmYOgyqeL8AFt7V/+Ie7hxtgRrnOk8HE3szz3kk m1XLTgG4GqfOjE2N8DEiXms28SSVULtueh8LLQIlpnb3zF2f3SaHPsi57PTPopkd BuNnijN9TUDMyDaDCEIYx6Bz31jHGx6TSBe1BgRrBUNmSaYxoiKAXPQHLPYSpq9h VO00lTayJGcUsD86M5w== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3uvrmjkmhg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 10 Dec 2023 23:19:41 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Sun, 10 Dec 2023 23:19:38 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Sun, 10 Dec 2023 23:19:38 -0800 Received: from localhost.localdomain (unknown [10.28.36.175]) by maili.marvell.com (Postfix) with ESMTP id 2E41E3F7097; Sun, 10 Dec 2023 23:19:32 -0800 (PST) From: Srujana Challa To: , , CC: , , , , , , , , , , , , , , , , Subject: [PATCH net-next v1 03/10] crypto: octeontx2: add devlink option to set max_rxc_icb_cnt Date: Mon, 11 Dec 2023 12:49:06 +0530 Message-ID: <20231211071913.151225-4-schalla@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231211071913.151225-1-schalla@marvell.com> References: <20231211071913.151225-1-schalla@marvell.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: 2erNIo0ptaBiogBJb-8p4d6aa8rLqfat X-Proofpoint-ORIG-GUID: 2erNIo0ptaBiogBJb-8p4d6aa8rLqfat X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 On CN10KA B0/CN10KB HW, maximum icb entries that RX can use, can be configured through HW CSR. This patch adds option to set max icb entries through devlink parameter and also sets max_rxc_icb_cnt to 0xc0 as default to match inline inbound peak performance compared to other chip versions. This patch also documents the devlink parameter under Documentation/crypto/device_drivers. Signed-off-by: Srujana Challa --- Documentation/crypto/device_drivers/index.rst | 9 ++++ .../crypto/device_drivers/octeontx2.rst | 25 +++++++++++ Documentation/crypto/index.rst | 1 + .../marvell/octeontx2/otx2_cpt_common.h | 8 ++++ .../marvell/octeontx2/otx2_cpt_devlink.c | 45 +++++++++++++++++++ .../net/ethernet/marvell/octeontx2/af/rvu.h | 20 +++++++++ .../ethernet/marvell/octeontx2/af/rvu_cpt.c | 14 ++++++ .../ethernet/marvell/octeontx2/af/rvu_reg.h | 1 + 8 files changed, 123 insertions(+) create mode 100644 Documentation/crypto/device_drivers/index.rst create mode 100644 Documentation/crypto/device_drivers/octeontx2.rst diff --git a/Documentation/crypto/device_drivers/index.rst b/Documentation/crypto/device_drivers/index.rst new file mode 100644 index 000000000000..c81d311ac61b --- /dev/null +++ b/Documentation/crypto/device_drivers/index.rst @@ -0,0 +1,9 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Hardware Device Driver Specific Documentation +--------------------------------------------- + +.. toctree:: + :maxdepth: 1 + + octeontx2 diff --git a/Documentation/crypto/device_drivers/octeontx2.rst b/Documentation/crypto/device_drivers/octeontx2.rst new file mode 100644 index 000000000000..0481bdcd77e9 --- /dev/null +++ b/Documentation/crypto/device_drivers/octeontx2.rst @@ -0,0 +1,25 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================= +octeontx2 devlink support +========================= + +This document describes the devlink features implemented by the ``octeontx2 CPT`` +device drivers. + +Parameters +========== + +The ``octeontx2`` driver implements the following driver-specific parameters. + +.. list-table:: Driver-specific parameters implemented + :widths: 5 5 5 85 + + * - Name + - Type + - Mode + - Description + * - ``max_rxc_icb_cnt`` + - u16 + - runtime + - Configures maximum icb entries that HW can use in RX path. diff --git a/Documentation/crypto/index.rst b/Documentation/crypto/index.rst index da5d5ad2bdf3..945ca1505ad9 100644 --- a/Documentation/crypto/index.rst +++ b/Documentation/crypto/index.rst @@ -28,3 +28,4 @@ for cryptographic use cases, as well as programming examples. api api-samples descore-readme + device_drivers/index diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h index 2a7544252444..1539828aa981 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h @@ -192,6 +192,14 @@ static inline void otx2_cpt_set_hw_caps(struct pci_dev *pdev, } } +static inline bool cpt_feature_rxc_icb_cnt(struct pci_dev *pdev) +{ + if (!is_dev_otx2(pdev) && !is_dev_cn10ka_ax(pdev)) + return true; + + return false; +} + int otx2_cpt_send_ready_msg(struct otx2_mbox *mbox, struct pci_dev *pdev); int otx2_cpt_send_mbox_msg(struct otx2_mbox *mbox, struct pci_dev *pdev); diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_devlink.c b/drivers/crypto/marvell/octeontx2/otx2_cpt_devlink.c index a2aba0b0d68a..16000dd19011 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_devlink.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_devlink.c @@ -32,10 +32,49 @@ static int otx2_cpt_dl_uc_info(struct devlink *dl, u32 id, return 0; } +static int otx2_cpt_dl_max_rxc_icb_cnt(struct devlink *dl, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct otx2_cpt_devlink *cpt_dl = devlink_priv(dl); + struct otx2_cptpf_dev *cptpf = cpt_dl->cptpf; + struct pci_dev *pdev = cptpf->pdev; + u64 reg_val = 0; + + otx2_cpt_read_af_reg(&cptpf->afpf_mbox, pdev, CPT_AF_RXC_CFG1, ®_val, + BLKADDR_CPT0); + ctx->val.vu16 = (reg_val >> 32) & 0x1FF; + + return 0; +} + +static int otx2_cpt_dl_max_rxc_icb_cnt_set(struct devlink *dl, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct otx2_cpt_devlink *cpt_dl = devlink_priv(dl); + struct otx2_cptpf_dev *cptpf = cpt_dl->cptpf; + struct pci_dev *pdev = cptpf->pdev; + u64 reg_val = 0; + + if (cptpf->enabled_vfs != 0) + return -EPERM; + + if (cpt_feature_rxc_icb_cnt(pdev)) { + otx2_cpt_read_af_reg(&cptpf->afpf_mbox, pdev, CPT_AF_RXC_CFG1, + ®_val, BLKADDR_CPT0); + reg_val &= ~(0x1FFULL << 32); + reg_val |= (u64)ctx->val.vu16 << 32; + return otx2_cpt_write_af_reg(&cptpf->afpf_mbox, pdev, + CPT_AF_RXC_CFG1, reg_val, + BLKADDR_CPT0); + } + return 0; +} + enum otx2_cpt_dl_param_id { OTX2_CPT_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX, OTX2_CPT_DEVLINK_PARAM_ID_EGRP_CREATE, OTX2_CPT_DEVLINK_PARAM_ID_EGRP_DELETE, + OTX2_CPT_DEVLINK_PARAM_ID_MAX_RXC_ICB_CNT, }; static const struct devlink_param otx2_cpt_dl_params[] = { @@ -49,6 +88,12 @@ static const struct devlink_param otx2_cpt_dl_params[] = { BIT(DEVLINK_PARAM_CMODE_RUNTIME), otx2_cpt_dl_uc_info, otx2_cpt_dl_egrp_delete, NULL), + DEVLINK_PARAM_DRIVER(OTX2_CPT_DEVLINK_PARAM_ID_MAX_RXC_ICB_CNT, + "max_rxc_icb_cnt", DEVLINK_PARAM_TYPE_U16, + BIT(DEVLINK_PARAM_CMODE_RUNTIME), + otx2_cpt_dl_max_rxc_icb_cnt, + otx2_cpt_dl_max_rxc_icb_cnt_set, + NULL), }; static int otx2_cpt_dl_info_firmware_version_put(struct devlink_info_req *req, diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h index c4d999ef5ab4..afc9a1c77513 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -646,6 +646,26 @@ static inline bool is_cnf10ka_a0(struct rvu *rvu) return false; } +static inline bool is_cn10ka_a0(struct rvu *rvu) +{ + struct pci_dev *pdev = rvu->pdev; + + if (pdev->subsystem_device == PCI_SUBSYS_DEVID_CN10K_A && + (pdev->revision & 0x0F) == 0x0) + return true; + return false; +} + +static inline bool is_cn10ka_a1(struct rvu *rvu) +{ + struct pci_dev *pdev = rvu->pdev; + + if (pdev->subsystem_device == PCI_SUBSYS_DEVID_CN10K_A && + (pdev->revision & 0x0F) == 0x1) + return true; + return false; +} + static inline bool is_rvu_npc_hash_extract_en(struct rvu *rvu) { u64 npc_const3; diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cpt.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cpt.c index f047185f38e0..f36a5ee55ac3 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cpt.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cpt.c @@ -673,6 +673,7 @@ static bool is_valid_offset(struct rvu *rvu, struct cpt_rd_wr_reg_msg *req) case CPT_AF_BLK_RST: case CPT_AF_CONSTANTS1: case CPT_AF_CTX_FLUSH_TIMER: + case CPT_AF_RXC_CFG1: return true; } @@ -1213,8 +1214,21 @@ int rvu_cpt_ctx_flush(struct rvu *rvu, u16 pcifunc) int rvu_cpt_init(struct rvu *rvu) { + u64 reg_val; + /* Retrieve CPT PF number */ rvu->cpt_pf_num = get_cpt_pf_num(rvu); + if (is_block_implemented(rvu->hw, BLKADDR_CPT0) && + (!is_rvu_otx2(rvu) && !is_cn10ka_a0(rvu) && !is_cn10ka_a1(rvu))) { + /* Set CPT_AF_RXC_CFG1:max_rxc_icb_cnt to 0xc0 to not effect + * inline inbound peak performance + */ + reg_val = rvu_read64(rvu, BLKADDR_CPT0, CPT_AF_RXC_CFG1); + reg_val &= ~(0x1FFULL << 32); + reg_val |= 0xC0ULL << 32; + rvu_write64(rvu, BLKADDR_CPT0, CPT_AF_RXC_CFG1, reg_val); + } + spin_lock_init(&rvu->cpt_intr_lock); return 0; diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h b/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h index b42e631e52d0..b3f1442d5196 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h @@ -532,6 +532,7 @@ #define CPT_AF_CTX_PSH_PC (0x49450ull) #define CPT_AF_CTX_PSH_LATENCY_PC (0x49458ull) #define CPT_AF_CTX_CAM_DATA(a) (0x49800ull | (u64)(a) << 3) +#define CPT_AF_RXC_CFG1 (0x50000ull) #define CPT_AF_RXC_TIME (0x50010ull) #define CPT_AF_RXC_TIME_CFG (0x50018ull) #define CPT_AF_RXC_DFRG (0x50020ull) From patchwork Mon Dec 11 07:19:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srujana Challa X-Patchwork-Id: 753211 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="QWmrRAqA" Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1CCA98E; Sun, 10 Dec 2023 23:19:58 -0800 (PST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BAMrXDn012812; Sun, 10 Dec 2023 23:19:47 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=jHSz//mmw13HNwSxadbtAutphKrxCAWXYgzS6Oin/ik=; b=QWm rRAqASL0qB2Qa+/Y3s8AaM1ehLMfojY/B21HfkGyVjC0KgI4E0qIjumCGKa1w7pm PYVtsFgbodNjAl15RqDHydKfamKgKV8WTb9UUdgC5/QCcrI/UdYxh2KFfjBwZ4jW qEY/YV5RjCfkh3yuNxzEfsnRxRfkw01NosnPig/QQBhZs8cgK+79BePlYohCdotK 39OUj1sIQhrtsF42F2w3e5OfAJhe18zwW9DhO0QGzRYx9JnmkdOj7YBtZcrAgMor qsU/bBrlWorxQFdMeIDcxZhdgHJnHxcyzk9H6nVhAS2wPp3laSIePpgZoj22XiDd FcRyTHfLD7CMwNY2ZSg== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3uvrmjkmj1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 10 Dec 2023 23:19:47 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Sun, 10 Dec 2023 23:19:45 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Sun, 10 Dec 2023 23:19:44 -0800 Received: from localhost.localdomain (unknown [10.28.36.175]) by maili.marvell.com (Postfix) with ESMTP id 5DCBC3F709C; Sun, 10 Dec 2023 23:19:39 -0800 (PST) From: Srujana Challa To: , , CC: , , , , , , , , , , , , , , , , Subject: [PATCH net-next v1 04/10] crypto: octeontx2: add devlink option to set t106 mode Date: Mon, 11 Dec 2023 12:49:07 +0530 Message-ID: <20231211071913.151225-5-schalla@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231211071913.151225-1-schalla@marvell.com> References: <20231211071913.151225-1-schalla@marvell.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: 8K8d384yJ-MGVYqNGMPyWv_rEJHQNMJm X-Proofpoint-ORIG-GUID: 8K8d384yJ-MGVYqNGMPyWv_rEJHQNMJm X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 On CN10KA B0/CN10KB, CPT scatter gather format has modified to support multi-seg in inline IPsec. Due to this CPT requires new firmware and doesn't work with CN10KA0/A1 firmware. To make HW works in backward compatibility mode or works with CN10KA0/A1 firmware, a bit(T106_MODE) is introduced in HW CSR. This patch adds devlink parameter for configuring T106_MODE. Signed-off-by: Srujana Challa --- .../crypto/device_drivers/octeontx2.rst | 4 ++ .../marvell/octeontx2/otx2_cpt_common.h | 8 +++ .../marvell/octeontx2/otx2_cpt_devlink.c | 50 +++++++++++++++++-- .../marvell/octeontx2/otx2_cptpf_main.c | 4 +- 4 files changed, 59 insertions(+), 7 deletions(-) diff --git a/Documentation/crypto/device_drivers/octeontx2.rst b/Documentation/crypto/device_drivers/octeontx2.rst index 0481bdcd77e9..288998c10b3d 100644 --- a/Documentation/crypto/device_drivers/octeontx2.rst +++ b/Documentation/crypto/device_drivers/octeontx2.rst @@ -23,3 +23,7 @@ The ``octeontx2`` driver implements the following driver-specific parameters. - u16 - runtime - Configures maximum icb entries that HW can use in RX path. + * - ``t106_mode`` + - u8 + - runtime + - Used to configure CN10KA B0/CN10KB CPT to work as CN10KA A0/A1. diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h index 1539828aa981..847f539cdd0e 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h @@ -200,6 +200,14 @@ static inline bool cpt_feature_rxc_icb_cnt(struct pci_dev *pdev) return false; } +static inline bool cpt_feature_sgv2(struct pci_dev *pdev) +{ + if (!is_dev_otx2(pdev) && !is_dev_cn10ka_ax(pdev)) + return true; + + return false; +} + int otx2_cpt_send_ready_msg(struct otx2_mbox *mbox, struct pci_dev *pdev); int otx2_cpt_send_mbox_msg(struct otx2_mbox *mbox, struct pci_dev *pdev); diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_devlink.c b/drivers/crypto/marvell/octeontx2/otx2_cpt_devlink.c index 16000dd19011..4654b8438154 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_devlink.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_devlink.c @@ -24,10 +24,7 @@ static int otx2_cpt_dl_egrp_delete(struct devlink *dl, u32 id, static int otx2_cpt_dl_uc_info(struct devlink *dl, u32 id, struct devlink_param_gset_ctx *ctx) { - struct otx2_cpt_devlink *cpt_dl = devlink_priv(dl); - struct otx2_cptpf_dev *cptpf = cpt_dl->cptpf; - - otx2_cpt_print_uc_dbg_info(cptpf); + ctx->val.vstr[0] = '\0'; return 0; } @@ -70,11 +67,50 @@ static int otx2_cpt_dl_max_rxc_icb_cnt_set(struct devlink *dl, u32 id, return 0; } +static int otx2_cpt_dl_t106_mode_get(struct devlink *dl, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct otx2_cpt_devlink *cpt_dl = devlink_priv(dl); + struct otx2_cptpf_dev *cptpf = cpt_dl->cptpf; + struct pci_dev *pdev = cptpf->pdev; + u64 reg_val = 0; + + otx2_cpt_read_af_reg(&cptpf->afpf_mbox, pdev, CPT_AF_CTL, ®_val, + BLKADDR_CPT0); + ctx->val.vu8 = (reg_val >> 18) & 0x1; + + return 0; +} + +static int otx2_cpt_dl_t106_mode_set(struct devlink *dl, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct otx2_cpt_devlink *cpt_dl = devlink_priv(dl); + struct otx2_cptpf_dev *cptpf = cpt_dl->cptpf; + struct pci_dev *pdev = cptpf->pdev; + u64 reg_val = 0; + + if (cptpf->enabled_vfs != 0 || cptpf->eng_grps.is_grps_created) + return -EPERM; + + if (cpt_feature_sgv2(pdev)) { + otx2_cpt_read_af_reg(&cptpf->afpf_mbox, pdev, CPT_AF_CTL, + ®_val, BLKADDR_CPT0); + reg_val &= ~(0x1ULL << 18); + reg_val |= ((u64)ctx->val.vu8 & 0x1) << 18; + return otx2_cpt_write_af_reg(&cptpf->afpf_mbox, pdev, + CPT_AF_CTL, reg_val, BLKADDR_CPT0); + } + + return 0; +} + enum otx2_cpt_dl_param_id { OTX2_CPT_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX, OTX2_CPT_DEVLINK_PARAM_ID_EGRP_CREATE, OTX2_CPT_DEVLINK_PARAM_ID_EGRP_DELETE, OTX2_CPT_DEVLINK_PARAM_ID_MAX_RXC_ICB_CNT, + OTX2_CPT_DEVLINK_PARAM_ID_T106_MODE, }; static const struct devlink_param otx2_cpt_dl_params[] = { @@ -94,6 +130,11 @@ static const struct devlink_param otx2_cpt_dl_params[] = { otx2_cpt_dl_max_rxc_icb_cnt, otx2_cpt_dl_max_rxc_icb_cnt_set, NULL), + DEVLINK_PARAM_DRIVER(OTX2_CPT_DEVLINK_PARAM_ID_T106_MODE, + "t106_mode", DEVLINK_PARAM_TYPE_U8, + BIT(DEVLINK_PARAM_CMODE_RUNTIME), + otx2_cpt_dl_t106_mode_get, otx2_cpt_dl_t106_mode_set, + NULL), }; static int otx2_cpt_dl_info_firmware_version_put(struct devlink_info_req *req, @@ -165,7 +206,6 @@ int otx2_cpt_register_dl(struct otx2_cptpf_dev *cptpf) devlink_free(dl); return ret; } - devlink_register(dl); return 0; diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c index c64c50a964ed..7d44b54659bf 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c @@ -600,10 +600,10 @@ static int cptpf_get_rid(struct pci_dev *pdev, struct otx2_cptpf_dev *cptpf) } otx2_cpt_read_af_reg(&cptpf->afpf_mbox, pdev, CPT_AF_CTL, ®_val, BLKADDR_CPT0); - if ((is_dev_cn10ka_b0(pdev) && (reg_val & BIT_ULL(18))) || + if ((cpt_feature_sgv2(pdev) && (reg_val & BIT_ULL(18))) || is_dev_cn10ka_ax(pdev)) eng_grps->rid = CPT_UC_RID_CN10K_A; - else if (is_dev_cn10kb(pdev) || is_dev_cn10ka_b0(pdev)) + else if (cpt_feature_sgv2(pdev)) eng_grps->rid = CPT_UC_RID_CN10K_B; return 0; From patchwork Mon Dec 11 07:19:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srujana Challa X-Patchwork-Id: 752628 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="QRjp9Ell" Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 22D038E; Sun, 10 Dec 2023 23:20:03 -0800 (PST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BAMh88S026392; Sun, 10 Dec 2023 23:19:53 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=aRjwBcyQurjYx2gnRf/+QgF7bF7tu+amylWotPqEUL4=; b=QRj p9EllfW+lJugThxsD3CJ6goA2HovMbdPzIU70qYjUyNpgoxCAHv18HvO5TTp8ede /m6AvP/f76XXLK5/QZBr2Y2AkXynkhwYJQbTD6ttTL9hVf9AGlivZOsSNHxWjD91 WSPndQgcaXdjkHSrhWBYoIxz9boLluYf41uKs68aoTgHrTBgKFkh8BVW4h4wrxeI 1nIyafmtPkqljv+Vl2Yj3liFd2NNPP9bbcOZMn9shpTZ5VS5s5X2kz5+VOceqxKW /bgTVy7ctnq+vmyzQEBWCTewdx0aFUo0HJAXt61ym2m969rdThW7QEUlsR5brduh 0e5HAx6eMu32mmTR61w== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3uvrmjkmjh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 10 Dec 2023 23:19:53 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Sun, 10 Dec 2023 23:19:51 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Sun, 10 Dec 2023 23:19:51 -0800 Received: from localhost.localdomain (unknown [10.28.36.175]) by maili.marvell.com (Postfix) with ESMTP id 73C643F7097; Sun, 10 Dec 2023 23:19:45 -0800 (PST) From: Srujana Challa To: , , CC: , , , , , , , , , , , , , , , , Subject: [PATCH net-next v1 05/10] crypto: octeontx2: remove errata workaround for CN10KB or CN10KA B0 chip. Date: Mon, 11 Dec 2023 12:49:08 +0530 Message-ID: <20231211071913.151225-6-schalla@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231211071913.151225-1-schalla@marvell.com> References: <20231211071913.151225-1-schalla@marvell.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: VtU_8zHEHqwyCtlTICLOlizmIloqu8FC X-Proofpoint-ORIG-GUID: VtU_8zHEHqwyCtlTICLOlizmIloqu8FC X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 Adds code to not execute CPT errata "when CPT_AF_DIAG[FLT_DIS] = 0 and a CPT engine access to LLC/DRAM encounters a fault/poison, a rare case may result in unpredictable data being delivered to a CPT engine" workaround on CN10KA B0/CN10KB HW as it is fixed on these chips. Signed-off-by: Srujana Challa --- drivers/crypto/marvell/octeontx2/otx2_cpt_common.h | 8 ++++++++ drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c | 10 ++++++---- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h index 847f539cdd0e..568efac3d8a6 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h @@ -192,6 +192,14 @@ static inline void otx2_cpt_set_hw_caps(struct pci_dev *pdev, } } +static inline bool cpt_is_errata_38550_exists(struct pci_dev *pdev) +{ + if (is_dev_otx2(pdev) || is_dev_cn10ka_ax(pdev)) + return true; + + return false; +} + static inline bool cpt_feature_rxc_icb_cnt(struct pci_dev *pdev) { if (!is_dev_otx2(pdev) && !is_dev_cn10ka_ax(pdev)) diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c index 7fccc348f66e..e319aa1ff119 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c @@ -1254,10 +1254,12 @@ int otx2_cpt_create_eng_grps(struct otx2_cptpf_dev *cptpf, * encounters a fault/poison, a rare case may result in * unpredictable data being delivered to a CPT engine. */ - otx2_cpt_read_af_reg(&cptpf->afpf_mbox, pdev, CPT_AF_DIAG, ®_val, - BLKADDR_CPT0); - otx2_cpt_write_af_reg(&cptpf->afpf_mbox, pdev, CPT_AF_DIAG, - reg_val | BIT_ULL(24), BLKADDR_CPT0); + if (cpt_is_errata_38550_exists(pdev)) { + otx2_cpt_read_af_reg(&cptpf->afpf_mbox, pdev, CPT_AF_DIAG, + ®_val, BLKADDR_CPT0); + otx2_cpt_write_af_reg(&cptpf->afpf_mbox, pdev, CPT_AF_DIAG, + reg_val | BIT_ULL(24), BLKADDR_CPT0); + } mutex_unlock(&eng_grps->lock); return 0; From patchwork Mon Dec 11 07:19:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srujana Challa X-Patchwork-Id: 753210 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="FzQO83M+" Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C0BF0D7; Sun, 10 Dec 2023 23:20:09 -0800 (PST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BAMrXDq012812; Sun, 10 Dec 2023 23:19:59 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=N18/kp16RTpQI4FddQQeEqyZYd4XAmPWOcNzgVDTp4c=; b=FzQ O83M+VGC+rNJvlZQL5dmKR5vFIDD6+IptK71k1sOUDRoh/+nddaQe6+A8ZCmUuhk qQxzjGuL/xa5w89z/ctnGG3oZ+0LopYgpWPc/tofixbJ36WBt3lkKLfQDHDhGAPJ HURXbQeE2drHz7G+MVdOLYJDPmPmjYBmOHCqGjmFIjSJBxlCDn8EcogbY/mSKYwX jyHwbNipQ0w2KllN2NgyNYWrhoLfkJgM15Tnc4137Fzj7nChJULHal8qebtCsbUJ BHUffESVTAFUV5iYU8NaakD6xV6j7IaDEJPMl/Z4jW+YWxW5fzF6u5hk5WYoqeg4 GXz9+1JJy2eUe85pSCw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3uvrmjkmk2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 10 Dec 2023 23:19:59 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Sun, 10 Dec 2023 23:19:57 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Sun, 10 Dec 2023 23:19:57 -0800 Received: from localhost.localdomain (unknown [10.28.36.175]) by maili.marvell.com (Postfix) with ESMTP id 8CC0E3F7098; Sun, 10 Dec 2023 23:19:51 -0800 (PST) From: Srujana Challa To: , , CC: , , , , , , , , , , , , , , , , Subject: [PATCH net-next v1 06/10] crypto: octeontx2: add LF reset on queue disable Date: Mon, 11 Dec 2023 12:49:09 +0530 Message-ID: <20231211071913.151225-7-schalla@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231211071913.151225-1-schalla@marvell.com> References: <20231211071913.151225-1-schalla@marvell.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: RsFBPCCpOSluKESU8lFy50uHhgtLagCu X-Proofpoint-ORIG-GUID: RsFBPCCpOSluKESU8lFy50uHhgtLagCu X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 CPT LF must be reset and follow CPT LF disable sequence suggested by HW team, when driver exits. This patch adds code for the same. Signed-off-by: Srujana Challa --- .../marvell/octeontx2/otx2_cpt_common.h | 1 + .../marvell/octeontx2/otx2_cpt_mbox_common.c | 26 +++++++ drivers/crypto/marvell/octeontx2/otx2_cptlf.h | 77 ++++++++++++------- .../marvell/octeontx2/otx2_cptpf_mbox.c | 9 ++- .../marvell/octeontx2/otx2_cptvf_mbox.c | 2 + 5 files changed, 86 insertions(+), 29 deletions(-) diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h index 568efac3d8a6..f5e3392b9b19 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h @@ -232,5 +232,6 @@ int otx2_cpt_attach_rscrs_msg(struct otx2_cptlfs_info *lfs); int otx2_cpt_detach_rsrcs_msg(struct otx2_cptlfs_info *lfs); int otx2_cpt_msix_offset_msg(struct otx2_cptlfs_info *lfs); int otx2_cpt_sync_mbox_msg(struct otx2_mbox *mbox); +int otx2_cpt_lf_reset_msg(struct otx2_cptlfs_info *lfs, int slot); #endif /* __OTX2_CPT_COMMON_H */ diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_mbox_common.c b/drivers/crypto/marvell/octeontx2/otx2_cpt_mbox_common.c index 273ee5352a50..da8e4e4e7aed 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_mbox_common.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_mbox_common.c @@ -229,3 +229,29 @@ int otx2_cpt_sync_mbox_msg(struct otx2_mbox *mbox) return otx2_mbox_check_rsp_msgs(mbox, 0); } EXPORT_SYMBOL_NS_GPL(otx2_cpt_sync_mbox_msg, CRYPTO_DEV_OCTEONTX2_CPT); + +int otx2_cpt_lf_reset_msg(struct otx2_cptlfs_info *lfs, int slot) +{ + struct otx2_mbox *mbox = lfs->mbox; + struct pci_dev *pdev = lfs->pdev; + struct cpt_lf_rst_req *req; + int ret; + + req = (struct cpt_lf_rst_req *)otx2_mbox_alloc_msg_rsp(mbox, 0, sizeof(*req), + sizeof(struct msg_rsp)); + if (req == NULL) { + dev_err(&pdev->dev, "RVU MBOX failed to get message.\n"); + return -EFAULT; + } + + req->hdr.id = MBOX_MSG_CPT_LF_RESET; + req->hdr.sig = OTX2_MBOX_REQ_SIG; + req->hdr.pcifunc = 0; + req->slot = slot; + ret = otx2_cpt_send_mbox_msg(mbox, pdev); + if (ret) + return ret; + + return ret; +} +EXPORT_SYMBOL_NS_GPL(otx2_cpt_lf_reset_msg, CRYPTO_DEV_OCTEONTX2_CPT); diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptlf.h b/drivers/crypto/marvell/octeontx2/otx2_cptlf.h index 70138d65ba9e..8d6f10f67ac0 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptlf.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cptlf.h @@ -5,6 +5,7 @@ #define __OTX2_CPTLF_H #include +#include #include #include #include "otx2_cpt_common.h" @@ -118,6 +119,7 @@ struct otx2_cptlfs_info { u8 kvf_limits; /* Kernel crypto limits */ atomic_t state; /* LF's state. started/reset */ int blkaddr; /* CPT blkaddr: BLKADDR_CPT0/BLKADDR_CPT1 */ + int global_slot; /* Global slot across the blocks */ }; static inline void otx2_cpt_free_instruction_queues( @@ -205,48 +207,71 @@ static inline void otx2_cptlf_set_iqueues_size(struct otx2_cptlfs_info *lfs) otx2_cptlf_do_set_iqueue_size(&lfs->lf[slot]); } +#define INFLIGHT GENMASK_ULL(8, 0) +#define GRB_CNT GENMASK_ULL(39, 32) +#define GWB_CNT GENMASK_ULL(47, 40) +#define XQ_XOR GENMASK_ULL(63, 63) +#define DQPTR GENMASK_ULL(19, 0) +#define NQPTR GENMASK_ULL(51, 32) + static inline void otx2_cptlf_do_disable_iqueue(struct otx2_cptlf_info *lf) { - union otx2_cptx_lf_ctl lf_ctl = { .u = 0x0 }; - union otx2_cptx_lf_inprog lf_inprog; + void __iomem *reg_base = lf->lfs->reg_base; + struct pci_dev *pdev = lf->lfs->pdev; u8 blkaddr = lf->lfs->blkaddr; - int timeout = 20; + int timeout = 1000000; + u64 inprog, inst_ptr; + u64 slot = lf->slot; + u64 qsize, pending; + int i = 0; /* Disable instructions enqueuing */ - otx2_cpt_write64(lf->lfs->reg_base, blkaddr, lf->slot, - OTX2_CPT_LF_CTL, lf_ctl.u); + otx2_cpt_write64(reg_base, blkaddr, slot, OTX2_CPT_LF_CTL, 0x0); - /* Wait for instruction queue to become empty */ + inprog = otx2_cpt_read64(reg_base, blkaddr, slot, OTX2_CPT_LF_INPROG); + inprog |= BIT_ULL(16); + otx2_cpt_write64(reg_base, blkaddr, slot, OTX2_CPT_LF_INPROG, inprog); + + qsize = otx2_cpt_read64(reg_base, blkaddr, slot, OTX2_CPT_LF_Q_SIZE) & 0x7FFF; + do { + inst_ptr = otx2_cpt_read64(reg_base, blkaddr, slot, OTX2_CPT_LF_Q_INST_PTR); + pending = (FIELD_GET(XQ_XOR, inst_ptr) * qsize * 40) + + FIELD_GET(NQPTR, inst_ptr) - FIELD_GET(DQPTR, inst_ptr); + udelay(1); + timeout--; + } while ((pending != 0) && (timeout != 0)); + + if (timeout == 0) + dev_warn(&pdev->dev, "TIMEOUT: CPT poll on pending instructions\n"); + + timeout = 1000000; + /* Wait for CPT queue to become execution-quiescent */ do { - lf_inprog.u = otx2_cpt_read64(lf->lfs->reg_base, blkaddr, - lf->slot, OTX2_CPT_LF_INPROG); - if (!lf_inprog.s.inflight) - break; - - usleep_range(10000, 20000); - if (timeout-- < 0) { - dev_err(&lf->lfs->pdev->dev, - "Error LF %d is still busy.\n", lf->slot); - break; + inprog = otx2_cpt_read64(reg_base, blkaddr, slot, OTX2_CPT_LF_INPROG); + + if ((FIELD_GET(INFLIGHT, inprog) == 0) && + (FIELD_GET(GRB_CNT, inprog) == 0)) { + i++; + } else { + i = 0; + timeout--; } + } while ((timeout != 0) && (i < 10)); - } while (1); - - /* - * Disable executions in the LF's queue, - * the queue should be empty at this point - */ - lf_inprog.s.eena = 0x0; - otx2_cpt_write64(lf->lfs->reg_base, blkaddr, lf->slot, - OTX2_CPT_LF_INPROG, lf_inprog.u); + if (timeout == 0) + dev_warn(&pdev->dev, "TIMEOUT: CPT poll on inflight count\n"); + /* Wait for 2 us to flush all queue writes to memory */ + udelay(2); } static inline void otx2_cptlf_disable_iqueues(struct otx2_cptlfs_info *lfs) { int slot; - for (slot = 0; slot < lfs->lfs_num; slot++) + for (slot = 0; slot < lfs->lfs_num; slot++) { otx2_cptlf_do_disable_iqueue(&lfs->lf[slot]); + otx2_cpt_lf_reset_msg(lfs, lfs->global_slot + slot); + } } static inline void otx2_cptlf_set_iqueue_enq(struct otx2_cptlf_info *lf, diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c index 390ed146d309..a6f16438bd4a 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c @@ -201,8 +201,8 @@ static int handle_msg_rx_inline_ipsec_lf_cfg(struct otx2_cptpf_dev *cptpf, struct mbox_msghdr *req) { struct otx2_cpt_rx_inline_lf_cfg *cfg_req; + int num_lfs = 1, ret; u8 egrp; - int ret; cfg_req = (struct otx2_cpt_rx_inline_lf_cfg *)req; if (cptpf->lfs.lfs_num) { @@ -223,8 +223,9 @@ static int handle_msg_rx_inline_ipsec_lf_cfg(struct otx2_cptpf_dev *cptpf, otx2_cptlf_set_dev_info(&cptpf->lfs, cptpf->pdev, cptpf->reg_base, &cptpf->afpf_mbox, BLKADDR_CPT0); + cptpf->lfs.global_slot = 0; ret = otx2_cptlf_init(&cptpf->lfs, 1 << egrp, OTX2_CPT_QUEUE_HI_PRIO, - 1); + num_lfs); if (ret) { dev_err(&cptpf->pdev->dev, "LF configuration failed for RX inline ipsec.\n"); @@ -236,8 +237,9 @@ static int handle_msg_rx_inline_ipsec_lf_cfg(struct otx2_cptpf_dev *cptpf, otx2_cptlf_set_dev_info(&cptpf->cpt1_lfs, cptpf->pdev, cptpf->reg_base, &cptpf->afpf_mbox, BLKADDR_CPT1); + cptpf->cpt1_lfs.global_slot = num_lfs; ret = otx2_cptlf_init(&cptpf->cpt1_lfs, 1 << egrp, - OTX2_CPT_QUEUE_HI_PRIO, 1); + OTX2_CPT_QUEUE_HI_PRIO, num_lfs); if (ret) { dev_err(&cptpf->pdev->dev, "LF configuration failed for RX inline ipsec.\n"); @@ -449,6 +451,7 @@ static void process_afpf_mbox_msg(struct otx2_cptpf_dev *cptpf, break; case MBOX_MSG_CPT_INLINE_IPSEC_CFG: case MBOX_MSG_NIX_INLINE_IPSEC_CFG: + case MBOX_MSG_CPT_LF_RESET: break; default: diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptvf_mbox.c b/drivers/crypto/marvell/octeontx2/otx2_cptvf_mbox.c index f68da1d08fdf..d9fa5f6e204d 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptvf_mbox.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptvf_mbox.c @@ -133,6 +133,8 @@ static void process_pfvf_mbox_mbox_msg(struct otx2_cptvf_dev *cptvf, memcpy(cptvf->eng_caps, eng_caps->eng_caps, sizeof(cptvf->eng_caps)); break; + case MBOX_MSG_CPT_LF_RESET: + break; default: dev_err(&cptvf->pdev->dev, "Unsupported msg %d received.\n", msg->id); From patchwork Mon Dec 11 07:19:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srujana Challa X-Patchwork-Id: 752627 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="PJNLb7RF" Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 19DF6D8; Sun, 10 Dec 2023 23:20:13 -0800 (PST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BAMu0AE008719; Sun, 10 Dec 2023 23:20:05 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=FynVzIlTChxeB4gIwk9DFNX0ycRoF18OMnfERKMfJqc=; b=PJN Lb7RF769u1KeM+oOiKncnEBtO5UhZ2VX+gjTyH82D6mtJw/s/IfOFDyS5bo+Nfyh ILkoiN1vmSd0dyi41ujxm3tNYIRYVygnxNDGR6nUaxUFT84ezaj5JdShuKoVanVA tej/gRu3hlO/h3ht8wiN3rvDL3gyiZMkk+xY/seqjGAnV+8GopDvF1YRQrH08cCn 2g/YKlUXw0HC1g+hMv+8Xr/sDwQgyy1hJX1h4pG5tUZ02aJANOFl70+HcA0oTAQ9 n/ctFIfqvg+bYzemy+tc3YqxT3+1nvlYG8saxPMRvveby7/MpXbSnxriJ6AMHFE+ 4Y8YFDYGM47D1zGaPKQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3uw8xqa9f2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 10 Dec 2023 23:20:04 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Sun, 10 Dec 2023 23:20:03 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Sun, 10 Dec 2023 23:20:03 -0800 Received: from localhost.localdomain (unknown [10.28.36.175]) by maili.marvell.com (Postfix) with ESMTP id A375D3F709C; Sun, 10 Dec 2023 23:19:57 -0800 (PST) From: Srujana Challa To: , , CC: , , , , , , , , , , , , , , , , Subject: [PATCH net-next v1 07/10] octeontx2-af: update CPT inbound inline IPsec mailbox Date: Mon, 11 Dec 2023 12:49:10 +0530 Message-ID: <20231211071913.151225-8-schalla@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231211071913.151225-1-schalla@marvell.com> References: <20231211071913.151225-1-schalla@marvell.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: zj05TOVakhvPFsW3RJSis0GqnVD6G0fN X-Proofpoint-ORIG-GUID: zj05TOVakhvPFsW3RJSis0GqnVD6G0fN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 Updates CPT inbound inline IPsec configure mailbox to take CPT credit threshold and bpid, which are introduced in CN10KB. Signed-off-by: Srujana Challa --- drivers/crypto/marvell/octeontx2/otx2_cpt_common.h | 2 ++ drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c | 2 ++ drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c | 6 +++++- 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h index f5e3392b9b19..db19dba4f3d9 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h @@ -56,6 +56,8 @@ struct otx2_cpt_rx_inline_lf_cfg { u16 param2; u16 opcode; u32 credit; + u32 credit_th; + u16 bpid; u32 reserved; }; diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c index a6f16438bd4a..bbabb57b4665 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c @@ -171,6 +171,8 @@ static int rx_inline_ipsec_lf_cfg(struct otx2_cptpf_dev *cptpf, u8 egrp, nix_req->hdr.id = MBOX_MSG_NIX_INLINE_IPSEC_CFG; nix_req->hdr.sig = OTX2_MBOX_REQ_SIG; nix_req->enable = 1; + nix_req->credit_th = req->credit_th; + nix_req->bpid = req->bpid; if (!req->credit || req->credit > OTX2_CPT_INST_QLEN_MSGS) nix_req->cpt_credit = OTX2_CPT_INST_QLEN_MSGS - 1; else diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c index e319aa1ff119..5c9484646172 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c @@ -16,7 +16,11 @@ #define LOADFVC_MAJOR_OP 0x01 #define LOADFVC_MINOR_OP 0x08 -#define CTX_FLUSH_TIMER_CNT 0xFFFFFF +/* + * Interval to flush dirty data for next CTX entry. The interval is measured + * in increments of 10ns(interval time = CTX_FLUSH_TIMER_COUNT * 10ns). + */ +#define CTX_FLUSH_TIMER_CNT 0x2FAF0 struct fw_info_t { struct list_head ucodes; From patchwork Mon Dec 11 07:19:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srujana Challa X-Patchwork-Id: 753209 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="McTLtrhP" Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F0B3D8E; Sun, 10 Dec 2023 23:20:21 -0800 (PST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BALjive014653; Sun, 10 Dec 2023 23:20:12 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=lf+QG2KuLkmQmY+ZfBeLAXeQGzDdc5Fv2MxMinPcCDc=; b=McT LtrhPt5sZ7Kp2zuc4Wwc05yOje8fng9PkpHly/7m4zFd3yZrcblMKUfnFfH0XTtL pJACswq4OMjhCdks9soujLd0PPAS0zR+9+WbJiVToeBf/BauaxV8PvFUrlRJ17s0 k+XT833nay7IYN20k4RuDR1dg68Ek+3/pXAp0fDgpQ0co3xCx9TDg8kNOcuM/eQO 0S3j8pHNNXtsMApNE2mjF/9kdDvcAQvLbZNxe4l7aqW7/buHxLleyxTgKXqgQc8b oQaMKRpse7yi9rFvVUgJSY23Vw+BU8ChK1zJuQsbsWQqiGI/+kx8T7J1xkhxTb+A QWU4auau8yCONc3BGEQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3uvrmjkmm1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 10 Dec 2023 23:20:11 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Sun, 10 Dec 2023 23:20:09 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Sun, 10 Dec 2023 23:20:09 -0800 Received: from localhost.localdomain (unknown [10.28.36.175]) by maili.marvell.com (Postfix) with ESMTP id CA8383F70AC; Sun, 10 Dec 2023 23:20:03 -0800 (PST) From: Srujana Challa To: , , CC: , , , , , , , , , , , , , , , , Subject: [PATCH net-next v1 08/10] crypto: octeontx2: add ctx_val workaround Date: Mon, 11 Dec 2023 12:49:11 +0530 Message-ID: <20231211071913.151225-9-schalla@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231211071913.151225-1-schalla@marvell.com> References: <20231211071913.151225-1-schalla@marvell.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: HZbv6a9lWzPU-jjNky2mmlzt_L_GNHUe X-Proofpoint-ORIG-GUID: HZbv6a9lWzPU-jjNky2mmlzt_L_GNHUe X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 HW has a errata that CPT HW may hit an issue, while processing CPT instructions with CTX_VAL set and CTX_VAL not set. So, this patch adds the code to always set the CTX_VAL as a workaround. Signed-off-by: Srujana Challa --- drivers/crypto/marvell/octeontx2/cn10k_cpt.c | 70 +++++++++++++++++++ drivers/crypto/marvell/octeontx2/cn10k_cpt.h | 26 +++++++ .../marvell/octeontx2/otx2_cpt_hw_types.h | 2 + .../marvell/octeontx2/otx2_cpt_reqmgr.h | 2 + .../marvell/octeontx2/otx2_cptvf_algs.c | 31 ++++++++ .../marvell/octeontx2/otx2_cptvf_algs.h | 5 ++ .../marvell/octeontx2/otx2_cptvf_reqmgr.c | 2 +- 7 files changed, 137 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/marvell/octeontx2/cn10k_cpt.c b/drivers/crypto/marvell/octeontx2/cn10k_cpt.c index b23ae3a020e0..0ec9a3b440da 100644 --- a/drivers/crypto/marvell/octeontx2/cn10k_cpt.c +++ b/drivers/crypto/marvell/octeontx2/cn10k_cpt.c @@ -96,6 +96,76 @@ int cn10k_cptvf_lmtst_init(struct otx2_cptvf_dev *cptvf) } EXPORT_SYMBOL_NS_GPL(cn10k_cptvf_lmtst_init, CRYPTO_DEV_OCTEONTX2_CPT); +void cn10k_cpt_hw_ctx_clear(struct pci_dev *pdev, + struct cn10k_cpt_errata_ctx *er_ctx) +{ + u64 cptr_dma; + + if (!is_dev_cn10ka_ax(pdev)) + return; + + cptr_dma = er_ctx->cptr_dma & ~(BIT_ULL(60)); + cn10k_cpt_ctx_flush(pdev, cptr_dma, true); + dma_unmap_single(&pdev->dev, cptr_dma, CN10K_CPT_HW_CTX_SIZE, + DMA_BIDIRECTIONAL); + kfree(er_ctx->hw_ctx); +} +EXPORT_SYMBOL_NS_GPL(cn10k_cpt_hw_ctx_clear, CRYPTO_DEV_OCTEONTX2_CPT); + +void cn10k_cpt_hw_ctx_set(union cn10k_cpt_hw_ctx *hctx, u16 ctx_sz) +{ + hctx->w0.aop_valid = 1; + hctx->w0.ctx_hdr_sz = 0; + hctx->w0.ctx_sz = ctx_sz; + hctx->w0.ctx_push_sz = 1; +} +EXPORT_SYMBOL_NS_GPL(cn10k_cpt_hw_ctx_set, CRYPTO_DEV_OCTEONTX2_CPT); + +int cn10k_cpt_hw_ctx_init(struct pci_dev *pdev, + struct cn10k_cpt_errata_ctx *er_ctx) +{ + union cn10k_cpt_hw_ctx *hctx; + u64 cptr_dma; + + er_ctx->cptr_dma = 0; + er_ctx->hw_ctx = NULL; + + if (!is_dev_cn10ka_ax(pdev)) + return 0; + + hctx = kmalloc(CN10K_CPT_HW_CTX_SIZE, GFP_KERNEL); + if (unlikely(!hctx)) + return -ENOMEM; + cptr_dma = dma_map_single(&pdev->dev, hctx, CN10K_CPT_HW_CTX_SIZE, + DMA_BIDIRECTIONAL); + + cn10k_cpt_hw_ctx_set(hctx, 1); + er_ctx->hw_ctx = hctx; + er_ctx->cptr_dma = cptr_dma | BIT_ULL(60); + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cn10k_cpt_hw_ctx_init, CRYPTO_DEV_OCTEONTX2_CPT); + +void cn10k_cpt_ctx_flush(struct pci_dev *pdev, u64 cptr, bool inval) +{ + struct otx2_cptvf_dev *cptvf = pci_get_drvdata(pdev); + struct otx2_cptlfs_info *lfs = &cptvf->lfs; + u64 reg; + + reg = (uintptr_t)cptr >> 7; + if (inval) + reg = reg | BIT_ULL(46); + + otx2_cpt_write64(lfs->reg_base, lfs->blkaddr, lfs->lf[0].slot, + OTX2_CPT_LF_CTX_FLUSH, reg); + /* Make sure that the FLUSH operation is complete */ + wmb(); + otx2_cpt_read64(lfs->reg_base, lfs->blkaddr, lfs->lf[0].slot, + OTX2_CPT_LF_CTX_ERR); +} +EXPORT_SYMBOL_NS_GPL(cn10k_cpt_ctx_flush, CRYPTO_DEV_OCTEONTX2_CPT); + int cptvf_hw_ops_get(struct otx2_cptvf_dev *cptvf) { if (!test_bit(CN10K_LMTST, &cptvf->cap_flag)) { diff --git a/drivers/crypto/marvell/octeontx2/cn10k_cpt.h b/drivers/crypto/marvell/octeontx2/cn10k_cpt.h index 0f714ee564f5..0de29d2c7b0f 100644 --- a/drivers/crypto/marvell/octeontx2/cn10k_cpt.h +++ b/drivers/crypto/marvell/octeontx2/cn10k_cpt.h @@ -8,6 +8,26 @@ #include "otx2_cptpf.h" #include "otx2_cptvf.h" +#define CN10K_CPT_HW_CTX_SIZE 256 + +union cn10k_cpt_hw_ctx { + u64 u; + struct { + u64 reserved_0_47:48; + u64 ctx_push_sz:7; + u64 reserved_55:1; + u64 ctx_hdr_sz:2; + u64 aop_valid:1; + u64 reserved_59:1; + u64 ctx_sz:4; + } w0; +}; + +struct cn10k_cpt_errata_ctx { + union cn10k_cpt_hw_ctx *hw_ctx; + u64 cptr_dma; +}; + static inline u8 cn10k_cpt_get_compcode(union otx2_cpt_res_s *result) { return ((struct cn10k_cpt_res_s *)result)->compcode; @@ -30,6 +50,12 @@ static inline u8 otx2_cpt_get_uc_compcode(union otx2_cpt_res_s *result) int cn10k_cptpf_lmtst_init(struct otx2_cptpf_dev *cptpf); int cn10k_cptvf_lmtst_init(struct otx2_cptvf_dev *cptvf); +void cn10k_cpt_ctx_flush(struct pci_dev *pdev, u64 cptr, bool inval); +int cn10k_cpt_hw_ctx_init(struct pci_dev *pdev, + struct cn10k_cpt_errata_ctx *er_ctx); +void cn10k_cpt_hw_ctx_clear(struct pci_dev *pdev, + struct cn10k_cpt_errata_ctx *er_ctx); +void cn10k_cpt_hw_ctx_set(union cn10k_cpt_hw_ctx *hctx, u16 ctx_sz); int cptvf_hw_ops_get(struct otx2_cptvf_dev *cptvf); #endif /* __CN10K_CPTLF_H */ diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_hw_types.h b/drivers/crypto/marvell/octeontx2/otx2_cpt_hw_types.h index 756aee0c2b05..06bcf49ee379 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_hw_types.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_hw_types.h @@ -102,6 +102,8 @@ #define OTX2_CPT_LF_Q_INST_PTR (0x110) #define OTX2_CPT_LF_Q_GRP_PTR (0x120) #define OTX2_CPT_LF_NQX(a) (0x400 | (a) << 3) +#define OTX2_CPT_LF_CTX_FLUSH (0x510) +#define OTX2_CPT_LF_CTX_ERR (0x520) #define OTX2_CPT_RVU_FUNC_BLKADDR_SHIFT 20 /* LMT LF registers */ #define OTX2_CPT_LMT_LFBASE BIT_ULL(OTX2_CPT_RVU_FUNC_BLKADDR_SHIFT) diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_reqmgr.h b/drivers/crypto/marvell/octeontx2/otx2_cpt_reqmgr.h index b0f426531a42..58e02d4fa0f2 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_reqmgr.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_reqmgr.h @@ -40,6 +40,8 @@ struct otx2_cptvf_request { u32 param2; u16 dlen; union otx2_cpt_opcode opcode; + dma_addr_t cptr_dma; + void *cptr; }; /* diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptvf_algs.c b/drivers/crypto/marvell/octeontx2/otx2_cptvf_algs.c index e27ddd3c4e55..1604fc58dc13 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptvf_algs.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptvf_algs.c @@ -17,6 +17,7 @@ #include "otx2_cptvf.h" #include "otx2_cptvf_algs.h" #include "otx2_cpt_reqmgr.h" +#include "cn10k_cpt.h" /* Size of salt in AES GCM mode */ #define AES_GCM_SALT_SIZE 4 @@ -384,6 +385,9 @@ static inline int cpt_enc_dec(struct skcipher_request *req, u32 enc) req_info->is_trunc_hmac = false; req_info->ctrl.s.grp = otx2_cpt_get_kcrypto_eng_grp_num(pdev); + req_info->req.cptr = ctx->er_ctx.hw_ctx; + req_info->req.cptr_dma = ctx->er_ctx.cptr_dma; + /* * We perform an asynchronous send and once * the request is completed the driver would @@ -530,6 +534,8 @@ static int otx2_cpt_enc_dec_init(struct crypto_skcipher *stfm) struct otx2_cpt_enc_ctx *ctx = crypto_skcipher_ctx(stfm); struct crypto_tfm *tfm = crypto_skcipher_tfm(stfm); struct crypto_alg *alg = tfm->__crt_alg; + struct pci_dev *pdev; + int ret, cpu_num; memset(ctx, 0, sizeof(*ctx)); /* @@ -541,6 +547,15 @@ static int otx2_cpt_enc_dec_init(struct crypto_skcipher *stfm) stfm, sizeof(struct otx2_cpt_req_ctx) + sizeof(struct skcipher_request)); + ret = get_se_device(&pdev, &cpu_num); + if (ret) + return ret; + + ctx->pdev = pdev; + ret = cn10k_cpt_hw_ctx_init(pdev, &ctx->er_ctx); + if (ret) + return ret; + return cpt_skcipher_fallback_init(ctx, alg); } @@ -552,6 +567,7 @@ static void otx2_cpt_skcipher_exit(struct crypto_skcipher *tfm) crypto_free_skcipher(ctx->fbk_cipher); ctx->fbk_cipher = NULL; } + cn10k_cpt_hw_ctx_clear(ctx->pdev, &ctx->er_ctx); } static int cpt_aead_fallback_init(struct otx2_cpt_aead_ctx *ctx, @@ -576,6 +592,8 @@ static int cpt_aead_init(struct crypto_aead *atfm, u8 cipher_type, u8 mac_type) struct otx2_cpt_aead_ctx *ctx = crypto_aead_ctx_dma(atfm); struct crypto_tfm *tfm = crypto_aead_tfm(atfm); struct crypto_alg *alg = tfm->__crt_alg; + struct pci_dev *pdev; + int ret, cpu_num; ctx->cipher_type = cipher_type; ctx->mac_type = mac_type; @@ -632,6 +650,15 @@ static int cpt_aead_init(struct crypto_aead *atfm, u8 cipher_type, u8 mac_type) } crypto_aead_set_reqsize_dma(atfm, sizeof(struct otx2_cpt_req_ctx)); + ret = get_se_device(&pdev, &cpu_num); + if (ret) + return ret; + + ctx->pdev = pdev; + ret = cn10k_cpt_hw_ctx_init(pdev, &ctx->er_ctx); + if (ret) + return ret; + return cpt_aead_fallback_init(ctx, alg); } @@ -694,6 +721,7 @@ static void otx2_cpt_aead_exit(struct crypto_aead *tfm) crypto_free_aead(ctx->fbk_cipher); ctx->fbk_cipher = NULL; } + cn10k_cpt_hw_ctx_clear(ctx->pdev, &ctx->er_ctx); } static int otx2_cpt_aead_gcm_set_authsize(struct crypto_aead *tfm, @@ -1299,6 +1327,9 @@ static int cpt_aead_enc_dec(struct aead_request *req, u8 reg_type, u8 enc) req_info->is_enc = enc; req_info->is_trunc_hmac = false; + req_info->req.cptr = ctx->er_ctx.hw_ctx; + req_info->req.cptr_dma = ctx->er_ctx.cptr_dma; + switch (reg_type) { case OTX2_CPT_AEAD_ENC_DEC_REQ: status = create_aead_input_list(req, enc); diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptvf_algs.h b/drivers/crypto/marvell/octeontx2/otx2_cptvf_algs.h index f04184bd1744..d29f84f01cee 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptvf_algs.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cptvf_algs.h @@ -9,6 +9,7 @@ #include #include #include "otx2_cpt_common.h" +#include "cn10k_cpt.h" #define OTX2_CPT_MAX_ENC_KEY_SIZE 32 #define OTX2_CPT_MAX_HASH_KEY_SIZE 64 @@ -123,6 +124,8 @@ struct otx2_cpt_enc_ctx { u8 key_type; u8 enc_align_len; struct crypto_skcipher *fbk_cipher; + struct pci_dev *pdev; + struct cn10k_cpt_errata_ctx er_ctx; }; union otx2_cpt_offset_ctrl { @@ -161,6 +164,8 @@ struct otx2_cpt_aead_ctx { struct crypto_shash *hashalg; struct otx2_cpt_sdesc *sdesc; struct crypto_aead *fbk_cipher; + struct cn10k_cpt_errata_ctx er_ctx; + struct pci_dev *pdev; u8 *ipad; u8 *opad; u32 enc_key_len; diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptvf_reqmgr.c b/drivers/crypto/marvell/octeontx2/otx2_cptvf_reqmgr.c index 997a2eb60c66..5387c68f3c9d 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptvf_reqmgr.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptvf_reqmgr.c @@ -159,7 +159,7 @@ static int process_request(struct pci_dev *pdev, struct otx2_cpt_req_info *req, cpu_to_be64s(&iq_cmd.cmd.u); iq_cmd.dptr = info->dptr_baddr | info->gthr_sz << 60; iq_cmd.rptr = info->rptr_baddr | info->sctr_sz << 60; - iq_cmd.cptr.u = 0; + iq_cmd.cptr.s.cptr = cpt_req->cptr_dma; iq_cmd.cptr.s.grp = ctrl->s.grp; /* Fill in the CPT_INST_S type command for HW interpretation */ From patchwork Mon Dec 11 07:19:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srujana Challa X-Patchwork-Id: 752626 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="h1GfdTtA" Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95CE1D7; Sun, 10 Dec 2023 23:20:28 -0800 (PST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BAMrKVe012215; Sun, 10 Dec 2023 23:20:18 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=qWu10LIIjAUzPyjrQCtmutOXP4sApr3ruooGwJKgmoM=; b=h1G fdTtAtzGS07Yz1jL96MgWIQhzi3z0Fl/9FL8wYMMCAuhusMCWZWQqbblKIp980Ci GwlNc+lpPORDPLrR+/F99KhAVmXUfHr9RIlTMzfguB6FDQhCRSzpiqxtBroTZhKX r3BY8QOjVnan3373G2t2XG5GwlFpAiuSC1kr1VX+GF+an89hghjA4iLerg+MZ2ob kDfytTFN4xuUPz1Ip7cBmhWl3TSYKGaPOTH8HISeF8VOhW9e/IauCXaC8eLSj3c1 vnBlcHs4MNcIfw7JgCa1+I8Kq+SH6/x1dfVqcVBNiQD3Ijy9AYChKew9KkSJl2d8 jOUz2KB5Ute88WG2BJg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3uvrmjkmme-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 10 Dec 2023 23:20:18 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Sun, 10 Dec 2023 23:20:15 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Sun, 10 Dec 2023 23:20:15 -0800 Received: from localhost.localdomain (unknown [10.28.36.175]) by maili.marvell.com (Postfix) with ESMTP id 13FF73F70A7; Sun, 10 Dec 2023 23:20:09 -0800 (PST) From: Srujana Challa To: , , CC: , , , , , , , , , , , , , , , , Subject: [PATCH net-next v1 09/10] crypto/octeontx2: register error interrupts for inline cptlf Date: Mon, 11 Dec 2023 12:49:12 +0530 Message-ID: <20231211071913.151225-10-schalla@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231211071913.151225-1-schalla@marvell.com> References: <20231211071913.151225-1-schalla@marvell.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: 4Loi2JVzUgUBQuSoEvwiWKix7JDe6Rk3 X-Proofpoint-ORIG-GUID: 4Loi2JVzUgUBQuSoEvwiWKix7JDe6Rk3 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 From: Nithin Dabilpuram Register errors interrupts for inline cptlf attached to PF driver so that SMMU faults and other errors can be reported. Signed-off-by: Nithin Dabilpuram --- drivers/crypto/marvell/octeontx2/otx2_cptlf.c | 101 +++++++++++------- drivers/crypto/marvell/octeontx2/otx2_cptlf.h | 6 +- drivers/crypto/marvell/octeontx2/otx2_cptpf.h | 4 + .../marvell/octeontx2/otx2_cptpf_main.c | 17 ++- .../marvell/octeontx2/otx2_cptpf_mbox.c | 68 ++++++++++-- .../marvell/octeontx2/otx2_cptvf_main.c | 12 ++- 6 files changed, 153 insertions(+), 55 deletions(-) diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptlf.c b/drivers/crypto/marvell/octeontx2/otx2_cptlf.c index 6edd27ff8c4e..c3bafd9a11dd 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptlf.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptlf.c @@ -151,26 +151,14 @@ static void cptlf_set_misc_intrs(struct otx2_cptlfs_info *lfs, u8 enable) irq_misc.u); } -static void cptlf_enable_intrs(struct otx2_cptlfs_info *lfs) +static void cptlf_set_done_intrs(struct otx2_cptlfs_info *lfs, u8 enable) { + u64 reg = enable ? OTX2_CPT_LF_DONE_INT_ENA_W1S : + OTX2_CPT_LF_DONE_INT_ENA_W1C; int slot; - /* Enable done interrupts */ for (slot = 0; slot < lfs->lfs_num; slot++) - otx2_cpt_write64(lfs->reg_base, lfs->blkaddr, slot, - OTX2_CPT_LF_DONE_INT_ENA_W1S, 0x1); - /* Enable Misc interrupts */ - cptlf_set_misc_intrs(lfs, true); -} - -static void cptlf_disable_intrs(struct otx2_cptlfs_info *lfs) -{ - int slot; - - for (slot = 0; slot < lfs->lfs_num; slot++) - otx2_cpt_write64(lfs->reg_base, lfs->blkaddr, slot, - OTX2_CPT_LF_DONE_INT_ENA_W1C, 0x1); - cptlf_set_misc_intrs(lfs, false); + otx2_cpt_write64(lfs->reg_base, lfs->blkaddr, slot, reg, 0x1); } static inline int cptlf_read_done_cnt(struct otx2_cptlf_info *lf) @@ -257,24 +245,44 @@ static irqreturn_t cptlf_done_intr_handler(int irq, void *arg) return IRQ_HANDLED; } -void otx2_cptlf_unregister_interrupts(struct otx2_cptlfs_info *lfs) +void otx2_cptlf_unregister_misc_interrupts(struct otx2_cptlfs_info *lfs) { - int i, offs, vector; + int i, irq_offs, vector; + irq_offs = OTX2_CPT_LF_INT_VEC_E_MISC; for (i = 0; i < lfs->lfs_num; i++) { - for (offs = 0; offs < OTX2_CPT_LF_MSIX_VECTORS; offs++) { - if (!lfs->lf[i].is_irq_reg[offs]) - continue; + if (!lfs->lf[i].is_irq_reg[irq_offs]) + continue; - vector = pci_irq_vector(lfs->pdev, - lfs->lf[i].msix_offset + offs); - free_irq(vector, &lfs->lf[i]); - lfs->lf[i].is_irq_reg[offs] = false; - } + vector = pci_irq_vector(lfs->pdev, + lfs->lf[i].msix_offset + irq_offs); + free_irq(vector, &lfs->lf[i]); + lfs->lf[i].is_irq_reg[irq_offs] = false; + } + + cptlf_set_misc_intrs(lfs, false); +} +EXPORT_SYMBOL_NS_GPL(otx2_cptlf_unregister_misc_interrupts, + CRYPTO_DEV_OCTEONTX2_CPT); + +void otx2_cptlf_unregister_done_interrupts(struct otx2_cptlfs_info *lfs) +{ + int i, irq_offs, vector; + + irq_offs = OTX2_CPT_LF_INT_VEC_E_DONE; + for (i = 0; i < lfs->lfs_num; i++) { + if (!lfs->lf[i].is_irq_reg[irq_offs]) + continue; + + vector = pci_irq_vector(lfs->pdev, + lfs->lf[i].msix_offset + irq_offs); + free_irq(vector, &lfs->lf[i]); + lfs->lf[i].is_irq_reg[irq_offs] = false; } - cptlf_disable_intrs(lfs); + + cptlf_set_done_intrs(lfs, false); } -EXPORT_SYMBOL_NS_GPL(otx2_cptlf_unregister_interrupts, +EXPORT_SYMBOL_NS_GPL(otx2_cptlf_unregister_done_interrupts, CRYPTO_DEV_OCTEONTX2_CPT); static int cptlf_do_register_interrrupts(struct otx2_cptlfs_info *lfs, @@ -296,34 +304,53 @@ static int cptlf_do_register_interrrupts(struct otx2_cptlfs_info *lfs, return ret; } -int otx2_cptlf_register_interrupts(struct otx2_cptlfs_info *lfs) +int otx2_cptlf_register_misc_interrupts(struct otx2_cptlfs_info *lfs) { + bool is_cpt1 = (lfs->blkaddr == BLKADDR_CPT1); int irq_offs, ret, i; + irq_offs = OTX2_CPT_LF_INT_VEC_E_MISC; for (i = 0; i < lfs->lfs_num; i++) { - irq_offs = OTX2_CPT_LF_INT_VEC_E_MISC; - snprintf(lfs->lf[i].irq_name[irq_offs], 32, "CPTLF Misc%d", i); + snprintf(lfs->lf[i].irq_name[irq_offs], 32, "CPT%dLF Misc%d", + is_cpt1, i); ret = cptlf_do_register_interrrupts(lfs, i, irq_offs, cptlf_misc_intr_handler); if (ret) goto free_irq; + } + cptlf_set_misc_intrs(lfs, true); + return 0; - irq_offs = OTX2_CPT_LF_INT_VEC_E_DONE; - snprintf(lfs->lf[i].irq_name[irq_offs], 32, "OTX2_CPTLF Done%d", - i); +free_irq: + otx2_cptlf_unregister_misc_interrupts(lfs); + return ret; +} +EXPORT_SYMBOL_NS_GPL(otx2_cptlf_register_misc_interrupts, + CRYPTO_DEV_OCTEONTX2_CPT); + +int otx2_cptlf_register_done_interrupts(struct otx2_cptlfs_info *lfs) +{ + bool is_cpt1 = (lfs->blkaddr == BLKADDR_CPT1); + int irq_offs, ret, i; + + irq_offs = OTX2_CPT_LF_INT_VEC_E_DONE; + for (i = 0; i < lfs->lfs_num; i++) { + snprintf(lfs->lf[i].irq_name[irq_offs], 32, + "OTX2_CPT%dLF Done%d", is_cpt1, i); ret = cptlf_do_register_interrrupts(lfs, i, irq_offs, cptlf_done_intr_handler); if (ret) goto free_irq; } - cptlf_enable_intrs(lfs); + cptlf_set_done_intrs(lfs, true); return 0; free_irq: - otx2_cptlf_unregister_interrupts(lfs); + otx2_cptlf_unregister_done_interrupts(lfs); return ret; } -EXPORT_SYMBOL_NS_GPL(otx2_cptlf_register_interrupts, CRYPTO_DEV_OCTEONTX2_CPT); +EXPORT_SYMBOL_NS_GPL(otx2_cptlf_register_done_interrupts, + CRYPTO_DEV_OCTEONTX2_CPT); void otx2_cptlf_free_irqs_affinity(struct otx2_cptlfs_info *lfs) { diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptlf.h b/drivers/crypto/marvell/octeontx2/otx2_cptlf.h index 8d6f10f67ac0..315a89a9bcb6 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptlf.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cptlf.h @@ -409,8 +409,10 @@ static inline void otx2_cptlf_set_dev_info(struct otx2_cptlfs_info *lfs, int otx2_cptlf_init(struct otx2_cptlfs_info *lfs, u8 eng_grp_msk, int pri, int lfs_num); void otx2_cptlf_shutdown(struct otx2_cptlfs_info *lfs); -int otx2_cptlf_register_interrupts(struct otx2_cptlfs_info *lfs); -void otx2_cptlf_unregister_interrupts(struct otx2_cptlfs_info *lfs); +int otx2_cptlf_register_misc_interrupts(struct otx2_cptlfs_info *lfs); +int otx2_cptlf_register_done_interrupts(struct otx2_cptlfs_info *lfs); +void otx2_cptlf_unregister_misc_interrupts(struct otx2_cptlfs_info *lfs); +void otx2_cptlf_unregister_done_interrupts(struct otx2_cptlfs_info *lfs); void otx2_cptlf_free_irqs_affinity(struct otx2_cptlfs_info *lfs); int otx2_cptlf_set_irqs_affinity(struct otx2_cptlfs_info *lfs); diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf.h b/drivers/crypto/marvell/octeontx2/otx2_cptpf.h index a209ec5af381..e5859a1e1c60 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf.h @@ -71,4 +71,8 @@ void otx2_cptpf_afpf_mbox_up_handler(struct work_struct *work); irqreturn_t otx2_cptpf_vfpf_mbox_intr(int irq, void *arg); void otx2_cptpf_vfpf_mbox_handler(struct work_struct *work); +int otx2_inline_cptlf_setup(struct otx2_cptpf_dev *cptpf, + struct otx2_cptlfs_info *lfs, u8 egrp, int num_lfs); +void otx2_inline_cptlf_cleanup(struct otx2_cptlfs_info *lfs); + #endif /* __OTX2_CPTPF_H */ diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c index 7d44b54659bf..79afa3a451a7 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c @@ -725,7 +725,7 @@ static int otx2_cptpf_probe(struct pci_dev *pdev, { struct device *dev = &pdev->dev; struct otx2_cptpf_dev *cptpf; - int err; + int err, num_vec; cptpf = devm_kzalloc(dev, sizeof(*cptpf), GFP_KERNEL); if (!cptpf) @@ -760,8 +760,11 @@ static int otx2_cptpf_probe(struct pci_dev *pdev, if (err) goto clear_drvdata; - err = pci_alloc_irq_vectors(pdev, RVU_PF_INT_VEC_CNT, - RVU_PF_INT_VEC_CNT, PCI_IRQ_MSIX); + num_vec = pci_msix_vec_count(cptpf->pdev); + if (num_vec <= 0) + goto clear_drvdata; + + err = pci_alloc_irq_vectors(pdev, num_vec, num_vec, PCI_IRQ_MSIX); if (err < 0) { dev_err(dev, "Request for %d msix vectors failed\n", RVU_PF_INT_VEC_CNT); @@ -825,6 +828,14 @@ static void otx2_cptpf_remove(struct pci_dev *pdev) cptpf_sriov_disable(pdev); otx2_cpt_unregister_dl(cptpf); + + /* Cleanup Inline CPT LF's if attached */ + if (cptpf->lfs.lfs_num) + otx2_inline_cptlf_cleanup(&cptpf->lfs); + + if (cptpf->cpt1_lfs.lfs_num) + otx2_inline_cptlf_cleanup(&cptpf->cpt1_lfs); + /* Delete sysfs entry created for kernel VF limits */ sysfs_remove_group(&pdev->dev.kobj, &cptpf_sysfs_group); /* Cleanup engine groups */ diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c index bbabb57b4665..5d63763db2af 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c @@ -199,6 +199,47 @@ static int rx_inline_ipsec_lf_cfg(struct otx2_cptpf_dev *cptpf, u8 egrp, return send_inline_ipsec_inbound_msg(cptpf, req->sso_pf_func, 0); } +int +otx2_inline_cptlf_setup(struct otx2_cptpf_dev *cptpf, + struct otx2_cptlfs_info *lfs, u8 egrp, int num_lfs) +{ + int ret; + + ret = otx2_cptlf_init(lfs, 1 << egrp, OTX2_CPT_QUEUE_HI_PRIO, 1); + if (ret) { + dev_err(&cptpf->pdev->dev, + "LF configuration failed for RX inline ipsec.\n"); + return ret; + } + + /* Get msix offsets for attached LFs */ + ret = otx2_cpt_msix_offset_msg(lfs); + if (ret) + goto cleanup_lf; + + /* Register for CPT LF Misc interrupts */ + ret = otx2_cptlf_register_misc_interrupts(lfs); + if (ret) + goto free_irq; + + return 0; +free_irq: + otx2_cptlf_unregister_misc_interrupts(lfs); +cleanup_lf: + otx2_cptlf_shutdown(lfs); + return ret; +} + +void +otx2_inline_cptlf_cleanup(struct otx2_cptlfs_info *lfs) +{ + /* Unregister misc interrupt */ + otx2_cptlf_unregister_misc_interrupts(lfs); + + /* Cleanup LFs */ + otx2_cptlf_shutdown(lfs); +} + static int handle_msg_rx_inline_ipsec_lf_cfg(struct otx2_cptpf_dev *cptpf, struct mbox_msghdr *req) { @@ -226,11 +267,9 @@ static int handle_msg_rx_inline_ipsec_lf_cfg(struct otx2_cptpf_dev *cptpf, otx2_cptlf_set_dev_info(&cptpf->lfs, cptpf->pdev, cptpf->reg_base, &cptpf->afpf_mbox, BLKADDR_CPT0); cptpf->lfs.global_slot = 0; - ret = otx2_cptlf_init(&cptpf->lfs, 1 << egrp, OTX2_CPT_QUEUE_HI_PRIO, - num_lfs); + ret = otx2_inline_cptlf_setup(cptpf, &cptpf->lfs, egrp, num_lfs); if (ret) { - dev_err(&cptpf->pdev->dev, - "LF configuration failed for RX inline ipsec.\n"); + dev_err(&cptpf->pdev->dev, "Inline-Ipsec CPT0 LF setup failed.\n"); return ret; } @@ -240,11 +279,10 @@ static int handle_msg_rx_inline_ipsec_lf_cfg(struct otx2_cptpf_dev *cptpf, cptpf->reg_base, &cptpf->afpf_mbox, BLKADDR_CPT1); cptpf->cpt1_lfs.global_slot = num_lfs; - ret = otx2_cptlf_init(&cptpf->cpt1_lfs, 1 << egrp, - OTX2_CPT_QUEUE_HI_PRIO, num_lfs); + ret = otx2_inline_cptlf_setup(cptpf, &cptpf->cpt1_lfs, egrp, + num_lfs); if (ret) { - dev_err(&cptpf->pdev->dev, - "LF configuration failed for RX inline ipsec.\n"); + dev_err(&cptpf->pdev->dev, "Inline CPT1 LF setup failed.\n"); goto lf_cleanup; } cptpf->rsrc_req_blkaddr = 0; @@ -257,9 +295,9 @@ static int handle_msg_rx_inline_ipsec_lf_cfg(struct otx2_cptpf_dev *cptpf, return 0; lf1_cleanup: - otx2_cptlf_shutdown(&cptpf->cpt1_lfs); + otx2_inline_cptlf_cleanup(&cptpf->cpt1_lfs); lf_cleanup: - otx2_cptlf_shutdown(&cptpf->lfs); + otx2_inline_cptlf_cleanup(&cptpf->lfs); return ret; } @@ -414,6 +452,8 @@ static void process_afpf_mbox_msg(struct otx2_cptpf_dev *cptpf, struct otx2_cptlfs_info *lfs = &cptpf->lfs; struct device *dev = &cptpf->pdev->dev; struct cpt_rd_wr_reg_msg *rsp_rd_wr; + struct msix_offset_rsp *rsp_msix; + int i; if (msg->id >= MBOX_MSG_MAX) { dev_err(dev, "MBOX msg with unknown ID %d\n", msg->id); @@ -432,6 +472,14 @@ static void process_afpf_mbox_msg(struct otx2_cptpf_dev *cptpf, cptpf->pf_id = (msg->pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK; break; + case MBOX_MSG_MSIX_OFFSET: + rsp_msix = (struct msix_offset_rsp *) msg; + for (i = 0; i < rsp_msix->cptlfs; i++) + lfs->lf[i].msix_offset = rsp_msix->cptlf_msixoff[i]; + + for (i = 0; i < rsp_msix->cpt1_lfs; i++) + lfs->lf[i].msix_offset = rsp_msix->cpt1_lf_msixoff[i]; + break; case MBOX_MSG_CPT_RD_WR_REGISTER: rsp_rd_wr = (struct cpt_rd_wr_reg_msg *)msg; if (msg->rc) { diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptvf_main.c b/drivers/crypto/marvell/octeontx2/otx2_cptvf_main.c index 5d1e11135c17..83c3fd8e2e57 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptvf_main.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptvf_main.c @@ -246,7 +246,8 @@ static void cptvf_lf_shutdown(struct otx2_cptlfs_info *lfs) /* Unregister crypto algorithms */ otx2_cpt_crypto_exit(lfs->pdev, THIS_MODULE); /* Unregister LFs interrupts */ - otx2_cptlf_unregister_interrupts(lfs); + otx2_cptlf_unregister_misc_interrupts(lfs); + otx2_cptlf_unregister_done_interrupts(lfs); /* Cleanup LFs software side */ lf_sw_cleanup(lfs); /* Send request to detach LFs */ @@ -298,7 +299,11 @@ static int cptvf_lf_init(struct otx2_cptvf_dev *cptvf) goto cleanup_lf; /* Register LFs interrupts */ - ret = otx2_cptlf_register_interrupts(lfs); + ret = otx2_cptlf_register_misc_interrupts(lfs); + if (ret) + goto cleanup_lf_sw; + + ret = otx2_cptlf_register_done_interrupts(lfs); if (ret) goto cleanup_lf_sw; @@ -319,7 +324,8 @@ static int cptvf_lf_init(struct otx2_cptvf_dev *cptvf) disable_irqs: otx2_cptlf_free_irqs_affinity(lfs); unregister_intr: - otx2_cptlf_unregister_interrupts(lfs); + otx2_cptlf_unregister_misc_interrupts(lfs); + otx2_cptlf_unregister_done_interrupts(lfs); cleanup_lf_sw: lf_sw_cleanup(lfs); cleanup_lf: From patchwork Mon Dec 11 07:19:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srujana Challa X-Patchwork-Id: 753208 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="bX99QMAU" Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 84178D7; Sun, 10 Dec 2023 23:20:35 -0800 (PST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BANVUVQ031546; Sun, 10 Dec 2023 23:20:24 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=yNYAIrwTD20qL7CUtnmLkE9u1cyS+7sEopBVngAZgiY=; b=bX9 9QMAUgwNt/aXiHYQ2cZbrQQEH92MAKui7wkSICJOQZWclwKs0Ux6WjjMAgkpDCUP h07pltm7n7ktzd1x6/Dlc7GYUo6b6tduF16G+Hf9fKlCjKHbDEfBAZ2BRs0EYwuw J/AgY0lHbi89rrxUNLrPKrWw+H6LVX4YcsleUqnX1S1chdndhUgPp4YMx5ssWRtV oTA3fbKnU9gQQeHsEKzRVpDzP0stvsJLTNY+Rq8y4w5iDJcieZKYX7g0s0PNa83k bBrW8U1rCZ7obsqWZMnBIyhTfM5KD0g64zN5HPtGa3dkXDqee+b/WRHOiZfcU+s5 +FQNpPJJzUNQ5UCEHJA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3uvrmjkmmv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 10 Dec 2023 23:20:24 -0800 (PST) Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Sun, 10 Dec 2023 23:20:21 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Sun, 10 Dec 2023 23:20:21 -0800 Received: from localhost.localdomain (unknown [10.28.36.175]) by maili.marvell.com (Postfix) with ESMTP id 3FDF43F70AF; Sun, 10 Dec 2023 23:20:15 -0800 (PST) From: Srujana Challa To: , , CC: , , , , , , , , , , , , , , , , Subject: [PATCH net-next v1 10/10] crypto: octeontx2: support setting ctx ilen for inline CPT LF Date: Mon, 11 Dec 2023 12:49:13 +0530 Message-ID: <20231211071913.151225-11-schalla@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231211071913.151225-1-schalla@marvell.com> References: <20231211071913.151225-1-schalla@marvell.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: F8B28BQLOLFHqX0OeVJOCAvfC_6Z_syF X-Proofpoint-ORIG-GUID: F8B28BQLOLFHqX0OeVJOCAvfC_6Z_syF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 From: Nithin Dabilpuram Provide an option in Inline IPsec configure mailbox to configure the CPT_AF_LFX_CTL:CTX_ILEN for inline CPT LF attached to CPT RVU PF. This is needed to set the ctx ilen to size of inbound SA for HW errata IPBUCPT-38756. Not setting this would lead to new context's not being fetched. Also set FLR_FLUSH in CPT_LF_CTX_CTL for CPT LF's as workaround for same errata. Signed-off-by: Nithin Dabilpuram --- .../marvell/octeontx2/otx2_cpt_common.h | 2 ++ .../marvell/octeontx2/otx2_cpt_hw_types.h | 4 ++- drivers/crypto/marvell/octeontx2/otx2_cptlf.c | 32 +++++++++++++++++++ drivers/crypto/marvell/octeontx2/otx2_cptlf.h | 19 +++++++++++ .../marvell/octeontx2/otx2_cptpf_mbox.c | 5 +++ 5 files changed, 61 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h index db19dba4f3d9..26c7d1bd6ef0 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h @@ -59,6 +59,8 @@ struct otx2_cpt_rx_inline_lf_cfg { u32 credit_th; u16 bpid; u32 reserved; + u8 ctx_ilen_valid : 1; + u8 ctx_ilen : 7; }; /* diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_hw_types.h b/drivers/crypto/marvell/octeontx2/otx2_cpt_hw_types.h index 06bcf49ee379..7e746a4def86 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cpt_hw_types.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_hw_types.h @@ -102,6 +102,7 @@ #define OTX2_CPT_LF_Q_INST_PTR (0x110) #define OTX2_CPT_LF_Q_GRP_PTR (0x120) #define OTX2_CPT_LF_NQX(a) (0x400 | (a) << 3) +#define OTX2_CPT_LF_CTX_CTL (0x500) #define OTX2_CPT_LF_CTX_FLUSH (0x510) #define OTX2_CPT_LF_CTX_ERR (0x520) #define OTX2_CPT_RVU_FUNC_BLKADDR_SHIFT 20 @@ -472,7 +473,8 @@ union otx2_cptx_af_lf_ctrl { u64 cont_err:1; u64 reserved_11_15:5; u64 nixtx_en:1; - u64 reserved_17_47:31; + u64 ctx_ilen:3; + u64 reserved_17_47:28; u64 grp:8; u64 reserved_56_63:8; } s; diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptlf.c b/drivers/crypto/marvell/octeontx2/otx2_cptlf.c index c3bafd9a11dd..777474d48ae4 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptlf.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptlf.c @@ -106,6 +106,32 @@ static int cptlf_set_grp_and_pri(struct otx2_cptlfs_info *lfs, return ret; } +static int cptlf_set_ctx_ilen(struct otx2_cptlfs_info *lfs, int ctx_ilen) +{ + union otx2_cptx_af_lf_ctrl lf_ctrl; + struct otx2_cptlf_info *lf; + int slot, ret = 0; + + for (slot = 0; slot < lfs->lfs_num; slot++) { + lf = &lfs->lf[slot]; + + ret = otx2_cpt_read_af_reg(lfs->mbox, lfs->pdev, + CPT_AF_LFX_CTL(lf->slot), + &lf_ctrl.u, lfs->blkaddr); + if (ret) + return ret; + + lf_ctrl.s.ctx_ilen = ctx_ilen; + + ret = otx2_cpt_write_af_reg(lfs->mbox, lfs->pdev, + CPT_AF_LFX_CTL(lf->slot), + lf_ctrl.u, lfs->blkaddr); + if (ret) + return ret; + } + return ret; +} + static void cptlf_hw_init(struct otx2_cptlfs_info *lfs) { /* Disable instruction queues */ @@ -443,6 +469,12 @@ int otx2_cptlf_init(struct otx2_cptlfs_info *lfs, u8 eng_grp_mask, int pri, if (ret) goto free_iq; + if (lfs->ctx_ilen_ovrd) { + ret = cptlf_set_ctx_ilen(lfs, lfs->ctx_ilen); + if (ret) + goto free_iq; + } + return 0; free_iq: diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptlf.h b/drivers/crypto/marvell/octeontx2/otx2_cptlf.h index 315a89a9bcb6..6e5dd9af94b7 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptlf.h +++ b/drivers/crypto/marvell/octeontx2/otx2_cptlf.h @@ -120,6 +120,8 @@ struct otx2_cptlfs_info { atomic_t state; /* LF's state. started/reset */ int blkaddr; /* CPT blkaddr: BLKADDR_CPT0/BLKADDR_CPT1 */ int global_slot; /* Global slot across the blocks */ + u8 ctx_ilen; + u8 ctx_ilen_ovrd; }; static inline void otx2_cpt_free_instruction_queues( @@ -309,6 +311,19 @@ static inline void otx2_cptlf_set_iqueue_exec(struct otx2_cptlf_info *lf, OTX2_CPT_LF_INPROG, lf_inprog.u); } +static inline void otx2_cptlf_set_ctx_flr_flush(struct otx2_cptlf_info *lf) +{ + u8 blkaddr = lf->lfs->blkaddr; + u64 val; + + val = otx2_cpt_read64(lf->lfs->reg_base, blkaddr, lf->slot, + OTX2_CPT_LF_CTX_CTL); + val |= BIT_ULL(0); + + otx2_cpt_write64(lf->lfs->reg_base, blkaddr, lf->slot, + OTX2_CPT_LF_CTX_CTL, val); +} + static inline void otx2_cptlf_enable_iqueue_exec(struct otx2_cptlf_info *lf) { otx2_cptlf_set_iqueue_exec(lf, true); @@ -324,6 +339,10 @@ static inline void otx2_cptlf_enable_iqueues(struct otx2_cptlfs_info *lfs) int slot; for (slot = 0; slot < lfs->lfs_num; slot++) { + /* Enable flush on FLR for Errata */ + if (is_dev_cn10kb(lfs->pdev)) + otx2_cptlf_set_ctx_flr_flush(&lfs->lf[slot]); + otx2_cptlf_enable_iqueue_exec(&lfs->lf[slot]); otx2_cptlf_enable_iqueue_enq(&lfs->lf[slot]); } diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c index 5d63763db2af..ec1ac7e836a3 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_mbox.c @@ -267,6 +267,9 @@ static int handle_msg_rx_inline_ipsec_lf_cfg(struct otx2_cptpf_dev *cptpf, otx2_cptlf_set_dev_info(&cptpf->lfs, cptpf->pdev, cptpf->reg_base, &cptpf->afpf_mbox, BLKADDR_CPT0); cptpf->lfs.global_slot = 0; + cptpf->lfs.ctx_ilen_ovrd = cfg_req->ctx_ilen_valid; + cptpf->lfs.ctx_ilen = cfg_req->ctx_ilen; + ret = otx2_inline_cptlf_setup(cptpf, &cptpf->lfs, egrp, num_lfs); if (ret) { dev_err(&cptpf->pdev->dev, "Inline-Ipsec CPT0 LF setup failed.\n"); @@ -279,6 +282,8 @@ static int handle_msg_rx_inline_ipsec_lf_cfg(struct otx2_cptpf_dev *cptpf, cptpf->reg_base, &cptpf->afpf_mbox, BLKADDR_CPT1); cptpf->cpt1_lfs.global_slot = num_lfs; + cptpf->cpt1_lfs.ctx_ilen_ovrd = cfg_req->ctx_ilen_valid; + cptpf->cpt1_lfs.ctx_ilen = cfg_req->ctx_ilen; ret = otx2_inline_cptlf_setup(cptpf, &cptpf->cpt1_lfs, egrp, num_lfs); if (ret) {