From patchwork Thu Dec 16 09:13:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Chiappero X-Patchwork-Id: 524710 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 549A2C433F5 for ; Thu, 16 Dec 2021 09:11:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232930AbhLPJLQ (ORCPT ); Thu, 16 Dec 2021 04:11:16 -0500 Received: from mga12.intel.com ([192.55.52.136]:9653 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232925AbhLPJLO (ORCPT ); Thu, 16 Dec 2021 04:11:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1639645874; x=1671181874; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NAnphNHT/gBugaCkRUMP+RZ7Sjjpk/Tmol23ImOmNCY=; b=BSg9ahoBO8Dxg53izy422AyEedBXfINN/Ggp9p9nEtoHE14iaiBKoCkA 0nyOpK2RLUsvcyXNWXyMYm6If24lNcSquPriGNoMfgXgFhpS6Ysi0LdNe u2mobAFOV/GRecGa63oQEPO+AtL6jyDBpko32i2PLSp5Yg+tmHq/32maJ l13hQOTq4DEDC/gKvOSAg4br/SLAlr9/efqjIcZtdEwersjzokTLD6SeF R+NTvkI/3txW3D14/Fis7+dpBXIVNf2zAkxWx74VcfCgfVQFZOAG8gxCY wSf/eOf2FclbGOJvkfTjFbvo1p5kSeHohaB6q0Qx20CZUdLP7fUfqVJ9+ Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10199"; a="219458340" X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="219458340" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2021 01:11:13 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="465968408" Received: from silpixa00393544.ir.intel.com ([10.237.213.118]) by orsmga006.jf.intel.com with ESMTP; 16 Dec 2021 01:11:12 -0800 From: Marco Chiappero To: herbert@gondor.apana.org.au Cc: linux-crypto@vger.kernel.org, qat-linux@intel.com, giovanni.cabiddu@intel.com, marco.chiappero@intel.com, Fiona Trahe Subject: [PATCH 02/24] crypto: qat - set CIPHER capability for QAT GEN2 Date: Thu, 16 Dec 2021 09:13:12 +0000 Message-Id: <20211216091334.402420-3-marco.chiappero@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211216091334.402420-1-marco.chiappero@intel.com> References: <20211216091334.402420-1-marco.chiappero@intel.com> MIME-Version: 1.0 Organization: Intel Research and Development Ireland Ltd - Co. Reg. #308263 - Collinstown Industrial Park, Leixlip, County Kildare - Ireland Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Giovanni Cabiddu Set the CIPHER capability for QAT GEN2 devices if the hardware supports it. This is done if both the CIPHER and the AUTHENTICATION engines are available on the device. Signed-off-by: Giovanni Cabiddu Signed-off-by: Marco Chiappero Reviewed-by: Fiona Trahe Reviewed-by: Marco Chiappero --- drivers/crypto/qat/qat_common/adf_gen2_hw_data.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/crypto/qat/qat_common/adf_gen2_hw_data.c b/drivers/crypto/qat/qat_common/adf_gen2_hw_data.c index 3b48fdaaff6d..3ea26f2f4a22 100644 --- a/drivers/crypto/qat/qat_common/adf_gen2_hw_data.c +++ b/drivers/crypto/qat/qat_common/adf_gen2_hw_data.c @@ -211,17 +211,23 @@ u32 adf_gen2_get_accel_cap(struct adf_accel_dev *accel_dev) u32 legfuses; u32 capabilities = ICP_ACCEL_CAPABILITIES_CRYPTO_SYMMETRIC | ICP_ACCEL_CAPABILITIES_CRYPTO_ASYMMETRIC | - ICP_ACCEL_CAPABILITIES_AUTHENTICATION; + ICP_ACCEL_CAPABILITIES_AUTHENTICATION | + ICP_ACCEL_CAPABILITIES_CIPHER; /* Read accelerator capabilities mask */ pci_read_config_dword(pdev, ADF_DEVICE_LEGFUSE_OFFSET, &legfuses); - if (legfuses & ICP_ACCEL_MASK_CIPHER_SLICE) + /* A set bit in legfuses means the feature is OFF in this SKU */ + if (legfuses & ICP_ACCEL_MASK_CIPHER_SLICE) { capabilities &= ~ICP_ACCEL_CAPABILITIES_CRYPTO_SYMMETRIC; + capabilities &= ~ICP_ACCEL_CAPABILITIES_CIPHER; + } if (legfuses & ICP_ACCEL_MASK_PKE_SLICE) capabilities &= ~ICP_ACCEL_CAPABILITIES_CRYPTO_ASYMMETRIC; - if (legfuses & ICP_ACCEL_MASK_AUTH_SLICE) + if (legfuses & ICP_ACCEL_MASK_AUTH_SLICE) { capabilities &= ~ICP_ACCEL_CAPABILITIES_AUTHENTICATION; + capabilities &= ~ICP_ACCEL_CAPABILITIES_CIPHER; + } if ((straps | fuses) & ADF_POWERGATE_PKE) capabilities &= ~ICP_ACCEL_CAPABILITIES_CRYPTO_ASYMMETRIC; From patchwork Thu Dec 16 09:13:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Chiappero X-Patchwork-Id: 524709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC3B7C433EF for ; Thu, 16 Dec 2021 09:11:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232918AbhLPJLV (ORCPT ); Thu, 16 Dec 2021 04:11:21 -0500 Received: from mga12.intel.com ([192.55.52.136]:9662 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232926AbhLPJLS (ORCPT ); Thu, 16 Dec 2021 04:11:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1639645878; x=1671181878; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dV59uIF5JkSeVWQspixFMwEK3gXirCFjVDr7IRBwYlg=; b=i+gwtdPfUfPWITnrmHRWiUTy+7zE1N3sU4LAX200DgaW2GRcCOh9sYan iDuxqhJMuV/kfnpjBqK0CSdCw57Vy+xm6DDlnK0B3BpGO/dva4u678dDh Q2pPx/pGi634s9G8MGBykGqch9N9pX++z+KJGjO5J86tZxlrprVUozwvx db9udiODjcSZAxpwjNjy3/0lJHWRzyenxM+3v8uNA/10IMi8LG/aR3OKk WJ1NebhvMLSGLe354eY3YNv5S0tujHsGR7UGErac1FQAva1x82lJDk5jP 3JubdqR6FQLlkkr+jAiXqFjOyJCtttfE/Li0uU+fKdcir7c/EbUg1va01 w==; X-IronPort-AV: E=McAfee;i="6200,9189,10199"; a="219458359" X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="219458359" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2021 01:11:18 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="465968426" Received: from silpixa00393544.ir.intel.com ([10.237.213.118]) by orsmga006.jf.intel.com with ESMTP; 16 Dec 2021 01:11:16 -0800 From: Marco Chiappero To: herbert@gondor.apana.org.au Cc: linux-crypto@vger.kernel.org, qat-linux@intel.com, giovanni.cabiddu@intel.com, marco.chiappero@intel.com, Fiona Trahe Subject: [PATCH 04/24] crypto: qat - extend crypto capability detection for 4xxx Date: Thu, 16 Dec 2021 09:13:14 +0000 Message-Id: <20211216091334.402420-5-marco.chiappero@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211216091334.402420-1-marco.chiappero@intel.com> References: <20211216091334.402420-1-marco.chiappero@intel.com> MIME-Version: 1.0 Organization: Intel Research and Development Ireland Ltd - Co. Reg. #308263 - Collinstown Industrial Park, Leixlip, County Kildare - Ireland Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Giovanni Cabiddu Extended the capability detection logic for 4xxx devices. Mask out unsupported algorithms and services based on the value read in the fuse register. This includes only capabilities for the crypto service. Signed-off-by: Giovanni Cabiddu Signed-off-by: Marco Chiappero Reviewed-by: Fiona Trahe Reviewed-by: Marco Chiappero --- .../crypto/qat/qat_4xxx/adf_4xxx_hw_data.c | 29 +++++++++++++++++-- drivers/crypto/qat/qat_common/icp_qat_hw.h | 9 +++++- 2 files changed, 34 insertions(+), 4 deletions(-) diff --git a/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c b/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c index 4658b7bf76da..d320c50c4561 100644 --- a/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c +++ b/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c @@ -98,18 +98,41 @@ static u32 get_accel_cap(struct adf_accel_dev *accel_dev) u32 fusectl1; u32 capabilities = ICP_ACCEL_CAPABILITIES_CRYPTO_SYMMETRIC | ICP_ACCEL_CAPABILITIES_CRYPTO_ASYMMETRIC | + ICP_ACCEL_CAPABILITIES_CIPHER | ICP_ACCEL_CAPABILITIES_AUTHENTICATION | + ICP_ACCEL_CAPABILITIES_SHA3 | + ICP_ACCEL_CAPABILITIES_SHA3_EXT | + ICP_ACCEL_CAPABILITIES_HKDF | + ICP_ACCEL_CAPABILITIES_ECEDMONT | + ICP_ACCEL_CAPABILITIES_CHACHA_POLY | + ICP_ACCEL_CAPABILITIES_AESGCM_SPC | ICP_ACCEL_CAPABILITIES_AES_V2; /* Read accelerator capabilities mask */ pci_read_config_dword(pdev, ADF_4XXX_FUSECTL1_OFFSET, &fusectl1); - if (fusectl1 & ICP_ACCEL_4XXX_MASK_CIPHER_SLICE) + /* A set bit in fusectl1 means the feature is OFF in this SKU */ + if (fusectl1 & ICP_ACCEL_4XXX_MASK_CIPHER_SLICE) { capabilities &= ~ICP_ACCEL_CAPABILITIES_CRYPTO_SYMMETRIC; - if (fusectl1 & ICP_ACCEL_4XXX_MASK_AUTH_SLICE) + capabilities &= ~ICP_ACCEL_CAPABILITIES_HKDF; + capabilities &= ~ICP_ACCEL_CAPABILITIES_CIPHER; + } + if (fusectl1 & ICP_ACCEL_4XXX_MASK_UCS_SLICE) { + capabilities &= ~ICP_ACCEL_CAPABILITIES_CHACHA_POLY; + capabilities &= ~ICP_ACCEL_CAPABILITIES_AESGCM_SPC; + capabilities &= ~ICP_ACCEL_CAPABILITIES_AES_V2; + capabilities &= ~ICP_ACCEL_CAPABILITIES_CIPHER; + } + if (fusectl1 & ICP_ACCEL_4XXX_MASK_AUTH_SLICE) { capabilities &= ~ICP_ACCEL_CAPABILITIES_AUTHENTICATION; - if (fusectl1 & ICP_ACCEL_4XXX_MASK_PKE_SLICE) + capabilities &= ~ICP_ACCEL_CAPABILITIES_SHA3; + capabilities &= ~ICP_ACCEL_CAPABILITIES_SHA3_EXT; + capabilities &= ~ICP_ACCEL_CAPABILITIES_CIPHER; + } + if (fusectl1 & ICP_ACCEL_4XXX_MASK_PKE_SLICE) { capabilities &= ~ICP_ACCEL_CAPABILITIES_CRYPTO_ASYMMETRIC; + capabilities &= ~ICP_ACCEL_CAPABILITIES_ECEDMONT; + } return capabilities; } diff --git a/drivers/crypto/qat/qat_common/icp_qat_hw.h b/drivers/crypto/qat/qat_common/icp_qat_hw.h index e39e8a2d51a7..5770b2b2c09e 100644 --- a/drivers/crypto/qat/qat_common/icp_qat_hw.h +++ b/drivers/crypto/qat/qat_common/icp_qat_hw.h @@ -91,7 +91,14 @@ enum icp_qat_capabilities_mask { ICP_ACCEL_CAPABILITIES_RAND = BIT(7), ICP_ACCEL_CAPABILITIES_ZUC = BIT(8), ICP_ACCEL_CAPABILITIES_SHA3 = BIT(9), - /* Bits 10-25 are currently reserved */ + /* Bits 10-11 are currently reserved */ + ICP_ACCEL_CAPABILITIES_HKDF = BIT(12), + ICP_ACCEL_CAPABILITIES_ECEDMONT = BIT(13), + /* Bit 14 is currently reserved */ + ICP_ACCEL_CAPABILITIES_SHA3_EXT = BIT(15), + ICP_ACCEL_CAPABILITIES_AESGCM_SPC = BIT(16), + ICP_ACCEL_CAPABILITIES_CHACHA_POLY = BIT(17), + /* Bits 18-25 are currently reserved */ ICP_ACCEL_CAPABILITIES_AES_V2 = BIT(26) }; From patchwork Thu Dec 16 09:13:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Chiappero X-Patchwork-Id: 524708 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FAEFC4332F for ; Thu, 16 Dec 2021 09:11:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232940AbhLPJLX (ORCPT ); Thu, 16 Dec 2021 04:11:23 -0500 Received: from mga12.intel.com ([192.55.52.136]:9672 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232927AbhLPJLW (ORCPT ); Thu, 16 Dec 2021 04:11:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1639645882; x=1671181882; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NuRd4IDkvME/gxL7utS9tAshVn6Sl0FNq4MureM6XS4=; b=mVxi2ruA3h70SYy4as8t9c54lUTg48WZwoZAE9YB+wIFnXmtutzZzC9t GPNxevT0Y1Eo0X8i0nIox6kMIBQBu7MsC3k7u6g8u0sKVVDp84lhbCopV pAZO81KPJnCgUtbS73TDYkHNSt8eGTTOJB6mnWnRcuQrjYpP+mMqtilQG z15iHGvlAPyH9rWkmQtyMirxUOEYEH7wnqNri70kg/z3XuEd+7dkam6r5 T8vKYzYX9mVYs9ktA8N97aCFidKIWAt1eFmqQIOkYFlCSeVOHjS4Yyema VQWXgW3H4A7aCcuhUtbcrlzS7Sd8zwMajp6liLqk6tXRoDokTeE0Pbncn g==; X-IronPort-AV: E=McAfee;i="6200,9189,10199"; a="219458375" X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="219458375" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2021 01:11:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="465968445" Received: from silpixa00393544.ir.intel.com ([10.237.213.118]) by orsmga006.jf.intel.com with ESMTP; 16 Dec 2021 01:11:20 -0800 From: Marco Chiappero To: herbert@gondor.apana.org.au Cc: linux-crypto@vger.kernel.org, qat-linux@intel.com, giovanni.cabiddu@intel.com, marco.chiappero@intel.com, Fiona Trahe Subject: [PATCH 06/24] crypto: qat - add the adf_get_pmisc_base() helper function Date: Thu, 16 Dec 2021 09:13:16 +0000 Message-Id: <20211216091334.402420-7-marco.chiappero@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211216091334.402420-1-marco.chiappero@intel.com> References: <20211216091334.402420-1-marco.chiappero@intel.com> MIME-Version: 1.0 Organization: Intel Research and Development Ireland Ltd - Co. Reg. #308263 - Collinstown Industrial Park, Leixlip, County Kildare - Ireland Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Add and use the new helper function adf_get_pmisc_base() where convenient. Also: - remove no longer shared variables - leverage other utilities, such as GET_PFVF_OPS(), as a consequence - consistently use the "pmisc_addr" name for the returned value of this new helper Signed-off-by: Marco Chiappero Co-developed-by: Giovanni Cabiddu Signed-off-by: Giovanni Cabiddu Reviewed-by: Fiona Trahe --- drivers/crypto/qat/qat_common/adf_admin.c | 10 ++--- .../crypto/qat/qat_common/adf_common_drv.h | 11 +++++ .../crypto/qat/qat_common/adf_gen2_hw_data.c | 39 ++++++------------ drivers/crypto/qat/qat_common/adf_gen2_pfvf.c | 24 ++++------- .../crypto/qat/qat_common/adf_gen4_hw_data.c | 9 +--- drivers/crypto/qat/qat_common/adf_isr.c | 28 ++++--------- drivers/crypto/qat/qat_common/adf_vf_isr.c | 14 ++----- drivers/crypto/qat/qat_common/qat_hal.c | 41 +++++-------------- 8 files changed, 61 insertions(+), 115 deletions(-) diff --git a/drivers/crypto/qat/qat_common/adf_admin.c b/drivers/crypto/qat/qat_common/adf_admin.c index c381b89d548d..498eb6f690e3 100644 --- a/drivers/crypto/qat/qat_common/adf_admin.c +++ b/drivers/crypto/qat/qat_common/adf_admin.c @@ -255,9 +255,7 @@ int adf_init_admin_comms(struct adf_accel_dev *accel_dev) { struct adf_admin_comms *admin; struct adf_hw_device_data *hw_data = accel_dev->hw_device; - struct adf_bar *pmisc = - &GET_BARS(accel_dev)[hw_data->get_misc_bar_id(hw_data)]; - void __iomem *csr = pmisc->virt_addr; + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); struct admin_info admin_csrs_info; u32 mailbox_offset, adminmsg_u, adminmsg_l; void __iomem *mailbox; @@ -291,13 +289,13 @@ int adf_init_admin_comms(struct adf_accel_dev *accel_dev) hw_data->get_admin_info(&admin_csrs_info); mailbox_offset = admin_csrs_info.mailbox_offset; - mailbox = csr + mailbox_offset; + mailbox = pmisc_addr + mailbox_offset; adminmsg_u = admin_csrs_info.admin_msg_ur; adminmsg_l = admin_csrs_info.admin_msg_lr; reg_val = (u64)admin->phy_addr; - ADF_CSR_WR(csr, adminmsg_u, upper_32_bits(reg_val)); - ADF_CSR_WR(csr, adminmsg_l, lower_32_bits(reg_val)); + ADF_CSR_WR(pmisc_addr, adminmsg_u, upper_32_bits(reg_val)); + ADF_CSR_WR(pmisc_addr, adminmsg_l, lower_32_bits(reg_val)); mutex_init(&admin->lock); admin->mailbox_addr = mailbox; diff --git a/drivers/crypto/qat/qat_common/adf_common_drv.h b/drivers/crypto/qat/qat_common/adf_common_drv.h index 2d8b72085505..5212891344a9 100644 --- a/drivers/crypto/qat/qat_common/adf_common_drv.h +++ b/drivers/crypto/qat/qat_common/adf_common_drv.h @@ -243,4 +243,15 @@ static inline void adf_flush_vf_wq(struct adf_accel_dev *accel_dev) } #endif + +static inline void __iomem *adf_get_pmisc_base(struct adf_accel_dev *accel_dev) +{ + struct adf_hw_device_data *hw_data = accel_dev->hw_device; + struct adf_bar *pmisc; + + pmisc = &GET_BARS(accel_dev)[hw_data->get_misc_bar_id(hw_data)]; + + return pmisc->virt_addr; +} + #endif diff --git a/drivers/crypto/qat/qat_common/adf_gen2_hw_data.c b/drivers/crypto/qat/qat_common/adf_gen2_hw_data.c index 688dd6f53b0b..57035b7dd4b2 100644 --- a/drivers/crypto/qat/qat_common/adf_gen2_hw_data.c +++ b/drivers/crypto/qat/qat_common/adf_gen2_hw_data.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0-only) /* Copyright(c) 2020 Intel Corporation */ +#include "adf_common_drv.h" #include "adf_gen2_hw_data.h" #include "icp_qat_hw.h" #include @@ -25,31 +26,29 @@ EXPORT_SYMBOL_GPL(adf_gen2_get_num_aes); void adf_gen2_enable_error_correction(struct adf_accel_dev *accel_dev) { struct adf_hw_device_data *hw_data = accel_dev->hw_device; - struct adf_bar *misc_bar = &GET_BARS(accel_dev) - [hw_data->get_misc_bar_id(hw_data)]; + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); unsigned long accel_mask = hw_data->accel_mask; unsigned long ae_mask = hw_data->ae_mask; - void __iomem *csr = misc_bar->virt_addr; unsigned int val, i; /* Enable Accel Engine error detection & correction */ for_each_set_bit(i, &ae_mask, hw_data->num_engines) { - val = ADF_CSR_RD(csr, ADF_GEN2_AE_CTX_ENABLES(i)); + val = ADF_CSR_RD(pmisc_addr, ADF_GEN2_AE_CTX_ENABLES(i)); val |= ADF_GEN2_ENABLE_AE_ECC_ERR; - ADF_CSR_WR(csr, ADF_GEN2_AE_CTX_ENABLES(i), val); - val = ADF_CSR_RD(csr, ADF_GEN2_AE_MISC_CONTROL(i)); + ADF_CSR_WR(pmisc_addr, ADF_GEN2_AE_CTX_ENABLES(i), val); + val = ADF_CSR_RD(pmisc_addr, ADF_GEN2_AE_MISC_CONTROL(i)); val |= ADF_GEN2_ENABLE_AE_ECC_PARITY_CORR; - ADF_CSR_WR(csr, ADF_GEN2_AE_MISC_CONTROL(i), val); + ADF_CSR_WR(pmisc_addr, ADF_GEN2_AE_MISC_CONTROL(i), val); } /* Enable shared memory error detection & correction */ for_each_set_bit(i, &accel_mask, hw_data->num_accel) { - val = ADF_CSR_RD(csr, ADF_GEN2_UERRSSMSH(i)); + val = ADF_CSR_RD(pmisc_addr, ADF_GEN2_UERRSSMSH(i)); val |= ADF_GEN2_ERRSSMSH_EN; - ADF_CSR_WR(csr, ADF_GEN2_UERRSSMSH(i), val); - val = ADF_CSR_RD(csr, ADF_GEN2_CERRSSMSH(i)); + ADF_CSR_WR(pmisc_addr, ADF_GEN2_UERRSSMSH(i), val); + val = ADF_CSR_RD(pmisc_addr, ADF_GEN2_CERRSSMSH(i)); val |= ADF_GEN2_ERRSSMSH_EN; - ADF_CSR_WR(csr, ADF_GEN2_CERRSSMSH(i), val); + ADF_CSR_WR(pmisc_addr, ADF_GEN2_CERRSSMSH(i), val); } } EXPORT_SYMBOL_GPL(adf_gen2_enable_error_correction); @@ -57,15 +56,9 @@ EXPORT_SYMBOL_GPL(adf_gen2_enable_error_correction); void adf_gen2_cfg_iov_thds(struct adf_accel_dev *accel_dev, bool enable, int num_a_regs, int num_b_regs) { - struct adf_hw_device_data *hw_data = accel_dev->hw_device; - void __iomem *pmisc_addr; - struct adf_bar *pmisc; - int pmisc_id, i; + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); u32 reg; - - pmisc_id = hw_data->get_misc_bar_id(hw_data); - pmisc = &GET_BARS(accel_dev)[pmisc_id]; - pmisc_addr = pmisc->virt_addr; + int i; /* Set/Unset Valid bit in AE Thread to PCIe Function Mapping Group A */ for (i = 0; i < num_a_regs; i++) { @@ -245,18 +238,12 @@ EXPORT_SYMBOL_GPL(adf_gen2_get_accel_cap); void adf_gen2_set_ssm_wdtimer(struct adf_accel_dev *accel_dev) { struct adf_hw_device_data *hw_data = accel_dev->hw_device; + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); u32 timer_val_pke = ADF_SSM_WDT_PKE_DEFAULT_VALUE; u32 timer_val = ADF_SSM_WDT_DEFAULT_VALUE; unsigned long accel_mask = hw_data->accel_mask; - void __iomem *pmisc_addr; - struct adf_bar *pmisc; - int pmisc_id; u32 i = 0; - pmisc_id = hw_data->get_misc_bar_id(hw_data); - pmisc = &GET_BARS(accel_dev)[pmisc_id]; - pmisc_addr = pmisc->virt_addr; - /* Configures WDT timers */ for_each_set_bit(i, &accel_mask, hw_data->num_accel) { /* Enable WDT for sym and dc */ diff --git a/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c b/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c index 099e39808d13..5ac69ece34a8 100644 --- a/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c +++ b/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c @@ -75,15 +75,12 @@ static void adf_gen2_disable_vf2pf_interrupts(void __iomem *pmisc_addr, static int adf_gen2_pfvf_send(struct adf_accel_dev *accel_dev, u32 msg, u8 vf_nr) { - struct adf_accel_pci *pci_info = &accel_dev->accel_pci_dev; - struct adf_hw_device_data *hw_data = accel_dev->hw_device; - void __iomem *pmisc_bar_addr = - pci_info->pci_bars[hw_data->get_misc_bar_id(hw_data)].virt_addr; - u32 val, pfvf_offset, count = 0; - u32 local_in_use_mask, local_in_use_pattern; + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); + unsigned int retries = ADF_PFVF_MSG_MAX_RETRIES; u32 remote_in_use_mask, remote_in_use_pattern; + u32 local_in_use_mask, local_in_use_pattern; + u32 val, pfvf_offset, count = 0; struct mutex *lock; /* lock preventing concurrent acces of CSR */ - unsigned int retries = ADF_PFVF_MSG_MAX_RETRIES; u32 int_bit; int ret; @@ -114,7 +111,7 @@ static int adf_gen2_pfvf_send(struct adf_accel_dev *accel_dev, u32 msg, ret = 0; /* Check if the PFVF CSR is in use by remote function */ - val = ADF_CSR_RD(pmisc_bar_addr, pfvf_offset); + val = ADF_CSR_RD(pmisc_addr, pfvf_offset); if ((val & remote_in_use_mask) == remote_in_use_pattern) { dev_dbg(&GET_DEV(accel_dev), "PFVF CSR in use by remote function\n"); @@ -122,12 +119,12 @@ static int adf_gen2_pfvf_send(struct adf_accel_dev *accel_dev, u32 msg, } /* Attempt to get ownership of the PFVF CSR */ - ADF_CSR_WR(pmisc_bar_addr, pfvf_offset, msg | int_bit); + ADF_CSR_WR(pmisc_addr, pfvf_offset, msg | int_bit); /* Wait for confirmation from remote func it received the message */ do { msleep(ADF_PFVF_MSG_ACK_DELAY); - val = ADF_CSR_RD(pmisc_bar_addr, pfvf_offset); + val = ADF_CSR_RD(pmisc_addr, pfvf_offset); } while ((val & int_bit) && (count++ < ADF_PFVF_MSG_ACK_MAX_RETRY)); if (val & int_bit) { @@ -143,7 +140,7 @@ static int adf_gen2_pfvf_send(struct adf_accel_dev *accel_dev, u32 msg, } /* Finished with the PFVF CSR; relinquish it and leave msg in CSR */ - ADF_CSR_WR(pmisc_bar_addr, pfvf_offset, val & ~local_in_use_mask); + ADF_CSR_WR(pmisc_addr, pfvf_offset, val & ~local_in_use_mask); out: mutex_unlock(lock); return ret; @@ -160,10 +157,7 @@ static int adf_gen2_pfvf_send(struct adf_accel_dev *accel_dev, u32 msg, static u32 adf_gen2_pfvf_recv(struct adf_accel_dev *accel_dev, u8 vf_nr) { - struct adf_accel_pci *pci_info = &accel_dev->accel_pci_dev; - struct adf_hw_device_data *hw_data = accel_dev->hw_device; - void __iomem *pmisc_addr = - pci_info->pci_bars[hw_data->get_misc_bar_id(hw_data)].virt_addr; + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); u32 pfvf_offset; u32 msg_origin; u32 int_bit; diff --git a/drivers/crypto/qat/qat_common/adf_gen4_hw_data.c b/drivers/crypto/qat/qat_common/adf_gen4_hw_data.c index c7808ff2aba1..3148a62938fd 100644 --- a/drivers/crypto/qat/qat_common/adf_gen4_hw_data.c +++ b/drivers/crypto/qat/qat_common/adf_gen4_hw_data.c @@ -111,20 +111,13 @@ static inline void adf_gen4_unpack_ssm_wdtimer(u64 value, u32 *upper, void adf_gen4_set_ssm_wdtimer(struct adf_accel_dev *accel_dev) { - struct adf_hw_device_data *hw_data = accel_dev->hw_device; + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); u64 timer_val_pke = ADF_SSM_WDT_PKE_DEFAULT_VALUE; u64 timer_val = ADF_SSM_WDT_DEFAULT_VALUE; u32 ssm_wdt_pke_high = 0; u32 ssm_wdt_pke_low = 0; u32 ssm_wdt_high = 0; u32 ssm_wdt_low = 0; - void __iomem *pmisc_addr; - struct adf_bar *pmisc; - int pmisc_id; - - pmisc_id = hw_data->get_misc_bar_id(hw_data); - pmisc = &GET_BARS(accel_dev)[pmisc_id]; - pmisc_addr = pmisc->virt_addr; /* Convert 64bit WDT timer value into 32bit values for * mmio write to 32bit CSRs. diff --git a/drivers/crypto/qat/qat_common/adf_isr.c b/drivers/crypto/qat/qat_common/adf_isr.c index 522e0c10d9b9..4ca482aa69f7 100644 --- a/drivers/crypto/qat/qat_common/adf_isr.c +++ b/drivers/crypto/qat/qat_common/adf_isr.c @@ -57,54 +57,42 @@ static irqreturn_t adf_msix_isr_bundle(int irq, void *bank_ptr) #ifdef CONFIG_PCI_IOV void adf_enable_vf2pf_interrupts(struct adf_accel_dev *accel_dev, u32 vf_mask) { - struct adf_hw_device_data *hw_data = accel_dev->hw_device; - u32 misc_bar_id = hw_data->get_misc_bar_id(hw_data); - struct adf_bar *pmisc = &GET_BARS(accel_dev)[misc_bar_id]; - void __iomem *pmisc_addr = pmisc->virt_addr; + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); unsigned long flags; spin_lock_irqsave(&accel_dev->pf.vf2pf_ints_lock, flags); - hw_data->pfvf_ops.enable_vf2pf_interrupts(pmisc_addr, vf_mask); + GET_PFVF_OPS(accel_dev)->enable_vf2pf_interrupts(pmisc_addr, vf_mask); spin_unlock_irqrestore(&accel_dev->pf.vf2pf_ints_lock, flags); } void adf_disable_vf2pf_interrupts(struct adf_accel_dev *accel_dev, u32 vf_mask) { - struct adf_hw_device_data *hw_data = accel_dev->hw_device; - u32 misc_bar_id = hw_data->get_misc_bar_id(hw_data); - struct adf_bar *pmisc = &GET_BARS(accel_dev)[misc_bar_id]; - void __iomem *pmisc_addr = pmisc->virt_addr; + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); unsigned long flags; spin_lock_irqsave(&accel_dev->pf.vf2pf_ints_lock, flags); - hw_data->pfvf_ops.disable_vf2pf_interrupts(pmisc_addr, vf_mask); + GET_PFVF_OPS(accel_dev)->disable_vf2pf_interrupts(pmisc_addr, vf_mask); spin_unlock_irqrestore(&accel_dev->pf.vf2pf_ints_lock, flags); } static void adf_disable_vf2pf_interrupts_irq(struct adf_accel_dev *accel_dev, u32 vf_mask) { - struct adf_hw_device_data *hw_data = accel_dev->hw_device; - u32 misc_bar_id = hw_data->get_misc_bar_id(hw_data); - struct adf_bar *pmisc = &GET_BARS(accel_dev)[misc_bar_id]; - void __iomem *pmisc_addr = pmisc->virt_addr; + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); spin_lock(&accel_dev->pf.vf2pf_ints_lock); - hw_data->pfvf_ops.disable_vf2pf_interrupts(pmisc_addr, vf_mask); + GET_PFVF_OPS(accel_dev)->disable_vf2pf_interrupts(pmisc_addr, vf_mask); spin_unlock(&accel_dev->pf.vf2pf_ints_lock); } static bool adf_handle_vf2pf_int(struct adf_accel_dev *accel_dev) { - struct adf_hw_device_data *hw_data = accel_dev->hw_device; - int bar_id = hw_data->get_misc_bar_id(hw_data); - struct adf_bar *pmisc = &GET_BARS(accel_dev)[bar_id]; - void __iomem *pmisc_addr = pmisc->virt_addr; + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); bool irq_handled = false; unsigned long vf_mask; /* Get the interrupt sources triggered by VFs */ - vf_mask = hw_data->pfvf_ops.get_vf2pf_sources(pmisc_addr); + vf_mask = GET_PFVF_OPS(accel_dev)->get_vf2pf_sources(pmisc_addr); if (vf_mask) { struct adf_accel_vf_info *vf_info; diff --git a/drivers/crypto/qat/qat_common/adf_vf_isr.c b/drivers/crypto/qat/qat_common/adf_vf_isr.c index fe094178f065..86c3bd0c9c2b 100644 --- a/drivers/crypto/qat/qat_common/adf_vf_isr.c +++ b/drivers/crypto/qat/qat_common/adf_vf_isr.c @@ -30,22 +30,16 @@ struct adf_vf_stop_data { void adf_enable_pf2vf_interrupts(struct adf_accel_dev *accel_dev) { - struct adf_accel_pci *pci_info = &accel_dev->accel_pci_dev; - struct adf_hw_device_data *hw_data = accel_dev->hw_device; - void __iomem *pmisc_bar_addr = - pci_info->pci_bars[hw_data->get_misc_bar_id(hw_data)].virt_addr; + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); - ADF_CSR_WR(pmisc_bar_addr, ADF_VINTMSK_OFFSET, 0x0); + ADF_CSR_WR(pmisc_addr, ADF_VINTMSK_OFFSET, 0x0); } void adf_disable_pf2vf_interrupts(struct adf_accel_dev *accel_dev) { - struct adf_accel_pci *pci_info = &accel_dev->accel_pci_dev; - struct adf_hw_device_data *hw_data = accel_dev->hw_device; - void __iomem *pmisc_bar_addr = - pci_info->pci_bars[hw_data->get_misc_bar_id(hw_data)].virt_addr; + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); - ADF_CSR_WR(pmisc_bar_addr, ADF_VINTMSK_OFFSET, 0x2); + ADF_CSR_WR(pmisc_addr, ADF_VINTMSK_OFFSET, 0x2); } EXPORT_SYMBOL_GPL(adf_disable_pf2vf_interrupts); diff --git a/drivers/crypto/qat/qat_common/qat_hal.c b/drivers/crypto/qat/qat_common/qat_hal.c index 12ca6b8764aa..4bfd8f3566f7 100644 --- a/drivers/crypto/qat/qat_common/qat_hal.c +++ b/drivers/crypto/qat/qat_common/qat_hal.c @@ -684,8 +684,7 @@ static int qat_hal_chip_init(struct icp_qat_fw_loader_handle *handle, { struct adf_accel_pci *pci_info = &accel_dev->accel_pci_dev; struct adf_hw_device_data *hw_data = accel_dev->hw_device; - struct adf_bar *misc_bar = - &pci_info->pci_bars[hw_data->get_misc_bar_id(hw_data)]; + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); unsigned int max_en_ae_id = 0; struct adf_bar *sram_bar; unsigned int csr_val = 0; @@ -715,18 +714,12 @@ static int qat_hal_chip_init(struct icp_qat_fw_loader_handle *handle, handle->chip_info->fcu_loaded_ae_csr = FCU_AE_LOADED_4XXX; handle->chip_info->fcu_loaded_ae_pos = 0; - handle->hal_cap_g_ctl_csr_addr_v = - (void __iomem *)((uintptr_t)misc_bar->virt_addr + - ICP_QAT_CAP_OFFSET_4XXX); - handle->hal_cap_ae_xfer_csr_addr_v = - (void __iomem *)((uintptr_t)misc_bar->virt_addr + - ICP_QAT_AE_OFFSET_4XXX); - handle->hal_ep_csr_addr_v = - (void __iomem *)((uintptr_t)misc_bar->virt_addr + - ICP_QAT_EP_OFFSET_4XXX); + handle->hal_cap_g_ctl_csr_addr_v = pmisc_addr + ICP_QAT_CAP_OFFSET_4XXX; + handle->hal_cap_ae_xfer_csr_addr_v = pmisc_addr + ICP_QAT_AE_OFFSET_4XXX; + handle->hal_ep_csr_addr_v = pmisc_addr + ICP_QAT_EP_OFFSET_4XXX; handle->hal_cap_ae_local_csr_addr_v = (void __iomem *)((uintptr_t)handle->hal_cap_ae_xfer_csr_addr_v - + LOCAL_TO_XFER_REG_OFFSET); + + LOCAL_TO_XFER_REG_OFFSET); break; case PCI_DEVICE_ID_INTEL_QAT_C62X: case PCI_DEVICE_ID_INTEL_QAT_C3XXX: @@ -749,15 +742,9 @@ static int qat_hal_chip_init(struct icp_qat_fw_loader_handle *handle, handle->chip_info->fcu_dram_addr_lo = FCU_DRAM_ADDR_LO; handle->chip_info->fcu_loaded_ae_csr = FCU_STATUS; handle->chip_info->fcu_loaded_ae_pos = FCU_LOADED_AE_POS; - handle->hal_cap_g_ctl_csr_addr_v = - (void __iomem *)((uintptr_t)misc_bar->virt_addr + - ICP_QAT_CAP_OFFSET); - handle->hal_cap_ae_xfer_csr_addr_v = - (void __iomem *)((uintptr_t)misc_bar->virt_addr + - ICP_QAT_AE_OFFSET); - handle->hal_ep_csr_addr_v = - (void __iomem *)((uintptr_t)misc_bar->virt_addr + - ICP_QAT_EP_OFFSET); + handle->hal_cap_g_ctl_csr_addr_v = pmisc_addr + ICP_QAT_CAP_OFFSET; + handle->hal_cap_ae_xfer_csr_addr_v = pmisc_addr + ICP_QAT_AE_OFFSET; + handle->hal_ep_csr_addr_v = pmisc_addr + ICP_QAT_EP_OFFSET; handle->hal_cap_ae_local_csr_addr_v = (void __iomem *)((uintptr_t)handle->hal_cap_ae_xfer_csr_addr_v + LOCAL_TO_XFER_REG_OFFSET); @@ -782,15 +769,9 @@ static int qat_hal_chip_init(struct icp_qat_fw_loader_handle *handle, handle->chip_info->fcu_dram_addr_lo = 0; handle->chip_info->fcu_loaded_ae_csr = 0; handle->chip_info->fcu_loaded_ae_pos = 0; - handle->hal_cap_g_ctl_csr_addr_v = - (void __iomem *)((uintptr_t)misc_bar->virt_addr + - ICP_QAT_CAP_OFFSET); - handle->hal_cap_ae_xfer_csr_addr_v = - (void __iomem *)((uintptr_t)misc_bar->virt_addr + - ICP_QAT_AE_OFFSET); - handle->hal_ep_csr_addr_v = - (void __iomem *)((uintptr_t)misc_bar->virt_addr + - ICP_QAT_EP_OFFSET); + handle->hal_cap_g_ctl_csr_addr_v = pmisc_addr + ICP_QAT_CAP_OFFSET; + handle->hal_cap_ae_xfer_csr_addr_v = pmisc_addr + ICP_QAT_AE_OFFSET; + handle->hal_ep_csr_addr_v = pmisc_addr + ICP_QAT_EP_OFFSET; handle->hal_cap_ae_local_csr_addr_v = (void __iomem *)((uintptr_t)handle->hal_cap_ae_xfer_csr_addr_v + LOCAL_TO_XFER_REG_OFFSET); From patchwork Thu Dec 16 09:13:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Chiappero X-Patchwork-Id: 524707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BA15C433FE for ; Thu, 16 Dec 2021 09:11:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232629AbhLPJL3 (ORCPT ); Thu, 16 Dec 2021 04:11:29 -0500 Received: from mga12.intel.com ([192.55.52.136]:9675 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232971AbhLPJL0 (ORCPT ); Thu, 16 Dec 2021 04:11:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1639645886; x=1671181886; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8fw1/SMP6K/M9KVhfpSaIj7VQtnYVbSI7od1Q+eBp0g=; b=Lry3oIY57dTEbJ7Zq4EQJq8foO+yy58HHUu0CKFZ8F6WIii1ttQsJGXY TScC6gzNZCilr4KBBIgAsLLyuU2/YMiXf5qUH0YfXmUEUCvNRV6U55b5y JO2fFsRTX5jykT9uoV64fwbhPwLhHAnDfPRFa4DvALmnwvZ3TTfVDIblK cFnax/sNf0/Gg2EJR8t27FQEZcGPW0fr0Q8K1aWHRj/aQ08OnKYKLL69O EXN15vwwMTBaGZlYjjJkipTXWivxmqALsDIozdVRjEJSrdRek5zZa7oaL HoEPMY3ZweQPINtCl6lHBe1M79asLSBAKEUpdb8eQ/L6pMH5WtRTfpqRe Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10199"; a="219458394" X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="219458394" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2021 01:11:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="465968460" Received: from silpixa00393544.ir.intel.com ([10.237.213.118]) by orsmga006.jf.intel.com with ESMTP; 16 Dec 2021 01:11:24 -0800 From: Marco Chiappero To: herbert@gondor.apana.org.au Cc: linux-crypto@vger.kernel.org, qat-linux@intel.com, giovanni.cabiddu@intel.com, marco.chiappero@intel.com, Fiona Trahe Subject: [PATCH 08/24] crypto: qat - make PFVF send and receive direction agnostic Date: Thu, 16 Dec 2021 09:13:18 +0000 Message-Id: <20211216091334.402420-9-marco.chiappero@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211216091334.402420-1-marco.chiappero@intel.com> References: <20211216091334.402420-1-marco.chiappero@intel.com> MIME-Version: 1.0 Organization: Intel Research and Development Ireland Ltd - Co. Reg. #308263 - Collinstown Industrial Park, Leixlip, County Kildare - Ireland Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Currently PF and VF share the same send and receive logic for the PFVF protocol. However, the inner behaviour still depends on the specific direction, requiring a test to determine the if the sender is a PF or a VF. Moreover the vf_nr parameter is only required for PF2VF messages and ignored for the opposite direction. Make the GEN2 send and recv completely direction agnostic, by calculating and determining any direction specific input in the caller instead, and feeding the send and the receive functions with the same arguments for both PF and VF. In order to accommodate for this change, the API of the pfvf_ops send and recv has been modified to remove any reference to vf_nr. Signed-off-by: Marco Chiappero Reviewed-by: Giovanni Cabiddu Reviewed-by: Fiona Trahe --- .../crypto/qat/qat_common/adf_accel_devices.h | 5 +- drivers/crypto/qat/qat_common/adf_gen2_pfvf.c | 99 +++++++++++++------ .../crypto/qat/qat_common/adf_pfvf_pf_proto.c | 11 ++- .../crypto/qat/qat_common/adf_pfvf_vf_proto.c | 11 ++- 4 files changed, 89 insertions(+), 37 deletions(-) diff --git a/drivers/crypto/qat/qat_common/adf_accel_devices.h b/drivers/crypto/qat/qat_common/adf_accel_devices.h index cc8b10b23145..d65d8dda8fda 100644 --- a/drivers/crypto/qat/qat_common/adf_accel_devices.h +++ b/drivers/crypto/qat/qat_common/adf_accel_devices.h @@ -154,8 +154,9 @@ struct adf_pfvf_ops { u32 (*get_vf2pf_sources)(void __iomem *pmisc_addr); void (*enable_vf2pf_interrupts)(void __iomem *pmisc_addr, u32 vf_mask); void (*disable_vf2pf_interrupts)(void __iomem *pmisc_addr, u32 vf_mask); - int (*send_msg)(struct adf_accel_dev *accel_dev, u32 msg, u8 vf_nr); - u32 (*recv_msg)(struct adf_accel_dev *accel_dev, u8 vf_nr); + int (*send_msg)(struct adf_accel_dev *accel_dev, u32 msg, + u32 pfvf_offset, struct mutex *csr_lock); + u32 (*recv_msg)(struct adf_accel_dev *accel_dev, u32 pfvf_offset); }; struct adf_hw_device_data { diff --git a/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c b/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c index 2e0b9ac27393..1e45f3230c19 100644 --- a/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c +++ b/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c @@ -115,15 +115,22 @@ static bool is_legacy_user_pfvf_message(u32 msg) return !(msg & ADF_PFVF_MSGORIGIN_SYSTEM); } +struct pfvf_gen2_params { + u32 pfvf_offset; + struct mutex *csr_lock; /* lock preventing concurrent access of CSR */ + enum gen2_csr_pos local_offset; + enum gen2_csr_pos remote_offset; +}; + static int adf_gen2_pfvf_send(struct adf_accel_dev *accel_dev, u32 msg, - u8 vf_nr) + struct pfvf_gen2_params *params) { void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); + enum gen2_csr_pos remote_offset = params->remote_offset; + enum gen2_csr_pos local_offset = params->local_offset; unsigned int retries = ADF_PFVF_MSG_MAX_RETRIES; - enum gen2_csr_pos remote_offset; - enum gen2_csr_pos local_offset; - struct mutex *lock; /* lock preventing concurrent acces of CSR */ - u32 pfvf_offset; + struct mutex *lock = params->csr_lock; + u32 pfvf_offset = params->pfvf_offset; u32 count = 0; u32 int_bit; u32 csr_val; @@ -136,17 +143,6 @@ static int adf_gen2_pfvf_send(struct adf_accel_dev *accel_dev, u32 msg, * it and after encoding it. Which one to shift depends on the * direction. */ - if (accel_dev->is_vf) { - pfvf_offset = GET_PFVF_OPS(accel_dev)->get_vf2pf_offset(0); - lock = &accel_dev->vf.vf2pf_lock; - local_offset = ADF_GEN2_CSR_VF2PF_OFFSET; - remote_offset = ADF_GEN2_CSR_PF2VF_OFFSET; - } else { - pfvf_offset = GET_PFVF_OPS(accel_dev)->get_pf2vf_offset(vf_nr); - lock = &accel_dev->pf.vf_info[vf_nr].pf2vf_lock; - local_offset = ADF_GEN2_CSR_PF2VF_OFFSET; - remote_offset = ADF_GEN2_CSR_VF2PF_OFFSET; - } int_bit = gen2_csr_get_int_bit(local_offset); @@ -208,23 +204,16 @@ static int adf_gen2_pfvf_send(struct adf_accel_dev *accel_dev, u32 msg, } } -static u32 adf_gen2_pfvf_recv(struct adf_accel_dev *accel_dev, u8 vf_nr) +static u32 adf_gen2_pfvf_recv(struct adf_accel_dev *accel_dev, + struct pfvf_gen2_params *params) { void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); - enum gen2_csr_pos local_offset; - u32 pfvf_offset; + enum gen2_csr_pos local_offset = params->local_offset; + u32 pfvf_offset = params->pfvf_offset; u32 int_bit; u32 csr_val; u32 msg; - if (accel_dev->is_vf) { - pfvf_offset = GET_PFVF_OPS(accel_dev)->get_pf2vf_offset(0); - local_offset = ADF_GEN2_CSR_PF2VF_OFFSET; - } else { - pfvf_offset = GET_PFVF_OPS(accel_dev)->get_vf2pf_offset(vf_nr); - local_offset = ADF_GEN2_CSR_VF2PF_OFFSET; - } - int_bit = gen2_csr_get_int_bit(local_offset); /* Read message */ @@ -252,6 +241,54 @@ static u32 adf_gen2_pfvf_recv(struct adf_accel_dev *accel_dev, u8 vf_nr) return msg; } +static int adf_gen2_pf2vf_send(struct adf_accel_dev *accel_dev, u32 msg, + u32 pfvf_offset, struct mutex *csr_lock) +{ + struct pfvf_gen2_params params = { + .csr_lock = csr_lock, + .pfvf_offset = pfvf_offset, + .local_offset = ADF_GEN2_CSR_PF2VF_OFFSET, + .remote_offset = ADF_GEN2_CSR_VF2PF_OFFSET, + }; + + return adf_gen2_pfvf_send(accel_dev, msg, ¶ms); +} + +static int adf_gen2_vf2pf_send(struct adf_accel_dev *accel_dev, u32 msg, + u32 pfvf_offset, struct mutex *csr_lock) +{ + struct pfvf_gen2_params params = { + .csr_lock = csr_lock, + .pfvf_offset = pfvf_offset, + .local_offset = ADF_GEN2_CSR_VF2PF_OFFSET, + .remote_offset = ADF_GEN2_CSR_PF2VF_OFFSET, + }; + + return adf_gen2_pfvf_send(accel_dev, msg, ¶ms); +} + +static u32 adf_gen2_pf2vf_recv(struct adf_accel_dev *accel_dev, u32 pfvf_offset) +{ + struct pfvf_gen2_params params = { + .pfvf_offset = pfvf_offset, + .local_offset = ADF_GEN2_CSR_PF2VF_OFFSET, + .remote_offset = ADF_GEN2_CSR_VF2PF_OFFSET, + }; + + return adf_gen2_pfvf_recv(accel_dev, ¶ms); +} + +static u32 adf_gen2_vf2pf_recv(struct adf_accel_dev *accel_dev, u32 pfvf_offset) +{ + struct pfvf_gen2_params params = { + .pfvf_offset = pfvf_offset, + .local_offset = ADF_GEN2_CSR_VF2PF_OFFSET, + .remote_offset = ADF_GEN2_CSR_PF2VF_OFFSET, + }; + + return adf_gen2_pfvf_recv(accel_dev, ¶ms); +} + void adf_gen2_init_pf_pfvf_ops(struct adf_pfvf_ops *pfvf_ops) { pfvf_ops->enable_comms = adf_enable_pf2vf_comms; @@ -260,8 +297,8 @@ void adf_gen2_init_pf_pfvf_ops(struct adf_pfvf_ops *pfvf_ops) pfvf_ops->get_vf2pf_sources = adf_gen2_get_vf2pf_sources; pfvf_ops->enable_vf2pf_interrupts = adf_gen2_enable_vf2pf_interrupts; pfvf_ops->disable_vf2pf_interrupts = adf_gen2_disable_vf2pf_interrupts; - pfvf_ops->send_msg = adf_gen2_pfvf_send; - pfvf_ops->recv_msg = adf_gen2_pfvf_recv; + pfvf_ops->send_msg = adf_gen2_pf2vf_send; + pfvf_ops->recv_msg = adf_gen2_vf2pf_recv; } EXPORT_SYMBOL_GPL(adf_gen2_init_pf_pfvf_ops); @@ -270,7 +307,7 @@ void adf_gen2_init_vf_pfvf_ops(struct adf_pfvf_ops *pfvf_ops) pfvf_ops->enable_comms = adf_enable_vf2pf_comms; pfvf_ops->get_pf2vf_offset = adf_gen2_vf_get_pfvf_offset; pfvf_ops->get_vf2pf_offset = adf_gen2_vf_get_pfvf_offset; - pfvf_ops->send_msg = adf_gen2_pfvf_send; - pfvf_ops->recv_msg = adf_gen2_pfvf_recv; + pfvf_ops->send_msg = adf_gen2_vf2pf_send; + pfvf_ops->recv_msg = adf_gen2_pf2vf_recv; } EXPORT_SYMBOL_GPL(adf_gen2_init_vf_pfvf_ops); diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c b/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c index fb477eb89fef..4e4daec2ae34 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c @@ -19,7 +19,11 @@ */ int adf_send_pf2vf_msg(struct adf_accel_dev *accel_dev, u8 vf_nr, u32 msg) { - return GET_PFVF_OPS(accel_dev)->send_msg(accel_dev, msg, vf_nr); + struct adf_pfvf_ops *pfvf_ops = GET_PFVF_OPS(accel_dev); + u32 pfvf_offset = pfvf_ops->get_pf2vf_offset(vf_nr); + + return pfvf_ops->send_msg(accel_dev, msg, pfvf_offset, + &accel_dev->pf.vf_info[vf_nr].pf2vf_lock); } /** @@ -33,7 +37,10 @@ int adf_send_pf2vf_msg(struct adf_accel_dev *accel_dev, u8 vf_nr, u32 msg) */ static u32 adf_recv_vf2pf_msg(struct adf_accel_dev *accel_dev, u8 vf_nr) { - return GET_PFVF_OPS(accel_dev)->recv_msg(accel_dev, vf_nr); + struct adf_pfvf_ops *pfvf_ops = GET_PFVF_OPS(accel_dev); + u32 pfvf_offset = pfvf_ops->get_vf2pf_offset(vf_nr); + + return pfvf_ops->recv_msg(accel_dev, pfvf_offset); } static int adf_handle_vf2pf_msg(struct adf_accel_dev *accel_dev, u32 vf_nr, diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c b/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c index f8d1c7d0ec4e..56e8185a9630 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c @@ -27,7 +27,11 @@ */ int adf_send_vf2pf_msg(struct adf_accel_dev *accel_dev, u32 msg) { - return GET_PFVF_OPS(accel_dev)->send_msg(accel_dev, msg, 0); + struct adf_pfvf_ops *pfvf_ops = GET_PFVF_OPS(accel_dev); + u32 pfvf_offset = pfvf_ops->get_vf2pf_offset(0); + + return pfvf_ops->send_msg(accel_dev, msg, pfvf_offset, + &accel_dev->vf.vf2pf_lock); } /** @@ -40,7 +44,10 @@ int adf_send_vf2pf_msg(struct adf_accel_dev *accel_dev, u32 msg) */ static u32 adf_recv_pf2vf_msg(struct adf_accel_dev *accel_dev) { - return GET_PFVF_OPS(accel_dev)->recv_msg(accel_dev, 0); + struct adf_pfvf_ops *pfvf_ops = GET_PFVF_OPS(accel_dev); + u32 pfvf_offset = pfvf_ops->get_pf2vf_offset(0); + + return pfvf_ops->recv_msg(accel_dev, pfvf_offset); } /** From patchwork Thu Dec 16 09:13:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Chiappero X-Patchwork-Id: 524706 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11271C433FE for ; Thu, 16 Dec 2021 09:11:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232938AbhLPJLe (ORCPT ); Thu, 16 Dec 2021 04:11:34 -0500 Received: from mga12.intel.com ([192.55.52.136]:9683 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232619AbhLPJLc (ORCPT ); Thu, 16 Dec 2021 04:11:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1639645892; x=1671181892; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZPoBss+8BrjSnslk/OP6CDGYWVGRc9nCraeKvdVLrjc=; b=hyMUT9nsUqf4y5OJYVAsZmGbHpQM97u3FQvxo8voTiy+8N5ccawEuqXH MxvjI5eHy0Q8qJVBe/+Y9VPPdCyn4SPvfBTgtly7TyAqy7dhZgEHS48Dc AZZ8n3DwtAAHWKwbQ1n61Se4PXuq+8oMzfg8FGInOV9QUfvay0v8beeiz pKLnTxGqeHgHlJzPLGJtHq0dI3gcNSYJFb9n9qXPmNSJDyGzNmSq0Jjy/ M3Dldwog0StkmwWfiBebLlHG3v/p0ta53Q7FwhLjTj2i8rU0hz1P2FSRZ gzFLpdTAOsMWnUxcO8wRL0dOmZvf+KW5rdtcEU4GjRhMSOaZyJcix9A0t A==; X-IronPort-AV: E=McAfee;i="6200,9189,10199"; a="219458432" X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="219458432" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2021 01:11:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="465968497" Received: from silpixa00393544.ir.intel.com ([10.237.213.118]) by orsmga006.jf.intel.com with ESMTP; 16 Dec 2021 01:11:30 -0800 From: Marco Chiappero To: herbert@gondor.apana.org.au Cc: linux-crypto@vger.kernel.org, qat-linux@intel.com, giovanni.cabiddu@intel.com, marco.chiappero@intel.com, Fiona Trahe Subject: [PATCH 11/24] crypto: qat - leverage bitfield.h utils for PFVF messages Date: Thu, 16 Dec 2021 09:13:21 +0000 Message-Id: <20211216091334.402420-12-marco.chiappero@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211216091334.402420-1-marco.chiappero@intel.com> References: <20211216091334.402420-1-marco.chiappero@intel.com> MIME-Version: 1.0 Organization: Intel Research and Development Ireland Ltd - Co. Reg. #308263 - Collinstown Industrial Park, Leixlip, County Kildare - Ireland Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The PFVF protocol defines messages composed of a number of control bitfields. Replace all the code setting and retrieving such bits with the utilities from bitfield.h, to improve code quality and readability. Signed-off-by: Marco Chiappero Reviewed-by: Giovanni Cabiddu Reviewed-by: Fiona Trahe --- drivers/crypto/qat/qat_common/adf_pfvf_msg.h | 8 ++++---- drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c | 13 +++++++------ drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c | 7 +++---- 3 files changed, 14 insertions(+), 14 deletions(-) diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_msg.h b/drivers/crypto/qat/qat_common/adf_pfvf_msg.h index 26eb27853e83..daee3d7ceb8c 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_msg.h +++ b/drivers/crypto/qat/qat_common/adf_pfvf_msg.h @@ -3,6 +3,8 @@ #ifndef ADF_PFVF_MSG_H #define ADF_PFVF_MSG_H +#include + /* * PF<->VF Messaging * The PF has an array of 32-bit PF2VF registers, one for each VF. The @@ -86,10 +88,8 @@ enum pfvf_compatibility_version { }; /* PF->VF Version Response */ -#define ADF_PF2VF_VERSION_RESP_VERS_SHIFT 0 -#define ADF_PF2VF_VERSION_RESP_VERS_MASK 0xFF -#define ADF_PF2VF_VERSION_RESP_RESULT_SHIFT 8 -#define ADF_PF2VF_VERSION_RESP_RESULT_MASK 0x03 +#define ADF_PF2VF_VERSION_RESP_VERS_MASK GENMASK(7, 0) +#define ADF_PF2VF_VERSION_RESP_RESULT_MASK GENMASK(9, 8) enum pf2vf_compat_response { ADF_PF2VF_VF_COMPATIBLE = 0x01, diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c b/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c index bb4d7db68579..8785b9d1df91 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0-only) /* Copyright(c) 2015 - 2021 Intel Corporation */ +#include #include #include #include "adf_accel_devices.h" @@ -64,9 +65,9 @@ static int adf_handle_vf2pf_msg(struct adf_accel_dev *accel_dev, u8 vf_nr, compat = ADF_PF2VF_VF_COMPAT_UNKNOWN; resp->type = ADF_PF2VF_MSGTYPE_VERSION_RESP; - resp->data = ADF_PFVF_COMPAT_THIS_VERSION << - ADF_PF2VF_VERSION_RESP_VERS_SHIFT; - resp->data |= compat << ADF_PF2VF_VERSION_RESP_RESULT_SHIFT; + resp->data = FIELD_PREP(ADF_PF2VF_VERSION_RESP_VERS_MASK, + ADF_PFVF_COMPAT_THIS_VERSION) | + FIELD_PREP(ADF_PF2VF_VERSION_RESP_RESULT_MASK, compat); } break; case ADF_VF2PF_MSGTYPE_VERSION_REQ: @@ -80,10 +81,10 @@ static int adf_handle_vf2pf_msg(struct adf_accel_dev *accel_dev, u8 vf_nr, /* PF always newer than legacy VF */ compat = ADF_PF2VF_VF_COMPATIBLE; - resp->type = ADF_PF2VF_MSGTYPE_VERSION_RESP; /* Set legacy major and minor version to the latest, 1.1 */ - resp->data |= 0x11; - resp->data |= compat << ADF_PF2VF_VERSION_RESP_RESULT_SHIFT; + resp->type = ADF_PF2VF_MSGTYPE_VERSION_RESP; + resp->data = FIELD_PREP(ADF_PF2VF_VERSION_RESP_VERS_MASK, 0x11) | + FIELD_PREP(ADF_PF2VF_VERSION_RESP_RESULT_MASK, compat); } break; case ADF_VF2PF_MSGTYPE_INIT: diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c b/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c index 5184a77598d2..130d7b9c12ea 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0-only) /* Copyright(c) 2015 - 2021 Intel Corporation */ +#include #include "adf_accel_devices.h" #include "adf_common_drv.h" #include "adf_pfvf_msg.h" @@ -67,10 +68,8 @@ int adf_vf2pf_request_version(struct adf_accel_dev *accel_dev) return ret; } - pf_version = (resp.data >> ADF_PF2VF_VERSION_RESP_VERS_SHIFT) - & ADF_PF2VF_VERSION_RESP_VERS_MASK; - compat = (resp.data >> ADF_PF2VF_VERSION_RESP_RESULT_SHIFT) - & ADF_PF2VF_VERSION_RESP_RESULT_MASK; + pf_version = FIELD_GET(ADF_PF2VF_VERSION_RESP_VERS_MASK, resp.data); + compat = FIELD_GET(ADF_PF2VF_VERSION_RESP_RESULT_MASK, resp.data); /* Response from PF received, check compatibility */ switch (compat) { From patchwork Thu Dec 16 09:13:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Chiappero X-Patchwork-Id: 524705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56A8FC433EF for ; Thu, 16 Dec 2021 09:11:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232968AbhLPJLg (ORCPT ); Thu, 16 Dec 2021 04:11:36 -0500 Received: from mga12.intel.com ([192.55.52.136]:9683 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232888AbhLPJLe (ORCPT ); Thu, 16 Dec 2021 04:11:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1639645894; x=1671181894; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=U25gde491WTGhSUqTUeM0lvLr3rTCGjvib2coeNmkRE=; b=m5F1qwjr8GtZf/Q4L/gwysP7tLLv7+5F7R8yszZB6gO3/hSieZgn2nB9 HIBVVJ3PfZpu5b0Iq14Ze6dQYtq73Mb1QEPFHMUlfacB7Sq2djRZcxvll iz9poTsGOeKOGJbu6gqx67VhPHdE3owBKf1YpOlgRR7y7DXHVgnTcOF3U 82SxGarOJDOh3dX/9SlLRSehr18jwYCbuQIxnIeCh9LjRJ7323G/NCiLo Fb3PdwdNbubfTQylRwvyNrxSrarTrbPpiUCzgWLH4A3CM7i2vWhG2XxfI qBL9wm93nbjXxQml24G7/Vtk02eN0TPZeOSuU7y6/Tdw+3qRgZB4ZIAzH g==; X-IronPort-AV: E=McAfee;i="6200,9189,10199"; a="219458441" X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="219458441" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2021 01:11:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="465968507" Received: from silpixa00393544.ir.intel.com ([10.237.213.118]) by orsmga006.jf.intel.com with ESMTP; 16 Dec 2021 01:11:32 -0800 From: Marco Chiappero To: herbert@gondor.apana.org.au Cc: linux-crypto@vger.kernel.org, qat-linux@intel.com, giovanni.cabiddu@intel.com, marco.chiappero@intel.com, Fiona Trahe Subject: [PATCH 12/24] crypto: qat - leverage read_poll_timeout in PFVF send Date: Thu, 16 Dec 2021 09:13:22 +0000 Message-Id: <20211216091334.402420-13-marco.chiappero@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211216091334.402420-1-marco.chiappero@intel.com> References: <20211216091334.402420-1-marco.chiappero@intel.com> MIME-Version: 1.0 Organization: Intel Research and Development Ireland Ltd - Co. Reg. #308263 - Collinstown Industrial Park, Leixlip, County Kildare - Ireland Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Replace the polling loop, waiting for the remote end to acknowledge the reception of the message, with the equivalent and standard read_poll_timeout() in adf_gen2_pfvf_send(). Also, the use of the read_poll_timeout(): - implies the use of microseconds for the timings, so update the previous values from ms to us - allows to leverage the return value for both success and error, removing the need for the reset of the 'ret' variable soon after the 'start' label. Signed-off-by: Marco Chiappero Co-developed-by: Giovanni Cabiddu Signed-off-by: Giovanni Cabiddu Reviewed-by: Fiona Trahe --- drivers/crypto/qat/qat_common/adf_gen2_pfvf.c | 20 ++++++++----------- 1 file changed, 8 insertions(+), 12 deletions(-) diff --git a/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c b/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c index 7a927bea4ac6..53c2e124944d 100644 --- a/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c +++ b/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0-only) /* Copyright(c) 2021 Intel Corporation */ #include +#include #include #include #include "adf_accel_devices.h" @@ -36,8 +37,8 @@ static const struct pfvf_csr_format csr_gen2_fmt = { { ADF_PFVF_GEN2_MSGDATA_SHIFT, ADF_PFVF_GEN2_MSGDATA_MASK }, }; -#define ADF_PFVF_MSG_ACK_DELAY 2 -#define ADF_PFVF_MSG_ACK_MAX_RETRY 100 +#define ADF_PFVF_MSG_ACK_DELAY_US 2000 +#define ADF_PFVF_MSG_ACK_MAX_DELAY_US (ADF_PFVF_MSG_ACK_DELAY_US * 100) #define ADF_PFVF_MSG_RETRY_DELAY 5 #define ADF_PFVF_MSG_MAX_RETRIES 3 @@ -143,7 +144,6 @@ static int adf_gen2_pfvf_send(struct adf_accel_dev *accel_dev, unsigned int retries = ADF_PFVF_MSG_MAX_RETRIES; struct mutex *lock = params->csr_lock; u32 pfvf_offset = params->pfvf_offset; - u32 count = 0; u32 int_bit; u32 csr_val; u32 csr_msg; @@ -172,8 +172,6 @@ static int adf_gen2_pfvf_send(struct adf_accel_dev *accel_dev, mutex_lock(lock); start: - ret = 0; - /* Check if the PFVF CSR is in use by remote function */ csr_val = ADF_CSR_RD(pmisc_addr, pfvf_offset); if (gen2_csr_is_in_use(csr_val, local_offset)) { @@ -186,15 +184,13 @@ static int adf_gen2_pfvf_send(struct adf_accel_dev *accel_dev, ADF_CSR_WR(pmisc_addr, pfvf_offset, csr_msg | int_bit); /* Wait for confirmation from remote func it received the message */ - do { - msleep(ADF_PFVF_MSG_ACK_DELAY); - csr_val = ADF_CSR_RD(pmisc_addr, pfvf_offset); - } while ((csr_val & int_bit) && (count++ < ADF_PFVF_MSG_ACK_MAX_RETRY)); - - if (csr_val & int_bit) { + ret = read_poll_timeout(ADF_CSR_RD, csr_val, !(csr_val & int_bit), + ADF_PFVF_MSG_ACK_DELAY_US, + ADF_PFVF_MSG_ACK_MAX_DELAY_US, + true, pmisc_addr, pfvf_offset); + if (unlikely(ret < 0)) { dev_dbg(&GET_DEV(accel_dev), "ACK not received from remote\n"); csr_val &= ~int_bit; - ret = -EIO; } if (csr_val != csr_msg) { From patchwork Thu Dec 16 09:13:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Chiappero X-Patchwork-Id: 524704 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7279AC433EF for ; Thu, 16 Dec 2021 09:11:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232888AbhLPJLv (ORCPT ); Thu, 16 Dec 2021 04:11:51 -0500 Received: from mga12.intel.com ([192.55.52.136]:9692 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232596AbhLPJLi (ORCPT ); Thu, 16 Dec 2021 04:11:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1639645898; x=1671181898; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=m6vmzx024UB2uvhlh0srr97NoXKZYZPYpv/vWiidmCc=; b=Eznxk9qxZoXR1JCdWzApcdEA5Qv/CUiTRyHBByOeIj4UvpveZuLOXAsg 2TQaAxpWtWjuW+y+LX0oDr/axrm/Y8AO4ilV6FZHLTmBxZc5Ek0alTl0l 29UEBYtOOywcb88+U9g9wO0TmNRKuR+2aTjAgcKdIhNK04kHp9zsxxhLB 8OCmSGb18bT2u5VONGl2CWfY6j4+RQX3FEhrIrhaENn/QGHk43L+qqyTF iJaTfPV4CiF4rncz3pLH+nV+mXuSiZtAn7Iwdg0ugjrg6iAIbKe1zrjmp hcdxpAs5gaV82e8nramQxO1sUL//Zn8BJnlE6eqR7TLrZFf7aSYs0L5+b w==; X-IronPort-AV: E=McAfee;i="6200,9189,10199"; a="219458453" X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="219458453" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2021 01:11:37 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="465968524" Received: from silpixa00393544.ir.intel.com ([10.237.213.118]) by orsmga006.jf.intel.com with ESMTP; 16 Dec 2021 01:11:36 -0800 From: Marco Chiappero To: herbert@gondor.apana.org.au Cc: linux-crypto@vger.kernel.org, qat-linux@intel.com, giovanni.cabiddu@intel.com, marco.chiappero@intel.com, Fiona Trahe Subject: [PATCH 14/24] crypto: qat - store the PFVF protocol version of the endpoints Date: Thu, 16 Dec 2021 09:13:24 +0000 Message-Id: <20211216091334.402420-15-marco.chiappero@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211216091334.402420-1-marco.chiappero@intel.com> References: <20211216091334.402420-1-marco.chiappero@intel.com> MIME-Version: 1.0 Organization: Intel Research and Development Ireland Ltd - Co. Reg. #308263 - Collinstown Industrial Park, Leixlip, County Kildare - Ireland Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org This patch adds an entry for storing the PFVF protocol version for both PF and VFs. While not currently used, knowing and storing the remote protocol version enables more complex compatibility checks and/or newer features for compatible PFVF endpoints in the future. Signed-off-by: Marco Chiappero Reviewed-by: Giovanni Cabiddu Reviewed-by: Fiona Trahe --- drivers/crypto/qat/qat_common/adf_accel_devices.h | 3 ++- drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c | 5 +++++ drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c | 2 +- drivers/crypto/qat/qat_common/adf_sriov.c | 1 + 4 files changed, 9 insertions(+), 2 deletions(-) diff --git a/drivers/crypto/qat/qat_common/adf_accel_devices.h b/drivers/crypto/qat/qat_common/adf_accel_devices.h index d5ccefc04153..1fb32f3e78df 100644 --- a/drivers/crypto/qat/qat_common/adf_accel_devices.h +++ b/drivers/crypto/qat/qat_common/adf_accel_devices.h @@ -249,6 +249,7 @@ struct adf_accel_vf_info { struct ratelimit_state vf2pf_ratelimit; u32 vf_nr; bool init; + u8 vf_compat_ver; }; struct adf_accel_dev { @@ -278,7 +279,7 @@ struct adf_accel_dev { struct mutex vf2pf_lock; /* protect CSR access */ struct completion msg_received; struct pfvf_message response; /* temp field holding pf2vf response */ - u8 pf_version; + u8 pf_compat_ver; } vf; }; bool is_vf; diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c b/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c index 8785b9d1df91..f461aa0a95c7 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c @@ -64,6 +64,8 @@ static int adf_handle_vf2pf_msg(struct adf_accel_dev *accel_dev, u8 vf_nr, else compat = ADF_PF2VF_VF_COMPAT_UNKNOWN; + vf_info->vf_compat_ver = vf_compat_ver; + resp->type = ADF_PF2VF_MSGTYPE_VERSION_RESP; resp->data = FIELD_PREP(ADF_PF2VF_VERSION_RESP_VERS_MASK, ADF_PFVF_COMPAT_THIS_VERSION) | @@ -78,6 +80,9 @@ static int adf_handle_vf2pf_msg(struct adf_accel_dev *accel_dev, u8 vf_nr, "Legacy VersionRequest received from VF%d to PF (vers 1.1)\n", vf_nr); + /* legacy driver, VF compat_ver is 0 */ + vf_info->vf_compat_ver = 0; + /* PF always newer than legacy VF */ compat = ADF_PF2VF_VF_COMPATIBLE; diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c b/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c index 130d7b9c12ea..307d593042f3 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c @@ -89,6 +89,6 @@ int adf_vf2pf_request_version(struct adf_accel_dev *accel_dev) return -EINVAL; } - accel_dev->vf.pf_version = pf_version; + accel_dev->vf.pf_compat_ver = pf_version; return 0; } diff --git a/drivers/crypto/qat/qat_common/adf_sriov.c b/drivers/crypto/qat/qat_common/adf_sriov.c index 429990c5e0f3..6366622ff8fd 100644 --- a/drivers/crypto/qat/qat_common/adf_sriov.c +++ b/drivers/crypto/qat/qat_common/adf_sriov.c @@ -58,6 +58,7 @@ static int adf_enable_sriov(struct adf_accel_dev *accel_dev) /* This ptr will be populated when VFs will be created */ vf_info->accel_dev = accel_dev; vf_info->vf_nr = i; + vf_info->vf_compat_ver = 0; mutex_init(&vf_info->pf2vf_lock); ratelimit_state_init(&vf_info->vf2pf_ratelimit, From patchwork Thu Dec 16 09:13:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Chiappero X-Patchwork-Id: 524702 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8273FC4332F for ; Thu, 16 Dec 2021 09:11:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232969AbhLPJLw (ORCPT ); Thu, 16 Dec 2021 04:11:52 -0500 Received: from mga12.intel.com ([192.55.52.136]:9683 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232975AbhLPJLn (ORCPT ); Thu, 16 Dec 2021 04:11:43 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1639645903; x=1671181903; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=v0kOuOoWwxjjZ/ehkkqv1ecpLX08q/d/6I9XLy5GwKM=; b=QB4rEMCVwaCLHlTQKaVxvsAIyxlUtSGnLJL9Edkp1ulJGr9EvBRxeYFb 0UjEm+J7RP/25uKGL+lHjofKWkkxVtY+sZrDa8cmEtm+buTnSvNRzh/Qt YdyNGCiMeaNN8sqv5iuH+VdLINkY0AqAdf26qWP4cMiNKasLxfKwMY7oX Fd7swNr3cAyGClSpbu/RI/9LEs58uQmPV/AsTbDswRO2AjiyYFM/KWYno weHzpJdKALFQHqHn/gyfRrEbz4nMl2S3sROxWmKkrd7MNPiNZCG/Cwn3K 4HYMNmhsNzS0qJ7oXgIFwLualvm78/Q9B2rTbe3va0wThu+g+UJLW2D0A Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10199"; a="219458465" X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="219458465" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2021 01:11:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="465968551" Received: from silpixa00393544.ir.intel.com ([10.237.213.118]) by orsmga006.jf.intel.com with ESMTP; 16 Dec 2021 01:11:39 -0800 From: Marco Chiappero To: herbert@gondor.apana.org.au Cc: linux-crypto@vger.kernel.org, qat-linux@intel.com, giovanni.cabiddu@intel.com, marco.chiappero@intel.com, Fiona Trahe Subject: [PATCH 16/24] crypto: qat - introduce support for PFVF block messages Date: Thu, 16 Dec 2021 09:13:26 +0000 Message-Id: <20211216091334.402420-17-marco.chiappero@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211216091334.402420-1-marco.chiappero@intel.com> References: <20211216091334.402420-1-marco.chiappero@intel.com> MIME-Version: 1.0 Organization: Intel Research and Development Ireland Ltd - Co. Reg. #308263 - Collinstown Industrial Park, Leixlip, County Kildare - Ireland Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org GEN2 devices use a single CSR for PFVF messages, which leaves up to 10 bits of payload per single message. While such amount is sufficient for the currently defined messages, the transfer of bigger and more complex data streams from the PF to the VF requires a new mechanism that extends the protocol. This patch adds a new layer on top of the basic PFVF messaging, called Block Messages, to encapsulate up to 126 bytes of data in a single logical message across multiple PFVF messages of new types (SMALL, MEDIUM and LARGE BLOCK), including (sub)types (BLKMSG_TYPE) to carry the information about the actual Block Message. Regardless of the size, each Block Message uses a two bytes header, containing the version and size, to allow for extension while maintaining compatibility. The size and the types of Block Messages are defined as follow: - small block messages: up to 16 BLKMSG types of up to 30 bytes - medium block messages: up to 8 BLKMSG types of up to 62 bytes - large block messages: up to 4 BLKMSG types of up to 126 bytes It effectively works as reading a byte at a time from a block device and for each of these new Block Messages: - the requestor (always a VF) can either request a specific byte of the larger message, in order to retrieve the full message, or request the value of the CRC calculated for a specific message up to the provided size (to allow for messages to grow while maintaining forward compatibility) - the responder (always the PF) will either return a single data or CRC byte, along with the indication of response type (or error). This patch provides the basic infrastructure to perform the above operations, without defining any new message. As CRCs are required, this code now depends on the CRC8 module. Note: as a consequence of the Block Messages design, sending multiple PFVF messages in bursts, the interrupt rate limiting values on the PF are increased. Signed-off-by: Marco Chiappero Co-developed-by: Giovanni Cabiddu Signed-off-by: Giovanni Cabiddu Reviewed-by: Fiona Trahe --- drivers/crypto/qat/Kconfig | 1 + drivers/crypto/qat/qat_common/adf_pfvf_msg.h | 74 +++++++ .../crypto/qat/qat_common/adf_pfvf_pf_msg.h | 3 + .../crypto/qat/qat_common/adf_pfvf_pf_proto.c | 137 ++++++++++++ .../crypto/qat/qat_common/adf_pfvf_utils.c | 15 ++ .../crypto/qat/qat_common/adf_pfvf_utils.h | 3 + .../crypto/qat/qat_common/adf_pfvf_vf_proto.c | 203 +++++++++++++++++- .../crypto/qat/qat_common/adf_pfvf_vf_proto.h | 2 + drivers/crypto/qat/qat_common/adf_sriov.c | 7 +- 9 files changed, 442 insertions(+), 3 deletions(-) diff --git a/drivers/crypto/qat/Kconfig b/drivers/crypto/qat/Kconfig index 77783feb62b2..4b90c0f22b03 100644 --- a/drivers/crypto/qat/Kconfig +++ b/drivers/crypto/qat/Kconfig @@ -13,6 +13,7 @@ config CRYPTO_DEV_QAT select CRYPTO_SHA512 select CRYPTO_LIB_AES select FW_LOADER + select CRC8 config CRYPTO_DEV_QAT_DH895xCC tristate "Support for Intel(R) DH895xCC" diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_msg.h b/drivers/crypto/qat/qat_common/adf_pfvf_msg.h index daee3d7ceb8c..6abbb5e05809 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_msg.h +++ b/drivers/crypto/qat/qat_common/adf_pfvf_msg.h @@ -71,6 +71,7 @@ struct pfvf_message { enum pf2vf_msgtype { ADF_PF2VF_MSGTYPE_RESTARTING = 0x01, ADF_PF2VF_MSGTYPE_VERSION_RESP = 0x02, + ADF_PF2VF_MSGTYPE_BLKMSG_RESP = 0x03, }; /* VF->PF messages */ @@ -79,6 +80,9 @@ enum vf2pf_msgtype { ADF_VF2PF_MSGTYPE_SHUTDOWN = 0x04, ADF_VF2PF_MSGTYPE_VERSION_REQ = 0x05, ADF_VF2PF_MSGTYPE_COMPAT_VER_REQ = 0x06, + ADF_VF2PF_MSGTYPE_LARGE_BLOCK_REQ = 0x07, + ADF_VF2PF_MSGTYPE_MEDIUM_BLOCK_REQ = 0x08, + ADF_VF2PF_MSGTYPE_SMALL_BLOCK_REQ = 0x09, }; /* VF/PF compatibility version. */ @@ -97,4 +101,74 @@ enum pf2vf_compat_response { ADF_PF2VF_VF_COMPAT_UNKNOWN = 0x03, }; +/* PF->VF Block Responses */ +#define ADF_PF2VF_BLKMSG_RESP_TYPE_MASK GENMASK(1, 0) +#define ADF_PF2VF_BLKMSG_RESP_DATA_MASK GENMASK(9, 2) + +enum pf2vf_blkmsg_resp_type { + ADF_PF2VF_BLKMSG_RESP_TYPE_DATA = 0x00, + ADF_PF2VF_BLKMSG_RESP_TYPE_CRC = 0x01, + ADF_PF2VF_BLKMSG_RESP_TYPE_ERROR = 0x02, +}; + +/* PF->VF Block Error Code */ +enum pf2vf_blkmsg_error { + ADF_PF2VF_INVALID_BLOCK_TYPE = 0x00, + ADF_PF2VF_INVALID_BYTE_NUM_REQ = 0x01, + ADF_PF2VF_PAYLOAD_TRUNCATED = 0x02, + ADF_PF2VF_UNSPECIFIED_ERROR = 0x03, +}; + +/* VF->PF Block Requests */ +#define ADF_VF2PF_LARGE_BLOCK_TYPE_MASK GENMASK(1, 0) +#define ADF_VF2PF_LARGE_BLOCK_BYTE_MASK GENMASK(8, 2) +#define ADF_VF2PF_MEDIUM_BLOCK_TYPE_MASK GENMASK(2, 0) +#define ADF_VF2PF_MEDIUM_BLOCK_BYTE_MASK GENMASK(8, 3) +#define ADF_VF2PF_SMALL_BLOCK_TYPE_MASK GENMASK(3, 0) +#define ADF_VF2PF_SMALL_BLOCK_BYTE_MASK GENMASK(8, 4) +#define ADF_VF2PF_BLOCK_CRC_REQ_MASK BIT(9) + +/* PF->VF Block Request Types + * 0..15 - 32 byte message + * 16..23 - 64 byte message + * 24..27 - 128 byte message + */ +/* No block messages as of yet */ + +#define ADF_VF2PF_SMALL_BLOCK_TYPE_MAX \ + (FIELD_MAX(ADF_VF2PF_SMALL_BLOCK_TYPE_MASK)) + +#define ADF_VF2PF_MEDIUM_BLOCK_TYPE_MAX \ + (FIELD_MAX(ADF_VF2PF_MEDIUM_BLOCK_TYPE_MASK) + \ + ADF_VF2PF_SMALL_BLOCK_TYPE_MAX + 1) + +#define ADF_VF2PF_LARGE_BLOCK_TYPE_MAX \ + (FIELD_MAX(ADF_VF2PF_LARGE_BLOCK_TYPE_MASK) + \ + ADF_VF2PF_MEDIUM_BLOCK_TYPE_MAX) + +#define ADF_VF2PF_SMALL_BLOCK_BYTE_MAX \ + FIELD_MAX(ADF_VF2PF_SMALL_BLOCK_BYTE_MASK) + +#define ADF_VF2PF_MEDIUM_BLOCK_BYTE_MAX \ + FIELD_MAX(ADF_VF2PF_MEDIUM_BLOCK_BYTE_MASK) + +#define ADF_VF2PF_LARGE_BLOCK_BYTE_MAX \ + FIELD_MAX(ADF_VF2PF_LARGE_BLOCK_BYTE_MASK) + +struct pfvf_blkmsg_header { + u8 version; + u8 payload_size; +} __packed; + +#define ADF_PFVF_BLKMSG_HEADER_SIZE (sizeof(struct pfvf_blkmsg_header)) +#define ADF_PFVF_BLKMSG_PAYLOAD_SIZE(blkmsg) (sizeof(blkmsg) - \ + ADF_PFVF_BLKMSG_HEADER_SIZE) +#define ADF_PFVF_BLKMSG_MSG_SIZE(blkmsg) (ADF_PFVF_BLKMSG_HEADER_SIZE + \ + (blkmsg)->hdr.payload_size) +#define ADF_PFVF_BLKMSG_MSG_MAX_SIZE 128 + +/* PF->VF Block message header bytes */ +#define ADF_PFVF_BLKMSG_VER_BYTE 0 +#define ADF_PFVF_BLKMSG_LEN_BYTE 1 + #endif /* ADF_PFVF_MSG_H */ diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_pf_msg.h b/drivers/crypto/qat/qat_common/adf_pfvf_pf_msg.h index 187807b1ff88..5c669f1de6e4 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_pf_msg.h +++ b/drivers/crypto/qat/qat_common/adf_pfvf_pf_msg.h @@ -7,4 +7,7 @@ void adf_pf2vf_notify_restarting(struct adf_accel_dev *accel_dev); +typedef int (*adf_pf2vf_blkmsg_provider)(struct adf_accel_dev *accel_dev, + u8 *buffer, u8 compat); + #endif /* ADF_PFVF_PF_MSG_H */ diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c b/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c index f461aa0a95c7..850b5f4414a6 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c @@ -6,7 +6,16 @@ #include "adf_accel_devices.h" #include "adf_common_drv.h" #include "adf_pfvf_msg.h" +#include "adf_pfvf_pf_msg.h" #include "adf_pfvf_pf_proto.h" +#include "adf_pfvf_utils.h" + +typedef u8 (*pf2vf_blkmsg_data_getter_fn)(u8 const *blkmsg, u8 byte); + +static const adf_pf2vf_blkmsg_provider pf2vf_blkmsg_providers[] = { + NULL, /* no message type defined for value 0 */ + NULL, /* no message type defined for value 1 */ +}; /** * adf_send_pf2vf_msg() - send PF to VF message @@ -44,6 +53,128 @@ static struct pfvf_message adf_recv_vf2pf_msg(struct adf_accel_dev *accel_dev, u return pfvf_ops->recv_msg(accel_dev, pfvf_offset); } +static adf_pf2vf_blkmsg_provider get_blkmsg_response_provider(u8 type) +{ + if (type >= ARRAY_SIZE(pf2vf_blkmsg_providers)) + return NULL; + + return pf2vf_blkmsg_providers[type]; +} + +/* Byte pf2vf_blkmsg_data_getter_fn callback */ +static u8 adf_pf2vf_blkmsg_get_byte(u8 const *blkmsg, u8 index) +{ + return blkmsg[index]; +} + +/* CRC pf2vf_blkmsg_data_getter_fn callback */ +static u8 adf_pf2vf_blkmsg_get_crc(u8 const *blkmsg, u8 count) +{ + /* count is 0-based, turn it into a length */ + return adf_pfvf_calc_blkmsg_crc(blkmsg, count + 1); +} + +static int adf_pf2vf_blkmsg_get_data(struct adf_accel_vf_info *vf_info, + u8 type, u8 byte, u8 max_size, u8 *data, + pf2vf_blkmsg_data_getter_fn data_getter) +{ + u8 blkmsg[ADF_PFVF_BLKMSG_MSG_MAX_SIZE] = { 0 }; + struct adf_accel_dev *accel_dev = vf_info->accel_dev; + adf_pf2vf_blkmsg_provider provider; + u8 msg_size; + + provider = get_blkmsg_response_provider(type); + + if (unlikely(!provider)) { + pr_err("QAT: No registered provider for message %d\n", type); + *data = ADF_PF2VF_INVALID_BLOCK_TYPE; + return -EINVAL; + } + + if (unlikely((*provider)(accel_dev, blkmsg, vf_info->vf_compat_ver))) { + pr_err("QAT: unknown error from provider for message %d\n", type); + *data = ADF_PF2VF_UNSPECIFIED_ERROR; + return -EINVAL; + } + + msg_size = ADF_PFVF_BLKMSG_HEADER_SIZE + blkmsg[ADF_PFVF_BLKMSG_LEN_BYTE]; + + if (unlikely(msg_size >= max_size)) { + pr_err("QAT: Invalid size %d provided for message type %d\n", + msg_size, type); + *data = ADF_PF2VF_PAYLOAD_TRUNCATED; + return -EINVAL; + } + + if (unlikely(byte >= msg_size)) { + pr_err("QAT: Out-of-bound byte number %d (msg size %d)\n", + byte, msg_size); + *data = ADF_PF2VF_INVALID_BYTE_NUM_REQ; + return -EINVAL; + } + + *data = data_getter(blkmsg, byte); + return 0; +} + +static struct pfvf_message handle_blkmsg_req(struct adf_accel_vf_info *vf_info, + struct pfvf_message req) +{ + u8 resp_type = ADF_PF2VF_BLKMSG_RESP_TYPE_ERROR; + struct pfvf_message resp = { 0 }; + u8 resp_data = 0; + u8 blk_type; + u8 blk_byte; + u8 byte_max; + + switch (req.type) { + case ADF_VF2PF_MSGTYPE_LARGE_BLOCK_REQ: + blk_type = FIELD_GET(ADF_VF2PF_LARGE_BLOCK_TYPE_MASK, req.data) + + ADF_VF2PF_MEDIUM_BLOCK_TYPE_MAX + 1; + blk_byte = FIELD_GET(ADF_VF2PF_LARGE_BLOCK_BYTE_MASK, req.data); + byte_max = ADF_VF2PF_LARGE_BLOCK_BYTE_MAX; + break; + case ADF_VF2PF_MSGTYPE_MEDIUM_BLOCK_REQ: + blk_type = FIELD_GET(ADF_VF2PF_MEDIUM_BLOCK_TYPE_MASK, req.data) + + ADF_VF2PF_SMALL_BLOCK_TYPE_MAX + 1; + blk_byte = FIELD_GET(ADF_VF2PF_MEDIUM_BLOCK_BYTE_MASK, req.data); + byte_max = ADF_VF2PF_MEDIUM_BLOCK_BYTE_MAX; + break; + case ADF_VF2PF_MSGTYPE_SMALL_BLOCK_REQ: + blk_type = FIELD_GET(ADF_VF2PF_SMALL_BLOCK_TYPE_MASK, req.data); + blk_byte = FIELD_GET(ADF_VF2PF_SMALL_BLOCK_BYTE_MASK, req.data); + byte_max = ADF_VF2PF_SMALL_BLOCK_BYTE_MAX; + break; + } + + /* Is this a request for CRC or data? */ + if (FIELD_GET(ADF_VF2PF_BLOCK_CRC_REQ_MASK, req.data)) { + dev_dbg(&GET_DEV(vf_info->accel_dev), + "BlockMsg of type %d for CRC over %d bytes received from VF%d\n", + blk_type, blk_byte, vf_info->vf_nr); + + if (!adf_pf2vf_blkmsg_get_data(vf_info, blk_type, blk_byte, + byte_max, &resp_data, + adf_pf2vf_blkmsg_get_crc)) + resp_type = ADF_PF2VF_BLKMSG_RESP_TYPE_CRC; + } else { + dev_dbg(&GET_DEV(vf_info->accel_dev), + "BlockMsg of type %d for data byte %d received from VF%d\n", + blk_type, blk_byte, vf_info->vf_nr); + + if (!adf_pf2vf_blkmsg_get_data(vf_info, blk_type, blk_byte, + byte_max, &resp_data, + adf_pf2vf_blkmsg_get_byte)) + resp_type = ADF_PF2VF_BLKMSG_RESP_TYPE_DATA; + } + + resp.type = ADF_PF2VF_MSGTYPE_BLKMSG_RESP; + resp.data = FIELD_PREP(ADF_PF2VF_BLKMSG_RESP_TYPE_MASK, resp_type) | + FIELD_PREP(ADF_PF2VF_BLKMSG_RESP_DATA_MASK, resp_data); + + return resp; +} + static int adf_handle_vf2pf_msg(struct adf_accel_dev *accel_dev, u8 vf_nr, struct pfvf_message msg, struct pfvf_message *resp) { @@ -106,6 +237,11 @@ static int adf_handle_vf2pf_msg(struct adf_accel_dev *accel_dev, u8 vf_nr, vf_info->init = false; } break; + case ADF_VF2PF_MSGTYPE_LARGE_BLOCK_REQ: + case ADF_VF2PF_MSGTYPE_MEDIUM_BLOCK_REQ: + case ADF_VF2PF_MSGTYPE_SMALL_BLOCK_REQ: + *resp = handle_blkmsg_req(vf_info, msg); + break; default: dev_dbg(&GET_DEV(accel_dev), "Unknown message from VF%d (type 0x%.4x, data: 0x%.4x)\n", @@ -147,6 +283,7 @@ bool adf_recv_and_handle_vf2pf_msg(struct adf_accel_dev *accel_dev, u32 vf_nr) */ int adf_enable_pf2vf_comms(struct adf_accel_dev *accel_dev) { + adf_pfvf_crc_init(); spin_lock_init(&accel_dev->pf.vf2pf_ints_lock); return 0; diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_utils.c b/drivers/crypto/qat/qat_common/adf_pfvf_utils.c index 494da89e2048..c5f6d77d4bb8 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_utils.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_utils.c @@ -1,11 +1,26 @@ // SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0-only) /* Copyright(c) 2021 Intel Corporation */ +#include #include #include #include "adf_accel_devices.h" #include "adf_pfvf_msg.h" #include "adf_pfvf_utils.h" +/* CRC Calculation */ +DECLARE_CRC8_TABLE(pfvf_crc8_table); +#define ADF_PFVF_CRC8_POLYNOMIAL 0x97 + +void adf_pfvf_crc_init(void) +{ + crc8_populate_msb(pfvf_crc8_table, ADF_PFVF_CRC8_POLYNOMIAL); +} + +u8 adf_pfvf_calc_blkmsg_crc(u8 const *buf, u8 buf_len) +{ + return crc8(pfvf_crc8_table, buf, buf_len, CRC8_INIT_VALUE); +} + static bool set_value_on_csr_msg(struct adf_accel_dev *accel_dev, u32 *csr_msg, u32 value, const struct pfvf_field_format *fmt) { diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_utils.h b/drivers/crypto/qat/qat_common/adf_pfvf_utils.h index 7676fdddbe26..2be048e2287b 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_utils.h +++ b/drivers/crypto/qat/qat_common/adf_pfvf_utils.h @@ -10,6 +10,9 @@ #define ADF_PFVF_MSG_ACK_DELAY_US 4 #define ADF_PFVF_MSG_ACK_MAX_DELAY_US (1 * USEC_PER_SEC) +u8 adf_pfvf_calc_blkmsg_crc(u8 const *buf, u8 buf_len); +void adf_pfvf_crc_init(void); + struct pfvf_field_format { u8 offset; u32 mask; diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c b/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c index 729c00c0d254..0fdd6b9892d3 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c @@ -1,10 +1,13 @@ // SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0-only) /* Copyright(c) 2015 - 2021 Intel Corporation */ +#include #include +#include #include #include "adf_accel_devices.h" #include "adf_common_drv.h" #include "adf_pfvf_msg.h" +#include "adf_pfvf_utils.h" #include "adf_pfvf_vf_msg.h" #include "adf_pfvf_vf_proto.h" @@ -94,6 +97,202 @@ int adf_send_vf2pf_req(struct adf_accel_dev *accel_dev, struct pfvf_message msg, return 0; } +static int adf_vf2pf_blkmsg_data_req(struct adf_accel_dev *accel_dev, bool crc, + u8 *type, u8 *data) +{ + struct pfvf_message req = { 0 }; + struct pfvf_message resp = { 0 }; + u8 blk_type; + u8 blk_byte; + u8 msg_type; + u8 max_data; + int err; + + /* Convert the block type to {small, medium, large} size category */ + if (*type <= ADF_VF2PF_SMALL_BLOCK_TYPE_MAX) { + msg_type = ADF_VF2PF_MSGTYPE_SMALL_BLOCK_REQ; + blk_type = FIELD_PREP(ADF_VF2PF_SMALL_BLOCK_TYPE_MASK, *type); + blk_byte = FIELD_PREP(ADF_VF2PF_SMALL_BLOCK_BYTE_MASK, *data); + max_data = ADF_VF2PF_SMALL_BLOCK_BYTE_MAX; + } else if (*type <= ADF_VF2PF_MEDIUM_BLOCK_TYPE_MAX) { + msg_type = ADF_VF2PF_MSGTYPE_MEDIUM_BLOCK_REQ; + blk_type = FIELD_PREP(ADF_VF2PF_MEDIUM_BLOCK_TYPE_MASK, + *type - ADF_VF2PF_SMALL_BLOCK_TYPE_MAX); + blk_byte = FIELD_PREP(ADF_VF2PF_MEDIUM_BLOCK_BYTE_MASK, *data); + max_data = ADF_VF2PF_MEDIUM_BLOCK_BYTE_MAX; + } else if (*type <= ADF_VF2PF_LARGE_BLOCK_TYPE_MAX) { + msg_type = ADF_VF2PF_MSGTYPE_LARGE_BLOCK_REQ; + blk_type = FIELD_PREP(ADF_VF2PF_LARGE_BLOCK_TYPE_MASK, + *type - ADF_VF2PF_MEDIUM_BLOCK_TYPE_MAX); + blk_byte = FIELD_PREP(ADF_VF2PF_LARGE_BLOCK_BYTE_MASK, *data); + max_data = ADF_VF2PF_LARGE_BLOCK_BYTE_MAX; + } else { + dev_err(&GET_DEV(accel_dev), "Invalid message type %u\n", *type); + return -EINVAL; + } + + /* Sanity check */ + if (*data > max_data) { + dev_err(&GET_DEV(accel_dev), + "Invalid byte %s %u for message type %u\n", + crc ? "count" : "index", *data, *type); + return -EINVAL; + } + + /* Build the block message */ + req.type = msg_type; + req.data = blk_type | blk_byte | FIELD_PREP(ADF_VF2PF_BLOCK_CRC_REQ_MASK, crc); + + err = adf_send_vf2pf_req(accel_dev, req, &resp); + if (err) + return err; + + *type = FIELD_GET(ADF_PF2VF_BLKMSG_RESP_TYPE_MASK, resp.data); + *data = FIELD_GET(ADF_PF2VF_BLKMSG_RESP_DATA_MASK, resp.data); + + return 0; +} + +static int adf_vf2pf_blkmsg_get_byte(struct adf_accel_dev *accel_dev, u8 type, + u8 index, u8 *data) +{ + int ret; + + ret = adf_vf2pf_blkmsg_data_req(accel_dev, false, &type, &index); + if (ret < 0) + return ret; + + if (unlikely(type != ADF_PF2VF_BLKMSG_RESP_TYPE_DATA)) { + dev_err(&GET_DEV(accel_dev), + "Unexpected BLKMSG response type %u, byte 0x%x\n", + type, index); + return -EFAULT; + } + + *data = index; + return 0; +} + +static int adf_vf2pf_blkmsg_get_crc(struct adf_accel_dev *accel_dev, u8 type, + u8 bytes, u8 *crc) +{ + int ret; + + /* The count of bytes refers to a length, however shift it to a 0-based + * count to avoid overflows. Thus, a request for 0 bytes is technically + * valid. + */ + --bytes; + + ret = adf_vf2pf_blkmsg_data_req(accel_dev, true, &type, &bytes); + if (ret < 0) + return ret; + + if (unlikely(type != ADF_PF2VF_BLKMSG_RESP_TYPE_CRC)) { + dev_err(&GET_DEV(accel_dev), + "Unexpected CRC BLKMSG response type %u, crc 0x%x\n", + type, bytes); + return -EFAULT; + } + + *crc = bytes; + return 0; +} + +/** + * adf_send_vf2pf_blkmsg_req() - retrieve block message + * @accel_dev: Pointer to acceleration VF device. + * @type: The block message type, see adf_pfvf_msg.h for allowed values + * @buffer: input buffer where to place the received data + * @buffer_len: buffer length as input, the amount of written bytes on output + * + * Request a message of type 'type' over the block message transport. + * This function will send the required amount block message requests and + * return the overall content back to the caller through the provided buffer. + * The buffer should be large enough to contain the requested message type, + * otherwise the response will be truncated. + * + * Return: 0 on success, error code otherwise. + */ +int adf_send_vf2pf_blkmsg_req(struct adf_accel_dev *accel_dev, u8 type, + u8 *buffer, unsigned int *buffer_len) +{ + unsigned int index; + unsigned int msg_len; + int ret; + u8 remote_crc; + u8 local_crc; + + if (unlikely(type > ADF_VF2PF_LARGE_BLOCK_TYPE_MAX)) { + dev_err(&GET_DEV(accel_dev), "Invalid block message type %d\n", + type); + return -EINVAL; + } + + if (unlikely(*buffer_len < ADF_PFVF_BLKMSG_HEADER_SIZE)) { + dev_err(&GET_DEV(accel_dev), + "Buffer size too small for a block message\n"); + return -EINVAL; + } + + ret = adf_vf2pf_blkmsg_get_byte(accel_dev, type, + ADF_PFVF_BLKMSG_VER_BYTE, + &buffer[ADF_PFVF_BLKMSG_VER_BYTE]); + if (unlikely(ret)) + return ret; + + if (unlikely(!buffer[ADF_PFVF_BLKMSG_VER_BYTE])) { + dev_err(&GET_DEV(accel_dev), + "Invalid version 0 received for block request %u", type); + return -EFAULT; + } + + ret = adf_vf2pf_blkmsg_get_byte(accel_dev, type, + ADF_PFVF_BLKMSG_LEN_BYTE, + &buffer[ADF_PFVF_BLKMSG_LEN_BYTE]); + if (unlikely(ret)) + return ret; + + if (unlikely(!buffer[ADF_PFVF_BLKMSG_LEN_BYTE])) { + dev_err(&GET_DEV(accel_dev), + "Invalid size 0 received for block request %u", type); + return -EFAULT; + } + + /* We need to pick the minimum since there is no way to request a + * specific version. As a consequence any scenario is possible: + * - PF has a newer (longer) version which doesn't fit in the buffer + * - VF expects a newer (longer) version, so we must not ask for + * bytes in excess + * - PF and VF share the same version, no problem + */ + msg_len = ADF_PFVF_BLKMSG_HEADER_SIZE + buffer[ADF_PFVF_BLKMSG_LEN_BYTE]; + msg_len = min(*buffer_len, msg_len); + + /* Get the payload */ + for (index = ADF_PFVF_BLKMSG_HEADER_SIZE; index < msg_len; index++) { + ret = adf_vf2pf_blkmsg_get_byte(accel_dev, type, index, + &buffer[index]); + if (unlikely(ret)) + return ret; + } + + ret = adf_vf2pf_blkmsg_get_crc(accel_dev, type, msg_len, &remote_crc); + if (unlikely(ret)) + return ret; + + local_crc = adf_pfvf_calc_blkmsg_crc(buffer, msg_len); + if (unlikely(local_crc != remote_crc)) { + dev_err(&GET_DEV(accel_dev), + "CRC error on msg type %d. Local %02X, remote %02X\n", + type, local_crc, remote_crc); + return -EIO; + } + + *buffer_len = msg_len; + return 0; +} + static bool adf_handle_pf2vf_msg(struct adf_accel_dev *accel_dev, struct pfvf_message msg) { @@ -104,6 +303,7 @@ static bool adf_handle_pf2vf_msg(struct adf_accel_dev *accel_dev, adf_pf2vf_handle_pf_restarting(accel_dev); return false; case ADF_PF2VF_MSGTYPE_VERSION_RESP: + case ADF_PF2VF_MSGTYPE_BLKMSG_RESP: dev_dbg(&GET_DEV(accel_dev), "Response Message received from PF (type 0x%.4x, data 0x%.4x)\n", msg.type, msg.data); @@ -135,12 +335,13 @@ bool adf_recv_and_handle_pf2vf_msg(struct adf_accel_dev *accel_dev) /** * adf_enable_vf2pf_comms() - Function enables communication from vf to pf * - * @accel_dev: Pointer to acceleration device virtual function. + * @accel_dev: Pointer to acceleration device virtual function. * * Return: 0 on success, error code otherwise. */ int adf_enable_vf2pf_comms(struct adf_accel_dev *accel_dev) { + adf_pfvf_crc_init(); adf_enable_pf2vf_interrupts(accel_dev); return adf_vf2pf_request_version(accel_dev); } diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.h b/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.h index e32d1bc3a740..f6ee9b38c0e1 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.h +++ b/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.h @@ -9,6 +9,8 @@ int adf_send_vf2pf_msg(struct adf_accel_dev *accel_dev, struct pfvf_message msg); int adf_send_vf2pf_req(struct adf_accel_dev *accel_dev, struct pfvf_message msg, struct pfvf_message *resp); +int adf_send_vf2pf_blkmsg_req(struct adf_accel_dev *accel_dev, u8 type, + u8 *buffer, unsigned int *buffer_len); int adf_enable_vf2pf_comms(struct adf_accel_dev *accel_dev); diff --git a/drivers/crypto/qat/qat_common/adf_sriov.c b/drivers/crypto/qat/qat_common/adf_sriov.c index 6366622ff8fd..971a05d62418 100644 --- a/drivers/crypto/qat/qat_common/adf_sriov.c +++ b/drivers/crypto/qat/qat_common/adf_sriov.c @@ -8,6 +8,9 @@ #include "adf_cfg.h" #include "adf_pfvf_pf_msg.h" +#define ADF_VF2PF_RATELIMIT_INTERVAL 8 +#define ADF_VF2PF_RATELIMIT_BURST 130 + static struct workqueue_struct *pf2vf_resp_wq; struct adf_pf2vf_resp { @@ -62,8 +65,8 @@ static int adf_enable_sriov(struct adf_accel_dev *accel_dev) mutex_init(&vf_info->pf2vf_lock); ratelimit_state_init(&vf_info->vf2pf_ratelimit, - DEFAULT_RATELIMIT_INTERVAL, - DEFAULT_RATELIMIT_BURST); + ADF_VF2PF_RATELIMIT_INTERVAL, + ADF_VF2PF_RATELIMIT_BURST); } /* Set Valid bits in AE Thread to PCIe Function Mapping */ From patchwork Thu Dec 16 09:13:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Chiappero X-Patchwork-Id: 524703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60F0AC43217 for ; Thu, 16 Dec 2021 09:11:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232975AbhLPJL4 (ORCPT ); Thu, 16 Dec 2021 04:11:56 -0500 Received: from mga12.intel.com ([192.55.52.136]:9697 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232990AbhLPJLv (ORCPT ); Thu, 16 Dec 2021 04:11:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1639645911; x=1671181911; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ouyMvUDAN3KXYSLCbWLt66R/lZCAu8o3pCcKUKN0fOU=; b=XnH376esNVxLlndh3VThK4T/mQUJ7/nZ09C3z4DW3qcUydLyKUJ9aFVt BzxuD1mSdiXamb+V6LDOEk8pf6eF6UJzJwLb+C38uiV6VFMfbO4IfyXV3 jTeMf6h6/tGxwTdRwdTpkf9lkSgXQTGbk+IeDuTlWyD0V85c3NAgbGYav xtI9AMkGiBvhfu75df884Iuc1V3P63atNDeSfpvHt3eYKUn8D1CD/1ABJ I7QRYOZjU5iWB5Jn01ogRSU2UssTSbe/XsPk/uZfm8nVnQ9CeVwuq9d6m QiiQJt36EkTLW9fN2KalzhA7OiIn8AmoBGpfOS1xGS5Zu/oV/jtc0iEgh Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10199"; a="219458474" X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="219458474" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2021 01:11:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="465968602" Received: from silpixa00393544.ir.intel.com ([10.237.213.118]) by orsmga006.jf.intel.com with ESMTP; 16 Dec 2021 01:11:44 -0800 From: Marco Chiappero To: herbert@gondor.apana.org.au Cc: linux-crypto@vger.kernel.org, qat-linux@intel.com, giovanni.cabiddu@intel.com, marco.chiappero@intel.com, Fiona Trahe Subject: [PATCH 18/24] crypto: qat - support fast ACKs in the PFVF protocol Date: Thu, 16 Dec 2021 09:13:28 +0000 Message-Id: <20211216091334.402420-19-marco.chiappero@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211216091334.402420-1-marco.chiappero@intel.com> References: <20211216091334.402420-1-marco.chiappero@intel.com> MIME-Version: 1.0 Organization: Intel Research and Development Ireland Ltd - Co. Reg. #308263 - Collinstown Industrial Park, Leixlip, County Kildare - Ireland Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The original design and current implementation of the PFVF protocol expects the sender to both acquire and relinquish the ownership of the shared CSR by setting and clearing the "in use" pattern on the remote half of the register when sending a message. This happens regardless of the acknowledgment of the reception, to guarantee changes, including collisions, are surely detected. However, in the case of a request that requires a response, collisions can also be detected by the lack of a reply. This can be exploited to speed up and simplify the above behaviour, letting the receiver both acknowledge the message and release the CSR in a single transaction: 1) the sender can return as soon as the message has been acknowledged 2) the receiver doesn't have to wait long before acquiring ownership of the CSR for the response message, greatly improving the overall throughput. Howerver, this improvement cannot be leveraged for fire-and-forget notifications, as it would be impossible for the sender to clearly distinguish between a collision and an ack immediately followed by a new message. This patch implements this optimization in a new version of the protocol (v3), which applies the fast-ack logic only whenever possible and guarantees backward compatibility with older versions. For requests, a new retry loop guarantees a correct behaviour. Signed-off-by: Marco Chiappero Reviewed-by: Giovanni Cabiddu Reviewed-by: Fiona Trahe --- .../crypto/qat/qat_common/adf_accel_devices.h | 2 +- drivers/crypto/qat/qat_common/adf_gen2_pfvf.c | 68 ++++++++++++++++--- drivers/crypto/qat/qat_common/adf_pfvf_msg.h | 4 +- .../crypto/qat/qat_common/adf_pfvf_pf_proto.c | 3 +- .../crypto/qat/qat_common/adf_pfvf_vf_proto.c | 44 ++++++------ 5 files changed, 91 insertions(+), 30 deletions(-) diff --git a/drivers/crypto/qat/qat_common/adf_accel_devices.h b/drivers/crypto/qat/qat_common/adf_accel_devices.h index 55e8948a8c5e..1b9d4ed03dd0 100644 --- a/drivers/crypto/qat/qat_common/adf_accel_devices.h +++ b/drivers/crypto/qat/qat_common/adf_accel_devices.h @@ -158,7 +158,7 @@ struct adf_pfvf_ops { int (*send_msg)(struct adf_accel_dev *accel_dev, struct pfvf_message msg, u32 pfvf_offset, struct mutex *csr_lock); struct pfvf_message (*recv_msg)(struct adf_accel_dev *accel_dev, - u32 pfvf_offset); + u32 pfvf_offset, u8 compat_ver); }; struct adf_hw_device_data { diff --git a/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c b/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c index feab01ec4bbb..1a9072aac2ca 100644 --- a/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c +++ b/drivers/crypto/qat/qat_common/adf_gen2_pfvf.c @@ -124,11 +124,34 @@ static bool is_legacy_user_pfvf_message(u32 msg) return !(msg & ADF_PFVF_MSGORIGIN_SYSTEM); } +static bool is_pf2vf_notification(u8 msg_type) +{ + switch (msg_type) { + case ADF_PF2VF_MSGTYPE_RESTARTING: + return true; + default: + return false; + } +} + +static bool is_vf2pf_notification(u8 msg_type) +{ + switch (msg_type) { + case ADF_VF2PF_MSGTYPE_INIT: + case ADF_VF2PF_MSGTYPE_SHUTDOWN: + return true; + default: + return false; + } +} + struct pfvf_gen2_params { u32 pfvf_offset; struct mutex *csr_lock; /* lock preventing concurrent access of CSR */ enum gen2_csr_pos local_offset; enum gen2_csr_pos remote_offset; + bool (*is_notification_message)(u8 msg_type); + u8 compat_ver; }; static int adf_gen2_pfvf_send(struct adf_accel_dev *accel_dev, @@ -190,15 +213,27 @@ static int adf_gen2_pfvf_send(struct adf_accel_dev *accel_dev, csr_val &= ~int_bit; } - if (csr_val != csr_msg) { - dev_dbg(&GET_DEV(accel_dev), - "Collision - PFVF CSR overwritten by remote function\n"); + /* For fire-and-forget notifications, the receiver does not clear + * the in-use pattern. This is used to detect collisions. + */ + if (params->is_notification_message(msg.type) && csr_val != csr_msg) { + /* Collision must have overwritten the message */ + dev_err(&GET_DEV(accel_dev), + "Collision on notification - PFVF CSR overwritten by remote function\n"); goto retry; } - /* Finished with the PFVF CSR; relinquish it and leave msg in CSR */ - gen2_csr_clear_in_use(&csr_val, remote_offset); - ADF_CSR_WR(pmisc_addr, pfvf_offset, csr_val); + /* If the far side did not clear the in-use pattern it is either + * 1) Notification - message left intact to detect collision + * 2) Older protocol (compatibility version < 3) on the far side + * where the sender is responsible for clearing the in-use + * pattern after the received has acknowledged receipt. + * In either case, clear the in-use pattern now. + */ + if (gen2_csr_is_in_use(csr_val, remote_offset)) { + gen2_csr_clear_in_use(&csr_val, remote_offset); + ADF_CSR_WR(pmisc_addr, pfvf_offset, csr_val); + } out: mutex_unlock(lock); @@ -218,6 +253,7 @@ static struct pfvf_message adf_gen2_pfvf_recv(struct adf_accel_dev *accel_dev, struct pfvf_gen2_params *params) { void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); + enum gen2_csr_pos remote_offset = params->remote_offset; enum gen2_csr_pos local_offset = params->local_offset; u32 pfvf_offset = params->pfvf_offset; struct pfvf_message msg = { 0 }; @@ -242,12 +278,22 @@ static struct pfvf_message adf_gen2_pfvf_recv(struct adf_accel_dev *accel_dev, if (unlikely(is_legacy_user_pfvf_message(csr_msg))) { dev_dbg(&GET_DEV(accel_dev), "Ignored non-system message (0x%.8x);\n", csr_val); + /* Because this must be a legacy message, the far side + * must clear the in-use pattern, so don't do it. + */ return msg; } /* Return the pfvf_message format */ msg = adf_pfvf_message_of(accel_dev, csr_msg, &csr_gen2_fmt); + /* The in-use pattern is not cleared for notifications (so that + * it can be used for collision detection) or older implementations + */ + if (params->compat_ver >= ADF_PFVF_COMPAT_FAST_ACK && + !params->is_notification_message(msg.type)) + gen2_csr_clear_in_use(&csr_val, remote_offset); + /* To ACK, clear the INT bit */ csr_val &= ~int_bit; ADF_CSR_WR(pmisc_addr, pfvf_offset, csr_val); @@ -263,6 +309,7 @@ static int adf_gen2_pf2vf_send(struct adf_accel_dev *accel_dev, struct pfvf_mess .pfvf_offset = pfvf_offset, .local_offset = ADF_GEN2_CSR_PF2VF_OFFSET, .remote_offset = ADF_GEN2_CSR_VF2PF_OFFSET, + .is_notification_message = is_pf2vf_notification, }; return adf_gen2_pfvf_send(accel_dev, msg, ¶ms); @@ -276,30 +323,35 @@ static int adf_gen2_vf2pf_send(struct adf_accel_dev *accel_dev, struct pfvf_mess .pfvf_offset = pfvf_offset, .local_offset = ADF_GEN2_CSR_VF2PF_OFFSET, .remote_offset = ADF_GEN2_CSR_PF2VF_OFFSET, + .is_notification_message = is_vf2pf_notification, }; return adf_gen2_pfvf_send(accel_dev, msg, ¶ms); } static struct pfvf_message adf_gen2_pf2vf_recv(struct adf_accel_dev *accel_dev, - u32 pfvf_offset) + u32 pfvf_offset, u8 compat_ver) { struct pfvf_gen2_params params = { .pfvf_offset = pfvf_offset, .local_offset = ADF_GEN2_CSR_PF2VF_OFFSET, .remote_offset = ADF_GEN2_CSR_VF2PF_OFFSET, + .is_notification_message = is_pf2vf_notification, + .compat_ver = compat_ver, }; return adf_gen2_pfvf_recv(accel_dev, ¶ms); } static struct pfvf_message adf_gen2_vf2pf_recv(struct adf_accel_dev *accel_dev, - u32 pfvf_offset) + u32 pfvf_offset, u8 compat_ver) { struct pfvf_gen2_params params = { .pfvf_offset = pfvf_offset, .local_offset = ADF_GEN2_CSR_VF2PF_OFFSET, .remote_offset = ADF_GEN2_CSR_PF2VF_OFFSET, + .is_notification_message = is_vf2pf_notification, + .compat_ver = compat_ver, }; return adf_gen2_pfvf_recv(accel_dev, ¶ms); diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_msg.h b/drivers/crypto/qat/qat_common/adf_pfvf_msg.h index f418dd26a742..df052194ece7 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_msg.h +++ b/drivers/crypto/qat/qat_common/adf_pfvf_msg.h @@ -89,8 +89,10 @@ enum vf2pf_msgtype { enum pfvf_compatibility_version { /* Support for extended capabilities */ ADF_PFVF_COMPAT_CAPABILITIES = 0x02, + /* In-use pattern cleared by receiver */ + ADF_PFVF_COMPAT_FAST_ACK = 0x03, /* Reference to the latest version */ - ADF_PFVF_COMPAT_THIS_VERSION = 0x02, + ADF_PFVF_COMPAT_THIS_VERSION = 0x03, }; /* PF->VF Version Response */ diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c b/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c index 1256d68c3efd..cd728d5ac9ab 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c @@ -48,10 +48,11 @@ int adf_send_pf2vf_msg(struct adf_accel_dev *accel_dev, u8 vf_nr, struct pfvf_me */ static struct pfvf_message adf_recv_vf2pf_msg(struct adf_accel_dev *accel_dev, u8 vf_nr) { + struct adf_accel_vf_info *vf_info = &accel_dev->pf.vf_info[vf_nr]; struct adf_pfvf_ops *pfvf_ops = GET_PFVF_OPS(accel_dev); u32 pfvf_offset = pfvf_ops->get_vf2pf_offset(vf_nr); - return pfvf_ops->recv_msg(accel_dev, pfvf_offset); + return pfvf_ops->recv_msg(accel_dev, pfvf_offset, vf_info->vf_compat_ver); } static adf_pf2vf_blkmsg_provider get_blkmsg_response_provider(u8 type) diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c b/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c index a85bd6dcb62a..b9a1cf5d58a9 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c @@ -15,6 +15,8 @@ #define ADF_PFVF_MSG_ACK_DELAY 2 #define ADF_PFVF_MSG_ACK_MAX_RETRY 100 +/* How often to retry if there is no response */ +#define ADF_PFVF_MSG_RESP_RETRIES 5 #define ADF_PFVF_MSG_RESP_TIMEOUT (ADF_PFVF_MSG_ACK_DELAY * \ ADF_PFVF_MSG_ACK_MAX_RETRY + \ ADF_PFVF_MSG_COLLISION_DETECT_DELAY) @@ -50,7 +52,7 @@ static struct pfvf_message adf_recv_pf2vf_msg(struct adf_accel_dev *accel_dev) struct adf_pfvf_ops *pfvf_ops = GET_PFVF_OPS(accel_dev); u32 pfvf_offset = pfvf_ops->get_pf2vf_offset(0); - return pfvf_ops->recv_msg(accel_dev, pfvf_offset); + return pfvf_ops->recv_msg(accel_dev, pfvf_offset, accel_dev->vf.pf_compat_ver); } /** @@ -68,33 +70,37 @@ int adf_send_vf2pf_req(struct adf_accel_dev *accel_dev, struct pfvf_message msg, struct pfvf_message *resp) { unsigned long timeout = msecs_to_jiffies(ADF_PFVF_MSG_RESP_TIMEOUT); + unsigned int retries = ADF_PFVF_MSG_RESP_RETRIES; int ret; reinit_completion(&accel_dev->vf.msg_received); /* Send request from VF to PF */ - ret = adf_send_vf2pf_msg(accel_dev, msg); - if (ret) { - dev_err(&GET_DEV(accel_dev), - "Failed to send request msg to PF\n"); - return ret; - } + do { + ret = adf_send_vf2pf_msg(accel_dev, msg); + if (ret) { + dev_err(&GET_DEV(accel_dev), + "Failed to send request msg to PF\n"); + return ret; + } - /* Wait for response */ - if (!wait_for_completion_timeout(&accel_dev->vf.msg_received, - timeout)) { - dev_err(&GET_DEV(accel_dev), - "PFVF request/response message timeout expired\n"); - return -EIO; - } + /* Wait for response, if it times out retry */ + ret = wait_for_completion_timeout(&accel_dev->vf.msg_received, + timeout); + if (ret) { + if (likely(resp)) + *resp = accel_dev->vf.response; - if (likely(resp)) - *resp = accel_dev->vf.response; + /* Once copied, set to an invalid value */ + accel_dev->vf.response.type = 0; - /* Once copied, set to an invalid value */ - accel_dev->vf.response.type = 0; + return 0; + } - return 0; + dev_err(&GET_DEV(accel_dev), "PFVF response message timeout\n"); + } while (--retries); + + return -EIO; } static int adf_vf2pf_blkmsg_data_req(struct adf_accel_dev *accel_dev, bool crc, From patchwork Thu Dec 16 09:13:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Chiappero X-Patchwork-Id: 524701 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A8B3C433EF for ; Thu, 16 Dec 2021 09:12:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232932AbhLPJMK (ORCPT ); Thu, 16 Dec 2021 04:12:10 -0500 Received: from mga12.intel.com ([192.55.52.136]:9714 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233020AbhLPJME (ORCPT ); Thu, 16 Dec 2021 04:12:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1639645924; x=1671181924; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=B8dblDQHwmC2tLTKguHb6fpvQ+cI//N1hj+Fdn/F2qc=; b=i+qq3rHD+kOm80EHEAaNeOJaVWemkDk/tyyeJE8FVG+b5t57H7AS2Z8y /JKafLxDF/82rAdnq4gd3IQiU+jaUyU7thx5Vxv9eVJJPYUdi7vTaOCbH xiL1pUG+bFDYHXo0464+eEZlWwdLWYbzOBUKnNTRM+2ToyyKBr5eeQgoH ANXISFLke+dGucmkOvuAkxLvt3rK7TA4Tz1iiG4Yqd9H54c1l5NPJKbp3 ZpcRVl1lDcib5GK/XEAfJkFfQ7MsXXps+UWrrLz5qpmAcz4NoPpVpmFDj qzTSHjoPIRVq2jc55YOqxWXWMWdSq7TUawRWou5QGX2hXqid+mJtlqj5B w==; X-IronPort-AV: E=McAfee;i="6200,9189,10199"; a="219458482" X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="219458482" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2021 01:11:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="465968623" Received: from silpixa00393544.ir.intel.com ([10.237.213.118]) by orsmga006.jf.intel.com with ESMTP; 16 Dec 2021 01:11:46 -0800 From: Marco Chiappero To: herbert@gondor.apana.org.au Cc: linux-crypto@vger.kernel.org, qat-linux@intel.com, giovanni.cabiddu@intel.com, marco.chiappero@intel.com, Fiona Trahe Subject: [PATCH 19/24] crypto: qat - exchange ring-to-service mappings over PFVF Date: Thu, 16 Dec 2021 09:13:29 +0000 Message-Id: <20211216091334.402420-20-marco.chiappero@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211216091334.402420-1-marco.chiappero@intel.com> References: <20211216091334.402420-1-marco.chiappero@intel.com> MIME-Version: 1.0 Organization: Intel Research and Development Ireland Ltd - Co. Reg. #308263 - Collinstown Industrial Park, Leixlip, County Kildare - Ireland Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org In addition to retrieving the device capabilities, a VF may also need to retrieve the mapping of its ring pairs to crypto and or compression services in order to work properly. Make the VF receive the ring-to-service mappings from the PF by means of a new REQ_RING_SVC_MAP Block Message and add the request and response logic on VF and PF respectively. This change requires to bump the PFVF protocol to version 4. Signed-off-by: Marco Chiappero Reviewed-by: Giovanni Cabiddu Reviewed-by: Fiona Trahe --- drivers/crypto/qat/qat_common/adf_pfvf_msg.h | 15 ++++++++++- .../crypto/qat/qat_common/adf_pfvf_pf_msg.c | 14 ++++++++++ .../crypto/qat/qat_common/adf_pfvf_pf_msg.h | 2 ++ .../crypto/qat/qat_common/adf_pfvf_pf_proto.c | 1 + .../crypto/qat/qat_common/adf_pfvf_vf_msg.c | 27 +++++++++++++++++++ .../crypto/qat/qat_common/adf_pfvf_vf_msg.h | 1 + .../crypto/qat/qat_common/adf_pfvf_vf_proto.c | 4 +++ 7 files changed, 63 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_msg.h b/drivers/crypto/qat/qat_common/adf_pfvf_msg.h index df052194ece7..1d3cad7d4999 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_msg.h +++ b/drivers/crypto/qat/qat_common/adf_pfvf_msg.h @@ -91,8 +91,10 @@ enum pfvf_compatibility_version { ADF_PFVF_COMPAT_CAPABILITIES = 0x02, /* In-use pattern cleared by receiver */ ADF_PFVF_COMPAT_FAST_ACK = 0x03, + /* Ring to service mapping support for non-standard mappings */ + ADF_PFVF_COMPAT_RING_TO_SVC_MAP = 0x04, /* Reference to the latest version */ - ADF_PFVF_COMPAT_THIS_VERSION = 0x03, + ADF_PFVF_COMPAT_THIS_VERSION = 0x04, }; /* PF->VF Version Response */ @@ -139,6 +141,7 @@ enum pf2vf_blkmsg_error { */ enum vf2pf_blkmsg_req_type { ADF_VF2PF_BLKMSG_REQ_CAP_SUMMARY = 0x02, + ADF_VF2PF_BLKMSG_REQ_RING_SVC_MAP = 0x03, }; #define ADF_VF2PF_SMALL_BLOCK_TYPE_MAX \ @@ -202,4 +205,14 @@ struct capabilities_v3 { u32 frequency; } __packed; +/* PF/VF Ring to service mapping values */ +enum blkmsg_ring_to_svc_versions { + ADF_PFVF_RING_TO_SVC_VERSION = 0x01, +}; + +struct ring_to_svc_map_v1 { + struct pfvf_blkmsg_header hdr; + u16 map; +} __packed; + #endif /* ADF_PFVF_MSG_H */ diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_pf_msg.c b/drivers/crypto/qat/qat_common/adf_pfvf_pf_msg.c index 5732cea1c7ca..14c069f0d71a 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_pf_msg.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_pf_msg.c @@ -36,3 +36,17 @@ int adf_pf_capabilities_msg_provider(struct adf_accel_dev *accel_dev, return 0; } + +int adf_pf_ring_to_svc_msg_provider(struct adf_accel_dev *accel_dev, + u8 *buffer, u8 compat) +{ + struct ring_to_svc_map_v1 rts_map_msg; + + rts_map_msg.map = accel_dev->hw_device->ring_to_svc_map; + rts_map_msg.hdr.version = ADF_PFVF_RING_TO_SVC_VERSION; + rts_map_msg.hdr.payload_size = ADF_PFVF_BLKMSG_PAYLOAD_SIZE(rts_map_msg); + + memcpy(buffer, &rts_map_msg, sizeof(rts_map_msg)); + + return 0; +} diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_pf_msg.h b/drivers/crypto/qat/qat_common/adf_pfvf_pf_msg.h index 401450bd30b0..e8982d1ac896 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_pf_msg.h +++ b/drivers/crypto/qat/qat_common/adf_pfvf_pf_msg.h @@ -12,5 +12,7 @@ typedef int (*adf_pf2vf_blkmsg_provider)(struct adf_accel_dev *accel_dev, int adf_pf_capabilities_msg_provider(struct adf_accel_dev *accel_dev, u8 *buffer, u8 comapt); +int adf_pf_ring_to_svc_msg_provider(struct adf_accel_dev *accel_dev, + u8 *buffer, u8 comapt); #endif /* ADF_PFVF_PF_MSG_H */ diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c b/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c index cd728d5ac9ab..84230aac67e6 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_pf_proto.c @@ -16,6 +16,7 @@ static const adf_pf2vf_blkmsg_provider pf2vf_blkmsg_providers[] = { NULL, /* no message type defined for value 0 */ NULL, /* no message type defined for value 1 */ adf_pf_capabilities_msg_provider, /* ADF_VF2PF_BLKMSG_REQ_CAP_SUMMARY */ + adf_pf_ring_to_svc_msg_provider, /* ADF_VF2PF_BLKMSG_REQ_RING_SVC_MAP */ }; /** diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c b/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c index b08f8544991a..14b222691c9c 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.c @@ -138,3 +138,30 @@ int adf_vf2pf_get_capabilities(struct adf_accel_dev *accel_dev) return 0; } + +int adf_vf2pf_get_ring_to_svc(struct adf_accel_dev *accel_dev) +{ + struct ring_to_svc_map_v1 rts_map_msg = { { 0 }, }; + unsigned int len = sizeof(rts_map_msg); + + if (accel_dev->vf.pf_compat_ver < ADF_PFVF_COMPAT_RING_TO_SVC_MAP) + /* Use already set default mappings */ + return 0; + + if (adf_send_vf2pf_blkmsg_req(accel_dev, ADF_VF2PF_BLKMSG_REQ_RING_SVC_MAP, + (u8 *)&rts_map_msg, &len)) { + dev_err(&GET_DEV(accel_dev), + "QAT: Failed to get block message response\n"); + return -EFAULT; + } + + if (unlikely(len < sizeof(struct ring_to_svc_map_v1))) { + dev_err(&GET_DEV(accel_dev), + "RING_TO_SVC message truncated to %d bytes\n", len); + return -EFAULT; + } + + /* Only v1 at present */ + accel_dev->hw_device->ring_to_svc_map = rts_map_msg.map; + return 0; +} diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.h b/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.h index c1f31354c138..71bc0e3f1d93 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.h +++ b/drivers/crypto/qat/qat_common/adf_pfvf_vf_msg.h @@ -8,6 +8,7 @@ int adf_vf2pf_notify_init(struct adf_accel_dev *accel_dev); void adf_vf2pf_notify_shutdown(struct adf_accel_dev *accel_dev); int adf_vf2pf_request_version(struct adf_accel_dev *accel_dev); int adf_vf2pf_get_capabilities(struct adf_accel_dev *accel_dev); +int adf_vf2pf_get_ring_to_svc(struct adf_accel_dev *accel_dev); #else static inline int adf_vf2pf_notify_init(struct adf_accel_dev *accel_dev) { diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c b/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c index b9a1cf5d58a9..0e4b8397cbe3 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c +++ b/drivers/crypto/qat/qat_common/adf_pfvf_vf_proto.c @@ -357,6 +357,10 @@ int adf_enable_vf2pf_comms(struct adf_accel_dev *accel_dev) return ret; ret = adf_vf2pf_get_capabilities(accel_dev); + if (ret) + return ret; + + ret = adf_vf2pf_get_ring_to_svc(accel_dev); return ret; } From patchwork Thu Dec 16 09:13:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Chiappero X-Patchwork-Id: 524700 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 795B2C4332F for ; Thu, 16 Dec 2021 09:12:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232972AbhLPJMM (ORCPT ); Thu, 16 Dec 2021 04:12:12 -0500 Received: from mga12.intel.com ([192.55.52.136]:9719 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232948AbhLPJMH (ORCPT ); Thu, 16 Dec 2021 04:12:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1639645927; x=1671181927; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=U6dvBEu1BouYkIMWbmnfTopexIZnXZofSykWPA1ErQg=; b=mIE8ChhzDYPWcnkQFZkNIixo4OwDAzt4kPD6YVXicbO0vT8+5Brj2TJi 3hJXrtmSTSMegjDqcS2qm2jeh+2JQQzDvDioUrKYxE58VquqEScbXypce /rcJugGpQmYigQljliNz855Vm/yW0RBHRNr+NZXMhoEFDTqsh0xNtjTQ4 CpqC90oj2avpya4Pmkcx2W1Ah8tUPyxNyq6F2wayVik3Bl13OTge1qQiR PwyzABLCM1aaOLtYTE7UqmAI6BZ9eyyrDb2luXzY5OrGys5lIh4ICwAZa 28Y9icdx6BINQ0Zu7yTtzXS2yxGBrAqtDBK8W4cCcVuHXkoMT+X9mfY3Q g==; X-IronPort-AV: E=McAfee;i="6200,9189,10199"; a="219458486" X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="219458486" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2021 01:11:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="465968657" Received: from silpixa00393544.ir.intel.com ([10.237.213.118]) by orsmga006.jf.intel.com with ESMTP; 16 Dec 2021 01:11:50 -0800 From: Marco Chiappero To: herbert@gondor.apana.org.au Cc: linux-crypto@vger.kernel.org, qat-linux@intel.com, giovanni.cabiddu@intel.com, marco.chiappero@intel.com, Fiona Trahe Subject: [PATCH 21/24] crypto: qat - add PFVF support to the GEN4 host driver Date: Thu, 16 Dec 2021 09:13:31 +0000 Message-Id: <20211216091334.402420-22-marco.chiappero@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211216091334.402420-1-marco.chiappero@intel.com> References: <20211216091334.402420-1-marco.chiappero@intel.com> MIME-Version: 1.0 Organization: Intel Research and Development Ireland Ltd - Co. Reg. #308263 - Collinstown Industrial Park, Leixlip, County Kildare - Ireland Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org So far PFVF support for GEN4 devices has been kept effectively disabled due to lack of support. This patch adds all the GEN4 specific logic to make PFVF fully functional on PF. Signed-off-by: Marco Chiappero Co-developed-by: Giovanni Cabiddu Signed-off-by: Giovanni Cabiddu Reviewed-by: Fiona Trahe --- .../crypto/qat/qat_4xxx/adf_4xxx_hw_data.c | 10 +- drivers/crypto/qat/qat_common/Makefile | 2 +- drivers/crypto/qat/qat_common/adf_gen4_pfvf.c | 148 ++++++++++++++++++ drivers/crypto/qat/qat_common/adf_gen4_pfvf.h | 17 ++ drivers/crypto/qat/qat_common/adf_pfvf_msg.h | 29 +++- 5 files changed, 196 insertions(+), 10 deletions(-) create mode 100644 drivers/crypto/qat/qat_common/adf_gen4_pfvf.c create mode 100644 drivers/crypto/qat/qat_common/adf_gen4_pfvf.h diff --git a/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c b/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c index 67cd20f443ab..ef71aa4efd64 100644 --- a/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c +++ b/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c @@ -4,6 +4,7 @@ #include #include #include +#include #include "adf_4xxx_hw_data.h" #include "icp_qat_hw.h" @@ -228,12 +229,6 @@ static u32 uof_get_ae_mask(u32 obj_num) return adf_4xxx_fw_config[obj_num].ae_mask; } -static u32 get_vf2pf_sources(void __iomem *pmisc_addr) -{ - /* For the moment do not report vf2pf sources */ - return 0; -} - void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data) { hw_data->dev_class = &adf_4xxx_class; @@ -278,12 +273,11 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data) hw_data->uof_get_ae_mask = uof_get_ae_mask; hw_data->set_msix_rttable = set_msix_default_rttable; hw_data->set_ssm_wdtimer = adf_gen4_set_ssm_wdtimer; - hw_data->pfvf_ops.enable_comms = adf_pfvf_comms_disabled; - hw_data->pfvf_ops.get_vf2pf_sources = get_vf2pf_sources; hw_data->disable_iov = adf_disable_sriov; hw_data->ring_pair_reset = adf_gen4_ring_pair_reset; adf_gen4_init_hw_csr_ops(&hw_data->csr_ops); + adf_gen4_init_pf_pfvf_ops(&hw_data->pfvf_ops); } void adf_clean_hw_data_4xxx(struct adf_hw_device_data *hw_data) diff --git a/drivers/crypto/qat/qat_common/Makefile b/drivers/crypto/qat/qat_common/Makefile index 80f6cb424753..7e191a42a5c7 100644 --- a/drivers/crypto/qat/qat_common/Makefile +++ b/drivers/crypto/qat/qat_common/Makefile @@ -22,4 +22,4 @@ intel_qat-$(CONFIG_DEBUG_FS) += adf_transport_debug.o intel_qat-$(CONFIG_PCI_IOV) += adf_sriov.o adf_vf_isr.o adf_pfvf_utils.o \ adf_pfvf_pf_msg.o adf_pfvf_pf_proto.o \ adf_pfvf_vf_msg.o adf_pfvf_vf_proto.o \ - adf_gen2_pfvf.o + adf_gen2_pfvf.o adf_gen4_pfvf.o diff --git a/drivers/crypto/qat/qat_common/adf_gen4_pfvf.c b/drivers/crypto/qat/qat_common/adf_gen4_pfvf.c new file mode 100644 index 000000000000..8efbedf63bc8 --- /dev/null +++ b/drivers/crypto/qat/qat_common/adf_gen4_pfvf.c @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0-only) +/* Copyright(c) 2021 Intel Corporation */ +#include +#include +#include +#include "adf_accel_devices.h" +#include "adf_common_drv.h" +#include "adf_gen4_pfvf.h" +#include "adf_pfvf_pf_proto.h" +#include "adf_pfvf_utils.h" + +#define ADF_4XXX_MAX_NUM_VFS 16 + +#define ADF_4XXX_PF2VM_OFFSET(i) (0x40B010 + ((i) * 0x20)) +#define ADF_4XXX_VM2PF_OFFSET(i) (0x40B014 + ((i) * 0x20)) + +/* VF2PF interrupt source registers */ +#define ADF_4XXX_VM2PF_SOU(i) (0x41A180 + ((i) * 4)) +#define ADF_4XXX_VM2PF_MSK(i) (0x41A1C0 + ((i) * 4)) +#define ADF_4XXX_VM2PF_INT_EN_MSK BIT(0) + +#define ADF_PFVF_GEN4_MSGTYPE_SHIFT 2 +#define ADF_PFVF_GEN4_MSGTYPE_MASK 0x3F +#define ADF_PFVF_GEN4_MSGDATA_SHIFT 8 +#define ADF_PFVF_GEN4_MSGDATA_MASK 0xFFFFFF + +static const struct pfvf_csr_format csr_gen4_fmt = { + { ADF_PFVF_GEN4_MSGTYPE_SHIFT, ADF_PFVF_GEN4_MSGTYPE_MASK }, + { ADF_PFVF_GEN4_MSGDATA_SHIFT, ADF_PFVF_GEN4_MSGDATA_MASK }, +}; + +static u32 adf_gen4_pf_get_pf2vf_offset(u32 i) +{ + return ADF_4XXX_PF2VM_OFFSET(i); +} + +static u32 adf_gen4_pf_get_vf2pf_offset(u32 i) +{ + return ADF_4XXX_VM2PF_OFFSET(i); +} + +static u32 adf_gen4_get_vf2pf_sources(void __iomem *pmisc_addr) +{ + int i; + u32 sou, mask; + int num_csrs = ADF_4XXX_MAX_NUM_VFS; + u32 vf_mask = 0; + + for (i = 0; i < num_csrs; i++) { + sou = ADF_CSR_RD(pmisc_addr, ADF_4XXX_VM2PF_SOU(i)); + mask = ADF_CSR_RD(pmisc_addr, ADF_4XXX_VM2PF_MSK(i)); + sou &= ~mask; + vf_mask |= sou << i; + } + + return vf_mask; +} + +static void adf_gen4_enable_vf2pf_interrupts(void __iomem *pmisc_addr, + u32 vf_mask) +{ + int num_csrs = ADF_4XXX_MAX_NUM_VFS; + unsigned long mask = vf_mask; + unsigned int val; + int i; + + for_each_set_bit(i, &mask, num_csrs) { + unsigned int offset = ADF_4XXX_VM2PF_MSK(i); + + val = ADF_CSR_RD(pmisc_addr, offset) & ~ADF_4XXX_VM2PF_INT_EN_MSK; + ADF_CSR_WR(pmisc_addr, offset, val); + } +} + +static void adf_gen4_disable_vf2pf_interrupts(void __iomem *pmisc_addr, + u32 vf_mask) +{ + int num_csrs = ADF_4XXX_MAX_NUM_VFS; + unsigned long mask = vf_mask; + unsigned int val; + int i; + + for_each_set_bit(i, &mask, num_csrs) { + unsigned int offset = ADF_4XXX_VM2PF_MSK(i); + + val = ADF_CSR_RD(pmisc_addr, offset) | ADF_4XXX_VM2PF_INT_EN_MSK; + ADF_CSR_WR(pmisc_addr, offset, val); + } +} + +static int adf_gen4_pfvf_send(struct adf_accel_dev *accel_dev, + struct pfvf_message msg, u32 pfvf_offset, + struct mutex *csr_lock) +{ + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); + u32 csr_val; + int ret; + + csr_val = adf_pfvf_csr_msg_of(accel_dev, msg, &csr_gen4_fmt); + if (unlikely(!csr_val)) + return -EINVAL; + + mutex_lock(csr_lock); + + ADF_CSR_WR(pmisc_addr, pfvf_offset, csr_val | ADF_PFVF_INT); + + /* Wait for confirmation from remote that it received the message */ + ret = read_poll_timeout(ADF_CSR_RD, csr_val, !(csr_val & ADF_PFVF_INT), + ADF_PFVF_MSG_ACK_DELAY_US, + ADF_PFVF_MSG_ACK_MAX_DELAY_US, + true, pmisc_addr, pfvf_offset); + if (ret < 0) + dev_dbg(&GET_DEV(accel_dev), "ACK not received from remote\n"); + + mutex_unlock(csr_lock); + return ret; +} + +static struct pfvf_message adf_gen4_pfvf_recv(struct adf_accel_dev *accel_dev, + u32 pfvf_offset, u8 compat_ver) +{ + void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev); + u32 csr_val; + + /* Read message from the CSR */ + csr_val = ADF_CSR_RD(pmisc_addr, pfvf_offset); + + /* We can now acknowledge the message reception by clearing the + * interrupt bit + */ + ADF_CSR_WR(pmisc_addr, pfvf_offset, csr_val & ~ADF_PFVF_INT); + + /* Return the pfvf_message format */ + return adf_pfvf_message_of(accel_dev, csr_val, &csr_gen4_fmt); +} + +void adf_gen4_init_pf_pfvf_ops(struct adf_pfvf_ops *pfvf_ops) +{ + pfvf_ops->enable_comms = adf_enable_pf2vf_comms; + pfvf_ops->get_pf2vf_offset = adf_gen4_pf_get_pf2vf_offset; + pfvf_ops->get_vf2pf_offset = adf_gen4_pf_get_vf2pf_offset; + pfvf_ops->get_vf2pf_sources = adf_gen4_get_vf2pf_sources; + pfvf_ops->enable_vf2pf_interrupts = adf_gen4_enable_vf2pf_interrupts; + pfvf_ops->disable_vf2pf_interrupts = adf_gen4_disable_vf2pf_interrupts; + pfvf_ops->send_msg = adf_gen4_pfvf_send; + pfvf_ops->recv_msg = adf_gen4_pfvf_recv; +} +EXPORT_SYMBOL_GPL(adf_gen4_init_pf_pfvf_ops); diff --git a/drivers/crypto/qat/qat_common/adf_gen4_pfvf.h b/drivers/crypto/qat/qat_common/adf_gen4_pfvf.h new file mode 100644 index 000000000000..17d1b774d4a8 --- /dev/null +++ b/drivers/crypto/qat/qat_common/adf_gen4_pfvf.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0-only) */ +/* Copyright(c) 2021 Intel Corporation */ +#ifndef ADF_GEN4_PFVF_H +#define ADF_GEN4_PFVF_H + +#include "adf_accel_devices.h" + +#ifdef CONFIG_PCI_IOV +void adf_gen4_init_pf_pfvf_ops(struct adf_pfvf_ops *pfvf_ops); +#else +static inline void adf_gen4_init_pf_pfvf_ops(struct adf_pfvf_ops *pfvf_ops) +{ + pfvf_ops->enable_comms = adf_pfvf_comms_disabled; +} +#endif + +#endif /* ADF_GEN4_PFVF_H */ diff --git a/drivers/crypto/qat/qat_common/adf_pfvf_msg.h b/drivers/crypto/qat/qat_common/adf_pfvf_msg.h index 1d3cad7d4999..f00e9e2c585b 100644 --- a/drivers/crypto/qat/qat_common/adf_pfvf_msg.h +++ b/drivers/crypto/qat/qat_common/adf_pfvf_msg.h @@ -6,7 +6,8 @@ #include /* - * PF<->VF Messaging + * PF<->VF Gen2 Messaging format + * * The PF has an array of 32-bit PF2VF registers, one for each VF. The * PF can access all these registers; each VF can access only the one * register associated with that particular VF. @@ -53,6 +54,32 @@ * respectively, the other 16 bits are written to first with a defined * IN_USE_BY pattern as part of a collision control scheme (see function * adf_gen2_pfvf_send() in adf_pf2vf_msg.c). + * + * + * PF<->VF Gen4 Messaging format + * + * Similarly to the gen2 messaging format, 32-bit long registers are used for + * communication between PF and VFs. However, each VF and PF share a pair of + * 32-bits register to avoid collisions: one for PV to VF messages and one + * for VF to PF messages. + * + * Both the Interrupt bit and the Message Origin bit retain the same position + * and meaning, although non-system messages are now deprecated and not + * expected. + * + * 31 30 9 8 7 6 5 4 3 2 1 0 + * _______________________________________________ + * | | | . . . | | | | | | | | | | | + * +-----------------------------------------------+ + * \_____________________/ \_______________/ ^ ^ + * ^ ^ | | + * | | | PF/VF Int + * | | Message Origin + * | Message Type + * Message-specific Data/Reserved + * + * For both formats, the message reception is acknowledged by lowering the + * interrupt bit on the register where the message was sent. */ /* PFVF message common bits */ From patchwork Thu Dec 16 09:13:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Chiappero X-Patchwork-Id: 524699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07357C43217 for ; Thu, 16 Dec 2021 09:12:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232948AbhLPJMN (ORCPT ); Thu, 16 Dec 2021 04:12:13 -0500 Received: from mga12.intel.com ([192.55.52.136]:9714 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232977AbhLPJMJ (ORCPT ); Thu, 16 Dec 2021 04:12:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1639645929; x=1671181929; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=z2sxZ2uQnoq1fsaRVTMa61/aLH0uOj6CDLNU41ztM24=; b=K+k7lOWAu48n1bGDubr7xJGQjakmwlxYssjHWRrGG2ruWn6OEj26ssLj Eye++RWmWxz/O7kwMV7KcJIan8kNhH9FGyUFXjPrYlQUin0wBNHX2vUZI xKehmL4d3PHz/Xm7Z7VN7oqNHGL3xPzIg/amyHJZ3J/icI9TM3sxBmICd HkyxwocWQNj6luLAImXBC4WPyTgI1eLatu8wT40QyBV2BMXIBrD5Hg0SK xQokTHwY8GOHPI4Gj+ENgC2F4SPAAzOdTs3dUvPqWti6Cu0BDPex/MOTJ Nm15uZeHImlAS2cu41QuZuTvN3fbmOOqDFhinPPvmA+K3BfEVGLDYEHhz w==; X-IronPort-AV: E=McAfee;i="6200,9189,10199"; a="219458497" X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="219458497" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2021 01:11:59 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="465968699" Received: from silpixa00393544.ir.intel.com ([10.237.213.118]) by orsmga006.jf.intel.com with ESMTP; 16 Dec 2021 01:11:56 -0800 From: Marco Chiappero To: herbert@gondor.apana.org.au Cc: linux-crypto@vger.kernel.org, qat-linux@intel.com, giovanni.cabiddu@intel.com, marco.chiappero@intel.com, Tomasz Kowalik , Mateuszx Potrola , Fiona Trahe Subject: [PATCH 24/24] crypto: qat - add support for compression for 4xxx Date: Thu, 16 Dec 2021 09:13:34 +0000 Message-Id: <20211216091334.402420-25-marco.chiappero@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211216091334.402420-1-marco.chiappero@intel.com> References: <20211216091334.402420-1-marco.chiappero@intel.com> MIME-Version: 1.0 Organization: Intel Research and Development Ireland Ltd - Co. Reg. #308263 - Collinstown Industrial Park, Leixlip, County Kildare - Ireland Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Tomasz Kowalik Add the logic required to enable the compression service for 4xxx devices. This allows to load the compression firmware image and report the appropriate compression capabilities. The firmware image selection for a given device is based on the 'ServicesEnabled' key stored in the internal configuration, which is added statically at the probe of the device according to the following rule, by default: - odd numbered devices assigned to compression services - even numbered devices assigned to crypto services In addition, restore the 'ServicesEnabled' key, if present, when SRIOV is enabled on the device. Signed-off-by: Tomasz Kowalik Co-developed-by: Mateuszx Potrola Signed-off-by: Mateuszx Potrola Co-developed-by: Giovanni Cabiddu Signed-off-by: Giovanni Cabiddu Signed-off-by: Marco Chiappero Reviewed-by: Fiona Trahe Reviewed-by: Marco Chiappero --- .../crypto/qat/qat_4xxx/adf_4xxx_hw_data.c | 79 +++++++++++++++++-- .../crypto/qat/qat_4xxx/adf_4xxx_hw_data.h | 1 + drivers/crypto/qat/qat_4xxx/adf_drv.c | 33 ++++++++ .../crypto/qat/qat_common/adf_accel_devices.h | 4 +- .../crypto/qat/qat_common/adf_accel_engine.c | 8 +- drivers/crypto/qat/qat_common/adf_cfg.c | 1 + .../crypto/qat/qat_common/adf_cfg_strings.h | 3 + drivers/crypto/qat/qat_common/adf_sriov.c | 31 +++++++- 8 files changed, 147 insertions(+), 13 deletions(-) diff --git a/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c b/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c index 40f684103b29..6d10edc40aca 100644 --- a/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c +++ b/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c @@ -2,6 +2,7 @@ /* Copyright(c) 2020 - 2021 Intel Corporation */ #include #include +#include #include #include #include @@ -13,12 +14,18 @@ struct adf_fw_config { char *obj_name; }; -static struct adf_fw_config adf_4xxx_fw_config[] = { +static struct adf_fw_config adf_4xxx_fw_cy_config[] = { {0xF0, ADF_4XXX_SYM_OBJ}, {0xF, ADF_4XXX_ASYM_OBJ}, {0x100, ADF_4XXX_ADMIN_OBJ}, }; +static struct adf_fw_config adf_4xxx_fw_dc_config[] = { + {0xF0, ADF_4XXX_DC_OBJ}, + {0xF, ADF_4XXX_DC_OBJ}, + {0x100, ADF_4XXX_ADMIN_OBJ}, +}; + /* Worker thread to service arbiter mappings */ static const u32 thrd_to_arb_map[ADF_4XXX_MAX_ACCELENGINES] = { 0x5555555, 0x5555555, 0x5555555, 0x5555555, @@ -32,6 +39,39 @@ static struct adf_hw_device_class adf_4xxx_class = { .instances = 0, }; +enum dev_services { + SVC_CY = 0, + SVC_DC, +}; + +static const char *const dev_cfg_services[] = { + [SVC_CY] = ADF_CFG_CY, + [SVC_DC] = ADF_CFG_DC, +}; + +static int get_service_enabled(struct adf_accel_dev *accel_dev) +{ + char services[ADF_CFG_MAX_VAL_LEN_IN_BYTES] = {0}; + u32 ret; + + ret = adf_cfg_get_param_value(accel_dev, ADF_GENERAL_SEC, + ADF_SERVICES_ENABLED, services); + if (ret) { + dev_err(&GET_DEV(accel_dev), + ADF_SERVICES_ENABLED " param not found\n"); + return ret; + } + + ret = match_string(dev_cfg_services, ARRAY_SIZE(dev_cfg_services), + services); + if (ret < 0) + dev_err(&GET_DEV(accel_dev), + "Invalid value of " ADF_SERVICES_ENABLED " param: %s\n", + services); + + return ret; +} + static u32 get_accel_mask(struct adf_hw_device_data *self) { return ADF_4XXX_ACCELERATORS_MASK; @@ -149,7 +189,14 @@ static u32 get_accel_cap(struct adf_accel_dev *accel_dev) capabilities_dc &= ~ICP_ACCEL_CAPABILITIES_CNV_INTEGRITY64; } - return capabilities_cy; + switch (get_service_enabled(accel_dev)) { + case SVC_CY: + return capabilities_cy; + case SVC_DC: + return capabilities_dc; + } + + return 0; } static enum dev_sku_info get_sku(struct adf_hw_device_data *self) @@ -230,17 +277,35 @@ static int adf_init_device(struct adf_accel_dev *accel_dev) static u32 uof_get_num_objs(void) { - return ARRAY_SIZE(adf_4xxx_fw_config); + BUILD_BUG_ON_MSG(ARRAY_SIZE(adf_4xxx_fw_cy_config) != + ARRAY_SIZE(adf_4xxx_fw_dc_config), + "Size mismatch between adf_4xxx_fw_*_config arrays"); + + return ARRAY_SIZE(adf_4xxx_fw_cy_config); } -static char *uof_get_name(u32 obj_num) +static char *uof_get_name(struct adf_accel_dev *accel_dev, u32 obj_num) { - return adf_4xxx_fw_config[obj_num].obj_name; + switch (get_service_enabled(accel_dev)) { + case SVC_CY: + return adf_4xxx_fw_cy_config[obj_num].obj_name; + case SVC_DC: + return adf_4xxx_fw_dc_config[obj_num].obj_name; + } + + return NULL; } -static u32 uof_get_ae_mask(u32 obj_num) +static u32 uof_get_ae_mask(struct adf_accel_dev *accel_dev, u32 obj_num) { - return adf_4xxx_fw_config[obj_num].ae_mask; + switch (get_service_enabled(accel_dev)) { + case SVC_CY: + return adf_4xxx_fw_cy_config[obj_num].ae_mask; + case SVC_DC: + return adf_4xxx_fw_dc_config[obj_num].ae_mask; + } + + return 0; } void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data) diff --git a/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.h b/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.h index a0c67752317f..12e4fb9b40ce 100644 --- a/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.h +++ b/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.h @@ -77,6 +77,7 @@ #define ADF_4XXX_FW "qat_4xxx.bin" #define ADF_4XXX_MMP "qat_4xxx_mmp.bin" #define ADF_4XXX_SYM_OBJ "qat_4xxx_sym.bin" +#define ADF_4XXX_DC_OBJ "qat_4xxx_dc.bin" #define ADF_4XXX_ASYM_OBJ "qat_4xxx_asym.bin" #define ADF_4XXX_ADMIN_OBJ "qat_4xxx_admin.bin" diff --git a/drivers/crypto/qat/qat_4xxx/adf_drv.c b/drivers/crypto/qat/qat_4xxx/adf_drv.c index 71ef065914b2..a6c78b9c730b 100644 --- a/drivers/crypto/qat/qat_4xxx/adf_drv.c +++ b/drivers/crypto/qat/qat_4xxx/adf_drv.c @@ -29,6 +29,29 @@ static void adf_cleanup_accel(struct adf_accel_dev *accel_dev) adf_devmgr_rm_dev(accel_dev, NULL); } +static int adf_cfg_dev_init(struct adf_accel_dev *accel_dev) +{ + const char *config; + int ret; + + config = accel_dev->accel_id % 2 ? ADF_CFG_DC : ADF_CFG_CY; + + ret = adf_cfg_section_add(accel_dev, ADF_GENERAL_SEC); + if (ret) + return ret; + + /* Default configuration is crypto only for even devices + * and compression for odd devices + */ + ret = adf_cfg_add_key_value_param(accel_dev, ADF_GENERAL_SEC, + ADF_SERVICES_ENABLED, config, + ADF_STR); + if (ret) + return ret; + + return 0; +} + static int adf_crypto_dev_config(struct adf_accel_dev *accel_dev) { char key[ADF_CFG_MAX_KEY_LEN_IN_BYTES]; @@ -227,8 +250,18 @@ static int adf_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto out_err; } + ret = adf_cfg_dev_init(accel_dev); + if (ret) { + dev_err(&pdev->dev, "Failed to initialize configuration.\n"); + goto out_err; + } + /* Get accelerator capabilities mask */ hw_data->accel_capabilities_mask = hw_data->get_accel_cap(accel_dev); + if (!hw_data->accel_capabilities_mask) { + dev_err(&pdev->dev, "Failed to get capabilities mask.\n"); + goto out_err; + } /* Find and map all the device's BARS */ bar_mask = pci_select_bars(pdev, IORESOURCE_MEM) & ADF_4XXX_BAR_MASK; diff --git a/drivers/crypto/qat/qat_common/adf_accel_devices.h b/drivers/crypto/qat/qat_common/adf_accel_devices.h index 1b9d4ed03dd0..2d4cd7c7cf33 100644 --- a/drivers/crypto/qat/qat_common/adf_accel_devices.h +++ b/drivers/crypto/qat/qat_common/adf_accel_devices.h @@ -192,9 +192,9 @@ struct adf_hw_device_data { int (*ring_pair_reset)(struct adf_accel_dev *accel_dev, u32 bank_nr); void (*reset_device)(struct adf_accel_dev *accel_dev); void (*set_msix_rttable)(struct adf_accel_dev *accel_dev); - char *(*uof_get_name)(u32 obj_num); + char *(*uof_get_name)(struct adf_accel_dev *accel_dev, u32 obj_num); u32 (*uof_get_num_objs)(void); - u32 (*uof_get_ae_mask)(u32 obj_num); + u32 (*uof_get_ae_mask)(struct adf_accel_dev *accel_dev, u32 obj_num); struct adf_pfvf_ops pfvf_ops; struct adf_hw_csr_ops csr_ops; const char *fw_name; diff --git a/drivers/crypto/qat/qat_common/adf_accel_engine.c b/drivers/crypto/qat/qat_common/adf_accel_engine.c index ca4eae8cdd0b..4ce2b666929e 100644 --- a/drivers/crypto/qat/qat_common/adf_accel_engine.c +++ b/drivers/crypto/qat/qat_common/adf_accel_engine.c @@ -22,8 +22,12 @@ static int adf_ae_fw_load_images(struct adf_accel_dev *accel_dev, void *fw_addr, num_objs = hw_device->uof_get_num_objs(); for (i = 0; i < num_objs; i++) { - obj_name = hw_device->uof_get_name(i); - ae_mask = hw_device->uof_get_ae_mask(i); + obj_name = hw_device->uof_get_name(accel_dev, i); + ae_mask = hw_device->uof_get_ae_mask(accel_dev, i); + if (!obj_name || !ae_mask) { + dev_err(&GET_DEV(accel_dev), "Invalid UOF image\n"); + goto out_err; + } if (qat_uclo_set_cfg_ae_mask(loader, ae_mask)) { dev_err(&GET_DEV(accel_dev), diff --git a/drivers/crypto/qat/qat_common/adf_cfg.c b/drivers/crypto/qat/qat_common/adf_cfg.c index 575b6f002303..b5b208cbe5a1 100644 --- a/drivers/crypto/qat/qat_common/adf_cfg.c +++ b/drivers/crypto/qat/qat_common/adf_cfg.c @@ -297,3 +297,4 @@ int adf_cfg_get_param_value(struct adf_accel_dev *accel_dev, up_read(&cfg->lock); return ret; } +EXPORT_SYMBOL_GPL(adf_cfg_get_param_value); diff --git a/drivers/crypto/qat/qat_common/adf_cfg_strings.h b/drivers/crypto/qat/qat_common/adf_cfg_strings.h index 09651e1f937a..655248dbf962 100644 --- a/drivers/crypto/qat/qat_common/adf_cfg_strings.h +++ b/drivers/crypto/qat/qat_common/adf_cfg_strings.h @@ -22,6 +22,9 @@ #define ADF_RING_ASYM_BANK_NUM "BankAsymNumber" #define ADF_CY "Cy" #define ADF_DC "Dc" +#define ADF_CFG_DC "dc" +#define ADF_CFG_CY "sym;asym" +#define ADF_SERVICES_ENABLED "ServicesEnabled" #define ADF_ETRMGR_COALESCING_ENABLED "InterruptCoalescingEnabled" #define ADF_ETRMGR_COALESCING_ENABLED_FORMAT \ ADF_ETRMGR_BANK "%d" ADF_ETRMGR_COALESCING_ENABLED diff --git a/drivers/crypto/qat/qat_common/adf_sriov.c b/drivers/crypto/qat/qat_common/adf_sriov.c index 971a05d62418..b960bca1f9d2 100644 --- a/drivers/crypto/qat/qat_common/adf_sriov.c +++ b/drivers/crypto/qat/qat_common/adf_sriov.c @@ -126,6 +126,32 @@ void adf_disable_sriov(struct adf_accel_dev *accel_dev) } EXPORT_SYMBOL_GPL(adf_disable_sriov); +static int adf_sriov_prepare_restart(struct adf_accel_dev *accel_dev) +{ + char services[ADF_CFG_MAX_VAL_LEN_IN_BYTES] = {0}; + int ret; + + ret = adf_cfg_get_param_value(accel_dev, ADF_GENERAL_SEC, + ADF_SERVICES_ENABLED, services); + + adf_dev_stop(accel_dev); + adf_dev_shutdown(accel_dev); + + if (!ret) { + ret = adf_cfg_section_add(accel_dev, ADF_GENERAL_SEC); + if (ret) + return ret; + + ret = adf_cfg_add_key_value_param(accel_dev, ADF_GENERAL_SEC, + ADF_SERVICES_ENABLED, + services, ADF_STR); + if (ret) + return ret; + } + + return 0; +} + /** * adf_sriov_configure() - Enable SRIOV for the device * @pdev: Pointer to PCI device. @@ -165,8 +191,9 @@ int adf_sriov_configure(struct pci_dev *pdev, int numvfs) return -EBUSY; } - adf_dev_stop(accel_dev); - adf_dev_shutdown(accel_dev); + ret = adf_sriov_prepare_restart(accel_dev); + if (ret) + return ret; } if (adf_cfg_section_add(accel_dev, ADF_KERNEL_SEC))