From patchwork Tue Oct 29 06:40:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhangfei Gao X-Patchwork-Id: 177992 Delivered-To: patch@linaro.org Received: by 2002:a92:409a:0:0:0:0:0 with SMTP id d26csp4426028ill; Mon, 28 Oct 2019 23:41:17 -0700 (PDT) X-Google-Smtp-Source: APXvYqw0hdlvBsgvLDCCm8bOYysTAsSl3XQ54cjxNkgIk81PYEbH9vnDC1oCOArMHuESPA443zY9 X-Received: by 2002:a17:906:66d2:: with SMTP id k18mr1606217ejp.278.1572331277663; Mon, 28 Oct 2019 23:41:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1572331277; cv=none; d=google.com; s=arc-20160816; b=mV+DcuBmV5XWH2+dXukZZXVdenWLZN6OcQSWzNqzgWnUZ+heZrBKCrSEiPFUgEnzJS nJ0pSZ1+fthNaccgtARd5QbloMyvDEMPyuscpW/tj9yRmz/zNkU/FrioVRE0NWSxC+rL 9swHGpo2+GjCrbJ0PxHtmB9OPjG6KiBgPgBhA2l+VEUzxJMrL0I2UcombzkI5KeAKa7r hgwzBpquHqYyEGUvWd8xQecMdntJBO59SdORXjlZRDMxW/fGyKhrmb3vpwQRRlvnkzsp QeBLCK5ozMWrHF8FlFYcdADJKKB53JBJGzOLqjvhDcF5tSmJhi9ktRU+OVxMfSkA7jIe KsIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=KIkOGoEfD3sC7Lm1oZeLGLIevyyonnKw4CrTMTarj0A=; b=iN1h/oT5ZCaYETxqm3AS/IwdhxstkWc58qyepe6hp0XlMf6+wo3uQZ5BeZHGvSPhQq /pF615oOFopWzNm6Z4XVRqhcMsle4sr2gVo/T6Tcrq9dFVnacHw366CA8srEy/sDdjku NVUWpe5dm6SLqAu+vnMkGEzdJXOdEWCP7yqOe6oWFD7fh0o1XoNKXgdVlGwlzo/+eQan 4FmU2UnrLpgvoZtU6YUyBHoSiU/RQz7EJutxXng71m/zpX+o5Ap54FyksxB+tzAKfVuh U8sc0694Wqjjs6/pT+aF9AsfOEheML8zcTxzArdDOw7aM+AXX6XzIvINmqyUl/vhLcLI eiVg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bIjXxvDe; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b2si7862971eja.89.2019.10.28.23.41.17; Mon, 28 Oct 2019 23:41:17 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bIjXxvDe; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732589AbfJ2GlJ (ORCPT + 3 others); Tue, 29 Oct 2019 02:41:09 -0400 Received: from mail-pl1-f193.google.com ([209.85.214.193]:36563 "EHLO mail-pl1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732576AbfJ2GlJ (ORCPT ); Tue, 29 Oct 2019 02:41:09 -0400 Received: by mail-pl1-f193.google.com with SMTP id g9so6471448plp.3 for ; Mon, 28 Oct 2019 23:41:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=KIkOGoEfD3sC7Lm1oZeLGLIevyyonnKw4CrTMTarj0A=; b=bIjXxvDeD2rWDSR6i985axak0TB8AlPFUcMAra8oKx8XhpAMLYgs03sHaaSUpqLxjJ GanF9IeQ9rcB7++fygVqlfUnt37XvO3LrlfTsw+0IgOkYY0Pa2fjK90vCRoJOVn1w+vK mQtOxJWYZctxQDXBOkOfslpDMwzIwsaydaFyUlhIZFLgSRqMbojBJ8PylPshovYPpXXk E7S4r9vuutLLOnm3PZ6H1kJiYGP+LrLZytPYO3UI17Itj1KFpChgA2MDBGHEZ6iYxRm7 RGSeHUEWyoNCcuJXshHSghG8Ge1Uw8MAi2hWEwZZFwTdA1b+UebAeRjOt4/luSzAKrnn GssA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=KIkOGoEfD3sC7Lm1oZeLGLIevyyonnKw4CrTMTarj0A=; b=N7VovGdL57GmlxqDZE0UVYbmu0pnG6aHSuZ7ZOwLeuutg6CeOggiQjNSYrvcThRY47 jVKLP79Smm5uefyUFfzwL3TEmnqDp44JDM7mdVnimO5/B3FD9LkfPiqwtN+/mJYva4hc L+3zmw38xSwwnLQ5sc+YZrQdPFmb/mAVdlJ8tmHplwwY9vtyKgfiTVacVSwOg7IIiYHS DiDx9pXel2vK0YmY5K5Irsl//yd497Twxi+nhYWpNSdo4QzAPDba5fB9C2YFGymozIgJ 3LWT2usVCcVdPo+8YGm0nOjgBRQK9UG4+Ht6YSYszTvMi3obUptrROsHOasiXg5puYDf XM0Q== X-Gm-Message-State: APjAAAU1TGOs4h13jS5I2Asmipbj72DRR54QHPanx77BZ6DCaO3Xa/sH nFthtCHmdVa4YNt8xt1M7ertFA== X-Received: by 2002:a17:902:8647:: with SMTP id y7mr2069000plt.75.1572331266831; Mon, 28 Oct 2019 23:41:06 -0700 (PDT) Received: from localhost.localdomain ([240e:362:4dc:3a00:892e:70f7:f486:8f02]) by smtp.gmail.com with ESMTPSA id e23sm13421834pgh.84.2019.10.28.23.40.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Oct 2019 23:41:06 -0700 (PDT) From: Zhangfei Gao To: Greg Kroah-Hartman , Arnd Bergmann , Herbert Xu , jonathan.cameron@huawei.com, grant.likely@arm.com, jean-philippe , Jerome Glisse , ilias.apalodimas@linaro.org, francois.ozog@linaro.org, kenneth-lee-2012@foxmail.com, Wangzhou , "haojian . zhuang" , guodong.xu@linaro.org Cc: linux-accelerators@lists.ozlabs.org, linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, iommu@lists.linux-foundation.org, Kenneth Lee , Zaibo Xu , Zhangfei Gao Subject: [PATCH v7 1/3] uacce: Add documents for uacce Date: Tue, 29 Oct 2019 14:40:14 +0800 Message-Id: <1572331216-9503-2-git-send-email-zhangfei.gao@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1572331216-9503-1-git-send-email-zhangfei.gao@linaro.org> References: <1572331216-9503-1-git-send-email-zhangfei.gao@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Kenneth Lee Uacce (Unified/User-space-access-intended Accelerator Framework) is a kernel module targets to provide Shared Virtual Addressing (SVA) between the accelerator and process. This patch add document to explain how it works. Signed-off-by: Kenneth Lee Signed-off-by: Zaibo Xu Signed-off-by: Zhou Wang Signed-off-by: Zhangfei Gao --- Documentation/misc-devices/uacce.rst | 160 +++++++++++++++++++++++++++++++++++ 1 file changed, 160 insertions(+) create mode 100644 Documentation/misc-devices/uacce.rst -- 2.7.4 diff --git a/Documentation/misc-devices/uacce.rst b/Documentation/misc-devices/uacce.rst new file mode 100644 index 0000000..ecd5d8b --- /dev/null +++ b/Documentation/misc-devices/uacce.rst @@ -0,0 +1,160 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Introduction of Uacce +========================= + +Uacce (Unified/User-space-access-intended Accelerator Framework) targets to +provide Shared Virtual Addressing (SVA) between accelerators and processes. +So accelerator can access any data structure of the main cpu. +This differs from the data sharing between cpu and io device, which share +data content rather than address. +Because of the unified address, hardware and user space of process can +share the same virtual address in the communication. +Uacce takes the hardware accelerator as a heterogeneous processor, while +IOMMU share the same CPU page tables and as a result the same translation +from va to pa. + + __________________________ __________________________ + | | | | + | User application (CPU) | | Hardware Accelerator | + |__________________________| |__________________________| + + | | + | va | va + V V + __________ __________ + | | | | + | MMU | | IOMMU | + |__________| |__________| + | | + | | + V pa V pa + _______________________________________ + | | + | Memory | + |_______________________________________| + + + +Architecture +------------ + +Uacce is the kernel module, taking charge of iommu and address sharing. +The user drivers and libraries are called WarpDrive. + +The uacce device, built around the IOMMU SVA API, can access multiple +address spaces, including the one without PASID. + +A virtual concept, queue, is used for the communication. It provides a +FIFO-like interface. And it maintains a unified address space between the +application and all involved hardware. + + ___________________ ________________ + | | user API | | + | WarpDrive library | ------------> | user driver | + |___________________| |________________| + | | + | | + | queue fd | + | | + | | + v | + ___________________ _________ | + | | | | | mmap memory + | Other framework | | uacce | | r/w interface + | crypto/nic/others | |_________| | + |___________________| | + | | | + | register | register | + | | | + | | | + | _________________ __________ | + | | | | | | + ------------- | Device Driver | | IOMMU | | + |_________________| |__________| | + | | + | V + | ___________________ + | | | + -------------------------- | Device(Hardware) | + |___________________| + + +How does it work +================ + +Uacce uses mmap and IOMMU to play the trick. + +Uacce create a chrdev for every device registered to it. New queue is +created when user application open the chrdev. The file descriptor is used +as the user handle of the queue. +The accelerator device present itself as an Uacce object, which exports as +chrdev to the user space. The user application communicates with the +hardware by ioctl (as control path) or share memory (as data path). + +The control path to the hardware is via file operation, while data path is +via mmap space of the queue fd. + +The queue file address space: +/** + * enum uacce_qfrt: qfrt type + * @UACCE_QFRT_MMIO: device mmio region + * @UACCE_QFRT_DUS: device user share region + */ +enum uacce_qfrt { + UACCE_QFRT_MMIO = 0, + UACCE_QFRT_DUS = 1, +}; + +All regions are optional and differ from device type to type. The +communication protocol is wrapped by the user driver. + +The device mmio region is mapped to the hardware mmio space. It is generally +used for doorbell or other notification to the hardware. It is not fast enough +as data channel. + +The device user share region is used for share data buffer between user process +and device. + + +The Uacce register API +----------------------- +The register API is defined in uacce.h. + +struct uacce_interface { + char name[UACCE_MAX_NAME_SIZE]; + enum uacce_dev_flag flags; + struct uacce_ops *ops; +}; + +According to the IOMMU capability, uacce_interface flags can be: + +/** + * enum uacce_dev_flag: Device flags: + * @UACCE_DEV_SVA: Shared Virtual Addresses + * Support PASID + * Support device page faults (PCI PRI or SMMU Stall) + */ +enum uacce_dev_flag { + UACCE_DEV_SVA = BIT(0), +}; + +struct uacce_device *uacce_register(struct device *parent, + struct uacce_interface *interface); +void uacce_unregister(struct uacce_device *uacce); + +uacce_register results can be: +a. If uacce module is not compiled, ERR_PTR(-ENODEV) +b. Succeed with the desired flags +c. Succeed with the negotiated flags, for example + uacce_interface.flags = UACCE_DEV_SVA but uacce->flags = ~UACCE_DEV_SVA +So user driver need check return value as well as the negotiated uacce->flags. + + +The user driver +--------------- + +The queue file mmap space will need a user driver to wrap the communication +protocol. Uacce provides some attributes in sysfs for the user driver to +match the right accelerator accordingly. +More details in Documentation/ABI/testing/sysfs-driver-uacce. From patchwork Tue Oct 29 06:40:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhangfei Gao X-Patchwork-Id: 177994 Delivered-To: patch@linaro.org Received: by 2002:a92:409a:0:0:0:0:0 with SMTP id d26csp4426457ill; Mon, 28 Oct 2019 23:41:45 -0700 (PDT) X-Google-Smtp-Source: APXvYqwnYu0RNp6PH2u3SZZ5jJZYyr9xpJXxuiouiM6GYsGuq4McykUYIBoxepYjATWexEKOhAGA X-Received: by 2002:a17:906:85d3:: with SMTP id i19mr1592015ejy.69.1572331305462; Mon, 28 Oct 2019 23:41:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1572331305; cv=none; d=google.com; s=arc-20160816; b=Kr0ZqtrsUsJCiWbT+hObcUfJpoct5kdMW8JHyXkj7l3JE3qEbzlI0UO6TwbD/SupOW QBFt6Wagb1iIgh0jE6gbX84XQo1Ff8c5Ud/zZBgwqoLk+krt57rpjHUh502rKlLSwd6c pe65hd5w1gHNYHa9g3GMXXacvmeYj7Qrzy8F4/nw73g0mXaVYY0qGmryCqejeJ2q1JcO Iftv5afIKTdHyeiipVyTDhYCQ9m4Ph5w75bwRuR3ln+U9U6acc86eqC9GuVrFaKf8Mm+ ziUiQ/TC7gIaZCOuhSAd168ReStgpnjRopgSs8RVEeBFIWyC0kTpwDhIR6EenEun3nZf RLsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=dekWozQpi8Yds4BtBBFvE6aNgm93yn3hRX+1DVnhSXA=; b=LceGnt5nHWaoBrR2IOR6zi9vRuC5buud+QgHtVEOYf8WVH53pbdoUxKsqIljXxRJPr 7NDc+UPFz5rGe7Pu2tz1cBHmYddG1OroGMll4dhEDHAhcFHFMUGmGXm3WyrgpeqQha24 20JXhgasRGdLWJk+rJMiuvwYpLz3lBwxHl/zufrmTPZ/vgrkZUS3K5ZtV/UedHcYFryp AUPiKrBTG4tTik2rsIJy0KphOvkbxs8G9oEcMUFYtlDFkl3QbUE7RiauosVmnnMBOLi1 zrZMIxdp0uYxhuOUPrLLfwmezOQG9J+CR7ygD+JJqTqWS9QnrpjZzG4U1iEFv+yCri69 QgTg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TxXdsa37; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v20si9094991edm.254.2019.10.28.23.41.45; Mon, 28 Oct 2019 23:41:45 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TxXdsa37; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732652AbfJ2Glj (ORCPT + 3 others); Tue, 29 Oct 2019 02:41:39 -0400 Received: from mail-pf1-f195.google.com ([209.85.210.195]:40226 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732630AbfJ2Gli (ORCPT ); Tue, 29 Oct 2019 02:41:38 -0400 Received: by mail-pf1-f195.google.com with SMTP id r4so3154964pfl.7 for ; Mon, 28 Oct 2019 23:41:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dekWozQpi8Yds4BtBBFvE6aNgm93yn3hRX+1DVnhSXA=; b=TxXdsa37VU48AW/7JNoNomAxHsjllorHtXzFiL55rXM8tizApaPdseiN4jRvWjh4hu 28njhgiqoINAZoT7iPK9J/uRgZsbDr9yfIp5ka2cak6LUudAOwPJgIcqmhv0F0hPzOkc P7DSdvH53puHBvbTXDDmMvOCq+RQRq+4QCeUPQdDUwkKQn5mOnGRux7Lw1+yJKxIMDAI qFTpQrsX/j0x4JMGE5Gz69GoeYz9rF0e1WZiBYJNfjbmsSlCSC2m5/x+cmuL7pLTYIB3 9nUPutB2IvHv4Y6Zq6aZIIV4/bp5l3/K0c3xrJc/1WLd8Ro6cYFJ4vIkJ7yrKtmPCckZ FhXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dekWozQpi8Yds4BtBBFvE6aNgm93yn3hRX+1DVnhSXA=; b=egM7jlvL3F8ibhNAEfy1BRASzqRuGuyL7ntGFANdjE+VmkvIC2jebt9sQq3F2SPP2i PkBwcxwIiGmAvtWVxEh6Ep1dpbWGsQlaFaPOQ2auE+jsEIShPxdN5r5xtou5ge4Gp7NJ SpWso7M7M90OuRBvZXrDQEe6V0+FMGjIbRQgBnvEvWHU69/PToo0hWPA+gAyXLyclwBM 1kAcplIWwD+xO+3XUFfnqCUCqS2gHvZHy3KoWheUdEjL6RiEYz6QipZnmeVNgEUsMj6B AwBqudvS1Q3TmeJdpbQzNac4kXPCn0q1rNqDUhc+xrK4fAQRq8AwpC3TSTzyRf6BVNNj flJw== X-Gm-Message-State: APjAAAUaKYaDV9PMCinJbAt0vH2ydUcmnAKPPepuGNe8P0ng3WS8nFFC TaGwGFISzRKrJNsZBXZmfhBosQ== X-Received: by 2002:a17:90a:25e1:: with SMTP id k88mr4361121pje.14.1572331298008; Mon, 28 Oct 2019 23:41:38 -0700 (PDT) Received: from localhost.localdomain ([240e:362:4dc:3a00:892e:70f7:f486:8f02]) by smtp.gmail.com with ESMTPSA id e23sm13421834pgh.84.2019.10.28.23.41.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Oct 2019 23:41:37 -0700 (PDT) From: Zhangfei Gao To: Greg Kroah-Hartman , Arnd Bergmann , Herbert Xu , jonathan.cameron@huawei.com, grant.likely@arm.com, jean-philippe , Jerome Glisse , ilias.apalodimas@linaro.org, francois.ozog@linaro.org, kenneth-lee-2012@foxmail.com, Wangzhou , "haojian . zhuang" , guodong.xu@linaro.org Cc: linux-accelerators@lists.ozlabs.org, linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, iommu@lists.linux-foundation.org, Zhangfei Gao Subject: [PATCH v7 3/3] crypto: hisilicon - register zip engine to uacce Date: Tue, 29 Oct 2019 14:40:16 +0800 Message-Id: <1572331216-9503-4-git-send-email-zhangfei.gao@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1572331216-9503-1-git-send-email-zhangfei.gao@linaro.org> References: <1572331216-9503-1-git-send-email-zhangfei.gao@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Register qm to uacce framework for user crypto driver Signed-off-by: Zhangfei Gao Signed-off-by: Zhou Wang --- drivers/crypto/hisilicon/qm.c | 253 ++++++++++++++++++++++++++++++-- drivers/crypto/hisilicon/qm.h | 13 +- drivers/crypto/hisilicon/zip/zip_main.c | 39 ++--- include/uapi/misc/uacce/qm.h | 23 +++ 4 files changed, 292 insertions(+), 36 deletions(-) create mode 100644 include/uapi/misc/uacce/qm.h -- 2.7.4 diff --git a/drivers/crypto/hisilicon/qm.c b/drivers/crypto/hisilicon/qm.c index a8ed6990..4b9cced 100644 --- a/drivers/crypto/hisilicon/qm.c +++ b/drivers/crypto/hisilicon/qm.c @@ -9,6 +9,9 @@ #include #include #include +#include +#include +#include #include "qm.h" /* eq/aeq irq enable */ @@ -465,17 +468,22 @@ static void qm_cq_head_update(struct hisi_qp *qp) static void qm_poll_qp(struct hisi_qp *qp, struct hisi_qm *qm) { - struct qm_cqe *cqe = qp->cqe + qp->qp_status.cq_head; - - if (qp->req_cb) { - while (QM_CQE_PHASE(cqe) == qp->qp_status.cqc_phase) { - dma_rmb(); - qp->req_cb(qp, qp->sqe + qm->sqe_size * cqe->sq_head); - qm_cq_head_update(qp); - cqe = qp->cqe + qp->qp_status.cq_head; - qm_db(qm, qp->qp_id, QM_DOORBELL_CMD_CQ, - qp->qp_status.cq_head, 0); - atomic_dec(&qp->qp_status.used); + struct qm_cqe *cqe; + + if (qp->event_cb) { + qp->event_cb(qp); + } else { + cqe = qp->cqe + qp->qp_status.cq_head; + + if (qp->req_cb) { + while (QM_CQE_PHASE(cqe) == qp->qp_status.cqc_phase) { + dma_rmb(); + qp->req_cb(qp, qp->sqe + qm->sqe_size * + cqe->sq_head); + qm_cq_head_update(qp); + cqe = qp->cqe + qp->qp_status.cq_head; + atomic_dec(&qp->qp_status.used); + } } /* set c_flag */ @@ -1397,6 +1405,220 @@ static void hisi_qm_cache_wb(struct hisi_qm *qm) } } +static void qm_qp_event_notifier(struct hisi_qp *qp) +{ + wake_up_interruptible(&qp->uacce_q->wait); +} + +static int hisi_qm_get_available_instances(struct uacce_device *uacce) +{ + int i, ret; + struct hisi_qm *qm = uacce->priv; + + read_lock(&qm->qps_lock); + for (i = 0, ret = 0; i < qm->qp_num; i++) + if (!qm->qp_array[i]) + ret++; + read_unlock(&qm->qps_lock); + + return ret; +} + +static int hisi_qm_uacce_get_queue(struct uacce_device *uacce, + unsigned long arg, + struct uacce_queue *q) +{ + struct hisi_qm *qm = uacce->priv; + struct hisi_qp *qp; + u8 alg_type = 0; + + qp = hisi_qm_create_qp(qm, alg_type); + if (IS_ERR(qp)) + return PTR_ERR(qp); + + q->priv = qp; + q->uacce = uacce; + qp->uacce_q = q; + qp->event_cb = qm_qp_event_notifier; + qp->pasid = arg; + + return 0; +} + +static void hisi_qm_uacce_put_queue(struct uacce_queue *q) +{ + struct hisi_qp *qp = q->priv; + + /* + * As put_queue is only called in uacce_mode=1, and only one queue can + * be used in this mode. we flush all sqc cache back in put queue. + */ + hisi_qm_cache_wb(qp->qm); + + /* need to stop hardware, but can not support in v1 */ + hisi_qm_release_qp(qp); +} + +/* map sq/cq/doorbell to user space */ +static int hisi_qm_uacce_mmap(struct uacce_queue *q, + struct vm_area_struct *vma, + struct uacce_qfile_region *qfr) +{ + struct hisi_qp *qp = q->priv; + struct hisi_qm *qm = qp->qm; + size_t sz = vma->vm_end - vma->vm_start; + struct pci_dev *pdev = qm->pdev; + struct device *dev = &pdev->dev; + unsigned long vm_pgoff; + int ret; + + switch (qfr->type) { + case UACCE_QFRT_MMIO: + if (qm->ver == QM_HW_V2) { + if (sz > PAGE_SIZE * (QM_DOORBELL_PAGE_NR + + QM_DOORBELL_SQ_CQ_BASE_V2 / PAGE_SIZE)) + return -EINVAL; + } else { + if (sz > PAGE_SIZE * QM_DOORBELL_PAGE_NR) + return -EINVAL; + } + + vma->vm_flags |= VM_IO; + + return remap_pfn_range(vma, vma->vm_start, + qm->phys_base >> PAGE_SHIFT, + sz, pgprot_noncached(vma->vm_page_prot)); + case UACCE_QFRT_DUS: + if (sz != qp->qdma.size) + return -EINVAL; + + /* dma_mmap_coherent() requires vm_pgoff as 0 + * restore vm_pfoff to initial value for mmap() + */ + vm_pgoff = vma->vm_pgoff; + vma->vm_pgoff = 0; + ret = dma_mmap_coherent(dev, vma, qp->qdma.va, + qp->qdma.dma, sz); + vma->vm_pgoff = vm_pgoff; + return ret; + + default: + return -EINVAL; + } +} + +static int hisi_qm_uacce_start_queue(struct uacce_queue *q) +{ + struct hisi_qp *qp = q->priv; + + return hisi_qm_start_qp(qp, qp->pasid); +} + +static void hisi_qm_uacce_stop_queue(struct uacce_queue *q) +{ + struct hisi_qp *qp = q->priv; + + hisi_qm_stop_qp(qp); +} + +static int qm_set_sqctype(struct uacce_queue *q, u16 type) +{ + struct hisi_qm *qm = q->uacce->priv; + struct hisi_qp *qp = q->priv; + + write_lock(&qm->qps_lock); + qp->alg_type = type; + write_unlock(&qm->qps_lock); + + return 0; +} + +static long hisi_qm_uacce_ioctl(struct uacce_queue *q, unsigned int cmd, + unsigned long arg) +{ + struct hisi_qp *qp = q->priv; + struct hisi_qp_ctx qp_ctx; + + if (cmd == UACCE_CMD_QM_SET_QP_CTX) { + if (copy_from_user(&qp_ctx, (void __user *)arg, + sizeof(struct hisi_qp_ctx))) + return -EFAULT; + + if (qp_ctx.qc_type != 0 && qp_ctx.qc_type != 1) + return -EINVAL; + + qm_set_sqctype(q, qp_ctx.qc_type); + qp_ctx.id = qp->qp_id; + + if (copy_to_user((void __user *)arg, &qp_ctx, + sizeof(struct hisi_qp_ctx))) + return -EFAULT; + } else { + return -EINVAL; + } + + return 0; +} + +static struct uacce_ops uacce_qm_ops = { + .get_available_instances = hisi_qm_get_available_instances, + .get_queue = hisi_qm_uacce_get_queue, + .put_queue = hisi_qm_uacce_put_queue, + .start_queue = hisi_qm_uacce_start_queue, + .stop_queue = hisi_qm_uacce_stop_queue, + .mmap = hisi_qm_uacce_mmap, + .ioctl = hisi_qm_uacce_ioctl, +}; + +static int qm_register_uacce(struct hisi_qm *qm) +{ + struct pci_dev *pdev = qm->pdev; + struct uacce_device *uacce; + unsigned long mmio_page_nr; + unsigned long dus_page_nr; + struct uacce_interface interface = { + .flags = UACCE_DEV_SVA, + .ops = &uacce_qm_ops, + }; + + strncpy(interface.name, pdev->driver->name, sizeof(interface.name)); + + uacce = uacce_register(&pdev->dev, &interface); + if (IS_ERR(uacce)) + return PTR_ERR(uacce); + + if (uacce->flags & UACCE_DEV_SVA) { + qm->use_sva = true; + } else { + /* only consider sva case */ + uacce_unregister(uacce); + return -EINVAL; + } + + uacce->is_vf = pdev->is_virtfn; + uacce->priv = qm; + uacce->algs = qm->algs; + + if (qm->ver == QM_HW_V1) { + mmio_page_nr = QM_DOORBELL_PAGE_NR; + uacce->api_ver = HISI_QM_API_VER_BASE; + } else { + mmio_page_nr = QM_DOORBELL_PAGE_NR + + QM_DOORBELL_SQ_CQ_BASE_V2 / PAGE_SIZE; + uacce->api_ver = HISI_QM_API_VER2_BASE; + } + + dus_page_nr = (PAGE_SIZE - 1 + qm->sqe_size * QM_Q_DEPTH + + sizeof(struct qm_cqe) * QM_Q_DEPTH) >> PAGE_SHIFT; + + uacce->qf_pg_size[UACCE_QFRT_MMIO] = mmio_page_nr; + uacce->qf_pg_size[UACCE_QFRT_DUS] = dus_page_nr; + + qm->uacce = uacce; + + return 0; +} + /** * hisi_qm_init() - Initialize configures about qm. * @qm: The qm needing init. @@ -1421,6 +1643,10 @@ int hisi_qm_init(struct hisi_qm *qm) return -EINVAL; } + ret = qm_register_uacce(qm); + if (ret < 0) + dev_warn(&pdev->dev, "fail to register uacce (%d)\n", ret); + ret = pci_enable_device_mem(pdev); if (ret < 0) { dev_err(&pdev->dev, "Failed to enable device mem!\n"); @@ -1433,6 +1659,8 @@ int hisi_qm_init(struct hisi_qm *qm) goto err_disable_pcidev; } + qm->phys_base = pci_resource_start(pdev, PCI_BAR_2); + qm->size = pci_resource_len(qm->pdev, PCI_BAR_2); qm->io_base = ioremap(pci_resource_start(pdev, PCI_BAR_2), pci_resource_len(qm->pdev, PCI_BAR_2)); if (!qm->io_base) { @@ -1504,6 +1732,9 @@ void hisi_qm_uninit(struct hisi_qm *qm) iounmap(qm->io_base); pci_release_mem_regions(pdev); pci_disable_device(pdev); + + if (qm->uacce) + uacce_unregister(qm->uacce); } EXPORT_SYMBOL_GPL(hisi_qm_uninit); diff --git a/drivers/crypto/hisilicon/qm.h b/drivers/crypto/hisilicon/qm.h index 103e2fd..84a3be9 100644 --- a/drivers/crypto/hisilicon/qm.h +++ b/drivers/crypto/hisilicon/qm.h @@ -77,6 +77,10 @@ #define HISI_ACC_SGL_SGE_NR_MAX 255 +/* page number for queue file region */ +#define QM_DOORBELL_PAGE_NR 1 + + enum qp_state { QP_STOP, }; @@ -161,7 +165,12 @@ struct hisi_qm { u32 error_mask; u32 msi_mask; + const char *algs; bool use_dma_api; + bool use_sva; + resource_size_t phys_base; + resource_size_t size; + struct uacce_device *uacce; }; struct hisi_qp_status { @@ -191,10 +200,12 @@ struct hisi_qp { struct hisi_qp_ops *hw_ops; void *qp_ctx; void (*req_cb)(struct hisi_qp *qp, void *data); + void (*event_cb)(struct hisi_qp *qp); struct work_struct work; struct workqueue_struct *wq; - struct hisi_qm *qm; + u16 pasid; + struct uacce_queue *uacce_q; }; int hisi_qm_init(struct hisi_qm *qm); diff --git a/drivers/crypto/hisilicon/zip/zip_main.c b/drivers/crypto/hisilicon/zip/zip_main.c index 1b2ee96..48860d2 100644 --- a/drivers/crypto/hisilicon/zip/zip_main.c +++ b/drivers/crypto/hisilicon/zip/zip_main.c @@ -316,8 +316,14 @@ static void hisi_zip_set_user_domain_and_cache(struct hisi_zip *hisi_zip) writel(AXUSER_BASE, base + HZIP_BD_RUSER_32_63); writel(AXUSER_BASE, base + HZIP_SGL_RUSER_32_63); writel(AXUSER_BASE, base + HZIP_BD_WUSER_32_63); - writel(AXUSER_BASE, base + HZIP_DATA_RUSER_32_63); - writel(AXUSER_BASE, base + HZIP_DATA_WUSER_32_63); + + if (hisi_zip->qm.use_sva) { + writel(AXUSER_BASE | AXUSER_SSV, base + HZIP_DATA_RUSER_32_63); + writel(AXUSER_BASE | AXUSER_SSV, base + HZIP_DATA_WUSER_32_63); + } else { + writel(AXUSER_BASE, base + HZIP_DATA_RUSER_32_63); + writel(AXUSER_BASE, base + HZIP_DATA_WUSER_32_63); + } /* let's open all compression/decompression cores */ writel(DECOMP_CHECK_ENABLE | ALL_COMP_DECOMP_EN, @@ -671,24 +677,12 @@ static int hisi_zip_probe(struct pci_dev *pdev, const struct pci_device_id *id) qm = &hisi_zip->qm; qm->pdev = pdev; qm->ver = rev_id; - + qm->use_dma_api = true; + qm->algs = "zlib\ngzip\n"; qm->sqe_size = HZIP_SQE_SIZE; qm->dev_name = hisi_zip_name; qm->fun_type = (pdev->device == PCI_DEVICE_ID_ZIP_PF) ? QM_HW_PF : QM_HW_VF; - switch (uacce_mode) { - case 0: - qm->use_dma_api = true; - break; - case 1: - qm->use_dma_api = false; - break; - case 2: - qm->use_dma_api = true; - break; - default: - return -EINVAL; - } ret = hisi_qm_init(qm); if (ret) { @@ -976,12 +970,10 @@ static int __init hisi_zip_init(void) goto err_pci; } - if (uacce_mode == 0 || uacce_mode == 2) { - ret = hisi_zip_register_to_crypto(); - if (ret < 0) { - pr_err("Failed to register driver to crypto.\n"); - goto err_crypto; - } + ret = hisi_zip_register_to_crypto(); + if (ret < 0) { + pr_err("Failed to register driver to crypto.\n"); + goto err_crypto; } return 0; @@ -996,8 +988,7 @@ static int __init hisi_zip_init(void) static void __exit hisi_zip_exit(void) { - if (uacce_mode == 0 || uacce_mode == 2) - hisi_zip_unregister_from_crypto(); + hisi_zip_unregister_from_crypto(); pci_unregister_driver(&hisi_zip_pci_driver); hisi_zip_unregister_debugfs(); } diff --git a/include/uapi/misc/uacce/qm.h b/include/uapi/misc/uacce/qm.h new file mode 100644 index 0000000..d79a8f2 --- /dev/null +++ b/include/uapi/misc/uacce/qm.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */ +#ifndef HISI_QM_USR_IF_H +#define HISI_QM_USR_IF_H + +#include + +/** + * struct hisi_qp_ctx - User data for hisi qp. + * @id: Specifies which Turbo decode algorithm to use + * @qc_type: Accelerator algorithm type + */ +struct hisi_qp_ctx { + __u16 id; + __u16 qc_type; +}; + +#define HISI_QM_API_VER_BASE "hisi_qm_v1" +#define HISI_QM_API_VER2_BASE "hisi_qm_v2" + +/* UACCE_CMD_QM_SET_QP_CTX: Set qp algorithm type */ +#define UACCE_CMD_QM_SET_QP_CTX _IOWR('H', 10, struct hisi_qp_ctx) + +#endif