From patchwork Tue Jun 13 17:20:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 692125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9C20EB64D8 for ; Tue, 13 Jun 2023 17:22:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239173AbjFMRWa (ORCPT ); Tue, 13 Jun 2023 13:22:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50256 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238972AbjFMRWO (ORCPT ); Tue, 13 Jun 2023 13:22:14 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B67C619BC; Tue, 13 Jun 2023 10:21:55 -0700 (PDT) Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35DEK2d9013277; Tue, 13 Jun 2023 17:21:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=PdECZRliiH8YtM8MM+yqmGlCI15fGQHGxP7gej4lxNI=; b=ZwigXcY2pHqhNR5Y4s505Dx/4Jycdg/1pPBtwVbCYR3EKXtOKvjOLrVuZ/bdy3gaYec9 +GgtrT2Ij21iF8XpIXLPAtiTjuhIGh/DpxYlB66NTAHKAG2R310LU9FJvxSDA0CbL+3g V8/8brDs1UcTmtkOrie8YCKqnl2Tpe0eaGBcDTY/zHD/mwUv0u9thlmv6unL5P2aC3R8 h8++U/sT4t/vZGt98Enx8xoUR/SJqIgTAthOExAYVvnbQdhR+q3qIMMh0lWbuxZAQyGM UoeiMgPd0/P7d3K3GeYkAjrDRPj9jrYFvETuNl4gFSXlF8dbBYZOQ76TGLNVHwf0Gp8K 8g== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r6amhj9dt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:20 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 35DHLKRa005347 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:20 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 13 Jun 2023 10:21:19 -0700 From: Elliot Berman To: Alex Elder , Srinivas Kandagatla , Elliot Berman , Prakruthi Deepak Heragu , Rob Herring , Krzysztof Kozlowski , Conor Dooley CC: Murali Nalajala , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Dmitry Baryshkov , Bjorn Andersson , "Konrad Dybcio" , Arnd Bergmann , "Greg Kroah-Hartman" , Jonathan Corbet , Bagas Sanjaya , Will Deacon , "Andy Gross" , Catalin Marinas , "Jassi Brar" , , , , , , "Rob Herring" Subject: [PATCH v14 02/25] dt-bindings: Add binding for gunyah hypervisor Date: Tue, 13 Jun 2023 10:20:30 -0700 Message-ID: <20230613172054.3959700-3-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230613172054.3959700-1-quic_eberman@quicinc.com> References: <20230613172054.3959700-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: RYglC-7spcDC-rv9qZRukDK3LKawB9IP X-Proofpoint-GUID: RYglC-7spcDC-rv9qZRukDK3LKawB9IP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-13_20,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 mlxscore=0 spamscore=0 phishscore=0 impostorscore=0 lowpriorityscore=0 bulkscore=0 mlxlogscore=999 adultscore=0 priorityscore=1501 malwarescore=0 clxscore=1011 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130153 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org When Linux is booted as a guest under the Gunyah hypervisor, the Gunyah Resource Manager applies a devicetree overlay describing the virtual platform configuration of the guest VM, such as the message queue capability IDs for communicating with the Resource Manager. This information is not otherwise discoverable by a VM: the Gunyah hypervisor core does not provide a direct interface to discover capability IDs nor a way to communicate with RM without having already known the corresponding message queue capability ID. Add the DT bindings that Gunyah adheres for the hypervisor node and message queues. Reviewed-by: Rob Herring Signed-off-by: Elliot Berman --- .../bindings/firmware/gunyah-hypervisor.yaml | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml diff --git a/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml new file mode 100644 index 0000000000000..3fc0b043ac3cf --- /dev/null +++ b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml @@ -0,0 +1,82 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/firmware/gunyah-hypervisor.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Gunyah Hypervisor + +maintainers: + - Prakruthi Deepak Heragu + - Elliot Berman + +description: |+ + Gunyah virtual machines use this information to determine the capability IDs + of the message queues used to communicate with the Gunyah Resource Manager. + See also: https://github.com/quic/gunyah-resource-manager/blob/develop/src/vm_creation/dto_construct.c + +properties: + compatible: + const: gunyah-hypervisor + + "#address-cells": + description: Number of cells needed to represent 64-bit capability IDs. + const: 2 + + "#size-cells": + description: must be 0, because capability IDs are not memory address + ranges and do not have a size. + const: 0 + +patternProperties: + "^gunyah-resource-mgr(@.*)?": + type: object + description: + Resource Manager node which is required to communicate to Resource + Manager VM using Gunyah Message Queues. + + properties: + compatible: + const: gunyah-resource-manager + + reg: + items: + - description: Gunyah capability ID of the TX message queue + - description: Gunyah capability ID of the RX message queue + + interrupts: + items: + - description: Interrupt for the TX message queue + - description: Interrupt for the RX message queue + + additionalProperties: false + + required: + - compatible + - reg + - interrupts + +additionalProperties: false + +required: + - compatible + - "#address-cells" + - "#size-cells" + +examples: + - | + #include + + hypervisor { + #address-cells = <2>; + #size-cells = <0>; + compatible = "gunyah-hypervisor"; + + gunyah-resource-mgr@0 { + compatible = "gunyah-resource-manager"; + interrupts = , /* TX full IRQ */ + ; /* RX empty IRQ */ + reg = <0x00000000 0x00000000>, <0x00000000 0x00000001>; + /* TX, RX cap ids */ + }; + }; From patchwork Tue Jun 13 17:20:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 692124 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A616EB64D0 for ; Tue, 13 Jun 2023 17:22:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239040AbjFMRWf (ORCPT ); Tue, 13 Jun 2023 13:22:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238755AbjFMRW1 (ORCPT ); Tue, 13 Jun 2023 13:22:27 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2ADFA1BCD; Tue, 13 Jun 2023 10:21:57 -0700 (PDT) Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35DAcLml013251; Tue, 13 Jun 2023 17:21:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=dB4o+zHipS/b54sgNdnhOb3uXQjLrxBiUIX6nI9viJg=; b=Y73ymzRLM2S7gDOoflfs7KBziVsKdJfan47Y3QT3AB7FPB93MtP7lY6LMNwPR0QAopI8 A01918kZ0fWg7sfo/GaBe2ULZk5Vnc1j6wrJRiptPV9SdgIF/66RF7wLWj2eBe7E7B9u yGsLhzJ7j0TSwrP3FaBr7hsOp/J2WkHxdUYfddJCyo3PHbqA16uSnjHDZq7g8k0drrqV ZZ2IGliCJkDfzjFo03qVU2btnBfHq930snbnasF6ZQkcosw3NQ6Eat20aL91t3YWfR2M 1ItDdDwz0JghOypcpfMJVP56NOCGjWY4EYDpWiu01A/msbUewkRp+Aip/cMpIaYGugY7 Fw== Received: from nasanppmta03.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r6km41d3y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:21 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 35DHLLw4012624 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:21 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 13 Jun 2023 10:21:20 -0700 From: Elliot Berman To: Alex Elder , Srinivas Kandagatla , Elliot Berman , Prakruthi Deepak Heragu CC: Murali Nalajala , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Dmitry Baryshkov , Bjorn Andersson , "Konrad Dybcio" , Arnd Bergmann , "Greg Kroah-Hartman" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Jonathan Corbet , Bagas Sanjaya , Will Deacon , Andy Gross , Catalin Marinas , Jassi Brar , , , , , Subject: [PATCH v14 03/25] gunyah: Common types and error codes for Gunyah hypercalls Date: Tue, 13 Jun 2023 10:20:31 -0700 Message-ID: <20230613172054.3959700-4-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230613172054.3959700-1-quic_eberman@quicinc.com> References: <20230613172054.3959700-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: V-dd59G4ln1UXt0XNZQg0RP3jVcKR9A_ X-Proofpoint-GUID: V-dd59G4ln1UXt0XNZQg0RP3jVcKR9A_ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-13_18,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 lowpriorityscore=0 malwarescore=0 priorityscore=1501 spamscore=0 impostorscore=0 phishscore=0 mlxscore=0 suspectscore=0 bulkscore=0 mlxlogscore=736 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130153 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Add architecture-independent standard error codes, types, and macros for Gunyah hypercalls. Reviewed-by: Dmitry Baryshkov Reviewed-by: Srinivas Kandagatla Reviewed-by: Alex Elder Signed-off-by: Elliot Berman --- include/linux/gunyah.h | 83 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 include/linux/gunyah.h diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h new file mode 100644 index 0000000000000..a4e8ec91961d1 --- /dev/null +++ b/include/linux/gunyah.h @@ -0,0 +1,83 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _LINUX_GUNYAH_H +#define _LINUX_GUNYAH_H + +#include +#include + +/******************************************************************************/ +/* Common arch-independent definitions for Gunyah hypercalls */ +#define GH_CAPID_INVAL U64_MAX +#define GH_VMID_ROOT_VM 0xff + +enum gh_error { + GH_ERROR_OK = 0, + GH_ERROR_UNIMPLEMENTED = -1, + GH_ERROR_RETRY = -2, + + GH_ERROR_ARG_INVAL = 1, + GH_ERROR_ARG_SIZE = 2, + GH_ERROR_ARG_ALIGN = 3, + + GH_ERROR_NOMEM = 10, + + GH_ERROR_ADDR_OVFL = 20, + GH_ERROR_ADDR_UNFL = 21, + GH_ERROR_ADDR_INVAL = 22, + + GH_ERROR_DENIED = 30, + GH_ERROR_BUSY = 31, + GH_ERROR_IDLE = 32, + + GH_ERROR_IRQ_BOUND = 40, + GH_ERROR_IRQ_UNBOUND = 41, + + GH_ERROR_CSPACE_CAP_NULL = 50, + GH_ERROR_CSPACE_CAP_REVOKED = 51, + GH_ERROR_CSPACE_WRONG_OBJ_TYPE = 52, + GH_ERROR_CSPACE_INSUF_RIGHTS = 53, + GH_ERROR_CSPACE_FULL = 54, + + GH_ERROR_MSGQUEUE_EMPTY = 60, + GH_ERROR_MSGQUEUE_FULL = 61, +}; + +/** + * gh_error_remap() - Remap Gunyah hypervisor errors into a Linux error code + * @gh_error: Gunyah hypercall return value + */ +static inline int gh_error_remap(enum gh_error gh_error) +{ + switch (gh_error) { + case GH_ERROR_OK: + return 0; + case GH_ERROR_NOMEM: + return -ENOMEM; + case GH_ERROR_DENIED: + case GH_ERROR_CSPACE_CAP_NULL: + case GH_ERROR_CSPACE_CAP_REVOKED: + case GH_ERROR_CSPACE_WRONG_OBJ_TYPE: + case GH_ERROR_CSPACE_INSUF_RIGHTS: + case GH_ERROR_CSPACE_FULL: + return -EACCES; + case GH_ERROR_BUSY: + case GH_ERROR_IDLE: + return -EBUSY; + case GH_ERROR_IRQ_BOUND: + case GH_ERROR_IRQ_UNBOUND: + case GH_ERROR_MSGQUEUE_FULL: + case GH_ERROR_MSGQUEUE_EMPTY: + return -EIO; + case GH_ERROR_UNIMPLEMENTED: + case GH_ERROR_RETRY: + return -EOPNOTSUPP; + default: + return -EINVAL; + } +} + +#endif From patchwork Tue Jun 13 17:20:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 692127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4547EB64D7 for ; Tue, 13 Jun 2023 17:21:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238792AbjFMRV5 (ORCPT ); Tue, 13 Jun 2023 13:21:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238491AbjFMRVw (ORCPT ); Tue, 13 Jun 2023 13:21:52 -0400 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 67B3C1732; Tue, 13 Jun 2023 10:21:48 -0700 (PDT) Received: from pps.filterd (m0279868.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35DDBCRj004806; Tue, 13 Jun 2023 17:21:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=LZphZJ6YgG7gabhn80grTEmhI7ZjP8+JCGi9JN3+Z2w=; b=CY2mItZ8K/tyeRJmY23fy34ttNAqIRN2HVSwsBlusNnkRHG5ug1/Q+uDrvtnpwiAZPXH Bo/JJ0HPzjZwRidOsoleK5+BLna3DqR2DTUJjpctOq0WEgkvwu5z7r8qZu8FmhbPCCfe t4VQL9b04ixK77CgWjVtgvJerQl04qZJ/lCx6lPkSpBVHmOBBKU4dE/yqhBbWszmDeZN hBHD7NdxIvfSr/Opq7tRGWHRmhibbPvEciYcaMk5Xrm6he/74SKN5DMQtDzXFKYkgHiN +rWzGcoyGTbWRIVOMwRLj4dSUjdSPrebVFjqB4mJ9k7ee1s+n3KqQIxX/zknR/N3T/hF gw== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r6s3wrnar-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:24 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 35DHLMkP020980 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:22 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 13 Jun 2023 10:21:21 -0700 From: Elliot Berman To: Alex Elder , Srinivas Kandagatla , Catalin Marinas , Will Deacon , Elliot Berman , Prakruthi Deepak Heragu CC: Murali Nalajala , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Dmitry Baryshkov , Bjorn Andersson , "Konrad Dybcio" , Arnd Bergmann , "Greg Kroah-Hartman" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Jonathan Corbet , Bagas Sanjaya , Andy Gross , Jassi Brar , , , , , Subject: [PATCH v14 04/25] virt: gunyah: Add hypercalls to identify Gunyah Date: Tue, 13 Jun 2023 10:20:32 -0700 Message-ID: <20230613172054.3959700-5-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230613172054.3959700-1-quic_eberman@quicinc.com> References: <20230613172054.3959700-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: h2ZyjYsrTu6891F2Bz--4fthh2Mhu1P1 X-Proofpoint-GUID: h2ZyjYsrTu6891F2Bz--4fthh2Mhu1P1 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-13_19,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 adultscore=0 mlxscore=0 mlxlogscore=999 clxscore=1015 impostorscore=0 lowpriorityscore=0 suspectscore=0 priorityscore=1501 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130153 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Add hypercalls to identify when Linux is running a virtual machine under Gunyah. There are two calls to help identify Gunyah: 1. gh_hypercall_get_uid() returns a UID when running under a Gunyah hypervisor. 2. gh_hypercall_hyp_identify() returns build information and a set of feature flags that are supported by Gunyah. Reviewed-by: Srinivas Kandagatla Reviewed-by: Alex Elder Signed-off-by: Elliot Berman --- arch/arm64/Kbuild | 1 + arch/arm64/gunyah/Makefile | 3 ++ arch/arm64/gunyah/gunyah_hypercall.c | 58 ++++++++++++++++++++++++++++ drivers/virt/Kconfig | 2 + drivers/virt/gunyah/Kconfig | 13 +++++++ include/linux/gunyah.h | 31 +++++++++++++++ 6 files changed, 108 insertions(+) create mode 100644 arch/arm64/gunyah/Makefile create mode 100644 arch/arm64/gunyah/gunyah_hypercall.c create mode 100644 drivers/virt/gunyah/Kconfig diff --git a/arch/arm64/Kbuild b/arch/arm64/Kbuild index 5bfbf7d79c99b..e4847ba0e3c95 100644 --- a/arch/arm64/Kbuild +++ b/arch/arm64/Kbuild @@ -3,6 +3,7 @@ obj-y += kernel/ mm/ net/ obj-$(CONFIG_KVM) += kvm/ obj-$(CONFIG_XEN) += xen/ obj-$(subst m,y,$(CONFIG_HYPERV)) += hyperv/ +obj-$(CONFIG_GUNYAH) += gunyah/ obj-$(CONFIG_CRYPTO) += crypto/ # for cleaning diff --git a/arch/arm64/gunyah/Makefile b/arch/arm64/gunyah/Makefile new file mode 100644 index 0000000000000..84f1e38cafb1e --- /dev/null +++ b/arch/arm64/gunyah/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_GUNYAH) += gunyah_hypercall.o diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c new file mode 100644 index 0000000000000..ef48e0b8b5448 --- /dev/null +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include + +/* {c1d58fcd-a453-5fdb-9265-ce36673d5f14} */ +static const uuid_t GUNYAH_UUID = + UUID_INIT(0xc1d58fcd, 0xa453, 0x5fdb, 0x92, 0x65, 0xce, 0x36, 0x67, 0x3d, 0x5f, 0x14); + +bool arch_is_gh_guest(void) +{ + struct arm_smccc_res res; + uuid_t uuid; + u32 *up; + + arm_smccc_1_1_hvc(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, &res); + + up = (u32 *)&uuid.b[0]; + up[0] = lower_32_bits(res.a0); + up[1] = lower_32_bits(res.a1); + up[2] = lower_32_bits(res.a2); + up[3] = lower_32_bits(res.a3); + + return uuid_equal(&uuid, &GUNYAH_UUID); +} +EXPORT_SYMBOL_GPL(arch_is_gh_guest); + +#define GH_HYPERCALL(fn) ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_VENDOR_HYP, \ + fn) + +#define GH_HYPERCALL_HYP_IDENTIFY GH_HYPERCALL(0x8000) + +/** + * gh_hypercall_hyp_identify() - Returns build information and feature flags + * supported by Gunyah. + * @hyp_identity: filled by the hypercall with the API info and feature flags. + */ +void gh_hypercall_hyp_identify(struct gh_hypercall_hyp_identify_resp *hyp_identity) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GH_HYPERCALL_HYP_IDENTIFY, &res); + + hyp_identity->api_info = res.a0; + hyp_identity->flags[0] = res.a1; + hyp_identity->flags[1] = res.a2; + hyp_identity->flags[2] = res.a3; +} +EXPORT_SYMBOL_GPL(gh_hypercall_hyp_identify); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/drivers/virt/Kconfig b/drivers/virt/Kconfig index f79ab13a5c28b..85bd6626ffc90 100644 --- a/drivers/virt/Kconfig +++ b/drivers/virt/Kconfig @@ -54,4 +54,6 @@ source "drivers/virt/coco/sev-guest/Kconfig" source "drivers/virt/coco/tdx-guest/Kconfig" +source "drivers/virt/gunyah/Kconfig" + endif diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig new file mode 100644 index 0000000000000..1a737694c333d --- /dev/null +++ b/drivers/virt/gunyah/Kconfig @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config GUNYAH + tristate "Gunyah Virtualization drivers" + depends on ARM64 + depends on MAILBOX + help + The Gunyah drivers are the helper interfaces that run in a guest VM + such as basic inter-VM IPC and signaling mechanisms, and higher level + services such as memory/device sharing, IRQ sharing, and so on. + + Say Y/M here to enable the drivers needed to interact in a Gunyah + virtual environment. diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index a4e8ec91961d1..6b36cf4787efb 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -6,8 +6,10 @@ #ifndef _LINUX_GUNYAH_H #define _LINUX_GUNYAH_H +#include #include #include +#include /******************************************************************************/ /* Common arch-independent definitions for Gunyah hypercalls */ @@ -80,4 +82,33 @@ static inline int gh_error_remap(enum gh_error gh_error) } } +enum gh_api_feature { + GH_FEATURE_DOORBELL = 1, + GH_FEATURE_MSGQUEUE = 2, + GH_FEATURE_VCPU = 5, + GH_FEATURE_MEMEXTENT = 6, +}; + +bool arch_is_gh_guest(void); + +#define GH_API_V1 1 + +/* Other bits reserved for future use and will be zero */ +#define GH_API_INFO_API_VERSION_MASK GENMASK_ULL(13, 0) +#define GH_API_INFO_BIG_ENDIAN BIT_ULL(14) +#define GH_API_INFO_IS_64BIT BIT_ULL(15) +#define GH_API_INFO_VARIANT_MASK GENMASK_ULL(63, 56) + +struct gh_hypercall_hyp_identify_resp { + u64 api_info; + u64 flags[3]; +}; + +static inline u16 gh_api_version(const struct gh_hypercall_hyp_identify_resp *gh_api) +{ + return FIELD_GET(GH_API_INFO_API_VERSION_MASK, gh_api->api_info); +} + +void gh_hypercall_hyp_identify(struct gh_hypercall_hyp_identify_resp *hyp_identity); + #endif From patchwork Tue Jun 13 17:20:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 692126 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2631DEB64DA for ; Tue, 13 Jun 2023 17:22:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238983AbjFMRWO (ORCPT ); Tue, 13 Jun 2023 13:22:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49854 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238720AbjFMRV4 (ORCPT ); Tue, 13 Jun 2023 13:21:56 -0400 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D08C7198D; Tue, 13 Jun 2023 10:21:51 -0700 (PDT) Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35DAf5wR024784; Tue, 13 Jun 2023 17:21:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=Jbmjr1kIvPaQFX4DgQkM3UtjljErKDw5g0nYVIiAnZ8=; b=itph5pYOXyBmpYC8rvuMSZiqIOmwHWtUtc8FT/7kehbpjggU5aBSBJle+Thyjxf/Ntt7 H0UkliMrlmOetSfzW4AQ1t4yCS5XzEw1IMgeyxqeRCMXtDIWo9y0iZA8SeRKuFIsxOM4 dLc9gWadKCp7yIc/sO+KWDCpj9PN4RdviwIKnsBmtMh5WdaxTvCHq1/luyxWdmFC45lc BBtDPYJ7GiWGW0KPBkf9ubISnGD9WPpxLKf/ZhWFPLW4GqMeR0Z+fHI8EGPys9Uvyft8 h7xANS+/ffaqGUybAFy03l6ZUw3rRS6giymrxtU7RGyH87qVf0MRE6Rk4MuNIN2PCE+L Pg== Received: from nasanppmta03.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r6ng4s64a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:32 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 35DHLUj6012791 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:30 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 13 Jun 2023 10:21:29 -0700 From: Elliot Berman To: Alex Elder , Srinivas Kandagatla , Elliot Berman , Prakruthi Deepak Heragu CC: Murali Nalajala , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Dmitry Baryshkov , Bjorn Andersson , "Konrad Dybcio" , Arnd Bergmann , "Greg Kroah-Hartman" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Jonathan Corbet , Bagas Sanjaya , Will Deacon , Andy Gross , Catalin Marinas , Jassi Brar , , , , , Subject: [PATCH v14 08/25] gunyah: rsc_mgr: Add VM lifecycle RPC Date: Tue, 13 Jun 2023 10:20:36 -0700 Message-ID: <20230613172054.3959700-9-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230613172054.3959700-1-quic_eberman@quicinc.com> References: <20230613172054.3959700-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: SuXKaEkIynOKVPwnwdXkdcGUdqe_5_BM X-Proofpoint-ORIG-GUID: SuXKaEkIynOKVPwnwdXkdcGUdqe_5_BM X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-13_20,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 lowpriorityscore=0 mlxlogscore=999 phishscore=0 spamscore=0 impostorscore=0 malwarescore=0 bulkscore=0 adultscore=0 priorityscore=1501 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130152 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Add Gunyah Resource Manager RPC to launch an unauthenticated VM. Reviewed-by: Alex Elder Reviewed-by: Srinivas Kandagatla Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/rsc_mgr_rpc.c | 259 ++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 73 +++++++++ 3 files changed, 333 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/rsc_mgr_rpc.c diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 0f5aec8346988..241bab357b86c 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,4 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 -gunyah-y += rsc_mgr.o +gunyah-y += rsc_mgr.o rsc_mgr_rpc.o obj-$(CONFIG_GUNYAH) += gunyah.o diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mgr_rpc.c new file mode 100644 index 0000000000000..a4a9f0ba4e1fc --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -0,0 +1,259 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include "rsc_mgr.h" + +/* Message IDs: VM Management */ +#define GH_RM_RPC_VM_ALLOC_VMID 0x56000001 +#define GH_RM_RPC_VM_DEALLOC_VMID 0x56000002 +#define GH_RM_RPC_VM_START 0x56000004 +#define GH_RM_RPC_VM_STOP 0x56000005 +#define GH_RM_RPC_VM_RESET 0x56000006 +#define GH_RM_RPC_VM_CONFIG_IMAGE 0x56000009 +#define GH_RM_RPC_VM_INIT 0x5600000B +#define GH_RM_RPC_VM_GET_HYP_RESOURCES 0x56000020 +#define GH_RM_RPC_VM_GET_VMID 0x56000024 + +struct gh_rm_vm_common_vmid_req { + __le16 vmid; + __le16 _padding; +} __packed; + +/* Call: VM_ALLOC */ +struct gh_rm_vm_alloc_vmid_resp { + __le16 vmid; + __le16 _padding; +} __packed; + +/* Call: VM_STOP */ +#define GH_RM_VM_STOP_FLAG_FORCE_STOP BIT(0) + +#define GH_RM_VM_STOP_REASON_FORCE_STOP 3 + +struct gh_rm_vm_stop_req { + __le16 vmid; + u8 flags; + u8 _padding; + __le32 stop_reason; +} __packed; + +/* Call: VM_CONFIG_IMAGE */ +struct gh_rm_vm_config_image_req { + __le16 vmid; + __le16 auth_mech; + __le32 mem_handle; + __le64 image_offset; + __le64 image_size; + __le64 dtb_offset; + __le64 dtb_size; +} __packed; + +/* + * Several RM calls take only a VMID as a parameter and give only standard + * response back. Deduplicate boilerplate code by using this common call. + */ +static int gh_rm_common_vmid_call(struct gh_rm *rm, u32 message_id, u16 vmid) +{ + struct gh_rm_vm_common_vmid_req req_payload = { + .vmid = cpu_to_le16(vmid), + }; + + return gh_rm_call(rm, message_id, &req_payload, sizeof(req_payload), NULL, NULL); +} + +/** + * gh_rm_alloc_vmid() - Allocate a new VM in Gunyah. Returns the VM identifier. + * @rm: Handle to a Gunyah resource manager + * @vmid: Use 0 to dynamically allocate a VM. A reserved VMID can be supplied + * to request allocation of a platform-defined VM. + * + * Returns - the allocated VMID or negative value on error + */ +int gh_rm_alloc_vmid(struct gh_rm *rm, u16 vmid) +{ + struct gh_rm_vm_common_vmid_req req_payload = { + .vmid = cpu_to_le16(vmid), + }; + struct gh_rm_vm_alloc_vmid_resp *resp_payload; + size_t resp_size; + void *resp; + int ret; + + ret = gh_rm_call(rm, GH_RM_RPC_VM_ALLOC_VMID, &req_payload, sizeof(req_payload), &resp, + &resp_size); + if (ret) + return ret; + + if (!vmid) { + resp_payload = resp; + ret = le16_to_cpu(resp_payload->vmid); + kfree(resp); + } + + return ret; +} + +/** + * gh_rm_dealloc_vmid() - Dispose of a VMID + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gh_rm_alloc_vmid + */ +int gh_rm_dealloc_vmid(struct gh_rm *rm, u16 vmid) +{ + return gh_rm_common_vmid_call(rm, GH_RM_RPC_VM_DEALLOC_VMID, vmid); +} + +/** + * gh_rm_vm_reset() - Reset a VM's resources + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gh_rm_alloc_vmid + * + * As part of tearing down the VM, request RM to clean up all the VM resources + * associated with the VM. Only after this, Linux can clean up all the + * references it maintains to resources. + */ +int gh_rm_vm_reset(struct gh_rm *rm, u16 vmid) +{ + return gh_rm_common_vmid_call(rm, GH_RM_RPC_VM_RESET, vmid); +} + +/** + * gh_rm_vm_start() - Move a VM into "ready to run" state + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gh_rm_alloc_vmid + * + * On VMs which use proxy scheduling, vcpu_run is needed to actually run the VM. + * On VMs which use Gunyah's scheduling, the vCPUs start executing in accordance with Gunyah + * scheduling policies. + */ +int gh_rm_vm_start(struct gh_rm *rm, u16 vmid) +{ + return gh_rm_common_vmid_call(rm, GH_RM_RPC_VM_START, vmid); +} + +/** + * gh_rm_vm_stop() - Send a request to Resource Manager VM to forcibly stop a VM. + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gh_rm_alloc_vmid + */ +int gh_rm_vm_stop(struct gh_rm *rm, u16 vmid) +{ + struct gh_rm_vm_stop_req req_payload = { + .vmid = cpu_to_le16(vmid), + .flags = GH_RM_VM_STOP_FLAG_FORCE_STOP, + .stop_reason = cpu_to_le32(GH_RM_VM_STOP_REASON_FORCE_STOP), + }; + + return gh_rm_call(rm, GH_RM_RPC_VM_STOP, &req_payload, sizeof(req_payload), NULL, NULL); +} + +/** + * gh_rm_vm_configure() - Prepare a VM to start and provide the common + * configuration needed by RM to configure a VM + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gh_rm_alloc_vmid + * @auth_mechanism: Authentication mechanism used by resource manager to verify + * the virtual machine + * @mem_handle: Handle to a previously shared memparcel that contains all parts + * of the VM image subject to authentication. + * @image_offset: Start address of VM image, relative to the start of memparcel + * @image_size: Size of the VM image + * @dtb_offset: Start address of the devicetree binary with VM configuration, + * relative to start of memparcel. + * @dtb_size: Maximum size of devicetree binary. + */ +int gh_rm_vm_configure(struct gh_rm *rm, u16 vmid, enum gh_rm_vm_auth_mechanism auth_mechanism, + u32 mem_handle, u64 image_offset, u64 image_size, u64 dtb_offset, u64 dtb_size) +{ + struct gh_rm_vm_config_image_req req_payload = { + .vmid = cpu_to_le16(vmid), + .auth_mech = cpu_to_le16(auth_mechanism), + .mem_handle = cpu_to_le32(mem_handle), + .image_offset = cpu_to_le64(image_offset), + .image_size = cpu_to_le64(image_size), + .dtb_offset = cpu_to_le64(dtb_offset), + .dtb_size = cpu_to_le64(dtb_size), + }; + + return gh_rm_call(rm, GH_RM_RPC_VM_CONFIG_IMAGE, &req_payload, sizeof(req_payload), + NULL, NULL); +} + +/** + * gh_rm_vm_init() - Move the VM to initialized state. + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier + * + * RM will allocate needed resources for the VM. + */ +int gh_rm_vm_init(struct gh_rm *rm, u16 vmid) +{ + return gh_rm_common_vmid_call(rm, GH_RM_RPC_VM_INIT, vmid); +} + +/** + * gh_rm_get_hyp_resources() - Retrieve hypervisor resources (capabilities) associated with a VM + * @rm: Handle to a Gunyah resource manager + * @vmid: VMID of the other VM to get the resources of + * @resources: Set by gh_rm_get_hyp_resources and contains the returned hypervisor resources. + * Caller must free the resources pointer if successful. + */ +int gh_rm_get_hyp_resources(struct gh_rm *rm, u16 vmid, + struct gh_rm_hyp_resources **resources) +{ + struct gh_rm_vm_common_vmid_req req_payload = { + .vmid = cpu_to_le16(vmid), + }; + struct gh_rm_hyp_resources *resp; + size_t resp_size; + int ret; + + ret = gh_rm_call(rm, GH_RM_RPC_VM_GET_HYP_RESOURCES, + &req_payload, sizeof(req_payload), + (void **)&resp, &resp_size); + if (ret) + return ret; + + if (!resp_size) + return -EBADMSG; + + if (resp_size < struct_size(resp, entries, 0) || + resp_size != struct_size(resp, entries, le32_to_cpu(resp->n_entries))) { + kfree(resp); + return -EBADMSG; + } + + *resources = resp; + return 0; +} + +/** + * gh_rm_get_vmid() - Retrieve VMID of this virtual machine + * @rm: Handle to a Gunyah resource manager + * @vmid: Filled with the VMID of this VM + */ +int gh_rm_get_vmid(struct gh_rm *rm, u16 *vmid) +{ + static u16 cached_vmid = GH_VMID_INVAL; + size_t resp_size; + __le32 *resp; + int ret; + + if (cached_vmid != GH_VMID_INVAL) { + *vmid = cached_vmid; + return 0; + } + + ret = gh_rm_call(rm, GH_RM_RPC_VM_GET_VMID, NULL, 0, (void **)&resp, &resp_size); + if (ret) + return ret; + + *vmid = cached_vmid = lower_16_bits(le32_to_cpu(*resp)); + kfree(resp); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_get_vmid); diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index f2a312e80af52..1ac66d9004d2b 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -18,4 +18,77 @@ int gh_rm_notifier_unregister(struct gh_rm *rm, struct notifier_block *nb); struct device *gh_rm_get(struct gh_rm *rm); void gh_rm_put(struct gh_rm *rm); +struct gh_rm_vm_exited_payload { + __le16 vmid; + __le16 exit_type; + __le32 exit_reason_size; + u8 exit_reason[]; +} __packed; + +#define GH_RM_NOTIFICATION_VM_EXITED 0x56100001 + +enum gh_rm_vm_status { + GH_RM_VM_STATUS_NO_STATE = 0, + GH_RM_VM_STATUS_INIT = 1, + GH_RM_VM_STATUS_READY = 2, + GH_RM_VM_STATUS_RUNNING = 3, + GH_RM_VM_STATUS_PAUSED = 4, + GH_RM_VM_STATUS_LOAD = 5, + GH_RM_VM_STATUS_AUTH = 6, + GH_RM_VM_STATUS_INIT_FAILED = 8, + GH_RM_VM_STATUS_EXITED = 9, + GH_RM_VM_STATUS_RESETTING = 10, + GH_RM_VM_STATUS_RESET = 11, +}; + +struct gh_rm_vm_status_payload { + __le16 vmid; + u16 reserved; + u8 vm_status; + u8 os_status; + __le16 app_status; +} __packed; + +#define GH_RM_NOTIFICATION_VM_STATUS 0x56100008 + +/* RPC Calls */ +int gh_rm_alloc_vmid(struct gh_rm *rm, u16 vmid); +int gh_rm_dealloc_vmid(struct gh_rm *rm, u16 vmid); +int gh_rm_vm_reset(struct gh_rm *rm, u16 vmid); +int gh_rm_vm_start(struct gh_rm *rm, u16 vmid); +int gh_rm_vm_stop(struct gh_rm *rm, u16 vmid); + +enum gh_rm_vm_auth_mechanism { + GH_RM_VM_AUTH_NONE = 0, + GH_RM_VM_AUTH_QCOM_PIL_ELF = 1, + GH_RM_VM_AUTH_QCOM_ANDROID_PVM = 2, +}; + +int gh_rm_vm_configure(struct gh_rm *rm, u16 vmid, enum gh_rm_vm_auth_mechanism auth_mechanism, + u32 mem_handle, u64 image_offset, u64 image_size, + u64 dtb_offset, u64 dtb_size); +int gh_rm_vm_init(struct gh_rm *rm, u16 vmid); + +struct gh_rm_hyp_resource { + u8 type; + u8 reserved; + __le16 partner_vmid; + __le32 resource_handle; + __le32 resource_label; + __le64 cap_id; + __le32 virq_handle; + __le32 virq; + __le64 base; + __le64 size; +} __packed; + +struct gh_rm_hyp_resources { + __le32 n_entries; + struct gh_rm_hyp_resource entries[]; +} __packed; + +int gh_rm_get_hyp_resources(struct gh_rm *rm, u16 vmid, + struct gh_rm_hyp_resources **resources); +int gh_rm_get_vmid(struct gh_rm *rm, u16 *vmid); + #endif From patchwork Tue Jun 13 17:20:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 692122 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE1B2C001DB for ; Tue, 13 Jun 2023 17:23:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239627AbjFMRXK (ORCPT ); Tue, 13 Jun 2023 13:23:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50258 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239129AbjFMRW3 (ORCPT ); Tue, 13 Jun 2023 13:22:29 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 98B231FC9; Tue, 13 Jun 2023 10:22:02 -0700 (PDT) Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35DAtVY0011423; Tue, 13 Jun 2023 17:21:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=ZR5DrOyn7BtztEo0/Q9Z+lMegXaU3nqBZWV0gGytyRA=; b=d1WqsbVzae/9zAhF+phosChOVnMDM0OmLX4euhtPRwwFsl/scuProcPCnsyhKZJ8E0wo yGKgKu8uf0cnojT8bZ0RIQd1A4D4153qZcM9dyDGbAhEDmuiXKxc9q9aZ2FPAcNoIyFr 5NY16SkYj4x40XD1SyuKfeCNCCRHa2C7gmOLoGiHVuNSqMWj2/wBs9/0TvPaLe4loSyd rppLrkDr2OeWwOun+U+PrIN1d+M0hUlVue/XjzRBnC9k2cOdktV6Zp5QO3uoUcXEp9Gf 0LhepuTwSYtlPVRJDsqRw8CW9XUumb+v4al19tWlCiicHUnpVWJxwtQI5TPqGhARU8xj QQ== Received: from nasanppmta03.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r6q4r0xeb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:36 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 35DHLacK013208 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:36 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 13 Jun 2023 10:21:35 -0700 From: Elliot Berman To: Alex Elder , Srinivas Kandagatla , Elliot Berman , Prakruthi Deepak Heragu CC: Murali Nalajala , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Dmitry Baryshkov , Bjorn Andersson , "Konrad Dybcio" , Arnd Bergmann , "Greg Kroah-Hartman" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Jonathan Corbet , Bagas Sanjaya , Will Deacon , Andy Gross , Catalin Marinas , Jassi Brar , , , , , Subject: [PATCH v14 11/25] gunyah: vm_mgr: Add/remove user memory regions Date: Tue, 13 Jun 2023 10:20:39 -0700 Message-ID: <20230613172054.3959700-12-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230613172054.3959700-1-quic_eberman@quicinc.com> References: <20230613172054.3959700-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: nYXa7leADpDLwEB6rdO-8hmlH3FACh5S X-Proofpoint-ORIG-GUID: nYXa7leADpDLwEB6rdO-8hmlH3FACh5S X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-13_19,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 clxscore=1015 priorityscore=1501 adultscore=0 bulkscore=0 suspectscore=0 impostorscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130153 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org When launching a virtual machine, Gunyah userspace allocates memory for the guest and informs Gunyah about these memory regions through SET_USER_MEMORY_REGION ioctl. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/vm_mgr.c | 59 +++++++- drivers/virt/gunyah/vm_mgr.h | 26 ++++ drivers/virt/gunyah/vm_mgr_mm.c | 232 ++++++++++++++++++++++++++++++++ include/uapi/linux/gunyah.h | 37 +++++ 5 files changed, 352 insertions(+), 4 deletions(-) create mode 100644 drivers/virt/gunyah/vm_mgr_mm.c diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index e47e25895299c..bacf78b8fa337 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,4 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 -gunyah-y += rsc_mgr.o rsc_mgr_rpc.o vm_mgr.o +gunyah-y += rsc_mgr.o rsc_mgr_rpc.o vm_mgr.o vm_mgr_mm.o obj-$(CONFIG_GUNYAH) += gunyah.o diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index a43401cb34f7d..297427952b8c7 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -15,6 +15,8 @@ #include "vm_mgr.h" +static void gh_vm_free(struct work_struct *work); + static __must_check struct gh_vm *gh_vm_alloc(struct gh_rm *rm) { struct gh_vm *ghvm; @@ -26,20 +28,72 @@ static __must_check struct gh_vm *gh_vm_alloc(struct gh_rm *rm) ghvm->parent = gh_rm_get(rm); ghvm->rm = rm; + mmgrab(current->mm); + ghvm->mm = current->mm; + mutex_init(&ghvm->mm_lock); + INIT_LIST_HEAD(&ghvm->memory_mappings); + INIT_WORK(&ghvm->free_work, gh_vm_free); + return ghvm; } -static int gh_vm_release(struct inode *inode, struct file *filp) +static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) { struct gh_vm *ghvm = filp->private_data; + void __user *argp = (void __user *)arg; + long r; + + switch (cmd) { + case GH_VM_SET_USER_MEM_REGION: { + struct gh_userspace_memory_region region; + + /* only allow owner task to add memory */ + if (ghvm->mm != current->mm) + return -EPERM; + + if (copy_from_user(®ion, argp, sizeof(region))) + return -EFAULT; + + /* All other flag bits are reserved for future use */ + if (region.flags & ~(GH_MEM_ALLOW_READ | GH_MEM_ALLOW_WRITE | GH_MEM_ALLOW_EXEC)) + return -EINVAL; + + r = gh_vm_mem_alloc(ghvm, ®ion); + break; + } + default: + r = -ENOTTY; + break; + } + return r; +} + +static void gh_vm_free(struct work_struct *work) +{ + struct gh_vm *ghvm = container_of(work, struct gh_vm, free_work); + + gh_vm_mem_reclaim(ghvm); gh_rm_put(ghvm->rm); + mmdrop(ghvm->mm); kfree(ghvm); +} + +static int gh_vm_release(struct inode *inode, struct file *filp) +{ + struct gh_vm *ghvm = filp->private_data; + + /* VM will be reset and make RM calls which can interruptible sleep. + * Defer to a work so this thread can receive signal. + */ + schedule_work(&ghvm->free_work); return 0; } static const struct file_operations gh_vm_fops = { .owner = THIS_MODULE, + .unlocked_ioctl = gh_vm_ioctl, + .compat_ioctl = compat_ptr_ioctl, .release = gh_vm_release, .llseek = noop_llseek, }; @@ -77,8 +131,7 @@ static long gh_dev_ioctl_create_vm(struct gh_rm *rm, unsigned long arg) err_put_fd: put_unused_fd(fd); err_destroy_vm: - gh_rm_put(ghvm->rm); - kfree(ghvm); + gh_vm_free(&ghvm->free_work); return err; } diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 1e94b58d7d34d..434ef9f662a7a 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -7,14 +7,40 @@ #define _GH_VM_MGR_H #include +#include +#include +#include #include long gh_dev_vm_mgr_ioctl(struct gh_rm *rm, unsigned int cmd, unsigned long arg); +enum gh_vm_mem_share_type { + VM_MEM_SHARE, + VM_MEM_LEND, +}; + +struct gh_vm_mem { + struct list_head list; + enum gh_vm_mem_share_type share_type; + struct gh_rm_mem_parcel parcel; + + __u64 guest_phys_addr; + struct page **pages; + unsigned long npages; +}; + struct gh_vm { struct gh_rm *rm; struct device *parent; + + struct work_struct free_work; + struct mm_struct *mm; /* userspace tied to this vm */ + struct mutex mm_lock; + struct list_head memory_mappings; }; +int gh_vm_mem_alloc(struct gh_vm *ghvm, struct gh_userspace_memory_region *region); +void gh_vm_mem_reclaim(struct gh_vm *ghvm); + #endif diff --git a/drivers/virt/gunyah/vm_mgr_mm.c b/drivers/virt/gunyah/vm_mgr_mm.c new file mode 100644 index 0000000000000..6974607f02edd --- /dev/null +++ b/drivers/virt/gunyah/vm_mgr_mm.c @@ -0,0 +1,232 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gh_vm_mgr: " fmt + +#include +#include + +#include + +#include "vm_mgr.h" + +static bool pages_are_mergeable(struct page *a, struct page *b) +{ + return page_to_pfn(a) + 1 == page_to_pfn(b); +} + +static bool gh_vm_mem_overlap(struct gh_vm_mem *a, u64 addr, u64 size) +{ + u64 a_end = a->guest_phys_addr + (a->npages << PAGE_SHIFT); + u64 end = addr + size; + + return a->guest_phys_addr < end && addr < a_end; +} + +static struct gh_vm_mem *__gh_vm_mem_find_by_label(struct gh_vm *ghvm, u32 label) + __must_hold(&ghvm->mm_lock) +{ + struct gh_vm_mem *mapping; + + list_for_each_entry(mapping, &ghvm->memory_mappings, list) + if (mapping->parcel.label == label) + return mapping; + + return NULL; +} + +static void gh_vm_mem_reclaim_mapping(struct gh_vm *ghvm, struct gh_vm_mem *mapping) + __must_hold(&ghvm->mm_lock) +{ + int ret = 0; + + if (mapping->parcel.mem_handle != GH_MEM_HANDLE_INVAL) { + ret = gh_rm_mem_reclaim(ghvm->rm, &mapping->parcel); + if (ret) + pr_warn("Failed to reclaim memory parcel for label %d: %d\n", + mapping->parcel.label, ret); + } + + if (!ret) { + unpin_user_pages(mapping->pages, mapping->npages); + account_locked_vm(ghvm->mm, mapping->npages, false); + } + + kfree(mapping->pages); + kfree(mapping->parcel.acl_entries); + kfree(mapping->parcel.mem_entries); + + list_del(&mapping->list); +} + +void gh_vm_mem_reclaim(struct gh_vm *ghvm) +{ + struct gh_vm_mem *mapping, *tmp; + + mutex_lock(&ghvm->mm_lock); + + list_for_each_entry_safe(mapping, tmp, &ghvm->memory_mappings, list) { + gh_vm_mem_reclaim_mapping(ghvm, mapping); + kfree(mapping); + } + + mutex_unlock(&ghvm->mm_lock); +} + +int gh_vm_mem_alloc(struct gh_vm *ghvm, struct gh_userspace_memory_region *region) +{ + struct gh_vm_mem *mapping, *tmp_mapping; + struct page *curr_page, *prev_page; + struct gh_rm_mem_parcel *parcel; + int i, j, pinned, ret = 0; + unsigned int gup_flags; + size_t entry_size; + u16 vmid; + + if (!region->memory_size || !PAGE_ALIGNED(region->memory_size) || + !PAGE_ALIGNED(region->userspace_addr) || + !PAGE_ALIGNED(region->guest_phys_addr)) + return -EINVAL; + + if (overflows_type(region->guest_phys_addr + region->memory_size, u64)) + return -EOVERFLOW; + + ret = mutex_lock_interruptible(&ghvm->mm_lock); + if (ret) + return ret; + + mapping = __gh_vm_mem_find_by_label(ghvm, region->label); + if (mapping) { + ret = -EEXIST; + goto unlock; + } + + list_for_each_entry(tmp_mapping, &ghvm->memory_mappings, list) { + if (gh_vm_mem_overlap(tmp_mapping, region->guest_phys_addr, + region->memory_size)) { + ret = -EEXIST; + goto unlock; + } + } + + mapping = kzalloc(sizeof(*mapping), GFP_KERNEL_ACCOUNT); + if (!mapping) { + ret = -ENOMEM; + goto unlock; + } + + mapping->guest_phys_addr = region->guest_phys_addr; + mapping->npages = region->memory_size >> PAGE_SHIFT; + parcel = &mapping->parcel; + parcel->label = region->label; + parcel->mem_handle = GH_MEM_HANDLE_INVAL; /* to be filled later by mem_share/mem_lend */ + parcel->mem_type = GH_RM_MEM_TYPE_NORMAL; + + ret = account_locked_vm(ghvm->mm, mapping->npages, true); + if (ret) + goto free_mapping; + + mapping->pages = kcalloc(mapping->npages, sizeof(*mapping->pages), GFP_KERNEL_ACCOUNT); + if (!mapping->pages) { + ret = -ENOMEM; + mapping->npages = 0; /* update npages for reclaim */ + goto unlock_pages; + } + + gup_flags = FOLL_LONGTERM; + if (region->flags & GH_MEM_ALLOW_WRITE) + gup_flags |= FOLL_WRITE; + + pinned = pin_user_pages_fast(region->userspace_addr, mapping->npages, + gup_flags, mapping->pages); + if (pinned < 0) { + ret = pinned; + goto free_pages; + } else if (pinned != mapping->npages) { + ret = -EFAULT; + mapping->npages = pinned; /* update npages for reclaim */ + goto unpin_pages; + } + + parcel->n_acl_entries = 2; + mapping->share_type = VM_MEM_SHARE; + parcel->acl_entries = kcalloc(parcel->n_acl_entries, sizeof(*parcel->acl_entries), + GFP_KERNEL); + if (!parcel->acl_entries) { + ret = -ENOMEM; + goto unpin_pages; + } + + /* acl_entries[0].vmid will be this VM's vmid. We'll fill it when the + * VM is starting and we know the VM's vmid. + */ + if (region->flags & GH_MEM_ALLOW_READ) + parcel->acl_entries[0].perms |= GH_RM_ACL_R; + if (region->flags & GH_MEM_ALLOW_WRITE) + parcel->acl_entries[0].perms |= GH_RM_ACL_W; + if (region->flags & GH_MEM_ALLOW_EXEC) + parcel->acl_entries[0].perms |= GH_RM_ACL_X; + + ret = gh_rm_get_vmid(ghvm->rm, &vmid); + if (ret) + goto free_acl; + + parcel->acl_entries[1].vmid = cpu_to_le16(vmid); + /* Host assumed to have all these permissions. Gunyah will not + * grant new permissions if host actually had less than RWX + */ + parcel->acl_entries[1].perms = GH_RM_ACL_R | GH_RM_ACL_W | GH_RM_ACL_X; + + parcel->n_mem_entries = 1; + for (i = 1; i < mapping->npages; i++) { + if (!pages_are_mergeable(mapping->pages[i - 1], mapping->pages[i])) + parcel->n_mem_entries++; + } + + parcel->mem_entries = kcalloc(parcel->n_mem_entries, + sizeof(parcel->mem_entries[0]), + GFP_KERNEL_ACCOUNT); + if (!parcel->mem_entries) { + ret = -ENOMEM; + goto free_acl; + } + + /* reduce number of entries by combining contiguous pages into single memory entry */ + prev_page = mapping->pages[0]; + parcel->mem_entries[0].phys_addr = cpu_to_le64(page_to_phys(prev_page)); + entry_size = PAGE_SIZE; + for (i = 1, j = 0; i < mapping->npages; i++) { + curr_page = mapping->pages[i]; + if (pages_are_mergeable(prev_page, curr_page)) { + entry_size += PAGE_SIZE; + } else { + parcel->mem_entries[j].size = cpu_to_le64(entry_size); + j++; + parcel->mem_entries[j].phys_addr = + cpu_to_le64(page_to_phys(curr_page)); + entry_size = PAGE_SIZE; + } + + prev_page = curr_page; + } + parcel->mem_entries[j].size = cpu_to_le64(entry_size); + + list_add(&mapping->list, &ghvm->memory_mappings); + mutex_unlock(&ghvm->mm_lock); + return 0; +free_acl: + kfree(parcel->acl_entries); +unpin_pages: + unpin_user_pages(mapping->pages, pinned); +free_pages: + kfree(mapping->pages); +unlock_pages: + account_locked_vm(ghvm->mm, mapping->npages, false); +free_mapping: + kfree(mapping); +unlock: + mutex_unlock(&ghvm->mm_lock); + return ret; +} diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index 86b9cb60118dd..91d6dd26fcc89 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -20,4 +20,41 @@ */ #define GH_CREATE_VM _IO(GH_IOCTL_TYPE, 0x0) /* Returns a Gunyah VM fd */ +/* + * ioctls for VM fds + */ + +/** + * enum gh_mem_flags - Possible flags on &struct gh_userspace_memory_region + * @GH_MEM_ALLOW_READ: Allow guest to read the memory + * @GH_MEM_ALLOW_WRITE: Allow guest to write to the memory + * @GH_MEM_ALLOW_EXEC: Allow guest to execute instructions in the memory + */ +enum gh_mem_flags { + GH_MEM_ALLOW_READ = 1UL << 0, + GH_MEM_ALLOW_WRITE = 1UL << 1, + GH_MEM_ALLOW_EXEC = 1UL << 2, +}; + +/** + * struct gh_userspace_memory_region - Userspace memory descripion for GH_VM_SET_USER_MEM_REGION + * @label: Identifer to the region which is unique to the VM. + * @flags: Flags for memory parcel behavior. See &enum gh_mem_flags. + * @guest_phys_addr: Location of the memory region in guest's memory space (page-aligned) + * @memory_size: Size of the region (page-aligned) + * @userspace_addr: Location of the memory region in caller (userspace)'s memory + * + * See Documentation/virt/gunyah/vm-manager.rst for further details. + */ +struct gh_userspace_memory_region { + __u32 label; + __u32 flags; + __u64 guest_phys_addr; + __u64 memory_size; + __u64 userspace_addr; +}; + +#define GH_VM_SET_USER_MEM_REGION _IOW(GH_IOCTL_TYPE, 0x1, \ + struct gh_userspace_memory_region) + #endif From patchwork Tue Jun 13 17:20:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 692123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 389A5EB64DA for ; Tue, 13 Jun 2023 17:23:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236598AbjFMRXE (ORCPT ); Tue, 13 Jun 2023 13:23:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50260 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239098AbjFMRW3 (ORCPT ); Tue, 13 Jun 2023 13:22:29 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3BA4D1FC8; Tue, 13 Jun 2023 10:22:02 -0700 (PDT) Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35DAxhWj025649; Tue, 13 Jun 2023 17:21:40 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=j1WJ1ACZtU3TBr3O7B4TkWADaRm3J6sg79CmXLkh6K4=; b=P8uHBCG5tLiNfP8vLVV+LCGeJzlNGtU2N+haARnGX+jgorl++btbxZrES1B5nlRYb06B 4kl9qmQfcbUxp5QxjQ3H40Flj7FCKeK3AIemqdcRyimHVTRjOFBjUZPpEvvXQfvl7dJI QTkaw8tjDGILbAv0DT0pMXVp1tawLRp15YdKaEWXbLTqpPuUxEIjfxLjCIR7iA13Yt+x xH6eON1HyjLA60z6z1LxcLOyW2/5odRxaPz48uRd81Gjebo+QnWSYQUC2vpFFiEIeQxq ZBgksjFxyJ6L4GQpUVsrQ6hFand04Sv6on9WFVQ/bmSfNYyD/jGYhxerxxa3rQoVtxsA YQ== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r68x9ae0t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:39 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 35DHLbB1004303 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:37 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 13 Jun 2023 10:21:36 -0700 From: Elliot Berman To: Alex Elder , Srinivas Kandagatla , Elliot Berman , Prakruthi Deepak Heragu CC: Murali Nalajala , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Dmitry Baryshkov , Bjorn Andersson , "Konrad Dybcio" , Arnd Bergmann , "Greg Kroah-Hartman" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Jonathan Corbet , Bagas Sanjaya , Will Deacon , Andy Gross , Catalin Marinas , Jassi Brar , , , , , Subject: [PATCH v14 12/25] gunyah: vm_mgr: Add ioctls to support basic non-proxy VM boot Date: Tue, 13 Jun 2023 10:20:40 -0700 Message-ID: <20230613172054.3959700-13-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230613172054.3959700-1-quic_eberman@quicinc.com> References: <20230613172054.3959700-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 8niVjKHBNKRZO9zuhHzATAYyAqSIGf2X X-Proofpoint-ORIG-GUID: 8niVjKHBNKRZO9zuhHzATAYyAqSIGf2X X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-13_19,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 priorityscore=1501 lowpriorityscore=0 impostorscore=0 spamscore=0 suspectscore=0 mlxlogscore=999 mlxscore=0 adultscore=0 malwarescore=0 clxscore=1015 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130153 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Add remaining ioctls to support non-proxy VM boot: - Gunyah Resource Manager uses the VM's devicetree to configure the virtual machine. The location of the devicetree in the guest's virtual memory can be declared via the SET_DTB_CONFIG ioctl. - Trigger start of the virtual machine with VM_START ioctl. Reviewed-by: Alex Elder Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/vm_mgr.c | 215 ++++++++++++++++++++++++++++++++ drivers/virt/gunyah/vm_mgr.h | 11 ++ drivers/virt/gunyah/vm_mgr_mm.c | 20 +++ include/uapi/linux/gunyah.h | 15 +++ 4 files changed, 261 insertions(+) diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index 297427952b8c7..562ae6ed4a5f1 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -17,6 +17,68 @@ static void gh_vm_free(struct work_struct *work); +static int gh_vm_rm_notification_status(struct gh_vm *ghvm, void *data) +{ + struct gh_rm_vm_status_payload *payload = data; + + if (le16_to_cpu(payload->vmid) != ghvm->vmid) + return NOTIFY_OK; + + /* All other state transitions are synchronous to a corresponding RM call */ + if (payload->vm_status == GH_RM_VM_STATUS_RESET) { + down_write(&ghvm->status_lock); + ghvm->vm_status = payload->vm_status; + up_write(&ghvm->status_lock); + wake_up(&ghvm->vm_status_wait); + } + + return NOTIFY_DONE; +} + +static int gh_vm_rm_notification_exited(struct gh_vm *ghvm, void *data) +{ + struct gh_rm_vm_exited_payload *payload = data; + + if (le16_to_cpu(payload->vmid) != ghvm->vmid) + return NOTIFY_OK; + + down_write(&ghvm->status_lock); + ghvm->vm_status = GH_RM_VM_STATUS_EXITED; + up_write(&ghvm->status_lock); + wake_up(&ghvm->vm_status_wait); + + return NOTIFY_DONE; +} + +static int gh_vm_rm_notification(struct notifier_block *nb, unsigned long action, void *data) +{ + struct gh_vm *ghvm = container_of(nb, struct gh_vm, nb); + + switch (action) { + case GH_RM_NOTIFICATION_VM_STATUS: + return gh_vm_rm_notification_status(ghvm, data); + case GH_RM_NOTIFICATION_VM_EXITED: + return gh_vm_rm_notification_exited(ghvm, data); + default: + return NOTIFY_OK; + } +} + +static void gh_vm_stop(struct gh_vm *ghvm) +{ + int ret; + + down_write(&ghvm->status_lock); + if (ghvm->vm_status == GH_RM_VM_STATUS_RUNNING) { + ret = gh_rm_vm_stop(ghvm->rm, ghvm->vmid); + if (ret) + dev_warn(ghvm->parent, "Failed to stop VM: %d\n", ret); + } + up_write(&ghvm->status_lock); + + wait_event(ghvm->vm_status_wait, ghvm->vm_status == GH_RM_VM_STATUS_EXITED); +} + static __must_check struct gh_vm *gh_vm_alloc(struct gh_rm *rm) { struct gh_vm *ghvm; @@ -26,17 +88,130 @@ static __must_check struct gh_vm *gh_vm_alloc(struct gh_rm *rm) return ERR_PTR(-ENOMEM); ghvm->parent = gh_rm_get(rm); + ghvm->vmid = GH_VMID_INVAL; ghvm->rm = rm; mmgrab(current->mm); ghvm->mm = current->mm; mutex_init(&ghvm->mm_lock); INIT_LIST_HEAD(&ghvm->memory_mappings); + init_rwsem(&ghvm->status_lock); + init_waitqueue_head(&ghvm->vm_status_wait); INIT_WORK(&ghvm->free_work, gh_vm_free); + ghvm->vm_status = GH_RM_VM_STATUS_NO_STATE; return ghvm; } +static int gh_vm_start(struct gh_vm *ghvm) +{ + struct gh_vm_mem *mapping; + u64 dtb_offset; + u32 mem_handle; + int ret; + + down_write(&ghvm->status_lock); + if (ghvm->vm_status != GH_RM_VM_STATUS_NO_STATE) { + up_write(&ghvm->status_lock); + return 0; + } + + ghvm->nb.notifier_call = gh_vm_rm_notification; + ret = gh_rm_notifier_register(ghvm->rm, &ghvm->nb); + if (ret) + goto err; + + ret = gh_rm_alloc_vmid(ghvm->rm, 0); + if (ret < 0) { + gh_rm_notifier_unregister(ghvm->rm, &ghvm->nb); + goto err; + } + ghvm->vmid = ret; + ghvm->vm_status = GH_RM_VM_STATUS_LOAD; + + mutex_lock(&ghvm->mm_lock); + list_for_each_entry(mapping, &ghvm->memory_mappings, list) { + mapping->parcel.acl_entries[0].vmid = cpu_to_le16(ghvm->vmid); + ret = gh_rm_mem_share(ghvm->rm, &mapping->parcel); + if (ret) { + dev_warn(ghvm->parent, "Failed to share parcel %d: %d\n", + mapping->parcel.label, ret); + mutex_unlock(&ghvm->mm_lock); + goto err; + } + } + mutex_unlock(&ghvm->mm_lock); + + mapping = gh_vm_mem_find_by_addr(ghvm, ghvm->dtb_config.guest_phys_addr, + ghvm->dtb_config.size); + if (!mapping) { + dev_warn(ghvm->parent, "Failed to find the memory_handle for DTB\n"); + ret = -EINVAL; + goto err; + } + + mem_handle = mapping->parcel.mem_handle; + dtb_offset = ghvm->dtb_config.guest_phys_addr - mapping->guest_phys_addr; + + ret = gh_rm_vm_configure(ghvm->rm, ghvm->vmid, ghvm->auth, mem_handle, + 0, 0, dtb_offset, ghvm->dtb_config.size); + if (ret) { + dev_warn(ghvm->parent, "Failed to configure VM: %d\n", ret); + goto err; + } + + ret = gh_rm_vm_init(ghvm->rm, ghvm->vmid); + if (ret) { + ghvm->vm_status = GH_RM_VM_STATUS_INIT_FAILED; + dev_warn(ghvm->parent, "Failed to initialize VM: %d\n", ret); + goto err; + } + ghvm->vm_status = GH_RM_VM_STATUS_READY; + + ret = gh_rm_vm_start(ghvm->rm, ghvm->vmid); + if (ret) { + dev_warn(ghvm->parent, "Failed to start VM: %d\n", ret); + goto err; + } + + ghvm->vm_status = GH_RM_VM_STATUS_RUNNING; + up_write(&ghvm->status_lock); + return ret; +err: + /* gh_vm_free will handle releasing resources and reclaiming memory */ + up_write(&ghvm->status_lock); + return ret; +} + +static int gh_vm_ensure_started(struct gh_vm *ghvm) +{ + int ret; + + ret = down_read_interruptible(&ghvm->status_lock); + if (ret) + return ret; + + /* Unlikely because VM is typically started */ + if (unlikely(ghvm->vm_status == GH_RM_VM_STATUS_NO_STATE)) { + up_read(&ghvm->status_lock); + ret = gh_vm_start(ghvm); + if (ret) + return ret; + /** gh_vm_start() is guaranteed to bring status out of + * GH_RM_VM_STATUS_LOAD, thus infinitely recursive call is not + * possible + */ + return gh_vm_ensure_started(ghvm); + } + + /* Unlikely because VM is typically running */ + if (unlikely(ghvm->vm_status != GH_RM_VM_STATUS_RUNNING)) + ret = -ENODEV; + + up_read(&ghvm->status_lock); + return ret; +} + static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) { struct gh_vm *ghvm = filp->private_data; @@ -61,6 +236,24 @@ static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) r = gh_vm_mem_alloc(ghvm, ®ion); break; } + case GH_VM_SET_DTB_CONFIG: { + struct gh_vm_dtb_config dtb_config; + + if (copy_from_user(&dtb_config, argp, sizeof(dtb_config))) + return -EFAULT; + + if (overflows_type(dtb_config.guest_phys_addr + dtb_config.size, u64)) + return -EOVERFLOW; + + ghvm->dtb_config = dtb_config; + + r = 0; + break; + } + case GH_VM_START: { + r = gh_vm_ensure_started(ghvm); + break; + } default: r = -ENOTTY; break; @@ -72,8 +265,30 @@ static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) static void gh_vm_free(struct work_struct *work) { struct gh_vm *ghvm = container_of(work, struct gh_vm, free_work); + int ret; + + if (ghvm->vm_status == GH_RM_VM_STATUS_RUNNING) + gh_vm_stop(ghvm); + + if (ghvm->vm_status != GH_RM_VM_STATUS_NO_STATE && + ghvm->vm_status != GH_RM_VM_STATUS_LOAD && + ghvm->vm_status != GH_RM_VM_STATUS_RESET) { + ret = gh_rm_vm_reset(ghvm->rm, ghvm->vmid); + if (ret) + dev_err(ghvm->parent, "Failed to reset the vm: %d\n", ret); + wait_event(ghvm->vm_status_wait, ghvm->vm_status == GH_RM_VM_STATUS_RESET); + } gh_vm_mem_reclaim(ghvm); + + if (ghvm->vm_status > GH_RM_VM_STATUS_NO_STATE) { + gh_rm_notifier_unregister(ghvm->rm, &ghvm->nb); + + ret = gh_rm_dealloc_vmid(ghvm->rm, ghvm->vmid); + if (ret) + dev_warn(ghvm->parent, "Failed to deallocate vmid: %d\n", ret); + } + gh_rm_put(ghvm->rm); mmdrop(ghvm->mm); kfree(ghvm); diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 434ef9f662a7a..4173bd51f83fe 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -10,6 +10,8 @@ #include #include #include +#include +#include #include @@ -31,8 +33,16 @@ struct gh_vm_mem { }; struct gh_vm { + u16 vmid; struct gh_rm *rm; struct device *parent; + enum gh_rm_vm_auth_mechanism auth; + struct gh_vm_dtb_config dtb_config; + + struct notifier_block nb; + enum gh_rm_vm_status vm_status; + wait_queue_head_t vm_status_wait; + struct rw_semaphore status_lock; struct work_struct free_work; struct mm_struct *mm; /* userspace tied to this vm */ @@ -42,5 +52,6 @@ struct gh_vm { int gh_vm_mem_alloc(struct gh_vm *ghvm, struct gh_userspace_memory_region *region); void gh_vm_mem_reclaim(struct gh_vm *ghvm); +struct gh_vm_mem *gh_vm_mem_find_by_addr(struct gh_vm *ghvm, u64 guest_phys_addr, u32 size); #endif diff --git a/drivers/virt/gunyah/vm_mgr_mm.c b/drivers/virt/gunyah/vm_mgr_mm.c index 6974607f02edd..f8a09e86d6bc1 100644 --- a/drivers/virt/gunyah/vm_mgr_mm.c +++ b/drivers/virt/gunyah/vm_mgr_mm.c @@ -75,6 +75,26 @@ void gh_vm_mem_reclaim(struct gh_vm *ghvm) mutex_unlock(&ghvm->mm_lock); } +struct gh_vm_mem *gh_vm_mem_find_by_addr(struct gh_vm *ghvm, u64 guest_phys_addr, u32 size) +{ + struct gh_vm_mem *mapping; + + if (overflows_type(guest_phys_addr + size, u64)) + return NULL; + + mutex_lock(&ghvm->mm_lock); + + list_for_each_entry(mapping, &ghvm->memory_mappings, list) { + if (gh_vm_mem_overlap(mapping, guest_phys_addr, size)) + goto unlock; + } + + mapping = NULL; +unlock: + mutex_unlock(&ghvm->mm_lock); + return mapping; +} + int gh_vm_mem_alloc(struct gh_vm *ghvm, struct gh_userspace_memory_region *region) { struct gh_vm_mem *mapping, *tmp_mapping; diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index 91d6dd26fcc89..4b63d0b9b8ba7 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -57,4 +57,19 @@ struct gh_userspace_memory_region { #define GH_VM_SET_USER_MEM_REGION _IOW(GH_IOCTL_TYPE, 0x1, \ struct gh_userspace_memory_region) +/** + * struct gh_vm_dtb_config - Set the location of the VM's devicetree blob + * @guest_phys_addr: Address of the VM's devicetree in guest memory. + * @size: Maximum size of the devicetree including space for overlays. + * Resource manager applies an overlay to the DTB and dtb_size should + * include room for the overlay. A page of memory is typicaly plenty. + */ +struct gh_vm_dtb_config { + __u64 guest_phys_addr; + __u64 size; +}; +#define GH_VM_SET_DTB_CONFIG _IOW(GH_IOCTL_TYPE, 0x2, struct gh_vm_dtb_config) + +#define GH_VM_START _IO(GH_IOCTL_TYPE, 0x3) + #endif From patchwork Tue Jun 13 17:20:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 692120 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC16EEB64D7 for ; Tue, 13 Jun 2023 17:23:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239797AbjFMRXs (ORCPT ); Tue, 13 Jun 2023 13:23:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239463AbjFMRXC (ORCPT ); Tue, 13 Jun 2023 13:23:02 -0400 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 618D9211D; Tue, 13 Jun 2023 10:22:12 -0700 (PDT) Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35DDKH59028476; Tue, 13 Jun 2023 17:21:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=f50WDWExzuV0WDZKHgEPO+SK5wQsm5rHlLHq49sA+NE=; b=iZrBt7arR1cCHDoGvudN85t5XigwPyaRz/BJVkKluMpBKkgt3rKcda7rnequ/m75ayyq D5La5vcO//fOd0XMEjAyd7XkfgXZmX91gpgSDP8oI872AbBa8imVLfpbw1/ZWvNy9WEj cPZ+UAN5+mh471FFZ6wuuwo0lG/qU78qenbvDhZubwQn7PIBVkl9rsFdIlC9bn1gH1Ek r9sJRcW6RLg6M4BN2atpUw5zhK+lPNsA3hGm2ZGF/vIIQ1TSxZKMAYM/QzJrx76R2V/a x33jpnq17YI35yGQ1ugaVfKBPnIt/Tg6hhckRPUgxdf6nKTkdpRqK5/mk2jTn9ZY0SG2 Vw== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r6nqh15fm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:44 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 35DHLguO029618 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:42 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 13 Jun 2023 10:21:41 -0700 From: Elliot Berman To: Alex Elder , Srinivas Kandagatla , Elliot Berman , Prakruthi Deepak Heragu CC: Murali Nalajala , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Dmitry Baryshkov , Bjorn Andersson , "Konrad Dybcio" , Arnd Bergmann , "Greg Kroah-Hartman" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Jonathan Corbet , Bagas Sanjaya , Will Deacon , Andy Gross , Catalin Marinas , Jassi Brar , , , , , Subject: [PATCH v14 15/25] virt: gunyah: Add Qualcomm Gunyah platform ops Date: Tue, 13 Jun 2023 10:20:43 -0700 Message-ID: <20230613172054.3959700-16-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230613172054.3959700-1-quic_eberman@quicinc.com> References: <20230613172054.3959700-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: nmODhVbsGYehKqWU9qWXL4rYeAJS1SiV X-Proofpoint-ORIG-GUID: nmODhVbsGYehKqWU9qWXL4rYeAJS1SiV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-13_18,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 lowpriorityscore=0 malwarescore=0 mlxlogscore=999 bulkscore=0 suspectscore=0 mlxscore=0 priorityscore=1501 impostorscore=0 spamscore=0 phishscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130153 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Qualcomm platforms have a firmware entity which performs access control to physical pages. Dynamically started Gunyah virtual machines use the QCOM_SCM_RM_MANAGED_VMID for access. Linux thus needs to assign access to the memory used by guest VMs. Gunyah doesn't do this operation for us since it is the current VM (typically VMID_HLOS) delegating the access and not Gunyah itself. Use the Gunyah platform ops to achieve this so that only Qualcomm platforms attempt to make the needed SCM calls. Reviewed-by: Alex Elder Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Kconfig | 13 +++ drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_qcom.c | 153 ++++++++++++++++++++++++++++++ 3 files changed, 167 insertions(+) create mode 100644 drivers/virt/gunyah/gunyah_qcom.c diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index de815189dab6c..0421b751aad4f 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -5,6 +5,7 @@ config GUNYAH depends on ARM64 depends on MAILBOX select GUNYAH_PLATFORM_HOOKS + imply GUNYAH_QCOM_PLATFORM if ARCH_QCOM help The Gunyah drivers are the helper interfaces that run in a guest VM such as basic inter-VM IPC and signaling mechanisms, and higher level @@ -15,3 +16,15 @@ config GUNYAH config GUNYAH_PLATFORM_HOOKS tristate + +config GUNYAH_QCOM_PLATFORM + tristate "Support for Gunyah on Qualcomm platforms" + depends on GUNYAH + select GUNYAH_PLATFORM_HOOKS + select QCOM_SCM + help + Enable support for interacting with Gunyah on Qualcomm + platforms. Interaction with Qualcomm firmware requires + extra platform-specific support. + + Say Y/M here to use Gunyah on Qualcomm platforms. diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 4fbeee521d60a..2aa9ff038ed02 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_GUNYAH_PLATFORM_HOOKS) += gunyah_platform_hooks.o +obj-$(CONFIG_GUNYAH_QCOM_PLATFORM) += gunyah_qcom.o gunyah-y += rsc_mgr.o rsc_mgr_rpc.o vm_mgr.o vm_mgr_mm.o obj-$(CONFIG_GUNYAH) += gunyah.o diff --git a/drivers/virt/gunyah/gunyah_qcom.c b/drivers/virt/gunyah/gunyah_qcom.c new file mode 100644 index 0000000000000..f06a598f2e1b3 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_qcom.c @@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include + +#define QCOM_SCM_RM_MANAGED_VMID 0x3A +#define QCOM_SCM_MAX_MANAGED_VMID 0x3F + +static int qcom_scm_gh_rm_pre_mem_share(struct gh_rm *rm, struct gh_rm_mem_parcel *mem_parcel) +{ + struct qcom_scm_vmperm *new_perms; + u64 src, src_cpy; + int ret = 0, i, n; + u16 vmid; + + new_perms = kcalloc(mem_parcel->n_acl_entries, sizeof(*new_perms), GFP_KERNEL); + if (!new_perms) + return -ENOMEM; + + for (n = 0; n < mem_parcel->n_acl_entries; n++) { + vmid = le16_to_cpu(mem_parcel->acl_entries[n].vmid); + if (vmid <= QCOM_SCM_MAX_MANAGED_VMID) + new_perms[n].vmid = vmid; + else + new_perms[n].vmid = QCOM_SCM_RM_MANAGED_VMID; + if (mem_parcel->acl_entries[n].perms & GH_RM_ACL_X) + new_perms[n].perm |= QCOM_SCM_PERM_EXEC; + if (mem_parcel->acl_entries[n].perms & GH_RM_ACL_W) + new_perms[n].perm |= QCOM_SCM_PERM_WRITE; + if (mem_parcel->acl_entries[n].perms & GH_RM_ACL_R) + new_perms[n].perm |= QCOM_SCM_PERM_READ; + } + + src = BIT_ULL(QCOM_SCM_VMID_HLOS); + + for (i = 0; i < mem_parcel->n_mem_entries; i++) { + src_cpy = src; + ret = qcom_scm_assign_mem(le64_to_cpu(mem_parcel->mem_entries[i].phys_addr), + le64_to_cpu(mem_parcel->mem_entries[i].size), + &src_cpy, new_perms, mem_parcel->n_acl_entries); + if (ret) + break; + } + + if (!ret) + goto out; + + src = 0; + for (n = 0; n < mem_parcel->n_acl_entries; n++) { + vmid = le16_to_cpu(mem_parcel->acl_entries[n].vmid); + if (vmid <= QCOM_SCM_MAX_MANAGED_VMID) + src |= BIT_ULL(vmid); + else + src |= BIT_ULL(QCOM_SCM_RM_MANAGED_VMID); + } + + new_perms[0].vmid = QCOM_SCM_VMID_HLOS; + + for (i--; i >= 0; i--) { + src_cpy = src; + WARN_ON_ONCE(qcom_scm_assign_mem( + le64_to_cpu(mem_parcel->mem_entries[i].phys_addr), + le64_to_cpu(mem_parcel->mem_entries[i].size), + &src_cpy, new_perms, 1)); + } + +out: + kfree(new_perms); + return ret; +} + +static int qcom_scm_gh_rm_post_mem_reclaim(struct gh_rm *rm, struct gh_rm_mem_parcel *mem_parcel) +{ + struct qcom_scm_vmperm new_perms; + u64 src = 0, src_cpy; + int ret = 0, i, n; + u16 vmid; + + new_perms.vmid = QCOM_SCM_VMID_HLOS; + new_perms.perm = QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | QCOM_SCM_PERM_READ; + + for (n = 0; n < mem_parcel->n_acl_entries; n++) { + vmid = le16_to_cpu(mem_parcel->acl_entries[n].vmid); + if (vmid <= QCOM_SCM_MAX_MANAGED_VMID) + src |= (1ull << vmid); + else + src |= (1ull << QCOM_SCM_RM_MANAGED_VMID); + } + + for (i = 0; i < mem_parcel->n_mem_entries; i++) { + src_cpy = src; + ret = qcom_scm_assign_mem(le64_to_cpu(mem_parcel->mem_entries[i].phys_addr), + le64_to_cpu(mem_parcel->mem_entries[i].size), + &src_cpy, &new_perms, 1); + WARN_ON_ONCE(ret); + } + + return ret; +} + +static struct gh_rm_platform_ops qcom_scm_gh_rm_platform_ops = { + .pre_mem_share = qcom_scm_gh_rm_pre_mem_share, + .post_mem_reclaim = qcom_scm_gh_rm_post_mem_reclaim, +}; + +/* {19bd54bd-0b37-571b-946f-609b54539de6} */ +static const uuid_t QCOM_EXT_UUID = + UUID_INIT(0x19bd54bd, 0x0b37, 0x571b, 0x94, 0x6f, 0x60, 0x9b, 0x54, 0x53, 0x9d, 0xe6); + +#define GH_QCOM_EXT_CALL_UUID_ID ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32, \ + ARM_SMCCC_OWNER_VENDOR_HYP, 0x3f01) + +static bool gh_has_qcom_extensions(void) +{ + struct arm_smccc_res res; + uuid_t uuid; + u32 *up; + + arm_smccc_1_1_smc(GH_QCOM_EXT_CALL_UUID_ID, &res); + + up = (u32 *)&uuid.b[0]; + up[0] = lower_32_bits(res.a0); + up[1] = lower_32_bits(res.a1); + up[2] = lower_32_bits(res.a2); + up[3] = lower_32_bits(res.a3); + + return uuid_equal(&uuid, &QCOM_EXT_UUID); +} + +static int __init qcom_gh_platform_hooks_register(void) +{ + if (!gh_has_qcom_extensions()) + return -ENODEV; + + return gh_rm_register_platform_ops(&qcom_scm_gh_rm_platform_ops); +} + +static void __exit qcom_gh_platform_hooks_unregister(void) +{ + gh_rm_unregister_platform_ops(&qcom_scm_gh_rm_platform_ops); +} + +module_init(qcom_gh_platform_hooks_register); +module_exit(qcom_gh_platform_hooks_unregister); +MODULE_DESCRIPTION("Qualcomm Technologies, Inc. Platform Hooks for Gunyah"); +MODULE_LICENSE("GPL"); From patchwork Tue Jun 13 17:20:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 692121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8239EB64DA for ; Tue, 13 Jun 2023 17:23:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239299AbjFMRX0 (ORCPT ); Tue, 13 Jun 2023 13:23:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239426AbjFMRXA (ORCPT ); Tue, 13 Jun 2023 13:23:00 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C4B162102; Tue, 13 Jun 2023 10:22:08 -0700 (PDT) Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35DEOTkM008970; Tue, 13 Jun 2023 17:21:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=0EoPusqu98de6/QYJRAJ5A5E0IpiAs/i2FtOF590R1g=; b=HRlNCUeMPEI4kXaSQbT6H/8DLqpQIC0iJgCN9qYH0RVfCJIcdLahkB9nnd6Ganc7zKii F1b9YxQQ0QWZVSi0hY6a/Lnj44/WHnSzazyV7h4S0apRRTHTRFhr53TnoJkS2pNVXwQo vRMa08t+z0EmhxaR7b3TWZZoJ3L6XlrPAbsaG8uK5MIWdmO5VjxuEFII56NNYPn2d8sz 5aoDZoY5cbTAm7Xl3kFPKQpT+h+ynRzebn1l3CoYZtHN9Ifgqog8JAEMvXhys8VSQHVG oXkhBgbhnDG8TokDtjaAjOROmPRQ9ydS1VHTqB0GZnq4zeZMq3hEZXZs+fseqfbXCBKV 9Q== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r6km41d4y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:44 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 35DHLiTb004391 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:44 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 13 Jun 2023 10:21:43 -0700 From: Elliot Berman To: Alex Elder , Srinivas Kandagatla , Elliot Berman , Prakruthi Deepak Heragu , Jonathan Corbet CC: Murali Nalajala , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Dmitry Baryshkov , Bjorn Andersson , "Konrad Dybcio" , Arnd Bergmann , "Greg Kroah-Hartman" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Bagas Sanjaya , Will Deacon , Andy Gross , Catalin Marinas , Jassi Brar , , , , , Subject: [PATCH v14 16/25] docs: gunyah: Document Gunyah VM Manager Date: Tue, 13 Jun 2023 10:20:44 -0700 Message-ID: <20230613172054.3959700-17-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230613172054.3959700-1-quic_eberman@quicinc.com> References: <20230613172054.3959700-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: 4Wj5x3wV_4OH3Cl_Y7491pAjq1I2qRVF X-Proofpoint-GUID: 4Wj5x3wV_4OH3Cl_Y7491pAjq1I2qRVF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-13_18,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 lowpriorityscore=0 malwarescore=0 priorityscore=1501 spamscore=0 impostorscore=0 phishscore=0 mlxscore=0 suspectscore=0 bulkscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130153 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Document the ioctls and usage of Gunyah VM Manager driver. Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/index.rst | 1 + Documentation/virt/gunyah/vm-manager.rst | 83 ++++++++++++++++++++++++ 2 files changed, 84 insertions(+) create mode 100644 Documentation/virt/gunyah/vm-manager.rst diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst index 74aa345e0a144..7058249825b16 100644 --- a/Documentation/virt/gunyah/index.rst +++ b/Documentation/virt/gunyah/index.rst @@ -7,6 +7,7 @@ Gunyah Hypervisor .. toctree:: :maxdepth: 1 + vm-manager message-queue Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in diff --git a/Documentation/virt/gunyah/vm-manager.rst b/Documentation/virt/gunyah/vm-manager.rst new file mode 100644 index 0000000000000..df0e1a8279bf5 --- /dev/null +++ b/Documentation/virt/gunyah/vm-manager.rst @@ -0,0 +1,83 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======================= +Virtual Machine Manager +======================= + +The Gunyah Virtual Machine Manager is a Linux driver to support launching +virtual machines using Gunyah. It presently supports launching virtual machines +scheduled by Gunyah's scheduler. + +Configuration of a Gunyah virtual machine is done via a devicetree. When the VM +is launched, memory is provided by the host VM which contains the devictree. +Gunyah reads the devicetree to configure the memory map and create resources +such as vCPUs for the VM. Memory can be shared with the VM with +`GH_VM_SET_USER_MEM_REGION`_. Userspace can interact with the resources in Linux +by adding "functions" to the VM. + +Sample Userspace VMM +==================== + +A sample userspace VMM is included in samples/gunyah/ along with a minimal +devicetree that can be used to launch a VM. To build this sample, enable +CONFIG_SAMPLE_GUNYAH. + +IOCTLs and userspace VMM flows +============================== + +The kernel exposes a char device interface at /dev/gunyah. + +To create a VM, use the `GH_CREATE_VM`_ ioctl. A successful call will return a +"Gunyah VM" file descriptor. + +/dev/gunyah API Descriptions +---------------------------- + +GH_CREATE_VM +~~~~~~~~~~~~ + +Creates a Gunyah VM. The argument is reserved for future use and must be 0. +A successful call will return a Gunyah VM file descriptor. See +`Gunyah VM API Descriptions`_ for list of IOCTLs that can be made on this file +file descriptor. + +Gunyah VM API Descriptions +-------------------------- + +GH_VM_SET_USER_MEM_REGION +~~~~~~~~~~~~~~~~~~~~~~~~~ + +This ioctl allows the user to create or delete a memory parcel for a guest +virtual machine. Each memory region is uniquely identified by a label; +attempting to create two regions with the same label is not allowed. Labels are +unique per virtual machine. + +While VMM is guest-agnostic and allows runtime addition of memory regions, +Linux guest virtual machines do not support accepting memory regions at runtime. +Thus, for Linux guests, memory regions should be provided before starting the VM +and the VM must be configured via the devicetree to accept these at boot-up. + +The guest physical address is used by Linux kernel to check that the requested +user regions do not overlap and to help find the corresponding memory region +for calls like `GH_VM_SET_DTB_CONFIG`_. It must be page aligned. + +To add a memory region, call `GH_VM_SET_USER_MEM_REGION`_ with fields set as +described above. + +.. kernel-doc:: include/uapi/linux/gunyah.h + :identifiers: gh_userspace_memory_region gh_mem_flags + +GH_VM_SET_DTB_CONFIG +~~~~~~~~~~~~~~~~~~~~ + +This ioctl sets the location of the VM's devicetree blob and is used by Gunyah +Resource Manager to allocate resources. The guest physical memory must be part +of the primary memory parcel provided to the VM prior to GH_VM_START. + +.. kernel-doc:: include/uapi/linux/gunyah.h + :identifiers: gh_vm_dtb_config + +GH_VM_START +~~~~~~~~~~~ + +This ioctl starts the VM. From patchwork Tue Jun 13 17:20:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 692119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9882DEB64DA for ; Tue, 13 Jun 2023 17:24:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239546AbjFMRYC (ORCPT ); Tue, 13 Jun 2023 13:24:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49954 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239536AbjFMRXJ (ORCPT ); Tue, 13 Jun 2023 13:23:09 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E77A213B; Tue, 13 Jun 2023 10:22:15 -0700 (PDT) Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35DAcSO0013828; Tue, 13 Jun 2023 17:21:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=2OkUID34BeYflc/zrfAetQILUsMucg1FpZUzkaqznGs=; b=FGBF7C9kzILd/1/Vwzp6VbHxRiUuQVNLJ37mfy2jbl9t6V6P+il9Hz10vj0zYpDI9aMh B5J7kxF9GPJWQ0rQLG3RFwekEZfPZI1SHAFFNSsVlyfiDLyGSholWqnEwKhAIfXdgcWs xlAUqbagMI6nVMBK/gt4Uj8akpWw5nNiOWgpbSwk7IyLrOgw4pe7cnEdlXjLlekSsl9B UZs4Ld8z3F2X+ZvEPkNNJwURn36vHjfoJPBkM6Xt/Yftg3RgdDY9dqrX3uvcXFAYw7G+ SI/ILoJD6h6FFPKAuFDyFksSPDLL08Tk2cmu1sKaxd8iHJT0XdQmMum9gPCI86AmlzpQ 2w== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r6km41d54-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:49 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 35DHLncn004530 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:49 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 13 Jun 2023 10:21:48 -0700 From: Elliot Berman To: Alex Elder , Srinivas Kandagatla , Elliot Berman , Prakruthi Deepak Heragu CC: Murali Nalajala , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Dmitry Baryshkov , Bjorn Andersson , "Konrad Dybcio" , Arnd Bergmann , "Greg Kroah-Hartman" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Jonathan Corbet , Bagas Sanjaya , Will Deacon , Andy Gross , Catalin Marinas , Jassi Brar , , , , , Subject: [PATCH v14 19/25] virt: gunyah: Add resource tickets Date: Tue, 13 Jun 2023 10:20:47 -0700 Message-ID: <20230613172054.3959700-20-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230613172054.3959700-1-quic_eberman@quicinc.com> References: <20230613172054.3959700-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: JL4NrveqiZ26zxkNvTOAO2zHPx0gTD42 X-Proofpoint-GUID: JL4NrveqiZ26zxkNvTOAO2zHPx0gTD42 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-13_18,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 lowpriorityscore=0 malwarescore=0 priorityscore=1501 spamscore=0 impostorscore=0 phishscore=0 mlxscore=0 suspectscore=0 bulkscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130153 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Some VM functions need to acquire Gunyah resources. For instance, Gunyah vCPUs are exposed to the host as a resource. The Gunyah vCPU function will register a resource ticket and be able to interact with the hypervisor once the resource ticket is filled. Resource tickets are the mechanism for functions to acquire ownership of Gunyah resources. Gunyah functions can be created before the VM's resources are created and made available to Linux. A resource ticket identifies a type of resource and a label of a resource which the ticket holder is interested in. Resources are created by Gunyah as configured in the VM's devicetree configuration. Gunyah doesn't process the label and that makes it possible for userspace to create multiple resources with the same label. Resource ticket owners need to be prepared for populate to be called multiple times if userspace created multiple resources with the same label. Reviewed-by: Alex Elder Signed-off-by: Elliot Berman --- drivers/virt/gunyah/vm_mgr.c | 117 +++++++++++++++++++++++++++++++++- drivers/virt/gunyah/vm_mgr.h | 4 ++ include/linux/gunyah_vm_mgr.h | 31 +++++++++ 3 files changed, 151 insertions(+), 1 deletion(-) diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index cfd79bc9900fd..d6aa1731148ae 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -183,6 +183,99 @@ void gh_vm_function_unregister(struct gh_vm_function *fn) } EXPORT_SYMBOL_GPL(gh_vm_function_unregister); +int gh_vm_add_resource_ticket(struct gh_vm *ghvm, struct gh_vm_resource_ticket *ticket) +{ + struct gh_vm_resource_ticket *iter; + struct gh_resource *ghrsc, *rsc_iter; + int ret = 0; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry(iter, &ghvm->resource_tickets, vm_list) { + if (iter->resource_type == ticket->resource_type && iter->label == ticket->label) { + ret = -EEXIST; + goto out; + } + } + + if (!try_module_get(ticket->owner)) { + ret = -ENODEV; + goto out; + } + + list_add(&ticket->vm_list, &ghvm->resource_tickets); + INIT_LIST_HEAD(&ticket->resources); + + list_for_each_entry_safe(ghrsc, rsc_iter, &ghvm->resources, list) { + if (ghrsc->type == ticket->resource_type && ghrsc->rm_label == ticket->label) { + if (ticket->populate(ticket, ghrsc)) + list_move(&ghrsc->list, &ticket->resources); + } + } +out: + mutex_unlock(&ghvm->resources_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gh_vm_add_resource_ticket); + +void gh_vm_remove_resource_ticket(struct gh_vm *ghvm, struct gh_vm_resource_ticket *ticket) +{ + struct gh_resource *ghrsc, *iter; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry_safe(ghrsc, iter, &ticket->resources, list) { + ticket->unpopulate(ticket, ghrsc); + list_move(&ghrsc->list, &ghvm->resources); + } + + module_put(ticket->owner); + list_del(&ticket->vm_list); + mutex_unlock(&ghvm->resources_lock); +} +EXPORT_SYMBOL_GPL(gh_vm_remove_resource_ticket); + +static void gh_vm_add_resource(struct gh_vm *ghvm, struct gh_resource *ghrsc) +{ + struct gh_vm_resource_ticket *ticket; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry(ticket, &ghvm->resource_tickets, vm_list) { + if (ghrsc->type == ticket->resource_type && ghrsc->rm_label == ticket->label) { + if (ticket->populate(ticket, ghrsc)) + list_add(&ghrsc->list, &ticket->resources); + else + list_add(&ghrsc->list, &ghvm->resources); + /* unconditonal -- we prevent multiple identical + * resource tickets so there will not be some other + * ticket elsewhere in the list if populate() failed. + */ + goto found; + } + } + list_add(&ghrsc->list, &ghvm->resources); +found: + mutex_unlock(&ghvm->resources_lock); +} + +static void gh_vm_clean_resources(struct gh_vm *ghvm) +{ + struct gh_vm_resource_ticket *ticket, *titer; + struct gh_resource *ghrsc, *riter; + + mutex_lock(&ghvm->resources_lock); + if (!list_empty(&ghvm->resource_tickets)) { + dev_warn(ghvm->parent, "Dangling resource tickets:\n"); + list_for_each_entry_safe(ticket, titer, &ghvm->resource_tickets, vm_list) { + dev_warn(ghvm->parent, " %pS\n", ticket->populate); + gh_vm_remove_resource_ticket(ghvm, ticket); + } + } + + list_for_each_entry_safe(ghrsc, riter, &ghvm->resources, list) { + gh_rm_free_resource(ghrsc); + } + mutex_unlock(&ghvm->resources_lock); +} + static int gh_vm_rm_notification_status(struct gh_vm *ghvm, void *data) { struct gh_rm_vm_status_payload *payload = data; @@ -265,6 +358,9 @@ static __must_check struct gh_vm *gh_vm_alloc(struct gh_rm *rm) init_waitqueue_head(&ghvm->vm_status_wait); INIT_WORK(&ghvm->free_work, gh_vm_free); kref_init(&ghvm->kref); + mutex_init(&ghvm->resources_lock); + INIT_LIST_HEAD(&ghvm->resources); + INIT_LIST_HEAD(&ghvm->resource_tickets); INIT_LIST_HEAD(&ghvm->functions); ghvm->vm_status = GH_RM_VM_STATUS_NO_STATE; @@ -274,9 +370,11 @@ static __must_check struct gh_vm *gh_vm_alloc(struct gh_rm *rm) static int gh_vm_start(struct gh_vm *ghvm) { struct gh_vm_mem *mapping; + struct gh_rm_hyp_resources *resources; + struct gh_resource *ghrsc; u64 dtb_offset; u32 mem_handle; - int ret; + int ret, i, n; down_write(&ghvm->status_lock); if (ghvm->vm_status != GH_RM_VM_STATUS_NO_STATE) { @@ -336,6 +434,22 @@ static int gh_vm_start(struct gh_vm *ghvm) } ghvm->vm_status = GH_RM_VM_STATUS_READY; + ret = gh_rm_get_hyp_resources(ghvm->rm, ghvm->vmid, &resources); + if (ret) { + dev_warn(ghvm->parent, "Failed to get hypervisor resources for VM: %d\n", ret); + goto err; + } + + for (i = 0, n = le32_to_cpu(resources->n_entries); i < n; i++) { + ghrsc = gh_rm_alloc_resource(ghvm->rm, &resources->entries[i]); + if (!ghrsc) { + ret = -ENOMEM; + goto err; + } + + gh_vm_add_resource(ghvm, ghrsc); + } + ret = gh_rm_vm_start(ghvm->rm, ghvm->vmid); if (ret) { dev_warn(ghvm->parent, "Failed to start VM: %d\n", ret); @@ -457,6 +571,7 @@ static void gh_vm_free(struct work_struct *work) gh_vm_stop(ghvm); gh_vm_remove_functions(ghvm); + gh_vm_clean_resources(ghvm); if (ghvm->vm_status != GH_RM_VM_STATUS_NO_STATE && ghvm->vm_status != GH_RM_VM_STATUS_LOAD && diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index c4bec1469ae8c..e5e0c92d4cb12 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -7,6 +7,7 @@ #define _GH_VM_MGR_H #include +#include #include #include #include @@ -52,6 +53,9 @@ struct gh_vm { struct list_head memory_mappings; struct mutex fn_lock; struct list_head functions; + struct mutex resources_lock; + struct list_head resources; + struct list_head resource_tickets; }; int gh_vm_mem_alloc(struct gh_vm *ghvm, struct gh_userspace_memory_region *region); diff --git a/include/linux/gunyah_vm_mgr.h b/include/linux/gunyah_vm_mgr.h index 6e4dd6fa5f96d..af97fec9e2ef5 100644 --- a/include/linux/gunyah_vm_mgr.h +++ b/include/linux/gunyah_vm_mgr.h @@ -94,4 +94,35 @@ void gh_vm_function_unregister(struct gh_vm_function *f); module_gh_vm_function(_name); \ MODULE_ALIAS_GH_VM_FUNCTION(_type, _idx) +/** + * struct gh_vm_resource_ticket - Represents a ticket to reserve exclusive access to VM resource(s) + * @vm_list: for @gh_vm->resource_tickets + * @resources: List of resource(s) associated with this ticket(members are from @gh_resource->list) + * @resource_type: Type of resource this ticket reserves + * @label: Label of the resource from resource manager this ticket reserves. + * @owner: owner of the ticket + * @populate: callback provided by the ticket owner and called when a resource is found that + * matches @resource_type and @label. Note that this callback could be called + * multiple times if userspace created mutliple resources with the same type/label. + * This callback may also have significant delay after gh_vm_add_resource_ticket() + * since gh_vm_add_resource_ticket() could be called before the VM starts. + * @unpopulate: callback provided by the ticket owner and called when the ticket owner should no + * no longer use the resource provided in the argument. When unpopulate() returns, + * the ticket owner should not be able to use the resource any more as the resource + * might being freed. + */ +struct gh_vm_resource_ticket { + struct list_head vm_list; + struct list_head resources; + enum gh_resource_type resource_type; + u32 label; + + struct module *owner; + bool (*populate)(struct gh_vm_resource_ticket *ticket, struct gh_resource *ghrsc); + void (*unpopulate)(struct gh_vm_resource_ticket *ticket, struct gh_resource *ghrsc); +}; + +int gh_vm_add_resource_ticket(struct gh_vm *ghvm, struct gh_vm_resource_ticket *ticket); +void gh_vm_remove_resource_ticket(struct gh_vm *ghvm, struct gh_vm_resource_ticket *ticket); + #endif From patchwork Tue Jun 13 17:20:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 692117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27845EB64DD for ; Tue, 13 Jun 2023 17:24:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240058AbjFMRYb (ORCPT ); Tue, 13 Jun 2023 13:24:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239553AbjFMRYC (ORCPT ); Tue, 13 Jun 2023 13:24:02 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1CC361BE9; Tue, 13 Jun 2023 10:22:34 -0700 (PDT) Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35DAwDK5017861; Tue, 13 Jun 2023 17:21:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=9kQi4bq7jBlBvrnldMXQVF7A85nY5zWmWChrhIWuX/A=; b=gQR+HG9cmNZpP/+e2WZbOs2E+YFakERdOiLTT/2+NgbBvmgaCjnpkJK9xf4C4CychIuc NEX8+XfFBeLzJeXFRJzUo6/YtoTST2XkAkrY+cwSb5r0L9+3+FqFh574uWDgkhDw+Wms V2giHHlZ4VX48odO6bQUFEzTqLGsv41cV+oqgqrkM1BS+Zo9CYGj3G+x9KlYx42Qe+ra w3k+6lwVbHH9ASi1y9oE4gW4bZM0+7hsI1HMbcFUdUt5K8jBRwlUd/tZEhjXoC1HxALZ YZdx9E7/aWEd2qx3q7Nz4mI0wjcrVwUzl4wYnH8/1fsfAnl9zB4lxlBnlbO5Psjpx7zT sA== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r6q4r0xf6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:53 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 35DHLqvU004597 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:52 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 13 Jun 2023 10:21:51 -0700 From: Elliot Berman To: Alex Elder , Srinivas Kandagatla , Elliot Berman , Prakruthi Deepak Heragu , Jonathan Corbet , Catalin Marinas , Will Deacon CC: Murali Nalajala , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Dmitry Baryshkov , Bjorn Andersson , "Konrad Dybcio" , Arnd Bergmann , "Greg Kroah-Hartman" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Bagas Sanjaya , Andy Gross , Jassi Brar , , , , , Subject: [PATCH v14 21/25] virt: gunyah: Add proxy-scheduled vCPUs Date: Tue, 13 Jun 2023 10:20:49 -0700 Message-ID: <20230613172054.3959700-22-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230613172054.3959700-1-quic_eberman@quicinc.com> References: <20230613172054.3959700-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 6KLf6egQYJlngVaGDgMYzODy76Ob-kWJ X-Proofpoint-ORIG-GUID: 6KLf6egQYJlngVaGDgMYzODy76Ob-kWJ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-13_19,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 clxscore=1015 priorityscore=1501 adultscore=0 bulkscore=0 suspectscore=0 impostorscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130153 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Gunyah allows host virtual machines to schedule guest virtual machines and handle their MMIO accesses. vCPUs are presented to the host as a Gunyah resource and represented to userspace as a Gunyah VM function. Creating the vcpu VM function will create a file descriptor that: - can run an ioctl: GH_VCPU_RUN to schedule the guest vCPU until the next interrupt occurs on the host or when the guest vCPU can no longer be run. - can be mmap'd to share a gh_vcpu_run structure which can look up the reason why GH_VCPU_RUN returned and provide return values for MMIO access. Acked-by: Alex Elder Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/vm-manager.rst | 46 ++- arch/arm64/gunyah/gunyah_hypercall.c | 28 ++ drivers/virt/gunyah/Kconfig | 11 + drivers/virt/gunyah/Makefile | 2 + drivers/virt/gunyah/gunyah_vcpu.c | 470 +++++++++++++++++++++++ drivers/virt/gunyah/vm_mgr.c | 4 + drivers/virt/gunyah/vm_mgr.h | 1 + include/linux/gunyah.h | 24 ++ include/uapi/linux/gunyah.h | 128 ++++++ 9 files changed, 712 insertions(+), 2 deletions(-) create mode 100644 drivers/virt/gunyah/gunyah_vcpu.c diff --git a/Documentation/virt/gunyah/vm-manager.rst b/Documentation/virt/gunyah/vm-manager.rst index cbc79bd8749b4..b0c3bde105ae9 100644 --- a/Documentation/virt/gunyah/vm-manager.rst +++ b/Documentation/virt/gunyah/vm-manager.rst @@ -5,8 +5,7 @@ Virtual Machine Manager ======================= The Gunyah Virtual Machine Manager is a Linux driver to support launching -virtual machines using Gunyah. It presently supports launching virtual machines -scheduled by Gunyah's scheduler. +virtual machines using Gunyah. Configuration of a Gunyah virtual machine is done via a devicetree. When the VM is launched, memory is provided by the host VM which contains the devictree. @@ -99,3 +98,46 @@ GH_VM_START ~~~~~~~~~~~ This ioctl starts the VM. + +GH_VM_ADD_FUNCTION +~~~~~~~~~~~~~~~~~~ + +This ioctl registers a Gunyah VM function with the VM manager. The VM function +is described with a &struct gh_fn_desc.type and some arguments for that type. +Typically, the function is added before the VM starts, but the function doesn't +"operate" until the VM starts with `GH_VM_START`_. For example, vCPU ioctls will +all return an error until the VM starts because the vCPUs don't exist until the +VM is started. This allows the VMM to set up all the kernel functions needed for +the VM *before* the VM starts. + +.. kernel-doc:: include/uapi/linux/gunyah.h + :identifiers: gh_fn_desc gh_fn_type + +The argument types are documented below: + +.. kernel-doc:: include/uapi/linux/gunyah.h + :identifiers: gh_fn_vcpu_arg + +Gunyah VCPU API Descriptions +---------------------------- + +A vCPU file descriptor is created after calling `GH_VM_ADD_FUNCTION` with the type `GH_FN_VCPU`. + +GH_VCPU_RUN +~~~~~~~~~~~ + +This ioctl is used to run a guest virtual cpu. While there are no +explicit parameters, there is an implicit parameter block that can be +obtained by mmap()ing the vcpu fd at offset 0, with the size given by +`GH_VCPU_MMAP_SIZE`_. The parameter block is formatted as a 'struct +gh_vcpu_run' (see below). + +GH_VCPU_MMAP_SIZE +~~~~~~~~~~~~~~~~~ + +The `GH_VCPU_RUN`_ ioctl communicates with userspace via a shared +memory region. This ioctl returns the size of that region. See the +`GH_VCPU_RUN`_ documentation for details. + +.. kernel-doc:: include/uapi/linux/gunyah.h + :identifiers: gh_vcpu_exit gh_vcpu_run gh_vm_status gh_vm_exit_info diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c index c5bf0dd468ec9..7d96491f84e00 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -37,6 +37,7 @@ EXPORT_SYMBOL_GPL(arch_is_gh_guest); #define GH_HYPERCALL_HYP_IDENTIFY GH_HYPERCALL(0x8000) #define GH_HYPERCALL_MSGQ_SEND GH_HYPERCALL(0x801B) #define GH_HYPERCALL_MSGQ_RECV GH_HYPERCALL(0x801C) +#define GH_HYPERCALL_VCPU_RUN GH_HYPERCALL(0x8065) /** * gh_hypercall_hyp_identify() - Returns build information and feature flags @@ -85,5 +86,32 @@ enum gh_error gh_hypercall_msgq_recv(u64 capid, void *buff, size_t size, size_t } EXPORT_SYMBOL_GPL(gh_hypercall_msgq_recv); +enum gh_error gh_hypercall_vcpu_run(u64 capid, u64 *resume_data, + struct gh_hypercall_vcpu_run_resp *resp) +{ + struct arm_smccc_1_2_regs args = { + .a0 = GH_HYPERCALL_VCPU_RUN, + .a1 = capid, + .a2 = resume_data[0], + .a3 = resume_data[1], + .a4 = resume_data[2], + /* C language says this will be implictly zero. Gunyah requires 0, so be explicit */ + .a5 = 0, + }; + struct arm_smccc_1_2_regs res; + + arm_smccc_1_2_hvc(&args, &res); + + if (res.a0 == GH_ERROR_OK) { + resp->sized_state = res.a1; + resp->state_data[0] = res.a2; + resp->state_data[1] = res.a3; + resp->state_data[2] = res.a4; + } + + return res.a0; +} +EXPORT_SYMBOL_GPL(gh_hypercall_vcpu_run); + MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 0421b751aad4f..0a58395f7d2c5 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -28,3 +28,14 @@ config GUNYAH_QCOM_PLATFORM extra platform-specific support. Say Y/M here to use Gunyah on Qualcomm platforms. + +config GUNYAH_VCPU + tristate "Runnable Gunyah vCPUs" + depends on GUNYAH + help + Enable kernel support for host-scheduled vCPUs running under Gunyah. + When selecting this option, userspace virtual machine managers (VMM) + can schedule the guest VM's vCPUs instead of using Gunyah's scheduler. + VMMs can also handle stage 2 faults of the vCPUs. + + Say Y/M here if unsure and you want to support Gunyah VMMs. diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 2aa9ff038ed02..cc16b6c19db92 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -5,3 +5,5 @@ obj-$(CONFIG_GUNYAH_QCOM_PLATFORM) += gunyah_qcom.o gunyah-y += rsc_mgr.o rsc_mgr_rpc.o vm_mgr.o vm_mgr_mm.o obj-$(CONFIG_GUNYAH) += gunyah.o + +obj-$(CONFIG_GUNYAH_VCPU) += gunyah_vcpu.o diff --git a/drivers/virt/gunyah/gunyah_vcpu.c b/drivers/virt/gunyah/gunyah_vcpu.c new file mode 100644 index 0000000000000..82a0cbf55caf6 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_vcpu.c @@ -0,0 +1,470 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "vm_mgr.h" + +#include + +#define MAX_VCPU_NAME 20 /* gh-vcpu:u32_max+NUL */ + +struct gh_vcpu { + struct gh_vm_function_instance *f; + struct gh_resource *rsc; + struct mutex run_lock; + /* Track why vcpu_run left last time around. */ + enum { + GH_VCPU_UNKNOWN = 0, + GH_VCPU_READY, + GH_VCPU_MMIO_READ, + GH_VCPU_SYSTEM_DOWN, + } state; + u8 mmio_read_len; + struct gh_vcpu_run *vcpu_run; + struct completion ready; + struct gh_vm *ghvm; + + struct notifier_block nb; + struct gh_vm_resource_ticket ticket; + struct kref kref; +}; + +static void vcpu_release(struct kref *kref) +{ + struct gh_vcpu *vcpu = container_of(kref, struct gh_vcpu, kref); + + free_page((unsigned long)vcpu->vcpu_run); + kfree(vcpu); +} + +/* + * When hypervisor allows us to schedule vCPU again, it gives us an interrupt + */ +static irqreturn_t gh_vcpu_irq_handler(int irq, void *data) +{ + struct gh_vcpu *vcpu = data; + + complete(&vcpu->ready); + return IRQ_HANDLED; +} + +static bool gh_handle_mmio(struct gh_vcpu *vcpu, + struct gh_hypercall_vcpu_run_resp *vcpu_run_resp) +{ + int ret = 0; + u64 addr = vcpu_run_resp->state_data[0], + len = vcpu_run_resp->state_data[1], + data = vcpu_run_resp->state_data[2]; + + if (WARN_ON(len > sizeof(u64))) + len = sizeof(u64); + + if (vcpu_run_resp->state == GH_VCPU_ADDRSPACE_VMMIO_READ) { + vcpu->vcpu_run->mmio.is_write = 0; + /* Record that we need to give vCPU user's supplied value next gh_vcpu_run() */ + vcpu->state = GH_VCPU_MMIO_READ; + vcpu->mmio_read_len = len; + } else { /* GH_VCPU_ADDRSPACE_VMMIO_WRITE */ + /* Try internal handlers first */ + ret = gh_vm_mmio_write(vcpu->f->ghvm, addr, len, data); + if (!ret) + return true; + + /* Give userspace the info */ + vcpu->vcpu_run->mmio.is_write = 1; + memcpy(vcpu->vcpu_run->mmio.data, &data, len); + } + + vcpu->vcpu_run->mmio.phys_addr = addr; + vcpu->vcpu_run->mmio.len = len; + vcpu->vcpu_run->exit_reason = GH_VCPU_EXIT_MMIO; + + return false; +} + +static int gh_vcpu_rm_notification(struct notifier_block *nb, unsigned long action, void *data) +{ + struct gh_vcpu *vcpu = container_of(nb, struct gh_vcpu, nb); + struct gh_rm_vm_exited_payload *exit_payload = data; + + if (action == GH_RM_NOTIFICATION_VM_EXITED && + le16_to_cpu(exit_payload->vmid) == vcpu->ghvm->vmid) + complete(&vcpu->ready); + + return NOTIFY_OK; +} + +static inline enum gh_vm_status remap_vm_status(enum gh_rm_vm_status rm_status) +{ + switch (rm_status) { + case GH_RM_VM_STATUS_INIT_FAILED: + return GH_VM_STATUS_LOAD_FAILED; + case GH_RM_VM_STATUS_EXITED: + return GH_VM_STATUS_EXITED; + default: + return GH_VM_STATUS_CRASHED; + } +} + +/** + * gh_vcpu_check_system() - Check whether VM as a whole is running + * @vcpu: Pointer to gh_vcpu + * + * Returns true if the VM is alive. + * Returns false if the vCPU is the VM is not alive (can only be that VM is shutting down). + */ +static bool gh_vcpu_check_system(struct gh_vcpu *vcpu) + __must_hold(&vcpu->run_lock) +{ + bool ret = true; + + down_read(&vcpu->ghvm->status_lock); + if (likely(vcpu->ghvm->vm_status == GH_RM_VM_STATUS_RUNNING)) + goto out; + + vcpu->vcpu_run->status.status = remap_vm_status(vcpu->ghvm->vm_status); + vcpu->vcpu_run->status.exit_info = vcpu->ghvm->exit_info; + vcpu->vcpu_run->exit_reason = GH_VCPU_EXIT_STATUS; + vcpu->state = GH_VCPU_SYSTEM_DOWN; + ret = false; +out: + up_read(&vcpu->ghvm->status_lock); + return ret; +} + +/** + * gh_vcpu_run() - Request Gunyah to begin scheduling this vCPU. + * @vcpu: The client descriptor that was obtained via gh_vcpu_alloc() + */ +static int gh_vcpu_run(struct gh_vcpu *vcpu) +{ + struct gh_hypercall_vcpu_run_resp vcpu_run_resp; + u64 state_data[3] = { 0 }; + enum gh_error gh_error; + int ret = 0; + + if (!vcpu->f) + return -ENODEV; + + if (mutex_lock_interruptible(&vcpu->run_lock)) + return -ERESTARTSYS; + + if (!vcpu->rsc) { + ret = -ENODEV; + goto out; + } + + switch (vcpu->state) { + case GH_VCPU_UNKNOWN: + if (vcpu->ghvm->vm_status != GH_RM_VM_STATUS_RUNNING) { + /* Check if VM is up. If VM is starting, will block until VM is fully up + * since that thread does down_write. + */ + if (!gh_vcpu_check_system(vcpu)) + goto out; + } + vcpu->state = GH_VCPU_READY; + break; + case GH_VCPU_MMIO_READ: + if (unlikely(vcpu->mmio_read_len > sizeof(state_data[0]))) + vcpu->mmio_read_len = sizeof(state_data[0]); + memcpy(&state_data[0], vcpu->vcpu_run->mmio.data, vcpu->mmio_read_len); + vcpu->state = GH_VCPU_READY; + break; + case GH_VCPU_SYSTEM_DOWN: + goto out; + default: + break; + } + + while (!ret && !signal_pending(current)) { + if (vcpu->vcpu_run->immediate_exit) { + ret = -EINTR; + goto out; + } + + gh_error = gh_hypercall_vcpu_run(vcpu->rsc->capid, state_data, &vcpu_run_resp); + if (gh_error == GH_ERROR_OK) { + switch (vcpu_run_resp.state) { + case GH_VCPU_STATE_READY: + if (need_resched()) + schedule(); + break; + case GH_VCPU_STATE_POWERED_OFF: + /* vcpu might be off because the VM is shut down. + * If so, it won't ever run again: exit back to user + */ + if (!gh_vcpu_check_system(vcpu)) + goto out; + /* Otherwise, another vcpu will turn it on (e.g. by PSCI) + * and hyp sends an interrupt to wake Linux up. + */ + fallthrough; + case GH_VCPU_STATE_EXPECTS_WAKEUP: + ret = wait_for_completion_interruptible(&vcpu->ready); + /* reinitialize completion before next hypercall. If we reinitialize + * after the hypercall, interrupt may have already come before + * re-initializing the completion and then end up waiting for + * event that already happened. + */ + reinit_completion(&vcpu->ready); + /* Check system status again. Completion might've + * come from gh_vcpu_rm_notification + */ + if (!ret && !gh_vcpu_check_system(vcpu)) + goto out; + break; + case GH_VCPU_STATE_BLOCKED: + schedule(); + break; + case GH_VCPU_ADDRSPACE_VMMIO_READ: + case GH_VCPU_ADDRSPACE_VMMIO_WRITE: + if (!gh_handle_mmio(vcpu, &vcpu_run_resp)) + goto out; + break; + default: + pr_warn_ratelimited("Unknown vCPU state: %llx\n", + vcpu_run_resp.sized_state); + schedule(); + break; + } + } else if (gh_error == GH_ERROR_RETRY) { + schedule(); + } else { + ret = gh_error_remap(gh_error); + } + } + +out: + mutex_unlock(&vcpu->run_lock); + + if (signal_pending(current)) + return -ERESTARTSYS; + + return ret; +} + +static long gh_vcpu_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) +{ + struct gh_vcpu *vcpu = filp->private_data; + long ret = -EINVAL; + + switch (cmd) { + case GH_VCPU_RUN: + ret = gh_vcpu_run(vcpu); + break; + case GH_VCPU_MMAP_SIZE: + ret = PAGE_SIZE; + break; + default: + break; + } + return ret; +} + +static int gh_vcpu_release(struct inode *inode, struct file *filp) +{ + struct gh_vcpu *vcpu = filp->private_data; + + gh_vm_put(vcpu->ghvm); + kref_put(&vcpu->kref, vcpu_release); + return 0; +} + +static vm_fault_t gh_vcpu_fault(struct vm_fault *vmf) +{ + struct gh_vcpu *vcpu = vmf->vma->vm_file->private_data; + struct page *page = NULL; + + if (vmf->pgoff == 0) + page = virt_to_page(vcpu->vcpu_run); + + get_page(page); + vmf->page = page; + return 0; +} + +static const struct vm_operations_struct gh_vcpu_ops = { + .fault = gh_vcpu_fault, +}; + +static int gh_vcpu_mmap(struct file *file, struct vm_area_struct *vma) +{ + vma->vm_ops = &gh_vcpu_ops; + return 0; +} + +static const struct file_operations gh_vcpu_fops = { + .owner = THIS_MODULE, + .unlocked_ioctl = gh_vcpu_ioctl, + .release = gh_vcpu_release, + .llseek = noop_llseek, + .mmap = gh_vcpu_mmap, +}; + +static bool gh_vcpu_populate(struct gh_vm_resource_ticket *ticket, struct gh_resource *ghrsc) +{ + struct gh_vcpu *vcpu = container_of(ticket, struct gh_vcpu, ticket); + int ret; + + mutex_lock(&vcpu->run_lock); + if (vcpu->rsc) { + pr_warn("vcpu%d already got a Gunyah resource. Check if multiple resources with same label were configured.\n", + vcpu->ticket.label); + ret = -EEXIST; + goto out; + } + + vcpu->rsc = ghrsc; + init_completion(&vcpu->ready); + + ret = request_irq(vcpu->rsc->irq, gh_vcpu_irq_handler, IRQF_TRIGGER_RISING, "gh_vcpu", + vcpu); + if (ret) + pr_warn("Failed to request vcpu irq %d: %d", vcpu->rsc->irq, ret); + + enable_irq_wake(vcpu->rsc->irq); + +out: + mutex_unlock(&vcpu->run_lock); + return !ret; +} + +static void gh_vcpu_unpopulate(struct gh_vm_resource_ticket *ticket, + struct gh_resource *ghrsc) +{ + struct gh_vcpu *vcpu = container_of(ticket, struct gh_vcpu, ticket); + + vcpu->vcpu_run->immediate_exit = true; + complete_all(&vcpu->ready); + mutex_lock(&vcpu->run_lock); + free_irq(vcpu->rsc->irq, vcpu); + vcpu->rsc = NULL; + mutex_unlock(&vcpu->run_lock); +} + +static long gh_vcpu_bind(struct gh_vm_function_instance *f) +{ + struct gh_fn_vcpu_arg *arg = f->argp; + struct gh_vcpu *vcpu; + char name[MAX_VCPU_NAME]; + struct file *file; + struct page *page; + int fd; + long r; + + if (f->arg_size != sizeof(*arg)) + return -EINVAL; + + vcpu = kzalloc(sizeof(*vcpu), GFP_KERNEL); + if (!vcpu) + return -ENOMEM; + + vcpu->f = f; + f->data = vcpu; + mutex_init(&vcpu->run_lock); + kref_init(&vcpu->kref); + + page = alloc_page(GFP_KERNEL | __GFP_ZERO); + if (!page) { + r = -ENOMEM; + goto err_destroy_vcpu; + } + vcpu->vcpu_run = page_address(page); + + vcpu->ticket.resource_type = GH_RESOURCE_TYPE_VCPU; + vcpu->ticket.label = arg->id; + vcpu->ticket.owner = THIS_MODULE; + vcpu->ticket.populate = gh_vcpu_populate; + vcpu->ticket.unpopulate = gh_vcpu_unpopulate; + + r = gh_vm_add_resource_ticket(f->ghvm, &vcpu->ticket); + if (r) + goto err_destroy_page; + + if (!gh_vm_get(f->ghvm)) { + r = -ENODEV; + goto err_remove_resource_ticket; + } + vcpu->ghvm = f->ghvm; + + vcpu->nb.notifier_call = gh_vcpu_rm_notification; + /* Ensure we run after the vm_mgr handles the notification and does + * any necessary state changes. We wake up to check the new state. + */ + vcpu->nb.priority = -1; + r = gh_rm_notifier_register(f->rm, &vcpu->nb); + if (r) + goto err_put_gh_vm; + + kref_get(&vcpu->kref); + + fd = get_unused_fd_flags(O_CLOEXEC); + if (fd < 0) { + r = fd; + goto err_notifier; + } + + snprintf(name, sizeof(name), "gh-vcpu:%u", vcpu->ticket.label); + file = anon_inode_getfile(name, &gh_vcpu_fops, vcpu, O_RDWR); + if (IS_ERR(file)) { + r = PTR_ERR(file); + goto err_put_fd; + } + + fd_install(fd, file); + + return fd; +err_put_fd: + put_unused_fd(fd); +err_notifier: + gh_rm_notifier_unregister(f->rm, &vcpu->nb); +err_put_gh_vm: + gh_vm_put(vcpu->ghvm); +err_remove_resource_ticket: + gh_vm_remove_resource_ticket(f->ghvm, &vcpu->ticket); +err_destroy_page: + free_page((unsigned long)vcpu->vcpu_run); +err_destroy_vcpu: + kfree(vcpu); + return r; +} + +static void gh_vcpu_unbind(struct gh_vm_function_instance *f) +{ + struct gh_vcpu *vcpu = f->data; + + gh_rm_notifier_unregister(f->rm, &vcpu->nb); + gh_vm_remove_resource_ticket(vcpu->f->ghvm, &vcpu->ticket); + vcpu->f = NULL; + + kref_put(&vcpu->kref, vcpu_release); +} + +static bool gh_vcpu_compare(const struct gh_vm_function_instance *f, + const void *arg, size_t size) +{ + const struct gh_fn_vcpu_arg *instance = f->argp, + *other = arg; + + if (sizeof(*other) != size) + return false; + + return instance->id == other->id; +} + +DECLARE_GH_VM_FUNCTION_INIT(vcpu, GH_FN_VCPU, 1, gh_vcpu_bind, gh_vcpu_unbind, gh_vcpu_compare); +MODULE_DESCRIPTION("Gunyah vCPU Function"); +MODULE_LICENSE("GPL"); diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index 28da1f95e49ec..afea4923847f1 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -405,6 +405,10 @@ static int gh_vm_rm_notification_exited(struct gh_vm *ghvm, void *data) down_write(&ghvm->status_lock); ghvm->vm_status = GH_RM_VM_STATUS_EXITED; + ghvm->exit_info.type = le16_to_cpu(payload->exit_type); + ghvm->exit_info.reason_size = le32_to_cpu(payload->exit_reason_size); + memcpy(&ghvm->exit_info.reason, payload->exit_reason, + min(GH_VM_MAX_EXIT_REASON_SIZE, ghvm->exit_info.reason_size)); up_write(&ghvm->status_lock); wake_up(&ghvm->vm_status_wait); diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 3fc0f91dfd1a9..fa2a61e10b57a 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -45,6 +45,7 @@ struct gh_vm { enum gh_rm_vm_status vm_status; wait_queue_head_t vm_status_wait; struct rw_semaphore status_lock; + struct gh_vm_exit_info exit_info; struct work_struct free_work; struct kref kref; diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 4b398b59c2c5a..cd5704a82c6a5 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -177,4 +177,28 @@ enum gh_error gh_hypercall_msgq_send(u64 capid, size_t size, void *buff, u64 tx_ enum gh_error gh_hypercall_msgq_recv(u64 capid, void *buff, size_t size, size_t *recv_size, bool *ready); +struct gh_hypercall_vcpu_run_resp { + union { + enum { + /* VCPU is ready to run */ + GH_VCPU_STATE_READY = 0, + /* VCPU is sleeping until an interrupt arrives */ + GH_VCPU_STATE_EXPECTS_WAKEUP = 1, + /* VCPU is powered off */ + GH_VCPU_STATE_POWERED_OFF = 2, + /* VCPU is blocked in EL2 for unspecified reason */ + GH_VCPU_STATE_BLOCKED = 3, + /* VCPU has returned for MMIO READ */ + GH_VCPU_ADDRSPACE_VMMIO_READ = 4, + /* VCPU has returned for MMIO WRITE */ + GH_VCPU_ADDRSPACE_VMMIO_WRITE = 5, + } state; + u64 sized_state; + }; + u64 state_data[3]; +}; + +enum gh_error gh_hypercall_vcpu_run(u64 capid, u64 *resume_data, + struct gh_hypercall_vcpu_run_resp *resp); + #endif diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index bb07118a351fd..434ffa8ffc783 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -72,8 +72,33 @@ struct gh_vm_dtb_config { #define GH_VM_START _IO(GH_IOCTL_TYPE, 0x3) +/** + * enum gh_fn_type - Valid types of Gunyah VM functions + * @GH_FN_VCPU: create a vCPU instance to control a vCPU + * &struct gh_fn_desc.arg is a pointer to &struct gh_fn_vcpu_arg + * Return: file descriptor to manipulate the vcpu. + */ +enum gh_fn_type { + GH_FN_VCPU = 1, +}; + #define GH_FN_MAX_ARG_SIZE 256 +/** + * struct gh_fn_vcpu_arg - Arguments to create a vCPU. + * @id: vcpu id + * + * Create this function with &GH_VM_ADD_FUNCTION using type &GH_FN_VCPU. + * + * The vcpu type will register with the VM Manager to expect to control + * vCPU number `vcpu_id`. It returns a file descriptor allowing interaction with + * the vCPU. See the Gunyah vCPU API description sections for interacting with + * the Gunyah vCPU file descriptors. + */ +struct gh_fn_vcpu_arg { + __u32 id; +}; + /** * struct gh_fn_desc - Arguments to create a VM function * @type: Type of the function. See &enum gh_fn_type. @@ -90,4 +115,107 @@ struct gh_fn_desc { #define GH_VM_ADD_FUNCTION _IOW(GH_IOCTL_TYPE, 0x4, struct gh_fn_desc) #define GH_VM_REMOVE_FUNCTION _IOW(GH_IOCTL_TYPE, 0x7, struct gh_fn_desc) +/* + * ioctls for vCPU fds + */ + +/** + * enum gh_vm_status - Stores status reason why VM is not runnable (exited). + * @GH_VM_STATUS_LOAD_FAILED: VM didn't start because it couldn't be loaded. + * @GH_VM_STATUS_EXITED: VM requested shutdown/reboot. + * Use &struct gh_vm_exit_info.reason for further details. + * @GH_VM_STATUS_CRASHED: VM state is unknown and has crashed. + */ +enum gh_vm_status { + GH_VM_STATUS_LOAD_FAILED = 1, + GH_VM_STATUS_EXITED = 2, + GH_VM_STATUS_CRASHED = 3, +}; + +/* + * Gunyah presently sends max 4 bytes of exit_reason. + * If that changes, this macro can be safely increased without breaking + * userspace so long as struct gh_vcpu_run < PAGE_SIZE. + */ +#define GH_VM_MAX_EXIT_REASON_SIZE 8u + +/** + * struct gh_vm_exit_info - Reason for VM exit as reported by Gunyah + * See Gunyah documentation for values. + * @type: Describes how VM exited + * @padding: padding bytes + * @reason_size: Number of bytes valid for `reason` + * @reason: See Gunyah documentation for interpretation. Note: these values are + * not interpreted by Linux and need to be converted from little-endian + * as applicable. + */ +struct gh_vm_exit_info { + __u16 type; + __u16 padding; + __u32 reason_size; + __u8 reason[GH_VM_MAX_EXIT_REASON_SIZE]; +}; + +/** + * enum gh_vcpu_exit - Stores reason why &GH_VCPU_RUN ioctl recently exited with status 0 + * @GH_VCPU_EXIT_UNKNOWN: Not used, status != 0 + * @GH_VCPU_EXIT_MMIO: vCPU performed a read or write that could not be handled + * by hypervisor or Linux. Use @struct gh_vcpu_run.mmio for + * details of the read/write. + * @GH_VCPU_EXIT_STATUS: vCPU not able to run because the VM has exited. + * Use @struct gh_vcpu_run.status for why VM has exited. + */ +enum gh_vcpu_exit { + GH_VCPU_EXIT_UNKNOWN, + GH_VCPU_EXIT_MMIO, + GH_VCPU_EXIT_STATUS, +}; + +/** + * struct gh_vcpu_run - Application code obtains a pointer to the gh_vcpu_run + * structure by mmap()ing a vcpu fd. + * @immediate_exit: polled when scheduling the vcpu. If set, immediately returns -EINTR. + * @padding: padding bytes + * @exit_reason: Set when GH_VCPU_RUN returns successfully and gives reason why + * GH_VCPU_RUN has stopped running the vCPU. See &enum gh_vcpu_exit. + * @mmio: Used when exit_reason == GH_VCPU_EXIT_MMIO + * The guest has faulted on an memory-mapped I/O instruction that + * couldn't be satisfied by gunyah. + * @mmio.phys_addr: Address guest tried to access + * @mmio.data: the value that was written if `is_write == 1`. Filled by + * user for reads (`is_write == 0`). + * @mmio.len: Length of write. Only the first `len` bytes of `data` + * are considered by Gunyah. + * @mmio.is_write: 1 if VM tried to perform a write, 0 for a read + * @status: Used when exit_reason == GH_VCPU_EXIT_STATUS. + * The guest VM is no longer runnable. This struct informs why. + * @status.status: See &enum gh_vm_status for possible values + * @status.exit_info: Used when status == GH_VM_STATUS_EXITED + */ +struct gh_vcpu_run { + /* in */ + __u8 immediate_exit; + __u8 padding[7]; + + /* out */ + __u32 exit_reason; + + union { + struct { + __u64 phys_addr; + __u8 data[8]; + __u32 len; + __u8 is_write; + } mmio; + + struct { + enum gh_vm_status status; + struct gh_vm_exit_info exit_info; + } status; + }; +}; + +#define GH_VCPU_RUN _IO(GH_IOCTL_TYPE, 0x5) +#define GH_VCPU_MMAP_SIZE _IO(GH_IOCTL_TYPE, 0x6) + #endif From patchwork Tue Jun 13 17:20:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 692118 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5CD08EB64DA for ; Tue, 13 Jun 2023 17:24:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240008AbjFMRYQ (ORCPT ); Tue, 13 Jun 2023 13:24:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50936 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231277AbjFMRXr (ORCPT ); Tue, 13 Jun 2023 13:23:47 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3FAE42701; Tue, 13 Jun 2023 10:22:25 -0700 (PDT) Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35DDE9mR017030; Tue, 13 Jun 2023 17:21:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=NSCOZ7VSWpZwQV9qvmb1c2Vez+hFpRt/AqMP2FgNDQ0=; b=ZMtmTfRzys/i5SqABaIvg3OLXexumFcanti974C1zJ0zTPV0LsuvOobrVfJVNqY1ZRp0 4rKEkji0bzS+MQpknZFX9Of8HXbcH4465O2AA7TVq9MOMKHYt+DgOL3CciNWP4zJzl5o QYtL+5UpmEsf3l65rxWpUXt5GB5VTfKJ9uvSZg8/8RqvtrziSAX0hx1Bv1F8XlxA8mBY J5U8S6URLrIOkTR01HerLWPs74hNj4lF+0Hbj7nBfFbSFz2FOu4LZyH7vFjyGBWzf7cV 4nmUStBdFFjBETBMeBCu/FieqD6flTHDoZiHb5AAhgqN/kZrfgKI2qZQNvuQCFG2fQn7 7Q== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r68x9ae1f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:54 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 35DHLsam021363 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:54 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 13 Jun 2023 10:21:53 -0700 From: Elliot Berman To: Alex Elder , Srinivas Kandagatla , Elliot Berman , Prakruthi Deepak Heragu , Catalin Marinas , Will Deacon CC: Murali Nalajala , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Dmitry Baryshkov , Bjorn Andersson , "Konrad Dybcio" , Arnd Bergmann , "Greg Kroah-Hartman" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Jonathan Corbet , Bagas Sanjaya , Andy Gross , Jassi Brar , , , , , Subject: [PATCH v14 22/25] virt: gunyah: Add hypercalls for sending doorbell Date: Tue, 13 Jun 2023 10:20:50 -0700 Message-ID: <20230613172054.3959700-23-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230613172054.3959700-1-quic_eberman@quicinc.com> References: <20230613172054.3959700-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: V5VLIexiiqSQyzT4qX_q9Ps_8Zytzvv6 X-Proofpoint-ORIG-GUID: V5VLIexiiqSQyzT4qX_q9Ps_8Zytzvv6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-13_19,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 priorityscore=1501 lowpriorityscore=0 impostorscore=0 spamscore=0 suspectscore=0 mlxlogscore=842 mlxscore=0 adultscore=0 malwarescore=0 clxscore=1015 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130153 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Gunyah doorbells allow two virtual machines to signal each other using interrupts. Add the hypercalls needed to assert the interrupt. Reviewed-by: Alex Elder Signed-off-by: Elliot Berman --- arch/arm64/gunyah/gunyah_hypercall.c | 25 +++++++++++++++++++++++++ include/linux/gunyah.h | 3 +++ 2 files changed, 28 insertions(+) diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c index 7d96491f84e00..e178b5a0f6917 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -35,6 +35,8 @@ EXPORT_SYMBOL_GPL(arch_is_gh_guest); fn) #define GH_HYPERCALL_HYP_IDENTIFY GH_HYPERCALL(0x8000) +#define GH_HYPERCALL_BELL_SEND GH_HYPERCALL(0x8012) +#define GH_HYPERCALL_BELL_SET_MASK GH_HYPERCALL(0x8015) #define GH_HYPERCALL_MSGQ_SEND GH_HYPERCALL(0x801B) #define GH_HYPERCALL_MSGQ_RECV GH_HYPERCALL(0x801C) #define GH_HYPERCALL_VCPU_RUN GH_HYPERCALL(0x8065) @@ -57,6 +59,29 @@ void gh_hypercall_hyp_identify(struct gh_hypercall_hyp_identify_resp *hyp_identi } EXPORT_SYMBOL_GPL(gh_hypercall_hyp_identify); +enum gh_error gh_hypercall_bell_send(u64 capid, u64 new_flags, u64 *old_flags) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GH_HYPERCALL_BELL_SEND, capid, new_flags, 0, &res); + + if (res.a0 == GH_ERROR_OK && old_flags) + *old_flags = res.a1; + + return res.a0; +} +EXPORT_SYMBOL_GPL(gh_hypercall_bell_send); + +enum gh_error gh_hypercall_bell_set_mask(u64 capid, u64 enable_mask, u64 ack_mask) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GH_HYPERCALL_BELL_SET_MASK, capid, enable_mask, ack_mask, 0, &res); + + return res.a0; +} +EXPORT_SYMBOL_GPL(gh_hypercall_bell_set_mask); + enum gh_error gh_hypercall_msgq_send(u64 capid, size_t size, void *buff, u64 tx_flags, bool *ready) { struct arm_smccc_res res; diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index cd5704a82c6a5..1f1685518bf3c 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -171,6 +171,9 @@ static inline u16 gh_api_version(const struct gh_hypercall_hyp_identify_resp *gh void gh_hypercall_hyp_identify(struct gh_hypercall_hyp_identify_resp *hyp_identity); +enum gh_error gh_hypercall_bell_send(u64 capid, u64 new_flags, u64 *old_flags); +enum gh_error gh_hypercall_bell_set_mask(u64 capid, u64 enable_mask, u64 ack_mask); + #define GH_HYPERCALL_MSGQ_TX_FLAGS_PUSH BIT(0) enum gh_error gh_hypercall_msgq_send(u64 capid, size_t size, void *buff, u64 tx_flags, bool *ready); From patchwork Tue Jun 13 17:20:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 692116 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56069EB64D7 for ; Tue, 13 Jun 2023 17:25:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241461AbjFMRZg (ORCPT ); Tue, 13 Jun 2023 13:25:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239942AbjFMRYN (ORCPT ); Tue, 13 Jun 2023 13:24:13 -0400 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30372273D; Tue, 13 Jun 2023 10:22:51 -0700 (PDT) Received: from pps.filterd (m0279868.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35DEHkMt009531; Tue, 13 Jun 2023 17:22:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=FlrxgEvt7L4TBZyxO2kgQkEc4ZQKY0Bf4bp//OqkEjg=; b=j4QE66esYU1XJU8eQSHHgaWURGZuSvC3eLWlyWWTH5NKKDdkuag+S+Q2LxKn23XEzEYo rLkNfCVWVKtbcBPdMR7WmJ514OvQHjcFNpn7WxXC36Tv2/qF/UUEw8dxtg6m6WqQc1B3 aBcFYCljFodsOqFMKNstAuWMve3/+uuC7GNReClZ1x2D605mg2qL8ETj3VNq2UigasG2 3yGvO2oFLPC4LkveZxSpuW8P98ynhtz5emKZAnF2qwX4jhYtl8+Cj3VNjdxDh4msS0em fFb1P7Chy46zjxeSHoxjAa48XaZdss2DYCj00Ro2AJgQCoRgSE86+auUm85yArbBgQ9j Rw== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r6s3wrnca-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:59 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 35DHLvfs029866 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:57 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 13 Jun 2023 10:21:56 -0700 From: Elliot Berman To: Alex Elder , Srinivas Kandagatla , Elliot Berman , Prakruthi Deepak Heragu , Jonathan Corbet CC: Murali Nalajala , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Dmitry Baryshkov , Bjorn Andersson , "Konrad Dybcio" , Arnd Bergmann , "Greg Kroah-Hartman" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Bagas Sanjaya , Will Deacon , Andy Gross , Catalin Marinas , Jassi Brar , , , , , Subject: [PATCH v14 24/25] virt: gunyah: Add ioeventfd Date: Tue, 13 Jun 2023 10:20:52 -0700 Message-ID: <20230613172054.3959700-25-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230613172054.3959700-1-quic_eberman@quicinc.com> References: <20230613172054.3959700-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: iBjD5XAYBg7uZ5J1W_W3nFDmwKmfPylL X-Proofpoint-GUID: iBjD5XAYBg7uZ5J1W_W3nFDmwKmfPylL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-13_19,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 adultscore=0 mlxscore=0 mlxlogscore=999 clxscore=1015 impostorscore=0 lowpriorityscore=0 suspectscore=0 priorityscore=1501 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130153 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Allow userspace to attach an ioeventfd to an mmio address within the guest. Reviewed-by: Alex Elder Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/vm-manager.rst | 2 +- drivers/virt/gunyah/Kconfig | 9 ++ drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_ioeventfd.c | 130 +++++++++++++++++++++++ include/uapi/linux/gunyah.h | 37 +++++++ 5 files changed, 178 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/gunyah_ioeventfd.c diff --git a/Documentation/virt/gunyah/vm-manager.rst b/Documentation/virt/gunyah/vm-manager.rst index 57a254827be26..415b11248e00b 100644 --- a/Documentation/virt/gunyah/vm-manager.rst +++ b/Documentation/virt/gunyah/vm-manager.rst @@ -116,7 +116,7 @@ the VM *before* the VM starts. The argument types are documented below: .. kernel-doc:: include/uapi/linux/gunyah.h - :identifiers: gh_fn_vcpu_arg gh_fn_irqfd_arg gh_irqfd_flags + :identifiers: gh_fn_vcpu_arg gh_fn_irqfd_arg gh_irqfd_flags gh_fn_ioeventfd_arg gh_ioeventfd_flags Gunyah VCPU API Descriptions ---------------------------- diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index bc2c46d9df946..63bebc5b9f825 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -48,3 +48,12 @@ config GUNYAH_IRQFD on Gunyah virtual machine. Say Y/M here if unsure and you want to support Gunyah VMMs. + +config GUNYAH_IOEVENTFD + tristate "Gunyah ioeventfd interface" + depends on GUNYAH + help + Enable kernel support for creating ioeventfds which can alert userspace + when a Gunyah virtual machine accesses a memory address. + + Say Y/M here if unsure and you want to support Gunyah VMMs. diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index ad212a1cf9671..63ca11e747961 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -8,3 +8,4 @@ obj-$(CONFIG_GUNYAH) += gunyah.o obj-$(CONFIG_GUNYAH_VCPU) += gunyah_vcpu.o obj-$(CONFIG_GUNYAH_IRQFD) += gunyah_irqfd.o +obj-$(CONFIG_GUNYAH_IOEVENTFD) += gunyah_ioeventfd.o diff --git a/drivers/virt/gunyah/gunyah_ioeventfd.c b/drivers/virt/gunyah/gunyah_ioeventfd.c new file mode 100644 index 0000000000000..5b1b9fd9ac3aa --- /dev/null +++ b/drivers/virt/gunyah/gunyah_ioeventfd.c @@ -0,0 +1,130 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include + +#include + +struct gh_ioeventfd { + struct gh_vm_function_instance *f; + struct gh_vm_io_handler io_handler; + + struct eventfd_ctx *ctx; +}; + +static int gh_write_ioeventfd(struct gh_vm_io_handler *io_dev, u64 addr, u32 len, u64 data) +{ + struct gh_ioeventfd *iofd = container_of(io_dev, struct gh_ioeventfd, io_handler); + + eventfd_signal(iofd->ctx, 1); + return 0; +} + +static struct gh_vm_io_handler_ops io_ops = { + .write = gh_write_ioeventfd, +}; + +static long gh_ioeventfd_bind(struct gh_vm_function_instance *f) +{ + const struct gh_fn_ioeventfd_arg *args = f->argp; + struct gh_ioeventfd *iofd; + struct eventfd_ctx *ctx; + int ret; + + if (f->arg_size != sizeof(*args)) + return -EINVAL; + + /* All other flag bits are reserved for future use */ + if (args->flags & ~GH_IOEVENTFD_FLAGS_DATAMATCH) + return -EINVAL; + + /* must be natural-word sized, or 0 to ignore length */ + switch (args->len) { + case 0: + case 1: + case 2: + case 4: + case 8: + break; + default: + return -EINVAL; + } + + /* check for range overflow */ + if (overflows_type(args->addr + args->len, u64)) + return -EINVAL; + + /* ioeventfd with no length can't be combined with DATAMATCH */ + if (!args->len && (args->flags & GH_IOEVENTFD_FLAGS_DATAMATCH)) + return -EINVAL; + + ctx = eventfd_ctx_fdget(args->fd); + if (IS_ERR(ctx)) + return PTR_ERR(ctx); + + iofd = kzalloc(sizeof(*iofd), GFP_KERNEL); + if (!iofd) { + ret = -ENOMEM; + goto err_eventfd; + } + + f->data = iofd; + iofd->f = f; + + iofd->ctx = ctx; + + if (args->flags & GH_IOEVENTFD_FLAGS_DATAMATCH) { + iofd->io_handler.datamatch = true; + iofd->io_handler.len = args->len; + iofd->io_handler.data = args->datamatch; + } + iofd->io_handler.addr = args->addr; + iofd->io_handler.ops = &io_ops; + + ret = gh_vm_add_io_handler(f->ghvm, &iofd->io_handler); + if (ret) + goto err_io_dev_add; + + return 0; + +err_io_dev_add: + kfree(iofd); +err_eventfd: + eventfd_ctx_put(ctx); + return ret; +} + +static void gh_ioevent_unbind(struct gh_vm_function_instance *f) +{ + struct gh_ioeventfd *iofd = f->data; + + eventfd_ctx_put(iofd->ctx); + gh_vm_remove_io_handler(iofd->f->ghvm, &iofd->io_handler); + kfree(iofd); +} + +static bool gh_ioevent_compare(const struct gh_vm_function_instance *f, + const void *arg, size_t size) +{ + const struct gh_fn_ioeventfd_arg *instance = f->argp, + *other = arg; + + if (sizeof(*other) != size) + return false; + + return instance->addr == other->addr; +} + +DECLARE_GH_VM_FUNCTION_INIT(ioeventfd, GH_FN_IOEVENTFD, 3, + gh_ioeventfd_bind, gh_ioevent_unbind, + gh_ioevent_compare); +MODULE_DESCRIPTION("Gunyah ioeventfd VM Function"); +MODULE_LICENSE("GPL"); diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index 0c480c622686a..fa1cae7419d2b 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -79,10 +79,13 @@ struct gh_vm_dtb_config { * Return: file descriptor to manipulate the vcpu. * @GH_FN_IRQFD: register eventfd to assert a Gunyah doorbell * &struct gh_fn_desc.arg is a pointer to &struct gh_fn_irqfd_arg + * @GH_FN_IOEVENTFD: register ioeventfd to trigger when VM faults on parameter + * &struct gh_fn_desc.arg is a pointer to &struct gh_fn_ioeventfd_arg */ enum gh_fn_type { GH_FN_VCPU = 1, GH_FN_IRQFD, + GH_FN_IOEVENTFD, }; #define GH_FN_MAX_ARG_SIZE 256 @@ -134,6 +137,40 @@ struct gh_fn_irqfd_arg { __u32 padding; }; +/** + * enum gh_ioeventfd_flags - flags for use in gh_fn_ioeventfd_arg + * @GH_IOEVENTFD_FLAGS_DATAMATCH: the event will be signaled only if the + * written value to the registered address is + * equal to &struct gh_fn_ioeventfd_arg.datamatch + */ +enum gh_ioeventfd_flags { + GH_IOEVENTFD_FLAGS_DATAMATCH = 1UL << 0, +}; + +/** + * struct gh_fn_ioeventfd_arg - Arguments to create an ioeventfd function + * @datamatch: data used when GH_IOEVENTFD_DATAMATCH is set + * @addr: Address in guest memory + * @len: Length of access + * @fd: When ioeventfd is matched, this eventfd is written + * @flags: See &enum gh_ioeventfd_flags + * @padding: padding bytes + * + * Create this function with &GH_VM_ADD_FUNCTION using type &GH_FN_IOEVENTFD. + * + * Attaches an ioeventfd to a legal mmio address within the guest. A guest write + * in the registered address will signal the provided event instead of triggering + * an exit on the GH_VCPU_RUN ioctl. + */ +struct gh_fn_ioeventfd_arg { + __u64 datamatch; + __u64 addr; /* legal mmio address */ + __u32 len; /* 1, 2, 4, or 8 bytes; or 0 to ignore length */ + __s32 fd; + __u32 flags; + __u32 padding; +}; + /** * struct gh_fn_desc - Arguments to create a VM function * @type: Type of the function. See &enum gh_fn_type.