From patchwork Tue Aug 23 18:52:43 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Suthikulpanit, Suravee" X-Patchwork-Id: 74522 Delivered-To: patch@linaro.org Received: by 10.140.29.52 with SMTP id a49csp2260547qga; Tue, 23 Aug 2016 12:32:18 -0700 (PDT) X-Received: by 10.98.51.131 with SMTP id z125mr56203376pfz.109.1471980738386; Tue, 23 Aug 2016 12:32:18 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j29si5239083pfk.53.2016.08.23.12.32.17; Tue, 23 Aug 2016 12:32:18 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@amdcloud.onmicrosoft.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755195AbcHWTcO (ORCPT + 27 others); Tue, 23 Aug 2016 15:32:14 -0400 Received: from mail-dm3nam03on0078.outbound.protection.outlook.com ([104.47.41.78]:38059 "EHLO NAM03-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754206AbcHWTcM (ORCPT ); Tue, 23 Aug 2016 15:32:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=0BuV4zYrlraH7xntm3rmKgpgAm7QBlynB/5p82ZABKk=; b=zmfIcZndBbDWGPiLGDDZOm+GxXh97cnceGkrjqjY5O6z7xKiUww+meuPVH6YNjsio4YjLVL4XifLG1Vp8NOmuZMxiV1qDM2KE99a0vck+hC4L2HQp+VzbgOMr19yCsmkm+qdbAHO/e1th7JXsNjPaGVoYQVJ/RTx4gxG7QwwkWQ= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Suravee.Suthikulpanit@amd.com; Received: from localhost.localdomain (114.109.128.54) by DM5PR12MB1452.namprd12.prod.outlook.com (10.172.38.141) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.557.21; Tue, 23 Aug 2016 18:53:46 +0000 From: Suravee Suthikulpanit To: , , , CC: , , , Suravee Suthikulpanit Subject: [PART2 PATCH v7 12/12] svm: Implements update_pi_irte hook to setup posted interrupt Date: Tue, 23 Aug 2016 13:52:43 -0500 Message-ID: <1471978363-13756-13-git-send-email-Suravee.Suthikulpanit@amd.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1471978363-13756-1-git-send-email-Suravee.Suthikulpanit@amd.com> References: <1471978363-13756-1-git-send-email-Suravee.Suthikulpanit@amd.com> MIME-Version: 1.0 X-Originating-IP: [114.109.128.54] X-ClientProxiedBy: KL1PR02CA0053.apcprd02.prod.outlook.com (10.167.54.21) To DM5PR12MB1452.namprd12.prod.outlook.com (10.172.38.141) X-MS-Office365-Filtering-Correlation-Id: 31ce2821-8502-494c-d44a-08d3cb86d1bc X-Microsoft-Exchange-Diagnostics: 1; DM5PR12MB1452; 2:nzIdXRJ+quSYD7wHzkubeEaGXR+LQ6PrNedt+ik4p9Az5+0H+iJ3XG3RhMpll1podj46unXy7VNtPSf+njaGufl49uC1tqW2jmxRUiujy5fLvoQAWxQyHIuIuNkkVcceqzO9LC8YycwyCDM22eQd4vsrvTl+MUiDN3DuDHp94HYt1fUHQ3kzEOx4wIhSXdz7; 3:iJXhQz9IngcSNWB3M9i4k5loGSDhWOvNEZpk38VJqGD0465cSjpBAc0TxF0bIdHVManaYBHBNbTszwsU/k58iCI9DCootSRppmOlqIOaT1xRbE9g9M9KTk+IrtLRrjA2 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DM5PR12MB1452; X-Microsoft-Exchange-Diagnostics: 1; DM5PR12MB1452; 25:2sORNIpn8C/5fxzvPopZimMmV4N611iaOzmMkmaomUbtEQwYhB0u3r/CUAwmFUX3BoRSTVsw26UJo52/THNeXOPXBLv3oJufS8LLIsTwZdGCcfHlSLpKFaDZyj6Sowcx0pEOhHEI4+YwyBh71WtmY1Xl2omfiv3CxSfboJFb6chDg2/6SupnXIAynzcKLupNJqHhT/4SeJZS+JAM4I8a+23HwUHhS+QTBwzxiVzMWduf6mqOP/U77OF8FfM8obq+cpWyXESylL8PDvk7/W4vx5BcXBi8hVer4iGFLH59b8+ZEQzshj64wilm/ZWElEwkn34oN/z8wpNXZXeyM8z25qkHKIR+SgCSNVwax4eLzFH9Jut25Qp+x92e8WeVwLIVRqytD/LCJ0zTV9jBXjnCDy3EAsK729EGpmHUIt66DoAvfeHB2af1XKisxXarFrQQICZeIxjZhu19yhXPrGHFhLPtxDu1fFN8cQpbMtLuYtGztzaE5yHoVLmmFfsjbPBUrNf3ZwlShUM8qzj3PQE0TSgGuaWAzSEZk4fk9S8uLls6GQ1AgY/fM+ezfJyL2ZSmZ5GlfEVucTXe4jczpyYKldlGtHLo0Kyexm0bzmbJnT/v2ry9Vz892fv0LIvLZlaxeSwB0ClMjg43MFRUJiZS0k8a0G16zf86BXZhNPqVbwX/zu2RFzyWdM/7b0p89lpJ0i3MOSI7brzAvIjlDD7ccrXOFxHlVRngQPm4fNV8wRd+v3nw065QYnBLEjj3A+D1 X-Microsoft-Exchange-Diagnostics: 1; DM5PR12MB1452; 31:QhsPaGaITltokhl7ElXrNvE9e0nTFUUBXv2FTN/C3OqbOZjyUvruWpi9LMqdJ5McO1cDgjdFWmAdfroCeCqo6Jzu7AG1Dj3e75V3+zRd9VlLLGUXnBv/iEjRjpX7K9P0YsYi//9xpiXslI3vaQ4pyzy50BiLEX7PHspskhLUgYHWfuL6CbUYZdhR0r7e38lEDLRNbzc4he8Z3GGWarvSFQWpFm5Juop3gz0tZ8B0fDw=; 20:83C65tXFlPD3zL3jgzTcBGFf0dhUE1f2N7aBPoxhfnthi/OpF601b63HQgp0///sW3GVMDgwytembO8AENuphOydcGNOS9X0bObD0scN45SJxEFFT7pSB77Biajj4ps+wV58+9QHnirmenV+sjxXT1k8CGimMk45LoUDJrcTd4yoC8Be9ZVazb7c4Li9nlrcov2kVx6xipE9Lsgq/lSE8Y0lSAewUXfNptzjUqO2dux9Z6kcqW+DTZNWYS755+ufdwqLaylYe+mJL2lHluKZHHZa8dZtk86/a37fNG2ndYcpjkxqc+NLfgmnH/k2s9qlda+9CdKrddTYCgMxfjtsjDyPZ5DNjKTLODpBirom50+usdugo5J46zSELD0gRyhXZja0ClpKwCdNr+/W76WxFkqvMFgPkh31PUKrK6G+a1ZbWU56/j4wCFt09iwlzWpXVmIP61tO4JDudUb8mBC7HpvMGvdAArzY/fFD9EMvH79bR3IuSVObIVurHhPEOQRN X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6055026); SRVR:DM5PR12MB1452; BCL:0; PCL:0; RULEID:; SRVR:DM5PR12MB1452; X-Microsoft-Exchange-Diagnostics: 1; DM5PR12MB1452; 4:IpldwSLRsrMhrybVdFBTIVlgazM0XS8LJLFxK5Wf9hG4G7Or5WtnykX0Kug2Cyq9V9GmporI75RZ6vVQvH3ZusYvn7aaztjNiVpLUZW73cZqXyyiL6pF8zo0iAsGGv8zO6Za5DiRXntXEleSDJ/GTOTV6USX4Uwb6PwAazEVjqVm+gbGxqO+0qYsLmb6NqUcjvR9TM5BaUl3OUsUE+tnR2tOuO1Me0pCl4YBoDGJRsCJImlHrPii7p6oZpJPqokDOHYiMuB0jhdfttdVBxe2oM0NgOEiyggObHDyKBLZDQWjiopiAV5T2tuCiiai+R+xsqUNZ8ICTF0LZvLO/OFJrZmwg/m2bAskgsLJq4bpem49GyGiQwEP21g/8+22TClmGggxw2y1YFwcgj4poZPtxRYhhpHxRjdD66bfK5AUEN/7/jp1QobROVqY8T5ss3uj X-Forefront-PRVS: 004395A01C X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6069001)(6009001)(7916002)(199003)(189002)(42186005)(76176999)(101416001)(48376002)(50466002)(77096005)(5660300001)(575784001)(92566002)(105586002)(5003940100001)(36756003)(86362001)(106356001)(66066001)(50986999)(5001770100001)(19580395003)(19580405001)(50226002)(97736004)(189998001)(229853001)(3846002)(2950100001)(47776003)(8676002)(7736002)(6116002)(586003)(81156014)(68736007)(4326007)(2201001)(81166006)(15650500001)(305945005)(7846002)(2906002); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR12MB1452; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM5PR12MB1452; 23:jtmYayjBaH4ZG16dhQ5bv/+3k5FJKSNlR0JG5hJRA?= =?us-ascii?Q?tdjmCoo3SIEUMbADhFw32MnS8yDcCQWT+ew+rDO1fcWFosw4gwWcqfkjga5U?= =?us-ascii?Q?yDC0XRNK9bbi72b1MNtsslHiEUnFKUlJElUWBAxoVWiOKOr0vmxFyjS63SoW?= =?us-ascii?Q?ZZ/8kzfNISiqFQRQzPOEth1k69y1jcW5fxuVA6bBCY7BUxf9JkiTqDc7iKrV?= =?us-ascii?Q?ihEjXnisF/EgPYrC+S4jPy7PApF/07HrU+8Mv0RA9MXTpHmMDmrCEu/SwPi7?= =?us-ascii?Q?EDGYfnyLC9575ubNV6F65iQ1ftXcMccuQ9edK0D/+2ho0Xxb7xkgHrlM0kX7?= =?us-ascii?Q?XQhjtpJYgE0/Tp34C7X3kCR5sANyGBZHW54s5DxX4jnwsss/geDGZ4zqnAEE?= =?us-ascii?Q?gCuKhIJ2L1jWstQW6mwvgo2W3jkPKsHL3x0DPHroMsDemLZO3+rog3UcZ2X2?= =?us-ascii?Q?5GyjPSCP2xLnVUQiIjDhM9PGFwcb23JpfW/xQ3eunhjLj3hd1QHF+cuCtXeV?= =?us-ascii?Q?/piJagutyeQ4ecJm0gWXQS8gKftG27zeGOTSNMsgnd4yGIeDIKv0/v15dY5/?= =?us-ascii?Q?Bp4ay2PJIM9hxbbtgYdmi//e2ZsyYa2kGSg76rWjuo+YdPBXFs9j52AS4RsT?= =?us-ascii?Q?meeJB7Gly+hQ9ke58L9hotX8yAHlCiTGXD+gWws8zHMyjZlqtZfpQ+k0qVwy?= =?us-ascii?Q?tbg0cLcSmqVInLYS5ZuUxHNkX7RCkq92X1ZJCb0wCTg4Sq/WJJRCDHWe30Di?= =?us-ascii?Q?loJqai2Oy2JGOZ/bDId6f5678RCgKyjpI0GsvxYLwuCYSI+pd1yEMVvmKbFA?= =?us-ascii?Q?mdB0bbM76DFJEnrCe1AxaaazUF4X5uk4CG4v9GW3Qn8KG82lWC+qc6H5DYKY?= =?us-ascii?Q?/9CL/DVQmzGhR4NjbaW8G2Qonnw/0QpVNShP8Fb9QLEKpNydL4j6X+KPKkQr?= =?us-ascii?Q?s1FX0X3X8cMqcuovt3GWjsKbkEQLzNA3uV3HjemeXD2GI9F+Z3JE3b0+Y94K?= =?us-ascii?Q?zy5d9OIflRtutPxNv3VBFcHYQyIDmanAoM7VYcO93vz6jQAyRRs7BiejzqWI?= =?us-ascii?Q?cA4hYq1gLl0EV+vwZWoUSicEVg1+K/v2v7I8wndycFwBFbY5CHTOf5Es0NOb?= =?us-ascii?Q?vMCHrXtfi814Dn+7JXBG54iOSClD148CffJlg6itaAazwPeWvClFA=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; DM5PR12MB1452; 6:eVgzo4GAOeuu2Qiitxsv+tZbX2nbRifQLp1SoE0vUgt4GI91rL700g24VAOk7eIaFp8mz9MJNCvHYdVLZcMPVUv8y7ollnu/ELIrrhCPnFLDyKUbfSR7g68/FD5QWs6Qb1ESppa5Eqq+Rx+LBwoT4yTNbB48tJnKAEBLHkEnXjbydKH1/UZcEaJOtD8NlAdDNDfuP5E2v+tZeLtdkab1LOnE4b50pTD/8dTrBLrbkEYnr5ufQJC/J7NLkOpnGslA/hahx2f2HwynSEbSQEQWQpw/5b07grg3adfwW9mLX8KNVICTBAkwzMtDGZ26iiU5MzSK/aGgfKgsS3l/t4CBbQ==; 5:nTZHRBfhr1tD+xY3idYQJS57klfvbumssjbekz+WSm3fcYtmgDoVWzhWwJ3kh4I//2KJdNs9Zem01mstuQD0NVn7+NwguY1Pz3vwKWyu1Z5vm151zfvxJ/tBaFtQkicEE1pqIEbh3N4H9evRfWHWkw==; 24:PLxtUq7CRsFqUqOY84zKzs7vNXu/od5kVC3qBe9KaFAT20Q+kUVwjs7aT/zbw4Z8PCVGygYAPLr2U9MMg3VuMzF1g8YDD46mOz3bytJV8A4=; 7:Iu+WSGz3ciQB9SG8/80iH35dYq/brHNGut8rOnheJSeVsx3tcOLqMNxz9MarVGOZHIpH7FH+pXauzy3A5iGDLHQ/HfBUU5wZlckW8Z/I75JhENnAqOO9WR+0eQMfEy4QvM3NrKgUoa8uvOpT2REUBTUZKCip2CWdUQxmRTpgK5hfA9bkgbWOtTIxDc13iGUJgfzuhatojpGrhiW6L0sgR4MJLmDNEpmNNbmjTOJjCQ4YcT4jO/Amk7ON5oUurodp SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DM5PR12MB1452; 20:ZZzsb6UrwYn5md2KBXo0TxNDQUIZjwC57pYIjTx1EEVOJOO00xTs4poKQS/mthCNHTE41pDXkSpQx+TVUkC4IOmLMJmyou7uAfqHFvAMsI8HK0iMqmvKEs6QL9c4ITdF1x37fOauuvmw1tXw7S0f6Go9W4VpNS9ZadmKE9MciuWaTIayJSZnqQ0JUVuWfzX/5cffbMF38JwDVRMEDXxQxyUk7BykkjlB2VVQhipb+42Ql5e7ncy0XoiLETL2oK6/ X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Aug 2016 18:53:46.5247 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1452 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Suravee Suthikulpanit This patch implements update_pi_irte function hook to allow SVM communicate to IOMMU driver regarding how to set up IRTE for handling posted interrupt. In case AVIC is enabled, during vcpu_load/unload, SVM needs to update IOMMU IRTE with appropriate host physical APIC ID. Also, when vcpu_blocking/unblocking, SVM needs to update the is-running bit in the IOMMU IRTE. Both are achieved via calling amd_iommu_update_ga(). However, if GA mode is not enabled for the pass-through device, IOMMU driver will simply just return when calling amd_iommu_update_ga. Signed-off-by: Suravee Suthikulpanit --- arch/x86/kvm/svm.c | 285 +++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 266 insertions(+), 19 deletions(-) -- 1.9.1 diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index 8f87a0a..c921024 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -43,6 +43,7 @@ #include #include #include +#include #include #include "trace.h" @@ -200,6 +201,23 @@ struct vcpu_svm { struct page *avic_backing_page; u64 *avic_physical_id_cache; bool avic_is_running; + + /* + * Per-vcpu list of struct amd_svm_iommu_ir: + * This is used mainly to store interrupt remapping information used + * when update the vcpu affinity. This avoids the need to scan for + * IRTE and try to match ga_tag in the IOMMU driver. + */ + struct list_head ir_list; + spinlock_t ir_list_lock; +}; + +/* + * This is a wrapper of struct amd_iommu_ir_data. + */ +struct amd_svm_iommu_ir { + struct list_head node; /* Used by SVM for per-vcpu ir_list */ + void *data; /* Storing pointer to struct amd_ir_data */ }; #define AVIC_LOGICAL_ID_ENTRY_GUEST_PHYSICAL_ID_MASK (0xFF) @@ -1440,31 +1458,34 @@ free_avic: return err; } -/** - * This function is called during VCPU halt/unhalt. - */ -static void avic_set_running(struct kvm_vcpu *vcpu, bool is_run) +static inline int +avic_update_iommu_vcpu_affinity(struct kvm_vcpu *vcpu, int cpu, bool r) { - u64 entry; - int h_physical_id = kvm_cpu_get_apicid(vcpu->cpu); + int ret = 0; + unsigned long flags; + struct amd_svm_iommu_ir *ir; struct vcpu_svm *svm = to_svm(vcpu); - if (!kvm_vcpu_apicv_active(vcpu)) - return; - - svm->avic_is_running = is_run; + if (!kvm_arch_has_assigned_device(vcpu->kvm)) + return 0; - /* ID = 0xff (broadcast), ID > 0xff (reserved) */ - if (WARN_ON(h_physical_id >= AVIC_MAX_PHYSICAL_ID_COUNT)) - return; + /* + * Here, we go through the per-vcpu ir_list to update all existing + * interrupt remapping table entry targeting this vcpu. + */ + spin_lock_irqsave(&svm->ir_list_lock, flags); - entry = READ_ONCE(*(svm->avic_physical_id_cache)); - WARN_ON(is_run == !!(entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK)); + if (list_empty(&svm->ir_list)) + goto out; - entry &= ~AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK; - if (is_run) - entry |= AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK; - WRITE_ONCE(*(svm->avic_physical_id_cache), entry); + list_for_each_entry(ir, &svm->ir_list, node) { + ret = amd_iommu_update_ga(cpu, r, ir->data); + if (ret) + break; + } +out: + spin_unlock_irqrestore(&svm->ir_list_lock, flags); + return ret; } static void avic_vcpu_load(struct kvm_vcpu *vcpu, int cpu) @@ -1491,6 +1512,8 @@ static void avic_vcpu_load(struct kvm_vcpu *vcpu, int cpu) entry |= AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK; WRITE_ONCE(*(svm->avic_physical_id_cache), entry); + avic_update_iommu_vcpu_affinity(vcpu, h_physical_id, + svm->avic_is_running); } static void avic_vcpu_put(struct kvm_vcpu *vcpu) @@ -1502,10 +1525,27 @@ static void avic_vcpu_put(struct kvm_vcpu *vcpu) return; entry = READ_ONCE(*(svm->avic_physical_id_cache)); + if (entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK) + avic_update_iommu_vcpu_affinity(vcpu, -1, 0); + entry &= ~AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK; WRITE_ONCE(*(svm->avic_physical_id_cache), entry); } +/** + * This function is called during VCPU halt/unhalt. + */ +static void avic_set_running(struct kvm_vcpu *vcpu, bool is_run) +{ + struct vcpu_svm *svm = to_svm(vcpu); + + svm->avic_is_running = is_run; + if (is_run) + avic_vcpu_load(vcpu, vcpu->cpu); + else + avic_vcpu_put(vcpu); +} + static void svm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) { struct vcpu_svm *svm = to_svm(vcpu); @@ -1567,6 +1607,9 @@ static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id) err = avic_init_backing_page(&svm->vcpu); if (err) goto free_page4; + + INIT_LIST_HEAD(&svm->ir_list); + spin_lock_init(&svm->ir_list_lock); } /* We initialize this flag to true to make sure that the is_running @@ -4363,6 +4406,209 @@ static void svm_deliver_avic_intr(struct kvm_vcpu *vcpu, int vec) kvm_vcpu_wake_up(vcpu); } +static void svm_ir_list_del(struct vcpu_svm *svm, struct amd_iommu_pi_data *pi) +{ + unsigned long flags; + struct amd_svm_iommu_ir *cur; + + spin_lock_irqsave(&svm->ir_list_lock, flags); + list_for_each_entry(cur, &svm->ir_list, node) { + if (cur->data != pi->ir_data) + continue; + list_del(&cur->node); + kfree(cur); + break; + } + spin_unlock_irqrestore(&svm->ir_list_lock, flags); +} + +static int svm_ir_list_add(struct vcpu_svm *svm, struct amd_iommu_pi_data *pi) +{ + int ret = 0; + unsigned long flags; + struct amd_svm_iommu_ir *ir; + + /** + * In some cases, the existing irte is updaed and re-set, + * so we need to check here if it's already been * added + * to the ir_list. + */ + if (pi->ir_data && (pi->prev_ga_tag != 0)) { + struct kvm *kvm = svm->vcpu.kvm; + u32 vcpu_id = AVIC_GATAG_TO_VCPUID(pi->prev_ga_tag); + struct kvm_vcpu *prev_vcpu = kvm_get_vcpu_by_id(kvm, vcpu_id); + struct vcpu_svm *prev_svm; + + if (!prev_vcpu) { + ret = -EINVAL; + goto out; + } + + prev_svm = to_svm(prev_vcpu); + svm_ir_list_del(prev_svm, pi); + } + + /** + * Allocating new amd_iommu_pi_data, which will get + * add to the per-vcpu ir_list. + */ + ir = kzalloc(sizeof(struct amd_svm_iommu_ir), GFP_KERNEL); + if (!ir) { + ret = -ENOMEM; + goto out; + } + ir->data = pi->ir_data; + + spin_lock_irqsave(&svm->ir_list_lock, flags); + list_add(&ir->node, &svm->ir_list); + spin_unlock_irqrestore(&svm->ir_list_lock, flags); +out: + return ret; +} + +/** + * Note: + * The HW cannot support posting multicast/broadcast + * interrupts to a vCPU. So, we still use legacy interrupt + * remapping for these kind of interrupts. + * + * For lowest-priority interrupts, we only support + * those with single CPU as the destination, e.g. user + * configures the interrupts via /proc/irq or uses + * irqbalance to make the interrupts single-CPU. + */ +static int +get_pi_vcpu_info(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e, + struct vcpu_data *vcpu_info, struct vcpu_svm **svm) +{ + struct kvm_lapic_irq irq; + struct kvm_vcpu *vcpu = NULL; + + kvm_set_msi_irq(e, &irq); + + if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu)) { + pr_debug("SVM: %s: use legacy intr remap mode for irq %u\n", + __func__, irq.vector); + return -1; + } + + pr_debug("SVM: %s: use GA mode for irq %u\n", __func__, + irq.vector); + *svm = to_svm(vcpu); + vcpu_info->pi_desc_addr = page_to_phys((*svm)->avic_backing_page); + vcpu_info->vector = irq.vector; + + return 0; +} + +/* + * svm_update_pi_irte - set IRTE for Posted-Interrupts + * + * @kvm: kvm + * @host_irq: host irq of the interrupt + * @guest_irq: gsi of the interrupt + * @set: set or unset PI + * returns 0 on success, < 0 on failure + */ +static int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq, + uint32_t guest_irq, bool set) +{ + struct kvm_kernel_irq_routing_entry *e; + struct kvm_irq_routing_table *irq_rt; + int idx, ret = -EINVAL; + + if (!kvm_arch_has_assigned_device(kvm) || + !irq_remapping_cap(IRQ_POSTING_CAP)) + return 0; + + pr_debug("SVM: %s: host_irq=%#x, guest_irq=%#x, set=%#x\n", + __func__, host_irq, guest_irq, set); + + idx = srcu_read_lock(&kvm->irq_srcu); + irq_rt = srcu_dereference(kvm->irq_routing, &kvm->irq_srcu); + WARN_ON(guest_irq >= irq_rt->nr_rt_entries); + + hlist_for_each_entry(e, &irq_rt->map[guest_irq], link) { + struct vcpu_data vcpu_info; + struct vcpu_svm *svm = NULL; + + if (e->type != KVM_IRQ_ROUTING_MSI) + continue; + + /** + * Here, we setup with legacy mode in the following cases: + * 1. When cannot target interrupt to a specific vcpu. + * 2. Unsetting posted interrupt. + * 3. APIC virtialization is disabled for the vcpu. + */ + if (!get_pi_vcpu_info(kvm, e, &vcpu_info, &svm) && set && + kvm_vcpu_apicv_active(&svm->vcpu)) { + struct amd_iommu_pi_data pi; + + /* Try to enable guest_mode in IRTE */ + pi.base = page_to_phys(svm->avic_backing_page) & AVIC_HPA_MASK; + pi.ga_tag = AVIC_GATAG(kvm->arch.avic_vm_id, + svm->vcpu.vcpu_id); + pi.is_guest_mode = true; + pi.vcpu_data = &vcpu_info; + ret = irq_set_vcpu_affinity(host_irq, &pi); + + /** + * Here, we successfully setting up vcpu affinity in + * IOMMU guest mode. Now, we need to store the posted + * interrupt information in a per-vcpu ir_list so that + * we can reference to them directly when we update vcpu + * scheduling information in IOMMU irte. + */ + if (!ret && pi.is_guest_mode) + svm_ir_list_add(svm, &pi); + } else { + /* Use legacy mode in IRTE */ + struct amd_iommu_pi_data pi; + + /** + * Here, pi is used to: + * - Tell IOMMU to use legacy mode for this interrupt. + * - Retrieve ga_tag of prior interrupt remapping data. + */ + pi.is_guest_mode = false; + ret = irq_set_vcpu_affinity(host_irq, &pi); + + /** + * Check if the posted interrupt was previously + * setup with the guest_mode by checking if the ga_tag + * was cached. If so, we need to clean up the per-vcpu + * ir_list. + */ + if (!ret && pi.prev_ga_tag) { + int id = AVIC_GATAG_TO_VCPUID(pi.prev_ga_tag); + struct kvm_vcpu *vcpu; + + vcpu = kvm_get_vcpu_by_id(kvm, id); + if (vcpu) + svm_ir_list_del(to_svm(vcpu), &pi); + } + } + + if (!ret && svm) { + trace_kvm_pi_irte_update(svm->vcpu.vcpu_id, + host_irq, e->gsi, + vcpu_info.vector, + vcpu_info.pi_desc_addr, set); + } + + if (ret < 0) { + pr_err("%s: failed to update PI IRTE\n", __func__); + goto out; + } + } + + ret = 0; +out: + srcu_read_unlock(&kvm->irq_srcu, idx); + return ret; +} + static int svm_nmi_allowed(struct kvm_vcpu *vcpu) { struct vcpu_svm *svm = to_svm(vcpu); @@ -5189,6 +5435,7 @@ static struct kvm_x86_ops svm_x86_ops = { .pmu_ops = &amd_pmu_ops, .deliver_posted_interrupt = svm_deliver_avic_intr, + .update_pi_irte = svm_update_pi_irte, }; static int __init svm_init(void)