From patchwork Tue Aug 18 14:33:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 247897 Delivered-To: patch@linaro.org Received: by 2002:a54:3b12:0:0:0:0:0 with SMTP id j18csp3219910ect; Tue, 18 Aug 2020 08:28:46 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyNd2rnywsgGzcSJvhqY9kf6G5+plLkfmdYANjMFUrAT9wpK82rMupAp6Z3DRNQ7K/6QXJx X-Received: by 2002:a25:cc12:: with SMTP id l18mr28387775ybf.224.1597764526235; Tue, 18 Aug 2020 08:28:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1597764526; cv=none; d=google.com; s=arc-20160816; b=jJ/5XzKaE5bFBXPUCYlynIZXawu6QXddKDVNx6crYa3f5Qqdm4EyyJd531reT+bLBg mSg0qQ3EgPPUcHraQhGdj+AheAwL7jnXzaI3HTja7uxMxuWi15JgLXk7KxGLdQ9w/s7P wbLf+VCIbzq4RqpPkmgdkmZTwvJEd6EIWHyO5GVss665cca5FUxliaQZ1oH+2FbjcyoX WWVbPGjvOAhWGIUH0z13de2r3+wDqSMBmt2V59+812EehDZbRJxcTHKx19sD9XOcDjkp sXsD0y5+XPqKVR39l0yOOLILGj3fpyQvDwNOQDc09/kRlyRAcGGcpURSFsMq+CIIroxY Dw1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=kaTbMGanT71s89DsJoOZGDCpWg1RR6kbH+cjUVEgFgU=; b=Bc9g1Jwj+YuDpDasdX9NSkFr5/pu5IabYTatAlZ7ypgHzPQxgoJvwi632ck13T98DA fUZfaW0noxtusbrJnTzRFPw9nQ/1n+cRHl1dKFL/h3Bs/nBgO5H+c7QxGRUDUi6SrP8M n7HovNrcCqIFIDG6rtLUJhAVkPYzPe9y1AbGUaeu0SN9NDzVpMYnM+ySD3+WlB76v1de KqYVK1AZRz3rVmQ+mcaG4qJBlaEXym8FhBSQchmIO1JF27XI5YV5gMQTNEDPG/jSa7YK wzsd84F46X4rOdu2el1V/uoLfO503K88xnHT4lxc9MITYmmCFHSJRnnKFBRD2G8rYofT 5S8Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@redhat.com header.s=mimecast20190719 header.b=bUCeU7Yk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x5si21262817ybg.315.2020.08.18.08.28.46 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 18 Aug 2020 08:28:46 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@redhat.com header.s=mimecast20190719 header.b=bUCeU7Yk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from localhost ([::1]:56278 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k83XZ-0003uM-NJ for patch@linaro.org; Tue, 18 Aug 2020 11:28:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39332) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k82gz-0004rF-RY for qemu-devel@nongnu.org; Tue, 18 Aug 2020 10:34:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:24110) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k82gv-0002Sc-TY for qemu-devel@nongnu.org; Tue, 18 Aug 2020 10:34:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1597761261; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kaTbMGanT71s89DsJoOZGDCpWg1RR6kbH+cjUVEgFgU=; b=bUCeU7YkZa9wDlAREaE7ZbGKkuilNzs0cdU6KodMvRZpOveHD8c4hRXww3ZtxD8MjEqgIx uxzF06mqCYM02XKl5RUIvsZ/5MSeWFt/3zFrZLZFjhX4AV5qfFFi0k6zgR7h4+Zm/oZGaX XYkVr1hsoTIek94oL9XE4Mbln/n7GMk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-241-TZ6N2GSJOE2UYmnKRps_6g-1; Tue, 18 Aug 2020 10:34:16 -0400 X-MC-Unique: TZ6N2GSJOE2UYmnKRps_6g-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CA11A8B8D22; Tue, 18 Aug 2020 14:34:04 +0000 (UTC) Received: from localhost (ovpn-114-160.ams2.redhat.com [10.36.114.160]) by smtp.corp.redhat.com (Postfix) with ESMTP id E07C16B8E0; Tue, 18 Aug 2020 14:34:03 +0000 (UTC) From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v6 1/7] Open 5.2 development tree Date: Tue, 18 Aug 2020 15:33:42 +0100 Message-Id: <20200818143348.310613-2-stefanha@redhat.com> In-Reply-To: <20200818143348.310613-1-stefanha@redhat.com> References: <20200818143348.310613-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0.0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/18 03:22:11 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Maydell , "Michael S. Tsirkin" , Max Reitz , qemu-block@nongnu.org, David Hildenbrand , Halil Pasic , Christian Borntraeger , Richard Henderson , Thomas Huth , Eduardo Habkost , qemu-s390x@nongnu.org, qemu-arm@nongnu.org, Stefan Hajnoczi , David Gibson , Kevin Wolf , Pankaj Gupta , cohuck@redhat.com, Raphael Norwitz , qemu-ppc@nongnu.org, Paolo Bonzini Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Peter Maydell Signed-off-by: Peter Maydell --- VERSION | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VERSION b/VERSION index 831446cbd2..7d40cb9d36 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -5.1.0 +5.1.50 From patchwork Tue Aug 18 14:33:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 276125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MIME_BASE64_TEXT,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E539C433E1 for ; Tue, 18 Aug 2020 15:45:11 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EB3F72076E for ; Tue, 18 Aug 2020 15:45:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="RgTT7fZn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EB3F72076E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36128 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k83nS-00068i-6h for qemu-devel@archiver.kernel.org; Tue, 18 Aug 2020 11:45:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39494) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k82h4-0004xn-KY for qemu-devel@nongnu.org; Tue, 18 Aug 2020 10:34:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:23125) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k82h1-0002UL-Eq for qemu-devel@nongnu.org; Tue, 18 Aug 2020 10:34:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1597761266; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ENnQwlZiy2s0di8Jd6xc923V8L9TxtXZ4z/RSHMFSr8=; b=RgTT7fZnqe3Ifq8Tfxo2L3bbuSmHBDIeTIQe97wrJBx+ctUTLljLZox3Ck0JPrKND7mtX6 zfW7pugnWqwFzJwkCuBJPGDFothgHdmtJgwOaI56I0kZn0Xzb4Qxh1Zz6xEMcELp8IAAuI zrAZpHIVAZm02eE4XkxzNyQeqaWZFuQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-511-KZGcSDqNOqiXxGbvOPpf-Q-1; Tue, 18 Aug 2020 10:34:23 -0400 X-MC-Unique: KZGcSDqNOqiXxGbvOPpf-Q-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B8453AD32C; Tue, 18 Aug 2020 14:34:08 +0000 (UTC) Received: from localhost (ovpn-114-160.ams2.redhat.com [10.36.114.160]) by smtp.corp.redhat.com (Postfix) with ESMTP id 34E712CFC6; Tue, 18 Aug 2020 14:34:08 +0000 (UTC) From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v6 3/7] virtio-pci: add virtio_pci_optimal_num_queues() helper Date: Tue, 18 Aug 2020 15:33:44 +0100 Message-Id: <20200818143348.310613-4-stefanha@redhat.com> In-Reply-To: <20200818143348.310613-1-stefanha@redhat.com> References: <20200818143348.310613-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/18 03:22:11 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Maydell , "Michael S. Tsirkin" , Max Reitz , qemu-block@nongnu.org, David Hildenbrand , Halil Pasic , Christian Borntraeger , Richard Henderson , Thomas Huth , Eduardo Habkost , qemu-s390x@nongnu.org, qemu-arm@nongnu.org, Stefan Hajnoczi , David Gibson , Kevin Wolf , Pankaj Gupta , cohuck@redhat.com, Raphael Norwitz , qemu-ppc@nongnu.org, Paolo Bonzini Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Multi-queue devices achieve the best performance when each vCPU has a dedicated queue. This ensures that virtqueue used notifications are handled on the same vCPU that submitted virtqueue buffers. When another vCPU handles the the notification an IPI will be necessary to wake the submission vCPU and this incurs a performance overhead. Provide a helper function that virtio-pci devices will use in later patches to automatically select the optimal number of queues. The function handles guests with large numbers of CPUs by limiting the number of queues to fit within the following constraints: 1. The maximum number of MSI-X vectors. 2. The maximum number of virtqueues. Signed-off-by: Stefan Hajnoczi Reviewed-by: Cornelia Huck --- hw/virtio/virtio-pci.h | 9 +++++++++ hw/virtio/virtio-pci.c | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/hw/virtio/virtio-pci.h b/hw/virtio/virtio-pci.h index e2eaaa9182..91096f0291 100644 --- a/hw/virtio/virtio-pci.h +++ b/hw/virtio/virtio-pci.h @@ -243,4 +243,13 @@ typedef struct VirtioPCIDeviceTypeInfo { /* Register virtio-pci type(s). @t must be static. */ void virtio_pci_types_register(const VirtioPCIDeviceTypeInfo *t); +/** + * virtio_pci_optimal_num_queues: + * @fixed_queues: number of queues that are always present + * + * Returns: The optimal number of queues for a multi-queue device, excluding + * @fixed_queues. + */ +unsigned virtio_pci_optimal_num_queues(unsigned fixed_queues); + #endif diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index ccdf54e81c..fc69570dcc 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -19,6 +19,7 @@ #include "exec/memop.h" #include "standard-headers/linux/virtio_pci.h" +#include "hw/boards.h" #include "hw/virtio/virtio.h" #include "migration/qemu-file-types.h" #include "hw/pci/pci.h" @@ -2058,6 +2059,37 @@ void virtio_pci_types_register(const VirtioPCIDeviceTypeInfo *t) g_free(base_name); } +unsigned virtio_pci_optimal_num_queues(unsigned fixed_queues) +{ + /* + * 1:1 vq to vCPU mapping is ideal because the same vCPU that submitted + * virtqueue buffers can handle their completion. When a different vCPU + * handles completion it may need to IPI the vCPU that submitted the + * request and this adds overhead. + * + * Virtqueues consume guest RAM and MSI-X vectors. This is wasteful in + * guests with very many vCPUs and a device that is only used by a few + * vCPUs. Unfortunately optimizing that case requires manual pinning inside + * the guest, so those users might as well manually set the number of + * queues. There is no upper limit that can be applied automatically and + * doing so arbitrarily would result in a sudden performance drop once the + * threshold number of vCPUs is exceeded. + */ + unsigned num_queues = current_machine->smp.cpus; + + /* + * The maximum number of MSI-X vectors is PCI_MSIX_FLAGS_QSIZE + 1, but the + * config change interrupt and the fixed virtqueues must be taken into + * account too. + */ + num_queues = MIN(num_queues, PCI_MSIX_FLAGS_QSIZE - fixed_queues); + + /* + * There is a limit to how many virtqueues a device can have. + */ + return MIN(num_queues, VIRTIO_QUEUE_MAX - fixed_queues); +} + /* virtio-pci-bus */ static void virtio_pci_bus_new(VirtioBusState *bus, size_t bus_size, From patchwork Tue Aug 18 14:33:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 276129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MIME_BASE64_TEXT,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 661AEC433E1 for ; Tue, 18 Aug 2020 15:33:03 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1F5EC20786 for ; Tue, 18 Aug 2020 15:33:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="C75EeahU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1F5EC20786 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:45850 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k83bi-0002yC-D2 for qemu-devel@archiver.kernel.org; Tue, 18 Aug 2020 11:33:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39582) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k82h8-000592-Cy for qemu-devel@nongnu.org; Tue, 18 Aug 2020 10:34:34 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:38472 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k82h4-0002VV-GG for qemu-devel@nongnu.org; Tue, 18 Aug 2020 10:34:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1597761269; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Wb1XamP5TGEehgB5L5m/KNklppAMpaQAEaCLfGMImwk=; b=C75EeahUQhTEnL+tHHEyA5g2dThoaeDeb6dKzyEkCGLab5rwHMTY2wvIE7R7lE2xX7isYr +zgddJxyrjNPsJB40rv3F1+PLLwprIZtVjVnzYFCTMjU46nGkMVMVwUcW2+iOCn7n1sPUH C3CrzCBCjaGDl2TByiKi+RUebyomFik= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-457-KdqfaL5LNdiWN4ne7qESyQ-1; Tue, 18 Aug 2020 10:34:24 -0400 X-MC-Unique: KdqfaL5LNdiWN4ne7qESyQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BA4658B5E64; Tue, 18 Aug 2020 14:34:12 +0000 (UTC) Received: from localhost (ovpn-114-160.ams2.redhat.com [10.36.114.160]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4CC915D9DC; Tue, 18 Aug 2020 14:34:12 +0000 (UTC) From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v6 5/7] virtio-scsi-pci: default num_queues to -smp N Date: Tue, 18 Aug 2020 15:33:46 +0100 Message-Id: <20200818143348.310613-6-stefanha@redhat.com> In-Reply-To: <20200818143348.310613-1-stefanha@redhat.com> References: <20200818143348.310613-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0.002 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.81; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/18 08:01:06 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Maydell , "Michael S. Tsirkin" , Max Reitz , qemu-block@nongnu.org, David Hildenbrand , Halil Pasic , Christian Borntraeger , Richard Henderson , Thomas Huth , Eduardo Habkost , qemu-s390x@nongnu.org, qemu-arm@nongnu.org, Stefan Hajnoczi , David Gibson , Kevin Wolf , Pankaj Gupta , cohuck@redhat.com, Raphael Norwitz , qemu-ppc@nongnu.org, Paolo Bonzini Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Automatically size the number of virtio-scsi-pci, vhost-scsi-pci, and vhost-user-scsi-pci request virtqueues to match the number of vCPUs. Other transports continue to default to 1 request virtqueue. A 1:1 virtqueue:vCPU mapping ensures that completion interrupts are handled on the same vCPU that submitted the request. No IPI is necessary to complete an I/O request and performance is improved. The maximum number of MSI-X vectors and virtqueues limit are respected. Signed-off-by: Stefan Hajnoczi --- include/hw/virtio/virtio-scsi.h | 2 ++ hw/core/machine.c | 3 +++ hw/scsi/vhost-scsi.c | 3 ++- hw/scsi/vhost-user-scsi.c | 3 ++- hw/scsi/virtio-scsi.c | 6 +++++- hw/virtio/vhost-scsi-pci.c | 10 +++++++--- hw/virtio/vhost-user-scsi-pci.c | 10 +++++++--- hw/virtio/virtio-scsi-pci.c | 10 +++++++--- 8 files changed, 35 insertions(+), 12 deletions(-) diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h index 9f293bcb80..c0b8e4dd7e 100644 --- a/include/hw/virtio/virtio-scsi.h +++ b/include/hw/virtio/virtio-scsi.h @@ -39,6 +39,8 @@ /* Number of virtqueues that are always present */ #define VIRTIO_SCSI_VQ_NUM_FIXED 2 +#define VIRTIO_SCSI_AUTO_NUM_QUEUES UINT32_MAX + typedef struct virtio_scsi_cmd_req VirtIOSCSICmdReq; typedef struct virtio_scsi_cmd_resp VirtIOSCSICmdResp; typedef struct virtio_scsi_ctrl_tmf_req VirtIOSCSICtrlTMFReq; diff --git a/hw/core/machine.c b/hw/core/machine.c index a6f7e4e8d7..9ee2aa0f7b 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -29,6 +29,9 @@ #include "migration/vmstate.h" GlobalProperty hw_compat_5_1[] = { + { "vhost-scsi", "num_queues", "1"}, + { "vhost-user-scsi", "num_queues", "1"}, + { "virtio-scsi-device", "num_queues", "1"}, }; const size_t hw_compat_5_1_len = G_N_ELEMENTS(hw_compat_5_1); diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c index 13b05af29b..a83ffeefc8 100644 --- a/hw/scsi/vhost-scsi.c +++ b/hw/scsi/vhost-scsi.c @@ -270,7 +270,8 @@ static Property vhost_scsi_properties[] = { DEFINE_PROP_STRING("vhostfd", VirtIOSCSICommon, conf.vhostfd), DEFINE_PROP_STRING("wwpn", VirtIOSCSICommon, conf.wwpn), DEFINE_PROP_UINT32("boot_tpgt", VirtIOSCSICommon, conf.boot_tpgt, 0), - DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues, 1), + DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues, + VIRTIO_SCSI_AUTO_NUM_QUEUES), DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSICommon, conf.virtqueue_size, 128), DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSICommon, conf.seg_max_adjust, diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index a8b821466f..7c0631656c 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -162,7 +162,8 @@ static void vhost_user_scsi_unrealize(DeviceState *dev) static Property vhost_user_scsi_properties[] = { DEFINE_PROP_CHR("chardev", VirtIOSCSICommon, conf.chardev), DEFINE_PROP_UINT32("boot_tpgt", VirtIOSCSICommon, conf.boot_tpgt, 0), - DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues, 1), + DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues, + VIRTIO_SCSI_AUTO_NUM_QUEUES), DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSICommon, conf.virtqueue_size, 128), DEFINE_PROP_UINT32("max_sectors", VirtIOSCSICommon, conf.max_sectors, diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c index eecdd05af5..3a71ea7097 100644 --- a/hw/scsi/virtio-scsi.c +++ b/hw/scsi/virtio-scsi.c @@ -891,6 +891,9 @@ void virtio_scsi_common_realize(DeviceState *dev, virtio_init(vdev, "virtio-scsi", VIRTIO_ID_SCSI, sizeof(VirtIOSCSIConfig)); + if (s->conf.num_queues == VIRTIO_SCSI_AUTO_NUM_QUEUES) { + s->conf.num_queues = 1; + } if (s->conf.num_queues == 0 || s->conf.num_queues > VIRTIO_QUEUE_MAX - VIRTIO_SCSI_VQ_NUM_FIXED) { error_setg(errp, "Invalid number of queues (= %" PRIu32 "), " @@ -964,7 +967,8 @@ static void virtio_scsi_device_unrealize(DeviceState *dev) } static Property virtio_scsi_properties[] = { - DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queues, 1), + DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queues, + VIRTIO_SCSI_AUTO_NUM_QUEUES), DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSI, parent_obj.conf.virtqueue_size, 256), DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSI, diff --git a/hw/virtio/vhost-scsi-pci.c b/hw/virtio/vhost-scsi-pci.c index 06e814d30e..a6bb0dc60d 100644 --- a/hw/virtio/vhost-scsi-pci.c +++ b/hw/virtio/vhost-scsi-pci.c @@ -47,11 +47,15 @@ static void vhost_scsi_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) { VHostSCSIPCI *dev = VHOST_SCSI_PCI(vpci_dev); DeviceState *vdev = DEVICE(&dev->vdev); - VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(vdev); + VirtIOSCSIConf *conf = &dev->vdev.parent_obj.parent_obj.conf; + + if (conf->num_queues == VIRTIO_SCSI_AUTO_NUM_QUEUES) { + conf->num_queues = + virtio_pci_optimal_num_queues(VIRTIO_SCSI_VQ_NUM_FIXED); + } if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) { - vpci_dev->nvectors = vs->conf.num_queues + - VIRTIO_SCSI_VQ_NUM_FIXED + 1; + vpci_dev->nvectors = conf->num_queues + VIRTIO_SCSI_VQ_NUM_FIXED + 1; } qdev_realize(vdev, BUS(&vpci_dev->bus), errp); diff --git a/hw/virtio/vhost-user-scsi-pci.c b/hw/virtio/vhost-user-scsi-pci.c index ab6dfb71a9..25e97ca54e 100644 --- a/hw/virtio/vhost-user-scsi-pci.c +++ b/hw/virtio/vhost-user-scsi-pci.c @@ -53,11 +53,15 @@ static void vhost_user_scsi_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) { VHostUserSCSIPCI *dev = VHOST_USER_SCSI_PCI(vpci_dev); DeviceState *vdev = DEVICE(&dev->vdev); - VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(vdev); + VirtIOSCSIConf *conf = &dev->vdev.parent_obj.parent_obj.conf; + + if (conf->num_queues == VIRTIO_SCSI_AUTO_NUM_QUEUES) { + conf->num_queues = + virtio_pci_optimal_num_queues(VIRTIO_SCSI_VQ_NUM_FIXED); + } if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) { - vpci_dev->nvectors = vs->conf.num_queues + - VIRTIO_SCSI_VQ_NUM_FIXED + 1; + vpci_dev->nvectors = conf->num_queues + VIRTIO_SCSI_VQ_NUM_FIXED + 1; } qdev_realize(vdev, BUS(&vpci_dev->bus), errp); diff --git a/hw/virtio/virtio-scsi-pci.c b/hw/virtio/virtio-scsi-pci.c index 3ff9eb7ef6..fa4b3bfb50 100644 --- a/hw/virtio/virtio-scsi-pci.c +++ b/hw/virtio/virtio-scsi-pci.c @@ -46,13 +46,17 @@ static void virtio_scsi_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) { VirtIOSCSIPCI *dev = VIRTIO_SCSI_PCI(vpci_dev); DeviceState *vdev = DEVICE(&dev->vdev); - VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(vdev); DeviceState *proxy = DEVICE(vpci_dev); + VirtIOSCSIConf *conf = &dev->vdev.parent_obj.conf; char *bus_name; + if (conf->num_queues == VIRTIO_SCSI_AUTO_NUM_QUEUES) { + conf->num_queues = + virtio_pci_optimal_num_queues(VIRTIO_SCSI_VQ_NUM_FIXED); + } + if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) { - vpci_dev->nvectors = vs->conf.num_queues + - VIRTIO_SCSI_VQ_NUM_FIXED + 1; + vpci_dev->nvectors = conf->num_queues + VIRTIO_SCSI_VQ_NUM_FIXED + 1; } /*