From patchwork Thu Oct 8 19:10:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 303406 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20ECFC433E7 for ; Thu, 8 Oct 2020 19:12:30 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7A12E21789 for ; Thu, 8 Oct 2020 19:12:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HpS+N+s9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7A12E21789 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:40872 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kQbL2-0003PV-3P for qemu-devel@archiver.kernel.org; Thu, 08 Oct 2020 15:12:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49216) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kQbJf-0001up-MG for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:33828) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kQbJd-00059H-Jo for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1602184260; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hE5tVBwV/RnyMnB02VZVAukfGpAPywHYUJDs3ZCWRMw=; b=HpS+N+s9H7wfvAEX+pWQ4W9twtjoFKSAJqV3+3RzA1wCdPFlF5ZY7bMfTUs0cgEEbGqbXo tuposC24R4rhsCZrE+H9JDEPB9ShEc+q3t0FfEyf7G33/QY6zvPeQq6QTVdmo0hCwU1reM vDzfBo1ZYjAmGxghikP/6QA9ef5dJ+M= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-440-McRSH0GEPAuOAozY2TMBdg-1; Thu, 08 Oct 2020 15:10:58 -0400 X-MC-Unique: McRSH0GEPAuOAozY2TMBdg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7B0E9103098A; Thu, 8 Oct 2020 19:10:57 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-212.ams2.redhat.com [10.36.114.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id 27D735C1BD; Thu, 8 Oct 2020 19:10:55 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, alex.bennee@linaro.org, zhengchuan@huawei.com, stefanha@redhat.com, peterx@redhat.com Subject: [PULL 01/10] virtiofsd: Silence gcc warning Date: Thu, 8 Oct 2020 20:10:37 +0100 Message-Id: <20201008191046.272549-2-dgilbert@redhat.com> In-Reply-To: <20201008191046.272549-1-dgilbert@redhat.com> References: <20201008191046.272549-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/08 01:56:49 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: "Dr. David Alan Gilbert" Gcc worries fd might be used unset, in reality it's always set if fi is set, and only used if fi is set so it's safe. Initialise it to -1 just to keep gcc happy for now. Signed-off-by: Dr. David Alan Gilbert Message-Id: <20200827153657.111098-2-dgilbert@redhat.com> Reviewed-by: Ján Tomko Signed-off-by: Dr. David Alan Gilbert --- tools/virtiofsd/passthrough_ll.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/virtiofsd/passthrough_ll.c b/tools/virtiofsd/passthrough_ll.c index 0b229ebd57..36ad46e0c0 100644 --- a/tools/virtiofsd/passthrough_ll.c +++ b/tools/virtiofsd/passthrough_ll.c @@ -620,7 +620,7 @@ static void lo_setattr(fuse_req_t req, fuse_ino_t ino, struct stat *attr, struct lo_inode *inode; int ifd; int res; - int fd; + int fd = -1; inode = lo_inode(req, ino); if (!inode) { From patchwork Thu Oct 8 19:10:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 303404 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFAB0C433DF for ; Thu, 8 Oct 2020 19:14:26 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 202C721789 for ; Thu, 8 Oct 2020 19:14:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="M3o9V4O2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 202C721789 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:48838 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kQbMv-0006g3-1G for qemu-devel@archiver.kernel.org; Thu, 08 Oct 2020 15:14:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49260) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kQbJh-0001wr-Eg for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:57845) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kQbJf-00059W-AU for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1602184262; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MIs8GSYndujj/IUhzFBYJYVlEAHzu8gUjnP34Xj8hpI=; b=M3o9V4O2x6fA7qSQdEJKCZ/pSPIJbkEYv6NhOCQiY5/NrAp1PnVtUOJ2u3y1qY4do25pi8 ZSV2on59ZNzaDhZvmNf3hTMf5wioAYBfhgkAMVteAvQUJJlODR948wSJQxqBtskBVba/bi WTrj5k3F50flxom2gld0vB7iVZeD0os= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-525-yXOcXuw5MkWT9eB7P_1LYw-1; Thu, 08 Oct 2020 15:11:00 -0400 X-MC-Unique: yXOcXuw5MkWT9eB7P_1LYw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 74A78107AFC7; Thu, 8 Oct 2020 19:10:59 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-212.ams2.redhat.com [10.36.114.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id 083485C1BD; Thu, 8 Oct 2020 19:10:57 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, alex.bennee@linaro.org, zhengchuan@huawei.com, stefanha@redhat.com, peterx@redhat.com Subject: [PULL 02/10] tools/virtiofsd: add support for --socket-group Date: Thu, 8 Oct 2020 20:10:38 +0100 Message-Id: <20201008191046.272549-3-dgilbert@redhat.com> In-Reply-To: <20201008191046.272549-1-dgilbert@redhat.com> References: <20201008191046.272549-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/08 02:56:27 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée If you like running QEMU as a normal user (very common for TCG runs) but you have to run virtiofsd as a root user you run into connection problems. Adding support for an optional --socket-group allows the users to keep using the command line. Signed-off-by: Alex Bennée Reviewed-by: Stefan Hajnoczi Message-Id: <20200925125147.26943-2-alex.bennee@linaro.org> Signed-off-by: Dr. David Alan Gilbert dgilbert: Split long line --- docs/tools/virtiofsd.rst | 4 ++++ tools/virtiofsd/fuse_i.h | 1 + tools/virtiofsd/fuse_lowlevel.c | 6 ++++++ tools/virtiofsd/fuse_virtio.c | 21 +++++++++++++++++++-- 4 files changed, 30 insertions(+), 2 deletions(-) diff --git a/docs/tools/virtiofsd.rst b/docs/tools/virtiofsd.rst index ae02938a95..7ecee49834 100644 --- a/docs/tools/virtiofsd.rst +++ b/docs/tools/virtiofsd.rst @@ -87,6 +87,10 @@ Options Listen on vhost-user UNIX domain socket at PATH. +.. option:: --socket-group=GROUP + + Set the vhost-user UNIX domain socket gid to GROUP. + .. option:: --fd=FDNUM Accept connections from vhost-user UNIX domain socket file descriptor FDNUM. diff --git a/tools/virtiofsd/fuse_i.h b/tools/virtiofsd/fuse_i.h index 1240828208..492e002181 100644 --- a/tools/virtiofsd/fuse_i.h +++ b/tools/virtiofsd/fuse_i.h @@ -68,6 +68,7 @@ struct fuse_session { size_t bufsize; int error; char *vu_socket_path; + char *vu_socket_group; int vu_listen_fd; int vu_socketfd; struct fv_VuDev *virtio_dev; diff --git a/tools/virtiofsd/fuse_lowlevel.c b/tools/virtiofsd/fuse_lowlevel.c index 2dd36ec03b..4d1ba2925d 100644 --- a/tools/virtiofsd/fuse_lowlevel.c +++ b/tools/virtiofsd/fuse_lowlevel.c @@ -2523,6 +2523,7 @@ static const struct fuse_opt fuse_ll_opts[] = { LL_OPTION("--debug", debug, 1), LL_OPTION("allow_root", deny_others, 1), LL_OPTION("--socket-path=%s", vu_socket_path, 0), + LL_OPTION("--socket-group=%s", vu_socket_group, 0), LL_OPTION("--fd=%d", vu_listen_fd, 0), LL_OPTION("--thread-pool-size=%d", thread_pool_size, 0), FUSE_OPT_END @@ -2630,6 +2631,11 @@ struct fuse_session *fuse_session_new(struct fuse_args *args, "fuse: --socket-path and --fd cannot be given together\n"); goto out4; } + if (se->vu_socket_group && !se->vu_socket_path) { + fuse_log(FUSE_LOG_ERR, + "fuse: --socket-group can only be used with --socket-path\n"); + goto out4; + } se->bufsize = FUSE_MAX_MAX_PAGES * getpagesize() + FUSE_BUFFER_HEADER_SIZE; diff --git a/tools/virtiofsd/fuse_virtio.c b/tools/virtiofsd/fuse_virtio.c index d5c8e98253..89f537f79b 100644 --- a/tools/virtiofsd/fuse_virtio.c +++ b/tools/virtiofsd/fuse_virtio.c @@ -31,6 +31,8 @@ #include #include #include +#include +#include #include #include "contrib/libvhost-user/libvhost-user.h" @@ -924,15 +926,30 @@ static int fv_create_listen_socket(struct fuse_session *se) /* * Unfortunately bind doesn't let you set the mask on the socket, - * so set umask to 077 and restore it later. + * so set umask appropriately and restore it later. */ - old_umask = umask(0077); + if (se->vu_socket_group) { + old_umask = umask(S_IROTH | S_IWOTH | S_IXOTH); + } else { + old_umask = umask(S_IRGRP | S_IWGRP | S_IXGRP | + S_IROTH | S_IWOTH | S_IXOTH); + } if (bind(listen_sock, (struct sockaddr *)&un, addr_len) == -1) { fuse_log(FUSE_LOG_ERR, "vhost socket bind: %m\n"); close(listen_sock); umask(old_umask); return -1; } + if (se->vu_socket_group) { + struct group *g = getgrnam(se->vu_socket_group); + if (g) { + if (!chown(se->vu_socket_path, -1, g->gr_gid)) { + fuse_log(FUSE_LOG_WARNING, + "vhost socket failed to set group to %s (%d)\n", + se->vu_socket_group, g->gr_gid); + } + } + } umask(old_umask); if (listen(listen_sock, 1) == -1) { From patchwork Thu Oct 8 19:10:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 303402 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 237BEC433DF for ; Thu, 8 Oct 2020 19:17:38 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8A069208C7 for ; Thu, 8 Oct 2020 19:17:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Esadjbf6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8A069208C7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:57260 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kQbQ0-0001uE-66 for qemu-devel@archiver.kernel.org; Thu, 08 Oct 2020 15:17:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49296) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kQbJj-000203-A6 for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:52203) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kQbJh-00059n-HI for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1602184264; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BJR6SWX9PO2YeU8RnGbYoAO1TtPIctVvTm8T9SR/Fy4=; b=Esadjbf6Ll+ONcTbPQyFaFP0QibV++B4b7TVBUQlPMfNFmQA+weH0SnY2MM2GVEJUTgOd8 jO34HscCELH1oIk7LCtuf8nDJ6EdjSJSqdN5ktMuianT3dR93tQxW2M4Jxp2JTbqcZWEQe D489NPvwVXe64hhqNxGYCy5dHa85L98= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-488-0P_4sym6NiaID5vLCJo8Cg-1; Thu, 08 Oct 2020 15:11:02 -0400 X-MC-Unique: 0P_4sym6NiaID5vLCJo8Cg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id ED24618CBC41; Thu, 8 Oct 2020 19:11:00 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-212.ams2.redhat.com [10.36.114.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9F8185C1BD; Thu, 8 Oct 2020 19:10:59 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, alex.bennee@linaro.org, zhengchuan@huawei.com, stefanha@redhat.com, peterx@redhat.com Subject: [PULL 03/10] virtiofsd: Call qemu_init_exec_dir Date: Thu, 8 Oct 2020 20:10:39 +0100 Message-Id: <20201008191046.272549-4-dgilbert@redhat.com> In-Reply-To: <20201008191046.272549-1-dgilbert@redhat.com> References: <20201008191046.272549-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/08 01:56:49 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: "Dr. David Alan Gilbert" Since fcb4f59c879 qemu_get_local_state_pathname relies on the init_exec_dir, and virtiofsd asserts because we never set it. Set it. Reported-by: Alex Bennée Signed-off-by: Dr. David Alan Gilbert Message-Id: <20201002124015.44820-1-dgilbert@redhat.com> Tested-by: Alex Bennée Reviewed-by: Alex Bennée Reviewed-by: Stefan Hajnoczi Signed-off-by: Dr. David Alan Gilbert --- tools/virtiofsd/passthrough_ll.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/virtiofsd/passthrough_ll.c b/tools/virtiofsd/passthrough_ll.c index 36ad46e0c0..477e6ee0b5 100644 --- a/tools/virtiofsd/passthrough_ll.c +++ b/tools/virtiofsd/passthrough_ll.c @@ -2839,6 +2839,8 @@ int main(int argc, char *argv[]) /* Don't mask creation mode, kernel already did that */ umask(0); + qemu_init_exec_dir(argv[0]); + pthread_mutex_init(&lo.mutex, NULL); lo.inodes = g_hash_table_new(lo_key_hash, lo_key_equal); lo.root.fd = -1; From patchwork Thu Oct 8 19:10:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 271797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 615A9C433DF for ; Thu, 8 Oct 2020 19:15:47 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C22E32184D for ; Thu, 8 Oct 2020 19:15:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Vx6k2tRR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C22E32184D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:53408 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kQbOC-000075-31 for qemu-devel@archiver.kernel.org; Thu, 08 Oct 2020 15:15:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49306) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kQbJk-00022a-JD for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:50600) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kQbJi-00059y-Mb for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1602184265; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xiPfig7jG9VC0GE1wxVAHcnqkY098QwCgL+T1oBSk+E=; b=Vx6k2tRRMMB5uvMIUr9eUOKSGaalSkBwicBdSwPL9Pom6tVrMdEA/UM/PR0VcVyN63rX9Q FeCjEzPp57/xsEBjMmp0un8/WL6Di/cx40lGy/kbV9XIZgSpFzRery61hLjKUIizdI1ihO w329dny6uY9WDaqQBzC6GEFy+yy4xlo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-120-Jp9XRHudMkCXvjjWDvrCEA-1; Thu, 08 Oct 2020 15:11:03 -0400 X-MC-Unique: Jp9XRHudMkCXvjjWDvrCEA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9484818CBC42; Thu, 8 Oct 2020 19:11:02 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-212.ams2.redhat.com [10.36.114.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id 450635C1BD; Thu, 8 Oct 2020 19:11:01 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, alex.bennee@linaro.org, zhengchuan@huawei.com, stefanha@redhat.com, peterx@redhat.com Subject: [PULL 04/10] virtiofsd: avoid /proc/self/fd tempdir Date: Thu, 8 Oct 2020 20:10:40 +0100 Message-Id: <20201008191046.272549-5-dgilbert@redhat.com> In-Reply-To: <20201008191046.272549-1-dgilbert@redhat.com> References: <20201008191046.272549-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/08 01:56:49 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Stefan Hajnoczi In order to prevent /proc/self/fd escapes a temporary directory is created where /proc/self/fd is bind-mounted. This doesn't work on read-only file systems. Avoid the temporary directory by bind-mounting /proc/self/fd over /proc. This does not affect other processes since we remounted / with MS_REC | MS_SLAVE. /proc must exist and virtiofsd does not use it so it's safe to do this. Path traversal can be tested with the following function: static void test_proc_fd_escape(struct lo_data *lo) { int fd; int level = 0; ino_t last_ino = 0; fd = lo->proc_self_fd; for (;;) { struct stat st; if (fstat(fd, &st) != 0) { perror("fstat"); return; } if (last_ino && st.st_ino == last_ino) { fprintf(stderr, "inode number unchanged, stopping\n"); return; } last_ino = st.st_ino; fprintf(stderr, "Level %d dev %lu ino %lu\n", level, (unsigned long)st.st_dev, (unsigned long)last_ino); fd = openat(fd, "..", O_PATH | O_DIRECTORY | O_NOFOLLOW); level++; } } Before and after this patch only Level 0 is displayed. Without /proc/self/fd bind-mount protection it is possible to traverse parent directories. Fixes: 397ae982f4df4 ("virtiofsd: jail lo->proc_self_fd") Cc: Miklos Szeredi Cc: Jens Freimann Signed-off-by: Stefan Hajnoczi Message-Id: <20201006095826.59813-1-stefanha@redhat.com> Reviewed-by: Dr. David Alan Gilbert Tested-by: Jens Freimann Reviewed-by: Jens Freimann Signed-off-by: Dr. David Alan Gilbert --- tools/virtiofsd/passthrough_ll.c | 34 +++++++++++--------------------- 1 file changed, 11 insertions(+), 23 deletions(-) diff --git a/tools/virtiofsd/passthrough_ll.c b/tools/virtiofsd/passthrough_ll.c index 477e6ee0b5..ff53df4451 100644 --- a/tools/virtiofsd/passthrough_ll.c +++ b/tools/virtiofsd/passthrough_ll.c @@ -2393,8 +2393,6 @@ static void setup_wait_parent_capabilities(void) static void setup_namespaces(struct lo_data *lo, struct fuse_session *se) { pid_t child; - char template[] = "virtiofsd-XXXXXX"; - char *tmpdir; /* * Create a new pid namespace for *child* processes. We'll have to @@ -2458,33 +2456,23 @@ static void setup_namespaces(struct lo_data *lo, struct fuse_session *se) exit(1); } - tmpdir = mkdtemp(template); - if (!tmpdir) { - fuse_log(FUSE_LOG_ERR, "tmpdir(%s): %m\n", template); - exit(1); - } - - if (mount("/proc/self/fd", tmpdir, NULL, MS_BIND, NULL) < 0) { - fuse_log(FUSE_LOG_ERR, "mount(/proc/self/fd, %s, MS_BIND): %m\n", - tmpdir); + /* + * We only need /proc/self/fd. Prevent ".." from accessing parent + * directories of /proc/self/fd by bind-mounting it over /proc. Since / was + * previously remounted with MS_REC | MS_SLAVE this mount change only + * affects our process. + */ + if (mount("/proc/self/fd", "/proc", NULL, MS_BIND, NULL) < 0) { + fuse_log(FUSE_LOG_ERR, "mount(/proc/self/fd, MS_BIND): %m\n"); exit(1); } - /* Now we can get our /proc/self/fd directory file descriptor */ - lo->proc_self_fd = open(tmpdir, O_PATH); + /* Get the /proc (actually /proc/self/fd, see above) file descriptor */ + lo->proc_self_fd = open("/proc", O_PATH); if (lo->proc_self_fd == -1) { - fuse_log(FUSE_LOG_ERR, "open(%s, O_PATH): %m\n", tmpdir); + fuse_log(FUSE_LOG_ERR, "open(/proc, O_PATH): %m\n"); exit(1); } - - if (umount2(tmpdir, MNT_DETACH) < 0) { - fuse_log(FUSE_LOG_ERR, "umount2(%s, MNT_DETACH): %m\n", tmpdir); - exit(1); - } - - if (rmdir(tmpdir) < 0) { - fuse_log(FUSE_LOG_ERR, "rmdir(%s): %m\n", tmpdir); - } } /* From patchwork Thu Oct 8 19:10:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 271800 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BE4AC433DF for ; Thu, 8 Oct 2020 19:12:47 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D735121789 for ; Thu, 8 Oct 2020 19:12:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="d2xvpemJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D735121789 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:41700 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kQbLJ-0003k7-Kv for qemu-devel@archiver.kernel.org; Thu, 08 Oct 2020 15:12:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49328) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kQbJn-00026m-84 for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:46813) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kQbJl-0005AL-D7 for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1602184268; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W85WJLLeimVgYQvZmhoPM6kQHkNmqQwrNZAwhc++t9I=; b=d2xvpemJc71UEZSEqf/aNcAizl4iXOvHfC7Q7EaN/8IgUyl7kxfE8Dnc5bHj4xpgh9kFsx 8hs3lcmu+xZvZJfhDXJXrQvk+EMnVhURf02CTyejxzAQCDHIRL7F39KTLMzHD5SguxkTiq KeGZaRWFdiAXWgDaJHOzdcXq3agUW8U= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-172-8VRy5tluNey0CSBwsjWzaA-1; Thu, 08 Oct 2020 15:11:05 -0400 X-MC-Unique: 8VRy5tluNey0CSBwsjWzaA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3A436107AFDB; Thu, 8 Oct 2020 19:11:04 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-212.ams2.redhat.com [10.36.114.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id DE2655C1BD; Thu, 8 Oct 2020 19:11:02 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, alex.bennee@linaro.org, zhengchuan@huawei.com, stefanha@redhat.com, peterx@redhat.com Subject: [PULL 05/10] migration: Pass incoming state into qemu_ufd_copy_ioctl() Date: Thu, 8 Oct 2020 20:10:41 +0100 Message-Id: <20201008191046.272549-6-dgilbert@redhat.com> In-Reply-To: <20201008191046.272549-1-dgilbert@redhat.com> References: <20201008191046.272549-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/08 01:56:49 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Peter Xu It'll be used in follow up patches to access more fields out of it. Meanwhile fetch the userfaultfd inside the function. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Message-Id: <20201002175336.30858-2-peterx@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/postcopy-ram.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 0a2f88a87d..722034dc01 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1128,10 +1128,12 @@ int postcopy_ram_incoming_setup(MigrationIncomingState *mis) return 0; } -static int qemu_ufd_copy_ioctl(int userfault_fd, void *host_addr, +static int qemu_ufd_copy_ioctl(MigrationIncomingState *mis, void *host_addr, void *from_addr, uint64_t pagesize, RAMBlock *rb) { + int userfault_fd = mis->userfault_fd; int ret; + if (from_addr) { struct uffdio_copy copy_struct; copy_struct.dst = (uint64_t)(uintptr_t)host_addr; @@ -1185,7 +1187,7 @@ int postcopy_place_page(MigrationIncomingState *mis, void *host, void *from, * which would be slightly cheaper, but we'd have to be careful * of the order of updating our page state. */ - if (qemu_ufd_copy_ioctl(mis->userfault_fd, host, from, pagesize, rb)) { + if (qemu_ufd_copy_ioctl(mis, host, from, pagesize, rb)) { int e = errno; error_report("%s: %s copy host: %p from: %p (size: %zd)", __func__, strerror(e), host, from, pagesize); @@ -1212,7 +1214,7 @@ int postcopy_place_page_zero(MigrationIncomingState *mis, void *host, * but it's not available for everything (e.g. hugetlbpages) */ if (qemu_ram_is_uf_zeroable(rb)) { - if (qemu_ufd_copy_ioctl(mis->userfault_fd, host, NULL, pagesize, rb)) { + if (qemu_ufd_copy_ioctl(mis, host, NULL, pagesize, rb)) { int e = errno; error_report("%s: %s zero host: %p", __func__, strerror(e), host); From patchwork Thu Oct 8 19:10:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 303400 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E90CC433E7 for ; Thu, 8 Oct 2020 19:19:20 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8D30D21789 for ; Thu, 8 Oct 2020 19:19:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="VJKJ3nTR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8D30D21789 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:37062 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kQbRe-0005GT-H3 for qemu-devel@archiver.kernel.org; Thu, 08 Oct 2020 15:19:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49354) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kQbJp-0002BI-Vd for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:30638) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kQbJo-0005Ac-0r for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1602184271; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ETXftxfOfwsfPpoqGQE70ovEFgQN8TGTkEmuE25RerY=; b=VJKJ3nTR2GTrMjN1ra1D4JZeboemW4ZbpHNwC707CuLVSAJ+OJ3kKFnlVfLI/GFSP8PNPC wDQtutn9T3RPMubyVq/bkZKfKBnfyrOLbHXcVbFQEEZ4M/uet5gffY84Bvt6Q6h0a8yC/a WVQ5gWDScpWzdyD1zcB9lmxNFD2IR2U= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-143-GB50m2cQMGODCgi5l9xzdA-1; Thu, 08 Oct 2020 15:11:07 -0400 X-MC-Unique: GB50m2cQMGODCgi5l9xzdA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D1E3C84A5E3; Thu, 8 Oct 2020 19:11:05 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-212.ams2.redhat.com [10.36.114.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id 845C55C1A3; Thu, 8 Oct 2020 19:11:04 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, alex.bennee@linaro.org, zhengchuan@huawei.com, stefanha@redhat.com, peterx@redhat.com Subject: [PULL 06/10] migration: Introduce migrate_send_rp_message_req_pages() Date: Thu, 8 Oct 2020 20:10:42 +0100 Message-Id: <20201008191046.272549-7-dgilbert@redhat.com> In-Reply-To: <20201008191046.272549-1-dgilbert@redhat.com> References: <20201008191046.272549-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/08 02:56:27 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Peter Xu This is another layer wrapper for sending a page request to the source VM. The new migrate_send_rp_message_req_pages() will be used elsewhere in coming patches. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Message-Id: <20201002175336.30858-3-peterx@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 10 ++++++++-- migration/migration.h | 2 ++ 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index aca7fdcd0b..b2dac6b39c 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -316,8 +316,8 @@ error: * Start: Address offset within the RB * Len: Length in bytes required - must be a multiple of pagesize */ -int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, - ram_addr_t start) +int migrate_send_rp_message_req_pages(MigrationIncomingState *mis, + RAMBlock *rb, ram_addr_t start) { uint8_t bufc[12 + 1 + 255]; /* start (8), len (4), rbname up to 256 */ size_t msglen = 12; /* start + len */ @@ -353,6 +353,12 @@ int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, return migrate_send_rp_message(mis, msg_type, msglen, bufc); } +int migrate_send_rp_req_pages(MigrationIncomingState *mis, + RAMBlock *rb, ram_addr_t start) +{ + return migrate_send_rp_message_req_pages(mis, rb, start); +} + static bool migration_colo_enabled; bool migration_incoming_colo_enabled(void) { diff --git a/migration/migration.h b/migration/migration.h index deb411aaad..e853ccf8b1 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -333,6 +333,8 @@ void migrate_send_rp_pong(MigrationIncomingState *mis, uint32_t value); int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, ram_addr_t start); +int migrate_send_rp_message_req_pages(MigrationIncomingState *mis, + RAMBlock *rb, ram_addr_t start); void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis, char *block_name); void migrate_send_rp_resume_ack(MigrationIncomingState *mis, uint32_t value); From patchwork Thu Oct 8 19:10:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 303405 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2821EC433E7 for ; Thu, 8 Oct 2020 19:12:59 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4A58621789 for ; Thu, 8 Oct 2020 19:12:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="R3JFsHAm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4A58621789 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:42382 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kQbLV-00040r-8W for qemu-devel@archiver.kernel.org; Thu, 08 Oct 2020 15:12:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49368) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kQbJs-0002G2-9R for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:44789) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kQbJp-0005Ai-Oy for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1602184272; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A0PIZIe2yf/h11Vk9+3KrQQ/0ry+dbRTrvMnpKQFI3U=; b=R3JFsHAmYV1dyLQZ0+OwnJ6HGKV2vmFnyzowpoI3DGXM8IPRKLZ/VciovJbtDzKaamAnq9 KLAeK7Z3X/8iEkD1AKIZ6FLF5y//RXVw+98KreMQwRbd/CAYMshG8vJaibSQH11q3LOCzF 6VQv6gRnniMPCNh8GnbuVXp3yXXr0jo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-428-Au6RgSylNF2upvmaD_qqSA-1; Thu, 08 Oct 2020 15:11:08 -0400 X-MC-Unique: Au6RgSylNF2upvmaD_qqSA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7BD79107B471; Thu, 8 Oct 2020 19:11:07 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-212.ams2.redhat.com [10.36.114.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id 297625C1A3; Thu, 8 Oct 2020 19:11:06 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, alex.bennee@linaro.org, zhengchuan@huawei.com, stefanha@redhat.com, peterx@redhat.com Subject: [PULL 07/10] migration: Maintain postcopy faulted addresses Date: Thu, 8 Oct 2020 20:10:43 +0100 Message-Id: <20201008191046.272549-8-dgilbert@redhat.com> In-Reply-To: <20201008191046.272549-1-dgilbert@redhat.com> References: <20201008191046.272549-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/08 01:56:49 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Peter Xu Maintain a list of faulted addresses on the destination host for which we're waiting on. This is implemented using a GTree rather than a real list to make sure even there're plenty of vCPUs/threads that are faulting, the lookup will still be fast with O(log(N)) (because we'll do that after placing each page). It should bring a slight overhead, but ideally that shouldn't be a big problem simply because in most cases the requested page list will be short. Actually we did similar things for postcopy blocktime measurements. This patch didn't use that simply because: (1) blocktime measurement is towards vcpu threads only, but here we need to record all faulted addresses, including main thread and external thread (like, DPDK via vhost-user). (2) blocktime measurement will require UFFD_FEATURE_THREAD_ID, but here we don't want to add that extra dependency on the kernel version since not necessary. E.g., we don't need to know which thread faulted on which page, we also don't care about multiple threads faulting on the same page. But we only care about what addresses are faulted so waiting for a page copying from src. (3) blocktime measurement is not enabled by default. However we need this by default especially for postcopy recover. Another thing to mention is that this patch introduced a new mutex to serialize the receivedmap and the page_requested tree, however that serialization does not cover other procedures like UFFDIO_COPY. Signed-off-by: Peter Xu Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 41 +++++++++++++++++++++++++++++++++++++++- migration/migration.h | 19 ++++++++++++++++++- migration/postcopy-ram.c | 17 ++++++++++++++--- migration/trace-events | 2 ++ 4 files changed, 74 insertions(+), 5 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index b2dac6b39c..d8a5c0de44 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -143,6 +143,13 @@ static int migration_maybe_pause(MigrationState *s, int new_state); static void migrate_fd_cancel(MigrationState *s); +static gint page_request_addr_cmp(gconstpointer ap, gconstpointer bp) +{ + uintptr_t a = (uintptr_t) ap, b = (uintptr_t) bp; + + return (a > b) - (a < b); +} + void migration_object_init(void) { MachineState *ms = MACHINE(qdev_get_machine()); @@ -165,6 +172,8 @@ void migration_object_init(void) qemu_event_init(¤t_incoming->main_thread_load_event, false); qemu_sem_init(¤t_incoming->postcopy_pause_sem_dst, 0); qemu_sem_init(¤t_incoming->postcopy_pause_sem_fault, 0); + qemu_mutex_init(¤t_incoming->page_request_mutex); + current_incoming->page_requested = g_tree_new(page_request_addr_cmp); if (!migration_object_check(current_migration, &err)) { error_report_err(err); @@ -240,6 +249,11 @@ void migration_incoming_state_destroy(void) qemu_event_reset(&mis->main_thread_load_event); + if (mis->page_requested) { + g_tree_destroy(mis->page_requested); + mis->page_requested = NULL; + } + if (mis->socket_address_list) { qapi_free_SocketAddressList(mis->socket_address_list); mis->socket_address_list = NULL; @@ -354,8 +368,33 @@ int migrate_send_rp_message_req_pages(MigrationIncomingState *mis, } int migrate_send_rp_req_pages(MigrationIncomingState *mis, - RAMBlock *rb, ram_addr_t start) + RAMBlock *rb, ram_addr_t start, uint64_t haddr) { + void *aligned = (void *)(uintptr_t)(haddr & (-qemu_target_page_size())); + bool received; + + WITH_QEMU_LOCK_GUARD(&mis->page_request_mutex) { + received = ramblock_recv_bitmap_test_byte_offset(rb, start); + if (!received && !g_tree_lookup(mis->page_requested, aligned)) { + /* + * The page has not been received, and it's not yet in the page + * request list. Queue it. Set the value of element to 1, so that + * things like g_tree_lookup() will return TRUE (1) when found. + */ + g_tree_insert(mis->page_requested, aligned, (gpointer)1); + mis->page_requested_count++; + trace_postcopy_page_req_add(aligned, mis->page_requested_count); + } + } + + /* + * If the page is there, skip sending the message. We don't even need the + * lock because as long as the page arrived, it'll be there forever. + */ + if (received) { + return 0; + } + return migrate_send_rp_message_req_pages(mis, rb, start); } diff --git a/migration/migration.h b/migration/migration.h index e853ccf8b1..8d2d1ce839 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -104,6 +104,23 @@ struct MigrationIncomingState { /* List of listening socket addresses */ SocketAddressList *socket_address_list; + + /* A tree of pages that we requested to the source VM */ + GTree *page_requested; + /* For debugging purpose only, but would be nice to keep */ + int page_requested_count; + /* + * The mutex helps to maintain the requested pages that we sent to the + * source, IOW, to guarantee coherent between the page_requests tree and + * the per-ramblock receivedmap. Note! This does not guarantee consistency + * of the real page copy procedures (using UFFDIO_[ZERO]COPY). E.g., even + * if one bit in receivedmap is cleared, UFFDIO_COPY could have happened + * for that page already. This is intended so that the mutex won't + * serialize and blocked by slow operations like UFFDIO_* ioctls. However + * this should be enough to make sure the page_requested tree always + * contains valid information. + */ + QemuMutex page_request_mutex; }; MigrationIncomingState *migration_incoming_get_current(void); @@ -332,7 +349,7 @@ void migrate_send_rp_shut(MigrationIncomingState *mis, void migrate_send_rp_pong(MigrationIncomingState *mis, uint32_t value); int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, - ram_addr_t start); + ram_addr_t start, uint64_t haddr); int migrate_send_rp_message_req_pages(MigrationIncomingState *mis, RAMBlock *rb, ram_addr_t start); void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis, diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 722034dc01..ca1daf0024 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -684,7 +684,7 @@ int postcopy_request_shared_page(struct PostCopyFD *pcfd, RAMBlock *rb, qemu_ram_get_idstr(rb), rb_offset); return postcopy_wake_shared(pcfd, client_addr, rb); } - migrate_send_rp_req_pages(mis, rb, aligned_rbo); + migrate_send_rp_req_pages(mis, rb, aligned_rbo, client_addr); return 0; } @@ -979,7 +979,8 @@ retry: * Send the request to the source - we want to request one * of our host page sizes (which is >= TPS) */ - ret = migrate_send_rp_req_pages(mis, rb, rb_offset); + ret = migrate_send_rp_req_pages(mis, rb, rb_offset, + msg.arg.pagefault.address); if (ret) { /* May be network failure, try to wait for recovery */ if (ret == -EIO && postcopy_pause_fault_thread(mis)) { @@ -1149,10 +1150,20 @@ static int qemu_ufd_copy_ioctl(MigrationIncomingState *mis, void *host_addr, ret = ioctl(userfault_fd, UFFDIO_ZEROPAGE, &zero_struct); } if (!ret) { + qemu_mutex_lock(&mis->page_request_mutex); ramblock_recv_bitmap_set_range(rb, host_addr, pagesize / qemu_target_page_size()); + /* + * If this page resolves a page fault for a previous recorded faulted + * address, take a special note to maintain the requested page list. + */ + if (g_tree_lookup(mis->page_requested, host_addr)) { + g_tree_remove(mis->page_requested, host_addr); + mis->page_requested_count--; + trace_postcopy_page_req_del(host_addr, mis->page_requested_count); + } + qemu_mutex_unlock(&mis->page_request_mutex); mark_postcopy_blocktime_end((uintptr_t)host_addr); - } return ret; } diff --git a/migration/trace-events b/migration/trace-events index 338f38b3dd..e4d5eb94ca 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -162,6 +162,7 @@ postcopy_pause_return_path(void) "" postcopy_pause_return_path_continued(void) "" postcopy_pause_continued(void) "" postcopy_start_set_run(void) "" +postcopy_page_req_add(void *addr, int count) "new page req %p total %d" source_return_path_thread_bad_end(void) "" source_return_path_thread_end(void) "" source_return_path_thread_entry(void) "" @@ -272,6 +273,7 @@ postcopy_ram_incoming_cleanup_blocktime(uint64_t total) "total blocktime %" PRIu postcopy_request_shared_page(const char *sharer, const char *rb, uint64_t rb_offset) "for %s in %s offset 0x%"PRIx64 postcopy_request_shared_page_present(const char *sharer, const char *rb, uint64_t rb_offset) "%s already %s offset 0x%"PRIx64 postcopy_wake_shared(uint64_t client_addr, const char *rb) "at 0x%"PRIx64" in %s" +postcopy_page_req_del(void *addr, int count) "resolved page req %p total %d" get_mem_fault_cpu_index(int cpu, uint32_t pid) "cpu: %d, pid: %u" From patchwork Thu Oct 8 19:10:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 271795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72849C433E7 for ; Thu, 8 Oct 2020 19:18:44 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C598F21789 for ; Thu, 8 Oct 2020 19:18:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NQV0tSdc" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C598F21789 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:33512 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kQbR4-0003mH-MS for qemu-devel@archiver.kernel.org; Thu, 08 Oct 2020 15:18:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49392) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kQbJw-0002Ok-4S for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:20 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:52850) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kQbJu-0005B2-7Q for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1602184277; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fqIuVZSLTsHJ7zWi6B3v7Jb9InR1IFs/G1h6Do57lEU=; b=NQV0tSdcgUTpLBoja41fu9+il/E7JAoqtm14UCaNRfQuFGWlxoBDCKT5qJ7SH9sjR/Bxy4 mFnEClMsBAO7k6N6i+LN6LMD23V0zI/C19naKPoBIaDrXQx7opIGKO6KMBXdpiiyszuNgN rRVMXfOhExaZVTMowNhVdfoxEE5J9hc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-175-1navZJ8CPoCd28yiKgubQQ-1; Thu, 08 Oct 2020 15:11:15 -0400 X-MC-Unique: 1navZJ8CPoCd28yiKgubQQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7DDF3107B470; Thu, 8 Oct 2020 19:11:14 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-212.ams2.redhat.com [10.36.114.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id C6E945C1A3; Thu, 8 Oct 2020 19:11:07 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, alex.bennee@linaro.org, zhengchuan@huawei.com, stefanha@redhat.com, peterx@redhat.com Subject: [PULL 08/10] migration: Sync requested pages after postcopy recovery Date: Thu, 8 Oct 2020 20:10:44 +0100 Message-Id: <20201008191046.272549-9-dgilbert@redhat.com> In-Reply-To: <20201008191046.272549-1-dgilbert@redhat.com> References: <20201008191046.272549-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/08 02:56:27 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Peter Xu We synchronize the requested pages right after a postcopy recovery happens. This helps to synchronize the prioritized pages on source so that the faulted threads can be served faster. Reported-by: Xiaohui Li Signed-off-by: Peter Xu Message-Id: <20201002175336.30858-5-peterx@redhat.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- migration/savevm.c | 57 ++++++++++++++++++++++++++++++++++++++++++ migration/trace-events | 1 + 2 files changed, 58 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index d2e141f7b1..33acbba1a4 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2011,6 +2011,49 @@ static int loadvm_postcopy_handle_run(MigrationIncomingState *mis) return LOADVM_QUIT; } +/* We must be with page_request_mutex held */ +static gboolean postcopy_sync_page_req(gpointer key, gpointer value, + gpointer data) +{ + MigrationIncomingState *mis = data; + void *host_addr = (void *) key; + ram_addr_t rb_offset; + RAMBlock *rb; + int ret; + + rb = qemu_ram_block_from_host(host_addr, true, &rb_offset); + if (!rb) { + /* + * This should _never_ happen. However be nice for a migrating VM to + * not crash/assert. Post an error (note: intended to not use *_once + * because we do want to see all the illegal addresses; and this can + * never be triggered by the guest so we're safe) and move on next. + */ + error_report("%s: illegal host addr %p", __func__, host_addr); + /* Try the next entry */ + return FALSE; + } + + ret = migrate_send_rp_message_req_pages(mis, rb, rb_offset); + if (ret) { + /* Please refer to above comment. */ + error_report("%s: send rp message failed for addr %p", + __func__, host_addr); + return FALSE; + } + + trace_postcopy_page_req_sync(host_addr); + + return FALSE; +} + +static void migrate_send_rp_req_pages_pending(MigrationIncomingState *mis) +{ + WITH_QEMU_LOCK_GUARD(&mis->page_request_mutex) { + g_tree_foreach(mis->page_requested, postcopy_sync_page_req, mis); + } +} + static int loadvm_postcopy_handle_resume(MigrationIncomingState *mis) { if (mis->state != MIGRATION_STATUS_POSTCOPY_RECOVER) { @@ -2033,6 +2076,20 @@ static int loadvm_postcopy_handle_resume(MigrationIncomingState *mis) /* Tell source that "we are ready" */ migrate_send_rp_resume_ack(mis, MIGRATION_RESUME_ACK_VALUE); + /* + * After a postcopy recovery, the source should have lost the postcopy + * queue, or potentially the requested pages could have been lost during + * the network down phase. Let's re-sync with the source VM by re-sending + * all the pending pages that we eagerly need, so these threads won't get + * blocked too long due to the recovery. + * + * Without this procedure, the faulted destination VM threads (waiting for + * page requests right before the postcopy is interrupted) can keep hanging + * until the pages are sent by the source during the background copying of + * pages, or another thread faulted on the same address accidentally. + */ + migrate_send_rp_req_pages_pending(mis); + return 0; } diff --git a/migration/trace-events b/migration/trace-events index e4d5eb94ca..0fbfd2da60 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -49,6 +49,7 @@ vmstate_save(const char *idstr, const char *vmsd_name) "%s, %s" vmstate_load(const char *idstr, const char *vmsd_name) "%s, %s" postcopy_pause_incoming(void) "" postcopy_pause_incoming_continued(void) "" +postcopy_page_req_sync(void *host_addr) "sync page req %p" # vmstate.c vmstate_load_field_error(const char *field, int ret) "field \"%s\" load failed, ret = %d" From patchwork Thu Oct 8 19:10:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 271798 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76FF0C43457 for ; Thu, 8 Oct 2020 19:14:51 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CA39021789 for ; Thu, 8 Oct 2020 19:14:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HCiQzGr3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CA39021789 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:49796 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kQbNJ-00074l-M4 for qemu-devel@archiver.kernel.org; Thu, 08 Oct 2020 15:14:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49422) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kQbJy-0002Uu-Ph for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:36382) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kQbJw-0005BC-Sn for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1602184280; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U0f7a6Fc7A+7mZcGUG5Hm7ANHiCXaOvgGvgfUjI3+ow=; b=HCiQzGr3vdGdwa8HEhjPMbcpB4pHncFO1lOW0y8AJXVsl1B40oeqP4xCrlzLbaT5cqvg9o 2rtgymWiJuTTJBdNLbQH/DZyHTBC8AZmiY3I5ZGRbLJHMgSt9GaTzlN7BkfllNnyXB7MEm RoPmbc7tALgmlJUF2vAEPj+qlYzDA6w= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-284-RIDE__bcNpmKMJLUVYvlzw-1; Thu, 08 Oct 2020 15:11:18 -0400 X-MC-Unique: RIDE__bcNpmKMJLUVYvlzw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 44FCBAE823; Thu, 8 Oct 2020 19:11:16 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-212.ams2.redhat.com [10.36.114.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id C8F065C1BD; Thu, 8 Oct 2020 19:11:14 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, alex.bennee@linaro.org, zhengchuan@huawei.com, stefanha@redhat.com, peterx@redhat.com Subject: [PULL 09/10] migration/dirtyrate: record start_time and calc_time while at the measuring state Date: Thu, 8 Oct 2020 20:10:45 +0100 Message-Id: <20201008191046.272549-10-dgilbert@redhat.com> In-Reply-To: <20201008191046.272549-1-dgilbert@redhat.com> References: <20201008191046.272549-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/08 01:56:49 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng Querying could include both the start-time and the calc-time while at the measuring state, allowing a caller to determine when they should expect to come back looking for a result. Signed-off-by: Chuan Zheng Message-Id: <1601350938-128320-2-git-send-email-zhengchuan@huawei.com> Reviewed-by: David Edmondson Signed-off-by: Dr. David Alan Gilbert --- migration/dirtyrate.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 68577ef250..40e41e793e 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -83,14 +83,14 @@ static struct DirtyRateInfo *query_dirty_rate_info(void) return info; } -static void reset_dirtyrate_stat(void) +static void init_dirtyrate_stat(int64_t start_time, int64_t calc_time) { DirtyStat.total_dirty_samples = 0; DirtyStat.total_sample_count = 0; DirtyStat.total_block_mem_MB = 0; DirtyStat.dirty_rate = -1; - DirtyStat.start_time = 0; - DirtyStat.calc_time = 0; + DirtyStat.start_time = start_time; + DirtyStat.calc_time = calc_time; } static void update_dirtyrate_stat(struct RamblockDirtyInfo *info) @@ -335,7 +335,6 @@ static void calculate_dirtyrate(struct DirtyRateConfig config) int64_t initial_time; rcu_register_thread(); - reset_dirtyrate_stat(); rcu_read_lock(); initial_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME); if (!record_ramblock_hash_info(&block_dinfo, config, &block_count)) { @@ -365,6 +364,8 @@ void *get_dirtyrate_thread(void *arg) { struct DirtyRateConfig config = *(struct DirtyRateConfig *)arg; int ret; + int64_t start_time; + int64_t calc_time; ret = dirtyrate_set_state(&CalculatingState, DIRTY_RATE_STATUS_UNSTARTED, DIRTY_RATE_STATUS_MEASURING); @@ -373,6 +374,10 @@ void *get_dirtyrate_thread(void *arg) return NULL; } + start_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME) / 1000; + calc_time = config.sample_period_seconds; + init_dirtyrate_stat(start_time, calc_time); + calculate_dirtyrate(config); ret = dirtyrate_set_state(&CalculatingState, DIRTY_RATE_STATUS_MEASURING, From patchwork Thu Oct 8 19:10:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 271793 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F29BC433DF for ; Thu, 8 Oct 2020 19:21:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B971C21789 for ; Thu, 8 Oct 2020 19:21:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HbeXoLOU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B971C21789 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:43266 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kQbTq-0007yZ-Lt for qemu-devel@archiver.kernel.org; Thu, 08 Oct 2020 15:21:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49434) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kQbJz-0002Ws-JD for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:24589) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kQbJx-0005BG-F2 for qemu-devel@nongnu.org; Thu, 08 Oct 2020 15:11:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1602184280; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=04mH/EQN2FMbm79HfXdBhiaZTkRTdOPvt3HTGNa6fxc=; b=HbeXoLOUExthTokkNswWvOMDSu1Pfrw90u+IjMS7fjLLShfsyJ+mOT+QEYcT2CURwz5GyL U4jCGKFhfLUjwPS9YwFWo7QB0M3Elyvi7WPGWqNBdcfIGuKaaq8Z9mA2ySrrCG2FAKwcWy oJHzZbSItu1WZZwyZZ73AGGH8uIybok= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-532-4mwwIaVXMJytjreyREq-Eg-1; Thu, 08 Oct 2020 15:11:18 -0400 X-MC-Unique: 4mwwIaVXMJytjreyREq-Eg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B860A18CBC40; Thu, 8 Oct 2020 19:11:17 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-212.ams2.redhat.com [10.36.114.212]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6D43E5C1A3; Thu, 8 Oct 2020 19:11:16 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, alex.bennee@linaro.org, zhengchuan@huawei.com, stefanha@redhat.com, peterx@redhat.com Subject: [PULL 10/10] migration/dirtyrate: present dirty rate only when querying the rate has completed Date: Thu, 8 Oct 2020 20:10:46 +0100 Message-Id: <20201008191046.272549-11-dgilbert@redhat.com> In-Reply-To: <20201008191046.272549-1-dgilbert@redhat.com> References: <20201008191046.272549-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/08 01:56:49 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng Make dirty_rate field optional, present dirty rate only when querying the rate has completed. The qmp results is shown as follow: @unstarted: {"return":{"status":"unstarted","start-time":0,"calc-time":0},"id":"libvirt-12"} @measuring: {"return":{"status":"measuring","start-time":102931,"calc-time":1},"id":"libvirt-85"} @measured: {"return":{"status":"measured","dirty-rate":4,"start-time":150146,"calc-time":1},"id":"libvirt-15"} Signed-off-by: Chuan Zheng Reviewed-by: David Edmondson Message-Id: <1601350938-128320-3-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/dirtyrate.c | 3 +-- qapi/migration.json | 8 +++----- 2 files changed, 4 insertions(+), 7 deletions(-) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 40e41e793e..ab9e1301f6 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -69,9 +69,8 @@ static struct DirtyRateInfo *query_dirty_rate_info(void) struct DirtyRateInfo *info = g_malloc0(sizeof(DirtyRateInfo)); if (qatomic_read(&CalculatingState) == DIRTY_RATE_STATUS_MEASURED) { + info->has_dirty_rate = true; info->dirty_rate = dirty_rate; - } else { - info->dirty_rate = -1; } info->status = CalculatingState; diff --git a/qapi/migration.json b/qapi/migration.json index 7f5e6fd681..974021a5c8 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1743,10 +1743,8 @@ # # Information about current dirty page rate of vm. # -# @dirty-rate: @dirtyrate describing the dirty page rate of vm -# in units of MB/s. -# If this field returns '-1', it means querying has not -# yet started or completed. +# @dirty-rate: an estimate of the dirty page rate of the VM in units of +# MB/s, present only when estimating the rate has completed. # # @status: status containing dirtyrate query status includes # 'unstarted' or 'measuring' or 'measured' @@ -1759,7 +1757,7 @@ # ## { 'struct': 'DirtyRateInfo', - 'data': {'dirty-rate': 'int64', + 'data': {'*dirty-rate': 'int64', 'status': 'DirtyRateStatus', 'start-time': 'int64', 'calc-time': 'int64'} }