From patchwork Fri Sep 25 12:06:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 272753 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09407C4363D for ; Fri, 25 Sep 2020 12:08:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2786A21D7A for ; Fri, 25 Sep 2020 12:08:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Sq1vfTOp" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2786A21D7A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:41686 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmWf-0003Sw-Rp for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:08:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38070) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmVS-0002B3-8X for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:19 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:52511) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmVO-0004Cd-5a for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:17 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035631; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ikZ5UtE3+vjTJczLSWLwfVuGSIdVlZO2PJ0ByXz+1L8=; b=Sq1vfTOpcs7vELZLNm4IHog+k3g44CZJBvMJ2/mTukgrj+UJa74fjVRXw9JSzaO1i9Idxl oFMHUjz6Myv3wZxjQaurBKkKEIPCahfxgB0TPCGSmKg3SMY99aJ3UBvf4MlncrciEC70zL vkJIaqsil9MV8RhMbYRDfAjoSquMmlk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-192-KF4qLxXBPGCAgDrw4VwhmA-1; Fri, 25 Sep 2020 08:07:09 -0400 X-MC-Unique: KF4qLxXBPGCAgDrw4VwhmA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D759410066FC; Fri, 25 Sep 2020 12:07:07 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8F92673692; Fri, 25 Sep 2020 12:07:05 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 01/26] migration: Properly destroy variables on incoming side Date: Fri, 25 Sep 2020 13:06:30 +0100 Message-Id: <20200925120655.295142-2-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Peter Xu In migration_incoming_state_destroy(), we've got a few variables that aren't destroyed properly, namely: main_thread_load_event postcopy_pause_sem_dst postcopy_pause_sem_fault rp_mutex Destroy them properly. Signed-off-by: Peter Xu Message-Id: <20200908203022.341615-2-peterx@redhat.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index d9d1e0b190..3495c9e542 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -238,12 +238,15 @@ void migration_incoming_state_destroy(void) mis->postcopy_remote_fds = NULL; } - qemu_event_reset(&mis->main_thread_load_event); - if (mis->socket_address_list) { qapi_free_SocketAddressList(mis->socket_address_list); mis->socket_address_list = NULL; } + + qemu_event_destroy(&mis->main_thread_load_event); + qemu_sem_destroy(&mis->postcopy_pause_sem_dst); + qemu_sem_destroy(&mis->postcopy_pause_sem_fault); + qemu_mutex_destroy(&mis->rp_mutex); } static void migrate_generate_event(int new_state) From patchwork Fri Sep 25 12:06:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 272751 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27E82C4363D for ; Fri, 25 Sep 2020 12:10:38 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 941062074B for ; Fri, 25 Sep 2020 12:10:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="RyYLb0yl" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 941062074B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:50118 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmYe-0006zw-I7 for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:10:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38084) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmVU-0002BQ-9C for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:42597) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmVQ-0004Cw-Sa for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:19 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035634; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=g11t0Rk3ousL47iNjK5Udtv7AJW1FtT5fTYIQ8CPVF0=; b=RyYLb0ylb36dJ46G7NRD37zhN5XT5kO7jMiRBZKoyJSETRfzJb0ggiTOyOI9fVTPprh8R2 5HThK+wpYiYLlyfKHY/YjnlR1ZqfZ9FCHuXQYzd+Q0eiFGUc0FtG7fYhUJaMOLyao63wUD B5Xk74iF9Uy75J1rpfuJ36rIDZzNt/w= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-515-pKc3RSS3PTSVFX7rX5ZN6w-1; Fri, 25 Sep 2020 08:07:12 -0400 X-MC-Unique: pKc3RSS3PTSVFX7rX5ZN6w-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 76C6210BBECB; Fri, 25 Sep 2020 12:07:10 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2EAAB73692; Fri, 25 Sep 2020 12:07:08 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 02/26] migration: Rework migrate_send_rp_req_pages() function Date: Fri, 25 Sep 2020 13:06:31 +0100 Message-Id: <20200925120655.295142-3-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 02:48:20 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Peter Xu We duplicated the logic of maintaining the last_rb variable at both callers of this function. Pass *rb pointer into the function so that we can avoid duplicating the logic. Also, when we have the rb pointer, it's also easier to remove the original 2nd & 4th parameters, because both of them (name of the ramblock when needed, or the page size) can be fetched from the ramblock pointer too. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Message-Id: <20200908203022.341615-3-peterx@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 26 ++++++++++++++++++-------- migration/migration.h | 4 ++-- migration/postcopy-ram.c | 24 ++---------------------- 3 files changed, 22 insertions(+), 32 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 3495c9e542..827f8be07d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -314,25 +314,35 @@ error: return ret; } -/* Request a range of pages from the source VM at the given - * start address. - * rbname: Name of the RAMBlock to request the page in, if NULL it's the same - * as the last request (a name must have been given previously) +/* Request one page from the source VM at the given start address. + * rb: the RAMBlock to request the page in * Start: Address offset within the RB * Len: Length in bytes required - must be a multiple of pagesize */ -int migrate_send_rp_req_pages(MigrationIncomingState *mis, const char *rbname, - ram_addr_t start, size_t len) +int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, + ram_addr_t start) { uint8_t bufc[12 + 1 + 255]; /* start (8), len (4), rbname up to 256 */ size_t msglen = 12; /* start + len */ + size_t len = qemu_ram_pagesize(rb); enum mig_rp_message_type msg_type; + const char *rbname; + int rbname_len; *(uint64_t *)bufc = cpu_to_be64((uint64_t)start); *(uint32_t *)(bufc + 8) = cpu_to_be32((uint32_t)len); - if (rbname) { - int rbname_len = strlen(rbname); + /* + * We maintain the last ramblock that we requested for page. Note that we + * don't need locking because this function will only be called within the + * postcopy ram fault thread. + */ + if (rb != mis->last_rb) { + mis->last_rb = rb; + + rbname = qemu_ram_get_idstr(rb); + rbname_len = strlen(rbname); + assert(rbname_len < 256); bufc[msglen++] = rbname_len; diff --git a/migration/migration.h b/migration/migration.h index bdc7450da3..e74042362d 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -326,8 +326,8 @@ void migrate_send_rp_shut(MigrationIncomingState *mis, uint32_t value); void migrate_send_rp_pong(MigrationIncomingState *mis, uint32_t value); -int migrate_send_rp_req_pages(MigrationIncomingState *mis, const char* rbname, - ram_addr_t start, size_t len); +int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, + ram_addr_t start); void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis, char *block_name); void migrate_send_rp_resume_ack(MigrationIncomingState *mis, uint32_t value); diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 1654ff11a5..0a2f88a87d 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -684,14 +684,7 @@ int postcopy_request_shared_page(struct PostCopyFD *pcfd, RAMBlock *rb, qemu_ram_get_idstr(rb), rb_offset); return postcopy_wake_shared(pcfd, client_addr, rb); } - if (rb != mis->last_rb) { - mis->last_rb = rb; - migrate_send_rp_req_pages(mis, qemu_ram_get_idstr(rb), - aligned_rbo, pagesize); - } else { - /* Save some space */ - migrate_send_rp_req_pages(mis, NULL, aligned_rbo, pagesize); - } + migrate_send_rp_req_pages(mis, rb, aligned_rbo); return 0; } @@ -986,20 +979,7 @@ retry: * Send the request to the source - we want to request one * of our host page sizes (which is >= TPS) */ - if (rb != mis->last_rb) { - mis->last_rb = rb; - ret = migrate_send_rp_req_pages(mis, - qemu_ram_get_idstr(rb), - rb_offset, - qemu_ram_pagesize(rb)); - } else { - /* Save some space */ - ret = migrate_send_rp_req_pages(mis, - NULL, - rb_offset, - qemu_ram_pagesize(rb)); - } - + ret = migrate_send_rp_req_pages(mis, rb, rb_offset); if (ret) { /* May be network failure, try to wait for recovery */ if (ret == -EIO && postcopy_pause_fault_thread(mis)) { From patchwork Fri Sep 25 12:06:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 304426 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A46AC4363D for ; Fri, 25 Sep 2020 12:11:26 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 817CB21D7A for ; Fri, 25 Sep 2020 12:11:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="U4dTRfsH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 817CB21D7A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:53524 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmZQ-0008Tj-4h for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:11:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38112) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmVX-0002Bq-9w for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:57022) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmVT-0004DN-TV for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:23 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035638; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8fT5Zma08Owo8hQzAyZxIdi4dLbrYqBsXoHvPgCnfPc=; b=U4dTRfsHzCLr1DFXn2GZ31OGIRu6OE4UNAYnwcxTJy1oflYVZYVPukUqGL5rlt36d2Zfze OFUSHSJ9q5D751G55mpq1G/lwy62epx8rNyNCxqR9ir6YUZfIEoVXMaxk3JRP5akKpekf/ YlGrtey0l+PS/OYFsi48Dvu7QCWWGGY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-567-DfDVf4bsOmWwYStUY9dzBg-1; Fri, 25 Sep 2020 08:07:14 -0400 X-MC-Unique: DfDVf4bsOmWwYStUY9dzBg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 18064805EEE; Fri, 25 Sep 2020 12:07:13 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id C314A73693; Fri, 25 Sep 2020 12:07:10 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 03/26] migration/dirtyrate: setup up query-dirtyrate framwork Date: Fri, 25 Sep 2020 13:06:32 +0100 Message-Id: <20200925120655.295142-4-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 02:48:20 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng Add get_dirtyrate_thread() functions to setup query-dirtyrate framework. Signed-off-by: Chuan Zheng Signed-off-by: YanYing Zhuang Reviewed-by: Dr. David Alan Gilbert Reviewed-by: David Edmondson Reviewed-by: Li Qiang Message-Id: <1600237327-33618-2-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/dirtyrate.c | 38 ++++++++++++++++++++++++++++++++++++++ migration/dirtyrate.h | 28 ++++++++++++++++++++++++++++ migration/meson.build | 2 +- 3 files changed, 67 insertions(+), 1 deletion(-) create mode 100644 migration/dirtyrate.c create mode 100644 migration/dirtyrate.h diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c new file mode 100644 index 0000000000..bf7fd24e75 --- /dev/null +++ b/migration/dirtyrate.c @@ -0,0 +1,38 @@ +/* + * Dirtyrate implement code + * + * Copyright (c) 2020 HUAWEI TECHNOLOGIES CO.,LTD. + * + * Authors: + * Chuan Zheng + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "cpu.h" +#include "qemu/config-file.h" +#include "exec/memory.h" +#include "exec/ramblock.h" +#include "exec/target_page.h" +#include "qemu/rcu_queue.h" +#include "qapi/qapi-commands-migration.h" +#include "migration.h" +#include "dirtyrate.h" + +static void calculate_dirtyrate(struct DirtyRateConfig config) +{ + /* todo */ + return; +} + +void *get_dirtyrate_thread(void *arg) +{ + struct DirtyRateConfig config = *(struct DirtyRateConfig *)arg; + + calculate_dirtyrate(config); + + return NULL; +} diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h new file mode 100644 index 0000000000..84ab9409ac --- /dev/null +++ b/migration/dirtyrate.h @@ -0,0 +1,28 @@ +/* + * Dirtyrate common functions + * + * Copyright (c) 2020 HUAWEI TECHNOLOGIES CO., LTD. + * + * Authors: + * Chuan Zheng + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef QEMU_MIGRATION_DIRTYRATE_H +#define QEMU_MIGRATION_DIRTYRATE_H + +/* + * Sample 512 pages per GB as default. + * TODO: Make it configurable. + */ +#define DIRTYRATE_DEFAULT_SAMPLE_PAGES 512 + +struct DirtyRateConfig { + uint64_t sample_pages_per_gigabytes; /* sample pages per GB */ + int64_t sample_period_seconds; /* time duration between two sampling */ +}; + +void *get_dirtyrate_thread(void *arg); +#endif diff --git a/migration/meson.build b/migration/meson.build index ac8ff1419f..b5b71c8060 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -37,4 +37,4 @@ softmmu_ss.add(when: ['CONFIG_RDMA', rdma], if_true: files('rdma.c')) softmmu_ss.add(when: 'CONFIG_LIVE_BLOCK_MIGRATION', if_true: files('block.c')) softmmu_ss.add(when: 'CONFIG_ZSTD', if_true: [files('multifd-zstd.c'), zstd]) -specific_ss.add(when: 'CONFIG_SOFTMMU', if_true: files('ram.c')) +specific_ss.add(when: 'CONFIG_SOFTMMU', if_true: files('dirtyrate.c', 'ram.c')) From patchwork Fri Sep 25 12:06:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 304428 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4987C4363D for ; Fri, 25 Sep 2020 12:09:12 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4F1422083B for ; Fri, 25 Sep 2020 12:09:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QMN0oJXh" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4F1422083B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:45180 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmXH-0004vX-8g for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:09:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38104) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmVW-0002Bm-9T for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:35726) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmVT-0004DQ-To for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:21 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035639; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KwoC26LP5CpaPJLEShkXwNEdUVef6paLbONKlxitJ44=; b=QMN0oJXhiYCb1kueCFVJbgzr+MFBxUShwoQldfTgt7mbIKKq/UzDJUkSaDaTTaowurlFmV T3qF+Ds3x97AzbQga93Sk38CQ934jCuJGV3VKuz0riFJsau4h+scOQjHcaJ66COtfTu865 tPohydDKEHzu3AJQO1UueLel+8uFOX0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-255-TSRwodWsMc-XCZJs2rr5Vw-1; Fri, 25 Sep 2020 08:07:17 -0400 X-MC-Unique: TSRwodWsMc-XCZJs2rr5Vw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B301B10BBECB; Fri, 25 Sep 2020 12:07:15 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6563E78437; Fri, 25 Sep 2020 12:07:13 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 04/26] migration/dirtyrate: add DirtyRateStatus to denote calculation status Date: Fri, 25 Sep 2020 13:06:33 +0100 Message-Id: <20200925120655.295142-5-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 02:48:20 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng add DirtyRateStatus to denote calculating status. Signed-off-by: Chuan Zheng Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Li Qiang Message-Id: <1600237327-33618-3-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert atomic name fixup --- migration/dirtyrate.c | 26 ++++++++++++++++++++++++++ qapi/migration.json | 17 +++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index bf7fd24e75..3edf000f45 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -22,6 +22,19 @@ #include "migration.h" #include "dirtyrate.h" +static int CalculatingState = DIRTY_RATE_STATUS_UNSTARTED; + +static int dirtyrate_set_state(int *state, int old_state, int new_state) +{ + assert(new_state < DIRTY_RATE_STATUS__MAX); + if (qatomic_cmpxchg(state, old_state, new_state) == old_state) { + return 0; + } else { + return -1; + } +} + + static void calculate_dirtyrate(struct DirtyRateConfig config) { /* todo */ @@ -31,8 +44,21 @@ static void calculate_dirtyrate(struct DirtyRateConfig config) void *get_dirtyrate_thread(void *arg) { struct DirtyRateConfig config = *(struct DirtyRateConfig *)arg; + int ret; + + ret = dirtyrate_set_state(&CalculatingState, DIRTY_RATE_STATUS_UNSTARTED, + DIRTY_RATE_STATUS_MEASURING); + if (ret == -1) { + error_report("change dirtyrate state failed."); + return NULL; + } calculate_dirtyrate(config); + ret = dirtyrate_set_state(&CalculatingState, DIRTY_RATE_STATUS_MEASURING, + DIRTY_RATE_STATUS_MEASURED); + if (ret == -1) { + error_report("change dirtyrate state failed."); + } return NULL; } diff --git a/qapi/migration.json b/qapi/migration.json index 675f70bb67..76a59b4f92 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1720,3 +1720,20 @@ ## { 'event': 'UNPLUG_PRIMARY', 'data': { 'device-id': 'str' } } + +## +# @DirtyRateStatus: +# +# An enumeration of dirtyrate status. +# +# @unstarted: the dirtyrate thread has not been started. +# +# @measuring: the dirtyrate thread is measuring. +# +# @measured: the dirtyrate thread has measured and results are available. +# +# Since: 5.2 +# +## +{ 'enum': 'DirtyRateStatus', + 'data': [ 'unstarted', 'measuring', 'measured'] } From patchwork Fri Sep 25 12:06:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 272748 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66D1CC4363D for ; Fri, 25 Sep 2020 12:14:07 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DBBF62083B for ; Fri, 25 Sep 2020 12:14:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IZSmJuLS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DBBF62083B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:33318 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmc1-0003Lz-TF for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:14:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38134) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmVe-0002G8-Im for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:51615) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmVc-0004Dr-8m for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:30 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035646; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5RtNjHNs3hEYTQX9+njr/qBDA45DXZrosh4HKnfGnHo=; b=IZSmJuLS2DsLKZX0rT49klLiP5HEuGnh+C7Nt2v5mA1d7f1xmSOLcOq02SdA/tXCRvFetf ofQZOEE6HkMzJiHp/EQk9C1ENCw+zYAkC4AtOXlyLCR9hMk4mejweKFh3xqAER3BraKXWc AIo0atWm6n9rUaX+HyeV0G5Wj845h7Q= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-433-rJDnEYYoN32iW2IG99GHQQ-1; Fri, 25 Sep 2020 08:07:22 -0400 X-MC-Unique: rJDnEYYoN32iW2IG99GHQQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 17ACF10066FC; Fri, 25 Sep 2020 12:07:21 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0AC5773692; Fri, 25 Sep 2020 12:07:15 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 05/26] migration/dirtyrate: Add RamblockDirtyInfo to store sampled page info Date: Fri, 25 Sep 2020 13:06:34 +0100 Message-Id: <20200925120655.295142-6-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 02:48:20 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng Add RamblockDirtyInfo to store sampled page info of each ramblock. Signed-off-by: Chuan Zheng Reviewed-by: Dr. David Alan Gilbert Reviewed-by: David Edmondson Reviewed-by: Li Qiang Message-Id: <1600237327-33618-4-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/dirtyrate.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h index 84ab9409ac..8707df852d 100644 --- a/migration/dirtyrate.h +++ b/migration/dirtyrate.h @@ -19,10 +19,28 @@ */ #define DIRTYRATE_DEFAULT_SAMPLE_PAGES 512 +/* + * Record ramblock idstr + */ +#define RAMBLOCK_INFO_MAX_LEN 256 + struct DirtyRateConfig { uint64_t sample_pages_per_gigabytes; /* sample pages per GB */ int64_t sample_period_seconds; /* time duration between two sampling */ }; +/* + * Store dirtypage info for each ramblock. + */ +struct RamblockDirtyInfo { + char idstr[RAMBLOCK_INFO_MAX_LEN]; /* idstr for each ramblock */ + uint8_t *ramblock_addr; /* base address of ramblock we measure */ + uint64_t ramblock_pages; /* ramblock size in TARGET_PAGE_SIZE */ + uint64_t *sample_page_vfn; /* relative offset address for sampled page */ + uint64_t sample_pages_count; /* count of sampled pages */ + uint64_t sample_dirty_count; /* count of dirty pages we measure */ + uint32_t *hash_result; /* array of hash result for sampled pages */ +}; + void *get_dirtyrate_thread(void *arg); #endif From patchwork Fri Sep 25 12:06:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 272750 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A26EC4363D for ; Fri, 25 Sep 2020 12:11:14 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0E58D2083B for ; Fri, 25 Sep 2020 12:11:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="MFCnccRG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0E58D2083B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:52482 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmZE-00082O-W6 for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:11:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38144) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmVg-0002J9-4v for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:22245) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmVe-0004Dy-7p for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:31 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035649; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c+p4cUwbZBePSB0cV+zQje3NVEUYNiC8OKPJnj/bUL4=; b=MFCnccRG2jI83Lwv7d2nDxFqMdslC2hG4tdoRE6wSl64Elo6YNXXkUnYKxocLQV6mxqp5J FMGy+jFpzqFQwWIbn/XXpEwgCZ6kMC4yI2QVlOwgKuUW52I7Jasd9QX5JJLIcy520deU8+ J+dYN+jhYKSquO/QzGFHPx4zji3Y+7Y= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-562-UpBs0oYDPgS2dZujpYroGw-1; Fri, 25 Sep 2020 08:07:27 -0400 X-MC-Unique: UpBs0oYDPgS2dZujpYroGw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 701C6186DD3D; Fri, 25 Sep 2020 12:07:26 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6316A73692; Fri, 25 Sep 2020 12:07:21 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 06/26] migration/dirtyrate: Add dirtyrate statistics series functions Date: Fri, 25 Sep 2020 13:06:35 +0100 Message-Id: <20200925120655.295142-7-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng Add dirtyrate statistics functions to record/update dirtyrate info. Signed-off-by: Chuan Zheng Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Li Qiang Message-Id: <1600237327-33618-5-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/dirtyrate.c | 32 ++++++++++++++++++++++++++++++++ migration/dirtyrate.h | 12 ++++++++++++ 2 files changed, 44 insertions(+) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 3edf000f45..94c4e173bc 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -23,6 +23,7 @@ #include "dirtyrate.h" static int CalculatingState = DIRTY_RATE_STATUS_UNSTARTED; +static struct DirtyRateStat DirtyStat; static int dirtyrate_set_state(int *state, int old_state, int new_state) { @@ -34,6 +35,37 @@ static int dirtyrate_set_state(int *state, int old_state, int new_state) } } +static void reset_dirtyrate_stat(void) +{ + DirtyStat.total_dirty_samples = 0; + DirtyStat.total_sample_count = 0; + DirtyStat.total_block_mem_MB = 0; + DirtyStat.dirty_rate = -1; + DirtyStat.start_time = 0; + DirtyStat.calc_time = 0; +} + +static void update_dirtyrate_stat(struct RamblockDirtyInfo *info) +{ + DirtyStat.total_dirty_samples += info->sample_dirty_count; + DirtyStat.total_sample_count += info->sample_pages_count; + /* size of total pages in MB */ + DirtyStat.total_block_mem_MB += (info->ramblock_pages * + TARGET_PAGE_SIZE) >> 20; +} + +static void update_dirtyrate(uint64_t msec) +{ + uint64_t dirtyrate; + uint64_t total_dirty_samples = DirtyStat.total_dirty_samples; + uint64_t total_sample_count = DirtyStat.total_sample_count; + uint64_t total_block_mem_MB = DirtyStat.total_block_mem_MB; + + dirtyrate = total_dirty_samples * total_block_mem_MB * + 1000 / (total_sample_count * msec); + + DirtyStat.dirty_rate = dirtyrate; +} static void calculate_dirtyrate(struct DirtyRateConfig config) { diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h index 8707df852d..312debca6f 100644 --- a/migration/dirtyrate.h +++ b/migration/dirtyrate.h @@ -42,5 +42,17 @@ struct RamblockDirtyInfo { uint32_t *hash_result; /* array of hash result for sampled pages */ }; +/* + * Store calculation statistics for each measure. + */ +struct DirtyRateStat { + uint64_t total_dirty_samples; /* total dirty sampled page */ + uint64_t total_sample_count; /* total sampled pages */ + uint64_t total_block_mem_MB; /* size of total sampled pages in MB */ + int64_t dirty_rate; /* dirty rate in MB/s */ + int64_t start_time; /* calculation start time in units of second */ + int64_t calc_time; /* time duration of two sampling in units of second */ +}; + void *get_dirtyrate_thread(void *arg); #endif From patchwork Fri Sep 25 12:06:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 304424 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0DF2C4363D for ; Fri, 25 Sep 2020 12:14:10 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 453C52083B for ; Fri, 25 Sep 2020 12:14:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ojqhya/e" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 453C52083B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:33656 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmc5-0003UL-6N for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:14:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38188) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmVn-0002Ve-0x for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:36349) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmVl-0004EU-6L for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:38 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035655; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pTjngOD6M5Zs+XGR87KKIEPcXJdlyB8CuV33zY5x0Wo=; b=Ojqhya/er/l0Gzxv89Api4uMbYU6wFW/A9RcFr7pf/KWKHEniwh8H9tcXfWUPtgDwFrewl o8q4dcGEKmARdx1eZf/RaHSrYMBukkZYaW28xSA3Z/BAMre1GLzNy8ZwkwiMl0hILtUZ6C J+y9VbXIgc3487PXXKHqw8tK7xkFBm4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-193-5Vlo2-GKNRiYjYpKipgvPw-1; Fri, 25 Sep 2020 08:07:31 -0400 X-MC-Unique: 5Vlo2-GKNRiYjYpKipgvPw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 113412FD03; Fri, 25 Sep 2020 12:07:30 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id BBE9073692; Fri, 25 Sep 2020 12:07:26 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 07/26] migration/dirtyrate: move RAMBLOCK_FOREACH_MIGRATABLE into ram.h Date: Fri, 25 Sep 2020 13:06:36 +0100 Message-Id: <20200925120655.295142-8-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 02:48:20 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng RAMBLOCK_FOREACH_MIGRATABLE is need in dirtyrate measure, move the existing definition up into migration/ram.h Signed-off-by: Chuan Zheng Reviewed-by: Dr. David Alan Gilbert Reviewed-by: David Edmondson Reviewed-by: Li Qiang Message-Id: <1600237327-33618-6-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/dirtyrate.c | 1 + migration/ram.c | 11 +---------- migration/ram.h | 10 ++++++++++ 3 files changed, 12 insertions(+), 10 deletions(-) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 94c4e173bc..10315f7a63 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -20,6 +20,7 @@ #include "qemu/rcu_queue.h" #include "qapi/qapi-commands-migration.h" #include "migration.h" +#include "ram.h" #include "dirtyrate.h" static int CalculatingState = DIRTY_RATE_STATUS_UNSTARTED; diff --git a/migration/ram.c b/migration/ram.c index c5f36aeae5..433489d633 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -158,21 +158,12 @@ out: return ret; } -static bool ramblock_is_ignored(RAMBlock *block) +bool ramblock_is_ignored(RAMBlock *block) { return !qemu_ram_is_migratable(block) || (migrate_ignore_shared() && qemu_ram_is_shared(block)); } -/* Should be holding either ram_list.mutex, or the RCU lock. */ -#define RAMBLOCK_FOREACH_NOT_IGNORED(block) \ - INTERNAL_RAMBLOCK_FOREACH(block) \ - if (ramblock_is_ignored(block)) {} else - -#define RAMBLOCK_FOREACH_MIGRATABLE(block) \ - INTERNAL_RAMBLOCK_FOREACH(block) \ - if (!qemu_ram_is_migratable(block)) {} else - #undef RAMBLOCK_FOREACH int foreach_not_ignored_block(RAMBlockIterFunc func, void *opaque) diff --git a/migration/ram.h b/migration/ram.h index 2eeaacfa13..011e85414e 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -37,6 +37,16 @@ extern MigrationStats ram_counters; extern XBZRLECacheStats xbzrle_counters; extern CompressionStats compression_counters; +bool ramblock_is_ignored(RAMBlock *block); +/* Should be holding either ram_list.mutex, or the RCU lock. */ +#define RAMBLOCK_FOREACH_NOT_IGNORED(block) \ + INTERNAL_RAMBLOCK_FOREACH(block) \ + if (ramblock_is_ignored(block)) {} else + +#define RAMBLOCK_FOREACH_MIGRATABLE(block) \ + INTERNAL_RAMBLOCK_FOREACH(block) \ + if (!qemu_ram_is_migratable(block)) {} else + int xbzrle_cache_resize(int64_t new_size, Error **errp); uint64_t ram_bytes_remaining(void); uint64_t ram_bytes_total(void); From patchwork Fri Sep 25 12:06:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 304425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0140C4363D for ; Fri, 25 Sep 2020 12:14:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 555F22083B for ; Fri, 25 Sep 2020 12:14:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KFMcCCwN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 555F22083B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:60922 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmbv-00036g-80 for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:13:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38208) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmVo-0002Yu-Sb for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:53170) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmVm-0004Eh-Sm for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:40 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035658; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rni6bUmwD3P+YAqyxuw4XH0MxQULNnL6YOVizGoMRjM=; b=KFMcCCwNFkubn6JeLD9UnW2uenHmndLbAopJseXscPRieljg6sM0RaU8Qj9U+nUFjZJpIx 34ZmbyfyBTT7Amits13nSii0pwaEYCA/xirZYrH5r9+arZPhxfZMbWuH7SZe6n7OOiz8QF BRGEOPrzkbNPBXBfCOfyyVqO0EGzsp4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-147-ynGf_mJAMk2j_w2Q9MUpLA-1; Fri, 25 Sep 2020 08:07:34 -0400 X-MC-Unique: ynGf_mJAMk2j_w2Q9MUpLA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A903E81C463; Fri, 25 Sep 2020 12:07:32 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5C25273695; Fri, 25 Sep 2020 12:07:30 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 08/26] migration/dirtyrate: Record hash results for each sampled page Date: Fri, 25 Sep 2020 13:06:37 +0100 Message-Id: <20200925120655.295142-9-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 02:48:20 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng Record hash results for each sampled page, crc32 is taken to calculate hash results for each sampled length in TARGET_PAGE_SIZE. Signed-off-by: Chuan Zheng Signed-off-by: YanYing Zhuang Reviewed-by: David Edmondson Reviewed-by: Li Qiang Message-Id: <1600237327-33618-7-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/dirtyrate.c | 109 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 109 insertions(+) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 10315f7a63..856d26dfe7 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -10,6 +10,7 @@ * See the COPYING file in the top-level directory. */ +#include #include "qemu/osdep.h" #include "qapi/error.h" #include "cpu.h" @@ -68,6 +69,114 @@ static void update_dirtyrate(uint64_t msec) DirtyStat.dirty_rate = dirtyrate; } +/* + * get hash result for the sampled memory with length of TARGET_PAGE_SIZE + * in ramblock, which starts from ramblock base address. + */ +static uint32_t get_ramblock_vfn_hash(struct RamblockDirtyInfo *info, + uint64_t vfn) +{ + uint32_t crc; + + crc = crc32(0, (info->ramblock_addr + + vfn * TARGET_PAGE_SIZE), TARGET_PAGE_SIZE); + + return crc; +} + +static bool save_ramblock_hash(struct RamblockDirtyInfo *info) +{ + unsigned int sample_pages_count; + int i; + GRand *rand; + + sample_pages_count = info->sample_pages_count; + + /* ramblock size less than one page, return success to skip this ramblock */ + if (unlikely(info->ramblock_pages == 0 || sample_pages_count == 0)) { + return true; + } + + info->hash_result = g_try_malloc0_n(sample_pages_count, + sizeof(uint32_t)); + if (!info->hash_result) { + return false; + } + + info->sample_page_vfn = g_try_malloc0_n(sample_pages_count, + sizeof(uint64_t)); + if (!info->sample_page_vfn) { + g_free(info->hash_result); + return false; + } + + rand = g_rand_new(); + for (i = 0; i < sample_pages_count; i++) { + info->sample_page_vfn[i] = g_rand_int_range(rand, 0, + info->ramblock_pages - 1); + info->hash_result[i] = get_ramblock_vfn_hash(info, + info->sample_page_vfn[i]); + } + g_rand_free(rand); + + return true; +} + +static void get_ramblock_dirty_info(RAMBlock *block, + struct RamblockDirtyInfo *info, + struct DirtyRateConfig *config) +{ + uint64_t sample_pages_per_gigabytes = config->sample_pages_per_gigabytes; + + /* Right shift 30 bits to calc ramblock size in GB */ + info->sample_pages_count = (qemu_ram_get_used_length(block) * + sample_pages_per_gigabytes) >> 30; + /* Right shift TARGET_PAGE_BITS to calc page count */ + info->ramblock_pages = qemu_ram_get_used_length(block) >> + TARGET_PAGE_BITS; + info->ramblock_addr = qemu_ram_get_host_addr(block); + strcpy(info->idstr, qemu_ram_get_idstr(block)); +} + +static bool record_ramblock_hash_info(struct RamblockDirtyInfo **block_dinfo, + struct DirtyRateConfig config, + int *block_count) +{ + struct RamblockDirtyInfo *info = NULL; + struct RamblockDirtyInfo *dinfo = NULL; + RAMBlock *block = NULL; + int total_count = 0; + int index = 0; + bool ret = false; + + RAMBLOCK_FOREACH_MIGRATABLE(block) { + total_count++; + } + + dinfo = g_try_malloc0_n(total_count, sizeof(struct RamblockDirtyInfo)); + if (dinfo == NULL) { + goto out; + } + + RAMBLOCK_FOREACH_MIGRATABLE(block) { + if (index >= total_count) { + break; + } + info = &dinfo[index]; + get_ramblock_dirty_info(block, info, &config); + if (!save_ramblock_hash(info)) { + goto out; + } + index++; + } + ret = true; + +out: + *block_count = index; + *block_dinfo = dinfo; + return ret; +} + static void calculate_dirtyrate(struct DirtyRateConfig config) { /* todo */ From patchwork Fri Sep 25 12:06:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 304429 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98A17C4363D for ; Fri, 25 Sep 2020 12:08:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2532A21D7A for ; Fri, 25 Sep 2020 12:08:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LnM2VhBq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2532A21D7A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:43420 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmWy-0004D6-Qe for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:08:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38212) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmVp-0002a8-En for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:31669) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmVn-0004Eq-Mu for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:41 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035659; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mn7LbChv+ICngI0HeE5AR67B3cUvSdaikYAC+lGnkE0=; b=LnM2VhBqaQObL3w2UX1XYNuUTjR3VYHhz6/TOt8a84wH1GrF3nBOx65/V8LhAGndfTK2Zh FUdJw01kIRrxCRcL6ypEVB4BdDjdcCLRNS0M+TCd5V4JLxSg6+pQhT8821PlKc4pmaOCi3 EIUUtpR/UdmDYdlvDeHNsE81uPtEC8g= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-424-ksj3urjSM_aSpnmzRi7iMg-1; Fri, 25 Sep 2020 08:07:36 -0400 X-MC-Unique: ksj3urjSM_aSpnmzRi7iMg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 501E18030CC; Fri, 25 Sep 2020 12:07:35 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 012F873692; Fri, 25 Sep 2020 12:07:32 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 09/26] migration/dirtyrate: Compare page hash results for recorded sampled page Date: Fri, 25 Sep 2020 13:06:38 +0100 Message-Id: <20200925120655.295142-10-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng Compare page hash results for recorded sampled page. Signed-off-by: Chuan Zheng Signed-off-by: YanYing Zhuang Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Li Qiang Message-Id: <1600237327-33618-8-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/dirtyrate.c | 63 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 856d26dfe7..0f78aa3af7 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -177,6 +177,69 @@ out: return ret; } +static void calc_page_dirty_rate(struct RamblockDirtyInfo *info) +{ + uint32_t crc; + int i; + + for (i = 0; i < info->sample_pages_count; i++) { + crc = get_ramblock_vfn_hash(info, info->sample_page_vfn[i]); + if (crc != info->hash_result[i]) { + info->sample_dirty_count++; + } + } +} + +static struct RamblockDirtyInfo * +find_block_matched(RAMBlock *block, int count, + struct RamblockDirtyInfo *infos) +{ + int i; + struct RamblockDirtyInfo *matched; + + for (i = 0; i < count; i++) { + if (!strcmp(infos[i].idstr, qemu_ram_get_idstr(block))) { + break; + } + } + + if (i == count) { + return NULL; + } + + if (infos[i].ramblock_addr != qemu_ram_get_host_addr(block) || + infos[i].ramblock_pages != + (qemu_ram_get_used_length(block) >> TARGET_PAGE_BITS)) { + return NULL; + } + + matched = &infos[i]; + + return matched; +} + +static bool compare_page_hash_info(struct RamblockDirtyInfo *info, + int block_count) +{ + struct RamblockDirtyInfo *block_dinfo = NULL; + RAMBlock *block = NULL; + + RAMBLOCK_FOREACH_MIGRATABLE(block) { + block_dinfo = find_block_matched(block, block_count, info); + if (block_dinfo == NULL) { + continue; + } + calc_page_dirty_rate(block_dinfo); + update_dirtyrate_stat(block_dinfo); + } + + if (DirtyStat.total_sample_count == 0) { + return false; + } + + return true; +} + static void calculate_dirtyrate(struct DirtyRateConfig config) { /* todo */ From patchwork Fri Sep 25 12:06:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 272746 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE25DC4363D for ; Fri, 25 Sep 2020 12:17:30 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 64AE72086A for ; Fri, 25 Sep 2020 12:17:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NmpDpMHz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 64AE72086A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:41720 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmfI-0006wW-H9 for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:17:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38240) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmVx-0002q5-1b for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:43714) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmVv-0004FO-4B for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:48 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035666; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZYwR8v70/BIXXGAJip2JIOAQsBVnpybo/HOqyw1ujpA=; b=NmpDpMHzk/n5BfCNMPoUNOhG+bVMoa1uPlexziq6Wsr84rHhMHgKKNl3BPsp2tRHx/5ydF /L1MT+j8shh4RYOVm7ZpKFADe8gahIKsrPbprjTS415pltbj5ZqAFK54E1wFnPVz73vibl /BtrLno/nGd22CyOwDaKTSy5rQ8kNg8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-94-GiOYJvevNtaY-PTt_Dqwkg-1; Fri, 25 Sep 2020 08:07:42 -0400 X-MC-Unique: GiOYJvevNtaY-PTt_Dqwkg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A9D2410BBECB; Fri, 25 Sep 2020 12:07:40 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9BCD973692; Fri, 25 Sep 2020 12:07:35 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 10/26] migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE Date: Fri, 25 Sep 2020 13:06:39 +0100 Message-Id: <20200925120655.295142-11-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng In order to sample real RAM, skip ramblock with size below MIN_RAMBLOCK_SIZE which is set as 128M. Signed-off-by: Chuan Zheng Reviewed-by: David Edmondson Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Li Qiang Message-Id: <1600237327-33618-9-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/dirtyrate.c | 21 +++++++++++++++++++++ migration/dirtyrate.h | 5 +++++ 2 files changed, 26 insertions(+) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 0f78aa3af7..ee47e2bc27 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -138,6 +138,18 @@ static void get_ramblock_dirty_info(RAMBlock *block, strcpy(info->idstr, qemu_ram_get_idstr(block)); } +static bool skip_sample_ramblock(RAMBlock *block) +{ + /* + * Sample only blocks larger than MIN_RAMBLOCK_SIZE. + */ + if (qemu_ram_get_used_length(block) < (MIN_RAMBLOCK_SIZE << 10)) { + return true; + } + + return false; +} + static bool record_ramblock_hash_info(struct RamblockDirtyInfo **block_dinfo, struct DirtyRateConfig config, int *block_count) @@ -150,6 +162,9 @@ static bool record_ramblock_hash_info(struct RamblockDirtyInfo **block_dinfo, bool ret = false; RAMBLOCK_FOREACH_MIGRATABLE(block) { + if (skip_sample_ramblock(block)) { + continue; + } total_count++; } @@ -159,6 +174,9 @@ static bool record_ramblock_hash_info(struct RamblockDirtyInfo **block_dinfo, } RAMBLOCK_FOREACH_MIGRATABLE(block) { + if (skip_sample_ramblock(block)) { + continue; + } if (index >= total_count) { break; } @@ -225,6 +243,9 @@ static bool compare_page_hash_info(struct RamblockDirtyInfo *info, RAMBlock *block = NULL; RAMBLOCK_FOREACH_MIGRATABLE(block) { + if (skip_sample_ramblock(block)) { + continue; + } block_dinfo = find_block_matched(block, block_count, info); if (block_dinfo == NULL) { continue; diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h index 312debca6f..be5b8ec2b1 100644 --- a/migration/dirtyrate.h +++ b/migration/dirtyrate.h @@ -24,6 +24,11 @@ */ #define RAMBLOCK_INFO_MAX_LEN 256 +/* + * Minimum RAMBlock size to sample, in megabytes. + */ +#define MIN_RAMBLOCK_SIZE 128 + struct DirtyRateConfig { uint64_t sample_pages_per_gigabytes; /* sample pages per GB */ int64_t sample_period_seconds; /* time duration between two sampling */ From patchwork Fri Sep 25 12:06:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 304427 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F364FC4363D for ; Fri, 25 Sep 2020 12:11:06 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6A3DF2083B for ; Fri, 25 Sep 2020 12:11:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iJvuxiA9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6A3DF2083B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:51782 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmZ7-0007i2-93 for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:11:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38254) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmW0-0002xz-C8 for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:52 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:28737) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmVy-0004Ff-DS for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:52 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035669; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bIx+v7hX+Kc77yn6My0x4/VOcV0NpTk52duItW9+GBA=; b=iJvuxiA9bA8hjgAE3Pfvlvunr8+Uf1gD6b1EKF0Heom8ZzW+yqdpi/TkE+Cq3yvf/PSNme 79h988C/BX1TaAZr0byEOpANRPyHXDLnxUBT4aKIvCcV3cxVKn5XErcASZFgv06K0cn8yz QbsynxPYs8WovlbgUS9hnnEYQpmv4ic= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-296-sHAIuk48NxKTyZrsY18tGA-1; Fri, 25 Sep 2020 08:07:47 -0400 X-MC-Unique: sHAIuk48NxKTyZrsY18tGA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EEB4B871805; Fri, 25 Sep 2020 12:07:45 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 008AE73693; Fri, 25 Sep 2020 12:07:40 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 11/26] migration/dirtyrate: Implement set_sample_page_period() and is_sample_period_valid() Date: Fri, 25 Sep 2020 13:06:40 +0100 Message-Id: <20200925120655.295142-12-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng Implement is_sample_period_valid() to check if the sample period is vaild and do set_sample_page_period() to sleep specific time between sample actions. Signed-off-by: Chuan Zheng Reviewed-by: Dr. David Alan Gilbert Reviewed-by: David Edmondson Reviewed-by: Li Qiang Message-Id: <1600237327-33618-10-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/dirtyrate.c | 24 ++++++++++++++++++++++++ migration/dirtyrate.h | 6 ++++++ 2 files changed, 30 insertions(+) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index ee47e2bc27..20216c5f09 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -27,6 +27,30 @@ static int CalculatingState = DIRTY_RATE_STATUS_UNSTARTED; static struct DirtyRateStat DirtyStat; +static int64_t set_sample_page_period(int64_t msec, int64_t initial_time) +{ + int64_t current_time; + + current_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME); + if ((current_time - initial_time) >= msec) { + msec = current_time - initial_time; + } else { + g_usleep((msec + initial_time - current_time) * 1000); + } + + return msec; +} + +static bool is_sample_period_valid(int64_t sec) +{ + if (sec < MIN_FETCH_DIRTYRATE_TIME_SEC || + sec > MAX_FETCH_DIRTYRATE_TIME_SEC) { + return false; + } + + return true; +} + static int dirtyrate_set_state(int *state, int old_state, int new_state) { assert(new_state < DIRTY_RATE_STATUS__MAX); diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h index be5b8ec2b1..6ec429534d 100644 --- a/migration/dirtyrate.h +++ b/migration/dirtyrate.h @@ -29,6 +29,12 @@ */ #define MIN_RAMBLOCK_SIZE 128 +/* + * Take 1s as minimum time for calculation duration + */ +#define MIN_FETCH_DIRTYRATE_TIME_SEC 1 +#define MAX_FETCH_DIRTYRATE_TIME_SEC 60 + struct DirtyRateConfig { uint64_t sample_pages_per_gigabytes; /* sample pages per GB */ int64_t sample_period_seconds; /* time duration between two sampling */ From patchwork Fri Sep 25 12:06:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 272744 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E43FC4741F for ; Fri, 25 Sep 2020 12:21:04 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B88F022B2D for ; Fri, 25 Sep 2020 12:21:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AhBjgfo7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B88F022B2D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:50188 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmik-0002Gp-I5 for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:21:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38274) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmW2-00032R-Hn for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:43775) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmW0-0004Fk-LU for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:54 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035671; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sYIPbhkmgu94Y8R7pvVnSSWLdcr7eX5vaM6ifZxTJmY=; b=AhBjgfo7PbOqFpQdiwBWfvWYultZhNf8iZA3XG4yURkGUPspuxJ3tGY6zzp9aMTpDM5dOY Nkd72+pxbGSLmjEW02vq2ysTvwpRpbPdqJuS0OAemKZMWyh52+o7Ig+hR0n2VdUW7WLq/o 0L30Qf4kO9hBbsJjqpvpY24qyjhk2XA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-490-1ncJuC8pMw2yiIX8SSf8KQ-1; Fri, 25 Sep 2020 08:07:50 -0400 X-MC-Unique: 1ncJuC8pMw2yiIX8SSf8KQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9E5B810BBECD; Fri, 25 Sep 2020 12:07:48 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 45C3C73692; Fri, 25 Sep 2020 12:07:46 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 12/26] migration/dirtyrate: Implement calculate_dirtyrate() function Date: Fri, 25 Sep 2020 13:06:41 +0100 Message-Id: <20200925120655.295142-13-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng Implement calculate_dirtyrate() function. Signed-off-by: Chuan Zheng Signed-off-by: YanYing Zhuang Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Li Qiang Message-Id: <1600237327-33618-11-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/dirtyrate.c | 45 +++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 43 insertions(+), 2 deletions(-) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 20216c5f09..419b0e30ba 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -162,6 +162,21 @@ static void get_ramblock_dirty_info(RAMBlock *block, strcpy(info->idstr, qemu_ram_get_idstr(block)); } +static void free_ramblock_dirty_info(struct RamblockDirtyInfo *infos, int count) +{ + int i; + + if (!infos) { + return; + } + + for (i = 0; i < count; i++) { + g_free(infos[i].sample_page_vfn); + g_free(infos[i].hash_result); + } + g_free(infos); +} + static bool skip_sample_ramblock(RAMBlock *block) { /* @@ -287,8 +302,34 @@ static bool compare_page_hash_info(struct RamblockDirtyInfo *info, static void calculate_dirtyrate(struct DirtyRateConfig config) { - /* todo */ - return; + struct RamblockDirtyInfo *block_dinfo = NULL; + int block_count = 0; + int64_t msec = 0; + int64_t initial_time; + + rcu_register_thread(); + reset_dirtyrate_stat(); + rcu_read_lock(); + initial_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME); + if (!record_ramblock_hash_info(&block_dinfo, config, &block_count)) { + goto out; + } + rcu_read_unlock(); + + msec = config.sample_period_seconds * 1000; + msec = set_sample_page_period(msec, initial_time); + + rcu_read_lock(); + if (!compare_page_hash_info(block_dinfo, block_count)) { + goto out; + } + + update_dirtyrate(msec); + +out: + rcu_read_unlock(); + free_ramblock_dirty_info(block_dinfo, block_count); + rcu_unregister_thread(); } void *get_dirtyrate_thread(void *arg) From patchwork Fri Sep 25 12:06:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 304423 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B3FEC4363D for ; Fri, 25 Sep 2020 12:16:50 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9C15621D7A for ; Fri, 25 Sep 2020 12:16:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QDeWKlcR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9C15621D7A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:41142 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmee-0006iG-EY for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:16:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38296) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmW6-0003Ao-7a for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:20525) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmW4-0004G7-2p for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:07:57 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035675; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+y4m8qUcZG9LZ6I4e9KlcqtI4wRE4aoiam6lpucP0c8=; b=QDeWKlcRi2nLJu2OnsEuKtSKkC/nm7G/TImKhdEn5oWNvnHN5Oc+Tvr3xNgTTy/bw+xhr3 D75OyT3YUB1t0h7K3dMcR0HozKXBN+fR2gUjZ907xly8IHSGuTi3BqEhhYq7kRUD+AoXyf oNk6eLeHpWjh0akuA9pGNgX0/xgjWGE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-257-65kStykoNQGQXuz2eDe1yQ-1; Fri, 25 Sep 2020 08:07:52 -0400 X-MC-Unique: 65kStykoNQGQXuz2eDe1yQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3EB9F1007482; Fri, 25 Sep 2020 12:07:51 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id E993473695; Fri, 25 Sep 2020 12:07:48 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 13/26] migration/dirtyrate: Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function Date: Fri, 25 Sep 2020 13:06:42 +0100 Message-Id: <20200925120655.295142-14-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 02:48:20 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function which could be called Signed-off-by: Chuan Zheng Message-Id: <1600237327-33618-12-git-send-email-zhengchuan@huawei.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert atomic function fixup Wording fixup in migration.json based on Eric's review --- migration/dirtyrate.c | 62 +++++++++++++++++++++++++++++++++++++++++++ qapi/migration.json | 50 ++++++++++++++++++++++++++++++++++ 2 files changed, 112 insertions(+) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 419b0e30ba..2ffd88d48e 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -61,6 +61,24 @@ static int dirtyrate_set_state(int *state, int old_state, int new_state) } } +static struct DirtyRateInfo *query_dirty_rate_info(void) +{ + int64_t dirty_rate = DirtyStat.dirty_rate; + struct DirtyRateInfo *info = g_malloc0(sizeof(DirtyRateInfo)); + + if (qatomic_read(&CalculatingState) == DIRTY_RATE_STATUS_MEASURED) { + info->dirty_rate = dirty_rate; + } else { + info->dirty_rate = -1; + } + + info->status = CalculatingState; + info->start_time = DirtyStat.start_time; + info->calc_time = DirtyStat.calc_time; + + return info; +} + static void reset_dirtyrate_stat(void) { DirtyStat.total_dirty_samples = 0; @@ -318,6 +336,8 @@ static void calculate_dirtyrate(struct DirtyRateConfig config) msec = config.sample_period_seconds * 1000; msec = set_sample_page_period(msec, initial_time); + DirtyStat.start_time = initial_time / 1000; + DirtyStat.calc_time = msec / 1000; rcu_read_lock(); if (!compare_page_hash_info(block_dinfo, block_count)) { @@ -353,3 +373,45 @@ void *get_dirtyrate_thread(void *arg) } return NULL; } + +void qmp_calc_dirty_rate(int64_t calc_time, Error **errp) +{ + static struct DirtyRateConfig config; + QemuThread thread; + int ret; + + /* + * If the dirty rate is already being measured, don't attempt to start. + */ + if (qatomic_read(&CalculatingState) == DIRTY_RATE_STATUS_MEASURING) { + error_setg(errp, "the dirty rate is already being measured."); + return; + } + + if (!is_sample_period_valid(calc_time)) { + error_setg(errp, "calc-time is out of range[%d, %d].", + MIN_FETCH_DIRTYRATE_TIME_SEC, + MAX_FETCH_DIRTYRATE_TIME_SEC); + return; + } + + /* + * Init calculation state as unstarted. + */ + ret = dirtyrate_set_state(&CalculatingState, CalculatingState, + DIRTY_RATE_STATUS_UNSTARTED); + if (ret == -1) { + error_setg(errp, "init dirty rate calculation state failed."); + return; + } + + config.sample_period_seconds = calc_time; + config.sample_pages_per_gigabytes = DIRTYRATE_DEFAULT_SAMPLE_PAGES; + qemu_thread_create(&thread, "get_dirtyrate", get_dirtyrate_thread, + (void *)&config, QEMU_THREAD_DETACHED); +} + +struct DirtyRateInfo *qmp_query_dirty_rate(Error **errp) +{ + return query_dirty_rate_info(); +} diff --git a/qapi/migration.json b/qapi/migration.json index 76a59b4f92..ce2216cfea 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1737,3 +1737,53 @@ ## { 'enum': 'DirtyRateStatus', 'data': [ 'unstarted', 'measuring', 'measured'] } + +## +# @DirtyRateInfo: +# +# Information about current dirty page rate of vm. +# +# @dirty-rate: @dirtyrate describing the dirty page rate of vm +# in units of MB/s. +# If this field returns '-1', it means querying has not +# yet started or completed. +# +# @status: status containing dirtyrate query status includes +# 'unstarted' or 'measuring' or 'measured' +# +# @start-time: start time in units of second for calculation +# +# @calc-time: time in units of second for sample dirty pages +# +# Since: 5.2 +# +## +{ 'struct': 'DirtyRateInfo', + 'data': {'dirty-rate': 'int64', + 'status': 'DirtyRateStatus', + 'start-time': 'int64', + 'calc-time': 'int64'} } + +## +# @calc-dirty-rate: +# +# start calculating dirty page rate for vm +# +# @calc-time: time in units of second for sample dirty pages +# +# Since: 5.2 +# +# Example: +# {"command": "calc-dirty-rate", "data": {"calc-time": 1} } +# +## +{ 'command': 'calc-dirty-rate', 'data': {'calc-time': 'int64'} } + +## +# @query-dirty-rate: +# +# query dirty page rate in units of MB/s for vm +# +# Since: 5.2 +## +{ 'command': 'query-dirty-rate', 'returns': 'DirtyRateInfo' } From patchwork Fri Sep 25 12:06:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 272749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A87AC4727C for ; Fri, 25 Sep 2020 12:13:52 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B265A2083B for ; Fri, 25 Sep 2020 12:13:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="JMa9CdcU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B265A2083B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:60134 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmbm-0002n2-Ie for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:13:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38330) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmW9-0003J6-Uv for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:01 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:43925) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmW7-0004GP-T5 for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:01 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035679; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UWc4hSQmzBB7vyJUeLm8dd7XRJRw3sM/5DoLuwgjqS4=; b=JMa9CdcUPX2RFF1cF/2JRSZPok2Tigw9ItwWnlutBZE+LhC1DKQE/eHN47193+HVs2+2Yq ZWVzAz/EhOuHPS9t/va6E7fy9H2nT0oXD/j9vrZkM9Hq7cYib4pcL/h1e1Gqii3/PYUKga g3z86nf4ZG9W9KfboGo6SjlmigwEpvM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-395-CmQm6c5LPmaHD9IPEQ9tvQ-1; Fri, 25 Sep 2020 08:07:55 -0400 X-MC-Unique: CmQm6c5LPmaHD9IPEQ9tvQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D33088030CC; Fri, 25 Sep 2020 12:07:53 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8A3D073692; Fri, 25 Sep 2020 12:07:51 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 14/26] migration/dirtyrate: Add trace_calls to make it easier to debug Date: Fri, 25 Sep 2020 13:06:43 +0100 Message-Id: <20200925120655.295142-15-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 02:48:20 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng Add trace_calls to make it easier to debug Signed-off-by: Chuan Zheng Reviewed-by: Dr. David Alan Gilbert Reviewed-by: David Edmondson Message-Id: <1600237327-33618-13-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/dirtyrate.c | 9 +++++++++ migration/trace-events | 8 ++++++++ 2 files changed, 17 insertions(+) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 2ffd88d48e..68577ef250 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -22,6 +22,7 @@ #include "qapi/qapi-commands-migration.h" #include "migration.h" #include "ram.h" +#include "trace.h" #include "dirtyrate.h" static int CalculatingState = DIRTY_RATE_STATUS_UNSTARTED; @@ -54,6 +55,7 @@ static bool is_sample_period_valid(int64_t sec) static int dirtyrate_set_state(int *state, int old_state, int new_state) { assert(new_state < DIRTY_RATE_STATUS__MAX); + trace_dirtyrate_set_state(DirtyRateStatus_str(new_state)); if (qatomic_cmpxchg(state, old_state, new_state) == old_state) { return 0; } else { @@ -76,6 +78,8 @@ static struct DirtyRateInfo *query_dirty_rate_info(void) info->start_time = DirtyStat.start_time; info->calc_time = DirtyStat.calc_time; + trace_query_dirty_rate_info(DirtyRateStatus_str(CalculatingState)); + return info; } @@ -123,6 +127,7 @@ static uint32_t get_ramblock_vfn_hash(struct RamblockDirtyInfo *info, crc = crc32(0, (info->ramblock_addr + vfn * TARGET_PAGE_SIZE), TARGET_PAGE_SIZE); + trace_get_ramblock_vfn_hash(info->idstr, vfn, crc); return crc; } @@ -201,6 +206,8 @@ static bool skip_sample_ramblock(RAMBlock *block) * Sample only blocks larger than MIN_RAMBLOCK_SIZE. */ if (qemu_ram_get_used_length(block) < (MIN_RAMBLOCK_SIZE << 10)) { + trace_skip_sample_ramblock(block->idstr, + qemu_ram_get_used_length(block)); return true; } @@ -260,6 +267,7 @@ static void calc_page_dirty_rate(struct RamblockDirtyInfo *info) for (i = 0; i < info->sample_pages_count; i++) { crc = get_ramblock_vfn_hash(info, info->sample_page_vfn[i]); if (crc != info->hash_result[i]) { + trace_calc_page_dirty_rate(info->idstr, crc, info->hash_result[i]); info->sample_dirty_count++; } } @@ -285,6 +293,7 @@ find_block_matched(RAMBlock *block, int count, if (infos[i].ramblock_addr != qemu_ram_get_host_addr(block) || infos[i].ramblock_pages != (qemu_ram_get_used_length(block) >> TARGET_PAGE_BITS)) { + trace_find_page_matched(block->idstr); return NULL; } diff --git a/migration/trace-events b/migration/trace-events index 7ba2fa6644..597a470b9d 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -313,3 +313,11 @@ dirty_bitmap_load_bits_zeroes(void) "" dirty_bitmap_load_header(uint32_t flags) "flags 0x%x" dirty_bitmap_load_enter(void) "" dirty_bitmap_load_success(void) "" + +# dirtyrate.c +dirtyrate_set_state(const char *new_state) "new state %s" +query_dirty_rate_info(const char *new_state) "current state %s" +get_ramblock_vfn_hash(const char *idstr, uint64_t vfn, uint32_t crc) "ramblock name: %s, vfn: %"PRIu64 ", crc: %" PRIu32 +calc_page_dirty_rate(const char *idstr, uint32_t new_crc, uint32_t old_crc) "ramblock name: %s, new crc: %" PRIu32 ", old crc: %" PRIu32 +skip_sample_ramblock(const char *idstr, uint64_t ramblock_size) "ramblock name: %s, ramblock size: %" PRIu64 +find_page_matched(const char *idstr) "ramblock %s addr or size changed" From patchwork Fri Sep 25 12:06:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 272747 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F107C4363D for ; Fri, 25 Sep 2020 12:16:15 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2336A2086A for ; Fri, 25 Sep 2020 12:16:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IdJw76QA" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2336A2086A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:40310 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLme5-0006M0-Ud for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:16:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38404) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmWI-0003dS-8E for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:48880) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmWD-0004Hz-GX for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:09 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035684; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UTt8X+r2NAe5sUSIUqOrvCV6Cqr7EQkVjHlDWo/HZE8=; b=IdJw76QARrlHM+qNlD2fsbhwGo0nG8CUFwoIPlEfPP7nRW9wafl0nb6ySB3d1WSrtnI8jV tpR0OyFOk4GYktpyBa3DQ4dN0E+M7JtP9jl+cNH9RsR38FDpj6fJymDgAuq+86BDRU4Krf IJSkrJm4n+63iEDEvb14jxjVPWEpUhE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-139-wIQFGDGLPAOWInNTFozYEQ-1; Fri, 25 Sep 2020 08:08:00 -0400 X-MC-Unique: wIQFGDGLPAOWInNTFozYEQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1D022186DD44; Fri, 25 Sep 2020 12:07:59 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 29FC673695; Fri, 25 Sep 2020 12:07:54 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 15/26] migration: Truncate state file in xen-save-devices-state Date: Fri, 25 Sep 2020 13:06:44 +0100 Message-Id: <20200925120655.295142-16-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 02:48:20 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Dov Murik When running the xen-save-devices-state QMP command, if the filename already exists it will be truncated before dumping the devices' state into it. Signed-off-by: Dov Murik Message-Id: <20200921094830.114028-1-dovmurik@linux.vnet.ibm.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- migration/savevm.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/migration/savevm.c b/migration/savevm.c index ee21e981ba..34e4b71052 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2803,7 +2803,8 @@ void qmp_xen_save_devices_state(const char *filename, bool has_live, bool live, vm_stop(RUN_STATE_SAVE_VM); global_state_store_running(); - ioc = qio_channel_file_new_path(filename, O_WRONLY | O_CREAT, 0660, errp); + ioc = qio_channel_file_new_path(filename, O_WRONLY | O_CREAT | O_TRUNC, + 0660, errp); if (!ioc) { goto the_end; } From patchwork Fri Sep 25 12:06:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 304422 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF508C4727D for ; Fri, 25 Sep 2020 12:17:58 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 42B9221D7A for ; Fri, 25 Sep 2020 12:17:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="T+Ow4eFS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 42B9221D7A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:42410 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmfk-0007Ec-VB for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:17:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38414) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmWJ-0003fZ-1Q for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:32654) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmWH-0004IX-0k for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:10 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035688; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=C8kuCzwewug5ZS+IEH9Np5RTfbg3mae2wgdLl7KapEs=; b=T+Ow4eFSa/tP5nFz2SDTKdOu4lhdNgmfrL0pQcAK/73Tb4UEIaKBlpJBS//8+UMKUBdD/F 40OXVResaZGRL8YL8au4vL3BRc2WleMLGsSHn3rgcJCD7hVDGDZScDNEFyrohkNa+zJ3RK PZRnI0TUFqjrFrBZzfph3zyu5jHL6H0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-130-4Nh1OWgDOa6u5ML4JSrMIA-1; Fri, 25 Sep 2020 08:08:05 -0400 X-MC-Unique: 4Nh1OWgDOa6u5ML4JSrMIA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8DE16186DD42; Fri, 25 Sep 2020 12:08:04 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 67D2E73697; Fri, 25 Sep 2020 12:07:59 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 16/26] migration: increase max-bandwidth to 128 MiB/s (1 Gib/s) Date: Fri, 25 Sep 2020 13:06:45 +0100 Message-Id: <20200925120655.295142-17-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Laurent Vivier max-bandwidth is set by default to 32 MiB/s (256 Mib/s) since 2008 (5bb7910af031c). Most of the CPUs can dirty memory faster than that now, and this is clearly a problem with POWER where the page size is 64 kiB and not 4 KiB. Signed-off-by: Laurent Vivier Message-Id: <20200921144957.979989-1-lvivier@redhat.com> Reviewed-by: David Gibson Reviewed-by: Greg Kurz Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 827f8be07d..de34c995af 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -57,7 +57,7 @@ #include "qemu/queue.h" #include "multifd.h" -#define MAX_THROTTLE (32 << 20) /* Migration transfer speed throttling */ +#define MAX_THROTTLE (128 << 20) /* Migration transfer speed throttling */ /* Amount of time to allocate to each "chunk" of bandwidth-throttled * data. */ From patchwork Fri Sep 25 12:06:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 304421 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43BE4C4363D for ; Fri, 25 Sep 2020 12:20:57 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9D9F12087D for ; Fri, 25 Sep 2020 12:20:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="d3ovRp5Y" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9D9F12087D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:49506 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmid-0001zD-Ak for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:20:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38464) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmWP-0003nw-RE for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:19 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:49184) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmWL-0004J1-EM for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:15 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035692; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MxEAC75mi2s7k0AhS1X4K1mrKuNvTrCWZYRq97YBjh0=; b=d3ovRp5Yp/2eN9yn/DW9DUVgSuI4S7gO11wMV4xSeVSXa13/8lZK+0tPTo9mVfx8GFMvzx SrHeHFX8sImSam/X+nwTK2XyM7Ia5DEcphaOioG/UanrG30TyfuTxWBkYrLHk1zxj4g3wE Rg7IOfmhutE9BdNKUP+J55RAqck3gWQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-428-lybUy8fMPN2g6CHcbX7olA-1; Fri, 25 Sep 2020 08:08:08 -0400 X-MC-Unique: lybUy8fMPN2g6CHcbX7olA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2D5AE425E6; Fri, 25 Sep 2020 12:08:07 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id D947273693; Fri, 25 Sep 2020 12:08:04 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 17/26] migration/tls: save hostname into MigrationState Date: Fri, 25 Sep 2020 13:06:46 +0100 Message-Id: <20200925120655.295142-18-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng hostname is need in multifd-tls, save hostname into MigrationState. Signed-off-by: Chuan Zheng Signed-off-by: Yan Jin Message-Id: <1600139042-104593-2-git-send-email-zhengchuan@huawei.com> Reviewed-by: Daniel P. Berrangé Signed-off-by: Dr. David Alan Gilbert --- migration/channel.c | 1 + migration/migration.c | 1 + migration/migration.h | 5 +++++ migration/tls.c | 2 ++ 4 files changed, 9 insertions(+) diff --git a/migration/channel.c b/migration/channel.c index 20e4c8e2dc..8a783baa0b 100644 --- a/migration/channel.c +++ b/migration/channel.c @@ -90,5 +90,6 @@ void migration_channel_connect(MigrationState *s, } } migrate_fd_connect(s, error); + g_free(s->hostname); error_free(error); } diff --git a/migration/migration.c b/migration/migration.c index de34c995af..838ca79f57 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1896,6 +1896,7 @@ void migrate_init(MigrationState *s) s->migration_thread_running = false; error_free(s->error); s->error = NULL; + s->hostname = NULL; migrate_set_state(&s->state, MIGRATION_STATUS_NONE, MIGRATION_STATUS_SETUP); diff --git a/migration/migration.h b/migration/migration.h index e74042362d..deb411aaad 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -259,6 +259,11 @@ struct MigrationState * (which is in 4M chunk). */ uint8_t clear_bitmap_shift; + + /* + * This save hostname when out-going migration starts + */ + char *hostname; }; void migrate_set_state(int *state, int old_state, int new_state); diff --git a/migration/tls.c b/migration/tls.c index 7a02ec8656..8fbf9ac796 100644 --- a/migration/tls.c +++ b/migration/tls.c @@ -154,6 +154,8 @@ void migration_tls_channel_connect(MigrationState *s, return; } + /* Save hostname into MigrationState for handshake */ + s->hostname = g_strdup(hostname); trace_migration_tls_outgoing_handshake_start(hostname); qio_channel_set_name(QIO_CHANNEL(tioc), "migration-tls-outgoing"); qio_channel_tls_handshake(tioc, From patchwork Fri Sep 25 12:06:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 272745 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B5F6C4363D for ; Fri, 25 Sep 2020 12:20:47 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A42162086A for ; Fri, 25 Sep 2020 12:20:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="L7BK30se" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A42162086A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:48598 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmiT-0001Yb-Gj for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:20:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38492) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmWT-0003oi-GC for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:20363) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmWN-0004J7-Ml for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:19 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035694; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qJcE2eKX4MSRYIivfXLil9oVYKvuM5PqU9DzpHd1rCw=; b=L7BK30seZc45QSS/OKR9xPVoCKm15DKBalk4M0E9i1RBJFUURa7hPShxx4Oy/SzYYtBOPF xvk7nJevZBj5ZrhhfUwxsgV4ymFD8wKX2+ZRpKaqTDmgdk/ZRbvuTadmG4ggNLd67E3suE M+wBvPxrthqTasoWabGJk+1d2rgyW4c= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-250-mWJxMNN1O9qRroVteRP5DQ-1; Fri, 25 Sep 2020 08:08:11 -0400 X-MC-Unique: mWJxMNN1O9qRroVteRP5DQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C3301805EE4; Fri, 25 Sep 2020 12:08:09 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7912773693; Fri, 25 Sep 2020 12:08:07 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 18/26] migration/tls: extract migration_tls_client_create for common-use Date: Fri, 25 Sep 2020 13:06:47 +0100 Message-Id: <20200925120655.295142-19-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng migration_tls_client_create will be used in multifd-tls, let's extract it. Signed-off-by: Chuan Zheng Signed-off-by: Yan Jin Reviewed-by: Daniel P. Berrangé Message-Id: <1600139042-104593-3-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/tls.c | 26 ++++++++++++++++++-------- migration/tls.h | 6 ++++++ 2 files changed, 24 insertions(+), 8 deletions(-) diff --git a/migration/tls.c b/migration/tls.c index 8fbf9ac796..66c6f43221 100644 --- a/migration/tls.c +++ b/migration/tls.c @@ -22,7 +22,6 @@ #include "channel.h" #include "migration.h" #include "tls.h" -#include "io/channel-tls.h" #include "crypto/tlscreds.h" #include "qemu/error-report.h" #include "qapi/error.h" @@ -125,11 +124,10 @@ static void migration_tls_outgoing_handshake(QIOTask *task, object_unref(OBJECT(ioc)); } - -void migration_tls_channel_connect(MigrationState *s, - QIOChannel *ioc, - const char *hostname, - Error **errp) +QIOChannelTLS *migration_tls_client_create(MigrationState *s, + QIOChannel *ioc, + const char *hostname, + Error **errp) { QCryptoTLSCreds *creds; QIOChannelTLS *tioc; @@ -137,7 +135,7 @@ void migration_tls_channel_connect(MigrationState *s, creds = migration_tls_get_creds( s, QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT, errp); if (!creds) { - return; + return NULL; } if (s->parameters.tls_hostname && *s->parameters.tls_hostname) { @@ -145,11 +143,23 @@ void migration_tls_channel_connect(MigrationState *s, } if (!hostname) { error_setg(errp, "No hostname available for TLS"); - return; + return NULL; } tioc = qio_channel_tls_new_client( ioc, creds, hostname, errp); + + return tioc; +} + +void migration_tls_channel_connect(MigrationState *s, + QIOChannel *ioc, + const char *hostname, + Error **errp) +{ + QIOChannelTLS *tioc; + + tioc = migration_tls_client_create(s, ioc, hostname, errp); if (!tioc) { return; } diff --git a/migration/tls.h b/migration/tls.h index cdd70001ed..0cfbe368ba 100644 --- a/migration/tls.h +++ b/migration/tls.h @@ -22,11 +22,17 @@ #define QEMU_MIGRATION_TLS_H #include "io/channel.h" +#include "io/channel-tls.h" void migration_tls_channel_process_incoming(MigrationState *s, QIOChannel *ioc, Error **errp); +QIOChannelTLS *migration_tls_client_create(MigrationState *s, + QIOChannel *ioc, + const char *hostname, + Error **errp); + void migration_tls_channel_connect(MigrationState *s, QIOChannel *ioc, const char *hostname, From patchwork Fri Sep 25 12:06:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 272743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CD8CC4363D for ; Fri, 25 Sep 2020 12:23:47 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C407021D7A for ; Fri, 25 Sep 2020 12:23:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XooFcRqm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C407021D7A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:56708 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmlN-0005Ai-Pg for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:23:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38496) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmWV-0003ps-RL for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:25 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:35326) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmWP-0004JF-JJ for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:22 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035696; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=u9hQ0hMltJMExf+pAheHArc7xtnUmlp9SNigj7wOqJY=; b=XooFcRqmftislbbHpW94NiY9eqTAJOY8/UZnkJGlig5q4z68/x+BVFKTbuxrKhalAbB26J DTGyG8cdVRpz483BxKPySOuS4d9ofFm7yz/6aUUJtbY2CbIoRxolkqLAOTFqxJQSZcJqJ3 u6SIKyXwoyV0J/xz7zMLazX5jR1+y5E= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-33-Hpu-5s4KOc22Ys_oujHSHA-1; Fri, 25 Sep 2020 08:08:13 -0400 X-MC-Unique: Hpu-5s4KOc22Ys_oujHSHA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 61B09802EA3; Fri, 25 Sep 2020 12:08:12 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1AAC973697; Fri, 25 Sep 2020 12:08:09 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 19/26] migration/tls: add tls_hostname into MultiFDSendParams Date: Fri, 25 Sep 2020 13:06:48 +0100 Message-Id: <20200925120655.295142-20-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 02:48:20 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng Since multifd creation is async with migration_channel_connect, we should pass the hostname from MigrationState to MultiFDSendParams. Signed-off-by: Chuan Zheng Signed-off-by: Yan Jin Message-Id: <1600139042-104593-4-git-send-email-zhengchuan@huawei.com> Reviewed-by: Daniel P. Berrangé Signed-off-by: Dr. David Alan Gilbert --- migration/multifd.c | 5 +++++ migration/multifd.h | 2 ++ 2 files changed, 7 insertions(+) diff --git a/migration/multifd.c b/migration/multifd.c index fd57378db8..de34276c43 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -548,6 +548,8 @@ void multifd_save_cleanup(void) qemu_sem_destroy(&p->sem_sync); g_free(p->name); p->name = NULL; + g_free(p->tls_hostname); + p->tls_hostname = NULL; multifd_pages_clear(p->pages); p->pages = NULL; p->packet_len = 0; @@ -751,10 +753,12 @@ int multifd_save_setup(Error **errp) int thread_count; uint32_t page_count = MULTIFD_PACKET_SIZE / qemu_target_page_size(); uint8_t i; + MigrationState *s; if (!migrate_use_multifd()) { return 0; } + s = migrate_get_current(); thread_count = migrate_multifd_channels(); multifd_send_state = g_malloc0(sizeof(*multifd_send_state)); multifd_send_state->params = g_new0(MultiFDSendParams, thread_count); @@ -779,6 +783,7 @@ int multifd_save_setup(Error **errp) p->packet->magic = cpu_to_be32(MULTIFD_MAGIC); p->packet->version = cpu_to_be32(MULTIFD_VERSION); p->name = g_strdup_printf("multifdsend_%d", i); + p->tls_hostname = g_strdup(s->hostname); socket_send_channel_create(multifd_new_send_channel_async, p); } diff --git a/migration/multifd.h b/migration/multifd.h index 448a03d89a..8d6751f5ed 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -71,6 +71,8 @@ typedef struct { uint8_t id; /* channel thread name */ char *name; + /* tls hostname */ + char *tls_hostname; /* channel thread id */ QemuThread thread; /* communication channel */ From patchwork Fri Sep 25 12:06:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 304420 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06380C4363D for ; Fri, 25 Sep 2020 12:21:11 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 726CE21D91 for ; Fri, 25 Sep 2020 12:21:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="g4RT++n4" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 726CE21D91 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:50812 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmir-0002Wk-6k for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:21:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38530) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmWb-0003tF-QJ for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:53751) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmWW-0004Jh-2z for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:28 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035703; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rjg2puLCSQciui07//qPWEIhVNJyqtO/XfPrVLvAMdA=; b=g4RT++n4rNQe/wOrVb5SWBJa5Snxymb4tQIbJ1NNqE4M6LXoV6aYZyQ5ZCvUMXHKLn9oFf bvXfTO1ffmIlc7g+m9D62G0l4vV5H+gviSQwciVvm4gdhMnUAUshSE8EIhgA/s50BWYZBt 6HhuJ2CUsYwHZk5VYhxcDnTB0ZejEUg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-463-kw9FVfvZPO6tKYMPIJkhzA-1; Fri, 25 Sep 2020 08:08:19 -0400 X-MC-Unique: kw9FVfvZPO6tKYMPIJkhzA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A9762805EE4; Fri, 25 Sep 2020 12:08:17 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id AD2A173693; Fri, 25 Sep 2020 12:08:12 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 20/26] migration/tls: extract cleanup function for common-use Date: Fri, 25 Sep 2020 13:06:49 +0100 Message-Id: <20200925120655.295142-21-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 02:48:20 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng multifd channel cleanup is need if multifd handshake failed, let's extract it. Signed-off-by: Chuan Zheng Signed-off-by: Yan Jin Reviewed-by: Daniel P. Berrangé Message-Id: <1600139042-104593-5-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/multifd.c | 34 ++++++++++++++++++++++------------ 1 file changed, 22 insertions(+), 12 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index de34276c43..36d4a403a5 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -719,6 +719,23 @@ out: return NULL; } +static void multifd_new_send_channel_cleanup(MultiFDSendParams *p, + QIOChannel *ioc, Error *err) +{ + migrate_set_error(migrate_get_current(), err); + /* Error happen, we need to tell who pay attention to me */ + qemu_sem_post(&multifd_send_state->channels_ready); + qemu_sem_post(&p->sem_sync); + /* + * Although multifd_send_thread is not created, but main migration + * thread neet to judge whether it is running, so we need to mark + * its status. + */ + p->quit = true; + object_unref(OBJECT(ioc)); + error_free(err); +} + static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque) { MultiFDSendParams *p = opaque; @@ -727,25 +744,18 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque) trace_multifd_new_send_channel_async(p->id); if (qio_task_propagate_error(task, &local_err)) { - migrate_set_error(migrate_get_current(), local_err); - /* Error happen, we need to tell who pay attention to me */ - qemu_sem_post(&multifd_send_state->channels_ready); - qemu_sem_post(&p->sem_sync); - /* - * Although multifd_send_thread is not created, but main migration - * thread needs to judge whether it is running, so we need to mark - * its status. - */ - p->quit = true; - object_unref(OBJECT(sioc)); - error_free(local_err); + goto cleanup; } else { p->c = QIO_CHANNEL(sioc); qio_channel_set_delay(p->c, false); p->running = true; qemu_thread_create(&p->thread, p->name, multifd_send_thread, p, QEMU_THREAD_JOINABLE); + return; } + +cleanup: + multifd_new_send_channel_cleanup(p, sioc, local_err); } int multifd_save_setup(Error **errp) From patchwork Fri Sep 25 12:06:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 304419 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D14BC4727C for ; Fri, 25 Sep 2020 12:24:26 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F065921D7A for ; Fri, 25 Sep 2020 12:24:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="K7KqjJP6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F065921D7A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:57924 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmm0-0005fi-Ng for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:24:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38556) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmWe-0003zX-Mw for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:25551) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmWb-0004K0-I6 for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:32 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035708; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YFFWgewJEUCwYiM/+Aevl1Ftix7NLyf9/nEk16IBEow=; b=K7KqjJP6vp7LnE/jgDeRmjsHnS5ZJUwy9eyd4pRmH0+e5lZI5FhbTgvDxGGQzU6BfJlP+0 BbbnCHTUa5SpX041c74lygLvMj98k+/Hp0/0GiYasSnpS/1kwchDHI2YtfCJe6z6BsmNlN KIPyTn+WmjSKzSUsrSbTOa+AyU5YrMs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-348-y-Pi9oUrP6aOm4HqIsU-_w-1; Fri, 25 Sep 2020 08:08:24 -0400 X-MC-Unique: y-Pi9oUrP6aOm4HqIsU-_w-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0C7AE57088; Fri, 25 Sep 2020 12:08:23 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 011D273695; Fri, 25 Sep 2020 12:08:17 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 21/26] migration/tls: add support for multifd tls-handshake Date: Fri, 25 Sep 2020 13:06:50 +0100 Message-Id: <20200925120655.295142-22-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng Similar like migration main thread, we need to do handshake for each multifd thread. Signed-off-by: Chuan Zheng Signed-off-by: Yan Jin Reviewed-by: Daniel P. Berrangé Message-Id: <1600139042-104593-6-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/multifd.c | 77 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 2 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 36d4a403a5..67e39593a7 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -20,6 +20,7 @@ #include "ram.h" #include "migration.h" #include "socket.h" +#include "tls.h" #include "qemu-file.h" #include "trace.h" #include "multifd.h" @@ -719,6 +720,77 @@ out: return NULL; } +static bool multifd_channel_connect(MultiFDSendParams *p, + QIOChannel *ioc, + Error *error); + +static void multifd_tls_outgoing_handshake(QIOTask *task, + gpointer opaque) +{ + MultiFDSendParams *p = opaque; + QIOChannel *ioc = QIO_CHANNEL(qio_task_get_source(task)); + Error *err = NULL; + + qio_task_propagate_error(task, &err); + multifd_channel_connect(p, ioc, err); +} + +static void multifd_tls_channel_connect(MultiFDSendParams *p, + QIOChannel *ioc, + Error **errp) +{ + MigrationState *s = migrate_get_current(); + const char *hostname = p->tls_hostname; + QIOChannelTLS *tioc; + + tioc = migration_tls_client_create(s, ioc, hostname, errp); + if (!tioc) { + return; + } + + qio_channel_set_name(QIO_CHANNEL(tioc), "multifd-tls-outgoing"); + qio_channel_tls_handshake(tioc, + multifd_tls_outgoing_handshake, + p, + NULL, + NULL); + +} + +static bool multifd_channel_connect(MultiFDSendParams *p, + QIOChannel *ioc, + Error *error) +{ + MigrationState *s = migrate_get_current(); + + if (!error) { + if (s->parameters.tls_creds && + *s->parameters.tls_creds && + !object_dynamic_cast(OBJECT(ioc), + TYPE_QIO_CHANNEL_TLS)) { + multifd_tls_channel_connect(p, ioc, &error); + if (!error) { + /* + * tls_channel_connect will call back to this + * function after the TLS handshake, + * so we mustn't call multifd_send_thread until then + */ + return false; + } else { + return true; + } + } else { + /* update for tls qio channel */ + p->c = ioc; + qemu_thread_create(&p->thread, p->name, multifd_send_thread, p, + QEMU_THREAD_JOINABLE); + } + return false; + } + + return true; +} + static void multifd_new_send_channel_cleanup(MultiFDSendParams *p, QIOChannel *ioc, Error *err) { @@ -749,8 +821,9 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque) p->c = QIO_CHANNEL(sioc); qio_channel_set_delay(p->c, false); p->running = true; - qemu_thread_create(&p->thread, p->name, multifd_send_thread, p, - QEMU_THREAD_JOINABLE); + if (multifd_channel_connect(p, sioc, local_err)) { + goto cleanup; + } return; } From patchwork Fri Sep 25 12:06:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 304418 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6ADF1C4363D for ; Fri, 25 Sep 2020 12:26:19 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B312F21D7A for ; Fri, 25 Sep 2020 12:26:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ghH+SPIP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B312F21D7A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34584 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmnp-0007zt-NX for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:26:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38568) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmWg-00043S-ED for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:33184) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmWe-0004KM-BV for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:34 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035711; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0q4K1GoOimXoF+idVYBXV3zVvJCoT2zu/GupuV3fex0=; b=ghH+SPIPoiEqYKlRuLeLC31ds+vkvVqjToc/8tB4iqZBkERtxjqPkn/RNihnOxi5S7XJ9B WifKGJ2OyJyPSdCXGyetFCfj1gkqvS1IfJVpWS239Fa4/6hA/T88/l4vdr+vWwYFqdhthB mZeFUmXhcS47jmFQQMgpthD66Mx3uEM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-321-GaS2aTUPPO6JDpp2XjVWdg-1; Fri, 25 Sep 2020 08:08:27 -0400 X-MC-Unique: GaS2aTUPPO6JDpp2XjVWdg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9DACF8030CC; Fri, 25 Sep 2020 12:08:25 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 56D8373692; Fri, 25 Sep 2020 12:08:23 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 22/26] migration/tls: add trace points for multifd-tls Date: Fri, 25 Sep 2020 13:06:51 +0100 Message-Id: <20200925120655.295142-23-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 02:48:20 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chuan Zheng add trace points for multifd-tls for debug. Signed-off-by: Chuan Zheng Signed-off-by: Yan Jin Reviewed-by: Daniel P. Berrangé Message-Id: <1600139042-104593-7-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/multifd.c | 10 +++++++++- migration/trace-events | 4 ++++ 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/migration/multifd.c b/migration/multifd.c index 67e39593a7..776f963436 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -731,7 +731,11 @@ static void multifd_tls_outgoing_handshake(QIOTask *task, QIOChannel *ioc = QIO_CHANNEL(qio_task_get_source(task)); Error *err = NULL; - qio_task_propagate_error(task, &err); + if (qio_task_propagate_error(task, &err)) { + trace_multifd_tls_outgoing_handshake_error(ioc, error_get_pretty(err)); + } else { + trace_multifd_tls_outgoing_handshake_complete(ioc); + } multifd_channel_connect(p, ioc, err); } @@ -748,6 +752,7 @@ static void multifd_tls_channel_connect(MultiFDSendParams *p, return; } + trace_multifd_tls_outgoing_handshake_start(ioc, tioc, hostname); qio_channel_set_name(QIO_CHANNEL(tioc), "multifd-tls-outgoing"); qio_channel_tls_handshake(tioc, multifd_tls_outgoing_handshake, @@ -763,6 +768,9 @@ static bool multifd_channel_connect(MultiFDSendParams *p, { MigrationState *s = migrate_get_current(); + trace_multifd_set_outgoing_channel( + ioc, object_get_typename(OBJECT(ioc)), p->tls_hostname, error); + if (!error) { if (s->parameters.tls_creds && *s->parameters.tls_creds && diff --git a/migration/trace-events b/migration/trace-events index 597a470b9d..338f38b3dd 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -129,6 +129,10 @@ multifd_send_sync_main_wait(uint8_t id) "channel %d" multifd_send_terminate_threads(bool error) "error %d" multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "channel %d packets %" PRIu64 " pages %" PRIu64 multifd_send_thread_start(uint8_t id) "%d" +multifd_tls_outgoing_handshake_start(void *ioc, void *tioc, const char *hostname) "ioc=%p tioc=%p hostname=%s" +multifd_tls_outgoing_handshake_error(void *ioc, const char *err) "ioc=%p err=%s" +multifd_tls_outgoing_handshake_complete(void *ioc) "ioc=%p" +multifd_set_outgoing_channel(void *ioc, const char *ioctype, const char *hostname, void *err) "ioc=%p ioctype=%s hostname=%s err=%p" # migration.c await_return_path_close_on_source_close(void) "" From patchwork Fri Sep 25 12:06:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 272740 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AB8DC4363D for ; Fri, 25 Sep 2020 12:29:02 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AD63E21D7A for ; Fri, 25 Sep 2020 12:29:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="C3kdMd4G" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AD63E21D7A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:40814 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmqS-0002Oa-Dx for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:29:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38696) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmX9-0005Av-TJ for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:09:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:52168) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmX7-0004MD-HA for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:09:03 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035740; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lvwWMmmZmOnBNYbLnXQN+MLvZSAi50iX0Kz4Z6Qg0y0=; b=C3kdMd4G9lkji9/kdVS2ZSY4Wjt6IUiADUu0OCHrI637dZKHg8A/6GwTT7IJHHPc5SOC78 Lgw+zifO65r8RS6pW9Hm6FHn4/fN9sPKeucsw2HVEOV1o+w8f+VZJ8o9fqZwdoHaCODTbg ts1JUCezDtIu+zUKcPnYQ6ZJoZ92hno= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-344-pb9kJKqGMeilook8KtYXTg-1; Fri, 25 Sep 2020 08:08:29 -0400 X-MC-Unique: pb9kJKqGMeilook8KtYXTg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3FD4380732A; Fri, 25 Sep 2020 12:08:28 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id E97C173692; Fri, 25 Sep 2020 12:08:25 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 23/26] monitor: Use LOCK_GUARD macros Date: Fri, 25 Sep 2020 13:06:52 +0100 Message-Id: <20200925120655.295142-24-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: "Dr. David Alan Gilbert" Use the lock guard macros in monitor/misc.c - saves a lot of unlocks in error paths, and the occasional goto. Signed-off-by: Dr. David Alan Gilbert Message-Id: <20200922095741.101911-1-dgilbert@redhat.com> Reviewed-by: Markus Armbruster Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Dr. David Alan Gilbert --- monitor/misc.c | 44 ++++++++++++++------------------------------ 1 file changed, 14 insertions(+), 30 deletions(-) diff --git a/monitor/misc.c b/monitor/misc.c index 262f2bd951..6e0da0cb96 100644 --- a/monitor/misc.c +++ b/monitor/misc.c @@ -141,13 +141,13 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index, handle_hmp_command(&hmp, command_line); cur_mon = old_mon; - qemu_mutex_lock(&hmp.common.mon_lock); - if (qstring_get_length(hmp.common.outbuf) > 0) { - output = g_strdup(qstring_get_str(hmp.common.outbuf)); - } else { - output = g_strdup(""); + WITH_QEMU_LOCK_GUARD(&hmp.common.mon_lock) { + if (qstring_get_length(hmp.common.outbuf) > 0) { + output = g_strdup(qstring_get_str(hmp.common.outbuf)); + } else { + output = g_strdup(""); + } } - qemu_mutex_unlock(&hmp.common.mon_lock); out: monitor_data_destroy(&hmp.common); @@ -1248,7 +1248,7 @@ void qmp_getfd(const char *fdname, Error **errp) return; } - qemu_mutex_lock(&cur_mon->mon_lock); + QEMU_LOCK_GUARD(&cur_mon->mon_lock); QLIST_FOREACH(monfd, &cur_mon->fds, next) { if (strcmp(monfd->name, fdname) != 0) { continue; @@ -1256,7 +1256,6 @@ void qmp_getfd(const char *fdname, Error **errp) tmp_fd = monfd->fd; monfd->fd = fd; - qemu_mutex_unlock(&cur_mon->mon_lock); /* Make sure close() is outside critical section */ close(tmp_fd); return; @@ -1267,7 +1266,6 @@ void qmp_getfd(const char *fdname, Error **errp) monfd->fd = fd; QLIST_INSERT_HEAD(&cur_mon->fds, monfd, next); - qemu_mutex_unlock(&cur_mon->mon_lock); } void qmp_closefd(const char *fdname, Error **errp) @@ -1299,7 +1297,7 @@ int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp) { mon_fd_t *monfd; - qemu_mutex_lock(&mon->mon_lock); + QEMU_LOCK_GUARD(&mon->mon_lock); QLIST_FOREACH(monfd, &mon->fds, next) { int fd; @@ -1313,12 +1311,10 @@ int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp) QLIST_REMOVE(monfd, next); g_free(monfd->name); g_free(monfd); - qemu_mutex_unlock(&mon->mon_lock); return fd; } - qemu_mutex_unlock(&mon->mon_lock); error_setg(errp, "File descriptor named '%s' has not been found", fdname); return -1; } @@ -1350,11 +1346,10 @@ void monitor_fdsets_cleanup(void) MonFdset *mon_fdset; MonFdset *mon_fdset_next; - qemu_mutex_lock(&mon_fdsets_lock); + QEMU_LOCK_GUARD(&mon_fdsets_lock); QLIST_FOREACH_SAFE(mon_fdset, &mon_fdsets, next, mon_fdset_next) { monitor_fdset_cleanup(mon_fdset); } - qemu_mutex_unlock(&mon_fdsets_lock); } AddfdInfo *qmp_add_fd(bool has_fdset_id, int64_t fdset_id, bool has_opaque, @@ -1389,7 +1384,7 @@ void qmp_remove_fd(int64_t fdset_id, bool has_fd, int64_t fd, Error **errp) MonFdsetFd *mon_fdset_fd; char fd_str[60]; - qemu_mutex_lock(&mon_fdsets_lock); + QEMU_LOCK_GUARD(&mon_fdsets_lock); QLIST_FOREACH(mon_fdset, &mon_fdsets, next) { if (mon_fdset->id != fdset_id) { continue; @@ -1409,12 +1404,10 @@ void qmp_remove_fd(int64_t fdset_id, bool has_fd, int64_t fd, Error **errp) goto error; } monitor_fdset_cleanup(mon_fdset); - qemu_mutex_unlock(&mon_fdsets_lock); return; } error: - qemu_mutex_unlock(&mon_fdsets_lock); if (has_fd) { snprintf(fd_str, sizeof(fd_str), "fdset-id:%" PRId64 ", fd:%" PRId64, fdset_id, fd); @@ -1430,7 +1423,7 @@ FdsetInfoList *qmp_query_fdsets(Error **errp) MonFdsetFd *mon_fdset_fd; FdsetInfoList *fdset_list = NULL; - qemu_mutex_lock(&mon_fdsets_lock); + QEMU_LOCK_GUARD(&mon_fdsets_lock); QLIST_FOREACH(mon_fdset, &mon_fdsets, next) { FdsetInfoList *fdset_info = g_malloc0(sizeof(*fdset_info)); FdsetFdInfoList *fdsetfd_list = NULL; @@ -1460,7 +1453,6 @@ FdsetInfoList *qmp_query_fdsets(Error **errp) fdset_info->next = fdset_list; fdset_list = fdset_info; } - qemu_mutex_unlock(&mon_fdsets_lock); return fdset_list; } @@ -1554,7 +1546,7 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags) #else MonFdset *mon_fdset; - qemu_mutex_lock(&mon_fdsets_lock); + QEMU_LOCK_GUARD(&mon_fdsets_lock); QLIST_FOREACH(mon_fdset, &mon_fdsets, next) { MonFdsetFd *mon_fdset_fd; MonFdsetFd *mon_fdset_fd_dup; @@ -1569,7 +1561,6 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags) QLIST_FOREACH(mon_fdset_fd, &mon_fdset->fds, next) { mon_fd_flags = fcntl(mon_fdset_fd->fd, F_GETFL); if (mon_fd_flags == -1) { - qemu_mutex_unlock(&mon_fdsets_lock); return -1; } @@ -1580,25 +1571,21 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags) } if (fd == -1) { - qemu_mutex_unlock(&mon_fdsets_lock); errno = EACCES; return -1; } dup_fd = qemu_dup_flags(fd, flags); if (dup_fd == -1) { - qemu_mutex_unlock(&mon_fdsets_lock); return -1; } mon_fdset_fd_dup = g_malloc0(sizeof(*mon_fdset_fd_dup)); mon_fdset_fd_dup->fd = dup_fd; QLIST_INSERT_HEAD(&mon_fdset->dup_fds, mon_fdset_fd_dup, next); - qemu_mutex_unlock(&mon_fdsets_lock); return dup_fd; } - qemu_mutex_unlock(&mon_fdsets_lock); errno = ENOENT; return -1; #endif @@ -1609,7 +1596,7 @@ static int64_t monitor_fdset_dup_fd_find_remove(int dup_fd, bool remove) MonFdset *mon_fdset; MonFdsetFd *mon_fdset_fd_dup; - qemu_mutex_lock(&mon_fdsets_lock); + QEMU_LOCK_GUARD(&mon_fdsets_lock); QLIST_FOREACH(mon_fdset, &mon_fdsets, next) { QLIST_FOREACH(mon_fdset_fd_dup, &mon_fdset->dup_fds, next) { if (mon_fdset_fd_dup->fd == dup_fd) { @@ -1619,17 +1606,14 @@ static int64_t monitor_fdset_dup_fd_find_remove(int dup_fd, bool remove) if (QLIST_EMPTY(&mon_fdset->dup_fds)) { monitor_fdset_cleanup(mon_fdset); } - goto err; + return -1; } else { - qemu_mutex_unlock(&mon_fdsets_lock); return mon_fdset->id; } } } } -err: - qemu_mutex_unlock(&mon_fdsets_lock); return -1; } From patchwork Fri Sep 25 12:06:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 272741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57E20C4363D for ; Fri, 25 Sep 2020 12:26:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B28C621D91 for ; Fri, 25 Sep 2020 12:26:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="czmstkBa" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B28C621D91 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:35956 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmoP-00007v-OU for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:26:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38590) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmWi-000497-SE for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:33318) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmWg-0004KX-Ua for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:36 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035714; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NDkY8vx9475fHbufaT4W/pj7j6QGnvMzicuX3sSBg+Y=; b=czmstkBajoQf2odV2JPywZTr8EFsVUtx7/8zNhrR5vIWv6kL1BIXAKgKrsr3DAYSvoSKCM r8OGPBVQeUQ/PlCoax59rJVN2fsLImogBNJioqi/Jnodzr658bvQZiMmQD5O0KCWaZRAWw qoYiJAOMGMad3I1ssXE9zJ2rqevTwXM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-536-xiYIwzDcM7Cmztyqxk3iWA-1; Fri, 25 Sep 2020 08:08:32 -0400 X-MC-Unique: xiYIwzDcM7Cmztyqxk3iWA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D472A10BBECB; Fri, 25 Sep 2020 12:08:30 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8B42C73692; Fri, 25 Sep 2020 12:08:28 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 24/26] virtiofsd: document cache=auto default Date: Fri, 25 Sep 2020 13:06:53 +0100 Message-Id: <20200925120655.295142-25-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: "Harry G. Coin" The virtiofsd --help output documents the cache=auto default value but the man page does not. Fix this. Signed-off-by: Harry G. Coin Signed-off-by: Stefan Hajnoczi Message-Id: <20200916112250.760245-1-stefanha@redhat.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- docs/tools/virtiofsd.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/tools/virtiofsd.rst b/docs/tools/virtiofsd.rst index 7fe6a87291..ae02938a95 100644 --- a/docs/tools/virtiofsd.rst +++ b/docs/tools/virtiofsd.rst @@ -103,6 +103,7 @@ Options forbids the FUSE client from caching to achieve best coherency at the cost of performance. ``auto`` acts similar to NFS with a 1 second metadata cache timeout. ``always`` sets a long cache lifetime at the expense of coherency. + The default is ``auto``. Examples -------- From patchwork Fri Sep 25 12:06:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 272742 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B593C4363D for ; Fri, 25 Sep 2020 12:25:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E0B04208A9 for ; Fri, 25 Sep 2020 12:25:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="WYk60y5R" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E0B04208A9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:60594 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmn8-0006rk-S0 for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:25:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38610) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmWr-0004SG-8P for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:27994) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmWp-0004Ks-2p for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:44 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035722; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HmXaXdAlFgmhrqchmR+bcGyDr/FBCRdNoRreh0DBOyU=; b=WYk60y5R5YFkgEyIy5FHUZobVIAPB8huINrFsqOoDi2aNDlZ5xO46tKWlZ58Z1K2BPxnR5 AD3YaSTtZ69RIc3AQ6qo4rvZhFuGZhwzpYYGlJwWDRfvSum74MDH3oxmPIlhAPfDPftQXI RS/DCUJF6IxT3C1YoagO0rbVnGvV/Og= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-548-RtIvKDKlNSy-0kLx7ihOPQ-1; Fri, 25 Sep 2020 08:08:37 -0400 X-MC-Unique: RtIvKDKlNSy-0kLx7ihOPQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 418AB1007B03; Fri, 25 Sep 2020 12:08:36 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2A81A73692; Fri, 25 Sep 2020 12:08:31 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 25/26] virtiofsd: Used glib "shared" thread pool Date: Fri, 25 Sep 2020 13:06:54 +0100 Message-Id: <20200925120655.295142-26-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vivek Goyal glib offers thread pools and it seems to support "exclusive" and "shared" thread pools. https://developer.gnome.org/glib/stable/glib-Thread-Pools.html#g-thread-pool-new Currently we use "exlusive" thread pools but its performance seems to be poor. I tried using "shared" thread pools and performance seems much better. I posted performance results here. https://www.redhat.com/archives/virtio-fs/2020-September/msg00080.html So lets switch to shared thread pools. We can think of making it optional once somebody can show in what cases exclusive thread pools offer better results. For now, my simple performance tests across the board see better results with shared thread pools. Signed-off-by: Vivek Goyal Message-Id: <20200921213216.GE13362@redhat.com> Reviewed-by: Stefan Hajnoczi Signed-off-by: Dr. David Alan Gilbert With seccomp fix from Miklos --- tools/virtiofsd/fuse_virtio.c | 2 +- tools/virtiofsd/passthrough_seccomp.c | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/virtiofsd/fuse_virtio.c b/tools/virtiofsd/fuse_virtio.c index 9e5537506c..d5c8e98253 100644 --- a/tools/virtiofsd/fuse_virtio.c +++ b/tools/virtiofsd/fuse_virtio.c @@ -588,7 +588,7 @@ static void *fv_queue_thread(void *opaque) struct fuse_session *se = qi->virtio_dev->se; GThreadPool *pool; - pool = g_thread_pool_new(fv_queue_worker, qi, se->thread_pool_size, TRUE, + pool = g_thread_pool_new(fv_queue_worker, qi, se->thread_pool_size, FALSE, NULL); if (!pool) { fuse_log(FUSE_LOG_ERR, "%s: g_thread_pool_new failed\n", __func__); diff --git a/tools/virtiofsd/passthrough_seccomp.c b/tools/virtiofsd/passthrough_seccomp.c index 19fee60011..eb9af8265f 100644 --- a/tools/virtiofsd/passthrough_seccomp.c +++ b/tools/virtiofsd/passthrough_seccomp.c @@ -93,6 +93,8 @@ static const int syscall_whitelist[] = { SCMP_SYS(rt_sigaction), SCMP_SYS(rt_sigprocmask), SCMP_SYS(rt_sigreturn), + SCMP_SYS(sched_getattr), + SCMP_SYS(sched_setattr), SCMP_SYS(sendmsg), SCMP_SYS(setresgid), SCMP_SYS(setresuid), From patchwork Fri Sep 25 12:06:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 304417 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E4B7C4363D for ; Fri, 25 Sep 2020 12:27:43 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1FBBF21D7A for ; Fri, 25 Sep 2020 12:27:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="avscC13O" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1FBBF21D7A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:38572 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLmpC-0001Kt-5o for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 08:27:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38626) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLmWu-0004ZY-A9 for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:50766) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLmWr-0004L1-RQ for qemu-devel@nongnu.org; Fri, 25 Sep 2020 08:08:47 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601035725; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YsBaT4DJHkqdygiSvDUVi/ykdYW4lCujueR9t1v3JVc=; b=avscC13Ol+pqrUVes5G3e7ts37Zzdp7B5A8wC6IRl6hXygo4pAr+q6scr37DHVaFnTHBfF 3w/El2gK7oTHaXfYwcVbV+HCmBU4D6FAbiSTq7yo0zg/zu1Mw2oyG0YaFVRbNs6u5g8KRM JmBJ9C/hxdiU1SXF8x8THOMelftKVsE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-158-Dma4gezJPbyvSfJjcxVmxA-1; Fri, 25 Sep 2020 08:08:43 -0400 X-MC-Unique: Dma4gezJPbyvSfJjcxVmxA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 888308030CD; Fri, 25 Sep 2020 12:08:41 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-177.ams2.redhat.com [10.36.114.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8CD3A73695; Fri, 25 Sep 2020 12:08:36 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, zhengchuan@huawei.com, dovmurik@linux.vnet.ibm.com, hgcoin@gmail.com, zhangjiachen.jaycee@bytedance.com, lvivier@redhat.com, peterx@redhat.com, stefanha@redhat.com, vgoyal@redhat.com, jinyan12@huawei.com, ann.zhuangyanying@huawei.com Subject: [PULL 26/26] virtiofsd: Add -o allow_direct_io|no_allow_direct_io options Date: Fri, 25 Sep 2020 13:06:55 +0100 Message-Id: <20200925120655.295142-27-dgilbert@redhat.com> In-Reply-To: <20200925120655.295142-1-dgilbert@redhat.com> References: <20200925120655.295142-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Jiachen Zhang Due to the commit 65da4539803373ec4eec97ffc49ee90083e56efd, the O_DIRECT open flag of guest applications will be discarded by virtiofsd. While this behavior makes it consistent with the virtio-9p scheme when guest applications use direct I/O, we no longer have any chance to bypass the host page cache. Therefore, we add a flag 'allow_direct_io' to lo_data. If '-o no_allow_direct_io' option is added, or none of '-o allow_direct_io' or '-o no_allow_direct_io' is added, the 'allow_direct_io' will be set to 0, and virtiofsd discards O_DIRECT as before. If '-o allow_direct_io' is added to the starting command-line, 'allow_direct_io' will be set to 1, so that the O_DIRECT flags will be retained and host page cache can be bypassed. Signed-off-by: Jiachen Zhang Reviewed-by: Stefan Hajnoczi Message-Id: <20200824105957.61265-1-zhangjiachen.jaycee@bytedance.com> Signed-off-by: Dr. David Alan Gilbert --- tools/virtiofsd/helper.c | 4 ++++ tools/virtiofsd/passthrough_ll.c | 20 ++++++++++++++------ 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/tools/virtiofsd/helper.c b/tools/virtiofsd/helper.c index 7bc5d7dc5a..85770d63f1 100644 --- a/tools/virtiofsd/helper.c +++ b/tools/virtiofsd/helper.c @@ -178,6 +178,10 @@ void fuse_cmdline_help(void) " (0 leaves rlimit unchanged)\n" " default: min(1000000, fs.file-max - 16384)\n" " if the current rlimit is lower\n" + " -o allow_direct_io|no_allow_direct_io\n" + " retain/discard O_DIRECT flags passed down\n" + " to virtiofsd from guest applications.\n" + " default: no_allow_direct_io\n" ); } diff --git a/tools/virtiofsd/passthrough_ll.c b/tools/virtiofsd/passthrough_ll.c index 784330e0e4..0b229ebd57 100644 --- a/tools/virtiofsd/passthrough_ll.c +++ b/tools/virtiofsd/passthrough_ll.c @@ -151,6 +151,7 @@ struct lo_data { int timeout_set; int readdirplus_set; int readdirplus_clear; + int allow_direct_io; struct lo_inode root; GHashTable *inodes; /* protected by lo->mutex */ struct lo_map ino_map; /* protected by lo->mutex */ @@ -179,6 +180,8 @@ static const struct fuse_opt lo_opts[] = { { "cache=always", offsetof(struct lo_data, cache), CACHE_ALWAYS }, { "readdirplus", offsetof(struct lo_data, readdirplus_set), 1 }, { "no_readdirplus", offsetof(struct lo_data, readdirplus_clear), 1 }, + { "allow_direct_io", offsetof(struct lo_data, allow_direct_io), 1 }, + { "no_allow_direct_io", offsetof(struct lo_data, allow_direct_io), 0 }, FUSE_OPT_END }; static bool use_syslog = false; @@ -1516,7 +1519,8 @@ static void lo_releasedir(fuse_req_t req, fuse_ino_t ino, fuse_reply_err(req, 0); } -static void update_open_flags(int writeback, struct fuse_file_info *fi) +static void update_open_flags(int writeback, int allow_direct_io, + struct fuse_file_info *fi) { /* * With writeback cache, kernel may send read requests even @@ -1541,10 +1545,13 @@ static void update_open_flags(int writeback, struct fuse_file_info *fi) /* * O_DIRECT in guest should not necessarily mean bypassing page - * cache on host as well. If somebody needs that behavior, it - * probably should be a configuration knob in daemon. + * cache on host as well. Therefore, we discard it by default + * ('-o no_allow_direct_io'). If somebody needs that behavior, + * the '-o allow_direct_io' option should be set. */ - fi->flags &= ~O_DIRECT; + if (!allow_direct_io) { + fi->flags &= ~O_DIRECT; + } } static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, @@ -1576,7 +1583,7 @@ static void lo_create(fuse_req_t req, fuse_ino_t parent, const char *name, goto out; } - update_open_flags(lo->writeback, fi); + update_open_flags(lo->writeback, lo->allow_direct_io, fi); fd = openat(parent_inode->fd, name, (fi->flags | O_CREAT) & ~O_NOFOLLOW, mode); @@ -1786,7 +1793,7 @@ static void lo_open(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi) fuse_log(FUSE_LOG_DEBUG, "lo_open(ino=%" PRIu64 ", flags=%d)\n", ino, fi->flags); - update_open_flags(lo->writeback, fi); + update_open_flags(lo->writeback, lo->allow_direct_io, fi); sprintf(buf, "%i", lo_fd(req, ino)); fd = openat(lo->proc_self_fd, buf, fi->flags & ~O_NOFOLLOW); @@ -2823,6 +2830,7 @@ int main(int argc, char *argv[]) .debug = 0, .writeback = 0, .posix_lock = 0, + .allow_direct_io = 0, .proc_self_fd = -1, }; struct lo_map_elem *root_elem;