From patchwork Fri May 22 06:47:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Lukas Straub X-Patchwork-Id: 282058 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 288E3C433E2 for ; Fri, 22 May 2020 06:56:18 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ED11E2065C for ; Fri, 22 May 2020 06:56:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=web.de header.i=@web.de header.b="JR3HRrRb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ED11E2065C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=web.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:35758 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jc1bN-0008CW-03 for qemu-devel@archiver.kernel.org; Fri, 22 May 2020 02:56:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38256) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jc1TD-0007Qx-Re for qemu-devel@nongnu.org; Fri, 22 May 2020 02:47:51 -0400 Received: from mout.web.de ([212.227.15.4]:38647) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jc1TC-00015i-DM for qemu-devel@nongnu.org; Fri, 22 May 2020 02:47:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1590130056; bh=CNbl5+1Uwmg7N2/0VF7Mxo+xS+N3Lz/h8u8/y0OTFF8=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References; b=JR3HRrRbM5rXcwtk5qjKxitJytJzbMknHG7Ch8IpHRGhv4VrYIsnKpTi+kIibcAr6 TAwYxyjteQCINCoOzH/rcYkErJwYuj3q7k3t3m2AiXyfFpmwr3tdwE238Tj7ikIUZq 5GRy0jXAk6ZYtwb/V83/JqAkjIOUmwUJheMUPpvU= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from luklap ([89.247.255.67]) by smtp.web.de (mrweb005 [213.165.67.108]) with ESMTPSA (Nemesis) id 1MiMAE-1j5gze0IYI-00fTl5; Fri, 22 May 2020 08:47:36 +0200 Date: Fri, 22 May 2020 08:47:34 +0200 From: Lukas Straub To: qemu-devel Subject: [PATCH v6 2/6] chardev/char.c: Use qemu_co_sleep_ns if in coroutine Message-ID: <825997b334c98a48543df66ec3dd0caf670f6dc7.1590129793.git.lukasstraub2@web.de> In-Reply-To: References: MIME-Version: 1.0 X-Provags-ID: V03:K1:CodRI4uaUsPjbwHWOr8hbiBqqyEvq5qA4z1yzoB0/ZUZ0AAa/A6 c0cIjAatQszOTtMS2JnNvZf3FHaICGn0ert+Ulw/8zljPFYENgTkQOKnzfG5dyFv9LOfiHD 2db8/O+Npz3YfQ55R6ZeoQkXHiCn8MQAnbMny2f5PvllMpiM8PmihjkNP1+4x+Zk6ZDN6fH LByvCaeBgZbk+cBLoJmNQ== X-UI-Out-Filterresults: notjunk:1; V03:K0:0Rbm5mWd3qk=:XVYkUI9RpmOSnQLdLUxXuI RZCljwqOZTVjw80xaknR2Ka1dOflihQFoiVOYIELcNmb2wsL+HiODIxGuxpN4qxe6Skwp1cWp Uo16m2g1W5hgQTUT5A4sWv7MpQ9jTBxJvmq9u4waY/kTLKCO0tqfb5eSO7WGJ6YApamuAjrMp Txs271iGRYueDuDFGJ2R2G+Zm/DyEz+OntS0OeEVS+V6mBJLx1XQjV+4fp5oqx7tDTnfbTp93 FXw6JPvyoZ2Kci2Yt8E9L93JdjcHWw3KH2lSylMv1NQ6utdsLYqwx5A/qGJO7Lupsrp9F3in3 uCrvMTFft5L86dszan+3dxY+bZiTabCaEUC8XpRrA9skhXJnKmQ3OoAkqO0f2e/7Gzgxzi4vv 3a5URfbLRsfk7IKvJb6iJgEefHlwJLpp4qQwedR0lCMsviOws8XyDacFfu5yicJNhs06kThZa cdFRlWtqIFrTegzc/B3YkOQ+iuAc854e8jW2UG9lsw6J+fotfBfxLrvhAwofEKamsdf7fQaJU /IOFiP+RVgOqZoK5+1QpE+mR42SFEyvISOjeYxm2Z0ChFiv+1ZoPaeK1fsPMBTVPMrPjGzOuZ 27mSD0OZcggFZZau/tJv8X/4EgWCaSAbM3FUZ2nZkVyqZ8MT/6CT1ZO7HAkLo8/vSF+atiSjO lF6B54fVSkU4xPouJaiHoQTtxF+FOcfCT4iHJ923sVyK1b77CQ83jyXuyfFQqixxsx9USHb1Q 9dG5DMzjZVw+Uwj9xESAkRCupU4cfDrGEHPqqrni2GugNoDNH7qs6PxPxU1PjBkwWH53IwDXo KOd+zeHItVuEOg9smwPOh+GsJ2vBYuvPSZ6VShuKQb9lqwSv15HuaVW/iu8UpbEQJPVoRZuDc gc5VDJJugJqymfBwmiY9+V3Hh8a3YpPSU1aFjkZ+DloaqIYgkDV7XSiEV/3ZNCiZLr6LrsmpC jPY/ygTBk1pxWf72tpAU7tE1z/iLNpsDTdFO/bSJwB7CIzBFm4ZxB75JZ/9WRFHzgRG4aeq4z H8z05VRWeB4X3yB5wCE5QPSt20Bn2nkixYwj5Gk1kK2hu50sSnNTj8AMn77Jo1u8fJvGL7F7Y EsYVrjVzsP4Fgb28o2QJS2ODyqy63IcfAXLmotWqqEOj1jxd4Yoj4Dr/neJC+U2Tc8b6GZWKs RUT9xem7kG5xkqxJuIA4+w+fiyFpxGZNeyyYDhiojvCLE5p9uzBunwClfi4rVOmUMieJFsyJa T3RhBTlDR+ZULqpXW Received-SPF: pass client-ip=212.227.15.4; envelope-from=lukasstraub2@web.de; helo=mout.web.de X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/22 02:47:49 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , Jason Wang , Li Zhijian , =?utf-8?q?Marc-Andr=C3=A9?= Lureau Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This will be needed in the next patch so compare_chr_send can be converted to a coroutine. Signed-off-by: Lukas Straub Reviewed-by: Marc-André Lureau Reviewed-by: Zhang Chen Reviewed-by: Philippe Mathieu-Daudé --- chardev/char.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) -- 2.20.1 diff --git a/chardev/char.c b/chardev/char.c index e77564060d..5c8014199f 100644 --- a/chardev/char.c +++ b/chardev/char.c @@ -38,6 +38,7 @@ #include "qemu/module.h" #include "qemu/option.h" #include "qemu/id.h" +#include "qemu/coroutine.h" #include "chardev/char-mux.h" @@ -119,7 +120,11 @@ static int qemu_chr_write_buffer(Chardev *s, retry: res = cc->chr_write(s, buf + *offset, len - *offset); if (res < 0 && errno == EAGAIN && write_all) { - g_usleep(100); + if (qemu_in_coroutine()) { + qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 100000); + } else { + g_usleep(100); + } goto retry; } From patchwork Fri May 22 06:47:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Straub X-Patchwork-Id: 282057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNWANTED_LANGUAGE_BODY, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9AF8C433E0 for ; Fri, 22 May 2020 07:01:38 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9C2282065C for ; Fri, 22 May 2020 07:01:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=web.de header.i=@web.de header.b="MftmjxVI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9C2282065C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=web.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:39778 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jc1gX-0004He-KA for qemu-devel@archiver.kernel.org; Fri, 22 May 2020 03:01:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38268) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jc1TN-0007UW-GV for qemu-devel@nongnu.org; Fri, 22 May 2020 02:48:01 -0400 Received: from mout.web.de ([212.227.17.12]:37177) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jc1TL-00016I-GL for qemu-devel@nongnu.org; Fri, 22 May 2020 02:48:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1590130060; bh=A0nODjFo6KWPTkquLJJac83giA5N9yaIoEj+gjJApoo=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References; b=MftmjxVImSczz8h74HC/VoWjQAzxl2dIg1L66ta2rjjU3QVvfG+uiEeeREHZw6CvB +JyKevS2q8uFuAxjfnh8/KVVlspifFFUG7z+R4eqdMBWMSUm4XiZoJatKee3eRqB29 QCGcUOzartfZkW1yXAW5MzzhwDAERN/aThtBQBHU= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from luklap ([89.247.255.67]) by smtp.web.de (mrweb102 [213.165.67.124]) with ESMTPSA (Nemesis) id 0MeBDG-1jPIJ81EPW-00PtjY; Fri, 22 May 2020 08:47:40 +0200 Date: Fri, 22 May 2020 08:47:38 +0200 From: Lukas Straub To: qemu-devel Subject: [PATCH v6 3/6] net/colo-compare.c: Fix deadlock in compare_chr_send Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Provags-ID: V03:K1:43PcfcYO3Hpqzkfu3gu8XeKuXm6lu1WuIGums3nXmKW6sgiLcgh jEeNszt1QwBgZ5xokS78Ju+fsnwpCICYvJRJdt5UOiUkTNHk6WryZiZaEokcrDuZXo3lkUl BIjnlybr1bhf1hdg+0vw50QiDDbClNx1b+OXGdcck0edZMTLbxddnOphFgzygs6Rd10pe5r eSId1EY31RUZBgb8sPFrw== X-UI-Out-Filterresults: notjunk:1; V03:K0:Jc3DtTlEx3Y=:yhVn3HU4zRL/m6GnuV86/S PMtPuiSTwVs8ORFFXUeBAUIy4M3qI6Iu9Oj7N1g0jS97eTXh/iCbovKhE3IFBeG7Nz5DkdjBr lOUHZqMY1RRE0cRuvObYlqVa9pGQqWMHXVVo7m9tuSoXZ6Omfta0rn5I3fXx0vEaXRHcqaEv6 AmKwrwX/xreIN7Q1WDUU5YGiPCS4tZrgL82Am6gKwT2SkNo28XZQ4hMXzSbbVq0bcTz+agOab yEu07NN2nsrEqsNerLkxCSl0zVtxvPwXMl7V/AkKO8yGWuvlS1ngy/oSo0As1SqQTfVDWhOg8 PtWn7ISdlycxTkrGz+/DsTlphfRvOUd0ifT3mtAz3DDBcDI+9lbw5cRZUgRIk+U2f2JXDT9Ly W7T2HfSe4bkBEkmllN8EqguiUIvrlQaIPRRkRXCJ+EvmL2HscLVLDPrip3CcR7lSqLrmA76wA mnV1rz1KagoLjM33wmlpP/E3Tv2S0EexOEDCBqf7D2EYYGJjXqAqZg9XV4PHlSIXZz+Rx3WPN /fWyYnH/jYE1l1lj+pQoO2AaMWqj2UQ8rgGBH0GxF2xMX9esr+zHk+ICFgBTHhAHfGWsINfCn uPDwNigPXymyFAIHYB1W2Y1tc3rhL77Vxp1AjNSJtJxsYeRc7RPZrnMkUBoR85aobte3sdeMb R5tWl4U6PTQWX0WmeEbewJRnSq/bcZw2kRm78hqWObyDcLoQGs5/Gxxv8vbFFoChA3P9LXaQd wjpp0lfLvPhReAYLuD7yZYeSVk1QCDATRu+tW3csPHhAn6S8pLhcECiwXpITapsWmCW1cEXWM kRsjPYGO6u9MI3oV/ddgDZ7m+qhbRWSxTSHvUqRRaQCXbE8AHYZc7ij7kgWn5t5vUZBiqL2KF dZYtFd1KtjvY5OBa9IihDbmapxGVhY1/2AqnQAC52jzgRUxncFSe1T0TRnIpq3dkQKgX9CGgZ L3RPzqW51NC7hdY2rGYshSjF8X69qC5DlbBEKUdTJFg6HDg8u90rwlXP3XSPjHW9ajHBsFLxF V3gEWWh/VTVqcL7iwpmqRVBugPaFPAg3R5ZCeyunlmC7YPrNOD7Q5zefdF8O4F6RKSDNy9M6e VZllrytNDv18FbyUytb55ojwqB3RI3iW6SaQCpxGQOpM7EtiVna7WjmPh2Csd3jcvcu3r4pNv 58GtkSqcg2Nd+EsJIt9spKU8vWVhyzBOxIRxuquz3qvnqtyFdSJNMOda0niO7vtvNQB7wci0R FwwZYj2iZAn/ihEyv Received-SPF: pass client-ip=212.227.17.12; envelope-from=lukasstraub2@web.de; helo=mout.web.de X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/22 02:47:49 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , Jason Wang , Li Zhijian , =?utf-8?q?Marc-Andr=C3=A9?= Lureau Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The chr_out chardev is connected to a filter-redirector running in the main loop. qemu_chr_fe_write_all might block here in compare_chr_send if the (socket-)buffer is full. If another filter-redirector in the main loop want's to send data to chr_pri_in it might also block if the buffer is full. This leads to a deadlock because both event loops get blocked. Fix this by converting compare_chr_send to a coroutine and putting the packets in a send queue. Signed-off-by: Lukas Straub Reviewed-by: Zhang Chen Tested-by: Zhang Chen --- net/colo-compare.c | 193 ++++++++++++++++++++++++++++++++++----------- net/colo.c | 7 ++ net/colo.h | 1 + 3 files changed, 156 insertions(+), 45 deletions(-) -- 2.20.1 diff --git a/net/colo-compare.c b/net/colo-compare.c index 1de4220fe2..e47c8c6049 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -32,6 +32,9 @@ #include "migration/migration.h" #include "util.h" +#include "block/aio-wait.h" +#include "qemu/coroutine.h" + #define TYPE_COLO_COMPARE "colo-compare" #define COLO_COMPARE(obj) \ OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE) @@ -77,6 +80,23 @@ static int event_unhandled_count; * |packet | |packet + |packet | |packet + * +--------+ +--------+ +--------+ +--------+ */ + +typedef struct SendCo { + Coroutine *co; + struct CompareState *s; + CharBackend *chr; + GQueue send_list; + bool notify_remote_frame; + bool done; + int ret; +} SendCo; + +typedef struct SendEntry { + uint32_t size; + uint32_t vnet_hdr_len; + uint8_t *buf; +} SendEntry; + typedef struct CompareState { Object parent; @@ -91,6 +111,8 @@ typedef struct CompareState { SocketReadState pri_rs; SocketReadState sec_rs; SocketReadState notify_rs; + SendCo out_sendco; + SendCo notify_sendco; bool vnet_hdr; uint32_t compare_timeout; uint32_t expired_scan_cycle; @@ -124,10 +146,11 @@ enum { static int compare_chr_send(CompareState *s, - const uint8_t *buf, + uint8_t *buf, uint32_t size, uint32_t vnet_hdr_len, - bool notify_remote_frame); + bool notify_remote_frame, + bool zero_copy); static bool packet_matches_str(const char *str, const uint8_t *buf, @@ -145,7 +168,7 @@ static void notify_remote_frame(CompareState *s) char msg[] = "DO_CHECKPOINT"; int ret = 0; - ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true); + ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true, false); if (ret < 0) { error_report("Notify Xen COLO-frame failed"); } @@ -272,12 +295,13 @@ static void colo_release_primary_pkt(CompareState *s, Packet *pkt) pkt->data, pkt->size, pkt->vnet_hdr_len, - false); + false, + true); if (ret < 0) { error_report("colo send primary packet failed"); } trace_colo_compare_main("packet same and release packet"); - packet_destroy(pkt, NULL); + packet_destroy_partial(pkt, NULL); } /* @@ -699,65 +723,115 @@ static void colo_compare_connection(void *opaque, void *user_data) } } -static int compare_chr_send(CompareState *s, - const uint8_t *buf, - uint32_t size, - uint32_t vnet_hdr_len, - bool notify_remote_frame) +static void coroutine_fn _compare_chr_send(void *opaque) { + SendCo *sendco = opaque; + CompareState *s = sendco->s; int ret = 0; - uint32_t len = htonl(size); - if (!size) { - return 0; - } + while (!g_queue_is_empty(&sendco->send_list)) { + SendEntry *entry = g_queue_pop_tail(&sendco->send_list); + uint32_t len = htonl(entry->size); - if (notify_remote_frame) { - ret = qemu_chr_fe_write_all(&s->chr_notify_dev, - (uint8_t *)&len, - sizeof(len)); - } else { - ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)&len, sizeof(len)); - } + ret = qemu_chr_fe_write_all(sendco->chr, (uint8_t *)&len, sizeof(len)); - if (ret != sizeof(len)) { - goto err; - } + if (ret != sizeof(len)) { + g_free(entry->buf); + g_slice_free(SendEntry, entry); + goto err; + } - if (s->vnet_hdr) { - /* - * We send vnet header len make other module(like filter-redirector) - * know how to parse net packet correctly. - */ - len = htonl(vnet_hdr_len); + if (!sendco->notify_remote_frame && s->vnet_hdr) { + /* + * We send vnet header len make other module(like filter-redirector) + * know how to parse net packet correctly. + */ + len = htonl(entry->vnet_hdr_len); - if (!notify_remote_frame) { - ret = qemu_chr_fe_write_all(&s->chr_out, + ret = qemu_chr_fe_write_all(sendco->chr, (uint8_t *)&len, sizeof(len)); + + if (ret != sizeof(len)) { + g_free(entry->buf); + g_slice_free(SendEntry, entry); + goto err; + } } - if (ret != sizeof(len)) { + ret = qemu_chr_fe_write_all(sendco->chr, + (uint8_t *)entry->buf, + entry->size); + + if (ret != entry->size) { + g_free(entry->buf); + g_slice_free(SendEntry, entry); goto err; } + + g_free(entry->buf); + g_slice_free(SendEntry, entry); } + sendco->ret = 0; + goto out; + +err: + while (!g_queue_is_empty(&sendco->send_list)) { + SendEntry *entry = g_queue_pop_tail(&sendco->send_list); + g_free(entry->buf); + g_slice_free(SendEntry, entry); + } + sendco->ret = ret < 0 ? ret : -EIO; +out: + sendco->co = NULL; + sendco->done = true; + aio_wait_kick(); +} + +static int compare_chr_send(CompareState *s, + uint8_t *buf, + uint32_t size, + uint32_t vnet_hdr_len, + bool notify_remote_frame, + bool zero_copy) +{ + SendCo *sendco; + SendEntry *entry; + if (notify_remote_frame) { - ret = qemu_chr_fe_write_all(&s->chr_notify_dev, - (uint8_t *)buf, - size); + sendco = &s->notify_sendco; } else { - ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)buf, size); + sendco = &s->out_sendco; } - if (ret != size) { - goto err; + if (!size) { + return 0; } - return 0; + entry = g_slice_new(SendEntry); + entry->size = size; + entry->vnet_hdr_len = vnet_hdr_len; + if (zero_copy) { + entry->buf = buf; + } else { + entry->buf = g_malloc(size); + memcpy(entry->buf, buf, size); + } + g_queue_push_head(&sendco->send_list, entry); + + if (sendco->done) { + sendco->co = qemu_coroutine_create(_compare_chr_send, sendco); + sendco->done = false; + qemu_coroutine_enter(sendco->co); + if (sendco->done) { + /* report early errors */ + return sendco->ret; + } + } -err: - return ret < 0 ? ret : -EIO; + /* assume success */ + return 0; } static int compare_chr_can_read(void *opaque) @@ -1063,6 +1137,7 @@ static void compare_pri_rs_finalize(SocketReadState *pri_rs) pri_rs->buf, pri_rs->packet_len, pri_rs->vnet_hdr_len, + false, false); } else { /* compare packet in the specified connection */ @@ -1093,7 +1168,7 @@ static void compare_notify_rs_finalize(SocketReadState *notify_rs) if (packet_matches_str("COLO_USERSPACE_PROXY_INIT", notify_rs->buf, notify_rs->packet_len)) { - ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true); + ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true, false); if (ret < 0) { error_report("Notify Xen COLO-frame INIT failed"); } @@ -1199,6 +1274,20 @@ static void colo_compare_complete(UserCreatable *uc, Error **errp) QTAILQ_INSERT_TAIL(&net_compares, s, next); + s->out_sendco.s = s; + s->out_sendco.chr = &s->chr_out; + s->out_sendco.notify_remote_frame = false; + s->out_sendco.done = true; + g_queue_init(&s->out_sendco.send_list); + + if (s->notify_dev) { + s->notify_sendco.s = s; + s->notify_sendco.chr = &s->chr_notify_dev; + s->notify_sendco.notify_remote_frame = true; + s->notify_sendco.done = true; + g_queue_init(&s->notify_sendco.send_list); + } + g_queue_init(&s->conn_list); qemu_mutex_init(&event_mtx); @@ -1225,8 +1314,9 @@ static void colo_flush_packets(void *opaque, void *user_data) pkt->data, pkt->size, pkt->vnet_hdr_len, - false); - packet_destroy(pkt, NULL); + false, + true); + packet_destroy_partial(pkt, NULL); } while (!g_queue_is_empty(&conn->secondary_list)) { pkt = g_queue_pop_head(&conn->secondary_list); @@ -1301,10 +1391,23 @@ static void colo_compare_finalize(Object *obj) } } + AioContext *ctx = iothread_get_aio_context(s->iothread); + aio_context_acquire(ctx); + AIO_WAIT_WHILE(ctx, !s->out_sendco.done); + if (s->notify_dev) { + AIO_WAIT_WHILE(ctx, !s->notify_sendco.done); + } + aio_context_release(ctx); + /* Release all unhandled packets after compare thead exited */ g_queue_foreach(&s->conn_list, colo_flush_packets, s); + AIO_WAIT_WHILE(NULL, !s->out_sendco.done); g_queue_clear(&s->conn_list); + g_queue_clear(&s->out_sendco.send_list); + if (s->notify_dev) { + g_queue_clear(&s->notify_sendco.send_list); + } if (s->connection_track_table) { g_hash_table_destroy(s->connection_track_table); diff --git a/net/colo.c b/net/colo.c index 8196b35837..a6c66d829a 100644 --- a/net/colo.c +++ b/net/colo.c @@ -185,6 +185,13 @@ void packet_destroy(void *opaque, void *user_data) g_slice_free(Packet, pkt); } +void packet_destroy_partial(void *opaque, void *user_data) +{ + Packet *pkt = opaque; + + g_slice_free(Packet, pkt); +} + /* * Clear hashtable, stop this hash growing really huge */ diff --git a/net/colo.h b/net/colo.h index 679314b1ca..573ab91785 100644 --- a/net/colo.h +++ b/net/colo.h @@ -102,5 +102,6 @@ bool connection_has_tracked(GHashTable *connection_track_table, void connection_hashtable_reset(GHashTable *connection_track_table); Packet *packet_new(const void *data, int size, int vnet_hdr_len); void packet_destroy(void *opaque, void *user_data); +void packet_destroy_partial(void *opaque, void *user_data); #endif /* NET_COLO_H */ From patchwork Fri May 22 06:47:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Lukas Straub X-Patchwork-Id: 282059 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNWANTED_LANGUAGE_BODY autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FBEFC433DF for ; Fri, 22 May 2020 06:51:30 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 097B42073B for ; Fri, 22 May 2020 06:51:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=web.de header.i=@web.de header.b="H+VKf/Ad" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 097B42073B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=web.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:56930 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jc1Wj-0002Sh-0C for qemu-devel@archiver.kernel.org; Fri, 22 May 2020 02:51:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38262) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jc1TJ-0007TD-TS for qemu-devel@nongnu.org; Fri, 22 May 2020 02:47:57 -0400 Received: from mout.web.de ([212.227.15.4]:38849) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jc1TI-000167-PS for qemu-devel@nongnu.org; Fri, 22 May 2020 02:47:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1590130063; bh=ccn6+rX95KPnZc453AzARCcdDXrnJE3FRVs3VRcf8r8=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References; b=H+VKf/AdfUmmR5GSJpzr3xM6laegk3q1EgUm3PepEXVnHvCEwkhp+TVg+1RyheJy/ 9vYp02NA/2Duee8UaQpB+dRC933Daa8EYLQlV9elSxcy6eUo9IjeLsEvyr8Ei2sC/k GPX1VipTGAknv4LBmHKXHAfQIzBZ6co0vYIFJt1Y= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from luklap ([89.247.255.67]) by smtp.web.de (mrweb005 [213.165.67.108]) with ESMTPSA (Nemesis) id 1MHEXc-1jpAdD2DT6-00DKkS; Fri, 22 May 2020 08:47:43 +0200 Date: Fri, 22 May 2020 08:47:42 +0200 From: Lukas Straub To: qemu-devel Subject: [PATCH v6 4/6] net/colo-compare.c: Only hexdump packets if tracing is enabled Message-ID: <8a4db9598051334ae3fa97b73556cdbaac5bbe43.1590129793.git.lukasstraub2@web.de> In-Reply-To: References: MIME-Version: 1.0 X-Provags-ID: V03:K1:kRqQK+J+UXSe/5J61Czr8joRBqKJwVkRhvNs85XNF3cQQMCew0M wJllyPnCDe9qsnbvdry1Q/yLkkQJ30V8aqRxPxOPnKFIEjljhaZE/Bflv3HAmZFluFxbxpp MPpc51in6IJ0Y2Eg0Bz75n4YUWmqruAzATww30i7WbQN/Sz75xa381GPYlXLZFKjdLFRTaU Lu/RR39FhDiPShC+Cm0qw== X-UI-Out-Filterresults: notjunk:1; V03:K0:tJW2wBz5+UU=:nGtIWeJ/W8oQ0qwWnyDl08 gzNY/rqiS/VwAUMDYygXbf0qcXG1eLf0Nmn4xw2+b37ctjGj3ZVi5kSfeQr0HFWHfwzj/8xof byTrydJJKtlZLsKAWzWvx8FX2cmef+U1/nKE4YHKZ9YPsXnVAV0CBqN/gurGkAPfSLjJaFH1t H0/M3cP5aOTRp5MPlbve8qJWLT0ceBLlt6cQk41KMj/FWf6M4qmNjuv3HFC/gWUMkaTUptyPw 5EIDp65/5Hc0Vr2zovzK1ItKiIQjVQqICG0gvr5iH5JsH4PymnNaZTw1ZVzX/o0SuMgI9QOVb jujkwwBVVYZrTT2bRij+YbBjWAq0EApldbvuN1T3detnJecPYq+VJlAyzloDrfsQnvQjpHIm7 8p0W8YYfWwhzvlwB9G8qYhb5TyJTm14Ni/YspSD5wFXc/Ea1/OKnYbvxSjocLn2b8+/fLQLOS mUAUJe9e/ly3BLFenxjFbhq23ofvaE7Jz2i0SIJUgzFadID9bmVSwTBRjZrp24ZvxbAtHhNd7 JMyu45pPqC52rcj0etllG0eKvX/EXZ2qIw5/tNUHsR2xONRI4Hrqu3BkZ4TAk4XX2BIDxl0QS qiW699rHXG8kuSAfB5N4ghjaaBCbmtv+WX6yz10lniaPmZINte8HmbdKS6sS0mgJyS1fI2HBW e9jluAY5OLAZ05Q21GqF7PTP6mhLl1FIYRmWz8PSl8U6OIx/W0OSGMfjCb4wXqLL4loQAe2EE ZGhqk86xrcxdqgzazJ1IQyGi+gP45Xs1Pn/iP8+IFX2WDOsi46AZfxg9xb/pizDBvUQMmF+aY dXd5Ur9b3/vmlSrugahL2nM0tz8Khz9ROkUiFiBQYwoY6UrDkQh8P9ID09MO3mf5dzojLIqdY AX2maJrIyIelBeYs/RC/DuCKYaDo+bwjz1BzgRIjkhJUSRhNJhUBy0W2M8Nm3fRt4JfiJ4j4l OhJG74YTqrEzY9bntrn4PTRk/BOVwbiwhxD/OHC2eWMw14fx6Sgi96oXGAubaE0RKSmGxSz9t knoDNldmebGeyscp4NqDnCvu43qv45AQHqATp+N5ia3sgsQo6rryt4N2gTSbOTBGalQoKkJu4 x2xRZol98ZJmC8qz5NvgoQYlFBGLlf4emXZrAOt5KxpaklJXrVedqHFsOwtO1i+1XW9FBZFGj KatYUCs5a8s5OQCW9yQYVoxGoS7AVi+EZ8zWTNyI3EUs5EGAB4KEFAitvyffEXTx1UTwLk5SS MEmiX/b2domU85OOz Received-SPF: pass client-ip=212.227.15.4; envelope-from=lukasstraub2@web.de; helo=mout.web.de X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/22 02:47:49 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , Jason Wang , Li Zhijian , =?utf-8?q?Marc-Andr=C3=A9?= Lureau Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Else the log will be flooded if there is a lot of network traffic. Signed-off-by: Lukas Straub Reviewed-by: Zhang Chen Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé --- net/colo-compare.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) -- 2.20.1 diff --git a/net/colo-compare.c b/net/colo-compare.c index e47c8c6049..7886444cdf 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -483,10 +483,12 @@ sec: g_queue_push_head(&conn->primary_list, ppkt); g_queue_push_head(&conn->secondary_list, spkt); - qemu_hexdump((char *)ppkt->data, stderr, - "colo-compare ppkt", ppkt->size); - qemu_hexdump((char *)spkt->data, stderr, - "colo-compare spkt", spkt->size); + if (trace_event_get_state_backends(TRACE_COLO_COMPARE_MISCOMPARE)) { + qemu_hexdump((char *)ppkt->data, stderr, + "colo-compare ppkt", ppkt->size); + qemu_hexdump((char *)spkt->data, stderr, + "colo-compare spkt", spkt->size); + } colo_compare_inconsistency_notify(s); }