From patchwork Tue Oct 31 11:26:10 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pavel Dovgalyuk X-Patchwork-Id: 117597 Delivered-To: patch@linaro.org Received: by 10.80.245.45 with SMTP id t42csp3850255edm; Tue, 31 Oct 2017 04:47:07 -0700 (PDT) X-Google-Smtp-Source: ABhQp+R7xqYVE7QVbDrEfikRhzjn0CiQFyfh4ML97EcR4OngUzYvn+EVYcnbbBRH/i73OBrypxGc X-Received: by 10.129.148.68 with SMTP id l65mr934409ywg.457.1509450427575; Tue, 31 Oct 2017 04:47:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1509450427; cv=none; d=google.com; s=arc-20160816; b=JU75jOEVbXb40kAbKShJbi/OUGGtHWTKPriKWZbOJJnLmQLj8SiHF7YKmVzaFSJ1if t5lUQegSBBS3W5/LjYac5MrwM2Fyb5jv30bTdZjBdP7wXMZsYBOUZhUctz0pdWxtdbMY zrZ9r00IB60W6K555NefSfquT9y4QYjl8oKG+he5pEL1P1rDLdumrqRPWUJOFW5Tukke 7pQftjZZAMQTDf1qKzsV4wYuNV73mRofUXI16GU/4sc3Ue7GZmvtqtzAn/A/B5Yz+K/J PL3ZQhlldTi76n+iIn2Dqufu/KxaTMvqoxKiuqyERjxJej4Q+PWJhSOv8UQM6UepVk6F /U6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=y5Y8wRaGEaQFk+6f2M/5y78fSKFSpjuvUCaIuyse3e4=; b=as5XAIHEABXr0mAFt/4PemtPFMjx/7AxLx49NDEE/vTqyYiLHBLS7sA4py1xdQq9Up uZuiyscP8oC/FhXdYSnPd3wz/md0BgDxj+/1Vpj9VFDx84LGIKJmVdFR0g+JvFv2DkaQ +TgEyKxPYFEx0wDQbZ0onDfMueDZ5QKberxTnO4K7yA6YZm0BeUo68sPIkSVyys1OztH xYzMPEAgRtCb7s9JY974+G24LS5fkCH9WG0raRcF+dPiDUXk/3YRBVRqhv1DgUHnj/J0 t0R4OwvbjVlkMF3OUpDZxmq0qEO4QeztTPNmoi0YVCQHPwYirv3ryikgbfn3/k0JCk3P Q00w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id 127si329393ybr.14.2017.10.31.04.47.07 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 31 Oct 2017 04:47:07 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:44976 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e9V0d-0004Gd-1i for patch@linaro.org; Tue, 31 Oct 2017 07:47:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50675) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e9UgO-0003VI-4E for qemu-devel@nongnu.org; Tue, 31 Oct 2017 07:26:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e9UgM-0008Iz-W4 for qemu-devel@nongnu.org; Tue, 31 Oct 2017 07:26:12 -0400 Received: from mail.ispras.ru ([83.149.199.45]:45180) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e9UgM-0008Hf-KD for qemu-devel@nongnu.org; Tue, 31 Oct 2017 07:26:10 -0400 Received: from [127.0.1.1] (unknown [85.142.117.226]) by mail.ispras.ru (Postfix) with ESMTPSA id C09D954006A; Tue, 31 Oct 2017 14:26:09 +0300 (MSK) From: Pavel Dovgalyuk To: qemu-devel@nongnu.org Date: Tue, 31 Oct 2017 14:26:10 +0300 Message-ID: <20171031112610.10516.78685.stgit@pasha-VirtualBox> In-Reply-To: <20171031112457.10516.8971.stgit@pasha-VirtualBox> References: <20171031112457.10516.8971.stgit@pasha-VirtualBox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 83.149.199.45 Subject: [Qemu-devel] [RFC PATCH 13/26] cpus: only take BQL for sleeping threads X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, pavel.dovgaluk@ispras.ru, mst@redhat.com, jasowang@redhat.com, quintela@redhat.com, zuban32s@gmail.com, maria.klimushenkova@ispras.ru, dovgaluk@ispras.ru, kraxel@redhat.com, boost.lists@gmail.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée Now the only real need to hold the BQL is for when we sleep on the cpu->halt conditional. The lock is actually dropped while the thread sleeps so the actual window for contention is pretty small. This also means we can remove the special case hack for exclusive work and simply declare that work no longer has an implicit BQL held. This isn't a major problem async work is generally only changing things in the context of its own vCPU. If it needs to work across vCPUs it should be using the exclusive mechanism or possibly taking the lock itself. Signed-off-by: Alex Bennée Tested-by: Pavel Dovgalyuk --- cpus-common.c | 13 +++++-------- cpus.c | 10 ++++------ 2 files changed, 9 insertions(+), 14 deletions(-) diff --git a/cpus-common.c b/cpus-common.c index 59f751e..64661c3 100644 --- a/cpus-common.c +++ b/cpus-common.c @@ -310,6 +310,11 @@ void async_safe_run_on_cpu(CPUState *cpu, run_on_cpu_func func, queue_work_on_cpu(cpu, wi); } +/* Work items run outside of the BQL. This is essential for avoiding a + * deadlock for exclusive work but also applies to non-exclusive work. + * If the work requires cross-vCPU changes then it should use the + * exclusive mechanism. + */ void process_queued_cpu_work(CPUState *cpu) { struct qemu_work_item *wi; @@ -327,17 +332,9 @@ void process_queued_cpu_work(CPUState *cpu) } qemu_mutex_unlock(&cpu->work_mutex); if (wi->exclusive) { - /* Running work items outside the BQL avoids the following deadlock: - * 1) start_exclusive() is called with the BQL taken while another - * CPU is running; 2) cpu_exec in the other CPU tries to takes the - * BQL, so it goes to sleep; start_exclusive() is sleeping too, so - * neither CPU can proceed. - */ - qemu_mutex_unlock_iothread(); start_exclusive(); wi->func(cpu, wi->data); end_exclusive(); - qemu_mutex_lock_iothread(); } else { wi->func(cpu, wi->data); } diff --git a/cpus.c b/cpus.c index efde5c1..de6dfce 100644 --- a/cpus.c +++ b/cpus.c @@ -1127,31 +1127,29 @@ static bool qemu_tcg_should_sleep(CPUState *cpu) static void qemu_tcg_wait_io_event(CPUState *cpu) { - qemu_mutex_lock_iothread(); while (qemu_tcg_should_sleep(cpu)) { + qemu_mutex_lock_iothread(); stop_tcg_kick_timer(); qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); + qemu_mutex_unlock_iothread(); } start_tcg_kick_timer(); qemu_wait_io_event_common(cpu); - - qemu_mutex_unlock_iothread(); } static void qemu_kvm_wait_io_event(CPUState *cpu) { - qemu_mutex_lock_iothread(); while (cpu_thread_is_idle(cpu)) { + qemu_mutex_lock_iothread(); qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); + qemu_mutex_unlock_iothread(); } qemu_wait_io_event_common(cpu); - - qemu_mutex_unlock_iothread(); } static void *qemu_kvm_cpu_thread_fn(void *arg)