From patchwork Tue Nov 14 08:17:39 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pavel Dovgalyuk X-Patchwork-Id: 118846 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp2764923qgn; Tue, 14 Nov 2017 00:27:10 -0800 (PST) X-Google-Smtp-Source: AGs4zMZ3kIg0UsIPkmwnRLGdqmLERZJiYnL6ejVAexFUlakiM+BDUtZOXeKrWLiXsrRxC0oR1HzK X-Received: by 10.13.196.196 with SMTP id g187mr7464816ywd.41.1510648030474; Tue, 14 Nov 2017 00:27:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510648030; cv=none; d=google.com; s=arc-20160816; b=XOUmpqKv76dwBqL1fPyaqCmp1FPsCYQKlx5T/p1IG07rui5x/kKaTvDfTBthaT3i2p QFtyAA8bxmtCguVKi8iIYqqZC5DfEu9JkIkBN5+GyjeSXnk79MoI67U7FQDAEvMtaKFg cr5kZMO9Hmu653RFhbP41hbinQPkJHDayVQDBZij78NmDBWa2vKPLRBbgsBHqskfu01h wu138O1ixkeqELC8NTLiQHl/jfGbjtzTQfOqxB55+qpn7Pa81foYXxyV99tLZZugGluP NHocKa4+PmiitVFhrI9014jII/pS/2dAzOUMX7Fj5QNxKg0IFnZwAwn2Bf69cbrJuYNu nBxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=W2nPWdW+rG+q+9wtIhToZC1Oxe/q+JpCTLim+Yyb0JM=; b=xrmqH6ytVR6skqZuinkpq1f3JUhtoWdnFIaWn573azM+BnS5YZe0jwn7rlLu/JmqNH O13wuCnlcSHZyOnHOunb7eY2r0S5CpFLQuNw4OD02HwY10Qi5OZ4O1TAbHKDvmLNamnX e4Fms+OpyX+VrUphxt2/+gDVUOaNM5YZFEQhJhtyENvwOzRWYbFvJ2bLDBrpur4dUiOA +lf2ms9eqTKm1+/ZJUmPeyon2jLvTTnChAGk5CQnSYwge1sQ8HX5V1LyV3se222V1kY7 4twBpp1nzn6t4xlIhIflz6NpGnIO8bKKRPGkaAmBK2/cjgNPM1CY4LMBKmH2QYpcB+01 M0OA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id u126si1432626ywa.593.2017.11.14.00.27.10 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 14 Nov 2017 00:27:10 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:58208 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWYn-0008Oc-V9 for patch@linaro.org; Tue, 14 Nov 2017 03:27:10 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40429) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPV-0008Mt-8S for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEWPT-0006Ql-UP for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:33 -0500 Received: from mail.ispras.ru ([83.149.199.45]:52660) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPT-0006Q4-Fq for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:31 -0500 Received: from [127.0.1.1] (unknown [85.142.117.226]) by mail.ispras.ru (Postfix) with ESMTPSA id A4E2F54006A; Tue, 14 Nov 2017 11:17:30 +0300 (MSK) From: Pavel Dovgalyuk To: qemu-devel@nongnu.org Date: Tue, 14 Nov 2017 11:17:39 +0300 Message-ID: <20171114081739.27640.85769.stgit@pasha-VirtualBox> In-Reply-To: <20171114081630.27640.53933.stgit@pasha-VirtualBox> References: <20171114081630.27640.53933.stgit@pasha-VirtualBox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 83.149.199.45 Subject: [Qemu-devel] [RFC PATCH v2 12/26] cpus: push BQL lock to qemu_*_wait_io_event X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, pavel.dovgaluk@ispras.ru, mst@redhat.com, jasowang@redhat.com, quintela@redhat.com, zuban32s@gmail.com, maria.klimushenkova@ispras.ru, dovgaluk@ispras.ru, kraxel@redhat.com, boost.lists@gmail.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée We only really need to grab the lock for initial setup (so we don't race with the thread-spawning thread). After that we can drop the lock for the whole main loop and only grab it for waiting for IO events. There is a slight wrinkle for the round-robin TCG thread as we also expire timers which needs to be done under BQL as they are in the main-loop. This is stage one of reducing the lock impact as we can drop the requirement of implicit BQL for async work and only grab the lock when we need to sleep on the cpu->halt_cond. Signed-off-by: Alex Bennée Tested-by: Pavel Dovgalyuk --- accel/kvm/kvm-all.c | 4 ---- cpus.c | 27 ++++++++++++++++++++------- target/i386/hax-all.c | 2 -- 3 files changed, 20 insertions(+), 13 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index f290f48..8d1d2c4 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -1857,9 +1857,7 @@ int kvm_cpu_exec(CPUState *cpu) return EXCP_HLT; } - qemu_mutex_unlock_iothread(); cpu_exec_start(cpu); - do { MemTxAttrs attrs; @@ -1989,8 +1987,6 @@ int kvm_cpu_exec(CPUState *cpu) } while (ret == 0); cpu_exec_end(cpu); - qemu_mutex_lock_iothread(); - if (ret < 0) { cpu_dump_state(cpu, stderr, fprintf, CPU_DUMP_CODE); vm_stop(RUN_STATE_INTERNAL_ERROR); diff --git a/cpus.c b/cpus.c index 7a3abea..3aeba5d 100644 --- a/cpus.c +++ b/cpus.c @@ -1127,6 +1127,8 @@ static bool qemu_tcg_should_sleep(CPUState *cpu) static void qemu_tcg_wait_io_event(CPUState *cpu) { + qemu_mutex_lock_iothread(); + while (qemu_tcg_should_sleep(cpu)) { stop_tcg_kick_timer(); qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); @@ -1135,15 +1137,21 @@ static void qemu_tcg_wait_io_event(CPUState *cpu) start_tcg_kick_timer(); qemu_wait_io_event_common(cpu); + + qemu_mutex_unlock_iothread(); } static void qemu_kvm_wait_io_event(CPUState *cpu) { + qemu_mutex_lock_iothread(); + while (cpu_thread_is_idle(cpu)) { qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); } qemu_wait_io_event_common(cpu); + + qemu_mutex_unlock_iothread(); } static void *qemu_kvm_cpu_thread_fn(void *arg) @@ -1169,6 +1177,8 @@ static void *qemu_kvm_cpu_thread_fn(void *arg) /* signal CPU creation */ cpu->created = true; + qemu_mutex_unlock_iothread(); + qemu_cond_signal(&qemu_cpu_cond); do { @@ -1211,10 +1221,10 @@ static void *qemu_dummy_cpu_thread_fn(void *arg) /* signal CPU creation */ cpu->created = true; + qemu_mutex_unlock_iothread(); qemu_cond_signal(&qemu_cpu_cond); while (1) { - qemu_mutex_unlock_iothread(); do { int sig; r = sigwait(&waitset, &sig); @@ -1225,6 +1235,7 @@ static void *qemu_dummy_cpu_thread_fn(void *arg) } qemu_mutex_lock_iothread(); qemu_wait_io_event_common(cpu); + qemu_mutex_unlock_iothread(); } return NULL; @@ -1313,11 +1324,9 @@ static int tcg_cpu_exec(CPUState *cpu) #ifdef CONFIG_PROFILER ti = profile_getclock(); #endif - qemu_mutex_unlock_iothread(); cpu_exec_start(cpu); ret = cpu_exec(cpu); cpu_exec_end(cpu); - qemu_mutex_lock_iothread(); #ifdef CONFIG_PROFILER tcg_time += profile_getclock() - ti; #endif @@ -1377,6 +1386,7 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) qemu_wait_io_event_common(cpu); } } + qemu_mutex_unlock_iothread(); start_tcg_kick_timer(); @@ -1386,6 +1396,9 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) cpu->exit_request = 1; while (1) { + + qemu_mutex_lock_iothread(); + /* Account partial waits to QEMU_CLOCK_VIRTUAL. */ qemu_account_warp_timer(); @@ -1394,6 +1407,8 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) */ handle_icount_deadline(); + qemu_mutex_unlock_iothread(); + if (!cpu) { cpu = first_cpu; } @@ -1419,9 +1434,7 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) cpu_handle_guest_debug(cpu); break; } else if (r == EXCP_ATOMIC) { - qemu_mutex_unlock_iothread(); cpu_exec_step_atomic(cpu); - qemu_mutex_lock_iothread(); break; } } else if (cpu->stop) { @@ -1462,6 +1475,7 @@ static void *qemu_hax_cpu_thread_fn(void *arg) current_cpu = cpu; hax_init_vcpu(cpu); + qemu_mutex_unlock_iothread(); qemu_cond_signal(&qemu_cpu_cond); while (1) { @@ -1512,6 +1526,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) cpu->created = true; cpu->can_do_io = 1; current_cpu = cpu; + qemu_mutex_unlock_iothread(); qemu_cond_signal(&qemu_cpu_cond); /* process any pending work */ @@ -1536,9 +1551,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) g_assert(cpu->halted); break; case EXCP_ATOMIC: - qemu_mutex_unlock_iothread(); cpu_exec_step_atomic(cpu); - qemu_mutex_lock_iothread(); default: /* Ignore everything else? */ break; diff --git a/target/i386/hax-all.c b/target/i386/hax-all.c index 3ce6950..9fd60d9 100644 --- a/target/i386/hax-all.c +++ b/target/i386/hax-all.c @@ -513,11 +513,9 @@ static int hax_vcpu_hax_exec(CPUArchState *env) hax_vcpu_interrupt(env); - qemu_mutex_unlock_iothread(); cpu_exec_start(cpu); hax_ret = hax_vcpu_run(vcpu); cpu_exec_end(cpu); - qemu_mutex_lock_iothread(); /* Simply continue the vcpu_run if system call interrupted */ if (hax_ret == -EINTR || hax_ret == -EAGAIN) {