From patchwork Tue Nov 14 08:17:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pavel Dovgalyuk X-Patchwork-Id: 118844 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp2763187qgn; Tue, 14 Nov 2017 00:24:39 -0800 (PST) X-Google-Smtp-Source: AGs4zMZFHAP0ck80LBSnM61ulvAXsOxRGY5r1LCUnApNT00tK3Cjx/QNW8PdLt2bO6ReOfXTgLES X-Received: by 10.37.163.1 with SMTP id d1mr6936695ybi.367.1510647879377; Tue, 14 Nov 2017 00:24:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510647879; cv=none; d=google.com; s=arc-20160816; b=qyZl2fCUy+zEsgEwlV7X3EBAfTBIo68zcY2spvum295EdR4fyNbGPOwHVccjl9to7l psHLibdManSUf+O2Yfpx2srq4OKjr/MHlm2Tvt80ZWMatTSG2+FHgwsHTH//FR5g0zB5 FybyFcr07PDTUjmyens8oDst8TKUOKh6LL9lCy2HOt0IVT+BDvhyiYKrGJ1atUm++l3c VLBM1uWYkU/o+2+v0gIsgOatCaMEAgWtK8GNLSIIcwhOwFc+XJWfrWyINB47wXh8zOuG 2SuSSKUJD3i82EFqH8RkJMnxSbzdwkloLVguZ1VyjgUZen6CpckdXUOoD6uVWCsTrm9I BkEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=rs1qWJWv8UDbdM1egYNaxIk0TpwVSiUjobYQ5TjYI7s=; b=KfDWt9xneTvAde7TOQGWKPZZlgySghGZPH3DF6zkKpIg1Xdeo90dMkwJFg0lL8O3cG 0I8EJUb/HPk84cTLo4iWzWNleJwZHTxo8ajwUDGrBxt9xZi+4GpcJPxCtXslTAbpRXv5 BjLhEOFU3ASxudvhkwtKocayxtW1q/M3MwKzwUxS78clYa005RN3U4k/xt9RUB/bZ3iu 8MTPI6KN0LV+k0ytdQoQQwIh2WdrhcG3DQVJ230Ix7NZuX7BItEP6ztq6OD0VnJMNaqX DepwqvVvwRKabffbyFQDfX4TR/yaGc4kl0m2Zsi50P1hTtj3DPsj122uAcHLRHfMTwhq lovw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id a66si659459ybg.439.2017.11.14.00.24.39 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 14 Nov 2017 00:24:39 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:58191 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWWM-00067K-R9 for patch@linaro.org; Tue, 14 Nov 2017 03:24:38 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40388) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPS-0008G6-86 for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:31 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEWPO-0006LU-7Z for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:30 -0500 Received: from mail.ispras.ru ([83.149.199.45]:52640) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPN-0006L7-Us for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:26 -0500 Received: from [127.0.1.1] (unknown [85.142.117.226]) by mail.ispras.ru (Postfix) with ESMTPSA id 16DC154006A; Tue, 14 Nov 2017 11:17:25 +0300 (MSK) From: Pavel Dovgalyuk To: qemu-devel@nongnu.org Date: Tue, 14 Nov 2017 11:17:33 +0300 Message-ID: <20171114081733.27640.74292.stgit@pasha-VirtualBox> In-Reply-To: <20171114081630.27640.53933.stgit@pasha-VirtualBox> References: <20171114081630.27640.53933.stgit@pasha-VirtualBox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 83.149.199.45 Subject: [Qemu-devel] [RFC PATCH v2 11/26] target/arm/arm-powertctl: drop BQL assertions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, pavel.dovgaluk@ispras.ru, mst@redhat.com, jasowang@redhat.com, quintela@redhat.com, zuban32s@gmail.com, maria.klimushenkova@ispras.ru, dovgaluk@ispras.ru, kraxel@redhat.com, boost.lists@gmail.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée The powerctl code is run in the context of the vCPU changing power state. It does not need the BQL to protect its changes. Signed-off-by: Alex Bennée --- target/arm/arm-powerctl.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/target/arm/arm-powerctl.c b/target/arm/arm-powerctl.c index 25207cb..9661a59 100644 --- a/target/arm/arm-powerctl.c +++ b/target/arm/arm-powerctl.c @@ -124,7 +124,6 @@ static void arm_set_cpu_on_async_work(CPUState *target_cpu_state, g_free(info); /* Finally set the power status */ - assert(qemu_mutex_iothread_locked()); target_cpu->power_state = PSCI_ON; } @@ -135,8 +134,6 @@ int arm_set_cpu_on(uint64_t cpuid, uint64_t entry, uint64_t context_id, ARMCPU *target_cpu; struct CpuOnInfo *info; - assert(qemu_mutex_iothread_locked()); - DPRINTF("cpu %" PRId64 " (EL %d, %s) @ 0x%" PRIx64 " with R0 = 0x%" PRIx64 "\n", cpuid, target_el, target_aa64 ? "aarch64" : "aarch32", entry, context_id); @@ -227,7 +224,6 @@ static void arm_set_cpu_off_async_work(CPUState *target_cpu_state, { ARMCPU *target_cpu = ARM_CPU(target_cpu_state); - assert(qemu_mutex_iothread_locked()); target_cpu->power_state = PSCI_OFF; target_cpu_state->halted = 1; target_cpu_state->exception_index = EXCP_HLT; @@ -238,8 +234,6 @@ int arm_set_cpu_off(uint64_t cpuid) CPUState *target_cpu_state; ARMCPU *target_cpu; - assert(qemu_mutex_iothread_locked()); - DPRINTF("cpu %" PRId64 "\n", cpuid); /* change to the cpu we are powering up */ @@ -274,8 +268,6 @@ int arm_reset_cpu(uint64_t cpuid) CPUState *target_cpu_state; ARMCPU *target_cpu; - assert(qemu_mutex_iothread_locked()); - DPRINTF("cpu %" PRId64 "\n", cpuid); /* change to the cpu we are resetting */ From patchwork Tue Nov 14 08:17:39 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pavel Dovgalyuk X-Patchwork-Id: 118846 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp2764923qgn; Tue, 14 Nov 2017 00:27:10 -0800 (PST) X-Google-Smtp-Source: AGs4zMZ3kIg0UsIPkmwnRLGdqmLERZJiYnL6ejVAexFUlakiM+BDUtZOXeKrWLiXsrRxC0oR1HzK X-Received: by 10.13.196.196 with SMTP id g187mr7464816ywd.41.1510648030474; Tue, 14 Nov 2017 00:27:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510648030; cv=none; d=google.com; s=arc-20160816; b=XOUmpqKv76dwBqL1fPyaqCmp1FPsCYQKlx5T/p1IG07rui5x/kKaTvDfTBthaT3i2p QFtyAA8bxmtCguVKi8iIYqqZC5DfEu9JkIkBN5+GyjeSXnk79MoI67U7FQDAEvMtaKFg cr5kZMO9Hmu653RFhbP41hbinQPkJHDayVQDBZij78NmDBWa2vKPLRBbgsBHqskfu01h wu138O1ixkeqELC8NTLiQHl/jfGbjtzTQfOqxB55+qpn7Pa81foYXxyV99tLZZugGluP NHocKa4+PmiitVFhrI9014jII/pS/2dAzOUMX7Fj5QNxKg0IFnZwAwn2Bf69cbrJuYNu nBxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=W2nPWdW+rG+q+9wtIhToZC1Oxe/q+JpCTLim+Yyb0JM=; b=xrmqH6ytVR6skqZuinkpq1f3JUhtoWdnFIaWn573azM+BnS5YZe0jwn7rlLu/JmqNH O13wuCnlcSHZyOnHOunb7eY2r0S5CpFLQuNw4OD02HwY10Qi5OZ4O1TAbHKDvmLNamnX e4Fms+OpyX+VrUphxt2/+gDVUOaNM5YZFEQhJhtyENvwOzRWYbFvJ2bLDBrpur4dUiOA +lf2ms9eqTKm1+/ZJUmPeyon2jLvTTnChAGk5CQnSYwge1sQ8HX5V1LyV3se222V1kY7 4twBpp1nzn6t4xlIhIflz6NpGnIO8bKKRPGkaAmBK2/cjgNPM1CY4LMBKmH2QYpcB+01 M0OA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id u126si1432626ywa.593.2017.11.14.00.27.10 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 14 Nov 2017 00:27:10 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:58208 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWYn-0008Oc-V9 for patch@linaro.org; Tue, 14 Nov 2017 03:27:10 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40429) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPV-0008Mt-8S for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEWPT-0006Ql-UP for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:33 -0500 Received: from mail.ispras.ru ([83.149.199.45]:52660) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPT-0006Q4-Fq for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:31 -0500 Received: from [127.0.1.1] (unknown [85.142.117.226]) by mail.ispras.ru (Postfix) with ESMTPSA id A4E2F54006A; Tue, 14 Nov 2017 11:17:30 +0300 (MSK) From: Pavel Dovgalyuk To: qemu-devel@nongnu.org Date: Tue, 14 Nov 2017 11:17:39 +0300 Message-ID: <20171114081739.27640.85769.stgit@pasha-VirtualBox> In-Reply-To: <20171114081630.27640.53933.stgit@pasha-VirtualBox> References: <20171114081630.27640.53933.stgit@pasha-VirtualBox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 83.149.199.45 Subject: [Qemu-devel] [RFC PATCH v2 12/26] cpus: push BQL lock to qemu_*_wait_io_event X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, pavel.dovgaluk@ispras.ru, mst@redhat.com, jasowang@redhat.com, quintela@redhat.com, zuban32s@gmail.com, maria.klimushenkova@ispras.ru, dovgaluk@ispras.ru, kraxel@redhat.com, boost.lists@gmail.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée We only really need to grab the lock for initial setup (so we don't race with the thread-spawning thread). After that we can drop the lock for the whole main loop and only grab it for waiting for IO events. There is a slight wrinkle for the round-robin TCG thread as we also expire timers which needs to be done under BQL as they are in the main-loop. This is stage one of reducing the lock impact as we can drop the requirement of implicit BQL for async work and only grab the lock when we need to sleep on the cpu->halt_cond. Signed-off-by: Alex Bennée Tested-by: Pavel Dovgalyuk --- accel/kvm/kvm-all.c | 4 ---- cpus.c | 27 ++++++++++++++++++++------- target/i386/hax-all.c | 2 -- 3 files changed, 20 insertions(+), 13 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index f290f48..8d1d2c4 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -1857,9 +1857,7 @@ int kvm_cpu_exec(CPUState *cpu) return EXCP_HLT; } - qemu_mutex_unlock_iothread(); cpu_exec_start(cpu); - do { MemTxAttrs attrs; @@ -1989,8 +1987,6 @@ int kvm_cpu_exec(CPUState *cpu) } while (ret == 0); cpu_exec_end(cpu); - qemu_mutex_lock_iothread(); - if (ret < 0) { cpu_dump_state(cpu, stderr, fprintf, CPU_DUMP_CODE); vm_stop(RUN_STATE_INTERNAL_ERROR); diff --git a/cpus.c b/cpus.c index 7a3abea..3aeba5d 100644 --- a/cpus.c +++ b/cpus.c @@ -1127,6 +1127,8 @@ static bool qemu_tcg_should_sleep(CPUState *cpu) static void qemu_tcg_wait_io_event(CPUState *cpu) { + qemu_mutex_lock_iothread(); + while (qemu_tcg_should_sleep(cpu)) { stop_tcg_kick_timer(); qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); @@ -1135,15 +1137,21 @@ static void qemu_tcg_wait_io_event(CPUState *cpu) start_tcg_kick_timer(); qemu_wait_io_event_common(cpu); + + qemu_mutex_unlock_iothread(); } static void qemu_kvm_wait_io_event(CPUState *cpu) { + qemu_mutex_lock_iothread(); + while (cpu_thread_is_idle(cpu)) { qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); } qemu_wait_io_event_common(cpu); + + qemu_mutex_unlock_iothread(); } static void *qemu_kvm_cpu_thread_fn(void *arg) @@ -1169,6 +1177,8 @@ static void *qemu_kvm_cpu_thread_fn(void *arg) /* signal CPU creation */ cpu->created = true; + qemu_mutex_unlock_iothread(); + qemu_cond_signal(&qemu_cpu_cond); do { @@ -1211,10 +1221,10 @@ static void *qemu_dummy_cpu_thread_fn(void *arg) /* signal CPU creation */ cpu->created = true; + qemu_mutex_unlock_iothread(); qemu_cond_signal(&qemu_cpu_cond); while (1) { - qemu_mutex_unlock_iothread(); do { int sig; r = sigwait(&waitset, &sig); @@ -1225,6 +1235,7 @@ static void *qemu_dummy_cpu_thread_fn(void *arg) } qemu_mutex_lock_iothread(); qemu_wait_io_event_common(cpu); + qemu_mutex_unlock_iothread(); } return NULL; @@ -1313,11 +1324,9 @@ static int tcg_cpu_exec(CPUState *cpu) #ifdef CONFIG_PROFILER ti = profile_getclock(); #endif - qemu_mutex_unlock_iothread(); cpu_exec_start(cpu); ret = cpu_exec(cpu); cpu_exec_end(cpu); - qemu_mutex_lock_iothread(); #ifdef CONFIG_PROFILER tcg_time += profile_getclock() - ti; #endif @@ -1377,6 +1386,7 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) qemu_wait_io_event_common(cpu); } } + qemu_mutex_unlock_iothread(); start_tcg_kick_timer(); @@ -1386,6 +1396,9 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) cpu->exit_request = 1; while (1) { + + qemu_mutex_lock_iothread(); + /* Account partial waits to QEMU_CLOCK_VIRTUAL. */ qemu_account_warp_timer(); @@ -1394,6 +1407,8 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) */ handle_icount_deadline(); + qemu_mutex_unlock_iothread(); + if (!cpu) { cpu = first_cpu; } @@ -1419,9 +1434,7 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) cpu_handle_guest_debug(cpu); break; } else if (r == EXCP_ATOMIC) { - qemu_mutex_unlock_iothread(); cpu_exec_step_atomic(cpu); - qemu_mutex_lock_iothread(); break; } } else if (cpu->stop) { @@ -1462,6 +1475,7 @@ static void *qemu_hax_cpu_thread_fn(void *arg) current_cpu = cpu; hax_init_vcpu(cpu); + qemu_mutex_unlock_iothread(); qemu_cond_signal(&qemu_cpu_cond); while (1) { @@ -1512,6 +1526,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) cpu->created = true; cpu->can_do_io = 1; current_cpu = cpu; + qemu_mutex_unlock_iothread(); qemu_cond_signal(&qemu_cpu_cond); /* process any pending work */ @@ -1536,9 +1551,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) g_assert(cpu->halted); break; case EXCP_ATOMIC: - qemu_mutex_unlock_iothread(); cpu_exec_step_atomic(cpu); - qemu_mutex_lock_iothread(); default: /* Ignore everything else? */ break; diff --git a/target/i386/hax-all.c b/target/i386/hax-all.c index 3ce6950..9fd60d9 100644 --- a/target/i386/hax-all.c +++ b/target/i386/hax-all.c @@ -513,11 +513,9 @@ static int hax_vcpu_hax_exec(CPUArchState *env) hax_vcpu_interrupt(env); - qemu_mutex_unlock_iothread(); cpu_exec_start(cpu); hax_ret = hax_vcpu_run(vcpu); cpu_exec_end(cpu); - qemu_mutex_lock_iothread(); /* Simply continue the vcpu_run if system call interrupted */ if (hax_ret == -EINTR || hax_ret == -EAGAIN) { From patchwork Tue Nov 14 08:17:44 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pavel Dovgalyuk X-Patchwork-Id: 118852 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp2772765qgn; Tue, 14 Nov 2017 00:37:24 -0800 (PST) X-Google-Smtp-Source: AGs4zMZuF32A3NZDvJfyNpOgviZCbz6D3y/yk20KdJvQzVDdAnUsFNP5QnquTTfcwUnhl3NglgA2 X-Received: by 10.129.85.198 with SMTP id j189mr2885075ywb.504.1510648644640; Tue, 14 Nov 2017 00:37:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510648644; cv=none; d=google.com; s=arc-20160816; b=kVyytgGIB6Oixm9HNg8Xt2wLDxJLDieBkLDS8dEAT10rlyP+0XHPXriE8l9IQkberP Stj0+6zBaOiAHorRZSed+ZiDIC0IY3d01SMZ9hTcV2YXC6b+Yusyiq1Yo4uMwJ49BoBY tUTn6DV3a3rls7kPH0IrotI51lC543x5+tw/8YrsBslg2J+93x/mNaasBcuyKBJyHVNk hUW4wG3FoNd54mH3JPsWuaiTWNBXlVfgdWJLPSsWPNAIpEgV4QUdo97hb6ASlxo776ob 8qw90N4jnM6eUkb/Q4A6u+QlnOdYBQB4oi46sgS7TUGluAia7aoRc++/U8BR7A38eM6f m8ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=KTldzkGteZd5nXNJsshgm9XwU0W5Aa64fuyzC+n/2BA=; b=QteggGl77ABMuFi2lLG41BImrV24GwgsuBX+J2169h8JRtugYF0pqrqYiM2eJZwxDC VlxRmVFTBnGCh82W9Szvuk/U4jMAUJwkKtpeUJEO4luQF5VLhURp9RUfzucRHDpsyNUi 10L2y04+WO6bN2YWIr92jZ48FC9EXZ6KhDJeLx4e2a/6JYUGeDMyIXZlFcJzgkudIRud c2hhCG9YFUk0wX/Dv1M3UtS1vFg9g6nJMLK2PSRdBDHc6TsQTC2yx2o9OKtn+T2DqPIc huHv3ZxnjIY5aIGdbGHQyIqx304gsMxBpMFUf6dRUFns9XUh9ixn8qeBwoY9pxa7t0Ba O/PA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id e127si753550ywh.257.2017.11.14.00.37.24 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 14 Nov 2017 00:37:24 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:58273 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWii-0001mZ-60 for patch@linaro.org; Tue, 14 Nov 2017 03:37:24 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40536) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPd-0008T0-Dx for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:42 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEWPZ-0006VA-Dd for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:41 -0500 Received: from mail.ispras.ru ([83.149.199.45]:52674) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPZ-0006UT-1q for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:37 -0500 Received: from [127.0.1.1] (unknown [85.142.117.226]) by mail.ispras.ru (Postfix) with ESMTPSA id 3829754006A; Tue, 14 Nov 2017 11:17:36 +0300 (MSK) From: Pavel Dovgalyuk To: qemu-devel@nongnu.org Date: Tue, 14 Nov 2017 11:17:44 +0300 Message-ID: <20171114081744.27640.58377.stgit@pasha-VirtualBox> In-Reply-To: <20171114081630.27640.53933.stgit@pasha-VirtualBox> References: <20171114081630.27640.53933.stgit@pasha-VirtualBox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 83.149.199.45 Subject: [Qemu-devel] [RFC PATCH v2 13/26] cpus: only take BQL for sleeping threads X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, pavel.dovgaluk@ispras.ru, mst@redhat.com, jasowang@redhat.com, quintela@redhat.com, zuban32s@gmail.com, maria.klimushenkova@ispras.ru, dovgaluk@ispras.ru, kraxel@redhat.com, boost.lists@gmail.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée Now the only real need to hold the BQL is for when we sleep on the cpu->halt conditional. The lock is actually dropped while the thread sleeps so the actual window for contention is pretty small. This also means we can remove the special case hack for exclusive work and simply declare that work no longer has an implicit BQL held. This isn't a major problem async work is generally only changing things in the context of its own vCPU. If it needs to work across vCPUs it should be using the exclusive mechanism or possibly taking the lock itself. Signed-off-by: Alex Bennée Tested-by: Pavel Dovgalyuk --- cpus-common.c | 13 +++++-------- cpus.c | 10 ++++------ 2 files changed, 9 insertions(+), 14 deletions(-) diff --git a/cpus-common.c b/cpus-common.c index 59f751e..64661c3 100644 --- a/cpus-common.c +++ b/cpus-common.c @@ -310,6 +310,11 @@ void async_safe_run_on_cpu(CPUState *cpu, run_on_cpu_func func, queue_work_on_cpu(cpu, wi); } +/* Work items run outside of the BQL. This is essential for avoiding a + * deadlock for exclusive work but also applies to non-exclusive work. + * If the work requires cross-vCPU changes then it should use the + * exclusive mechanism. + */ void process_queued_cpu_work(CPUState *cpu) { struct qemu_work_item *wi; @@ -327,17 +332,9 @@ void process_queued_cpu_work(CPUState *cpu) } qemu_mutex_unlock(&cpu->work_mutex); if (wi->exclusive) { - /* Running work items outside the BQL avoids the following deadlock: - * 1) start_exclusive() is called with the BQL taken while another - * CPU is running; 2) cpu_exec in the other CPU tries to takes the - * BQL, so it goes to sleep; start_exclusive() is sleeping too, so - * neither CPU can proceed. - */ - qemu_mutex_unlock_iothread(); start_exclusive(); wi->func(cpu, wi->data); end_exclusive(); - qemu_mutex_lock_iothread(); } else { wi->func(cpu, wi->data); } diff --git a/cpus.c b/cpus.c index 3aeba5d..1800a3b 100644 --- a/cpus.c +++ b/cpus.c @@ -1127,31 +1127,29 @@ static bool qemu_tcg_should_sleep(CPUState *cpu) static void qemu_tcg_wait_io_event(CPUState *cpu) { - qemu_mutex_lock_iothread(); while (qemu_tcg_should_sleep(cpu)) { + qemu_mutex_lock_iothread(); stop_tcg_kick_timer(); qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); + qemu_mutex_unlock_iothread(); } start_tcg_kick_timer(); qemu_wait_io_event_common(cpu); - - qemu_mutex_unlock_iothread(); } static void qemu_kvm_wait_io_event(CPUState *cpu) { - qemu_mutex_lock_iothread(); while (cpu_thread_is_idle(cpu)) { + qemu_mutex_lock_iothread(); qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); + qemu_mutex_unlock_iothread(); } qemu_wait_io_event_common(cpu); - - qemu_mutex_unlock_iothread(); } static void *qemu_kvm_cpu_thread_fn(void *arg) From patchwork Tue Nov 14 08:17:50 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pavel Dovgalyuk X-Patchwork-Id: 118848 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp2766410qgn; Tue, 14 Nov 2017 00:29:16 -0800 (PST) X-Google-Smtp-Source: AGs4zMbYx9kTd+am+tuTFtPypD1n4wN6Ali0EYJ9exexl+0DKe3Bhw23M99fmdfiFzDBdDLJK6G+ X-Received: by 10.13.218.133 with SMTP id c127mr818567ywe.68.1510648156394; Tue, 14 Nov 2017 00:29:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510648156; cv=none; d=google.com; s=arc-20160816; b=TINELzMopNvDfs2Nw6n3xLr9KOiyUGU40B51JonJ1cMPnMBzbb/WHebCtvN5h05E7/ 8SeixtPfBq7kjwGHnLhAnnTVAk4VZjGOBLpO9fhcL+eFfzt5FHwYmjsCRQES3ZW7D/qN BpAsdkouQuWnERkMKsH3B4+Z/47jNQndHFsHEkjJDZ6Qn43DtKxMx4PMQ1Sahjtuf6AZ 9kMXY/NGBLkpuPo7e8ro2h/45/+DaeTO+f18scuzXAHxTXxTMuSMY1UsU3kB3zsKXhmM VGMFzEXaHwEKeVd2w6b3qflXkKAiQLrcI98kPE0E0N+keyEYsEHq87bD4jG+xG6RXCAS Ilow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=wB0Ogw6PcNL9/dEgXbcFgFmGjbFBmF92fbxgyIPdfqw=; b=0qxrFiGOX8uwg8qy/W9gHbHiIYEtC5oGcsXUVIEs65Sh2Z29OZYnulNW9zUEWJ6Ed+ YSc4QbEYlarHAE6VajqsW33vwnmfaUI45MgdCEpzzfkSgNRSnegTQQi2SKdFgNXZ/2u5 DOK51EZta/iBMi9wNBMy1aCpSOqZBGf4+tI9mDQQE0p2S3o9KqLXAMP2mP06/8T8ELHU k6GRLYNSQGUt59YbpuhIMAZkDMWFC+pUtD/+xsrcOUcXXWMI73F1Pvjh06DWz7z4DXT6 QTecC+WF/GbWaYoS7NBSOCrMHxINVidzFDcrbDt7hXdl7YgaInsWli7InaJhauLN2O0l ltQg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id q14si3668046ybm.781.2017.11.14.00.29.16 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 14 Nov 2017 00:29:16 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:58216 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWap-0002fo-RA for patch@linaro.org; Tue, 14 Nov 2017 03:29:15 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40571) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPf-0008V9-OV for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEWPe-0006ZR-RB for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:43 -0500 Received: from mail.ispras.ru ([83.149.199.45]:52696) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPe-0006Yt-Ji for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:42 -0500 Received: from [127.0.1.1] (unknown [85.142.117.226]) by mail.ispras.ru (Postfix) with ESMTPSA id BF5B754006A; Tue, 14 Nov 2017 11:17:41 +0300 (MSK) From: Pavel Dovgalyuk To: qemu-devel@nongnu.org Date: Tue, 14 Nov 2017 11:17:50 +0300 Message-ID: <20171114081750.27640.53609.stgit@pasha-VirtualBox> In-Reply-To: <20171114081630.27640.53933.stgit@pasha-VirtualBox> References: <20171114081630.27640.53933.stgit@pasha-VirtualBox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 83.149.199.45 Subject: [Qemu-devel] [RFC PATCH v2 14/26] replay/replay.c: bump REPLAY_VERSION again X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, pavel.dovgaluk@ispras.ru, mst@redhat.com, jasowang@redhat.com, quintela@redhat.com, zuban32s@gmail.com, maria.klimushenkova@ispras.ru, dovgaluk@ispras.ru, kraxel@redhat.com, boost.lists@gmail.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée This time commit 802f045a5f61b781df55e4492d896b4d20503ba7 broke the replay file format. Also add a comment about this to replay-internal.h. Signed-off-by: Alex Bennée Reviewed-off-by: Pavel Dovgalyuk --- replay/replay-internal.h | 2 +- replay/replay.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) Acked-by: Paolo Bonzini diff --git a/replay/replay-internal.h b/replay/replay-internal.h index be96d7e..8e4c701 100644 --- a/replay/replay-internal.h +++ b/replay/replay-internal.h @@ -12,7 +12,7 @@ * */ - +/* Any changes to order/number of events will need to bump REPLAY_VERSION */ enum ReplayEvents { /* for instruction event */ EVENT_INSTRUCTION, diff --git a/replay/replay.c b/replay/replay.c index ff58a5a..4f24498 100644 --- a/replay/replay.c +++ b/replay/replay.c @@ -22,7 +22,7 @@ /* Current version of the replay mechanism. Increase it when file format changes. */ -#define REPLAY_VERSION 0xe02006 +#define REPLAY_VERSION 0xe02007 /* Size of replay log header */ #define HEADER_SIZE (sizeof(uint32_t) + sizeof(uint64_t)) From patchwork Tue Nov 14 08:17:55 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pavel Dovgalyuk X-Patchwork-Id: 118845 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp2763661qgn; Tue, 14 Nov 2017 00:25:26 -0800 (PST) X-Google-Smtp-Source: AGs4zMaYHvNnUpUuf7ebxMWdPMcTJTP2WzJFUNH7rR2apGckqMLbKVBtku0w1jLBxJ2tryVc2m75 X-Received: by 10.129.125.212 with SMTP id y203mr7128597ywc.510.1510647926092; Tue, 14 Nov 2017 00:25:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510647926; cv=none; d=google.com; s=arc-20160816; b=CmEwPtNW4bOssm0C5W4B50LKNinqpiBk09uD5BT0c3mx/KnLb5OAt0YYT1og8zF7ee aymmq8/M1ID3vHRcu7Oewl8A0DINMwYtT87N/S8EenyXvUIiC9A+wgMNbnMKmKy5bdUx gOOOdHx7kHGdyDouOa9rcXbbpEn3c+VxU4CNnYG7umFIFWxkRYqDoqHRM2bZRDW1MDZh 2LgVSw1jDfefRZ+/qkZ8u3dC0AfgSiW+X2wPcOYgQ5rRcHRS3cSUwKODPrD5CsMgYxTG uC6TDtymEga3hjplVt5xsm3OeRR7rvYw4H+bOxTMXrcC/MhtG7gy8NlyuBxOrqRHY1mB EkzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=eaWXh7eJ96bPE1dM/ljeqUReo1DLKNLjyU5VNVCDfdE=; b=vcrt/ckBwP4KdXr1xZG0nk+kLvHwOe+fKjoBSA3Xv/hf95lxtKEYaZCX7mdW1zPzmY H7krYke3MC5W0pTzoJZ9qgQkmI/7JyuaxO+vBhHCpRu3Bxi7oVPwN5f/6/cqOClJT5C1 xwB/0hyNMWC9F9zvI8/CclytgBOy9jHhWeqkfbVe7AudN33pN+SKwI6aoDcGs+7T5sNR k1ArSDt39/9vcxNmwt2bxe+UhIMu072g5TFxt/jkooSvI/vmUkustdTbi7FnvWx+dm8Z eTW1Y7BQM9WXpQP9N1qnK1AbLaSeI2IDuvv+ifX241fEKR/tVCec2RjVfwTkfOHWVI/s SVRw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id 1si97927ywm.453.2017.11.14.00.25.26 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 14 Nov 2017 00:25:26 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:58192 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWX7-0006nj-Im for patch@linaro.org; Tue, 14 Nov 2017 03:25:25 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40657) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPo-0000AX-C6 for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:53 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEWPk-0006cv-Db for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:52 -0500 Received: from mail.ispras.ru ([83.149.199.45]:52710) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPk-0006cN-6L for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:48 -0500 Received: from [127.0.1.1] (unknown [85.142.117.226]) by mail.ispras.ru (Postfix) with ESMTPSA id 4992154006A; Tue, 14 Nov 2017 11:17:47 +0300 (MSK) From: Pavel Dovgalyuk To: qemu-devel@nongnu.org Date: Tue, 14 Nov 2017 11:17:55 +0300 Message-ID: <20171114081755.27640.21686.stgit@pasha-VirtualBox> In-Reply-To: <20171114081630.27640.53933.stgit@pasha-VirtualBox> References: <20171114081630.27640.53933.stgit@pasha-VirtualBox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 83.149.199.45 Subject: [Qemu-devel] [RFC PATCH v2 15/26] replay/replay-internal.c: track holding of replay_lock X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, pavel.dovgaluk@ispras.ru, mst@redhat.com, jasowang@redhat.com, quintela@redhat.com, zuban32s@gmail.com, maria.klimushenkova@ispras.ru, dovgaluk@ispras.ru, kraxel@redhat.com, boost.lists@gmail.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée This is modelled after the iothread mutex lock. We keep a TLS flag to indicate when that thread has acquired the lock and assert we don't double-lock or release when we shouldn't have. Signed-off-by: Alex Bennée Tested-by: Pavel Dovgalyuk --- replay/replay-internal.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/replay/replay-internal.c b/replay/replay-internal.c index fca8514..157c863 100644 --- a/replay/replay-internal.c +++ b/replay/replay-internal.c @@ -179,13 +179,24 @@ void replay_mutex_destroy(void) qemu_mutex_destroy(&lock); } +static __thread bool replay_locked; + +static bool replay_mutex_locked(void) +{ + return replay_locked; +} + void replay_mutex_lock(void) { + g_assert(!replay_mutex_locked()); qemu_mutex_lock(&lock); + replay_locked = true; } void replay_mutex_unlock(void) { + g_assert(replay_mutex_locked()); + replay_locked = false; qemu_mutex_unlock(&lock); } From patchwork Tue Nov 14 08:18:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pavel Dovgalyuk X-Patchwork-Id: 118849 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp2766727qgn; Tue, 14 Nov 2017 00:29:44 -0800 (PST) X-Google-Smtp-Source: AGs4zMZh2SezlcIk533nVaNn7r1Dn+PXIbhChYEjOxKYHdJxxh8GSJJQARYzs+cShyTe+runcp+7 X-Received: by 10.129.227.76 with SMTP id w12mr7619193ywl.81.1510648184693; Tue, 14 Nov 2017 00:29:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510648184; cv=none; d=google.com; s=arc-20160816; b=Enieo5Ilqk7jg+mPA8KgXLxVS8ScgE3Aer9svxsNIYCoDDKxb75CS3i9sHj4QIABay QX7tE9qzZghRwGdi+q50Q1h4HlLz4e7Mfo54NGKLn5XNpqT8EXDGvn4fOlaWI05mLi8v kSbB9jOkPgu0cTJnnKg1jB76i2m1QZSvBgSvPOSC6Z2uE5i9hQCIWYpW2/4F2NAAwOa0 iMXhHKhOWQ4cDiRzAHXvp9On+DQ7tOdqJglZPfpXDyIbPBa/3ynZCHNKJGRApW0KB4uv RuJ7PtBiXAQXsnn1J6RzbQBpiJ0BCH1z8UqiAWfXXAOfEeM5NH3v4TIwKfjuD//KsNk2 dQ1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=XVGeX3Cfx8cqy8mesBOY0NylnirozKMOBlZ86Slq83w=; b=Yu1url0SSUstl8X1Abvs/grj8iX12P2LPJmhYSzQHWXtbphEbthFs/peil0NGEtVrh Xw7VhREQtj0j4shygCP87sMWyDvJJijGOS56FmeCs3Qiz081zQqRICEau0K4lkTDJ/oX +rKoxiY7tzX1CdYkubq+fO2rM6sXigARGCADJ1JFhPubV67P9MWLvExyhuG8x3cLy7a1 wmR3XBtkHMRDvyn1TyL0N74oxDjPDQxfnNUEaaa8AznjbhK1LcqY1wfcMyI4Zewh9C6I UG2JuRLMxi6hX6PtCanZ2d31b4uurzRw2My2lYkUSGV8vicrRcGVROB4Ttf8ZC/+v9GP gN8g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id e68si761986ywh.691.2017.11.14.00.29.44 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 14 Nov 2017 00:29:44 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:58221 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWbI-000352-7m for patch@linaro.org; Tue, 14 Nov 2017 03:29:44 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40718) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPu-0000Hj-P2 for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEWPq-0006iI-Jc for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:58 -0500 Received: from mail.ispras.ru ([83.149.199.45]:52728) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPq-0006hq-6A for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:17:54 -0500 Received: from [127.0.1.1] (unknown [85.142.117.226]) by mail.ispras.ru (Postfix) with ESMTPSA id E083C54006A; Tue, 14 Nov 2017 11:17:52 +0300 (MSK) From: Pavel Dovgalyuk To: qemu-devel@nongnu.org Date: Tue, 14 Nov 2017 11:18:01 +0300 Message-ID: <20171114081801.27640.98687.stgit@pasha-VirtualBox> In-Reply-To: <20171114081630.27640.53933.stgit@pasha-VirtualBox> References: <20171114081630.27640.53933.stgit@pasha-VirtualBox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 83.149.199.45 Subject: [Qemu-devel] [RFC PATCH v2 16/26] replay: make locking visible outside replay code X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, pavel.dovgaluk@ispras.ru, mst@redhat.com, jasowang@redhat.com, quintela@redhat.com, zuban32s@gmail.com, maria.klimushenkova@ispras.ru, dovgaluk@ispras.ru, kraxel@redhat.com, boost.lists@gmail.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée The replay_mutex_lock/unlock/locked functions are now going to be used for ensuring lock-step behaviour between the two threads. Make them public API functions and also provide stubs for non-QEMU builds on common paths. Signed-off-by: Alex Bennée Tested-by: Pavel Dovgalyuk --- include/sysemu/replay.h | 14 ++++++++++++++ replay/replay-internal.c | 9 ++++----- replay/replay-internal.h | 5 ++--- stubs/replay.c | 15 +++++++++++++++ 4 files changed, 35 insertions(+), 8 deletions(-) diff --git a/include/sysemu/replay.h b/include/sysemu/replay.h index b86d6bb..9973849 100644 --- a/include/sysemu/replay.h +++ b/include/sysemu/replay.h @@ -47,6 +47,20 @@ extern ReplayMode replay_mode; /* Name of the initial VM snapshot */ extern char *replay_snapshot; +/* Replay locking + * + * The locks are needed to protect the shared structures and log file + * when doing record/replay. They also are the main sync-point between + * the main-loop thread and the vCPU thread. This was a role + * previously filled by the BQL which has been busy trying to reduce + * its impact across the code. This ensures blocks of events stay + * sequential and reproducible. + */ + +void replay_mutex_lock(void); +void replay_mutex_unlock(void); +bool replay_mutex_locked(void); + /* Replay process control functions */ /*! Enables recording or saving event log with specified parameters */ diff --git a/replay/replay-internal.c b/replay/replay-internal.c index 157c863..a9a6a64 100644 --- a/replay/replay-internal.c +++ b/replay/replay-internal.c @@ -169,6 +169,8 @@ void replay_finish_event(void) replay_fetch_data_kind(); } +static __thread bool replay_locked; + void replay_mutex_init(void) { qemu_mutex_init(&lock); @@ -179,9 +181,7 @@ void replay_mutex_destroy(void) qemu_mutex_destroy(&lock); } -static __thread bool replay_locked; - -static bool replay_mutex_locked(void) +bool replay_mutex_locked(void) { return replay_locked; } @@ -204,7 +204,7 @@ void replay_mutex_unlock(void) void replay_save_instructions(void) { if (replay_file && replay_mode == REPLAY_MODE_RECORD) { - replay_mutex_lock(); + g_assert(replay_mutex_locked()); int diff = (int)(replay_get_current_step() - replay_state.current_step); /* Time can only go forward */ @@ -215,6 +215,5 @@ void replay_save_instructions(void) replay_put_dword(diff); replay_state.current_step += diff; } - replay_mutex_unlock(); } } diff --git a/replay/replay-internal.h b/replay/replay-internal.h index 8e4c701..f5f8e96 100644 --- a/replay/replay-internal.h +++ b/replay/replay-internal.h @@ -100,12 +100,11 @@ int64_t replay_get_qword(void); void replay_get_array(uint8_t *buf, size_t *size); void replay_get_array_alloc(uint8_t **buf, size_t *size); -/* Mutex functions for protecting replay log file */ +/* Mutex functions for protecting replay log file and ensuring + * synchronisation between vCPU and main-loop threads. */ void replay_mutex_init(void); void replay_mutex_destroy(void); -void replay_mutex_lock(void); -void replay_mutex_unlock(void); /*! Checks error status of the file. */ void replay_check_error(void); diff --git a/stubs/replay.c b/stubs/replay.c index 9991ee5..cb050ef 100644 --- a/stubs/replay.c +++ b/stubs/replay.c @@ -73,3 +73,18 @@ uint64_t blkreplay_next_id(void) { return 0; } + +void replay_mutex_lock(void) +{ + abort(); +} + +void replay_mutex_unlock(void) +{ + abort(); +} + +bool replay_mutex_locked(void) +{ + return false; +} From patchwork Tue Nov 14 08:18:07 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pavel Dovgalyuk X-Patchwork-Id: 118847 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp2765565qgn; Tue, 14 Nov 2017 00:28:05 -0800 (PST) X-Google-Smtp-Source: AGs4zMZTlbcnb9eNBsADymU6Gw/8c5teDSJzGCKv5+svU7LyXKfVwoVKyfYNj8TH+3AFZQcUTIH+ X-Received: by 10.129.163.144 with SMTP id a138mr6332424ywh.238.1510648085187; Tue, 14 Nov 2017 00:28:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510648085; cv=none; d=google.com; s=arc-20160816; b=k1zcJX/shaJNUpxmzhCcPT5PgSsA56ISHtoyfK7ATXgTt0jyI4qUhVcHiECSm8jVDH OvSjDvIBwfvgUnrXIEinjtr9GhQKY/iSAT6DulW5NS3yX2tx6Gca8XBL5OiY5zELJboy +DhXxUUKHD4zyb8RLe/Xo3ixQt7aH4bpPpBEfv0upWUCY9yOs7tx5fNhpSeA3l3YspOp iVFe2xM79FgRCgfZkQht3W9nWXLYnb/EbNNs9OYW3dLO0lTT+u2dbgDcSpPKT0/SV87J d/AxUpfkQnS6mTUQIZXo9vgnfaYG95eRww2+CGTBCLGUEs50QSP/e2z46rkuSmEbNNh1 qAHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=31MPdGRF39GLqQh5VtT+zjJ916SJf1kDaCQaDm007RE=; b=tWtQQKI/MRS5mDu/isEybBeq06xI2PBaoutK7NuaRA8jlpm8jTbPL1TFJAmnB87bg0 xk5rYkDbGlkxQsUe3FppCxD530zETVMYAb3MTG0O8CW5P3JtY1QxUNbuWDk8baKdbufz cgrAZtVgSZ6pgMMM+Gs3i9fs7HC4Ck7iPDQlegeWN4h0/r3KM474Be8IJFQAw6aCMhN1 K2uqPPUJ08Rtza8KJOeoJLVv3EmroCFh7iXtNuFJhm+4YcggyibxX4V6qiTP3Ni4CW9b EhBhKoADOdYrg1vVRZSPvZJuijnyUkSHYa2nZaaklvXorbsVRDuGrdZ/TbN9bPjPqPo8 htNg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id m64si3709659yba.816.2017.11.14.00.28.04 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 14 Nov 2017 00:28:04 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:58211 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWZf-0000y8-El for patch@linaro.org; Tue, 14 Nov 2017 03:28:03 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40809) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWQ1-0000NA-Ey for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:11 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEWPw-0006mh-6G for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:05 -0500 Received: from mail.ispras.ru ([83.149.199.45]:52750) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWPv-0006mE-Kk for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:00 -0500 Received: from [127.0.1.1] (unknown [85.142.117.226]) by mail.ispras.ru (Postfix) with ESMTPSA id B03B154006A; Tue, 14 Nov 2017 11:17:58 +0300 (MSK) From: Pavel Dovgalyuk To: qemu-devel@nongnu.org Date: Tue, 14 Nov 2017 11:18:07 +0300 Message-ID: <20171114081807.27640.41031.stgit@pasha-VirtualBox> In-Reply-To: <20171114081630.27640.53933.stgit@pasha-VirtualBox> References: <20171114081630.27640.53933.stgit@pasha-VirtualBox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 83.149.199.45 Subject: [Qemu-devel] [RFC PATCH v2 17/26] replay: push replay_mutex_lock up the call tree X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, pavel.dovgaluk@ispras.ru, mst@redhat.com, jasowang@redhat.com, quintela@redhat.com, zuban32s@gmail.com, maria.klimushenkova@ispras.ru, dovgaluk@ispras.ru, kraxel@redhat.com, boost.lists@gmail.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée Now instead of using the replay_lock to guard the output of the log we now use it to protect the whole execution section. This replaces what the BQL used to do when it was held during TCG execution. We also introduce some rules for locking order - mainly that you cannot take the replay_mutex while holding the BQL. This leads to some slight sophistry during start-up and extending the replay_mutex_destroy function to unlock the mutex without checking for the BQL condition so it can be cleanly dropped in the non-replay case. Signed-off-by: Alex Bennée Tested-by: Pavel Dovgalyuk -- v2: updated replay_mutex_lock/unlock functions as suggested by Paolo Bonzini updated docs --- cpus.c | 17 +++++++++++++++++ docs/replay.txt | 22 ++++++++++++++++++++++ include/sysemu/replay.h | 2 ++ replay/replay-char.c | 21 ++++++++------------- replay/replay-events.c | 18 +++++------------- replay/replay-internal.c | 33 +++++++++++++++++++++++---------- replay/replay-time.c | 10 +++++----- replay/replay.c | 39 +++++++++++++++++++-------------------- util/main-loop.c | 17 +++++++++++++---- vl.c | 2 ++ 10 files changed, 116 insertions(+), 65 deletions(-) diff --git a/cpus.c b/cpus.c index 1800a3b..75a5e63 100644 --- a/cpus.c +++ b/cpus.c @@ -1293,6 +1293,8 @@ static void prepare_icount_for_run(CPUState *cpu) insns_left = MIN(0xffff, cpu->icount_budget); cpu->icount_decr.u16.low = insns_left; cpu->icount_extra = cpu->icount_budget - insns_left; + + replay_mutex_lock(); } } @@ -1308,6 +1310,8 @@ static void process_icount_data(CPUState *cpu) cpu->icount_budget = 0; replay_account_executed_instructions(); + + replay_mutex_unlock(); } } @@ -1395,6 +1399,8 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) while (1) { + replay_mutex_lock(); + qemu_mutex_lock_iothread(); /* Account partial waits to QEMU_CLOCK_VIRTUAL. */ @@ -1407,6 +1413,8 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) qemu_mutex_unlock_iothread(); + replay_mutex_unlock(); + if (!cpu) { cpu = first_cpu; } @@ -1677,12 +1685,21 @@ void pause_all_vcpus(void) cpu_stop_current(); } + /* We need to drop the replay_lock so any vCPU threads woken up + * can finish their replay tasks + */ + replay_mutex_unlock(); + while (!all_vcpus_paused()) { qemu_cond_wait(&qemu_pause_cond, &qemu_global_mutex); CPU_FOREACH(cpu) { qemu_cpu_kick(cpu); } } + + qemu_mutex_unlock_iothread(); + replay_mutex_lock(); + qemu_mutex_lock_iothread(); } void cpu_resume(CPUState *cpu) diff --git a/docs/replay.txt b/docs/replay.txt index c52407f..959633e 100644 --- a/docs/replay.txt +++ b/docs/replay.txt @@ -49,6 +49,28 @@ Modifications of qemu include: * recording/replaying user input (mouse and keyboard) * adding internal checkpoints for cpu and io synchronization +Locking and thread synchronisation +---------------------------------- + +Previously the synchronisation of the main thread and the vCPU thread +was ensured by the holding of the BQL. However the trend has been to +reduce the time the BQL was held across the system including under TCG +system emulation. As it is important that batches of events are kept +in sequence (e.g. expiring timers and checkpoints in the main thread +while instruction checkpoints are written by the vCPU thread) we need +another lock to keep things in lock-step. This role is now handled by +the replay_mutex_lock. It used to be held only for each event being +written but now it is held for a whole execution period. This results +in a deterministic ping-pong between the two main threads. + +As the BQL is now a finer grained lock than the replay_lock it is almost +certainly a bug, and a source of deadlocks, to take the +replay_mutex_lock while the BQL is held. This is enforced by an assert. +While the unlocks are usually in the reverse order, this is not +necessary; you can drop the replay_lock while holding the BQL, without +doing a more complicated unlock_iothread/replay_unlock/lock_iothread +sequence. + Non-deterministic events ------------------------ diff --git a/include/sysemu/replay.h b/include/sysemu/replay.h index 9973849..d026b28 100644 --- a/include/sysemu/replay.h +++ b/include/sysemu/replay.h @@ -63,6 +63,8 @@ bool replay_mutex_locked(void); /* Replay process control functions */ +/*! Enables and take replay locks (even if we don't use it) */ +void replay_init_locks(void); /*! Enables recording or saving event log with specified parameters */ void replay_configure(struct QemuOpts *opts); /*! Initializes timers used for snapshotting and enables events recording */ diff --git a/replay/replay-char.c b/replay/replay-char.c index cbf7c04..736cc8c 100755 --- a/replay/replay-char.c +++ b/replay/replay-char.c @@ -96,25 +96,24 @@ void *replay_event_char_read_load(void) void replay_char_write_event_save(int res, int offset) { + g_assert(replay_mutex_locked()); + replay_save_instructions(); - replay_mutex_lock(); replay_put_event(EVENT_CHAR_WRITE); replay_put_dword(res); replay_put_dword(offset); - replay_mutex_unlock(); } void replay_char_write_event_load(int *res, int *offset) { + g_assert(replay_mutex_locked()); + replay_account_executed_instructions(); - replay_mutex_lock(); if (replay_next_event_is(EVENT_CHAR_WRITE)) { *res = replay_get_dword(); *offset = replay_get_dword(); replay_finish_event(); - replay_mutex_unlock(); } else { - replay_mutex_unlock(); error_report("Missing character write event in the replay log"); exit(1); } @@ -122,23 +121,21 @@ void replay_char_write_event_load(int *res, int *offset) int replay_char_read_all_load(uint8_t *buf) { - replay_mutex_lock(); + g_assert(replay_mutex_locked()); + if (replay_next_event_is(EVENT_CHAR_READ_ALL)) { size_t size; int res; replay_get_array(buf, &size); replay_finish_event(); - replay_mutex_unlock(); res = (int)size; assert(res >= 0); return res; } else if (replay_next_event_is(EVENT_CHAR_READ_ALL_ERROR)) { int res = replay_get_dword(); replay_finish_event(); - replay_mutex_unlock(); return res; } else { - replay_mutex_unlock(); error_report("Missing character read all event in the replay log"); exit(1); } @@ -146,19 +143,17 @@ int replay_char_read_all_load(uint8_t *buf) void replay_char_read_all_save_error(int res) { + g_assert(replay_mutex_locked()); assert(res < 0); replay_save_instructions(); - replay_mutex_lock(); replay_put_event(EVENT_CHAR_READ_ALL_ERROR); replay_put_dword(res); - replay_mutex_unlock(); } void replay_char_read_all_save_buf(uint8_t *buf, int offset) { + g_assert(replay_mutex_locked()); replay_save_instructions(); - replay_mutex_lock(); replay_put_event(EVENT_CHAR_READ_ALL); replay_put_array(buf, offset); - replay_mutex_unlock(); } diff --git a/replay/replay-events.c b/replay/replay-events.c index e858254..a941efb 100644 --- a/replay/replay-events.c +++ b/replay/replay-events.c @@ -79,16 +79,14 @@ bool replay_has_events(void) void replay_flush_events(void) { - replay_mutex_lock(); + g_assert(replay_mutex_locked()); + while (!QTAILQ_EMPTY(&events_list)) { Event *event = QTAILQ_FIRST(&events_list); - replay_mutex_unlock(); replay_run_event(event); - replay_mutex_lock(); QTAILQ_REMOVE(&events_list, event, events); g_free(event); } - replay_mutex_unlock(); } void replay_disable_events(void) @@ -102,14 +100,14 @@ void replay_disable_events(void) void replay_clear_events(void) { - replay_mutex_lock(); + g_assert(replay_mutex_locked()); + while (!QTAILQ_EMPTY(&events_list)) { Event *event = QTAILQ_FIRST(&events_list); QTAILQ_REMOVE(&events_list, event, events); g_free(event); } - replay_mutex_unlock(); } /*! Adds specified async event to the queue */ @@ -136,9 +134,8 @@ void replay_add_event(ReplayAsyncEventKind event_kind, event->opaque2 = opaque2; event->id = id; - replay_mutex_lock(); + g_assert(replay_mutex_locked()); QTAILQ_INSERT_TAIL(&events_list, event, events); - replay_mutex_unlock(); } void replay_bh_schedule_event(QEMUBH *bh) @@ -210,10 +207,7 @@ void replay_save_events(int checkpoint) while (!QTAILQ_EMPTY(&events_list)) { Event *event = QTAILQ_FIRST(&events_list); replay_save_event(event, checkpoint); - - replay_mutex_unlock(); replay_run_event(event); - replay_mutex_lock(); QTAILQ_REMOVE(&events_list, event, events); g_free(event); } @@ -299,9 +293,7 @@ void replay_read_events(int checkpoint) } replay_finish_event(); read_event_kind = -1; - replay_mutex_unlock(); replay_run_event(event); - replay_mutex_lock(); g_free(event); } diff --git a/replay/replay-internal.c b/replay/replay-internal.c index a9a6a64..a1a7686 100644 --- a/replay/replay-internal.c +++ b/replay/replay-internal.c @@ -174,30 +174,43 @@ static __thread bool replay_locked; void replay_mutex_init(void) { qemu_mutex_init(&lock); + /* Hold the mutex while we start-up */ + qemu_mutex_lock(&lock); + replay_locked = true; } -void replay_mutex_destroy(void) +bool replay_mutex_locked(void) { - qemu_mutex_destroy(&lock); + return replay_locked; } -bool replay_mutex_locked(void) +void replay_mutex_destroy(void) { - return replay_locked; + if (replay_mutex_locked()) { + qemu_mutex_unlock(&lock); + } + qemu_mutex_destroy(&lock); } +/* Ordering constraints, replay_lock must be taken before BQL */ void replay_mutex_lock(void) { - g_assert(!replay_mutex_locked()); - qemu_mutex_lock(&lock); - replay_locked = true; + if (replay_mode != REPLAY_MODE_NONE) { + g_assert(!qemu_mutex_iothread_locked()); + g_assert(!replay_mutex_locked()); + qemu_mutex_lock(&lock); + replay_locked = true; + } } +/* BQL can't be held when releasing the replay_lock */ void replay_mutex_unlock(void) { - g_assert(replay_mutex_locked()); - replay_locked = false; - qemu_mutex_unlock(&lock); + if (replay_mode != REPLAY_MODE_NONE) { + g_assert(replay_mutex_locked()); + replay_locked = false; + qemu_mutex_unlock(&lock); + } } /*! Saves cached instructions. */ diff --git a/replay/replay-time.c b/replay/replay-time.c index f70382a..6a7565e 100644 --- a/replay/replay-time.c +++ b/replay/replay-time.c @@ -17,13 +17,13 @@ int64_t replay_save_clock(ReplayClockKind kind, int64_t clock) { - replay_save_instructions(); if (replay_file) { - replay_mutex_lock(); + g_assert(replay_mutex_locked()); + + replay_save_instructions(); replay_put_event(EVENT_CLOCK + kind); replay_put_qword(clock); - replay_mutex_unlock(); } return clock; @@ -46,16 +46,16 @@ void replay_read_next_clock(ReplayClockKind kind) /*! Reads next clock event from the input. */ int64_t replay_read_clock(ReplayClockKind kind) { + g_assert(replay_file && replay_mutex_locked()); + replay_account_executed_instructions(); if (replay_file) { int64_t ret; - replay_mutex_lock(); if (replay_next_event_is(EVENT_CLOCK + kind)) { replay_read_next_clock(kind); } ret = replay_state.cached_clock[kind]; - replay_mutex_unlock(); return ret; } diff --git a/replay/replay.c b/replay/replay.c index 4f24498..6e34b79 100644 --- a/replay/replay.c +++ b/replay/replay.c @@ -80,8 +80,9 @@ int replay_get_instructions(void) void replay_account_executed_instructions(void) { + g_assert(replay_mutex_locked()); + if (replay_mode == REPLAY_MODE_PLAY) { - replay_mutex_lock(); if (replay_state.instructions_count > 0) { int count = (int)(replay_get_current_step() - replay_state.current_step); @@ -100,24 +101,22 @@ void replay_account_executed_instructions(void) qemu_notify_event(); } } - replay_mutex_unlock(); } } bool replay_exception(void) { + if (replay_mode == REPLAY_MODE_RECORD) { + g_assert(replay_mutex_locked()); replay_save_instructions(); - replay_mutex_lock(); replay_put_event(EVENT_EXCEPTION); - replay_mutex_unlock(); return true; } else if (replay_mode == REPLAY_MODE_PLAY) { + g_assert(replay_mutex_locked()); bool res = replay_has_exception(); if (res) { - replay_mutex_lock(); replay_finish_event(); - replay_mutex_unlock(); } return res; } @@ -129,10 +128,9 @@ bool replay_has_exception(void) { bool res = false; if (replay_mode == REPLAY_MODE_PLAY) { + g_assert(replay_mutex_locked()); replay_account_executed_instructions(); - replay_mutex_lock(); res = replay_next_event_is(EVENT_EXCEPTION); - replay_mutex_unlock(); } return res; @@ -141,17 +139,15 @@ bool replay_has_exception(void) bool replay_interrupt(void) { if (replay_mode == REPLAY_MODE_RECORD) { + g_assert(replay_mutex_locked()); replay_save_instructions(); - replay_mutex_lock(); replay_put_event(EVENT_INTERRUPT); - replay_mutex_unlock(); return true; } else if (replay_mode == REPLAY_MODE_PLAY) { + g_assert(replay_mutex_locked()); bool res = replay_has_interrupt(); if (res) { - replay_mutex_lock(); replay_finish_event(); - replay_mutex_unlock(); } return res; } @@ -163,10 +159,9 @@ bool replay_has_interrupt(void) { bool res = false; if (replay_mode == REPLAY_MODE_PLAY) { + g_assert(replay_mutex_locked()); replay_account_executed_instructions(); - replay_mutex_lock(); res = replay_next_event_is(EVENT_INTERRUPT); - replay_mutex_unlock(); } return res; } @@ -174,9 +169,8 @@ bool replay_has_interrupt(void) void replay_shutdown_request(ShutdownCause cause) { if (replay_mode == REPLAY_MODE_RECORD) { - replay_mutex_lock(); + g_assert(replay_mutex_locked()); replay_put_event(EVENT_SHUTDOWN + cause); - replay_mutex_unlock(); } } @@ -190,9 +184,9 @@ bool replay_checkpoint(ReplayCheckpoint checkpoint) return true; } - replay_mutex_lock(); if (replay_mode == REPLAY_MODE_PLAY) { + g_assert(replay_mutex_locked()); if (replay_next_event_is(EVENT_CHECKPOINT + checkpoint)) { replay_finish_event(); } else if (replay_state.data_kind != EVENT_ASYNC) { @@ -205,15 +199,20 @@ bool replay_checkpoint(ReplayCheckpoint checkpoint) checkpoint were processed */ res = replay_state.data_kind != EVENT_ASYNC; } else if (replay_mode == REPLAY_MODE_RECORD) { + g_assert(replay_mutex_locked()); replay_put_event(EVENT_CHECKPOINT + checkpoint); replay_save_events(checkpoint); res = true; } out: - replay_mutex_unlock(); return res; } +void replay_init_locks(void) +{ + replay_mutex_init(); +} + static void replay_enable(const char *fname, int mode) { const char *fmode = NULL; @@ -233,8 +232,6 @@ static void replay_enable(const char *fname, int mode) atexit(replay_finish); - replay_mutex_init(); - replay_file = fopen(fname, fmode); if (replay_file == NULL) { fprintf(stderr, "Replay: open %s: %s\n", fname, strerror(errno)); @@ -274,6 +271,8 @@ void replay_configure(QemuOpts *opts) Location loc; if (!opts) { + /* we no longer need this lock */ + replay_mutex_destroy(); return; } diff --git a/util/main-loop.c b/util/main-loop.c index 7558eb5..1e928b3 100644 --- a/util/main-loop.c +++ b/util/main-loop.c @@ -29,6 +29,7 @@ #include "qemu/sockets.h" // struct in_addr needed for libslirp.h #include "sysemu/qtest.h" #include "sysemu/cpus.h" +#include "sysemu/replay.h" #include "slirp/libslirp.h" #include "qemu/main-loop.h" #include "block/aio.h" @@ -245,18 +246,21 @@ static int os_host_main_loop_wait(int64_t timeout) timeout = SCALE_MS; } + if (timeout) { spin_counter = 0; - qemu_mutex_unlock_iothread(); } else { spin_counter++; } + qemu_mutex_unlock_iothread(); + + replay_mutex_unlock(); ret = qemu_poll_ns((GPollFD *)gpollfds->data, gpollfds->len, timeout); - if (timeout) { - qemu_mutex_lock_iothread(); - } + replay_mutex_lock(); + + qemu_mutex_lock_iothread(); glib_pollfds_poll(); @@ -463,8 +467,13 @@ static int os_host_main_loop_wait(int64_t timeout) poll_timeout_ns = qemu_soonest_timeout(poll_timeout_ns, timeout); qemu_mutex_unlock_iothread(); + + replay_mutex_unlock(); + g_poll_ret = qemu_poll_ns(poll_fds, n_poll_fds + w->num, poll_timeout_ns); + replay_mutex_lock(); + qemu_mutex_lock_iothread(); if (g_poll_ret > 0) { for (i = 0; i < w->num; i++) { diff --git a/vl.c b/vl.c index a8e0d03..77fc1ef 100644 --- a/vl.c +++ b/vl.c @@ -3137,6 +3137,8 @@ int main(int argc, char **argv, char **envp) qemu_init_cpu_list(); qemu_init_cpu_loop(); + + replay_init_locks(); qemu_mutex_lock_iothread(); atexit(qemu_run_exit_notifiers); From patchwork Tue Nov 14 08:18:35 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pavel Dovgalyuk X-Patchwork-Id: 118851 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp2772074qgn; Tue, 14 Nov 2017 00:36:24 -0800 (PST) X-Google-Smtp-Source: AGs4zMaxmFcU763R1y3lZccBb3q/MS4yleFqHBeq4zu1aMe1VF4/VwZN7gKRmgPMuiCnG2G75nJk X-Received: by 10.37.2.140 with SMTP id 134mr2512840ybc.327.1510648584940; Tue, 14 Nov 2017 00:36:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510648584; cv=none; d=google.com; s=arc-20160816; b=iHRk4SWBP6qJgft0i0QGIXP/LJ9zxWsGO8K83PYpm4RQ01FlEWk0iyAFnu50r8/4lD 1jvckRjg+5r8dt/50AnbdPcw0TSRCm2Boh2viNr9NTpOWkNBogr9wiDYH+KdpyVTd/mK pMGr5l22fZdGUNmqcolhq+YnN5kpA4mfOd6NQVNFYMozrYe/t70L5mK0HhC6J44GFlMk Kis+JNM76XsNvncYodvsrLvmMQQYmcI6p/ookgW+/6vc0lcuaEs4SvNBKmTVX4R7Smoq EFVNZ6g2v7wK6Q13dQ7VWu3u6bW5HhVwU6JgbOr7+aTi3FstUnYIlI4AF4uApKqD3hfg iiDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=8/Vw8eOcmoeZm82a8jz9Lqv6EaUsIFnni1lghWpg+Ng=; b=bcW77ajVaeXnMUJpltG0YXKbg6+gDd4QVwklz8/P/AnMOQfpswsBeQNWfsopBGrpfg YOgNYwV2ZLaUM0JGqyROyywwrIwHzX0zA+lCTrrxtWwPyMvLcBva1zqJU2hi1c3joLus FadCkffT08m4g80nPeQciDeULqWAHdOxECDr2RtwlsjuAi8IOm2kwqv4S8QMU/iiEXTQ J8S9KM0yRS2OH3PM1ZK2yKg3QWDK1eNK0pgJEsI8qk8wWgpiVPewARy1D6nFCp22pD/1 b5q/50DUZFUy9iAmaqWKbvhCVawBICktVBsJ3rZI7RaU1oE19FcoycY7B9xapIiDchmA tbHw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id j13si3888913ybl.208.2017.11.14.00.36.24 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 14 Nov 2017 00:36:24 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:58263 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWhk-00010f-Fi for patch@linaro.org; Tue, 14 Nov 2017 03:36:24 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41080) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWQO-0000ff-Sr for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEWQN-0007B5-Pi for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:28 -0500 Received: from mail.ispras.ru ([83.149.199.45]:52842) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWQN-0007Al-Hn for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:27 -0500 Received: from [127.0.1.1] (unknown [85.142.117.226]) by mail.ispras.ru (Postfix) with ESMTPSA id B5ED054006A; Tue, 14 Nov 2017 11:18:26 +0300 (MSK) From: Pavel Dovgalyuk To: qemu-devel@nongnu.org Date: Tue, 14 Nov 2017 11:18:35 +0300 Message-ID: <20171114081835.27640.75770.stgit@pasha-VirtualBox> In-Reply-To: <20171114081630.27640.53933.stgit@pasha-VirtualBox> References: <20171114081630.27640.53933.stgit@pasha-VirtualBox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 83.149.199.45 Subject: [Qemu-devel] [RFC PATCH v2 22/26] scripts/qemu-gdb: add simple tcg lock status helper X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, pavel.dovgaluk@ispras.ru, mst@redhat.com, jasowang@redhat.com, quintela@redhat.com, zuban32s@gmail.com, maria.klimushenkova@ispras.ru, dovgaluk@ispras.ru, kraxel@redhat.com, boost.lists@gmail.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée Add a simple helper to dump lock state. Signed-off-by: Alex Bennée --- scripts/qemu-gdb.py | 3 ++- scripts/qemugdb/tcg.py | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+), 1 deletion(-) create mode 100644 scripts/qemugdb/tcg.py diff --git a/scripts/qemu-gdb.py b/scripts/qemu-gdb.py index b3f8e04..d58213e 100644 --- a/scripts/qemu-gdb.py +++ b/scripts/qemu-gdb.py @@ -26,7 +26,7 @@ import os, sys sys.path.append(os.path.dirname(__file__)) -from qemugdb import aio, mtree, coroutine +from qemugdb import aio, mtree, coroutine, tcg class QemuCommand(gdb.Command): '''Prefix for QEMU debug support commands''' @@ -38,6 +38,7 @@ QemuCommand() coroutine.CoroutineCommand() mtree.MtreeCommand() aio.HandlersCommand() +tcg.TCGLockStatusCommand() coroutine.CoroutineSPFunction() coroutine.CoroutinePCFunction() diff --git a/scripts/qemugdb/tcg.py b/scripts/qemugdb/tcg.py new file mode 100644 index 0000000..8c7f1d7 --- /dev/null +++ b/scripts/qemugdb/tcg.py @@ -0,0 +1,46 @@ +#!/usr/bin/python +# -*- coding: utf-8 -*- +# +# GDB debugging support, TCG status +# +# Copyright 2016 Linaro Ltd +# +# Authors: +# Alex Bennée +# +# This work is licensed under the terms of the GNU GPL, version 2. See +# the COPYING file in the top-level directory. +# +# Contributions after 2012-01-13 are licensed under the terms of the +# GNU GPL, version 2 or (at your option) any later version. + +# 'qemu tcg-lock-status' -- display the TCG lock status across threads + +import gdb + +class TCGLockStatusCommand(gdb.Command): + '''Display TCG Execution Status''' + def __init__(self): + gdb.Command.__init__(self, 'qemu tcg-lock-status', gdb.COMMAND_DATA, + gdb.COMPLETE_NONE) + + def invoke(self, arg, from_tty): + gdb.write("Thread, BQL (iothread_mutex), Replay, Blocked?\n") + for thread in gdb.inferiors()[0].threads(): + thread.switch() + + iothread = gdb.parse_and_eval("iothread_locked") + replay = gdb.parse_and_eval("replay_locked") + + frame = gdb.selected_frame() + if frame.name() == "__lll_lock_wait": + frame.older().select() + mutex = gdb.parse_and_eval("mutex") + owner = gdb.parse_and_eval("mutex->__data.__owner") + blocked = ("__lll_lock_wait waiting on %s from %d" % + (mutex, owner)) + else: + blocked = "not blocked" + + gdb.write("%d/%d, %s, %s, %s\n" % (thread.num, thread.ptid[1], + iothread, replay, blocked)) From patchwork Tue Nov 14 08:18:40 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pavel Dovgalyuk X-Patchwork-Id: 118853 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp2773578qgn; Tue, 14 Nov 2017 00:38:31 -0800 (PST) X-Google-Smtp-Source: AGs4zMYVLnvIRKgYfQ+QDVwKaNxbF8709jdu4X2DKWPx3FRfqzahLsZbX9zx4cD8ik11wvHHKhej X-Received: by 10.37.131.134 with SMTP id t6mr6184587ybk.179.1510648711611; Tue, 14 Nov 2017 00:38:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510648711; cv=none; d=google.com; s=arc-20160816; b=aXyuLnaRHC3ewB4r4mZ8hFLqB2I1bGfG2kgsWDMdd46fYHzs52/sGLugX9k0DeEsTQ 44D3vFWFt2Bw6SUzK/SapNNYnJXbyw6aXq3wVhcx6GZJLJr8wfi7ivbSCIoCqMog/up7 a1oM6O53RKgrItZx3FtsUWtru9oPRFJugeW3Vrga0kStRlQidmLfMncXyeZwyTfmnAnJ jS8CRqq8RmqtHeMwxQrXsiAaNlM98zoq+yTXCAtxPoYSReDULWpuSBW4JdoYW44YBKrg o/4hPtLQ60icTNtiyCKDr8w1CO6fidDSHV4F+eiPOMhDi74QwX0tZVydJV2rfEMvt0V6 GbNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=DxLEvgVZnhx6Th1Z2OcxhG+6S/6xQvOXu2TUf1VJeLk=; b=jRtdm6D22Uo4WsUw4WmuvRPIGvM0GyeX+UtJ2tIUlCd1a2+K/G/UlwnlH6BAagPANG aJcZL++bQUxWQE6QOxd6VuWSzM4xZtJ6LiimJE3rlRmZ1bfeXgLRsCDkmuhErFwMZSCg 767CTk+0pzyorlJ2Tx+2GsKWvsUo1baOrEGVs+YESrcIzwAd8tGRzNz+vibUVyXZs96L f9TnLpvnNgIVg3aFaZUR2VUmPVWpv6Oq/QjcJMEAxLbbkwWx+d0r1bJGC+VVn6zw4UlA 1Jq92sdH/s3H3gSXKf6XLhZfcUbkGyseyp5+5lHbXfIMbExNNjsb1BslZhXf1dxZ69kC Ipag== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id l9si17865ywa.49.2017.11.14.00.38.31 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 14 Nov 2017 00:38:31 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:58281 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWjn-0002gt-4i for patch@linaro.org; Tue, 14 Nov 2017 03:38:31 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41177) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWQW-0000mz-NZ for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:37 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEWQT-0007GP-Fk for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:36 -0500 Received: from mail.ispras.ru ([83.149.199.45]:52870) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWQT-0007Fo-40 for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:33 -0500 Received: from [127.0.1.1] (unknown [85.142.117.226]) by mail.ispras.ru (Postfix) with ESMTPSA id 475BF54006A; Tue, 14 Nov 2017 11:18:32 +0300 (MSK) From: Pavel Dovgalyuk To: qemu-devel@nongnu.org Date: Tue, 14 Nov 2017 11:18:40 +0300 Message-ID: <20171114081840.27640.76952.stgit@pasha-VirtualBox> In-Reply-To: <20171114081630.27640.53933.stgit@pasha-VirtualBox> References: <20171114081630.27640.53933.stgit@pasha-VirtualBox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 83.149.199.45 Subject: [Qemu-devel] [RFC PATCH v2 23/26] util/qemu-thread-*: add qemu_lock, locked and unlock trace events X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, pavel.dovgaluk@ispras.ru, mst@redhat.com, jasowang@redhat.com, quintela@redhat.com, zuban32s@gmail.com, maria.klimushenkova@ispras.ru, dovgaluk@ispras.ru, kraxel@redhat.com, boost.lists@gmail.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée Signed-off-by: Alex Bennée --- v1 - fix merge conflicts with existing tracing - add trylock/cond_wait traces --- include/qemu/thread.h | 14 ++++++++++---- util/qemu-thread-posix.c | 21 ++++++++++++--------- util/trace-events | 7 ++++--- 3 files changed, 26 insertions(+), 16 deletions(-) diff --git a/include/qemu/thread.h b/include/qemu/thread.h index 9910f49..c51a7f1 100644 --- a/include/qemu/thread.h +++ b/include/qemu/thread.h @@ -22,9 +22,13 @@ typedef struct QemuThread QemuThread; void qemu_mutex_init(QemuMutex *mutex); void qemu_mutex_destroy(QemuMutex *mutex); -void qemu_mutex_lock(QemuMutex *mutex); -int qemu_mutex_trylock(QemuMutex *mutex); -void qemu_mutex_unlock(QemuMutex *mutex); +int qemu_mutex_trylock_impl(QemuMutex *mutex, const char *file, const int line); +void qemu_mutex_lock_impl(QemuMutex *mutex, const char *file, const int line); +void qemu_mutex_unlock_impl(QemuMutex *mutex, const char *file, const int line); + +#define qemu_mutex_lock(mutex) qemu_mutex_lock_impl(mutex, __FILE__, __LINE__) +#define qemu_mutex_trylock(mutex) qemu_mutex_trylock_impl(mutex, __FILE__, __LINE__) +#define qemu_mutex_unlock(mutex) qemu_mutex_unlock_impl(mutex, __FILE__, __LINE__) /* Prototypes for other functions are in thread-posix.h/thread-win32.h. */ void qemu_rec_mutex_init(QemuRecMutex *mutex); @@ -39,7 +43,9 @@ void qemu_cond_destroy(QemuCond *cond); */ void qemu_cond_signal(QemuCond *cond); void qemu_cond_broadcast(QemuCond *cond); -void qemu_cond_wait(QemuCond *cond, QemuMutex *mutex); +void qemu_cond_wait_impl(QemuCond *cond, QemuMutex *mutex, const char *file, const int line); + +#define qemu_cond_wait(cond, mutex) qemu_cond_wait_impl(cond, mutex, __FILE__, __LINE__) void qemu_sem_init(QemuSemaphore *sem, int init); void qemu_sem_post(QemuSemaphore *sem); diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c index 7306475..1a838a9 100644 --- a/util/qemu-thread-posix.c +++ b/util/qemu-thread-posix.c @@ -57,26 +57,28 @@ void qemu_mutex_destroy(QemuMutex *mutex) error_exit(err, __func__); } -void qemu_mutex_lock(QemuMutex *mutex) +void qemu_mutex_lock_impl(QemuMutex *mutex, const char *file, const int line) { int err; assert(mutex->initialized); + trace_qemu_mutex_lock(mutex, file, line); + err = pthread_mutex_lock(&mutex->lock); if (err) error_exit(err, __func__); - trace_qemu_mutex_locked(mutex); + trace_qemu_mutex_locked(mutex, file, line); } -int qemu_mutex_trylock(QemuMutex *mutex) +int qemu_mutex_trylock_impl(QemuMutex *mutex, const char *file, const int line) { int err; assert(mutex->initialized); err = pthread_mutex_trylock(&mutex->lock); if (err == 0) { - trace_qemu_mutex_locked(mutex); + trace_qemu_mutex_locked(mutex, file, line); return 0; } if (err != EBUSY) { @@ -85,15 +87,16 @@ int qemu_mutex_trylock(QemuMutex *mutex) return -EBUSY; } -void qemu_mutex_unlock(QemuMutex *mutex) +void qemu_mutex_unlock_impl(QemuMutex *mutex, const char *file, const int line) { int err; assert(mutex->initialized); - trace_qemu_mutex_unlocked(mutex); err = pthread_mutex_unlock(&mutex->lock); if (err) error_exit(err, __func__); + + trace_qemu_mutex_unlock(mutex, file, line); } void qemu_rec_mutex_init(QemuRecMutex *mutex) @@ -152,14 +155,14 @@ void qemu_cond_broadcast(QemuCond *cond) error_exit(err, __func__); } -void qemu_cond_wait(QemuCond *cond, QemuMutex *mutex) +void qemu_cond_wait_impl(QemuCond *cond, QemuMutex *mutex, const char *file, const int line) { int err; assert(cond->initialized); - trace_qemu_mutex_unlocked(mutex); + trace_qemu_mutex_unlock(mutex, file, line); err = pthread_cond_wait(&cond->cond, &mutex->lock); - trace_qemu_mutex_locked(mutex); + trace_qemu_mutex_locked(mutex, file, line); if (err) error_exit(err, __func__); } diff --git a/util/trace-events b/util/trace-events index 025499f..515e625 100644 --- a/util/trace-events +++ b/util/trace-events @@ -56,6 +56,7 @@ lockcnt_futex_wait(const void *lockcnt, int val) "lockcnt %p waiting on %d" lockcnt_futex_wait_resume(const void *lockcnt, int new) "lockcnt %p after wait: %d" lockcnt_futex_wake(const void *lockcnt) "lockcnt %p waking up one waiter" -# util/qemu-thread-posix.c -qemu_mutex_locked(void *lock) "locked mutex %p" -qemu_mutex_unlocked(void *lock) "unlocked mutex %p" +# util/qemu-thread.c +qemu_mutex_lock(void *mutex, const char *file, const int line) "waiting on mutex %p (%s:%d)" +qemu_mutex_locked(void *mutex, const char *file, const int line) "taken mutex %p (%s:%d)" +qemu_mutex_unlock(void *mutex, const char *file, const int line) "released mutex %p (%s:%d)" From patchwork Tue Nov 14 08:18:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pavel Dovgalyuk X-Patchwork-Id: 118850 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp2771673qgn; Tue, 14 Nov 2017 00:35:52 -0800 (PST) X-Google-Smtp-Source: AGs4zMa84/PjjFE3VbW5bpBPxNQHTwLLmRqVvIO/PSysgA49HIVf+ZvAAV5eBMaX5adZNOUovspQ X-Received: by 10.13.209.70 with SMTP id t67mr7427153ywd.102.1510648552336; Tue, 14 Nov 2017 00:35:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510648552; cv=none; d=google.com; s=arc-20160816; b=xrfM0kruEk1cMkUC1q7zh8yGiPu7MMrtB0wV1x09NdCcPUNd5B+pf1ug61wVdivdbt fxbHHTS+CFi/pbEKi+LP+V3ic/8PIzQssfnh1FseySNEymok1PUWuoMJ78tnD4xKJR4M M5yqZOmZve4UXbTa1XM4kRH8gJXhUFpTDvooQUDDaNQlgY0hNw2qVq8slOk79sse0trK gQSyIvUPmu0ehShRfAU/2Rp3wJuy8M2TiTwFyl6WSXa8PoWKbcqF7ABtRJNSBvSTRaKp 21PfpbYsbsUdJfZaWf4CEqOO4a/JgLGCCaiayKDPG6Ujw7+SfGTGjh7NKRArmIIJplQC FnxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=aT+AWYNLUZIktc+qFKMvvIRgCCvYlgEwbMv7GDlIC3k=; b=G2JoV9nBaVhmqMVZgZ8V26Etzi/Rm879CxVJ8gZZozv/TpM1xKRdj2UfFHSj72RbvY nCaiJRn+dJkco6x6wllMBrxlKpK2FyJ2pUWQG8OFaAYvq3BoQvTpGimVyhLClkcGHe9Y cMnM570n3AuorzvY7+bgdSSbJfnJ6S/7mgK1yVtC7lviYS/2+uhFgRcXP02yQUyRLIph W+go1z6Q78rKJaSc69tnggw3W4+XcZd4Ze1/CriYpPJE2Qi4IKPe1WTpL5EVTMoR8BZU if0/Kc2PmKzbTZuuJIdlcCrUebyKN7Z3OHIFTcU2Qh3jOUQ8KuBqdSOdyOmzX1IE17NE pSSw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id i190si193906ywf.520.2017.11.14.00.35.52 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 14 Nov 2017 00:35:52 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:58254 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWhD-0000Tg-Py for patch@linaro.org; Tue, 14 Nov 2017 03:35:51 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41218) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWQa-0000w4-Dm for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:41 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEWQZ-0007L6-4Z for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:40 -0500 Received: from mail.ispras.ru ([83.149.199.45]:52890) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWQY-0007Kh-On for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:39 -0500 Received: from [127.0.1.1] (unknown [85.142.117.226]) by mail.ispras.ru (Postfix) with ESMTPSA id CD86D54006A; Tue, 14 Nov 2017 11:18:37 +0300 (MSK) From: Pavel Dovgalyuk To: qemu-devel@nongnu.org Date: Tue, 14 Nov 2017 11:18:46 +0300 Message-ID: <20171114081846.27640.12424.stgit@pasha-VirtualBox> In-Reply-To: <20171114081630.27640.53933.stgit@pasha-VirtualBox> References: <20171114081630.27640.53933.stgit@pasha-VirtualBox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 83.149.199.45 Subject: [Qemu-devel] [RFC PATCH v2 24/26] scripts/analyse-locks-simpletrace.py: script to analyse lock times X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, pavel.dovgaluk@ispras.ru, mst@redhat.com, jasowang@redhat.com, quintela@redhat.com, zuban32s@gmail.com, maria.klimushenkova@ispras.ru, dovgaluk@ispras.ru, kraxel@redhat.com, boost.lists@gmail.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée This script allows analysis of mutex acquisition and hold times based on a trace file. Given a trace control file of: qemu_mutex_lock qemu_mutex_locked qemu_mutex_unlock And running with: $QEMU $QEMU_ARGS -trace events=./lock-trace You can analyse the results with: ./scripts/analyse-locks-simpletrace.py trace-events-all ./trace-21812 Signed-off-by: Alex Bennée --- scripts/analyse-locks-simpletrace.py | 99 ++++++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100755 scripts/analyse-locks-simpletrace.py diff --git a/scripts/analyse-locks-simpletrace.py b/scripts/analyse-locks-simpletrace.py new file mode 100755 index 0000000..b72c951 --- /dev/null +++ b/scripts/analyse-locks-simpletrace.py @@ -0,0 +1,99 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- +# +# Analyse lock events and +# +# Author: Alex Bennée +# + +import os +import simpletrace +import argparse +import numpy as np + +class MutexAnalyser(simpletrace.Analyzer): + "A simpletrace Analyser for checking locks." + + def __init__(self): + self.locks = 0 + self.locked = 0 + self.unlocks = 0 + self.mutex_records = {} + + def _get_mutex(self, mutex): + if not mutex in self.mutex_records: + self.mutex_records[mutex] = {"locks": 0, + "lock_time": 0, + "acquire_times": [], + "locked": 0, + "locked_time": 0, + "held_times": [], + "unlocked": 0} + + return self.mutex_records[mutex] + + def qemu_mutex_lock(self, timestamp, mutex, filename, line): + self.locks += 1 + rec = self._get_mutex(mutex) + rec["locks"] += 1 + rec["lock_time"] = timestamp[0] + rec["lock_loc"] = (filename, line) + + def qemu_mutex_locked(self, timestamp, mutex, filename, line): + self.locked += 1 + rec = self._get_mutex(mutex) + rec["locked"] += 1 + rec["locked_time"] = timestamp[0] + acquire_time = rec["locked_time"] - rec["lock_time"] + rec["locked_loc"] = (filename, line) + rec["acquire_times"].append(acquire_time) + + def qemu_mutex_unlock(self, timestamp, mutex, filename, line): + self.unlocks += 1 + rec = self._get_mutex(mutex) + rec["unlocked"] += 1 + held_time = timestamp[0] - rec["locked_time"] + rec["held_times"].append(held_time) + rec["unlock_loc"] = (filename, line) + + +def get_args(): + "Grab options" + parser = argparse.ArgumentParser() + parser.add_argument("--output", "-o", type=str, help="Render plot to file") + parser.add_argument("events", type=str, help='trace file read from') + parser.add_argument("tracefile", type=str, help='trace file read from') + return parser.parse_args() + +if __name__ == '__main__': + args = get_args() + + # Gather data from the trace + analyser = MutexAnalyser() + simpletrace.process(args.events, args.tracefile, analyser) + + print ("Total locks: %d, locked: %d, unlocked: %d" % + (analyser.locks, analyser.locked, analyser.unlocks)) + + # Now dump the individual lock stats + for key, val in sorted(analyser.mutex_records.iteritems(), + key=lambda (k,v): v["locks"]): + print ("Lock: %#x locks: %d, locked: %d, unlocked: %d" % + (key, val["locks"], val["locked"], val["unlocked"])) + + acquire_times = np.array(val["acquire_times"]) + if len(acquire_times) > 0: + print (" Acquire Time: min:%d median:%d avg:%.2f max:%d" % + (acquire_times.min(), np.median(acquire_times), + acquire_times.mean(), acquire_times.max())) + + held_times = np.array(val["held_times"]) + if len(held_times) > 0: + print (" Held Time: min:%d median:%d avg:%.2f max:%d" % + (held_times.min(), np.median(held_times), + held_times.mean(), held_times.max())) + + # Check if any locks still held + if val["locks"] > val["locked"]: + print (" LOCK HELD (%s:%s)" % (val["locked_loc"])) + print (" BLOCKED (%s:%s)" % (val["lock_loc"])) From patchwork Tue Nov 14 08:18:51 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pavel Dovgalyuk X-Patchwork-Id: 118854 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp2774397qgn; Tue, 14 Nov 2017 00:39:37 -0800 (PST) X-Google-Smtp-Source: AGs4zMZuLF/l+gdiE9WYGkGZJ5SJnJVzw57r30c3iQK76CoboE6hxtPX26DPwQQ9QAVAY5omdeex X-Received: by 10.37.1.198 with SMTP id 189mr1161861ybb.420.1510648777525; Tue, 14 Nov 2017 00:39:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510648777; cv=none; d=google.com; s=arc-20160816; b=aBenAcvzvgTNapBtjnZDFDAm9WL+NG/7sb65IW6fzzwzzEITlR05u7kuNuF1JnY7uV SNGFCZPIRZ+dEetKAPDDQVX3Uvrhk+g3k/UHZJFFX6MxrjKKxLO4XtVWYcBCkxTAa2I4 RMcVNH3peqkinQ7e7KC37E5hDjVm9iSbMmKqkWx0+ZSFJX6sdFkyhLNx8I3JAi/L610k ptOWKv2STP2+AEUhXPiFVuMtcLvcmryGThs4Qw5DsmpZ+ZhqREw6PjPE+op2TjLHRlpj PMkjA6KcCMXagHeabyAAjnQaXSvM2/fuLRGfSDH1CXTBtLxu9HhsLVdchqiy9dj3H5/q uhLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=t7Q16W46hGY0/4vznypm79ggdT9i2ZsOOIaY54eKsk4=; b=QVISqlwYbG2PecASlWOFINeIBBVVBCrjORHGYRvUEpDGziTohT+GdUBqlHTRZy33Wi byhIVQs+1p5izmJKfOP7jq4mduKADPFWyTVyG5hKPnrxhheb7m+Fjh+zr9r+TjRtO/m7 p0AFdw6fzC81QJlMc6FNgua97Xm223OSscPRYU3yVZMzra0Nm0Kjz5pQIUGv5MGVx/qv dnmNlKCjXCcd4jz1+MvT4dY7bGxZGIXk5PpFhsJo9DyVcwP0CouU8Y99S/vDSatWXi6s yTJoBLzOd1ioKbt3f4/KDItfrFRWyMOmsLT2iuAw65wApyEzl8IySXjNFgcOfQlGvs26 5HKg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id t2si3983659ywi.690.2017.11.14.00.39.37 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 14 Nov 2017 00:39:37 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:58284 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWkq-0003mm-W8 for patch@linaro.org; Tue, 14 Nov 2017 03:39:37 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41326) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWQg-00017X-M6 for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:48 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEWQe-0007Ph-JH for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:46 -0500 Received: from mail.ispras.ru ([83.149.199.45]:52916) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWQe-0007P3-5W for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:44 -0500 Received: from [127.0.1.1] (unknown [85.142.117.226]) by mail.ispras.ru (Postfix) with ESMTPSA id 538F254006A; Tue, 14 Nov 2017 11:18:43 +0300 (MSK) From: Pavel Dovgalyuk To: qemu-devel@nongnu.org Date: Tue, 14 Nov 2017 11:18:51 +0300 Message-ID: <20171114081851.27640.19418.stgit@pasha-VirtualBox> In-Reply-To: <20171114081630.27640.53933.stgit@pasha-VirtualBox> References: <20171114081630.27640.53933.stgit@pasha-VirtualBox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 83.149.199.45 Subject: [Qemu-devel] [RFC PATCH v2 25/26] scripts/replay-dump.py: replay log dumper X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, pavel.dovgaluk@ispras.ru, mst@redhat.com, jasowang@redhat.com, quintela@redhat.com, zuban32s@gmail.com, maria.klimushenkova@ispras.ru, dovgaluk@ispras.ru, kraxel@redhat.com, boost.lists@gmail.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée This script is a debugging tool for looking through the contents of a replay log file. It is incomplete but should fail gracefully at events it doesn't understand. It currently understands two different log formats as the audio record/replay support was merged during since MTTCG. It was written to help debug what has caused the BQL changes to break replay support. Signed-off-by: Alex Bennée --- v2 - yet another update to the log format --- scripts/replay-dump.py | 308 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 308 insertions(+) create mode 100755 scripts/replay-dump.py diff --git a/scripts/replay-dump.py b/scripts/replay-dump.py new file mode 100755 index 0000000..203bb31 --- /dev/null +++ b/scripts/replay-dump.py @@ -0,0 +1,308 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- +# +# Dump the contents of a recorded execution stream +# +# Copyright (c) 2017 Alex Bennée +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, see . + +import argparse +import struct +from collections import namedtuple + +# This mirrors some of the global replay state which some of the +# stream loading refers to. Some decoders may read the next event so +# we need handle that case. Calling reuse_event will ensure the next +# event is read from the cache rather than advancing the file. + +class ReplayState(object): + def __init__(self): + self.event = -1 + self.event_count = 0 + self.already_read = False + self.current_checkpoint = 0 + self.checkpoint = 0 + + def set_event(self, ev): + self.event = ev + self.event_count += 1 + + def get_event(self): + self.already_read = False + return self.event + + def reuse_event(self, ev): + self.event = ev + self.already_read = True + + def set_checkpoint(self): + self.checkpoint = self.event - self.checkpoint_start + + def get_checkpoint(self): + return self.checkpoint + +replay_state = ReplayState() + +# Simple read functions that mirror replay-internal.c +# The file-stream is big-endian and manually written out a byte at a time. + +def read_byte(fin): + "Read a single byte" + return struct.unpack('>B', fin.read(1))[0] + +def read_event(fin): + "Read a single byte event, but save some state" + if replay_state.already_read: + return replay_state.get_event() + else: + replay_state.set_event(read_byte(fin)) + return replay_state.event + +def read_word(fin): + "Read a 16 bit word" + return struct.unpack('>H', fin.read(2))[0] + +def read_dword(fin): + "Read a 32 bit word" + return struct.unpack('>I', fin.read(4))[0] + +def read_qword(fin): + "Read a 64 bit word" + return struct.unpack('>Q', fin.read(8))[0] + +# Generic decoder structure +Decoder = namedtuple("Decoder", "eid name fn") + +def call_decode(table, index, dumpfile): + "Search decode table for next step" + decoder = next((d for d in table if d.eid == index), None) + if not decoder: + print "Could not decode index: %d" % (index) + print "Entry is: %s" % (decoder) + print "Decode Table is:\n%s" % (table) + return False + else: + return decoder.fn(decoder.eid, decoder.name, dumpfile) + +# Print event +def print_event(eid, name, string=None, event_count=None): + "Print event with count" + if not event_count: + event_count = replay_state.event_count + + if string: + print "%d:%s(%d) %s" % (event_count, name, eid, string) + else: + print "%d:%s(%d)" % (event_count, name, eid) + + +# Decoders for each event type + +def decode_unimp(eid, name, _unused_dumpfile): + "Unimplimented decoder, will trigger exit" + print "%s not handled - will now stop" % (name) + return False + +# Checkpoint decoder +def swallow_async_qword(eid, name, dumpfile): + "Swallow a qword of data without looking at it" + step_id = read_qword(dumpfile) + print " %s(%d) @ %d" % (name, eid, step_id) + return True + +async_decode_table = [ Decoder(0, "REPLAY_ASYNC_EVENT_BH", swallow_async_qword), + Decoder(1, "REPLAY_ASYNC_INPUT", decode_unimp), + Decoder(2, "REPLAY_ASYNC_INPUT_SYNC", decode_unimp), + Decoder(3, "REPLAY_ASYNC_CHAR_READ", decode_unimp), + Decoder(4, "REPLAY_ASYNC_EVENT_BLOCK", decode_unimp), + Decoder(5, "REPLAY_ASYNC_EVENT_NET", decode_unimp), +] +# See replay_read_events/replay_read_event +def decode_async(eid, name, dumpfile): + """Decode an ASYNC event""" + + print_event(eid, name) + + async_event_kind = read_byte(dumpfile) + async_event_checkpoint = read_byte(dumpfile) + + if async_event_checkpoint != replay_state.current_checkpoint: + print " mismatch between checkpoint %d and async data %d" % ( + replay_state.current_checkpoint, async_event_checkpoint) + return True + + return call_decode(async_decode_table, async_event_kind, dumpfile) + + +def decode_instruction(eid, name, dumpfile): + ins_diff = read_dword(dumpfile) + print_event(eid, name, "0x%x" % (ins_diff)) + return True + +def decode_audio_out(eid, name, dumpfile): + audio_data = read_dword(dumpfile) + print_event(eid, name, "%d" % (audio_data)) + return True + +def decode_checkpoint(eid, name, dumpfile): + """Decode a checkpoint. + + Checkpoints contain a series of async events with their own specific data. + """ + replay_state.set_checkpoint() + # save event count as we peek ahead + event_number = replay_state.event_count + next_event = read_event(dumpfile) + + # if the next event is EVENT_ASYNC there are a bunch of + # async events to read, otherwise we are done + if next_event != 3: + print_event(eid, name, "no additional data", event_number) + else: + print_event(eid, name, "more data follows", event_number) + + replay_state.reuse_event(next_event) + return True + +def decode_checkpoint_init(eid, name, dumpfile): + print_event(eid, name) + return True + +def decode_interrupt(eid, name, dumpfile): + print_event(eid, name) + return True + +def decode_clock(eid, name, dumpfile): + clock_data = read_qword(dumpfile) + print_event(eid, name, "0x%x" % (clock_data)) + return True + + +# pre-MTTCG merge +v5_event_table = [Decoder(0, "EVENT_INSTRUCTION", decode_instruction), + Decoder(1, "EVENT_INTERRUPT", decode_interrupt), + Decoder(2, "EVENT_EXCEPTION", decode_unimp), + Decoder(3, "EVENT_ASYNC", decode_async), + Decoder(4, "EVENT_SHUTDOWN", decode_unimp), + Decoder(5, "EVENT_CHAR_WRITE", decode_unimp), + Decoder(6, "EVENT_CHAR_READ_ALL", decode_unimp), + Decoder(7, "EVENT_CHAR_READ_ALL_ERROR", decode_unimp), + Decoder(8, "EVENT_CLOCK_HOST", decode_clock), + Decoder(9, "EVENT_CLOCK_VIRTUAL_RT", decode_clock), + Decoder(10, "EVENT_CP_CLOCK_WARP_START", decode_checkpoint), + Decoder(11, "EVENT_CP_CLOCK_WARP_ACCOUNT", decode_checkpoint), + Decoder(12, "EVENT_CP_RESET_REQUESTED", decode_checkpoint), + Decoder(13, "EVENT_CP_SUSPEND_REQUESTED", decode_checkpoint), + Decoder(14, "EVENT_CP_CLOCK_VIRTUAL", decode_checkpoint), + Decoder(15, "EVENT_CP_CLOCK_HOST", decode_checkpoint), + Decoder(16, "EVENT_CP_CLOCK_VIRTUAL_RT", decode_checkpoint), + Decoder(17, "EVENT_CP_INIT", decode_checkpoint_init), + Decoder(18, "EVENT_CP_RESET", decode_checkpoint), +] + +# post-MTTCG merge, AUDIO support added +v6_event_table = [Decoder(0, "EVENT_INSTRUCTION", decode_instruction), + Decoder(1, "EVENT_INTERRUPT", decode_interrupt), + Decoder(2, "EVENT_EXCEPTION", decode_unimp), + Decoder(3, "EVENT_ASYNC", decode_async), + Decoder(4, "EVENT_SHUTDOWN", decode_unimp), + Decoder(5, "EVENT_CHAR_WRITE", decode_unimp), + Decoder(6, "EVENT_CHAR_READ_ALL", decode_unimp), + Decoder(7, "EVENT_CHAR_READ_ALL_ERROR", decode_unimp), + Decoder(8, "EVENT_AUDIO_OUT", decode_audio_out), + Decoder(9, "EVENT_AUDIO_IN", decode_unimp), + Decoder(10, "EVENT_CLOCK_HOST", decode_clock), + Decoder(11, "EVENT_CLOCK_VIRTUAL_RT", decode_clock), + Decoder(12, "EVENT_CP_CLOCK_WARP_START", decode_checkpoint), + Decoder(13, "EVENT_CP_CLOCK_WARP_ACCOUNT", decode_checkpoint), + Decoder(14, "EVENT_CP_RESET_REQUESTED", decode_checkpoint), + Decoder(15, "EVENT_CP_SUSPEND_REQUESTED", decode_checkpoint), + Decoder(16, "EVENT_CP_CLOCK_VIRTUAL", decode_checkpoint), + Decoder(17, "EVENT_CP_CLOCK_HOST", decode_checkpoint), + Decoder(18, "EVENT_CP_CLOCK_VIRTUAL_RT", decode_checkpoint), + Decoder(19, "EVENT_CP_INIT", decode_checkpoint_init), + Decoder(20, "EVENT_CP_RESET", decode_checkpoint), +] + +# Shutdown cause added +v7_event_table = [Decoder(0, "EVENT_INSTRUCTION", decode_instruction), + Decoder(1, "EVENT_INTERRUPT", decode_interrupt), + Decoder(2, "EVENT_EXCEPTION", decode_unimp), + Decoder(3, "EVENT_ASYNC", decode_async), + Decoder(4, "EVENT_SHUTDOWN", decode_unimp), + Decoder(5, "EVENT_SHUTDOWN_HOST_ERR", decode_unimp), + Decoder(6, "EVENT_SHUTDOWN_HOST_QMP", decode_unimp), + Decoder(7, "EVENT_SHUTDOWN_HOST_SIGNAL", decode_unimp), + Decoder(8, "EVENT_SHUTDOWN_HOST_UI", decode_unimp), + Decoder(9, "EVENT_SHUTDOWN_GUEST_SHUTDOWN", decode_unimp), + Decoder(10, "EVENT_SHUTDOWN_GUEST_RESET", decode_unimp), + Decoder(11, "EVENT_SHUTDOWN_GUEST_PANIC", decode_unimp), + Decoder(12, "EVENT_SHUTDOWN___MAX", decode_unimp), + Decoder(13, "EVENT_CHAR_WRITE", decode_unimp), + Decoder(14, "EVENT_CHAR_READ_ALL", decode_unimp), + Decoder(15, "EVENT_CHAR_READ_ALL_ERROR", decode_unimp), + Decoder(16, "EVENT_AUDIO_OUT", decode_audio_out), + Decoder(17, "EVENT_AUDIO_IN", decode_unimp), + Decoder(18, "EVENT_CLOCK_HOST", decode_clock), + Decoder(19, "EVENT_CLOCK_VIRTUAL_RT", decode_clock), + Decoder(20, "EVENT_CP_CLOCK_WARP_START", decode_checkpoint), + Decoder(21, "EVENT_CP_CLOCK_WARP_ACCOUNT", decode_checkpoint), + Decoder(22, "EVENT_CP_RESET_REQUESTED", decode_checkpoint), + Decoder(23, "EVENT_CP_SUSPEND_REQUESTED", decode_checkpoint), + Decoder(24, "EVENT_CP_CLOCK_VIRTUAL", decode_checkpoint), + Decoder(25, "EVENT_CP_CLOCK_HOST", decode_checkpoint), + Decoder(26, "EVENT_CP_CLOCK_VIRTUAL_RT", decode_checkpoint), + Decoder(27, "EVENT_CP_INIT", decode_checkpoint_init), + Decoder(28, "EVENT_CP_RESET", decode_checkpoint), +] + +def parse_arguments(): + "Grab arguments for script" + parser = argparse.ArgumentParser() + parser.add_argument("-f", "--file", help='record/replay dump to read from', + required=True) + return parser.parse_args() + +def decode_file(filename): + "Decode a record/replay dump" + dumpfile = open(filename, "rb") + + # read and throwaway the header + version = read_dword(dumpfile) + junk = read_qword(dumpfile) + + print "HEADER: version 0x%x" % (version) + + if version == 0xe02007: + event_decode_table = v7_event_table + replay_state.checkpoint_start = 12 + elif version == 0xe02006: + event_decode_table = v6_event_table + replay_state.checkpoint_start = 12 + else: + event_decode_table = v5_event_table + replay_state.checkpoint_start = 10 + + try: + decode_ok = True + while decode_ok: + event = read_event(dumpfile) + decode_ok = call_decode(event_decode_table, event, dumpfile) + finally: + dumpfile.close() + +if __name__ == "__main__": + args = parse_arguments() + decode_file(args.file) From patchwork Tue Nov 14 08:18:57 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pavel Dovgalyuk X-Patchwork-Id: 118855 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp2776194qgn; Tue, 14 Nov 2017 00:42:05 -0800 (PST) X-Google-Smtp-Source: AGs4zMaMA52N28zljyT+nd3KBRMlUz3Cwm04Njwf3hUq0tqz/szWD5nkVOV+9eezkr2qGRrzgKj+ X-Received: by 10.13.248.132 with SMTP id i126mr7348619ywf.306.1510648925205; Tue, 14 Nov 2017 00:42:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510648925; cv=none; d=google.com; s=arc-20160816; b=YZI2DI8IQB3Pm41+whLdfk9vg+tvnSNZyphDrmJle0J037+7c8TBtk6MzrEkvW6bZa v+DpakV4rMmM1OSqutgF4NMYjfy0SIfSbhUDjn+FkfbH2+JxKQgPytD+qkIjMzaXvkb9 lxOCL9e1ihSikorSE18dyWcpzuNC9xgytqjxAgzb5Dr5UAxLnd9aOIp1QDv0kYbgdQle mtLTb031uvp1q2nNcPIS7OPbXMnRkhmGx7CvwNLmmls57tBUMkmIW9Qo4fAI38f1QOEd bK9kq5OnIc8kfowI7HQL+e7YlUu6ByDWVot2mlz4AsyTys0S7FavG8jpNRu61TK+EJVd qKlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=0mNrYB/7CQ13/nCWNidSOnK31xeLWYbx0BprAi0IOhM=; b=tG4T7RixarMk5CHaHlwGPHywjn3y5IGGLzCLUAz0eDbySm70Zn0m9YFh3bD0iTl28y gHDu7i+iFC0+dnUOX30ziZdesUQvjx0rthIIj130ixG/4ffPswqg3Vh3GUvlADSerl62 Vgz5w96tQxcdaOG3mvUaVIrynkVUij8x7V6gOeSmOLBILd2lXR5N6W9dQtESNYzoPkNO NhwGR9qrxJDe6vljHcd9IGotLVazVik7zZemffDbWzzRlNDHDj22Gbb6ICKHKvmksvo4 mJumpVNs4St6YwRdSvo16jmiqtXl2ndSFhiqerTE7XGAdq2S5riyoahlbD7pLj2zjEkq CDZQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id t202si1071507ywg.226.2017.11.14.00.42.05 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 14 Nov 2017 00:42:05 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:58302 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWnE-0006W8-NE for patch@linaro.org; Tue, 14 Nov 2017 03:42:04 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41374) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWQl-0001BI-8O for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:57 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEWQk-0007UW-2a for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:51 -0500 Received: from mail.ispras.ru ([83.149.199.45]:52930) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEWQj-0007U4-Lg for qemu-devel@nongnu.org; Tue, 14 Nov 2017 03:18:50 -0500 Received: from [127.0.1.1] (unknown [85.142.117.226]) by mail.ispras.ru (Postfix) with ESMTPSA id DA80B54006A; Tue, 14 Nov 2017 11:18:48 +0300 (MSK) From: Pavel Dovgalyuk To: qemu-devel@nongnu.org Date: Tue, 14 Nov 2017 11:18:57 +0300 Message-ID: <20171114081857.27640.15221.stgit@pasha-VirtualBox> In-Reply-To: <20171114081630.27640.53933.stgit@pasha-VirtualBox> References: <20171114081630.27640.53933.stgit@pasha-VirtualBox> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 83.149.199.45 Subject: [Qemu-devel] [RFC PATCH v2 26/26] scripts/qemu-gdb/timers.py: new helper to dump timer state X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, pavel.dovgaluk@ispras.ru, mst@redhat.com, jasowang@redhat.com, quintela@redhat.com, zuban32s@gmail.com, maria.klimushenkova@ispras.ru, dovgaluk@ispras.ru, kraxel@redhat.com, boost.lists@gmail.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée This introduces the qemu-gdb command "qemu timers" which will dump the state of the main timers in the system. Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé --- scripts/qemu-gdb.py | 3 ++- scripts/qemugdb/timers.py | 54 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+), 1 deletion(-) create mode 100644 scripts/qemugdb/timers.py diff --git a/scripts/qemu-gdb.py b/scripts/qemu-gdb.py index d58213e..690827e 100644 --- a/scripts/qemu-gdb.py +++ b/scripts/qemu-gdb.py @@ -26,7 +26,7 @@ import os, sys sys.path.append(os.path.dirname(__file__)) -from qemugdb import aio, mtree, coroutine, tcg +from qemugdb import aio, mtree, coroutine, tcg, timers class QemuCommand(gdb.Command): '''Prefix for QEMU debug support commands''' @@ -39,6 +39,7 @@ coroutine.CoroutineCommand() mtree.MtreeCommand() aio.HandlersCommand() tcg.TCGLockStatusCommand() +timers.TimersCommand() coroutine.CoroutineSPFunction() coroutine.CoroutinePCFunction() diff --git a/scripts/qemugdb/timers.py b/scripts/qemugdb/timers.py new file mode 100644 index 0000000..be71a00 --- /dev/null +++ b/scripts/qemugdb/timers.py @@ -0,0 +1,54 @@ +#!/usr/bin/python +# GDB debugging support +# +# Copyright 2017 Linaro Ltd +# +# Author: Alex Bennée +# +# This work is licensed under the terms of the GNU GPL, version 2. See +# the COPYING file in the top-level directory. + +# 'qemu timers' -- display the current timerlists + +import gdb + +class TimersCommand(gdb.Command): + '''Display the current QEMU timers''' + + def __init__(self): + 'Register the class as a gdb command' + gdb.Command.__init__(self, 'qemu timers', gdb.COMMAND_DATA, + gdb.COMPLETE_NONE) + + def dump_timers(self, timer): + "Follow a timer and recursively dump each one in the list." + # timer should be of type QemuTimer + gdb.write(" timer %s/%s (cb:%s,opq:%s)\n" % ( + timer['expire_time'], + timer['scale'], + timer['cb'], + timer['opaque'])) + + if int(timer['next']) > 0: + self.dump_timers(timer['next']) + + + def process_timerlist(self, tlist, ttype): + gdb.write("Processing %s timers\n" % (ttype)) + gdb.write(" clock %s is enabled:%s, last:%s\n" % ( + tlist['clock']['type'], + tlist['clock']['enabled'], + tlist['clock']['last'])) + if int(tlist['active_timers']) > 0: + self.dump_timers(tlist['active_timers']) + + + def invoke(self, arg, from_tty): + 'Run the command' + main_timers = gdb.parse_and_eval("main_loop_tlg") + + # This will break if QEMUClockType in timer.h is redfined + self.process_timerlist(main_timers['tl'][0], "Realtime") + self.process_timerlist(main_timers['tl'][1], "Virtual") + self.process_timerlist(main_timers['tl'][2], "Host") + self.process_timerlist(main_timers['tl'][3], "Virtual RT")